Skip to contentSkip to conversation

compress/isal: ISA-L compression PMD

PATCH Superseded Lee Daly 4 participants 70 messages Mar 9, 2018 – May 9, 2018 Arm/Amlogic Meson SoC support

lore
Share & tools
lore

Lee Daly

author
#0 lore raw

PATCH 1/1 compress/isal: ISA-L compression PMD

Compression PMD using the Intel ISA-L library.

Features of this PMD:
- Stateless compression and decompression.
- Fixed/Semi-Dynamic compression,
- Supports 32K sliding window size,
- ISA-L levels 0 to 3,
- Adler32 & CRC32 checksums.

---

This patch has dependencies on the compressdev API. V2 features to be added include: SGL support & documentation.

This PMD uses the ISA-L library,
which can be found at https://github.com/01org/isa-l.
The user must download and compile this library before using the PMD.

 config/common_base                                 |   4 +
 drivers/Makefile                                   |   2 +
 drivers/compress/Makefile                          |   9 +
 drivers/compress/isa-l/Makefile                    |  31 ++
 drivers/compress/isa-l/isa-l_compress_pmd.c        | 491 +++++++++++++++++++++
 drivers/compress/isa-l/isa-l_compress_pmd_ops.c    | 321 ++++++++++++++
 .../compress/isa-l/isa-l_compress_pmd_private.h    |  57 +++
 drivers/compress/isa-l/rte_pmd_isa-l_version.map   |   3 +
 lib/librte_compressdev/rte_comp.h                  |   4 +-
 lib/librte_compressdev/rte_compressdev.h           |   2 +-
 mk/rte.app.mk                                      |   6 +
 11 files changed, 928 insertions(+), 2 deletions(-)
 create mode 100644 drivers/compress/Makefile
 create mode 100644 drivers/compress/isa-l/Makefile
 create mode 100644 drivers/compress/isa-l/isa-l_compress_pmd.c
 create mode 100644 drivers/compress/isa-l/isa-l_compress_pmd_ops.c
 create mode 100644 drivers/compress/isa-l/isa-l_compress_pmd_private.h
 create mode 100644 drivers/compress/isa-l/rte_pmd_isa-l_version.map
config/common_base +4 -0
index e0e5768..175209a 100644
@@ -540,6 +540,10 @@ CONFIG_RTE_LIBRTE_PMD_MRVL_CRYPTO_DEBUG=n
 CONFIG_RTE_LIBRTE_COMPRESSDEV=y
 CONFIG_RTE_COMPRESS_MAX_DEVS=64
 
+# Compile PMD for ISA-L device
+#
+CONFIG_RTE_LIBRTE_PMD_ISAL=n
+
 #
 # Compile generic security library
 #
drivers/Makefile +2 -0
index ee65c87..7dd4d2d 100644
@@ -10,6 +10,8 @@ DIRS-y += net
 DEPDIRS-net := bus mempool
 DIRS-$(CONFIG_RTE_LIBRTE_BBDEV) += bbdev
 DEPDIRS-bbdev := bus mempool
+DIRS-$(CONFIG_RTE_LIBRTE_COMPRESSDEV) += compress
+DEPDIRS-compress := bus mempool
 DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
 DEPDIRS-crypto := bus mempool
 DIRS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += event
drivers/compress/Makefile new file +9 -0
new file mode 100644
index 0000000..a7aed60
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isa-l
+
+include $(RTE_SDK)/mk/rte.subdir.mk
+
drivers/compress/isa-l/Makefile new file +31 -0
new file mode 100644
index 0000000..f732f90
@@ -0,0 +1,31 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_pmd_isal_comp.a
+
+# build flags
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+# external library dependencies
+LDLIBS += -lisal
+LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring
+LDLIBS += -lrte_compressdev
+LDLIBS += -lrte_bus_vdev
+
+# library version
+LIBABIVER := 1
+
+# versioning export map
+EXPORT_MAP := rte_pmd_isa-l_version.map
+
+# library source files
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isa-l_compress_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isa-l_compress_pmd_ops.c
+
+# export include files
+include $(RTE_SDK)/mk/rte.lib.mk
drivers/compress/isa-l/isa-l_compress_pmd.c new file +491 -0

491 lines not on this page · raw message

drivers/compress/isa-l/isa-l_compress_pmd_ops.c new file +321 -0

321 lines not on this page · raw message

drivers/compress/isa-l/isa-l_compress_pmd_private.h new file +57 -0

57 lines not on this page · raw message

drivers/compress/isa-l/rte_pmd_isa-l_version.map new file +3 -0
new file mode 100644
index 0000000..de8e412
@@ -0,0 +1,3 @@
+DPDK_18.05 {
+	local: *;
+};
lib/librte_compressdev/rte_comp.h +3 -1
index 0f5ee32..24b9e60 100644
@@ -41,7 +41,9 @@ enum rte_comp_op_status {
 
 /** Compression Algorithms */
 enum rte_comp_algorithm {
-	RTE_COMP_NULL = 0,
+	RTE_COMP_UNSPECIFIED = 0,
+	/** No Compression algorithm */
+	RTE_COMP_NULL,
 	/**< No compression.
 	 * Pass-through, data is copied unchanged from source buffer to
 	 * destination buffer.
lib/librte_compressdev/rte_compressdev.h +1 -1
index 72390b4..1b220ac 100644
@@ -94,7 +94,7 @@ struct rte_compressdev_capabilities {
 
 /** Macro used at end of comp PMD list */
 #define RTE_COMP_END_OF_CAPABILITIES_LIST() \
-	{ RTE_COMP_ALGO_LIST_END }
+	{ RTE_COMP_UNSPECIFIED }
 
 /**
  * compression device supported feature flags
mk/rte.app.mk +6 -0
index cff710a..c8bdb57 100644
@@ -218,6 +218,12 @@ endif # CONFIG_RTE_LIBRTE_DPAA_BUS
 
 endif # CONFIG_RTE_LIBRTE_CRYPTODEV
 
+ifeq ($(CONFIG_RTE_LIBRTE_COMPRESSDEV),y)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += -lrte_pmd_isal_comp
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) +=  -lisal
+endif # CONFIG_RTE_LIBRTE_COMPRESSDEV
+
+
 ifeq ($(CONFIG_RTE_LIBRTE_EVENTDEV),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SKELETON_EVENTDEV) += -lrte_pmd_skeleton_event
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV) += -lrte_pmd_sw_event
Signature · 1 line
2.7.4
author
#1 lore raw

PATCH v2 1/1 compress/isal: ISA-L compression PMD

Compression PMD, which utilizes Intel’s ISA-L compression library. The driver written under the DPDK compression API, compressdev, and therefore has dependencies on it, as well as the Intel's ISA-L library.

V2:
- Changes to keep in compliance with compressdev API,
- Enable meson build system.
- General rework & fixes.
- Documentation.

Future V3 for this release:
- Split into patchset

---
 MAINTAINERS                                        |  11 +
 config/common_base                                 |   5 +
 devtools/test-build.sh                             |   4 +
 doc/guides/compressdevs/features/default.ini       |  43 ++
 doc/guides/compressdevs/features/isal.ini          |  41 ++
 doc/guides/compressdevs/index.rst                  |  13 +
 doc/guides/compressdevs/isal.rst                   |  95 ++++
 doc/guides/compressdevs/overview.rst               |  13 +
 doc/guides/compressdevs/overview_feature_table.txt |  81 ++++
 doc/guides/index.rst                               |   1 +
 drivers/Makefile                                   |   2 +
 drivers/compress/Makefile                          |   9 +
 drivers/compress/isal/Makefile                     |  31 ++
 drivers/compress/isal/isa-l_compress_pmd.c         | 509 +++++++++++++++++++++
 drivers/compress/isal/isa-l_compress_pmd_ops.c     | 337 ++++++++++++++
 drivers/compress/isal/isa-l_compress_pmd_private.h |  56 +++
 drivers/compress/isal/meson.build                  |  15 +
 drivers/compress/isal/rte_pmd_isal_version.map     |   3 +
 drivers/compress/meson.build                       |   8 +
 drivers/meson.build                                |   1 +
 mk/rte.app.mk                                      |   5 +
 21 files changed, 1283 insertions(+)
 create mode 100644 doc/guides/compressdevs/features/default.ini
 create mode 100644 doc/guides/compressdevs/features/isal.ini
 create mode 100644 doc/guides/compressdevs/index.rst
 create mode 100644 doc/guides/compressdevs/isal.rst
 create mode 100644 doc/guides/compressdevs/overview.rst
 create mode 100644 doc/guides/compressdevs/overview_feature_table.txt
 create mode 100644 drivers/compress/Makefile
 create mode 100644 drivers/compress/isal/Makefile
 create mode 100644 drivers/compress/isal/isa-l_compress_pmd.c
 create mode 100644 drivers/compress/isal/isa-l_compress_pmd_ops.c
 create mode 100644 drivers/compress/isal/isa-l_compress_pmd_private.h
 create mode 100644 drivers/compress/isal/meson.build
 create mode 100644 drivers/compress/isal/rte_pmd_isal_version.map
 create mode 100644 drivers/compress/meson.build
MAINTAINERS +11 -0
index ed3251d..dc28288 100644
@@ -725,6 +725,17 @@ F: drivers/crypto/zuc/
 F: doc/guides/cryptodevs/zuc.rst
 F: doc/guides/cryptodevs/features/zuc.ini
 
+Compression Drivers
+-------------------
+M: Pablo de Lara <[email protected]>
+T: git://dpdk.org/next/dpdk-next-crypto
+F: doc/guides/compressdevs/features/default.ini
+
+ISA-L
+M: Lee Daly <[email protected]>
+F: drivers/compress/isal/
+F: doc/guides/compressdevs/isal.rst
+F: doc/guides/compressdevs/features/isal.ini
 
 Eventdev Drivers
 ----------------
config/common_base +5 -0
index c09c7cf..1def0d3 100644
@@ -533,6 +533,11 @@ CONFIG_RTE_LIBRTE_PMD_MRVL_CRYPTO=n
 CONFIG_RTE_LIBRTE_PMD_MRVL_CRYPTO_DEBUG=n
 
 #
+# Compile PMD for ISA-L device
+#
+CONFIG_RTE_LIBRTE_PMD_ISAL=n
+
+#
 # Compile generic security library
 #
 CONFIG_RTE_LIBRTE_SECURITY=y
devtools/test-build.sh +4 -0
index 3362edc..66f3ece 100755
@@ -45,6 +45,7 @@ default_path=$PATH
 # - DPDK_DEP_SSL (y/[n])
 # - DPDK_DEP_SZE (y/[n])
 # - DPDK_DEP_ZLIB (y/[n])
+# - DPDK_DEP_ISAL (y/[n])
 # - DPDK_MAKE_JOBS (int)
 # - DPDK_NOTIFY (notify-send)
 # - FLEXRAN_SDK
@@ -129,6 +130,7 @@ reset_env ()
 	unset DPDK_DEP_SSL
 	unset DPDK_DEP_SZE
 	unset DPDK_DEP_ZLIB
+	unset DPDK_DEP_ISAL
 	unset AESNI_MULTI_BUFFER_LIB_PATH
 	unset ARMV8_CRYPTO_LIB_PATH
 	unset FLEXRAN_SDK
@@ -178,6 +180,8 @@ config () # <directory> <target> <options>
 		test "$DPDK_DEP_ZLIB" != y || \
 		sed -ri          's,(BNX2X_PMD=)n,\1y,' $1/.config
 		sed -ri            's,(NFP_PMD=)n,\1y,' $1/.config
+		test "$DPDK_DEP_ISAL" != y || \
+		sed -ri          's,(ISAL_PMD=)n,\1y,' $1/.config
 		test "$DPDK_DEP_PCAP" != y || \
 		sed -ri               's,(PCAP=)n,\1y,' $1/.config
 		test -z "$ARMV8_CRYPTO_LIB_PATH" || \
doc/guides/compressdevs/features/default.ini new file +43 -0

43 lines not on this page · raw message

doc/guides/compressdevs/features/isal.ini new file +41 -0

41 lines not on this page · raw message

doc/guides/compressdevs/index.rst new file +13 -0
new file mode 100644
index 0000000..bc59ce8
@@ -0,0 +1,13 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Compression Device Drivers
+==========================
+
+
+.. toctree::
+    :maxdepth: 2
+    :numbered:
+
+    overview
+    isal
doc/guides/compressdevs/isal.rst new file +95 -0

95 lines not on this page · raw message

doc/guides/compressdevs/overview.rst new file +13 -0
new file mode 100644
index 0000000..b656af4
@@ -0,0 +1,13 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Compression Device Supported Functionality Matrices
+===================================================
+
+Supported Feature Flags
+-----------------------
+
+.. _table_compression_pmd_features:
+
+.. include:: overview_feature_table.txt
+
doc/guides/compressdevs/overview_feature_table.txt new file +81 -0

81 lines not on this page · raw message

doc/guides/index.rst +1 -0
index d60529d..18fe0ec 100644
@@ -17,6 +17,7 @@ DPDK documentation
    nics/index
    bbdevs/index
    cryptodevs/index
+   compressdevs/index
    eventdevs/index
    mempool/index
    platform/index
drivers/Makefile +2 -0
index 3d9f86b..c88638c 100644
@@ -13,6 +13,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_BBDEV) += baseband
 DEPDIRS-baseband := common bus mempool
 DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
 DEPDIRS-crypto := common bus mempool
+DIRS-$(CONFIG_RTE_LIBRTE_COMPRESSDEV) += compress
+DEPDIRS-compress := bus mempool
 DIRS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += event
 DEPDIRS-event := common bus mempool net
 DIRS-$(CONFIG_RTE_LIBRTE_RAWDEV) += raw
drivers/compress/Makefile new file +9 -0
new file mode 100644
index 0000000..4fff2c7
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal
+
+include $(RTE_SDK)/mk/rte.subdir.mk
+
drivers/compress/isal/Makefile new file +31 -0
new file mode 100644
index 0000000..7673429
@@ -0,0 +1,31 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_pmd_isal_comp.a
+
+# build flags
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+# external library dependencies
+LDLIBS += -lisal
+LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring
+LDLIBS += -lrte_compressdev
+LDLIBS += -lrte_bus_vdev
+
+# library version
+LIBABIVER := 1
+
+# versioning export map
+EXPORT_MAP := rte_pmd_isal_version.map
+
+# library source files
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isa-l_compress_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isa-l_compress_pmd_ops.c
+
+# export include files
+include $(RTE_SDK)/mk/rte.lib.mk
drivers/compress/isal/isa-l_compress_pmd.c new file +509 -0

509 lines not on this page · raw message

drivers/compress/isal/isa-l_compress_pmd_ops.c new file +337 -0

337 lines not on this page · raw message

drivers/compress/isal/isa-l_compress_pmd_private.h new file +56 -0

56 lines not on this page · raw message

drivers/compress/isal/meson.build new file +15 -0
new file mode 100644
index 0000000..87a5608
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2018 Intel Corporation
+
+dep = dependency('libisal', required: false)
+if not dep.found()
+	build =false
+endif
+
+deps += 'bus_vdev'
+sources = files('isa-l_compress_pmd.c', 'isa-l_compress_pmd_ops.c')
+ext_deps += dep
+pkgconfig_extra_libs += '-lisal'
+
+allow_experimental_apis = true
+
drivers/compress/isal/rte_pmd_isal_version.map new file +3 -0
new file mode 100644
index 0000000..de8e412
@@ -0,0 +1,3 @@
+DPDK_18.05 {
+	local: *;
+};
drivers/compress/meson.build new file +8 -0
new file mode 100644
index 0000000..fb136e1
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+drivers = ['isal']
+
+std_deps = ['compressdev'] # compressdev pulls in all other needed deps
+config_flag_fmt = 'RTE_LIBRTE_@0@_PMD'
+driver_name_fmt = 'rte_pmd_@0@'
drivers/meson.build +1 -0
index b146f09..fd42292 100644
@@ -7,6 +7,7 @@ driver_classes = ['common',
 	       'mempool', # depends on common and bus.
 	       'net',     # depends on common, bus and mempool.
 	       'crypto',  # depends on common, bus and mempool (net in future).
+	       'compress', # depends on common, bus.
 	       'event']   # depends on common, bus, mempool and net.
 
 foreach class:driver_classes
mk/rte.app.mk +5 -0
index 2585908..9cfb6c1 100644
@@ -227,6 +227,11 @@ endif # CONFIG_RTE_LIBRTE_DPAA_BUS
 
 endif # CONFIG_RTE_LIBRTE_CRYPTODEV
 
+ifeq ($(CONFIG_RTE_LIBRTE_COMPRESSDEV),y)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += -lrte_pmd_isal_comp
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) +=  -lisal
+endif # CONFIG_RTE_LIBRTE_COMPRESSDEV
+
 ifeq ($(CONFIG_RTE_LIBRTE_EVENTDEV),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SKELETON_EVENTDEV) += -lrte_pmd_skeleton_event
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV) += -lrte_pmd_sw_event
Signature · 1 line
2.7.4
author
#2 lore raw

v3 cover add ISA-L compression PMD

This patchset contains the first compression PMD written under the DPDK compression API, compressdev. The ISA-L compression driver utilizes Intel's ISA-L compression library. It therefore has dependencies on both compressdev and the ISA-L library, v2.22.0.

V2:
- Changes to keep in compliance with compressdev API,
- Enable meson build system,
- General rework & fixes,
- Documentation.

V3:
- Changes to keep in compliance with compressdev API,
- General rework,
- Split into patchset.

Lee Daly (11):
compress/isal: add skeleton ISA-L compression PMD compress/isal: add pmd device init and de-init compress/isal: add basic pmd ops compress/isal: add private xform related ops compress/isal: add queue pair related ops compress/isal: support enqueue/dequeue api compress/isal: add stats related ops compress/isal: add ISA-L compression functionality compress/isal: add ISA-L decomp functionality compress/isal: add generic compression driver docs compress/isal: add ISA-L compression PMD docs

 MAINTAINERS                                        |  11 +
 config/common_base                                 |   5 +
 devtools/test-build.sh                             |   4 +
 doc/guides/compressdevs/features/default.ini       |  42 ++
 doc/guides/compressdevs/features/isal.ini          |  40 ++
 doc/guides/compressdevs/index.rst                  |  13 +
 doc/guides/compressdevs/isal.rst                   |  94 ++++
 doc/guides/compressdevs/overview.rst               |  12 +
 doc/guides/compressdevs/overview_feature_table.txt |  81 ++++
 doc/guides/index.rst                               |   1 +
 doc/guides/rel_notes/release_18_05.rst             |   6 +
 drivers/Makefile                                   |   2 +
 drivers/compress/Makefile                          |   8 +
 drivers/compress/isal/Makefile                     |  31 ++
 drivers/compress/isal/isal_compress_pmd.c          | 508 +++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c      | 337 ++++++++++++++
 drivers/compress/isal/isal_compress_pmd_private.h  |  54 +++
 drivers/compress/isal/meson.build                  |  14 +
 drivers/compress/isal/rte_pmd_isal_version.map     |   3 +
 drivers/compress/meson.build                       |   8 +
 drivers/meson.build                                |   1 +
 mk/rte.app.mk                                      |   5 +
 22 files changed, 1280 insertions(+)
 create mode 100644 doc/guides/compressdevs/features/default.ini
 create mode 100644 doc/guides/compressdevs/features/isal.ini
 create mode 100644 doc/guides/compressdevs/index.rst
 create mode 100644 doc/guides/compressdevs/isal.rst
 create mode 100644 doc/guides/compressdevs/overview.rst
 create mode 100644 doc/guides/compressdevs/overview_feature_table.txt
 create mode 100644 drivers/compress/Makefile
 create mode 100644 drivers/compress/isal/Makefile
 create mode 100644 drivers/compress/isal/isal_compress_pmd.c
 create mode 100644 drivers/compress/isal/isal_compress_pmd_ops.c
 create mode 100644 drivers/compress/isal/isal_compress_pmd_private.h
 create mode 100644 drivers/compress/isal/meson.build
 create mode 100644 drivers/compress/isal/rte_pmd_isal_version.map
 create mode 100644 drivers/compress/meson.build
Signature · 1 line
2.7.4
author
#3 lore raw

PATCH v3 1/11 compress/isal: add skeleton ISA-L compression PMD

---
 config/common_base                             |  5 +++++
 doc/guides/rel_notes/release_18_05.rst         |  6 ++++++
 drivers/Makefile                               |  2 ++
 drivers/compress/Makefile                      |  8 +++++++
 drivers/compress/isal/Makefile                 | 30 ++++++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd.c      | 29 +++++++++++++++++++++++++
 drivers/compress/isal/meson.build              | 14 ++++++++++++
 drivers/compress/isal/rte_pmd_isal_version.map |  3 +++
 drivers/compress/meson.build                   |  8 +++++++
 drivers/meson.build                            |  1 +
 mk/rte.app.mk                                  |  5 +++++
 11 files changed, 111 insertions(+)
 create mode 100644 drivers/compress/Makefile
 create mode 100644 drivers/compress/isal/Makefile
 create mode 100644 drivers/compress/isal/isal_compress_pmd.c
 create mode 100644 drivers/compress/isal/meson.build
 create mode 100644 drivers/compress/isal/rte_pmd_isal_version.map
 create mode 100644 drivers/compress/meson.build
config/common_base +5 -0
index 5053c89..0992fe1 100644
@@ -557,6 +557,11 @@ CONFIG_RTE_LIBRTE_COMPRESSDEV=y
 CONFIG_RTE_COMPRESS_MAX_DEVS=64
 
 #
+# Compile PMD for ISA-L compression device
+#
+CONFIG_RTE_LIBRTE_PMD_ISAL=n
+
+#
 # Compile generic event device library
 #
 CONFIG_RTE_LIBRTE_EVENTDEV=y
doc/guides/rel_notes/release_18_05.rst +6 -0
index 5193a62..fa0a624 100644
@@ -75,6 +75,12 @@ New Features
   The compressdev library provides an API for offload of compression and
   decompression operations to hardware or software accelerator devices.
 
+* **Added a new compression poll mode driver using Intels ISA-L.**
+
+   Added the new ``ISA-L`` compression driver. See the
+   :doc:`../compressdevs/isal` compression driver guide for more details on
+   this new driver.
+
 
 
 
drivers/Makefile +2 -0
index 3d9f86b..c88638c 100644
@@ -13,6 +13,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_BBDEV) += baseband
 DEPDIRS-baseband := common bus mempool
 DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
 DEPDIRS-crypto := common bus mempool
+DIRS-$(CONFIG_RTE_LIBRTE_COMPRESSDEV) += compress
+DEPDIRS-compress := bus mempool
 DIRS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += event
 DEPDIRS-event := common bus mempool net
 DIRS-$(CONFIG_RTE_LIBRTE_RAWDEV) += raw
drivers/compress/Makefile new file +8 -0
new file mode 100644
index 0000000..592497f
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal
+
+include $(RTE_SDK)/mk/rte.subdir.mk
drivers/compress/isal/Makefile new file +30 -0
new file mode 100644
index 0000000..9b1d866
@@ -0,0 +1,30 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_pmd_isal_comp.a
+
+# build flags
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+# external library dependencies
+LDLIBS += -lisal
+LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring
+LDLIBS += -lrte_compressdev
+LDLIBS += -lrte_bus_vdev
+
+# library version
+LIBABIVER := 1
+
+# versioning export map
+EXPORT_MAP := rte_pmd_isal_version.map
+
+# library source files
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal_compress_pmd.c
+
+# export include files
+include $(RTE_SDK)/mk/rte.lib.mk
drivers/compress/isal/isal_compress_pmd.c new file +29 -0
new file mode 100644
index 0000000..d7137fd
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include <rte_bus_vdev.h>
+#include <rte_compressdev_pmd.h>
+
+/** Remove compression device */
+static int
+compdev_isal_remove_dev(struct rte_vdev_device *vdev __rte_unused)
+{
+	return 0;
+}
+
+/** Initialise ISA-L compression device */
+static int
+compdev_isal_probe(struct rte_vdev_device *dev __rte_unused)
+{
+	return 0;
+}
+
+static struct rte_vdev_driver compdev_isal_pmd_drv = {
+	.probe = compdev_isal_probe,
+	.remove = compdev_isal_remove_dev,
+};
+
+RTE_PMD_REGISTER_VDEV(COMPDEV_NAME_ISAL_PMD, compdev_isal_pmd_drv);
+RTE_PMD_REGISTER_PARAM_STRING(COMPDEV_NAME_ISAL_PMD,
+	"socket_id=<int>");
drivers/compress/isal/meson.build new file +14 -0
new file mode 100644
index 0000000..4447e20
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2018 Intel Corporation
+
+dep = dependency('libisal', required: false)
+if not dep.found()
+       build =false
+endif
+
+deps += 'bus_vdev'
+sources = files('isal_compress_pmd.c')
+ext_deps += dep
+pkgconfig_extra_libs += '-lisal'
+
+allow_experimental_apis = true
drivers/compress/isal/rte_pmd_isal_version.map new file +3 -0
new file mode 100644
index 0000000..de8e412
@@ -0,0 +1,3 @@
+DPDK_18.05 {
+	local: *;
+};
drivers/compress/meson.build new file +8 -0
new file mode 100644
index 0000000..fb136e1
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+drivers = ['isal']
+
+std_deps = ['compressdev'] # compressdev pulls in all other needed deps
+config_flag_fmt = 'RTE_LIBRTE_@0@_PMD'
+driver_name_fmt = 'rte_pmd_@0@'
drivers/meson.build +1 -0
index b146f09..ac7ba5e 100644
@@ -7,6 +7,7 @@ driver_classes = ['common',
 	       'mempool', # depends on common and bus.
 	       'net',     # depends on common, bus and mempool.
 	       'crypto',  # depends on common, bus and mempool (net in future).
+	       'compress', # depends on bus, mempool.
 	       'event']   # depends on common, bus, mempool and net.
 
 foreach class:driver_classes
mk/rte.app.mk +5 -0
index 8530ac5..7a6739c 100644
@@ -228,6 +228,11 @@ endif # CONFIG_RTE_LIBRTE_DPAA_BUS
 
 endif # CONFIG_RTE_LIBRTE_CRYPTODEV
 
+ifeq ($(CONFIG_RTE_LIBRTE_COMPRESSDEV),y)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += -lrte_pmd_isal_comp
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += -lisal
+endif # CONFIG_RTE_LIBRTE_COMPRESSDEV
+
 ifeq ($(CONFIG_RTE_LIBRTE_EVENTDEV),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SKELETON_EVENTDEV) += -lrte_pmd_skeleton_event
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV) += -lrte_pmd_sw_event
Signature · 1 line
2.7.4
#4 lore raw

Re: [PATCH v3 01/11] compress/isal: add skeleton ISA-L compression PMD

Hi Lee,

Quoted 9 lines

-----Original Message-----
From: Daly, Lee
Sent: Tuesday, April 17, 2018 2:35 PM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v3 01/11] compress/isal: add skeleton ISA-L compression PMD

Add some explanation on the commit message about the patch (especially for this one, where you are introducing the PMD).
Same applies for the other commits.

Quoted 43 lines
---
 config/common_base                             |  5 +++++
 doc/guides/rel_notes/release_18_05.rst         |  6 ++++++
 drivers/Makefile                               |  2 ++
 drivers/compress/Makefile                      |  8 +++++++
 drivers/compress/isal/Makefile                 | 30 ++++++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd.c      | 29
+++++++++++++++++++++++++
 drivers/compress/isal/meson.build              | 14 ++++++++++++
 drivers/compress/isal/rte_pmd_isal_version.map |  3 +++
 drivers/compress/meson.build                   |  8 +++++++
 drivers/meson.build                            |  1 +
 mk/rte.app.mk                                  |  5 +++++
 11 files changed, 111 insertions(+)
 create mode 100644 drivers/compress/Makefile  create mode 100644
drivers/compress/isal/Makefile  create mode 100644
drivers/compress/isal/isal_compress_pmd.c
 create mode 100644 drivers/compress/isal/meson.build  create mode 100644
drivers/compress/isal/rte_pmd_isal_version.map
 create mode 100644 drivers/compress/meson.build
config/common_base index renamed from config/common_base +0 -0

5053c89..0992fe1 100644

config/common_base +4 -0
@@ -557,6 +557,11 @@ CONFIG_RTE_LIBRTE_COMPRESSDEV=y
 CONFIG_RTE_COMPRESS_MAX_DEVS=64
 
 #
+# Compile PMD for ISA-L compression device #
+CONFIG_RTE_LIBRTE_PMD_ISAL=n
+
+#
 # Compile generic event device library
 #
 CONFIG_RTE_LIBRTE_EVENTDEV=y
doc/guides/rel_notes/release_18_05.rst +0 -0
index 5193a62..fa0a624 100644

doc/guides/rel_notes/release_18_05.rst New Features · line 82

Quoted 8 lines
doc/guides/rel_notes/release_18_05.rst +5 -0
@@ -75,6 +75,12 @@ New Features
   The compressdev library provides an API for offload of compression and
   decompression operations to hardware or software accelerator devices.
 
+* **Added a new compression poll mode driver using Intels ISA-L.**
+
+   Added the new ``ISA-L`` compression driver. See the
+   :doc:`../compressdevs/isal` compression driver guide for more details on
+   this new driver.

The reference to the document is not yet available, so remove it from here and add it in the last patch.

Quoted 107 lines
drivers/compress/isal/isal_compress_pmd.c new file +24 -0
new file mode 100644
index 0000000..d7137fd
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include <rte_bus_vdev.h>
+#include <rte_compressdev_pmd.h>
+
+/** Remove compression device */
+static int
+compdev_isal_remove_dev(struct rte_vdev_device *vdev __rte_unused) {
+	return 0;
+}
+
+/** Initialise ISA-L compression device */ static int
+compdev_isal_probe(struct rte_vdev_device *dev __rte_unused) {
+	return 0;
+}
+
+static struct rte_vdev_driver compdev_isal_pmd_drv = {
+	.probe = compdev_isal_probe,
+	.remove = compdev_isal_remove_dev,
+};
+
+RTE_PMD_REGISTER_VDEV(COMPDEV_NAME_ISAL_PMD,

compdev_isal_pmd_drv);

drivers/compress/isal/isal_compress_pmd.c new file +2 -0
+RTE_PMD_REGISTER_PARAM_STRING(COMPDEV_NAME_ISAL_PMD,
+	"socket_id=<int>");
drivers/compress/isal/meson.build new file +13 -0
new file mode 100644
index 0000000..4447e20
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause # Copyright 2018 Intel
+Corporation
+
+dep = dependency('libisal', required: false) if not dep.found()
+       build =false
+endif
+
+deps += 'bus_vdev'
+sources = files('isal_compress_pmd.c')
+ext_deps += dep
+pkgconfig_extra_libs += '-lisal'
+
+allow_experimental_apis = true
drivers/compress/isal/rte_pmd_isal_version.map new file +3 -0
new file mode 100644
index 0000000..de8e412
@@ -0,0 +1,3 @@
+DPDK_18.05 {
+	local: *;
+};
drivers/compress/meson.build new renamed from drivers/compress/meson.build +0 -0

file mode 100644 index 0000000..fb136e1

drivers/compress/meson.build new file +8 -0
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel
+Corporation
+
+drivers = ['isal']
+
+std_deps = ['compressdev'] # compressdev pulls in all other needed deps
+config_flag_fmt = 'RTE_LIBRTE_@0@_PMD'
+driver_name_fmt = 'rte_pmd_@0@'
drivers/meson.build index b146f09..ac7ba5e renamed from drivers/meson.build +0 -0

100644

drivers/meson.build +1 -0
@@ -7,6 +7,7 @@ driver_classes = ['common',
 	       'mempool', # depends on common and bus.
 	       'net',     # depends on common, bus and mempool.
 	       'crypto',  # depends on common, bus and mempool (net in future).
+	       'compress', # depends on bus, mempool.
 	       'event']   # depends on common, bus, mempool and net.
 
 foreach class:driver_classes
mk/rte.app.mk +5 -0
@@ -228,6 +228,11 @@ endif # CONFIG_RTE_LIBRTE_DPAA_BUS
 
 endif # CONFIG_RTE_LIBRTE_CRYPTODEV
 
+ifeq ($(CONFIG_RTE_LIBRTE_COMPRESSDEV),y)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += -lrte_pmd_isal_comp
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += -lisal endif #
+CONFIG_RTE_LIBRTE_COMPRESSDEV
+
 ifeq ($(CONFIG_RTE_LIBRTE_EVENTDEV),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SKELETON_EVENTDEV) += -

lrte_pmd_skeleton_event
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV) += -lrte_pmd_sw_event
--
2.7.4

author
#5 lore raw

PATCH v3 2/11 compress/isal: add pmd device init and de-init

In reply to Lee Daly
---
 drivers/compress/isal/Makefile                    |  1 +
 drivers/compress/isal/isal_compress_pmd.c         | 96 ++++++++++++++++++++++-
 drivers/compress/isal/isal_compress_pmd_ops.c     | 25 ++++++
 drivers/compress/isal/isal_compress_pmd_private.h | 24 ++++++
 drivers/compress/isal/meson.build                 |  2 +-
 5 files changed, 145 insertions(+), 3 deletions(-)
 create mode 100644 drivers/compress/isal/isal_compress_pmd_ops.c
 create mode 100644 drivers/compress/isal/isal_compress_pmd_private.h
drivers/compress/isal/Makefile +1 -0
index 9b1d866..95904f6 100644
@@ -25,6 +25,7 @@ EXPORT_MAP := rte_pmd_isal_version.map
 
 # library source files
 SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal_compress_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal_compress_pmd_ops.c
 
 # export include files
 include $(RTE_SDK)/mk/rte.lib.mk
drivers/compress/isal/isal_compress_pmd.c +94 -2

117 lines not on this page · raw message

drivers/compress/isal/isal_compress_pmd_ops.c new file +25 -0
new file mode 100644
index 0000000..cff05b4
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include <rte_compressdev_pmd.h>
+
+struct rte_compressdev_ops isal_pmd_ops = {
+		.dev_configure		= NULL,
+		.dev_start		= NULL,
+		.dev_stop		= NULL,
+		.dev_close		= NULL,
+
+		.stats_get		= NULL,
+		.stats_reset		= NULL,
+
+		.dev_infos_get		= NULL,
+
+		.queue_pair_setup	= NULL,
+		.queue_pair_release	= NULL,
+
+		.private_xform_create	= NULL,
+		.private_xform_free	= NULL,
+};
+
+struct rte_compressdev_ops *isal_compress_pmd_ops = &isal_pmd_ops;
drivers/compress/isal/isal_compress_pmd_private.h new file +24 -0
new file mode 100644
index 0000000..09ecfb7
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#ifndef _ISAL_COMP_PMD_PRIVATE_H_
+#define _ISAL_COMP_PMD_PRIVATE_H_
+
+#define COMPDEV_NAME_ISAL_PMD		compress_isal
+/**< ISA-L comp PMD device name */
+
+extern int isal_logtype_driver;
+#define ISAL_PMD_LOG(level, fmt, args...) \
+	rte_log(RTE_LOG_ ## level, isal_logtype_driver, "%s(): "fmt "\n", \
+			__func__, ##args)
+
+/* private data structure for each ISA-L compression device */
+struct isal_comp_private {
+	struct rte_mempool *priv_xform_mp;
+};
+
+/** device specific operations function pointer structure */
+extern struct rte_compressdev_ops *isal_compress_pmd_ops;
+
+#endif /* _ISAL_COMP_PMD_PRIVATE_H_ */
drivers/compress/isal/meson.build +1 -1
index 4447e20..94c10fd 100644
@@ -7,7 +7,7 @@ if not dep.found()
 endif
 
 deps += 'bus_vdev'
-sources = files('isal_compress_pmd.c')
+sources = files('isal_compress_pmd.c', 'isal_compress_pmd_ops.c')
 ext_deps += dep
 pkgconfig_extra_libs += '-lisal'
 
Signature · 1 line
2.7.4
#6 lore raw

Re: [PATCH v3 02/11] compress/isal: add pmd device init and de-init

Quoted 12 lines

-----Original Message-----
From: Daly, Lee
Sent: Tuesday, April 17, 2018 2:35 PM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v3 02/11] compress/isal: add pmd device init and de-init

...

Quoted 24 lines

+++ b/drivers/compress/isal/isal_compress_pmd.c

drivers/compress/isal/isal_compress_pmd.c +17 -0
@@ -3,20 +3,102 @@
  */
 
 #include <rte_bus_vdev.h>
+#include <rte_common.h>
+#include <rte_malloc.h>
 #include <rte_compressdev_pmd.h>
 
+#include "isal_compress_pmd_private.h"
+
+int isal_logtype_driver;
+
+/* Enqueue burst */
+static uint16_t
+isal_comp_pmd_enqueue_burst(void *queue_pair __rte_unused,
+			struct rte_comp_op **ops __rte_unused,
+			uint16_t nb_ops __rte_unused)
+{
+	uint16_t num_enq = 0;
+
+	return num_enq;
+}
+

drivers/compress/isal/isal_compress_pmd.c line 34

Quoted 10 lines
drivers/compress/isal/isal_compress_pmd.c +10 -0
+/* Dequeue burst */
+static uint16_t
+isal_comp_pmd_dequeue_burst(void *queue_pair __rte_unused,
+		struct rte_comp_op **ops __rte_unused,
+		uint16_t nb_ops __rte_unused)
+{
+	uint16_t nb_dequeued = 0;
+
+	return nb_dequeued;
+}

Since you have a separate patch implementing these two functions (enqueue/dequeuer), I would directly declare them in that patch. Therefore, also remove the function pointers setting in compdev_isal_create and leave it for that patch.

Quoted 21 lines
drivers/compress/isal/isal_compress_pmd.c +21 -0
+
+/* Create ISA-L compression device */
+static int
+compdev_isal_create(const char *name, struct rte_vdev_device *vdev,
+		struct rte_compressdev_pmd_init_params *init_params) {
+	struct rte_compressdev *dev;
+
+	dev = rte_compressdev_pmd_create(name, &vdev->device,
+			sizeof(struct isal_comp_private), init_params);
+	if (dev == NULL) {
+		ISAL_PMD_LOG(ERR, "failed to create compressdev vdev");
+		return -EFAULT;
+	}
+
+	dev->dev_ops = isal_compress_pmd_ops;
+
+	/* register rx/tx burst functions for data path */
+	dev->dequeue_burst = isal_comp_pmd_dequeue_burst;
+	dev->enqueue_burst = isal_comp_pmd_enqueue_burst;
+
+	return 0;
author
#7 lore raw

PATCH v3 3/11 compress/isal: add basic pmd ops

In reply to Lee Daly
---
 drivers/compress/isal/isal_compress_pmd_ops.c     | 62 +++++++++++++++++++++--
 drivers/compress/isal/isal_compress_pmd_private.h | 12 +++++
 2 files changed, 69 insertions(+), 5 deletions(-)
drivers/compress/isal/isal_compress_pmd_ops.c +57 -5
index cff05b4..ee1ee48 100644
@@ -2,18 +2,70 @@
  * Copyright(c) 2018 Intel Corporation
  */
 
+#include <rte_common.h>
 #include <rte_compressdev_pmd.h>
 
+#include "isal_compress_pmd_private.h"
+
+static const struct rte_compressdev_capabilities isal_pmd_capabilities[] = {
+	RTE_COMP_END_OF_CAPABILITIES_LIST()
+};
+
+/** Configure device */
+static int
+isal_comp_pmd_config(struct rte_compressdev *dev __rte_unused,
+		struct rte_compressdev_config *config __rte_unused)
+{
+	return 0;
+}
+
+/** Start device */
+static int
+isal_comp_pmd_start(__rte_unused struct rte_compressdev *dev)
+{
+	return 0;
+}
+
+/** Stop device */
+static void
+isal_comp_pmd_stop(__rte_unused struct rte_compressdev *dev)
+{
+}
+
+/** Close device */
+static int
+isal_comp_pmd_close(__rte_unused struct rte_compressdev *dev)
+{
+	return 0;
+}
+
+/** Get device info */
+static void
+isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
+		struct rte_compressdev_info *dev_info)
+{
+	if (dev_info != NULL) {
+		dev_info->capabilities = isal_pmd_capabilities;
+		dev_info->feature_flags = RTE_COMPDEV_FF_CPU_AVX512 |
+				RTE_COMPDEV_FF_CPU_AVX2 |
+				RTE_COMPDEV_FF_CPU_AVX |
+				RTE_COMPDEV_FF_CPU_SSE;
+	}
+}
+
+
+
+
 struct rte_compressdev_ops isal_pmd_ops = {
-		.dev_configure		= NULL,
-		.dev_start		= NULL,
-		.dev_stop		= NULL,
-		.dev_close		= NULL,
+		.dev_configure		= isal_comp_pmd_config,
+		.dev_start		= isal_comp_pmd_start,
+		.dev_stop		= isal_comp_pmd_stop,
+		.dev_close		= isal_comp_pmd_close,
 
 		.stats_get		= NULL,
 		.stats_reset		= NULL,
 
-		.dev_infos_get		= NULL,
+		.dev_infos_get		= isal_comp_pmd_info_get,
 
 		.queue_pair_setup	= NULL,
 		.queue_pair_release	= NULL,
drivers/compress/isal/isal_compress_pmd_private.h +12 -0
index 09ecfb7..efbe68b 100644
@@ -18,6 +18,18 @@ struct isal_comp_private {
 	struct rte_mempool *priv_xform_mp;
 };
 
+/** ISA-L queue pair */
+struct isal_comp_qp {
+	/* Queue Pair Identifier */
+	uint16_t id;
+	/* Unique Queue Pair Name */
+	char name[RTE_COMPRESSDEV_NAME_MAX_LEN];
+	/* Queue pair statistics */
+	struct rte_compressdev_stats qp_stats;
+	/* Number of free elements on ring */
+	uint16_t num_free_elements;
+} __rte_cache_aligned;
+
 /** device specific operations function pointer structure */
 extern struct rte_compressdev_ops *isal_compress_pmd_ops;
 
Signature · 1 line
2.7.4
#8 lore raw

Re: [PATCH v3 03/11] compress/isal: add basic pmd ops

Quoted 65 lines

-----Original Message-----
From: Daly, Lee
Sent: Tuesday, April 17, 2018 2:35 PM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v3 03/11] compress/isal: add basic pmd ops

---
 drivers/compress/isal/isal_compress_pmd_ops.c     | 62
+++++++++++++++++++++--
 drivers/compress/isal/isal_compress_pmd_private.h | 12 +++++
 2 files changed, 69 insertions(+), 5 deletions(-)
drivers/compress/isal/isal_compress_pmd_ops.c +36 -0
index cff05b4..ee1ee48 100644
@@ -2,18 +2,70 @@
  * Copyright(c) 2018 Intel Corporation
  */
 
+#include <rte_common.h>
 #include <rte_compressdev_pmd.h>
 
+#include "isal_compress_pmd_private.h"
+
+static const struct rte_compressdev_capabilities isal_pmd_capabilities[] = {
+	RTE_COMP_END_OF_CAPABILITIES_LIST()
+};
+
+/** Configure device */
+static int
+isal_comp_pmd_config(struct rte_compressdev *dev __rte_unused,
+		struct rte_compressdev_config *config __rte_unused) {
+	return 0;
+}
+
+/** Start device */
+static int
+isal_comp_pmd_start(__rte_unused struct rte_compressdev *dev) {
+	return 0;
+}
+
+/** Stop device */
+static void
+isal_comp_pmd_stop(__rte_unused struct rte_compressdev *dev) { }
+
+/** Close device */
+static int
+isal_comp_pmd_close(__rte_unused struct rte_compressdev *dev) {
+	return 0;
+}
+
+/** Get device info */
+static void
+isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
+		struct rte_compressdev_info *dev_info) {
+	if (dev_info != NULL) {
+		dev_info->capabilities = isal_pmd_capabilities;

drivers/compress/isal/isal_compress_pmd_ops.c line 52

Quoted 10 lines
drivers/compress/isal/isal_compress_pmd_ops.c +10 -0
+		dev_info->feature_flags = RTE_COMPDEV_FF_CPU_AVX512 |
+				RTE_COMPDEV_FF_CPU_AVX2 |
+				RTE_COMPDEV_FF_CPU_AVX |
+				RTE_COMPDEV_FF_CPU_SSE;
+	}
+}
+
+
+
+

Remove extra blank lines.

author
#9 lore raw

PATCH v3 4/11 compress/isal: add private xform related ops

In reply to Lee Daly
---
 drivers/compress/isal/isal_compress_pmd.c         | 187 ++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c     |  96 ++++++++++-
 drivers/compress/isal/isal_compress_pmd_private.h |  12 ++
 3 files changed, 292 insertions(+), 3 deletions(-)
drivers/compress/isal/isal_compress_pmd.c +187 -0
index 8e658b4..8db6380 100644
@@ -9,8 +9,195 @@
 
 #include "isal_compress_pmd_private.h"
 
+#define RTE_COMP_ISAL_WINDOW_SIZE 15
+#define RTE_COMP_ISAL_LEVEL_ZERO 0 /* ISA-L Level 0 used for fixed Huffman */
+#define RTE_COMP_ISAL_LEVEL_ONE 1
+#define RTE_COMP_ISAL_LEVEL_TWO 2
+#define RTE_COMP_ISAL_LEVEL_THREE 3 /* Optimised for AVX512 & AVX2 only */
+
 int isal_logtype_driver;
 
+/* Verify and set private xform parameters */
+int
+isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
+		const struct rte_comp_xform *xform)
+{
+	if (xform == NULL)
+		return -EINVAL;
+
+	/* Check for chained xforms */
+	if (xform->next != NULL) {
+		ISAL_PMD_LOG(ERR, "Chained xforms not supported\n");
+		return -ENOTSUP;
+	}
+
+	/* Set compression private xform variables */
+	if (xform->type == RTE_COMP_COMPRESS) {
+		/* Set private xform type - COMPRESS/DECOMPRESS */
+		priv_xform->type = RTE_COMP_COMPRESS;
+
+		/* Set private xform algorithm */
+		if (xform->compress.algo != RTE_COMP_ALGO_DEFLATE) {
+			if (xform->compress.algo == RTE_COMP_ALGO_NULL) {
+				ISAL_PMD_LOG(ERR, "By-pass not supported\n");
+				return -ENOTSUP;
+			}
+			ISAL_PMD_LOG(ERR, "Algorithm not supported\n");
+			return -ENOTSUP;
+		}
+		priv_xform->compress.algo = RTE_COMP_ALGO_DEFLATE;
+
+		/* Set private xform checksum */
+		switch (xform->compress.chksum) {
+		case(RTE_COMP_CHECKSUM_NONE):
+			priv_xform->compress.chksum = RTE_COMP_CHECKSUM_NONE;
+			break;
+		case(RTE_COMP_CHECKSUM_ADLER32):
+			priv_xform->compress.chksum = RTE_COMP_CHECKSUM_ADLER32;
+			break;
+		case(RTE_COMP_CHECKSUM_CRC32):
+			priv_xform->compress.chksum = RTE_COMP_CHECKSUM_CRC32;
+			break;
+		default:
+			ISAL_PMD_LOG(ERR, "Checksum not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform window size, 32K supported */
+		if (xform->compress.window_size == RTE_COMP_ISAL_WINDOW_SIZE)
+			priv_xform->compress.window_size =
+					RTE_COMP_ISAL_WINDOW_SIZE;
+		else {
+			ISAL_PMD_LOG(ERR, "Window size not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform huffman type */
+		switch (xform->compress.deflate.huffman) {
+		case(RTE_COMP_HUFFMAN_DEFAULT):
+			priv_xform->compress.deflate.huffman =
+					RTE_COMP_HUFFMAN_DEFAULT;
+			break;
+		case(RTE_COMP_HUFFMAN_FIXED):
+			priv_xform->compress.deflate.huffman =
+					RTE_COMP_HUFFMAN_FIXED;
+			break;
+		case(RTE_COMP_HUFFMAN_DYNAMIC):
+			priv_xform->compress.deflate.huffman =
+					RTE_COMP_HUFFMAN_DYNAMIC;
+			break;
+		default:
+			ISAL_PMD_LOG(ERR, "Huffman code not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform level.
+		 * Checking compliance with compressdev API, -1 <= level => 9
+		 */
+		if (xform->compress.level < RTE_COMP_LEVEL_PMD_DEFAULT ||
+				xform->compress.level > RTE_COMP_LEVEL_MAX) {
+			ISAL_PMD_LOG(ERR, "Compression level out of range\n");
+			return -EINVAL;
+		}
+		/* Check for Compressdev API level 0, No compression
+		 * not supported in ISA-L
+		 */
+		else if (xform->compress.level == RTE_COMP_LEVEL_NONE) {
+			ISAL_PMD_LOG(ERR, "No Compression not supported\n");
+			return -ENOTSUP;
+		}
+		/* If using fixed huffman code, level must be 0 */
+		else if (priv_xform->compress.deflate.huffman ==
+				RTE_COMP_HUFFMAN_FIXED) {
+			ISAL_PMD_LOG(DEBUG, "ISA-L level 0 used due to a"
+					" fixed huffman code\n");
+			priv_xform->compress.level = RTE_COMP_ISAL_LEVEL_ZERO;
+		} else {
+			/* Mapping API levels to ISA-L levels 1,2 & 3 */
+			switch (xform->compress.level) {
+			case RTE_COMP_LEVEL_PMD_DEFAULT:
+				/* Default is 1 if not using fixed huffman */
+				priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_ONE;
+				break;
+			case RTE_COMP_LEVEL_MIN:
+				priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_ONE;
+				break;
+			case RTE_COMP_ISAL_LEVEL_TWO:
+				priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_TWO;
+				break;
+			/* Level 3 or higher requested */
+			default:
+				/* Check for AVX512, to use ISA-L level 3 */
+				if (rte_cpu_get_flag_enabled(
+						RTE_CPUFLAG_AVX512F))
+					priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_THREE;
+				/* Check for AVX2, to use ISA-L level 3 */
+				else if (rte_cpu_get_flag_enabled(
+						RTE_CPUFLAG_AVX2))
+					priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_THREE;
+				else{
+					ISAL_PMD_LOG(DEBUG, "Requested ISA-L level"
+						" 3 or above; Level 3 optimized"
+						" for AVX512 & AVX2 only."
+						" level changed to 2.\n");
+					priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_TWO;
+				}
+			}
+		}
+	}
+
+	/* Set decompression private xform variables */
+	else if (xform->type == RTE_COMP_DECOMPRESS) {
+
+		/* Set private xform type - COMPRESS/DECOMPRESS */
+		priv_xform->type = RTE_COMP_DECOMPRESS;
+
+		/* Set private xform algorithm */
+		if (xform->decompress.algo != RTE_COMP_ALGO_DEFLATE) {
+			if (xform->decompress.algo == RTE_COMP_ALGO_NULL) {
+				ISAL_PMD_LOG(ERR, "By pass not supported\n");
+				return -ENOTSUP;
+			}
+			ISAL_PMD_LOG(ERR, "Algorithm not supported\n");
+			return -ENOTSUP;
+		}
+		priv_xform->decompress.algo = RTE_COMP_ALGO_DEFLATE;
+
+		/* Set private xform checksum */
+		switch (xform->decompress.chksum) {
+		case(RTE_COMP_CHECKSUM_NONE):
+			priv_xform->decompress.chksum = RTE_COMP_CHECKSUM_NONE;
+			break;
+		case(RTE_COMP_CHECKSUM_ADLER32):
+			priv_xform->decompress.chksum =
+					RTE_COMP_CHECKSUM_ADLER32;
+			break;
+		case(RTE_COMP_CHECKSUM_CRC32):
+			priv_xform->decompress.chksum = RTE_COMP_CHECKSUM_CRC32;
+			break;
+		default:
+			ISAL_PMD_LOG(ERR, "Checksum not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform window size, 32K supported */
+		if (xform->decompress.window_size == RTE_COMP_ISAL_WINDOW_SIZE)
+			priv_xform->decompress.window_size =
+					RTE_COMP_ISAL_WINDOW_SIZE;
+		else {
+			ISAL_PMD_LOG(ERR, "Window size not supported\n");
+			return -ENOTSUP;
+		}
+	}
+	return 0;
+}
+
 /* Enqueue burst */
 static uint16_t
 isal_comp_pmd_enqueue_burst(void *queue_pair __rte_unused,
drivers/compress/isal/isal_compress_pmd_ops.c +93 -3

116 lines not on this page · raw message

drivers/compress/isal/isal_compress_pmd_private.h +12 -0
index efbe68b..5c27939 100644
@@ -30,6 +30,18 @@ struct isal_comp_qp {
 	uint16_t num_free_elements;
 } __rte_cache_aligned;
 
+/** ISA-L private xform structure */
+struct isal_priv_xform {
+	enum rte_comp_xform_type type;
+	struct rte_comp_compress_xform compress;
+	struct rte_comp_decompress_xform decompress;
+} __rte_cache_aligned;
+
+/** Set and validate NULL comp private xform parameters */
+extern int
+isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
+			const struct rte_comp_xform *xform);
+
 /** device specific operations function pointer structure */
 extern struct rte_compressdev_ops *isal_compress_pmd_ops;
 
Signature · 1 line
2.7.4
#10 lore raw

Re: [PATCH v3 04/11] compress/isal: add private xform related ops

Quoted 38 lines

-----Original Message-----
From: Daly, Lee
Sent: Tuesday, April 17, 2018 2:35 PM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v3 04/11] compress/isal: add private xform related ops

---
 drivers/compress/isal/isal_compress_pmd.c         | 187
++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c     |  96 ++++++++++-
 drivers/compress/isal/isal_compress_pmd_private.h |  12 ++
 3 files changed, 292 insertions(+), 3 deletions(-)
drivers/compress/isal/isal_compress_pmd.c +8 -0
index 8e658b4..8db6380 100644
@@ -9,8 +9,195 @@
 
 #include "isal_compress_pmd_private.h"
 
+#define RTE_COMP_ISAL_WINDOW_SIZE 15
+#define RTE_COMP_ISAL_LEVEL_ZERO 0 /* ISA-L Level 0 used for fixed
+Huffman */ #define RTE_COMP_ISAL_LEVEL_ONE 1 #define
+RTE_COMP_ISAL_LEVEL_TWO 2 #define RTE_COMP_ISAL_LEVEL_THREE 3 /*
+Optimised for AVX512 & AVX2 only */
+
 int isal_logtype_driver;
 
+/* Verify and set private xform parameters */ int
+isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,

drivers/compress/isal/isal_compress_pmd.c line 31

Quoted 10 lines
drivers/compress/isal/isal_compress_pmd.c +10 -0
+		const struct rte_comp_xform *xform)
+{
+	if (xform == NULL)
+		return -EINVAL;
+
+	/* Check for chained xforms */
+	if (xform->next != NULL) {
+		ISAL_PMD_LOG(ERR, "Chained xforms not supported\n");
+		return -ENOTSUP;
+	}

The next pointer will be removed in the API v6 patchset, so you can remove this.

...

drivers/compress/isal/isal_compress_pmd.c static int · line 17

Quoted 4 lines
drivers/compress/isal/isal_compress_pmd.c +1 -0
+++ b/drivers/compress/isal/isal_compress_pmd_ops.c
@@ -16,7 +16,56 @@ static int
 isal_comp_pmd_config(struct rte_compressdev *dev __rte_unused,
 		struct rte_compressdev_config *config __rte_unused)  {

Remove the __rte_unused here, as you are now using these structure pointers.

...

Quoted 6 lines

git a/drivers/compress/isal/isal_compress_pmd_private.h b/drivers/compress/isal/isal_compress_pmd_private.h index efbe68b..5c27939 100644

drivers/compress/isal/isal_compress_pmd_private.h struct isal_comp_qp { · line 37

Quoted 8 lines
drivers/compress/isal/isal_compress_pmd_private.h +5 -0
@@ -30,6 +30,18 @@ struct isal_comp_qp {
 	uint16_t num_free_elements;
 } __rte_cache_aligned;
 
+/** ISA-L private xform structure */
+struct isal_priv_xform {
+	enum rte_comp_xform_type type;
+	struct rte_comp_compress_xform compress;
+	struct rte_comp_decompress_xform decompress; }

Xform will be either for compression or decompression, so use a union with compress and decompress structures inside.

author
#11 lore raw

PATCH v3 5/11 compress/isal: add queue pair related ops

In reply to Lee Daly
---
 drivers/compress/isal/isal_compress_pmd_ops.c     | 110 +++++++++++++++++++++-
 drivers/compress/isal/isal_compress_pmd_private.h |   2 +
 2 files changed, 110 insertions(+), 2 deletions(-)
drivers/compress/isal/isal_compress_pmd_ops.c +108 -2
index 2e9381d..73e4c84 100644
@@ -4,6 +4,7 @@
 
 #include <rte_common.h>
 #include <rte_compressdev_pmd.h>
+#include <rte_malloc.h>
 
 #include "isal_compress_pmd_private.h"
 
@@ -102,6 +103,111 @@ isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
 	}
 }
 
+
+/** Release queue pair */
+static int
+isal_comp_pmd_qp_release(struct rte_compressdev *dev, uint16_t qp_id)
+{
+struct isal_comp_qp *qp = dev->data->queue_pairs[qp_id];
+
+	if (qp == NULL)
+		return -EINVAL;
+
+	if (dev->data->queue_pairs[qp_id] != NULL)
+		rte_free(dev->data->queue_pairs[qp_id]);
+
+	return 0;
+}
+
+/** Create a ring to place process packets on */
+static struct rte_ring *
+isal_comp_pmd_qp_create_processed_pkts_ring(struct isal_comp_qp *qp,
+		unsigned int ring_size, int socket_id)
+{
+	struct rte_ring *r;
+
+	r = rte_ring_lookup(qp->name);
+	if (r) {
+		if (rte_ring_get_size(r) >= ring_size) {
+			ISAL_PMD_LOG(DEBUG,
+				"Reusing existing ring %s for processed packets",
+				qp->name);
+			return r;
+		}
+
+			ISAL_PMD_LOG(ERR,
+					"Unable to reuse existing ring %s for processed packets",
+			 qp->name);
+		return NULL;
+	}
+
+	return rte_ring_create(qp->name, ring_size, socket_id,
+			RING_F_SP_ENQ | RING_F_SC_DEQ);
+}
+
+/** set a unique name for the queue pair based on it's name, dev_id and qp_id */
+static int
+isal_comp_pmd_qp_set_unique_name(struct rte_compressdev *dev,
+struct isal_comp_qp *qp)
+{
+	unsigned int n = snprintf(qp->name, sizeof(qp->name),
+			"isal_compression_pmd_%u_qp_%u",
+			dev->data->dev_id, qp->id);
+
+	if (n >= sizeof(qp->name))
+		return -1;
+
+	return 0;
+}
+
+/* Setup a queue pair */
+static int
+isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
+		uint32_t max_inflight_ops, int socket_id)
+{
+	struct isal_comp_qp *qp = NULL;
+	int retval;
+
+	/* Free memory prior to re-allocation if needed. */
+	if (dev->data->queue_pairs[qp_id] != NULL)
+		isal_comp_pmd_qp_release(dev, qp_id);
+
+	/* Allocate the queue pair data structure. */
+	qp = rte_zmalloc_socket("Isa-l compression PMD Queue Pair", sizeof(*qp),
+					RTE_CACHE_LINE_SIZE, socket_id);
+	if (qp == NULL) {
+		ISAL_PMD_LOG(ERR, "Failed to allocate queue pair memory");
+		return (-ENOMEM);
+	}
+
+	qp->id = qp_id;
+	dev->data->queue_pairs[qp_id] = qp;
+
+	retval = isal_comp_pmd_qp_set_unique_name(dev, qp);
+	if (retval) {
+		ISAL_PMD_LOG(ERR, "Failed to create unique name for isal "
+				"compression device");
+		goto qp_setup_cleanup;
+	}
+
+	qp->processed_pkts = isal_comp_pmd_qp_create_processed_pkts_ring(qp,
+			max_inflight_ops, socket_id);
+	if (qp->processed_pkts == NULL) {
+		ISAL_PMD_LOG(ERR, "Failed to create unique name for isal "
+				"compression device");
+		goto qp_setup_cleanup;
+	}
+
+	memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
+	return 0;
+
+qp_setup_cleanup:
+	if (qp)
+		rte_free(qp);
+
+	return -1;
+}
+
 /** Set private xform data*/
 static int
 isal_comp_pmd_priv_xform_create(struct rte_compressdev *dev,
@@ -157,8 +263,8 @@ struct rte_compressdev_ops isal_pmd_ops = {
 
 		.dev_infos_get		= isal_comp_pmd_info_get,
 
-		.queue_pair_setup	= NULL,
-		.queue_pair_release	= NULL,
+		.queue_pair_setup	= isal_comp_pmd_qp_setup,
+		.queue_pair_release	= isal_comp_pmd_qp_release,
 
 		.private_xform_create	= isal_comp_pmd_priv_xform_create,
 		.private_xform_free	= isal_comp_pmd_priv_xform_free,
drivers/compress/isal/isal_compress_pmd_private.h +2 -0
index 5c27939..05679b8 100644
@@ -24,6 +24,8 @@ struct isal_comp_qp {
 	uint16_t id;
 	/* Unique Queue Pair Name */
 	char name[RTE_COMPRESSDEV_NAME_MAX_LEN];
+	/* Ring for placing process packets */
+	struct rte_ring *processed_pkts;
 	/* Queue pair statistics */
 	struct rte_compressdev_stats qp_stats;
 	/* Number of free elements on ring */
Signature · 1 line
2.7.4
#12 lore raw

Re: [PATCH v3 05/11] compress/isal: add queue pair related ops

Quoted 59 lines

-----Original Message-----
From: Daly, Lee
Sent: Tuesday, April 17, 2018 2:35 PM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v3 05/11] compress/isal: add queue pair related ops

---
 drivers/compress/isal/isal_compress_pmd_ops.c     | 110
+++++++++++++++++++++-
 drivers/compress/isal/isal_compress_pmd_private.h |   2 +
 2 files changed, 110 insertions(+), 2 deletions(-)
drivers/compress/isal/isal_compress_pmd_ops.c +24 -0
index 2e9381d..73e4c84 100644
@@ -4,6 +4,7 @@
 
 #include <rte_common.h>
 #include <rte_compressdev_pmd.h>
+#include <rte_malloc.h>
 
 #include "isal_compress_pmd_private.h"
 
@@ -102,6 +103,111 @@ isal_comp_pmd_info_get(struct rte_compressdev
 *dev __rte_unused,
 	}
 }
 
+
+/** Release queue pair */
+static int
+isal_comp_pmd_qp_release(struct rte_compressdev *dev, uint16_t qp_id) {
+struct isal_comp_qp *qp = dev->data->queue_pairs[qp_id];
+
+	if (qp == NULL)
+		return -EINVAL;
+
+	if (dev->data->queue_pairs[qp_id] != NULL)
+		rte_free(dev->data->queue_pairs[qp_id]);
+
+	return 0;
+}
+
+/** Create a ring to place process packets on */ static struct rte_ring
+* isal_comp_pmd_qp_create_processed_pkts_ring(struct isal_comp_qp *qp,
+		unsigned int ring_size, int socket_id) {
+	struct rte_ring *r;
+
+	r = rte_ring_lookup(qp->name);
+	if (r) {
+		if (rte_ring_get_size(r) >= ring_size) {

drivers/compress/isal/isal_compress_pmd_ops.c isal_comp_pmd_info_get(struct rte_compressdev · line 139

Quoted 10 lines
drivers/compress/isal/isal_compress_pmd_ops.c +8 -0
+			ISAL_PMD_LOG(DEBUG,
+				"Reusing existing ring %s for processed
 packets",
+				qp->name);
+			return r;
+		}
+
+			ISAL_PMD_LOG(ERR,
+					"Unable to reuse existing ring %s for
 processed packets",

Remove one tab here, as the line is too long.

drivers/compress/isal/isal_compress_pmd_ops.c isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,

Quoted 9 lines
drivers/compress/isal/isal_compress_pmd_ops.c +9 -0
+			 qp->name);
+		return NULL;
+	}
+
+	return rte_ring_create(qp->name, ring_size, socket_id,
+			RING_F_SP_ENQ | RING_F_SC_DEQ);
+}
+
+/** set a unique name for the queue pair based on it's name, dev_id and

"its"

drivers/compress/isal/isal_compress_pmd_ops.c isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused, · line 155

Quoted 5 lines
drivers/compress/isal/isal_compress_pmd_ops.c +5 -0
+qp_id */ static int isal_comp_pmd_qp_set_unique_name(struct
+rte_compressdev *dev, struct isal_comp_qp *qp) {
+	unsigned int n = snprintf(qp->name, sizeof(qp->name),
+			"isal_compression_pmd_%u_qp_%u",
+			dev->data->dev_id, qp->id);

Better to use strlcpy.

Quoted 6 lines
drivers/compress/isal/isal_compress_pmd_ops.c +6 -0
+
+	if (n >= sizeof(qp->name))
+		return -1;
+
+	return 0;
+}
author
#13 lore raw

Re: [PATCH v3 05/11] compress/isal: add queue pair related ops

<...>

Quoted 17 lines
Quoted 17 lines

-----Original Message-----
From: Daly, Lee
Sent: Tuesday, April 17, 2018 2:35 PM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v3 05/11] compress/isal: add queue pair related ops

---
 drivers/compress/isal/isal_compress_pmd_ops.c     | 110
+++++++++++++++++++++-
 drivers/compress/isal/isal_compress_pmd_private.h |   2 +
 2 files changed, 110 insertions(+), 2 deletions(-)
Quoted 4 lines

drivers/compress/isal/isal_compress_pmd_ops.c

Quoted 4 lines
drivers/compress/isal/isal_compress_pmd_ops.c +0 -1
index 2e9381d..73e4c84 100644
--- a/drivers/compress/isal/isal_compress_pmd_ops.c

//snip

Quoted 8 lines

drivers/compress/isal/isal_compress_pmd_ops.c isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused, · line 155

Quoted 6 lines
drivers/compress/isal/isal_compress_pmd_ops.c +5 -0
+qp_id */ static int isal_comp_pmd_qp_set_unique_name(struct
+rte_compressdev *dev, struct isal_comp_qp *qp) {
+	unsigned int n = snprintf(qp->name, sizeof(qp->name),
+			"isal_compression_pmd_%u_qp_%u",
+			dev->data->dev_id, qp->id);

Better to use strlcpy.

[Lee] I believe snprintf to be a better fit due to the fact strlcpy doesn't handle variable arguments as inputs, i.e

	"snprintf(dst, length, "%s", src)" 
	"strlcpy(dst, src, length)"

To complete this action with strlcpy would cause more overhead, and snprintf() is still safe as it always null terminates.

Quoted 7 lines
Quoted 7 lines
drivers/compress/isal/isal_compress_pmd_ops.c +6 -0
+
+	if (n >= sizeof(qp->name))
+		return -1;
+
+	return 0;
+}
author
#14 lore raw

PATCH v3 6/11 compress/isal: support enqueue/dequeue api

In reply to Lee Daly
---
 drivers/compress/isal/isal_compress_pmd.c     | 56 ++++++++++++++++++++++-----
 drivers/compress/isal/isal_compress_pmd_ops.c |  2 +
 2 files changed, 49 insertions(+), 9 deletions(-)
drivers/compress/isal/isal_compress_pmd.c +47 -9
index 8db6380..2e19b42 100644
@@ -198,24 +198,62 @@ isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
 	return 0;
 }
 
+/* Process compression operation */
+static int
+process_op(struct isal_comp_qp *qp __rte_unused,
+		struct rte_comp_op *op __rte_unused,
+		struct isal_priv_xform *priv_xform)
+{
+	switch (priv_xform->type) {
+	case RTE_COMP_COMPRESS:
+		break;
+	case RTE_COMP_DECOMPRESS:
+		break;
+	default:
+		ISAL_PMD_LOG(ERR, "Operation Not Supported\n");
+		return -ENOTSUP;
+	}
+	return 0;
+}
+
 /* Enqueue burst */
 static uint16_t
-isal_comp_pmd_enqueue_burst(void *queue_pair __rte_unused,
-			struct rte_comp_op **ops __rte_unused,
-			uint16_t nb_ops __rte_unused)
+isal_comp_pmd_enqueue_burst(void *queue_pair, struct rte_comp_op **ops,
+			uint16_t nb_ops)
 {
-	uint16_t num_enq = 0;
+	struct isal_comp_qp *qp = queue_pair;
+	uint16_t i;
+	int retval;
+	int16_t num_enq = RTE_MIN(qp->num_free_elements, nb_ops);
+
+	for (i = 0; i < num_enq; i++) {
+		retval = process_op(qp, ops[i], ops[i]->private_xform);
+		if (unlikely(retval < 0) ||
+				ops[i]->status != RTE_COMP_OP_STATUS_SUCCESS) {
+			qp->qp_stats.enqueue_err_count++;
+		}
+	}
+
+	retval = rte_ring_enqueue_burst(qp->processed_pkts, (void *)ops,
+			num_enq, NULL);
+	qp->num_free_elements -= retval;
+	qp->qp_stats.enqueued_count += retval;
 
-	return num_enq;
+	return retval;
 }
 
 /* Dequeue burst */
 static uint16_t
-isal_comp_pmd_dequeue_burst(void *queue_pair __rte_unused,
-		struct rte_comp_op **ops __rte_unused,
-		uint16_t nb_ops __rte_unused)
+isal_comp_pmd_dequeue_burst(void *queue_pair, struct rte_comp_op **ops,
+		uint16_t nb_ops)
 {
-	uint16_t nb_dequeued = 0;
+	struct isal_comp_qp *qp = queue_pair;
+	uint16_t nb_dequeued;
+
+	nb_dequeued = rte_ring_dequeue_burst(qp->processed_pkts, (void **)ops,
+			nb_ops, NULL);
+	qp->num_free_elements += nb_dequeued;
+	qp->qp_stats.dequeued_count += nb_dequeued;
 
 	return nb_dequeued;
 }
drivers/compress/isal/isal_compress_pmd_ops.c +2 -0
index 73e4c84..1690c34 100644
@@ -198,6 +198,8 @@ isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
 		goto qp_setup_cleanup;
 	}
 
+	qp->num_free_elements = rte_ring_free_count(qp->processed_pkts);
+
 	memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
 	return 0;
 
Signature · 1 line
2.7.4
author
#15 lore raw

PATCH v3 7/11 compress/isal: add stats related ops

In reply to Lee Daly
---
 drivers/compress/isal/isal_compress_pmd_ops.c | 33 +++++++++++++++++++++++++--
 1 file changed, 31 insertions(+), 2 deletions(-)
drivers/compress/isal/isal_compress_pmd_ops.c +31 -2
index 1690c34..0fcb899 100644
@@ -89,6 +89,24 @@ isal_comp_pmd_close(__rte_unused struct rte_compressdev *dev)
 	return 0;
 }
 
+/** Get device statistics */
+static void
+isal_comp_pmd_stats_get(struct rte_compressdev *dev,
+		struct rte_compressdev_stats *stats)
+{
+	uint16_t qp_id;
+
+	for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+		struct isal_comp_qp *qp = dev->data->queue_pairs[qp_id];
+
+		stats->enqueued_count += qp->qp_stats.enqueued_count;
+		stats->dequeued_count += qp->qp_stats.dequeued_count;
+
+		stats->enqueue_err_count += qp->qp_stats.enqueue_err_count;
+		stats->dequeue_err_count += qp->qp_stats.dequeue_err_count;
+	}
+}
+
 /** Get device info */
 static void
 isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
@@ -103,6 +121,17 @@ isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
 	}
 }
 
+/** Reset device statistics */
+static void
+isal_comp_pmd_stats_reset(struct rte_compressdev *dev)
+{
+	uint16_t qp_id;
+
+	for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+		struct isal_comp_qp *qp = dev->data->queue_pairs[qp_id];
+		memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
+	}
+}
 
 /** Release queue pair */
 static int
@@ -260,8 +289,8 @@ struct rte_compressdev_ops isal_pmd_ops = {
 		.dev_stop		= isal_comp_pmd_stop,
 		.dev_close		= isal_comp_pmd_close,
 
-		.stats_get		= NULL,
-		.stats_reset		= NULL,
+		.stats_get		= isal_comp_pmd_stats_get,
+		.stats_reset		= isal_comp_pmd_stats_reset,
 
 		.dev_infos_get		= isal_comp_pmd_info_get,
 
Signature · 1 line
2.7.4
author
#16 lore raw

PATCH v3 8/11 compress/isal: add ISA-L compression functionality

In reply to Lee Daly
---
 drivers/compress/isal/isal_compress_pmd.c         | 108 +++++++++++++++++++++-
 drivers/compress/isal/isal_compress_pmd_ops.c     |  25 +++++
 drivers/compress/isal/isal_compress_pmd_private.h |   2 +
 3 files changed, 133 insertions(+), 2 deletions(-)
drivers/compress/isal/isal_compress_pmd.c +106 -2
index 2e19b42..379e0a6 100644
@@ -1,10 +1,12 @@
 /* SPDX-License-Identifier: BSD-3-Clause
  * Copyright(c) 2018 Intel Corporation
  */
+#include <isa-l.h>
 
 #include <rte_bus_vdev.h>
 #include <rte_common.h>
 #include <rte_malloc.h>
+#include <rte_mbuf.h>
 #include <rte_compressdev_pmd.h>
 
 #include "isal_compress_pmd_private.h"
@@ -198,14 +200,116 @@ isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
 	return 0;
 }
 
+/* Stateless Compression Function */
+static int
+process_isal_deflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
+		struct isal_priv_xform *priv_xform)
+{
+	int ret = 0;
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;
+
+	/* Required due to init clearing level_buf */
+	uint8_t *temp_level_buf = qp->stream->level_buf;
+
+	/* Initialize compression stream */
+	isal_deflate_stateless_init(qp->stream);
+
+	qp->stream->level_buf = temp_level_buf;
+
+	qp->stream->flush = NO_FLUSH;
+
+	/* Set op checksum, none by default */
+	if (priv_xform->compress.chksum == RTE_COMP_CHECKSUM_CRC32)
+		qp->stream->gzip_flag = IGZIP_GZIP;
+	else if (priv_xform->compress.chksum == RTE_COMP_CHECKSUM_ADLER32)
+		qp->stream->gzip_flag = IGZIP_ZLIB;
+
+	/* set op level & intermediate level buffer */
+	if (priv_xform->compress.level == RTE_COMP_ISAL_LEVEL_ZERO) {
+		qp->stream->level = ISAL_DEF_MIN_LEVEL;
+		qp->stream->level_buf_size = ISAL_DEF_LVL0_DEFAULT;
+	} else if (priv_xform->compress.level == RTE_COMP_ISAL_LEVEL_ONE) {
+		qp->stream->level = RTE_COMP_ISAL_LEVEL_ONE;
+		qp->stream->level_buf_size = ISAL_DEF_LVL1_DEFAULT;
+	} else if (priv_xform->compress.level == RTE_COMP_ISAL_LEVEL_TWO) {
+		qp->stream->level = RTE_COMP_ISAL_LEVEL_TWO;
+		qp->stream->level_buf_size = ISAL_DEF_LVL2_DEFAULT;
+	} else {
+		qp->stream->level = ISAL_DEF_MAX_LEVEL;
+		qp->stream->level_buf_size = ISAL_DEF_LVL3_DEFAULT;
+	}
+
+	/* Point compression stream structure to input/output buffers */
+	qp->stream->avail_in = op->src.length;
+	qp->stream->next_in = rte_pktmbuf_mtod(op->m_src, uint8_t *);
+	qp->stream->avail_out = op->m_dst->data_len;
+	qp->stream->next_out  = rte_pktmbuf_mtod(op->m_dst, uint8_t *);
+	qp->stream->end_of_stream = 1; /* All input consumed in one go */
+
+	if (unlikely(!qp->stream->next_in || !qp->stream->next_out)) {
+		ISAL_PMD_LOG(ERR, "Invalid source or destination buffers\n");
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		return -1;
+	}
+
+	/* Set op huffman code */
+	if (priv_xform->compress.deflate.huffman == RTE_COMP_HUFFMAN_FIXED)
+		isal_deflate_set_hufftables(qp->stream, NULL,
+				IGZIP_HUFFTABLE_STATIC);
+	else if (priv_xform->compress.deflate.huffman ==
+			RTE_COMP_HUFFMAN_DEFAULT)
+		isal_deflate_set_hufftables(qp->stream, NULL,
+			IGZIP_HUFFTABLE_DEFAULT);
+	/* Dynamically change the huffman code to suit the input data */
+	else if (priv_xform->compress.deflate.huffman ==
+			RTE_COMP_HUFFMAN_DYNAMIC) {
+		struct isal_hufftables     hufftable;
+		struct isal_huff_histogram histogram;
+		memset(&histogram, 0, sizeof(struct isal_huff_histogram));
+
+		isal_update_histogram(qp->stream->next_in, qp->stream->avail_in,
+				&histogram);
+		isal_create_hufftables(&hufftable, &histogram);
+		isal_deflate_set_hufftables(qp->stream, &hufftable,
+				IGZIP_HUFFTABLE_CUSTOM);
+	}
+
+	/* Execute compression operation */
+	ret =  isal_deflate_stateless(qp->stream);
+
+	/* Check that output buffer did not run out of space */
+	if (ret == STATELESS_OVERFLOW) {
+		ISAL_PMD_LOG(ERR, "Output buffer not big enough\n");
+		op->status = RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED;
+		return ret;
+	}
+
+	/* Check that input buffer has been fully consumed */
+	if (qp->stream->avail_in != (uint32_t)0) {
+		ISAL_PMD_LOG(ERR, "Input buffer could not be read entirely\n");
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return -1;
+	}
+
+	if (ret != COMP_OK) {
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return ret;
+	}
+
+	op->consumed = op->src.length - qp->stream->avail_in;
+	op->produced = qp->stream->total_out;
+
+	return ret;
+}
+
 /* Process compression operation */
 static int
-process_op(struct isal_comp_qp *qp __rte_unused,
-		struct rte_comp_op *op __rte_unused,
+process_op(struct isal_comp_qp *qp, struct rte_comp_op *op,
 		struct isal_priv_xform *priv_xform)
 {
 	switch (priv_xform->type) {
 	case RTE_COMP_COMPRESS:
+		process_isal_deflate(op, qp, priv_xform);
 		break;
 	case RTE_COMP_DECOMPRESS:
 		break;
drivers/compress/isal/isal_compress_pmd_ops.c +25 -0
index 0fcb899..0b74733 100644
@@ -1,6 +1,7 @@
 /* SPDX-License-Identifier: BSD-3-Clause
  * Copyright(c) 2018 Intel Corporation
  */
+#include <isa-l.h>
 
 #include <rte_common.h>
 #include <rte_compressdev_pmd.h>
@@ -9,6 +10,17 @@
 #include "isal_compress_pmd_private.h"
 
 static const struct rte_compressdev_capabilities isal_pmd_capabilities[] = {
+	{
+		.algo = RTE_COMP_ALGO_DEFLATE,
+		.comp_feature_flags =	RTE_COMP_FF_ADLER32_CHECKSUM |
+					RTE_COMP_FF_CRC32_CHECKSUM |
+					RTE_COMP_FF_SHAREABLE_PRIV_XFORM,
+		.window_size = {
+			.min = 15,
+			.max = 15,
+			.increment = 0
+		},
+	},
 	RTE_COMP_END_OF_CAPABILITIES_LIST()
 };
 
@@ -142,6 +154,9 @@ struct isal_comp_qp *qp = dev->data->queue_pairs[qp_id];
 	if (qp == NULL)
 		return -EINVAL;
 
+	if (qp->stream != NULL)
+		rte_free(qp->stream);
+
 	if (dev->data->queue_pairs[qp_id] != NULL)
 		rte_free(dev->data->queue_pairs[qp_id]);
 
@@ -209,6 +224,16 @@ isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
 		return (-ENOMEM);
 	}
 
+	/* Initialize memory for compression stream structure */
+	qp->stream = rte_zmalloc_socket("Isa-l compression stream ",
+			sizeof(struct isal_zstream),  RTE_CACHE_LINE_SIZE,
+			socket_id);
+
+	/* Initialize memory for compression level buffer */
+	qp->stream->level_buf = rte_zmalloc_socket("Isa-l compression lev_buf",
+			ISAL_DEF_LVL3_DEFAULT, RTE_CACHE_LINE_SIZE,
+			socket_id);
+
 	qp->id = qp_id;
 	dev->data->queue_pairs[qp_id] = qp;
 
drivers/compress/isal/isal_compress_pmd_private.h +2 -0
index 05679b8..57f0ecc 100644
@@ -28,6 +28,8 @@ struct isal_comp_qp {
 	struct rte_ring *processed_pkts;
 	/* Queue pair statistics */
 	struct rte_compressdev_stats qp_stats;
+	/* Compression stream information*/
+	struct isal_zstream *stream;
 	/* Number of free elements on ring */
 	uint16_t num_free_elements;
 } __rte_cache_aligned;
Signature · 1 line
2.7.4
#17 lore raw

Re: [PATCH v3 08/11] compress/isal: add ISA-L compression functionality

Quoted 14 lines

-----Original Message-----
From: Daly, Lee
Sent: Tuesday, April 17, 2018 2:35 PM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v3 08/11] compress/isal: add ISA-L compression functionality

---
 drivers/compress/isal/isal_compress_pmd.c         | 108

...

Quoted 34 lines
drivers/compress/isal/isal_compress_pmd.c +22 -0
+/* Stateless Compression Function */
+static int
+process_isal_deflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
+		struct isal_priv_xform *priv_xform)
+{
+	int ret = 0;
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;
+
+	/* Required due to init clearing level_buf */
+	uint8_t *temp_level_buf = qp->stream->level_buf;
+
+	/* Initialize compression stream */
+	isal_deflate_stateless_init(qp->stream);
+
+	qp->stream->level_buf = temp_level_buf;
+
+	qp->stream->flush = NO_FLUSH;
+
+	/* Set op checksum, none by default */
+	if (priv_xform->compress.chksum == RTE_COMP_CHECKSUM_CRC32)
+		qp->stream->gzip_flag = IGZIP_GZIP;
+	else if (priv_xform->compress.chksum ==

RTE_COMP_CHECKSUM_ADLER32)

drivers/compress/isal/isal_compress_pmd.c +10 -0
+		qp->stream->gzip_flag = IGZIP_ZLIB;
+
+	/* set op level & intermediate level buffer */
+	if (priv_xform->compress.level == RTE_COMP_ISAL_LEVEL_ZERO) {
+		qp->stream->level = ISAL_DEF_MIN_LEVEL;
+		qp->stream->level_buf_size = ISAL_DEF_LVL0_DEFAULT;
+	} else if (priv_xform->compress.level == RTE_COMP_ISAL_LEVEL_ONE) {
+		qp->stream->level = RTE_COMP_ISAL_LEVEL_ONE;
+		qp->stream->level_buf_size = ISAL_DEF_LVL1_DEFAULT;
+	} else if (priv_xform->compress.level == RTE_COMP_ISAL_LEVEL_TWO)

{

drivers/compress/isal/isal_compress_pmd.c isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform, · line 240

Quoted 6 lines
drivers/compress/isal/isal_compress_pmd.c +6 -0
+		qp->stream->level = RTE_COMP_ISAL_LEVEL_TWO;
+		qp->stream->level_buf_size = ISAL_DEF_LVL2_DEFAULT;
+	} else {
+		qp->stream->level = ISAL_DEF_MAX_LEVEL;
+		qp->stream->level_buf_size = ISAL_DEF_LVL3_DEFAULT;
+	}

Better to store directly "level", "level_buf_size" and "gzip_flag" in priv_xform, and setting them when creating it. That way, you save these branches in data path.

author
#18 lore raw

PATCH v3 9/11 compress/isal: add ISA-L decomp functionality

In reply to Lee Daly
---
 drivers/compress/isal/isal_compress_pmd.c         | 58 +++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c     |  8 ++++
 drivers/compress/isal/isal_compress_pmd_private.h |  2 +
 3 files changed, 68 insertions(+)
drivers/compress/isal/isal_compress_pmd.c +58 -0
index 379e0a6..39674ec 100644
@@ -302,6 +302,63 @@ process_isal_deflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
 	return ret;
 }
 
+/* Stateless Decompression Function */
+static int
+process_isal_inflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
+		struct isal_priv_xform *priv_xform)
+{
+	int ret = 0;
+
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;
+
+	/* Initialize decompression state */
+	isal_inflate_init(qp->state);
+
+	/* Set op checksum, none by default */
+	if (priv_xform->decompress.chksum == RTE_COMP_CHECKSUM_CRC32)
+		qp->state->crc_flag = ISAL_GZIP;
+	else if (priv_xform->decompress.chksum == RTE_COMP_CHECKSUM_ADLER32)
+		qp->state->crc_flag = ISAL_ZLIB;
+
+	/* Point decompression state structure to input/output buffers */
+	qp->state->avail_in = op->src.length;
+	qp->state->next_in = rte_pktmbuf_mtod(op->m_src, uint8_t *);
+	qp->state->avail_out = op->m_dst->data_len;
+	qp->state->next_out  = rte_pktmbuf_mtod(op->m_dst, uint8_t *);
+
+	if (unlikely(!qp->state->next_in || !qp->state->next_out)) {
+		ISAL_PMD_LOG(ERR, "Invalid source or destination buffers\n");
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		return -1;
+	}
+
+	/* Execute decompression operation */
+	ret = isal_inflate_stateless(qp->state);
+
+	if (ret == ISAL_OUT_OVERFLOW) {
+		ISAL_PMD_LOG(ERR, "Output buffer not big enough\n");
+		op->status = RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED;
+		return ret;
+	}
+
+	/* Check that input buffer has been fully consumed */
+	if (qp->state->avail_in != (uint32_t)0) {
+		ISAL_PMD_LOG(ERR, "Input buffer could not be read entirely\n");
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return -1;
+	}
+
+	if (ret != ISAL_DECOMP_OK) {
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return ret;
+	}
+
+	op->consumed = op->src.length - qp->state->avail_in;
+	op->produced = qp->state->total_out;
+
+return ret;
+}
+
 /* Process compression operation */
 static int
 process_op(struct isal_comp_qp *qp, struct rte_comp_op *op,
@@ -312,6 +369,7 @@ process_op(struct isal_comp_qp *qp, struct rte_comp_op *op,
 		process_isal_deflate(op, qp, priv_xform);
 		break;
 	case RTE_COMP_DECOMPRESS:
+		process_isal_inflate(op, qp, priv_xform);
 		break;
 	default:
 		ISAL_PMD_LOG(ERR, "Operation Not Supported\n");
drivers/compress/isal/isal_compress_pmd_ops.c +8 -0
index 0b74733..943b342 100644
@@ -157,6 +157,9 @@ struct isal_comp_qp *qp = dev->data->queue_pairs[qp_id];
 	if (qp->stream != NULL)
 		rte_free(qp->stream);
 
+	if (qp->state != NULL)
+		rte_free(qp->state);
+
 	if (dev->data->queue_pairs[qp_id] != NULL)
 		rte_free(dev->data->queue_pairs[qp_id]);
 
@@ -234,6 +237,11 @@ isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
 			ISAL_DEF_LVL3_DEFAULT, RTE_CACHE_LINE_SIZE,
 			socket_id);
 
+	/* Initialize memory for decompression state structure */
+	qp->state = rte_zmalloc_socket("Isa-l decompression state",
+			sizeof(struct inflate_state), RTE_CACHE_LINE_SIZE,
+			socket_id);
+
 	qp->id = qp_id;
 	dev->data->queue_pairs[qp_id] = qp;
 
drivers/compress/isal/isal_compress_pmd_private.h +2 -0
index 57f0ecc..4943aad 100644
@@ -30,6 +30,8 @@ struct isal_comp_qp {
 	struct rte_compressdev_stats qp_stats;
 	/* Compression stream information*/
 	struct isal_zstream *stream;
+	/* Decompression state information*/
+	struct inflate_state *state;
 	/* Number of free elements on ring */
 	uint16_t num_free_elements;
 } __rte_cache_aligned;
Signature · 1 line
2.7.4
#19 lore raw

Re: [PATCH v3 09/11] compress/isal: add ISA-L decomp functionality

Quoted 37 lines

-----Original Message-----
From: Daly, Lee
Sent: Tuesday, April 17, 2018 2:36 PM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v3 09/11] compress/isal: add ISA-L decomp functionality

---
 drivers/compress/isal/isal_compress_pmd.c         | 58
+++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c     |  8 ++++
 drivers/compress/isal/isal_compress_pmd_private.h |  2 +
 3 files changed, 68 insertions(+)
drivers/compress/isal/isal_compress_pmd.c +8 -0
index 379e0a6..39674ec 100644
@@ -302,6 +302,63 @@ process_isal_deflate(struct rte_comp_op *op, struct
 isal_comp_qp *qp,
 	return ret;
 }
 
+/* Stateless Decompression Function */
+static int
+process_isal_inflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
+		struct isal_priv_xform *priv_xform)
+{
+	int ret = 0;
+
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;

drivers/compress/isal/isal_compress_pmd.c process_isal_deflate(struct rte_comp_op *op, struct · line 323

Quoted 10 lines
drivers/compress/isal/isal_compress_pmd.c +9 -0
+
+	/* Initialize decompression state */
+	isal_inflate_init(qp->state);
+
+	/* Set op checksum, none by default */
+	if (priv_xform->decompress.chksum == RTE_COMP_CHECKSUM_CRC32)
+		qp->state->crc_flag = ISAL_GZIP;
+	else if (priv_xform->decompress.chksum ==
 RTE_COMP_CHECKSUM_ADLER32)
+		qp->state->crc_flag = ISAL_ZLIB;

Same comment as in patch 8, you can store this information in priv_xform.

author
#20 lore raw

PATCH v3 10/11 compress/isal: add generic compression driver docs

In reply to Lee Daly
---
 MAINTAINERS                                        |  6 ++
 doc/guides/compressdevs/features/default.ini       | 42 +++++++++++
 doc/guides/compressdevs/index.rst                  | 12 ++++
 doc/guides/compressdevs/overview.rst               | 12 ++++
 doc/guides/compressdevs/overview_feature_table.txt | 81 ++++++++++++++++++++++
 doc/guides/index.rst                               |  1 +
 6 files changed, 154 insertions(+)
 create mode 100644 doc/guides/compressdevs/features/default.ini
 create mode 100644 doc/guides/compressdevs/index.rst
 create mode 100644 doc/guides/compressdevs/overview.rst
 create mode 100644 doc/guides/compressdevs/overview_feature_table.txt
MAINTAINERS +6 -0
index d2dd61c..37b9b1d 100644
@@ -766,6 +766,12 @@ F: doc/guides/cryptodevs/zuc.rst
 F: doc/guides/cryptodevs/features/zuc.ini
 
 
+Compression Drivers
+-------------------
+M: Pablo de Lara <[email protected]>
+T: git://dpdk.org/next/dpdk-next-crypto
+
+
 Eventdev Drivers
 ----------------
 M: Jerin Jacob <[email protected]>
doc/guides/compressdevs/features/default.ini new file +42 -0

42 lines not on this page · raw message

doc/guides/compressdevs/index.rst new file +12 -0
new file mode 100644
index 0000000..9271cee
@@ -0,0 +1,12 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Compression Device Drivers
+==========================
+
+
+.. toctree::
+    :maxdepth: 2
+    :numbered:
+
+    overview
doc/guides/compressdevs/overview.rst new file +12 -0
new file mode 100644
index 0000000..ca37de1
@@ -0,0 +1,12 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Compression Device Supported Functionality Matrices
+===================================================
+
+Supported Feature Flags
+-----------------------
+
+.. _table_compression_pmd_features:
+
+.. include:: overview_feature_table.txt
doc/guides/compressdevs/overview_feature_table.txt new file +81 -0

81 lines not on this page · raw message

doc/guides/index.rst +1 -0
index d60529d..18fe0ec 100644
@@ -17,6 +17,7 @@ DPDK documentation
    nics/index
    bbdevs/index
    cryptodevs/index
+   compressdevs/index
    eventdevs/index
    mempool/index
    platform/index
Signature · 1 line
2.7.4
#21 lore raw

Re: [PATCH v3 10/11] compress/isal: add generic compression driver docs

Found two typos

Quoted 8 lines
---
 MAINTAINERS                                        |  6 ++
 doc/guides/compressdevs/features/default.ini       | 42 +++++++++++
 doc/guides/compressdevs/index.rst                  | 12 ++++
 doc/guides/compressdevs/overview.rst               | 12 ++++
 doc/guides/compressdevs/overview_feature_table.txt | 81

++++++++++++++++++++++

<...>

Quoted 1 line

+; Supported algorithims of the 'ISA-L' compression driver.

algorithims / algorithms

Quoted 2 lines

+;
+[ALGORITHIM]

ALGORITHIM / ALGORITHM

doc/guides/compressdevs/features/default.ini line 24

Quoted 3 lines
doc/guides/compressdevs/features/default.ini new file +3 -0
+Deflate =
+LZS     =
+;

<...>

Marko K

#22 lore raw

Re: [PATCH v3 10/11] compress/isal: add generic compression driver docs

In reply to Lee Daly
Quoted 12 lines

-----Original Message-----
From: Daly, Lee
Sent: Tuesday, April 17, 2018 2:36 PM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v3 10/11] compress/isal: add generic compression driver docs

...

Quoted 15 lines

b/doc/guides/compressdevs/features/default.ini
new file mode 100644
index 0000000..b7fe1e1

doc/guides/compressdevs/features/default.ini new file +9 -0
@@ -0,0 +1,42 @@
+;
+; Features of a default compression driver.
+;
+; This file defines the features that are valid for inclusion in ; the
+other driver files and also the order that they appear in ; the
+features table in the documentation.
+;
+[FEATURES]
+HW Accelerated =

doc/guides/compressdevs/features/default.ini line 19

Quoted 10 lines
doc/guides/compressdevs/features/default.ini new file +10 -0
+CPU SSE        =
+CPU AVX        =
+CPU AVX2       =
+CPU AVX512     =
+CPU NEON       =
+Stateful       =
+By-Pass        =
+Chained mbufs  =
+;
+; Supported algorithims of the 'ISA-L' compression driver.

This file is generic for all PMDs, so remove the ISAL reference (same applicable below).

Quoted 6 lines
doc/guides/compressdevs/features/default.ini new file +6 -0
+;
+[ALGORITHIM]
+Deflate =
+LZS     =
+;
+; Supported checksums of the 'ISA-L' compression driver.

doc/guides/compressdevs/features/default.ini line 35

Quoted 10 lines
doc/guides/compressdevs/features/default.ini new file +10 -0
+;
+[CHECKSUMS]
+Adler32       =
+Crc32         =
+Adler32&Crc32 =
+;
+; Supported huffman codes of the 'ISA-L' compression driver.
+;
+[HUFFMAN CODES]
+Default      =

No need to add default here.

doc/guides/compressdevs/features/default.ini line 38

Quoted 3 lines
doc/guides/compressdevs/features/default.ini new file +3 -0
+Fixed        =
+Dynamic      =
+Semi-Dynamic =

Semi-dynamic Huffman coding is not in the API, so remove it.

doc/guides/compressdevs/features/default.ini line 42

Quoted 4 lines
doc/guides/compressdevs/features/default.ini new file +4 -0
+;
+; Supported others of the 'ISA-L' compression driver.
+;
+[OTHERS]

Remove this section if it is going to be empty.
...

doc/guides/compressdevs/overview_feature_table.txt

Quoted 20 lines

+++ b/doc/guides/compressdevs/overview.rst

doc/guides/compressdevs/features/default.ini new file +12 -0
@@ -0,0 +1,12 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Compression Device Supported Functionality Matrices
+===================================================
+
+Supported Feature Flags
+-----------------------
+
+.. _table_compression_pmd_features:
+
+.. include:: overview_feature_table.txt
doc/guides/compressdevs/overview_feature_table.txt new file +0 -0
new file mode 100644
index 0000000..b775124

This document gets generated with Sphinx. It parses the .ini files and generates it.
Look at how is done for cryptodev, in the setup function in doc/guides/conf.py,.
You should also extend the table for algorithm, checksum and Huffman encoding.
Then, maybe it is just better to have all these sections in the FEATURES section, to build a single table. Lastly, you need to add this file in the .gitignore file, to avoid adding it into git.

Quoted 3 lines
doc/guides/compressdevs/overview_feature_table.txt new file +2 -0
@@ -0,0 +1,81 @@
+
+.. raw:: html
#23 lore raw

Re: [PATCH v3 10/11] compress/isal: add generic compression driver docs

In reply to Lee Daly
Quoted 9 lines

-----Original Message-----
From: Daly, Lee
Sent: Tuesday, April 17, 2018 2:36 PM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v3 10/11] compress/isal: add generic compression driver docs

One last comment. Change title to: "doc: add compression feature guide",
since this is not adding any information about ISA-L, but it is common for all compression PMDs.

author
#24 lore raw

Re: [PATCH v3 10/11] compress/isal: add generic compression driver docs

Thanks for the feedback on the patch set Pablo,
V4 in progress.

Quoted 24 lines

-----Original Message-----
From: De Lara Guarch, Pablo
Sent: Tuesday, April 24, 2018 12:07 PM
To: Daly, Lee <[email protected]>; [email protected]
Cc: Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>
Subject: RE: [PATCH v3 10/11] compress/isal: add generic compression driver docs

Quoted 11 lines

-----Original Message-----
From: Daly, Lee
Sent: Tuesday, April 17, 2018 2:36 PM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v3 10/11] compress/isal: add generic compression driver docs

One last comment. Change title to: "doc: add compression feature guide", since this is not adding any information about ISA-L, but it is common for all compression PMDs.

author
#25 lore raw

PATCH v3 11/11 compress/isal: add ISA-L compression PMD docs

In reply to Lee Daly
---
 MAINTAINERS                               |  5 ++
 devtools/test-build.sh                    |  4 ++
 doc/guides/compressdevs/features/isal.ini | 40 +++++++++++++
 doc/guides/compressdevs/index.rst         |  1 +
 doc/guides/compressdevs/isal.rst          | 94 +++++++++++++++++++++++++++++++
 5 files changed, 144 insertions(+)
 create mode 100644 doc/guides/compressdevs/features/isal.ini
 create mode 100644 doc/guides/compressdevs/isal.rst
MAINTAINERS +5 -0
index 37b9b1d..baccae7 100644
@@ -771,6 +771,11 @@ Compression Drivers
 M: Pablo de Lara <[email protected]>
 T: git://dpdk.org/next/dpdk-next-crypto
 
+ISA-L PMD
+M: Lee Daly <[email protected]>
+F: drivers/compress/isal
+F: doc/guides/compressdevs/isal.rst
+F: doc/guides/compressdevs/features/isal.ini
 
 Eventdev Drivers
 ----------------
devtools/test-build.sh +4 -0
index 3362edc..66f3ece 100755
@@ -45,6 +45,7 @@ default_path=$PATH
 # - DPDK_DEP_SSL (y/[n])
 # - DPDK_DEP_SZE (y/[n])
 # - DPDK_DEP_ZLIB (y/[n])
+# - DPDK_DEP_ISAL (y/[n])
 # - DPDK_MAKE_JOBS (int)
 # - DPDK_NOTIFY (notify-send)
 # - FLEXRAN_SDK
@@ -129,6 +130,7 @@ reset_env ()
 	unset DPDK_DEP_SSL
 	unset DPDK_DEP_SZE
 	unset DPDK_DEP_ZLIB
+	unset DPDK_DEP_ISAL
 	unset AESNI_MULTI_BUFFER_LIB_PATH
 	unset ARMV8_CRYPTO_LIB_PATH
 	unset FLEXRAN_SDK
@@ -178,6 +180,8 @@ config () # <directory> <target> <options>
 		test "$DPDK_DEP_ZLIB" != y || \
 		sed -ri          's,(BNX2X_PMD=)n,\1y,' $1/.config
 		sed -ri            's,(NFP_PMD=)n,\1y,' $1/.config
+		test "$DPDK_DEP_ISAL" != y || \
+		sed -ri          's,(ISAL_PMD=)n,\1y,' $1/.config
 		test "$DPDK_DEP_PCAP" != y || \
 		sed -ri               's,(PCAP=)n,\1y,' $1/.config
 		test -z "$ARMV8_CRYPTO_LIB_PATH" || \
doc/guides/compressdevs/features/isal.ini new file +40 -0
new file mode 100644
index 0000000..82caa17
@@ -0,0 +1,40 @@
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; Supported features of 'ISA-L' compression driver.
+;
+[FEATURES]
+HW Accelerated = N
+CPU SSE        = Y
+CPU AVX        = Y
+CPU AVX2       = Y
+CPU AVX512     = Y
+CPU NEON       = N
+Stateful       = N
+By-Pass        = N
+Chained mbufs  = N
+;
+; Supported algorithims of the 'ISA-L' compression driver.
+;
+[ALGORITHIM]
+Deflate  = Y
+LZS      = N
+;
+; Supported checksums of the 'ISA-L' compression driver.
+;
+[CHECKSUMS]
+Adler32       = Y
+Crc32         = Y
+Adler32&Crc32 = N
+;
+; Supported huffman codes of the 'ISA-L' compression driver.
+;
+[HUFFMAN CODES]
+Default      = Y
+Fixed        = Y
+Dynamic      = N
+Semi-Dynamic = Y
+;
+; Supported others of the 'ISA-L' compression driver.
+;
+[OTHERS]
doc/guides/compressdevs/index.rst +1 -0
index 9271cee..bc59ce8 100644
@@ -10,3 +10,4 @@ Compression Device Drivers
     :numbered:
 
     overview
+    isal
doc/guides/compressdevs/isal.rst new file +94 -0

94 lines not on this page · raw message

Signature · 1 line
2.7.4
#26 lore raw

Re: [PATCH v3 11/11] compress/isal: add ISA-L compression PMD docs

Few typos below

Quoted 8 lines
---
 MAINTAINERS                               |  5 ++
 devtools/test-build.sh                    |  4 ++
 doc/guides/compressdevs/features/isal.ini | 40 +++++++++++++
 doc/guides/compressdevs/index.rst         |  1 +
 doc/guides/compressdevs/isal.rst          | 94

+++++++++++++++++++++++++++++++

<...>

Quoted 4 lines
doc/guides/compressdevs/features/isal.ini new file +4 -0
+; Supported features of 'ISA-L' compression driver.
+;
+[FEATURES]
+HW Accelerated = N

doc/guides/compressdevs/features/isal.ini line 17

Quoted 10 lines
doc/guides/compressdevs/features/isal.ini new file +10 -0
+CPU SSE        = Y
+CPU AVX        = Y
+CPU AVX2       = Y
+CPU AVX512     = Y
+CPU NEON       = N
+Stateful       = N
+By-Pass        = N
+Chained mbufs  = N
+;
+; Supported algorithims of the 'ISA-L' compression driver.

algorithims / algorithms

Quoted 2 lines

+;
+[ALGORITHIM]

ALGORITHIM / ALGORITHM

<...>

Review

Quoted 5 lines

+The ISA-L PMD (**librte_pmd_isal_comp**) provides poll mode compression
+& decompression driver support for utilizing Intel ISA-L library, which

(quoted diff) +2 -0
+implements the deflate algorithim for both compression and
+decompression

algorithim / algorithm

<...>

doc/guides/compressdevs/isal.rst line 61

Quoted 6 lines
doc/guides/compressdevs/isal.rst new file +6 -0
+
+are usualy sufficient::
+
+	./autogen.sh
+	./configure
+

usualy / usually

<...>

#27 lore raw

Re: [PATCH v3 11/11] compress/isal: add ISA-L compression PMD docs

In reply to Lee Daly
Quoted 9 lines
---
 MAINTAINERS                               |  5 ++
 devtools/test-build.sh                    |  4 ++
 doc/guides/compressdevs/features/isal.ini | 40 +++++++++++++
 doc/guides/compressdevs/index.rst         |  1 +
 doc/guides/compressdevs/isal.rst          | 94

+++++++++++++++++++++++++++++++

<...>

doc/guides/compressdevs/isal.rst line 42

Quoted 9 lines
doc/guides/compressdevs/isal.rst new file +9 -0
+Limitations
+-----------
+
+* Chained mbufs are not supported.
+
+* Compressdev level 0, no compression, is not supported. ISA-L level 0
+used for
+
+fixed huffman codes.

For the bit above have the two lines together as it messes up the format and look of the sentance

* Compressdev level 0, no compression, is not supported. ISA-L level 0 used for fixed huffman codes.

doc/guides/compressdevs/isal.rst line 53

Quoted 11 lines
doc/guides/compressdevs/isal.rst new file +11 -0
+* Out of order operations are not supported
+
+Installation
+------------
+
+* To build DPDK with Intel's ISA-L library, the user is required to
+download
+
+the library from
+
+`<https://github.com/01org/isa-l>`_.

Same here,

+* To build DPDK with Intel's ISA-L library, the user is required to
download the library from <https://github.com/01org/isa-l>_.

doc/guides/compressdevs/isal.rst line 61

Quoted 8 lines
doc/guides/compressdevs/isal.rst new file +8 -0
+* Once downloaded, the user needs to build the library, the ISA-L
+autotools
+
+are usualy sufficient::
+
+	./autogen.sh
+	./configure
+

Same above, also use 4 spaces to indent the code block not tabs,

* Once downloaded, the user needs to build the library, the ISA-L autotools are usualy sufficient::

./autogen.sh
./configure

doc/guides/compressdevs/isal.rst line 65

Quoted 5 lines

+make can be used to install the library on their system, before building DPDK::

doc/guides/compressdevs/isal.rst new file +3 -0
+
+	make
+	sudo make install

Spaces instead of tabs for the commands above

doc/guides/compressdevs/isal.rst line 75

Quoted 11 lines
doc/guides/compressdevs/isal.rst new file +11 -0
+* To build with meson, the "libisal.pc" file, must be copied into
+"pkgconfig",
+
+e.g. /usr/lib/pkgconfig or /usr/lib64/pkgconfig depending on your
+system,
+
+for meson to find the ISA-L library. "libisal.pc" is located in library sources,::
+
+       cp isal/libisal.pc /usr/lib/pkgconfig/
+
+instructions on how to download below.

Same changes as above also you can get rid of the comma after sources

* To build with meson, the "libisal.pc" file, must be copied into "pkgconfig", e.g. /usr/lib/pkgconfig or /usr/lib64/pkgconfig depending on your system, for meson to find the ISA-L library. "libisal.pc" is located in library sources ::

cp isal/libisal.pc /usr/lib/pkgconfig/

instructions on how to download below.

<...>

Quoted 20 lines
doc/guides/compressdevs/isal.rst new file +12 -0
+Initialization
+--------------
+
+In order to enable this virtual compression PMD, user must:
+
+* Set CONFIG_RTE_LIBRTE_PMD_ISAL=y in config/common_base.
+
+To use the PMD in an application, user must:
+
+* Call rte_vdev_init("compress_isal") within the application.
+
+* Use --vdev="compress_isal" in the EAL options, which will call

rte_vdev_init() internally.

(quoted diff) +2 -0
+
+The following parameters (all optional) can be provided in the previous two

calls:

(quoted diff) +4 -0
+
+* socket_id: Specify the socket where the memory for the device is
+going to be allocated
+  (by default, socket_id will be the socket where the core that is creating the
Quoted 1 line

PMD is running on).

In this section commands like CONFIG_RTE_LIBRTE_PMD_ISAL=y and --vdev="compress_isal"
Put them in `` `` quotes to highlight it's a command as it's a bit hard to see and also looks better.

Marko K

author
#28 lore raw

Re: [PATCH v3 11/11] compress/isal: add ISA-L compression PMD docs

//snip

Quoted 15 lines

Subject: RE: [dpdk-dev] [PATCH v3 11/11] compress/isal: add ISA-L compression PMD docs

Quoted 8 lines
---
 MAINTAINERS                               |  5 ++
 devtools/test-build.sh                    |  4 ++
 doc/guides/compressdevs/features/isal.ini | 40 +++++++++++++
 doc/guides/compressdevs/index.rst         |  1 +
 doc/guides/compressdevs/isal.rst          | 94

+++++++++++++++++++++++++++++++

<...>

Quoted 10 lines

doc/guides/compressdevs/isal.rst line 42

Quoted 9 lines
doc/guides/compressdevs/isal.rst new file +9 -0
+Limitations
+-----------
+
+* Chained mbufs are not supported.
+
+* Compressdev level 0, no compression, is not supported. ISA-L level
+0 used for
+
+fixed huffman codes.

[Lee] Thanks for the feedback on patchset, V4 to be sent in coming days.

Quoted 10 lines

For the bit above have the two lines together as it messes up the format and look of the sentance

* Compressdev level 0, no compression, is not supported. ISA-L level 0 used for fixed huffman codes.

doc/guides/compressdevs/isal.rst line 47

Quoted 4 lines
doc/guides/compressdevs/isal.rst new file +4 -0
+* Out of order operations are not supported
+
+Installation
+------------

//snip

#29 lore raw

Re: [PATCH v3 11/11] compress/isal: add ISA-L compression PMD docs

In reply to Lee Daly
Quoted 10 lines

-----Original Message-----
From: Daly, Lee
Sent: Tuesday, April 17, 2018 2:36 PM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v3 11/11] compress/isal: add ISA-L compression PMD docs

Change title to " doc: add compress isa-l PMD guide

Quoted 18 lines
---
 MAINTAINERS                               |  5 ++
 devtools/test-build.sh                    |  4 ++
 doc/guides/compressdevs/features/isal.ini | 40 +++++++++++++
 doc/guides/compressdevs/index.rst         |  1 +
 doc/guides/compressdevs/isal.rst          | 94
+++++++++++++++++++++++++++++++
 5 files changed, 144 insertions(+)
 create mode 100644 doc/guides/compressdevs/features/isal.ini
 create mode 100644 doc/guides/compressdevs/isal.rst
MAINTAINERS +0 -0
index 37b9b1d..baccae7 100644

MAINTAINERS Compression Drivers · line 772

Quoted 2 lines
MAINTAINERS +0 -0
@@ -771,6 +771,11 @@ Compression Drivers
 M: Pablo de Lara <[email protected]>
 T: git://dpdk.org/next/dpdk-next-crypto

Add F: doc/guides/compressdev/features/default.ini, but better in patch 10.

MAINTAINERS Compression Drivers · line 776

Quoted 4 lines
MAINTAINERS +3 -0
+ISA-L PMD
+M: Lee Daly <[email protected]>
+F: drivers/compress/isal

You can add these three lines above in the first patch and then add the files below in this patch.

Quoted 9 lines
MAINTAINERS +2 -0
+F: doc/guides/compressdevs/isal.rst
+F: doc/guides/compressdevs/features/isal.ini
 
 Eventdev Drivers
 ----------------
devtools/test-build.sh index renamed from devtools/test-build.sh +0 -0

3362edc..66f3ece 100755

devtools/test-build.sh +0 -0

...

devtools/test-build.sh config () # <directory> <target> <options>

Quoted 5 lines
devtools/test-build.sh +2 -0
+		test "$DPDK_DEP_ISAL" != y || \
+		sed -ri          's,(ISAL_PMD=)n,\1y,' $1/.config
 		test "$DPDK_DEP_PCAP" != y || \
 		sed -ri               's,(PCAP=)n,\1y,' $1/.config
 		test -z "$ARMV8_CRYPTO_LIB_PATH" || \ diff --git

Add the changes for test-build.sh in first patch.

...

Quoted 32 lines
doc/guides/compressdevs/index.rst +1 -0
index 9271cee..bc59ce8 100644
@@ -10,3 +10,4 @@ Compression Device Drivers
     :numbered:
 
     overview
+    isal
doc/guides/compressdevs/isal.rst new file +15 -0
new file mode 100644
index 0000000..d76f7ae
@@ -0,0 +1,94 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+ISA-L Compression Poll Mode Driver
+==================================
+
+The ISA-L PMD (**librte_pmd_isal_comp**) provides poll mode compression
+& decompression driver support for utilizing Intel ISA-L library, which
+implements the deflate algorithim for both compression and
+decompression
+
+Features
+--------
+
+ISA-L PMD has support for:

doc/guides/compressdevs/isal.rst line 25

Quoted 10 lines
doc/guides/compressdevs/isal.rst new file +10 -0
+
+Compression/Decompression algorithm:
+
+* DEFLATE
+
+Huffman code type:
+
+* DEFAULT
+* FIXED
+* DYNAMIC

Remove DEFAULT, which is supported by all PMDs.

Quoted 11 lines
doc/guides/compressdevs/isal.rst new file +11 -0
+
+Checksum support:
+
+* Adler32
+* CRC32
+
+Window size support:
+
+* 32K
+
+Limitations

doc/guides/compressdevs/isal.rst line 44

Quoted 10 lines
doc/guides/compressdevs/isal.rst new file +10 -0
+-----------
+
+* Chained mbufs are not supported.
+
+* Compressdev level 0, no compression, is not supported. ISA-L level 0
+used for
+
+fixed huffman codes.
+
+* Out of order operations are not supported

I don't think this is a limitation. What kind of "out of order" do you mean here? Some PMDs can process operations out of order, but that is expected from the API. If a PMD processes them in order, it is also OK.

...

Quoted 2 lines
doc/guides/compressdevs/isal.rst new file +1 -0
+The following parameters (all optional) can be provided in the previous two

calls:

There is only one parameter, so change to "the following parameter".

Quoted 7 lines
doc/guides/compressdevs/isal.rst new file +4 -0
+
+* socket_id: Specify the socket where the memory for the device is
+going to be allocated
+  (by default, socket_id will be the socket where the core that is creating the

PMD is running on).
--
2.7.4

author
#30 lore raw

v4 cover add ISA-L compression PMD

In reply to Lee Daly

This patchset contains the first compression PMD written under the DPDK compression API, compressdev. The ISA-L compression driver utilizes Intel's ISA-L compression library. It therefore has dependencies on both compressdev and the ISA-L library, v2.22.0.

V2:
- Changes to keep in compliance with compressdev API,
- Enable meson build system,
- General rework & fixes,
- Documentation.

V3:
- Changes to keep in compliance with compressdev API,
- General rework,
- Split into patchset.

V4:
- Changes to keep in compliance with the compressdev API,
- Removed unnecessary branching in compression function, process_isal_deflate,
- Some minor documentation fixes,
- Minor reworks.

Lee Daly (10):
compress/isal: add skeleton ISA-L compression PMD compress/isal: add pmd device init and de-init compress/isal: add basic pmd ops compress/isal: add private xform related ops compress/isal: add queue pair related ops compress/isal: support enqueue/dequeue api compress/isal: add stats related ops compress/isal: add ISA-L compression functionality compress/isal: add ISA-L decomp functionality
doc: add compression driver and ISA-L PMD docs

 .gitignore                                         |   1 +
 MAINTAINERS                                        |  12 +
 config/common_base                                 |   5 +
 devtools/test-build.sh                             |   4 +
 doc/guides/compressdevs/features/default.ini       |  24 +
 doc/guides/compressdevs/features/isal.ini          |  22 +
 doc/guides/compressdevs/index.rst                  |  13 +
 doc/guides/compressdevs/isal.rst                   |  77 ++++
 doc/guides/compressdevs/overview.rst               |  12 +
 doc/guides/compressdevs/overview_feature_table.txt |  84 ++++
 doc/guides/conf.py                                 |   5 +
 doc/guides/index.rst                               |   1 +
 doc/guides/rel_notes/release_18_05.rst             |   4 +
 drivers/Makefile                                   |   2 +
 drivers/compress/Makefile                          |   8 +
 drivers/compress/isal/Makefile                     |  31 ++
 drivers/compress/isal/isal_compress_pmd.c          | 500 +++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c      | 345 ++++++++++++++
 drivers/compress/isal/isal_compress_pmd_private.h  |  57 +++
 drivers/compress/isal/meson.build                  |  14 +
 drivers/compress/isal/rte_pmd_isal_version.map     |   3 +
 drivers/compress/meson.build                       |   8 +
 drivers/meson.build                                |   1 +
 mk/rte.app.mk                                      |   5 +
 24 files changed, 1238 insertions(+)
 create mode 100644 doc/guides/compressdevs/features/default.ini
 create mode 100644 doc/guides/compressdevs/features/isal.ini
 create mode 100644 doc/guides/compressdevs/index.rst
 create mode 100644 doc/guides/compressdevs/isal.rst
 create mode 100644 doc/guides/compressdevs/overview.rst
 create mode 100644 doc/guides/compressdevs/overview_feature_table.txt
 create mode 100644 drivers/compress/Makefile
 create mode 100644 drivers/compress/isal/Makefile
 create mode 100644 drivers/compress/isal/isal_compress_pmd.c
 create mode 100644 drivers/compress/isal/isal_compress_pmd_ops.c
 create mode 100644 drivers/compress/isal/isal_compress_pmd_private.h
 create mode 100644 drivers/compress/isal/meson.build
 create mode 100644 drivers/compress/isal/rte_pmd_isal_version.map
 create mode 100644 drivers/compress/meson.build
Signature · 1 line
2.7.4
author
#31 lore raw

PATCH v4 1/10 compress/isal: add skeleton ISA-L compression PMD

Adding basic skeleton of the ISA-L compression driver. No compression functionality, but lays the foundation for operations in the rest of the patchset. The ISA-L compression driver utilizes Intel's ISA-L compression library and compressdev API. It therefore has dependencies on both compressdev and the ISA-L library, v2.22.0.

---
 config/common_base                             |  5 +++++
 doc/guides/rel_notes/release_18_05.rst         |  3 +++
 drivers/Makefile                               |  2 ++
 drivers/compress/Makefile                      |  8 +++++++
 drivers/compress/isal/Makefile                 | 30 ++++++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd.c      | 29 +++++++++++++++++++++++++
 drivers/compress/isal/meson.build              | 14 ++++++++++++
 drivers/compress/isal/rte_pmd_isal_version.map |  3 +++
 drivers/compress/meson.build                   |  8 +++++++
 drivers/meson.build                            |  1 +
 mk/rte.app.mk                                  |  5 +++++
 11 files changed, 108 insertions(+)
 create mode 100644 drivers/compress/Makefile
 create mode 100644 drivers/compress/isal/Makefile
 create mode 100644 drivers/compress/isal/isal_compress_pmd.c
 create mode 100644 drivers/compress/isal/meson.build
 create mode 100644 drivers/compress/isal/rte_pmd_isal_version.map
 create mode 100644 drivers/compress/meson.build
config/common_base +5 -0
index dd836e8..a982570 100644
@@ -562,6 +562,11 @@ CONFIG_RTE_COMPRESS_MAX_DEVS=64
 CONFIG_RTE_COMPRESSDEV_TEST=y
 
 #
+# Compile PMD for ISA-L compression device
+#
+CONFIG_RTE_LIBRTE_PMD_ISAL=n
+
+#
 # Compile generic event device library
 #
 CONFIG_RTE_LIBRTE_EVENTDEV=y
doc/guides/rel_notes/release_18_05.rst +3 -0
index 5193a62..ce70d1f 100644
@@ -75,7 +75,10 @@ New Features
   The compressdev library provides an API for offload of compression and
   decompression operations to hardware or software accelerator devices.
 
+* **Added a new compression poll mode driver using Intels ISA-L.**
 
+   Added the new ``ISA-L`` compression driver, for compression and decompression
+   operations in software.
 
 
 API Changes
drivers/Makefile +2 -0
index 3d9f86b..c88638c 100644
@@ -13,6 +13,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_BBDEV) += baseband
 DEPDIRS-baseband := common bus mempool
 DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
 DEPDIRS-crypto := common bus mempool
+DIRS-$(CONFIG_RTE_LIBRTE_COMPRESSDEV) += compress
+DEPDIRS-compress := bus mempool
 DIRS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += event
 DEPDIRS-event := common bus mempool net
 DIRS-$(CONFIG_RTE_LIBRTE_RAWDEV) += raw
drivers/compress/Makefile new file +8 -0
new file mode 100644
index 0000000..592497f
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal
+
+include $(RTE_SDK)/mk/rte.subdir.mk
drivers/compress/isal/Makefile new file +30 -0
new file mode 100644
index 0000000..9b1d866
@@ -0,0 +1,30 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_pmd_isal_comp.a
+
+# build flags
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+# external library dependencies
+LDLIBS += -lisal
+LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring
+LDLIBS += -lrte_compressdev
+LDLIBS += -lrte_bus_vdev
+
+# library version
+LIBABIVER := 1
+
+# versioning export map
+EXPORT_MAP := rte_pmd_isal_version.map
+
+# library source files
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal_compress_pmd.c
+
+# export include files
+include $(RTE_SDK)/mk/rte.lib.mk
drivers/compress/isal/isal_compress_pmd.c new file +29 -0
new file mode 100644
index 0000000..d7137fd
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include <rte_bus_vdev.h>
+#include <rte_compressdev_pmd.h>
+
+/** Remove compression device */
+static int
+compdev_isal_remove_dev(struct rte_vdev_device *vdev __rte_unused)
+{
+	return 0;
+}
+
+/** Initialise ISA-L compression device */
+static int
+compdev_isal_probe(struct rte_vdev_device *dev __rte_unused)
+{
+	return 0;
+}
+
+static struct rte_vdev_driver compdev_isal_pmd_drv = {
+	.probe = compdev_isal_probe,
+	.remove = compdev_isal_remove_dev,
+};
+
+RTE_PMD_REGISTER_VDEV(COMPDEV_NAME_ISAL_PMD, compdev_isal_pmd_drv);
+RTE_PMD_REGISTER_PARAM_STRING(COMPDEV_NAME_ISAL_PMD,
+	"socket_id=<int>");
drivers/compress/isal/meson.build new file +14 -0
new file mode 100644
index 0000000..4447e20
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2018 Intel Corporation
+
+dep = dependency('libisal', required: false)
+if not dep.found()
+       build =false
+endif
+
+deps += 'bus_vdev'
+sources = files('isal_compress_pmd.c')
+ext_deps += dep
+pkgconfig_extra_libs += '-lisal'
+
+allow_experimental_apis = true
drivers/compress/isal/rte_pmd_isal_version.map new file +3 -0
new file mode 100644
index 0000000..de8e412
@@ -0,0 +1,3 @@
+DPDK_18.05 {
+	local: *;
+};
drivers/compress/meson.build new file +8 -0
new file mode 100644
index 0000000..fb136e1
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+drivers = ['isal']
+
+std_deps = ['compressdev'] # compressdev pulls in all other needed deps
+config_flag_fmt = 'RTE_LIBRTE_@0@_PMD'
+driver_name_fmt = 'rte_pmd_@0@'
drivers/meson.build +1 -0
index b146f09..ac7ba5e 100644
@@ -7,6 +7,7 @@ driver_classes = ['common',
 	       'mempool', # depends on common and bus.
 	       'net',     # depends on common, bus and mempool.
 	       'crypto',  # depends on common, bus and mempool (net in future).
+	       'compress', # depends on bus, mempool.
 	       'event']   # depends on common, bus, mempool and net.
 
 foreach class:driver_classes
mk/rte.app.mk +5 -0
index 8530ac5..7a6739c 100644
@@ -228,6 +228,11 @@ endif # CONFIG_RTE_LIBRTE_DPAA_BUS
 
 endif # CONFIG_RTE_LIBRTE_CRYPTODEV
 
+ifeq ($(CONFIG_RTE_LIBRTE_COMPRESSDEV),y)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += -lrte_pmd_isal_comp
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += -lisal
+endif # CONFIG_RTE_LIBRTE_COMPRESSDEV
+
 ifeq ($(CONFIG_RTE_LIBRTE_EVENTDEV),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SKELETON_EVENTDEV) += -lrte_pmd_skeleton_event
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV) += -lrte_pmd_sw_event
Signature · 1 line
2.7.4
author
#32 lore raw

v5 cover add ISA-L compression PMD

This patchset contains the first compression PMD written under the DPDK compression API, compressdev. The ISA-L compression driver utilizes Intel's ISA-L compression library. It therefore has dependencies on both compressdev and the ISA-L library, v2.22.0.

V2:
- Changes to keep in compliance with compressdev API,
- Enable meson build system,
- General rework & fixes,
- Documentation.

V3:
- Changes to keep in compliance with compressdev API,
- General rework,
- Split into patchset.

V4:
- Changes to keep in compliance with the compressdev API,
- Removed unnecessary branching in compression function, process_isal_deflate,
- Some minor documentation fixes,
- Minor reworks.

V5:
- Removed unnecessary overhead related to dynamic compression,
- Minor reworks.

Lee Daly (10):
compress/isal: add skeleton ISA-L compression PMD compress/isal: add pmd device init and de-init compress/isal: add basic pmd ops compress/isal: add private xform related ops compress/isal: add queue pair related ops compress/isal: support enqueue/dequeue api compress/isal: add stats related ops compress/isal: add ISA-L compression functionality compress/isal: add ISA-L decomp functionality
doc: add compression driver and ISA-L PMD docs

 .gitignore                                        |   1 +
 MAINTAINERS                                       |  12 +
 config/common_base                                |   5 +
 devtools/test-build.sh                            |   4 +
 doc/guides/compressdevs/features/default.ini      |  24 ++
 doc/guides/compressdevs/features/isal.ini         |  22 +
 doc/guides/compressdevs/index.rst                 |  13 +
 doc/guides/compressdevs/isal.rst                  |  77 ++++
 doc/guides/compressdevs/overview.rst              |  12 +
 doc/guides/conf.py                                |   5 +
 doc/guides/index.rst                              |   1 +
 doc/guides/rel_notes/release_18_05.rst            |   4 +
 drivers/Makefile                                  |   2 +
 drivers/compress/Makefile                         |   8 +
 drivers/compress/isal/Makefile                    |  31 ++
 drivers/compress/isal/isal_compress_pmd.c         | 498 ++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c     | 345 +++++++++++++++
 drivers/compress/isal/isal_compress_pmd_private.h |  57 +++
 drivers/compress/isal/meson.build                 |  14 +
 drivers/compress/isal/rte_pmd_isal_version.map    |   3 +
 drivers/compress/meson.build                      |   8 +
 drivers/meson.build                               |   1 +
 mk/rte.app.mk                                     |   5 +
 23 files changed, 1152 insertions(+)
 create mode 100644 doc/guides/compressdevs/features/default.ini
 create mode 100644 doc/guides/compressdevs/features/isal.ini
 create mode 100644 doc/guides/compressdevs/index.rst
 create mode 100644 doc/guides/compressdevs/isal.rst
 create mode 100644 doc/guides/compressdevs/overview.rst
 create mode 100644 drivers/compress/Makefile
 create mode 100644 drivers/compress/isal/Makefile
 create mode 100644 drivers/compress/isal/isal_compress_pmd.c
 create mode 100644 drivers/compress/isal/isal_compress_pmd_ops.c
 create mode 100644 drivers/compress/isal/isal_compress_pmd_private.h
 create mode 100644 drivers/compress/isal/meson.build
 create mode 100644 drivers/compress/isal/rte_pmd_isal_version.map
 create mode 100644 drivers/compress/meson.build
Signature · 1 line
2.7.4
author
#33 lore raw

PATCH v5 1/10 compress/isal: add skeleton ISA-L compression PMD

Adding basic skeleton of the ISA-L compression driver. No compression functionality, but lays the foundation for operations in the rest of the patchset. The ISA-L compression driver utilizes Intel's ISA-L compression library and compressdev API. It therefore has dependencies on both compressdev and the ISA-L library, v2.22.0.

---
 config/common_base                             |  5 +++++
 doc/guides/rel_notes/release_18_05.rst         |  3 +++
 drivers/Makefile                               |  2 ++
 drivers/compress/Makefile                      |  8 +++++++
 drivers/compress/isal/Makefile                 | 30 ++++++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd.c      | 29 +++++++++++++++++++++++++
 drivers/compress/isal/meson.build              | 14 ++++++++++++
 drivers/compress/isal/rte_pmd_isal_version.map |  3 +++
 drivers/compress/meson.build                   |  8 +++++++
 drivers/meson.build                            |  1 +
 mk/rte.app.mk                                  |  5 +++++
 11 files changed, 108 insertions(+)
 create mode 100644 drivers/compress/Makefile
 create mode 100644 drivers/compress/isal/Makefile
 create mode 100644 drivers/compress/isal/isal_compress_pmd.c
 create mode 100644 drivers/compress/isal/meson.build
 create mode 100644 drivers/compress/isal/rte_pmd_isal_version.map
 create mode 100644 drivers/compress/meson.build
config/common_base +5 -0
index dd836e8..a982570 100644
@@ -562,6 +562,11 @@ CONFIG_RTE_COMPRESS_MAX_DEVS=64
 CONFIG_RTE_COMPRESSDEV_TEST=y
 
 #
+# Compile PMD for ISA-L compression device
+#
+CONFIG_RTE_LIBRTE_PMD_ISAL=n
+
+#
 # Compile generic event device library
 #
 CONFIG_RTE_LIBRTE_EVENTDEV=y
doc/guides/rel_notes/release_18_05.rst +3 -0
index 5193a62..ce70d1f 100644
@@ -75,7 +75,10 @@ New Features
   The compressdev library provides an API for offload of compression and
   decompression operations to hardware or software accelerator devices.
 
+* **Added a new compression poll mode driver using Intels ISA-L.**
 
+   Added the new ``ISA-L`` compression driver, for compression and decompression
+   operations in software.
 
 
 API Changes
drivers/Makefile +2 -0
index 3d9f86b..c88638c 100644
@@ -13,6 +13,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_BBDEV) += baseband
 DEPDIRS-baseband := common bus mempool
 DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
 DEPDIRS-crypto := common bus mempool
+DIRS-$(CONFIG_RTE_LIBRTE_COMPRESSDEV) += compress
+DEPDIRS-compress := bus mempool
 DIRS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += event
 DEPDIRS-event := common bus mempool net
 DIRS-$(CONFIG_RTE_LIBRTE_RAWDEV) += raw
drivers/compress/Makefile new file +8 -0
new file mode 100644
index 0000000..592497f
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal
+
+include $(RTE_SDK)/mk/rte.subdir.mk
drivers/compress/isal/Makefile new file +30 -0
new file mode 100644
index 0000000..9b1d866
@@ -0,0 +1,30 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_pmd_isal_comp.a
+
+# build flags
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+# external library dependencies
+LDLIBS += -lisal
+LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring
+LDLIBS += -lrte_compressdev
+LDLIBS += -lrte_bus_vdev
+
+# library version
+LIBABIVER := 1
+
+# versioning export map
+EXPORT_MAP := rte_pmd_isal_version.map
+
+# library source files
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal_compress_pmd.c
+
+# export include files
+include $(RTE_SDK)/mk/rte.lib.mk
drivers/compress/isal/isal_compress_pmd.c new file +29 -0
new file mode 100644
index 0000000..d7137fd
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include <rte_bus_vdev.h>
+#include <rte_compressdev_pmd.h>
+
+/** Remove compression device */
+static int
+compdev_isal_remove_dev(struct rte_vdev_device *vdev __rte_unused)
+{
+	return 0;
+}
+
+/** Initialise ISA-L compression device */
+static int
+compdev_isal_probe(struct rte_vdev_device *dev __rte_unused)
+{
+	return 0;
+}
+
+static struct rte_vdev_driver compdev_isal_pmd_drv = {
+	.probe = compdev_isal_probe,
+	.remove = compdev_isal_remove_dev,
+};
+
+RTE_PMD_REGISTER_VDEV(COMPDEV_NAME_ISAL_PMD, compdev_isal_pmd_drv);
+RTE_PMD_REGISTER_PARAM_STRING(COMPDEV_NAME_ISAL_PMD,
+	"socket_id=<int>");
drivers/compress/isal/meson.build new file +14 -0
new file mode 100644
index 0000000..4447e20
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2018 Intel Corporation
+
+dep = dependency('libisal', required: false)
+if not dep.found()
+       build =false
+endif
+
+deps += 'bus_vdev'
+sources = files('isal_compress_pmd.c')
+ext_deps += dep
+pkgconfig_extra_libs += '-lisal'
+
+allow_experimental_apis = true
drivers/compress/isal/rte_pmd_isal_version.map new file +3 -0
new file mode 100644
index 0000000..de8e412
@@ -0,0 +1,3 @@
+DPDK_18.05 {
+	local: *;
+};
drivers/compress/meson.build new file +8 -0
new file mode 100644
index 0000000..fb136e1
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+drivers = ['isal']
+
+std_deps = ['compressdev'] # compressdev pulls in all other needed deps
+config_flag_fmt = 'RTE_LIBRTE_@0@_PMD'
+driver_name_fmt = 'rte_pmd_@0@'
drivers/meson.build +1 -0
index b146f09..ac7ba5e 100644
@@ -7,6 +7,7 @@ driver_classes = ['common',
 	       'mempool', # depends on common and bus.
 	       'net',     # depends on common, bus and mempool.
 	       'crypto',  # depends on common, bus and mempool (net in future).
+	       'compress', # depends on bus, mempool.
 	       'event']   # depends on common, bus, mempool and net.
 
 foreach class:driver_classes
mk/rte.app.mk +5 -0
index 8530ac5..7a6739c 100644
@@ -228,6 +228,11 @@ endif # CONFIG_RTE_LIBRTE_DPAA_BUS
 
 endif # CONFIG_RTE_LIBRTE_CRYPTODEV
 
+ifeq ($(CONFIG_RTE_LIBRTE_COMPRESSDEV),y)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += -lrte_pmd_isal_comp
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += -lisal
+endif # CONFIG_RTE_LIBRTE_COMPRESSDEV
+
 ifeq ($(CONFIG_RTE_LIBRTE_EVENTDEV),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SKELETON_EVENTDEV) += -lrte_pmd_skeleton_event
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV) += -lrte_pmd_sw_event
Signature · 1 line
2.7.4
author
#34 lore raw

v6 cover add ISA-L compression PMD

V2:
- Changes to keep in compliance with compressdev API,
- Enable meson build system,
- General rework & fixes,
- Documentation.

V3:
- Changes to keep in compliance with compressdev API,
- General rework,
- Split into patchset.

V4:
- Changes to keep in compliance with the compressdev API,
- Removed unnecessary branching in compression function, process_isal_deflate,
- Some minor documentation fixes,
- Minor reworks.

V5:
- Removed unnecessary overhead related to dynamic huffman code compression,
- Minor reworks.

V6:
- Removed partial checksum support for enablement of full checksum in future,
- Added fix to dynamic huffman code compression.

Lee Daly (10):
compress/isal: add skeleton ISA-L compression PMD compress/isal: add pmd device init and de-init compress/isal: add basic pmd ops compress/isal: add private xform related ops compress/isal: add queue pair related ops compress/isal: support enqueue/dequeue api compress/isal: add stats related ops compress/isal: add ISA-L compression functionality compress/isal: add ISA-L decomp functionality
doc: add compression driver and ISA-L PMD docs

 .gitignore                                        |   1 +
 MAINTAINERS                                       |  12 +
 config/common_base                                |   5 +
 devtools/test-build.sh                            |   4 +
 doc/guides/compressdevs/features/default.ini      |  24 ++
 doc/guides/compressdevs/features/isal.ini         |  22 +
 doc/guides/compressdevs/index.rst                 |  13 +
 doc/guides/compressdevs/isal.rst                  |  74 ++++
 doc/guides/compressdevs/overview.rst              |  12 +
 doc/guides/conf.py                                |   5 +
 doc/guides/index.rst                              |   1 +
 doc/guides/rel_notes/release_18_05.rst            |   4 +
 drivers/Makefile                                  |   2 +
 drivers/compress/Makefile                         |   8 +
 drivers/compress/isal/Makefile                    |  31 ++
 drivers/compress/isal/isal_compress_pmd.c         | 472 ++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c     | 343 ++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_private.h |  57 +++
 drivers/compress/isal/meson.build                 |  14 +
 drivers/compress/isal/rte_pmd_isal_version.map    |   3 +
 drivers/compress/meson.build                      |   8 +
 drivers/meson.build                               |   1 +
 mk/rte.app.mk                                     |   5 +
 23 files changed, 1121 insertions(+)
 create mode 100644 doc/guides/compressdevs/features/default.ini
 create mode 100644 doc/guides/compressdevs/features/isal.ini
 create mode 100644 doc/guides/compressdevs/index.rst
 create mode 100644 doc/guides/compressdevs/isal.rst
 create mode 100644 doc/guides/compressdevs/overview.rst
 create mode 100644 drivers/compress/Makefile
 create mode 100644 drivers/compress/isal/Makefile
 create mode 100644 drivers/compress/isal/isal_compress_pmd.c
 create mode 100644 drivers/compress/isal/isal_compress_pmd_ops.c
 create mode 100644 drivers/compress/isal/isal_compress_pmd_private.h
 create mode 100644 drivers/compress/isal/meson.build
 create mode 100644 drivers/compress/isal/rte_pmd_isal_version.map
 create mode 100644 drivers/compress/meson.build
Signature · 1 line
2.7.4
author
#35 lore raw

PATCH v6 1/10 compress/isal: add skeleton ISA-L compression PMD

Adding basic skeleton of the ISA-L compression driver. No compression functionality, but lays the foundation for operations in the rest of the patchset. The ISA-L compression driver utilizes Intel's ISA-L compression library and compressdev API. It therefore has dependencies on both compressdev and the ISA-L library, v2.22.0.

---
 config/common_base                             |  5 +++++
 doc/guides/rel_notes/release_18_05.rst         |  3 +++
 drivers/Makefile                               |  2 ++
 drivers/compress/Makefile                      |  8 +++++++
 drivers/compress/isal/Makefile                 | 30 ++++++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd.c      | 29 +++++++++++++++++++++++++
 drivers/compress/isal/meson.build              | 14 ++++++++++++
 drivers/compress/isal/rte_pmd_isal_version.map |  3 +++
 drivers/compress/meson.build                   |  8 +++++++
 drivers/meson.build                            |  1 +
 mk/rte.app.mk                                  |  5 +++++
 11 files changed, 108 insertions(+)
 create mode 100644 drivers/compress/Makefile
 create mode 100644 drivers/compress/isal/Makefile
 create mode 100644 drivers/compress/isal/isal_compress_pmd.c
 create mode 100644 drivers/compress/isal/meson.build
 create mode 100644 drivers/compress/isal/rte_pmd_isal_version.map
 create mode 100644 drivers/compress/meson.build
config/common_base +5 -0
index dd836e8..a982570 100644
@@ -562,6 +562,11 @@ CONFIG_RTE_COMPRESS_MAX_DEVS=64
 CONFIG_RTE_COMPRESSDEV_TEST=y
 
 #
+# Compile PMD for ISA-L compression device
+#
+CONFIG_RTE_LIBRTE_PMD_ISAL=n
+
+#
 # Compile generic event device library
 #
 CONFIG_RTE_LIBRTE_EVENTDEV=y
doc/guides/rel_notes/release_18_05.rst +3 -0
index 5193a62..ce70d1f 100644
@@ -75,7 +75,10 @@ New Features
   The compressdev library provides an API for offload of compression and
   decompression operations to hardware or software accelerator devices.
 
+* **Added a new compression poll mode driver using Intels ISA-L.**
 
+   Added the new ``ISA-L`` compression driver, for compression and decompression
+   operations in software.
 
 
 API Changes
drivers/Makefile +2 -0
index 3d9f86b..c88638c 100644
@@ -13,6 +13,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_BBDEV) += baseband
 DEPDIRS-baseband := common bus mempool
 DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
 DEPDIRS-crypto := common bus mempool
+DIRS-$(CONFIG_RTE_LIBRTE_COMPRESSDEV) += compress
+DEPDIRS-compress := bus mempool
 DIRS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += event
 DEPDIRS-event := common bus mempool net
 DIRS-$(CONFIG_RTE_LIBRTE_RAWDEV) += raw
drivers/compress/Makefile new file +8 -0
new file mode 100644
index 0000000..592497f
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal
+
+include $(RTE_SDK)/mk/rte.subdir.mk
drivers/compress/isal/Makefile new file +30 -0
new file mode 100644
index 0000000..9b1d866
@@ -0,0 +1,30 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_pmd_isal_comp.a
+
+# build flags
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+# external library dependencies
+LDLIBS += -lisal
+LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring
+LDLIBS += -lrte_compressdev
+LDLIBS += -lrte_bus_vdev
+
+# library version
+LIBABIVER := 1
+
+# versioning export map
+EXPORT_MAP := rte_pmd_isal_version.map
+
+# library source files
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal_compress_pmd.c
+
+# export include files
+include $(RTE_SDK)/mk/rte.lib.mk
drivers/compress/isal/isal_compress_pmd.c new file +29 -0
new file mode 100644
index 0000000..d7137fd
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include <rte_bus_vdev.h>
+#include <rte_compressdev_pmd.h>
+
+/** Remove compression device */
+static int
+compdev_isal_remove_dev(struct rte_vdev_device *vdev __rte_unused)
+{
+	return 0;
+}
+
+/** Initialise ISA-L compression device */
+static int
+compdev_isal_probe(struct rte_vdev_device *dev __rte_unused)
+{
+	return 0;
+}
+
+static struct rte_vdev_driver compdev_isal_pmd_drv = {
+	.probe = compdev_isal_probe,
+	.remove = compdev_isal_remove_dev,
+};
+
+RTE_PMD_REGISTER_VDEV(COMPDEV_NAME_ISAL_PMD, compdev_isal_pmd_drv);
+RTE_PMD_REGISTER_PARAM_STRING(COMPDEV_NAME_ISAL_PMD,
+	"socket_id=<int>");
drivers/compress/isal/meson.build new file +14 -0
new file mode 100644
index 0000000..4447e20
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2018 Intel Corporation
+
+dep = dependency('libisal', required: false)
+if not dep.found()
+       build =false
+endif
+
+deps += 'bus_vdev'
+sources = files('isal_compress_pmd.c')
+ext_deps += dep
+pkgconfig_extra_libs += '-lisal'
+
+allow_experimental_apis = true
drivers/compress/isal/rte_pmd_isal_version.map new file +3 -0
new file mode 100644
index 0000000..de8e412
@@ -0,0 +1,3 @@
+DPDK_18.05 {
+	local: *;
+};
drivers/compress/meson.build new file +8 -0
new file mode 100644
index 0000000..fb136e1
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+drivers = ['isal']
+
+std_deps = ['compressdev'] # compressdev pulls in all other needed deps
+config_flag_fmt = 'RTE_LIBRTE_@0@_PMD'
+driver_name_fmt = 'rte_pmd_@0@'
drivers/meson.build +1 -0
index b146f09..ac7ba5e 100644
@@ -7,6 +7,7 @@ driver_classes = ['common',
 	       'mempool', # depends on common and bus.
 	       'net',     # depends on common, bus and mempool.
 	       'crypto',  # depends on common, bus and mempool (net in future).
+	       'compress', # depends on bus, mempool.
 	       'event']   # depends on common, bus, mempool and net.
 
 foreach class:driver_classes
mk/rte.app.mk +5 -0
index 8530ac5..7a6739c 100644
@@ -228,6 +228,11 @@ endif # CONFIG_RTE_LIBRTE_DPAA_BUS
 
 endif # CONFIG_RTE_LIBRTE_CRYPTODEV
 
+ifeq ($(CONFIG_RTE_LIBRTE_COMPRESSDEV),y)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += -lrte_pmd_isal_comp
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += -lisal
+endif # CONFIG_RTE_LIBRTE_COMPRESSDEV
+
 ifeq ($(CONFIG_RTE_LIBRTE_EVENTDEV),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SKELETON_EVENTDEV) += -lrte_pmd_skeleton_event
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV) += -lrte_pmd_sw_event
Signature · 1 line
2.7.4
author
#36 lore raw

PATCH v6 2/10 compress/isal: add pmd device init and de-init

In reply to Lee Daly

This patch adds device initialization functionality such as probe and create, also adding deinitialize functionality.
Dynamic logging component also added.

---
 drivers/compress/isal/Makefile                    |  1 +
 drivers/compress/isal/isal_compress_pmd.c         | 74 +++++++++++++++++++++--
 drivers/compress/isal/isal_compress_pmd_ops.c     | 25 ++++++++
 drivers/compress/isal/isal_compress_pmd_private.h | 24 ++++++++
 drivers/compress/isal/meson.build                 |  2 +-
 5 files changed, 121 insertions(+), 5 deletions(-)
 create mode 100644 drivers/compress/isal/isal_compress_pmd_ops.c
 create mode 100644 drivers/compress/isal/isal_compress_pmd_private.h
drivers/compress/isal/Makefile +1 -0
index 9b1d866..95904f6 100644
@@ -25,6 +25,7 @@ EXPORT_MAP := rte_pmd_isal_version.map
 
 # library source files
 SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal_compress_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal_compress_pmd_ops.c
 
 # export include files
 include $(RTE_SDK)/mk/rte.lib.mk
drivers/compress/isal/isal_compress_pmd.c +70 -4

93 lines not on this page · raw message

drivers/compress/isal/isal_compress_pmd_ops.c new file +25 -0
new file mode 100644
index 0000000..cff05b4
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include <rte_compressdev_pmd.h>
+
+struct rte_compressdev_ops isal_pmd_ops = {
+		.dev_configure		= NULL,
+		.dev_start		= NULL,
+		.dev_stop		= NULL,
+		.dev_close		= NULL,
+
+		.stats_get		= NULL,
+		.stats_reset		= NULL,
+
+		.dev_infos_get		= NULL,
+
+		.queue_pair_setup	= NULL,
+		.queue_pair_release	= NULL,
+
+		.private_xform_create	= NULL,
+		.private_xform_free	= NULL,
+};
+
+struct rte_compressdev_ops *isal_compress_pmd_ops = &isal_pmd_ops;
drivers/compress/isal/isal_compress_pmd_private.h new file +24 -0
new file mode 100644
index 0000000..09ecfb7
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#ifndef _ISAL_COMP_PMD_PRIVATE_H_
+#define _ISAL_COMP_PMD_PRIVATE_H_
+
+#define COMPDEV_NAME_ISAL_PMD		compress_isal
+/**< ISA-L comp PMD device name */
+
+extern int isal_logtype_driver;
+#define ISAL_PMD_LOG(level, fmt, args...) \
+	rte_log(RTE_LOG_ ## level, isal_logtype_driver, "%s(): "fmt "\n", \
+			__func__, ##args)
+
+/* private data structure for each ISA-L compression device */
+struct isal_comp_private {
+	struct rte_mempool *priv_xform_mp;
+};
+
+/** device specific operations function pointer structure */
+extern struct rte_compressdev_ops *isal_compress_pmd_ops;
+
+#endif /* _ISAL_COMP_PMD_PRIVATE_H_ */
drivers/compress/isal/meson.build +1 -1
index 4447e20..94c10fd 100644
@@ -7,7 +7,7 @@ if not dep.found()
 endif
 
 deps += 'bus_vdev'
-sources = files('isal_compress_pmd.c')
+sources = files('isal_compress_pmd.c', 'isal_compress_pmd_ops.c')
 ext_deps += dep
 pkgconfig_extra_libs += '-lisal'
 
Signature · 1 line
2.7.4
author
#37 lore raw

PATCH v6 3/10 compress/isal: add basic pmd ops

In reply to Lee Daly

Add basic device poll mode driver operations to give ability to start, stop, close device etc.

---
 drivers/compress/isal/isal_compress_pmd_ops.c     | 63 +++++++++++++++++++++--
 drivers/compress/isal/isal_compress_pmd_private.h | 12 +++++
 2 files changed, 70 insertions(+), 5 deletions(-)
drivers/compress/isal/isal_compress_pmd_ops.c +58 -5
index cff05b4..2f828c5 100644
@@ -2,18 +2,71 @@
  * Copyright(c) 2018 Intel Corporation
  */
 
+#include <rte_common.h>
 #include <rte_compressdev_pmd.h>
 
+#include "isal_compress_pmd_private.h"
+
+static const struct rte_compressdev_capabilities isal_pmd_capabilities[] = {
+	RTE_COMP_END_OF_CAPABILITIES_LIST()
+};
+
+/** Configure device */
+static int
+isal_comp_pmd_config(struct rte_compressdev *dev __rte_unused,
+		struct rte_compressdev_config *config __rte_unused)
+{
+	return 0;
+}
+
+/** Start device */
+static int
+isal_comp_pmd_start(__rte_unused struct rte_compressdev *dev)
+{
+	return 0;
+}
+
+/** Stop device */
+static void
+isal_comp_pmd_stop(__rte_unused struct rte_compressdev *dev)
+{
+}
+
+/** Close device */
+static int
+isal_comp_pmd_close(struct rte_compressdev *dev)
+{
+	/* Free private data */
+	struct isal_comp_private *internals = dev->data->dev_private;
+
+	rte_mempool_free(internals->priv_xform_mp);
+	return 0;
+}
+
+/** Get device info */
+static void
+isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
+		struct rte_compressdev_info *dev_info)
+{
+	if (dev_info != NULL) {
+		dev_info->capabilities = isal_pmd_capabilities;
+		dev_info->feature_flags = RTE_COMPDEV_FF_CPU_AVX512 |
+				RTE_COMPDEV_FF_CPU_AVX2 |
+				RTE_COMPDEV_FF_CPU_AVX |
+				RTE_COMPDEV_FF_CPU_SSE;
+	}
+}
+
 struct rte_compressdev_ops isal_pmd_ops = {
-		.dev_configure		= NULL,
-		.dev_start		= NULL,
-		.dev_stop		= NULL,
-		.dev_close		= NULL,
+		.dev_configure		= isal_comp_pmd_config,
+		.dev_start		= isal_comp_pmd_start,
+		.dev_stop		= isal_comp_pmd_stop,
+		.dev_close		= isal_comp_pmd_close,
 
 		.stats_get		= NULL,
 		.stats_reset		= NULL,
 
-		.dev_infos_get		= NULL,
+		.dev_infos_get		= isal_comp_pmd_info_get,
 
 		.queue_pair_setup	= NULL,
 		.queue_pair_release	= NULL,
drivers/compress/isal/isal_compress_pmd_private.h +12 -0
index 09ecfb7..efbe68b 100644
@@ -18,6 +18,18 @@ struct isal_comp_private {
 	struct rte_mempool *priv_xform_mp;
 };
 
+/** ISA-L queue pair */
+struct isal_comp_qp {
+	/* Queue Pair Identifier */
+	uint16_t id;
+	/* Unique Queue Pair Name */
+	char name[RTE_COMPRESSDEV_NAME_MAX_LEN];
+	/* Queue pair statistics */
+	struct rte_compressdev_stats qp_stats;
+	/* Number of free elements on ring */
+	uint16_t num_free_elements;
+} __rte_cache_aligned;
+
 /** device specific operations function pointer structure */
 extern struct rte_compressdev_ops *isal_compress_pmd_ops;
 
Signature · 1 line
2.7.4
author
#38 lore raw

PATCH v6 4/10 compress/isal: add private xform related ops

In reply to Lee Daly

This patch creates, configures and frees the private xform, taking the applications xform and using it to populate the PMDs own private xform with the information which will be required for the compress/decompress functions. This information includes the level, algorithm, type of huffman code, type of checksum etc.

---
 drivers/compress/isal/isal_compress_pmd.c         | 176 ++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c     | 103 ++++++++++++-
 drivers/compress/isal/isal_compress_pmd_private.h |  15 ++
 3 files changed, 289 insertions(+), 5 deletions(-)
drivers/compress/isal/isal_compress_pmd.c +176 -0
index 5cc9409..1224dd0 100644
@@ -1,6 +1,7 @@
 /* SPDX-License-Identifier: BSD-3-Clause
  * Copyright(c) 2018 Intel Corporation
  */
+#include <isa-l.h>
 
 #include <rte_bus_vdev.h>
 #include <rte_common.h>
@@ -9,8 +10,183 @@
 
 #include "isal_compress_pmd_private.h"
 
+#define RTE_COMP_ISAL_WINDOW_SIZE 15
+#define RTE_COMP_ISAL_LEVEL_ZERO 0 /* ISA-L Level 0 used for fixed Huffman */
+#define RTE_COMP_ISAL_LEVEL_ONE 1
+#define RTE_COMP_ISAL_LEVEL_TWO 2
+#define RTE_COMP_ISAL_LEVEL_THREE 3 /* Optimised for AVX512 & AVX2 only */
+
 int isal_logtype_driver;
 
+/* Verify and set private xform parameters */
+int
+isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
+		const struct rte_comp_xform *xform)
+{
+	if (xform == NULL)
+		return -EINVAL;
+
+	/* Set compression private xform variables */
+	if (xform->type == RTE_COMP_COMPRESS) {
+		/* Set private xform type - COMPRESS/DECOMPRESS */
+		priv_xform->type = RTE_COMP_COMPRESS;
+
+		/* Set private xform algorithm */
+		if (xform->compress.algo != RTE_COMP_ALGO_DEFLATE) {
+			if (xform->compress.algo == RTE_COMP_ALGO_NULL) {
+				ISAL_PMD_LOG(ERR, "By-pass not supported\n");
+				return -ENOTSUP;
+			}
+			ISAL_PMD_LOG(ERR, "Algorithm not supported\n");
+			return -ENOTSUP;
+		}
+		priv_xform->compress.algo = RTE_COMP_ALGO_DEFLATE;
+
+		/* Set private xform checksum - raw deflate by default */
+		if (xform->compress.chksum != RTE_COMP_CHECKSUM_NONE) {
+			ISAL_PMD_LOG(ERR, "Checksum not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform window size, 32K supported */
+		if (xform->compress.window_size == RTE_COMP_ISAL_WINDOW_SIZE)
+			priv_xform->compress.window_size =
+					RTE_COMP_ISAL_WINDOW_SIZE;
+		else {
+			ISAL_PMD_LOG(ERR, "Window size not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform huffman type */
+		switch (xform->compress.deflate.huffman) {
+		case(RTE_COMP_HUFFMAN_DEFAULT):
+			priv_xform->compress.deflate.huffman =
+					RTE_COMP_HUFFMAN_DEFAULT;
+			break;
+		case(RTE_COMP_HUFFMAN_FIXED):
+			priv_xform->compress.deflate.huffman =
+					RTE_COMP_HUFFMAN_FIXED;
+			break;
+		case(RTE_COMP_HUFFMAN_DYNAMIC):
+			priv_xform->compress.deflate.huffman =
+					RTE_COMP_HUFFMAN_DYNAMIC;
+			break;
+		default:
+			ISAL_PMD_LOG(ERR, "Huffman code not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform level.
+		 * Checking compliance with compressdev API, -1 <= level => 9
+		 */
+		if (xform->compress.level < RTE_COMP_LEVEL_PMD_DEFAULT ||
+				xform->compress.level > RTE_COMP_LEVEL_MAX) {
+			ISAL_PMD_LOG(ERR, "Compression level out of range\n");
+			return -EINVAL;
+		}
+		/* Check for Compressdev API level 0, No compression
+		 * not supported in ISA-L
+		 */
+		else if (xform->compress.level == RTE_COMP_LEVEL_NONE) {
+			ISAL_PMD_LOG(ERR, "No Compression not supported\n");
+			return -ENOTSUP;
+		}
+		/* If using fixed huffman code, level must be 0 */
+		else if (priv_xform->compress.deflate.huffman ==
+				RTE_COMP_HUFFMAN_FIXED) {
+			ISAL_PMD_LOG(DEBUG, "ISA-L level 0 used due to a"
+					" fixed huffman code\n");
+			priv_xform->compress.level = RTE_COMP_ISAL_LEVEL_ZERO;
+			priv_xform->level_buffer_size =
+					ISAL_DEF_LVL0_DEFAULT;
+		} else {
+			/* Mapping API levels to ISA-L levels 1,2 & 3 */
+			switch (xform->compress.level) {
+			case RTE_COMP_LEVEL_PMD_DEFAULT:
+				/* Default is 1 if not using fixed huffman */
+				priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_ONE;
+				priv_xform->level_buffer_size =
+						ISAL_DEF_LVL1_DEFAULT;
+				break;
+			case RTE_COMP_LEVEL_MIN:
+				priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_ONE;
+				priv_xform->level_buffer_size =
+						ISAL_DEF_LVL1_DEFAULT;
+				break;
+			case RTE_COMP_ISAL_LEVEL_TWO:
+				priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_TWO;
+				priv_xform->level_buffer_size =
+						ISAL_DEF_LVL2_DEFAULT;
+				break;
+			/* Level 3 or higher requested */
+			default:
+				/* Check for AVX512, to use ISA-L level 3 */
+				if (rte_cpu_get_flag_enabled(
+						RTE_CPUFLAG_AVX512F)) {
+					priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_THREE;
+					priv_xform->level_buffer_size =
+						ISAL_DEF_LVL3_DEFAULT;
+				}
+				/* Check for AVX2, to use ISA-L level 3 */
+				else if (rte_cpu_get_flag_enabled(
+						RTE_CPUFLAG_AVX2)) {
+					priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_THREE;
+					priv_xform->level_buffer_size =
+						ISAL_DEF_LVL3_DEFAULT;
+				} else {
+					ISAL_PMD_LOG(DEBUG, "Requested ISA-L level"
+						" 3 or above; Level 3 optimized"
+						" for AVX512 & AVX2 only."
+						" level changed to 2.\n");
+					priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_TWO;
+					priv_xform->level_buffer_size =
+						ISAL_DEF_LVL2_DEFAULT;
+				}
+			}
+		}
+	}
+
+	/* Set decompression private xform variables */
+	else if (xform->type == RTE_COMP_DECOMPRESS) {
+
+		/* Set private xform type - COMPRESS/DECOMPRESS */
+		priv_xform->type = RTE_COMP_DECOMPRESS;
+
+		/* Set private xform algorithm */
+		if (xform->decompress.algo != RTE_COMP_ALGO_DEFLATE) {
+			if (xform->decompress.algo == RTE_COMP_ALGO_NULL) {
+				ISAL_PMD_LOG(ERR, "By pass not supported\n");
+				return -ENOTSUP;
+			}
+			ISAL_PMD_LOG(ERR, "Algorithm not supported\n");
+			return -ENOTSUP;
+		}
+		priv_xform->decompress.algo = RTE_COMP_ALGO_DEFLATE;
+
+		/* Set private xform checksum - raw deflate by default */
+		if (xform->compress.chksum != RTE_COMP_CHECKSUM_NONE) {
+			ISAL_PMD_LOG(ERR, "Checksum not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform window size, 32K supported */
+		if (xform->decompress.window_size == RTE_COMP_ISAL_WINDOW_SIZE)
+			priv_xform->decompress.window_size =
+					RTE_COMP_ISAL_WINDOW_SIZE;
+		else {
+			ISAL_PMD_LOG(ERR, "Window size not supported\n");
+			return -ENOTSUP;
+		}
+	}
+	return 0;
+}
+
 /* Create ISA-L compression device */
 static int
 compdev_isal_create(const char *name, struct rte_vdev_device *vdev,
drivers/compress/isal/isal_compress_pmd_ops.c +98 -5

122 lines not on this page · raw message

drivers/compress/isal/isal_compress_pmd_private.h +15 -0
index efbe68b..7e3b840 100644
@@ -30,6 +30,21 @@ struct isal_comp_qp {
 	uint16_t num_free_elements;
 } __rte_cache_aligned;
 
+/** ISA-L private xform structure */
+struct isal_priv_xform {
+	enum rte_comp_xform_type type;
+	union {
+		struct rte_comp_compress_xform compress;
+		struct rte_comp_decompress_xform decompress;
+	};
+	uint32_t level_buffer_size;
+} __rte_cache_aligned;
+
+/** Set and validate NULL comp private xform parameters */
+extern int
+isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
+			const struct rte_comp_xform *xform);
+
 /** device specific operations function pointer structure */
 extern struct rte_compressdev_ops *isal_compress_pmd_ops;
 
Signature · 1 line
2.7.4
author
#39 lore raw

PATCH v6 5/10 compress/isal: add queue pair related ops

In reply to Lee Daly

This adds queue pair operations such as setup and release.

---
 drivers/compress/isal/isal_compress_pmd_ops.c     | 111 +++++++++++++++++++++-
 drivers/compress/isal/isal_compress_pmd_private.h |   2 +
 2 files changed, 111 insertions(+), 2 deletions(-)
drivers/compress/isal/isal_compress_pmd_ops.c +109 -2
index 9c89df0..b0abf42 100644
@@ -4,6 +4,7 @@
 
 #include <rte_common.h>
 #include <rte_compressdev_pmd.h>
+#include <rte_malloc.h>
 
 #include "isal_compress_pmd_private.h"
 
@@ -106,6 +107,112 @@ isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
 	}
 }
 
+
+/** Release queue pair */
+static int
+isal_comp_pmd_qp_release(struct rte_compressdev *dev, uint16_t qp_id)
+{
+	struct isal_comp_qp *qp = dev->data->queue_pairs[qp_id];
+
+	if (qp == NULL)
+		return -EINVAL;
+
+	if (dev->data->queue_pairs[qp_id] != NULL)
+		rte_free(dev->data->queue_pairs[qp_id]);
+
+	return 0;
+}
+
+/** Create a ring to place process packets on */
+static struct rte_ring *
+isal_comp_pmd_qp_create_processed_pkts_ring(struct isal_comp_qp *qp,
+		unsigned int ring_size, int socket_id)
+{
+	struct rte_ring *r;
+
+	r = rte_ring_lookup(qp->name);
+	if (r) {
+		if (rte_ring_get_size(r) >= ring_size) {
+			ISAL_PMD_LOG(DEBUG,
+				"Reusing existing ring %s for processed packets",
+				qp->name);
+			return r;
+		}
+
+			ISAL_PMD_LOG(ERR,
+				"Unable to reuse existing ring %s"
+				" for processed packets",
+			 qp->name);
+		return NULL;
+	}
+
+	return rte_ring_create(qp->name, ring_size, socket_id,
+			RING_F_SP_ENQ | RING_F_SC_DEQ);
+}
+
+/** set a unique name for the queue pair based on its name, dev_id and qp_id */
+static int
+isal_comp_pmd_qp_set_unique_name(struct rte_compressdev *dev,
+struct isal_comp_qp *qp)
+{
+	unsigned int n = snprintf(qp->name, sizeof(qp->name),
+			"isal_compression_pmd_%u_qp_%u",
+			dev->data->dev_id, qp->id);
+
+	if (n >= sizeof(qp->name))
+		return -1;
+
+	return 0;
+}
+
+/* Setup a queue pair */
+static int
+isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
+		uint32_t max_inflight_ops, int socket_id)
+{
+	struct isal_comp_qp *qp = NULL;
+	int retval;
+
+	/* Free memory prior to re-allocation if needed. */
+	if (dev->data->queue_pairs[qp_id] != NULL)
+		isal_comp_pmd_qp_release(dev, qp_id);
+
+	/* Allocate the queue pair data structure. */
+	qp = rte_zmalloc_socket("Isa-l compression PMD Queue Pair", sizeof(*qp),
+					RTE_CACHE_LINE_SIZE, socket_id);
+	if (qp == NULL) {
+		ISAL_PMD_LOG(ERR, "Failed to allocate queue pair memory");
+		return (-ENOMEM);
+	}
+
+	qp->id = qp_id;
+	dev->data->queue_pairs[qp_id] = qp;
+
+	retval = isal_comp_pmd_qp_set_unique_name(dev, qp);
+	if (retval) {
+		ISAL_PMD_LOG(ERR, "Failed to create unique name for isal "
+				"compression device");
+		goto qp_setup_cleanup;
+	}
+
+	qp->processed_pkts = isal_comp_pmd_qp_create_processed_pkts_ring(qp,
+			max_inflight_ops, socket_id);
+	if (qp->processed_pkts == NULL) {
+		ISAL_PMD_LOG(ERR, "Failed to create unique name for isal "
+				"compression device");
+		goto qp_setup_cleanup;
+	}
+
+	memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
+	return 0;
+
+qp_setup_cleanup:
+	if (qp)
+		rte_free(qp);
+
+	return -1;
+}
+
 /** Set private xform data*/
 static int
 isal_comp_pmd_priv_xform_create(struct rte_compressdev *dev,
@@ -161,8 +268,8 @@ struct rte_compressdev_ops isal_pmd_ops = {
 
 		.dev_infos_get		= isal_comp_pmd_info_get,
 
-		.queue_pair_setup	= NULL,
-		.queue_pair_release	= NULL,
+		.queue_pair_setup	= isal_comp_pmd_qp_setup,
+		.queue_pair_release	= isal_comp_pmd_qp_release,
 
 		.private_xform_create	= isal_comp_pmd_priv_xform_create,
 		.private_xform_free	= isal_comp_pmd_priv_xform_free,
drivers/compress/isal/isal_compress_pmd_private.h +2 -0
index 7e3b840..7505c76 100644
@@ -24,6 +24,8 @@ struct isal_comp_qp {
 	uint16_t id;
 	/* Unique Queue Pair Name */
 	char name[RTE_COMPRESSDEV_NAME_MAX_LEN];
+	/* Ring for placing process packets */
+	struct rte_ring *processed_pkts;
 	/* Queue pair statistics */
 	struct rte_compressdev_stats qp_stats;
 	/* Number of free elements on ring */
Signature · 1 line
2.7.4
author
#40 lore raw

PATCH v6 6/10 compress/isal: support enqueue/dequeue api

In reply to Lee Daly

This patchs adds support for the compressdev enqueue_burst and dequeue_burst API operations.

---
 drivers/compress/isal/isal_compress_pmd.c     | 70 +++++++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c |  2 +
 2 files changed, 72 insertions(+)
drivers/compress/isal/isal_compress_pmd.c +70 -0
index 1224dd0..aaa0593 100644
@@ -187,6 +187,72 @@ isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
 	return 0;
 }
 
+/* Process compression/decompression operation */
+static int
+process_op(struct isal_comp_qp *qp __rte_unused,
+		struct rte_comp_op *op __rte_unused,
+		struct isal_priv_xform *priv_xform)
+{
+	switch (priv_xform->type) {
+	case RTE_COMP_COMPRESS:
+		break;
+	case RTE_COMP_DECOMPRESS:
+		break;
+	default:
+		ISAL_PMD_LOG(ERR, "Operation Not Supported\n");
+		return -ENOTSUP;
+	}
+	return 0;
+}
+
+/* Enqueue burst */
+static uint16_t
+isal_comp_pmd_enqueue_burst(void *queue_pair, struct rte_comp_op **ops,
+			uint16_t nb_ops)
+{
+	struct isal_comp_qp *qp = queue_pair;
+	uint16_t i;
+	int retval;
+	int16_t num_enq = RTE_MIN(qp->num_free_elements, nb_ops);
+
+	for (i = 0; i < num_enq; i++) {
+		if (unlikely(ops[i]->op_type != RTE_COMP_OP_STATELESS)) {
+			ops[i]->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+			ISAL_PMD_LOG(ERR, "Stateful operation not Supported\n");
+			qp->qp_stats.enqueue_err_count++;
+			continue;
+		}
+		retval = process_op(qp, ops[i], ops[i]->private_xform);
+		if (unlikely(retval < 0) ||
+				ops[i]->status != RTE_COMP_OP_STATUS_SUCCESS) {
+			qp->qp_stats.enqueue_err_count++;
+		}
+	}
+
+	retval = rte_ring_enqueue_burst(qp->processed_pkts, (void *)ops,
+			num_enq, NULL);
+	qp->num_free_elements -= retval;
+	qp->qp_stats.enqueued_count += retval;
+
+	return retval;
+}
+
+/* Dequeue burst */
+static uint16_t
+isal_comp_pmd_dequeue_burst(void *queue_pair, struct rte_comp_op **ops,
+		uint16_t nb_ops)
+{
+	struct isal_comp_qp *qp = queue_pair;
+	uint16_t nb_dequeued;
+
+	nb_dequeued = rte_ring_dequeue_burst(qp->processed_pkts, (void **)ops,
+			nb_ops, NULL);
+	qp->num_free_elements += nb_dequeued;
+	qp->qp_stats.dequeued_count += nb_dequeued;
+
+	return nb_dequeued;
+}
+
 /* Create ISA-L compression device */
 static int
 compdev_isal_create(const char *name, struct rte_vdev_device *vdev,
@@ -203,6 +269,10 @@ compdev_isal_create(const char *name, struct rte_vdev_device *vdev,
 
 	dev->dev_ops = isal_compress_pmd_ops;
 
+	/* register rx/tx burst functions for data path */
+	dev->dequeue_burst = isal_comp_pmd_dequeue_burst;
+	dev->enqueue_burst = isal_comp_pmd_enqueue_burst;
+
 	return 0;
 }
 
drivers/compress/isal/isal_compress_pmd_ops.c +2 -0
index b0abf42..4033864 100644
@@ -203,6 +203,8 @@ isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
 		goto qp_setup_cleanup;
 	}
 
+	qp->num_free_elements = rte_ring_free_count(qp->processed_pkts);
+
 	memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
 	return 0;
 
Signature · 1 line
2.7.4
author
#41 lore raw

PATCH v6 7/10 compress/isal: add stats related ops

In reply to Lee Daly

Add functions for statistic retrieval and resetting.

---
 drivers/compress/isal/isal_compress_pmd_ops.c | 33 +++++++++++++++++++++++++--
 1 file changed, 31 insertions(+), 2 deletions(-)
drivers/compress/isal/isal_compress_pmd_ops.c +31 -2
index 4033864..dcf79bf 100644
@@ -93,6 +93,24 @@ isal_comp_pmd_close(struct rte_compressdev *dev)
 	return 0;
 }
 
+/** Get device statistics */
+static void
+isal_comp_pmd_stats_get(struct rte_compressdev *dev,
+		struct rte_compressdev_stats *stats)
+{
+	uint16_t qp_id;
+
+	for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+		struct isal_comp_qp *qp = dev->data->queue_pairs[qp_id];
+
+		stats->enqueued_count += qp->qp_stats.enqueued_count;
+		stats->dequeued_count += qp->qp_stats.dequeued_count;
+
+		stats->enqueue_err_count += qp->qp_stats.enqueue_err_count;
+		stats->dequeue_err_count += qp->qp_stats.dequeue_err_count;
+	}
+}
+
 /** Get device info */
 static void
 isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
@@ -107,6 +125,17 @@ isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
 	}
 }
 
+/** Reset device statistics */
+static void
+isal_comp_pmd_stats_reset(struct rte_compressdev *dev)
+{
+	uint16_t qp_id;
+
+	for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+		struct isal_comp_qp *qp = dev->data->queue_pairs[qp_id];
+		memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
+	}
+}
 
 /** Release queue pair */
 static int
@@ -265,8 +294,8 @@ struct rte_compressdev_ops isal_pmd_ops = {
 		.dev_stop		= isal_comp_pmd_stop,
 		.dev_close		= isal_comp_pmd_close,
 
-		.stats_get		= NULL,
-		.stats_reset		= NULL,
+		.stats_get		= isal_comp_pmd_stats_get,
+		.stats_reset		= isal_comp_pmd_stats_reset,
 
 		.dev_infos_get		= isal_comp_pmd_info_get,
 
Signature · 1 line
2.7.4
author
#42 lore raw

PATCH v6 8/10 compress/isal: add ISA-L compression functionality

In reply to Lee Daly

Adds compression functionality, this sets internal ISA-L structures, provides input & output mbuf addresses, executes compression, which ISA-L calls deflate, and finally error checks.

---
 drivers/compress/isal/isal_compress_pmd.c         | 83 ++++++++++++++++++++++-
 drivers/compress/isal/isal_compress_pmd_ops.c     | 26 +++++++
 drivers/compress/isal/isal_compress_pmd_private.h |  2 +
 3 files changed, 109 insertions(+), 2 deletions(-)
drivers/compress/isal/isal_compress_pmd.c +81 -2
index aaa0593..dbdee39 100644
@@ -6,6 +6,7 @@
 #include <rte_bus_vdev.h>
 #include <rte_common.h>
 #include <rte_malloc.h>
+#include <rte_mbuf.h>
 #include <rte_compressdev_pmd.h>
 
 #include "isal_compress_pmd_private.h"
@@ -187,14 +188,92 @@ isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
 	return 0;
 }
 
+/* Stateless Compression Function */
+static int
+process_isal_deflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
+		struct isal_priv_xform *priv_xform)
+{
+	int ret = 0;
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;
+
+	/* Required due to init clearing level_buf */
+	uint8_t *temp_level_buf = qp->stream->level_buf;
+
+	/* Initialize compression stream */
+	isal_deflate_stateless_init(qp->stream);
+
+	qp->stream->level_buf = temp_level_buf;
+
+	/* Stateless operation, input will be consumed in one go */
+	qp->stream->flush = NO_FLUSH;
+
+	/* set op level & intermediate level buffer */
+	qp->stream->level = priv_xform->compress.level;
+	qp->stream->level_buf_size = priv_xform->level_buffer_size;
+
+	/* Point compression stream structure to input/output buffers */
+	qp->stream->avail_in = op->src.length;
+	qp->stream->next_in = rte_pktmbuf_mtod(op->m_src, uint8_t *);
+	qp->stream->avail_out = op->m_dst->data_len;
+	qp->stream->next_out  = rte_pktmbuf_mtod(op->m_dst, uint8_t *);
+	qp->stream->end_of_stream = 1; /* All input consumed in one go */
+
+	if (unlikely(!qp->stream->next_in || !qp->stream->next_out)) {
+		ISAL_PMD_LOG(ERR, "Invalid source or destination buffers\n");
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		return -1;
+	}
+
+	/* Set op huffman code */
+	if (priv_xform->compress.deflate.huffman == RTE_COMP_HUFFMAN_FIXED)
+		isal_deflate_set_hufftables(qp->stream, NULL,
+				IGZIP_HUFFTABLE_STATIC);
+	else if (priv_xform->compress.deflate.huffman ==
+			RTE_COMP_HUFFMAN_DEFAULT)
+		isal_deflate_set_hufftables(qp->stream, NULL,
+			IGZIP_HUFFTABLE_DEFAULT);
+	/* Dynamically change the huffman code to suit the input data */
+	else if (priv_xform->compress.deflate.huffman ==
+			RTE_COMP_HUFFMAN_DYNAMIC)
+		isal_deflate_set_hufftables(qp->stream, NULL,
+				IGZIP_HUFFTABLE_DEFAULT);
+
+	/* Execute compression operation */
+	ret =  isal_deflate_stateless(qp->stream);
+
+	/* Check that output buffer did not run out of space */
+	if (ret == STATELESS_OVERFLOW) {
+		ISAL_PMD_LOG(ERR, "Output buffer not big enough\n");
+		op->status = RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED;
+		return ret;
+	}
+
+	/* Check that input buffer has been fully consumed */
+	if (qp->stream->avail_in != (uint32_t)0) {
+		ISAL_PMD_LOG(ERR, "Input buffer could not be read entirely\n");
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return -1;
+	}
+
+	if (ret != COMP_OK) {
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return ret;
+	}
+
+	op->consumed = qp->stream->total_in;
+	op->produced = qp->stream->total_out;
+
+	return ret;
+}
+
 /* Process compression/decompression operation */
 static int
-process_op(struct isal_comp_qp *qp __rte_unused,
-		struct rte_comp_op *op __rte_unused,
+process_op(struct isal_comp_qp *qp, struct rte_comp_op *op,
 		struct isal_priv_xform *priv_xform)
 {
 	switch (priv_xform->type) {
 	case RTE_COMP_COMPRESS:
+		process_isal_deflate(op, qp, priv_xform);
 		break;
 	case RTE_COMP_DECOMPRESS:
 		break;
drivers/compress/isal/isal_compress_pmd_ops.c +26 -0
index dcf79bf..77d5d4b 100644
@@ -1,6 +1,7 @@
 /* SPDX-License-Identifier: BSD-3-Clause
  * Copyright(c) 2018 Intel Corporation
  */
+#include <isa-l.h>
 
 #include <rte_common.h>
 #include <rte_compressdev_pmd.h>
@@ -9,6 +10,15 @@
 #include "isal_compress_pmd_private.h"
 
 static const struct rte_compressdev_capabilities isal_pmd_capabilities[] = {
+	{
+		.algo = RTE_COMP_ALGO_DEFLATE,
+		.comp_feature_flags =	RTE_COMP_FF_SHAREABLE_PRIV_XFORM,
+		.window_size = {
+			.min = 15,
+			.max = 15,
+			.increment = 0
+		},
+	},
 	RTE_COMP_END_OF_CAPABILITIES_LIST()
 };
 
@@ -146,6 +156,12 @@ isal_comp_pmd_qp_release(struct rte_compressdev *dev, uint16_t qp_id)
 	if (qp == NULL)
 		return -EINVAL;
 
+	if (qp->stream != NULL)
+		rte_free(qp->stream);
+
+	if (qp->stream->level_buf != NULL)
+		rte_free(qp->stream->level_buf);
+
 	if (dev->data->queue_pairs[qp_id] != NULL)
 		rte_free(dev->data->queue_pairs[qp_id]);
 
@@ -214,6 +230,16 @@ isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
 		return (-ENOMEM);
 	}
 
+	/* Initialize memory for compression stream structure */
+	qp->stream = rte_zmalloc_socket("Isa-l compression stream ",
+			sizeof(struct isal_zstream),  RTE_CACHE_LINE_SIZE,
+			socket_id);
+
+	/* Initialize memory for compression level buffer */
+	qp->stream->level_buf = rte_zmalloc_socket("Isa-l compression lev_buf",
+			ISAL_DEF_LVL3_DEFAULT, RTE_CACHE_LINE_SIZE,
+			socket_id);
+
 	qp->id = qp_id;
 	dev->data->queue_pairs[qp_id] = qp;
 
drivers/compress/isal/isal_compress_pmd_private.h +2 -0
index 7505c76..6ba34a6 100644
@@ -28,6 +28,8 @@ struct isal_comp_qp {
 	struct rte_ring *processed_pkts;
 	/* Queue pair statistics */
 	struct rte_compressdev_stats qp_stats;
+	/* Compression stream information*/
+	struct isal_zstream *stream;
 	/* Number of free elements on ring */
 	uint16_t num_free_elements;
 } __rte_cache_aligned;
Signature · 1 line
2.7.4
Acked-by on v6 8/10
#43 lore raw

Re: [PATCH v6 08/10] compress/isal: add ISA-L compression functionality

Quoted 6 lines

Adds compression functionality, this sets internal ISA-L structures, provides input & output mbuf addresses, executes compression, which ISA-L calls deflate, and finally error checks.

Quoted 90 lines
---
 drivers/compress/isal/isal_compress_pmd.c         | 83 ++++++++++++++++++++++-
 drivers/compress/isal/isal_compress_pmd_ops.c     | 26 +++++++
 drivers/compress/isal/isal_compress_pmd_private.h |  2 +
 3 files changed, 109 insertions(+), 2 deletions(-)
drivers/compress/isal/isal_compress_pmd.c +69 -0
index aaa0593..dbdee39 100644
@@ -6,6 +6,7 @@
 #include <rte_bus_vdev.h>
 #include <rte_common.h>
 #include <rte_malloc.h>
+#include <rte_mbuf.h>
 #include <rte_compressdev_pmd.h>
 
 #include "isal_compress_pmd_private.h"
@@ -187,14 +188,92 @@ isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
 	return 0;
 }
 
+/* Stateless Compression Function */
+static int
+process_isal_deflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
+		struct isal_priv_xform *priv_xform)
+{
+	int ret = 0;
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;
+
+	/* Required due to init clearing level_buf */
+	uint8_t *temp_level_buf = qp->stream->level_buf;
+
+	/* Initialize compression stream */
+	isal_deflate_stateless_init(qp->stream);
+
+	qp->stream->level_buf = temp_level_buf;
+
+	/* Stateless operation, input will be consumed in one go */
+	qp->stream->flush = NO_FLUSH;
+
+	/* set op level & intermediate level buffer */
+	qp->stream->level = priv_xform->compress.level;
+	qp->stream->level_buf_size = priv_xform->level_buffer_size;
+
+	/* Point compression stream structure to input/output buffers */
+	qp->stream->avail_in = op->src.length;
+	qp->stream->next_in = rte_pktmbuf_mtod(op->m_src, uint8_t *);
+	qp->stream->avail_out = op->m_dst->data_len;
+	qp->stream->next_out  = rte_pktmbuf_mtod(op->m_dst, uint8_t *);
+	qp->stream->end_of_stream = 1; /* All input consumed in one go */
+
+	if (unlikely(!qp->stream->next_in || !qp->stream->next_out)) {
+		ISAL_PMD_LOG(ERR, "Invalid source or destination buffers\n");
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		return -1;
+	}
+
+	/* Set op huffman code */
+	if (priv_xform->compress.deflate.huffman == RTE_COMP_HUFFMAN_FIXED)
+		isal_deflate_set_hufftables(qp->stream, NULL,
+				IGZIP_HUFFTABLE_STATIC);
+	else if (priv_xform->compress.deflate.huffman ==
+			RTE_COMP_HUFFMAN_DEFAULT)
+		isal_deflate_set_hufftables(qp->stream, NULL,
+			IGZIP_HUFFTABLE_DEFAULT);
+	/* Dynamically change the huffman code to suit the input data */
+	else if (priv_xform->compress.deflate.huffman ==
+			RTE_COMP_HUFFMAN_DYNAMIC)
+		isal_deflate_set_hufftables(qp->stream, NULL,
+				IGZIP_HUFFTABLE_DEFAULT);
+
+	/* Execute compression operation */
+	ret =  isal_deflate_stateless(qp->stream);
+
+	/* Check that output buffer did not run out of space */
+	if (ret == STATELESS_OVERFLOW) {
+		ISAL_PMD_LOG(ERR, "Output buffer not big enough\n");
+		op->status = RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED;
+		return ret;
+	}
+
+	/* Check that input buffer has been fully consumed */
+	if (qp->stream->avail_in != (uint32_t)0) {
+		ISAL_PMD_LOG(ERR, "Input buffer could not be read entirely\n");
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return -1;
+	}
+
+	if (ret != COMP_OK) {

drivers/compress/isal/isal_compress_pmd.c isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform, · line 268

Quoted 10 lines
drivers/compress/isal/isal_compress_pmd.c +10 -0
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return ret;
+	}
+
+	op->consumed = qp->stream->total_in;
+	op->produced = qp->stream->total_out;
+
+	return ret;
+}
+

--snip--

author
#44 lore raw

PATCH v6 9/10 compress/isal: add ISA-L decomp functionality

In reply to Lee Daly

Adds decompression functionality, similar to compression, this sets internal ISA-L structures, provides input & output mbuf addresses, executes decompression, which ISA-L calls inflate and finally error checks.

---
 drivers/compress/isal/isal_compress_pmd.c         | 52 +++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c     |  8 ++++
 drivers/compress/isal/isal_compress_pmd_private.h |  2 +
 3 files changed, 62 insertions(+)
drivers/compress/isal/isal_compress_pmd.c +52 -0
index dbdee39..9d1d413 100644
@@ -266,6 +266,57 @@ process_isal_deflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
 	return ret;
 }
 
+/* Stateless Decompression Function */
+static int
+process_isal_inflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
+		struct isal_priv_xform *priv_xform)
+{
+	int ret = 0;
+
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;
+
+	/* Initialize decompression state */
+	isal_inflate_init(qp->state);
+
+	/* Point decompression state structure to input/output buffers */
+	qp->state->avail_in = op->src.length;
+	qp->state->next_in = rte_pktmbuf_mtod(op->m_src, uint8_t *);
+	qp->state->avail_out = op->m_dst->data_len;
+	qp->state->next_out  = rte_pktmbuf_mtod(op->m_dst, uint8_t *);
+
+	if (unlikely(!qp->state->next_in || !qp->state->next_out)) {
+		ISAL_PMD_LOG(ERR, "Invalid source or destination buffers\n");
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		return -1;
+	}
+
+	/* Execute decompression operation */
+	ret = isal_inflate_stateless(qp->state);
+
+	if (ret == ISAL_OUT_OVERFLOW) {
+		ISAL_PMD_LOG(ERR, "Output buffer not big enough\n");
+		op->status = RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED;
+		return ret;
+	}
+
+	/* Check that input buffer has been fully consumed */
+	if (qp->state->avail_in != (uint32_t)0) {
+		ISAL_PMD_LOG(ERR, "Input buffer could not be read entirely\n");
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return -1;
+	}
+
+	if (ret != ISAL_DECOMP_OK) {
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return ret;
+	}
+
+	op->consumed = op->src.length - qp->state->avail_in;
+	op->produced = qp->state->total_out;
+
+return ret;
+}
+
 /* Process compression/decompression operation */
 static int
 process_op(struct isal_comp_qp *qp, struct rte_comp_op *op,
@@ -276,6 +327,7 @@ process_op(struct isal_comp_qp *qp, struct rte_comp_op *op,
 		process_isal_deflate(op, qp, priv_xform);
 		break;
 	case RTE_COMP_DECOMPRESS:
+		process_isal_inflate(op, qp, priv_xform);
 		break;
 	default:
 		ISAL_PMD_LOG(ERR, "Operation Not Supported\n");
drivers/compress/isal/isal_compress_pmd_ops.c +8 -0
index 77d5d4b..970a041 100644
@@ -162,6 +162,9 @@ isal_comp_pmd_qp_release(struct rte_compressdev *dev, uint16_t qp_id)
 	if (qp->stream->level_buf != NULL)
 		rte_free(qp->stream->level_buf);
 
+	if (qp->state != NULL)
+		rte_free(qp->state);
+
 	if (dev->data->queue_pairs[qp_id] != NULL)
 		rte_free(dev->data->queue_pairs[qp_id]);
 
@@ -240,6 +243,11 @@ isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
 			ISAL_DEF_LVL3_DEFAULT, RTE_CACHE_LINE_SIZE,
 			socket_id);
 
+	/* Initialize memory for decompression state structure */
+	qp->state = rte_zmalloc_socket("Isa-l decompression state",
+			sizeof(struct inflate_state), RTE_CACHE_LINE_SIZE,
+			socket_id);
+
 	qp->id = qp_id;
 	dev->data->queue_pairs[qp_id] = qp;
 
drivers/compress/isal/isal_compress_pmd_private.h +2 -0
index 6ba34a6..46e9fcf 100644
@@ -30,6 +30,8 @@ struct isal_comp_qp {
 	struct rte_compressdev_stats qp_stats;
 	/* Compression stream information*/
 	struct isal_zstream *stream;
+	/* Decompression state information*/
+	struct inflate_state *state;
 	/* Number of free elements on ring */
 	uint16_t num_free_elements;
 } __rte_cache_aligned;
Signature · 1 line
2.7.4
Acked-by on v6 9/10
#45 lore raw

Re: [PATCH v6 09/10] compress/isal: add ISA-L decomp functionality

Quoted 6 lines

Adds decompression functionality, similar to compression, this sets internal ISA-L structures, provides input & output mbuf addresses, executes decompression, which ISA-L calls inflate and finally error checks.

drivers/compress/isal/isal_compress_pmd.c process_isal_deflate(struct rte_comp_op *op, struct isal_comp_qp *qp, · line 267

Quoted 13 lines
---
 drivers/compress/isal/isal_compress_pmd.c         | 52 +++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c     |  8 ++++
 drivers/compress/isal/isal_compress_pmd_private.h |  2 +
 3 files changed, 62 insertions(+)
drivers/compress/isal/isal_compress_pmd.c +0 -0
index dbdee39..9d1d413 100644
@@ -266,6 +266,57 @@ process_isal_deflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
 	return ret;
 }

--snip--

author
#46 lore raw

PATCH v6 10/10 doc: add compression driver and ISA-L PMD docs

In reply to Lee Daly

This adds general compression drivers feature guide as well as the ISA-L PMD documentation and guide.

---
 .gitignore                                   |  1 +
 MAINTAINERS                                  | 12 +++++
 devtools/test-build.sh                       |  4 ++
 doc/guides/compressdevs/features/default.ini | 24 +++++++++
 doc/guides/compressdevs/features/isal.ini    | 22 +++++++++
 doc/guides/compressdevs/index.rst            | 13 +++++
 doc/guides/compressdevs/isal.rst             | 74 ++++++++++++++++++++++++++++
 doc/guides/compressdevs/overview.rst         | 12 +++++
 doc/guides/conf.py                           |  5 ++
 doc/guides/index.rst                         |  1 +
 doc/guides/rel_notes/release_18_05.rst       |  3 +-
 11 files changed, 170 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/compressdevs/features/default.ini
 create mode 100644 doc/guides/compressdevs/features/isal.ini
 create mode 100644 doc/guides/compressdevs/index.rst
 create mode 100644 doc/guides/compressdevs/isal.rst
 create mode 100644 doc/guides/compressdevs/overview.rst
.gitignore +1 -0
index 6df5ba0..9105e26 100644
@@ -3,6 +3,7 @@ doc/guides/cryptodevs/overview_feature_table.txt
 doc/guides/cryptodevs/overview_cipher_table.txt
 doc/guides/cryptodevs/overview_auth_table.txt
 doc/guides/cryptodevs/overview_aead_table.txt
+doc/guides/compressdevs/overview_feature_table.txt
 cscope.out.po
 cscope.out.in
 cscope.out
MAINTAINERS +12 -0
index d2dd61c..a22f37f 100644
@@ -766,6 +766,18 @@ F: doc/guides/cryptodevs/zuc.rst
 F: doc/guides/cryptodevs/features/zuc.ini
 
 
+Compression Drivers
+-------------------
+M: Pablo de Lara <[email protected]>
+T: git://dpdk.org/next/dpdk-next-crypto
+F: doc/guides/compressdevs/features/default.ini
+
+ISA-L PMD
+M: Lee Daly <[email protected]>
+F: drivers/compress/isal
+F: doc/guides/compressdevs/isal.rst
+F: doc/guides/compressdevs/features/isal.ini
+
 Eventdev Drivers
 ----------------
 M: Jerin Jacob <[email protected]>
devtools/test-build.sh +4 -0
index 3362edc..66f3ece 100755
@@ -45,6 +45,7 @@ default_path=$PATH
 # - DPDK_DEP_SSL (y/[n])
 # - DPDK_DEP_SZE (y/[n])
 # - DPDK_DEP_ZLIB (y/[n])
+# - DPDK_DEP_ISAL (y/[n])
 # - DPDK_MAKE_JOBS (int)
 # - DPDK_NOTIFY (notify-send)
 # - FLEXRAN_SDK
@@ -129,6 +130,7 @@ reset_env ()
 	unset DPDK_DEP_SSL
 	unset DPDK_DEP_SZE
 	unset DPDK_DEP_ZLIB
+	unset DPDK_DEP_ISAL
 	unset AESNI_MULTI_BUFFER_LIB_PATH
 	unset ARMV8_CRYPTO_LIB_PATH
 	unset FLEXRAN_SDK
@@ -178,6 +180,8 @@ config () # <directory> <target> <options>
 		test "$DPDK_DEP_ZLIB" != y || \
 		sed -ri          's,(BNX2X_PMD=)n,\1y,' $1/.config
 		sed -ri            's,(NFP_PMD=)n,\1y,' $1/.config
+		test "$DPDK_DEP_ISAL" != y || \
+		sed -ri          's,(ISAL_PMD=)n,\1y,' $1/.config
 		test "$DPDK_DEP_PCAP" != y || \
 		sed -ri               's,(PCAP=)n,\1y,' $1/.config
 		test -z "$ARMV8_CRYPTO_LIB_PATH" || \
doc/guides/compressdevs/features/default.ini new file +24 -0
new file mode 100644
index 0000000..795fc55
@@ -0,0 +1,24 @@
+;
+; Features of a default compression driver.
+;
+; This file defines the features that are valid for inclusion in
+; the other driver files and also the order that they appear in
+; the features table in the documentation.
+;
+[Features]
+HW Accelerated =
+CPU SSE        =
+CPU AVX        =
+CPU AVX2       =
+CPU AVX512     =
+CPU NEON       =
+Stateful       =
+By-Pass        =
+Chained mbufs  =
+Deflate        =
+LZS            =
+Adler32        =
+Crc32          =
+Adler32&Crc32  =
+Fixed          =
+Dynamic        =
doc/guides/compressdevs/features/isal.ini new file +22 -0
new file mode 100644
index 0000000..ad2718d
@@ -0,0 +1,22 @@
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; Supported features of 'ISA-L' compression driver.
+;
+[Features]
+HW Accelerated =
+CPU SSE        = Y
+CPU AVX        = Y
+CPU AVX2       = Y
+CPU AVX512     = Y
+CPU NEON       =
+Stateful       =
+By-Pass        =
+Chained mbufs  =
+Deflate        = Y
+LZS            =
+Adler32        =
+Crc32          =
+Adler32&Crc32  =
+Fixed          = Y
+Dynamic        = Y
doc/guides/compressdevs/index.rst new file +13 -0
new file mode 100644
index 0000000..bc59ce8
@@ -0,0 +1,13 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Compression Device Drivers
+==========================
+
+
+.. toctree::
+    :maxdepth: 2
+    :numbered:
+
+    overview
+    isal
doc/guides/compressdevs/isal.rst new file +74 -0

74 lines not on this page · raw message

doc/guides/compressdevs/overview.rst new file +12 -0
new file mode 100644
index 0000000..ca37de1
@@ -0,0 +1,12 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Compression Device Supported Functionality Matrices
+===================================================
+
+Supported Feature Flags
+-----------------------
+
+.. _table_compression_pmd_features:
+
+.. include:: overview_feature_table.txt
doc/guides/conf.py +5 -0
index 38de280..c883306 100644
@@ -388,6 +388,11 @@ def setup(app):
                             'AEAD',
                             'AEAD algorithms in crypto drivers',
                             'AEAD algorithm')
+    table_file = dirname(__file__) + '/compressdevs/overview_feature_table.txt'
+    generate_overview_table(table_file, 1,
+                            'Features',
+                            'Features availability in compression drivers',
+                            'Feature')
 
     if LooseVersion(sphinx_version) < LooseVersion('1.3.1'):
         print('Upgrade sphinx to version >= 1.3.1 for '
doc/guides/index.rst +1 -0
index d60529d..18fe0ec 100644
@@ -17,6 +17,7 @@ DPDK documentation
    nics/index
    bbdevs/index
    cryptodevs/index
+   compressdevs/index
    eventdevs/index
    mempool/index
    platform/index
doc/guides/rel_notes/release_18_05.rst +2 -1
index ce70d1f..d6a9d6f 100644
@@ -78,7 +78,8 @@ New Features
 * **Added a new compression poll mode driver using Intels ISA-L.**
 
    Added the new ``ISA-L`` compression driver, for compression and decompression
-   operations in software.
+   operations in software. See the :doc:`../compressdevs/isal` compression driver
+   guide for details on this new driver.
 
 
 API Changes
Signature · 1 line
2.7.4
#47 lore raw

Re: [PATCH v6 00/10] add ISA-L compression PMD

In reply to Lee Daly
Quoted 11 lines

-----Original Message-----
From: Daly, Lee
Sent: Wednesday, May 9, 2018 5:14 PM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v6 00/10] add ISA-L compression PMD

#48 lore raw

Re: [PATCH v6 00/10] add ISA-L compression PMD

Quoted 23 lines

-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of De Lara Guarch, Pablo
Sent: Wednesday, May 9, 2018 9:57 PM
To: Daly, Lee <[email protected]>; [email protected]
Cc: Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v6 00/10] add ISA-L compression PMD

Quoted 10 lines

-----Original Message-----
From: Daly, Lee
Sent: Wednesday, May 9, 2018 5:14 PM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v6 00/10] add ISA-L compression PMD

Quoted 1 line

Series-applied-to dpdk-next-crypto.
Thanks for the work!

Pablo

author
#49 lore raw

PATCH v5 2/10 compress/isal: add pmd device init and de-init

In reply to Lee Daly

This patch adds device initialization functionality such as probe and create, also adding deinitialize functionality.
Dynamic logging component also added.

---
 drivers/compress/isal/Makefile                    |  1 +
 drivers/compress/isal/isal_compress_pmd.c         | 74 +++++++++++++++++++++--
 drivers/compress/isal/isal_compress_pmd_ops.c     | 25 ++++++++
 drivers/compress/isal/isal_compress_pmd_private.h | 24 ++++++++
 drivers/compress/isal/meson.build                 |  2 +-
 5 files changed, 121 insertions(+), 5 deletions(-)
 create mode 100644 drivers/compress/isal/isal_compress_pmd_ops.c
 create mode 100644 drivers/compress/isal/isal_compress_pmd_private.h
drivers/compress/isal/Makefile +1 -0
index 9b1d866..95904f6 100644
@@ -25,6 +25,7 @@ EXPORT_MAP := rte_pmd_isal_version.map
 
 # library source files
 SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal_compress_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal_compress_pmd_ops.c
 
 # export include files
 include $(RTE_SDK)/mk/rte.lib.mk
drivers/compress/isal/isal_compress_pmd.c +70 -4

93 lines not on this page · raw message

drivers/compress/isal/isal_compress_pmd_ops.c new file +25 -0
new file mode 100644
index 0000000..cff05b4
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include <rte_compressdev_pmd.h>
+
+struct rte_compressdev_ops isal_pmd_ops = {
+		.dev_configure		= NULL,
+		.dev_start		= NULL,
+		.dev_stop		= NULL,
+		.dev_close		= NULL,
+
+		.stats_get		= NULL,
+		.stats_reset		= NULL,
+
+		.dev_infos_get		= NULL,
+
+		.queue_pair_setup	= NULL,
+		.queue_pair_release	= NULL,
+
+		.private_xform_create	= NULL,
+		.private_xform_free	= NULL,
+};
+
+struct rte_compressdev_ops *isal_compress_pmd_ops = &isal_pmd_ops;
drivers/compress/isal/isal_compress_pmd_private.h new file +24 -0
new file mode 100644
index 0000000..09ecfb7
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#ifndef _ISAL_COMP_PMD_PRIVATE_H_
+#define _ISAL_COMP_PMD_PRIVATE_H_
+
+#define COMPDEV_NAME_ISAL_PMD		compress_isal
+/**< ISA-L comp PMD device name */
+
+extern int isal_logtype_driver;
+#define ISAL_PMD_LOG(level, fmt, args...) \
+	rte_log(RTE_LOG_ ## level, isal_logtype_driver, "%s(): "fmt "\n", \
+			__func__, ##args)
+
+/* private data structure for each ISA-L compression device */
+struct isal_comp_private {
+	struct rte_mempool *priv_xform_mp;
+};
+
+/** device specific operations function pointer structure */
+extern struct rte_compressdev_ops *isal_compress_pmd_ops;
+
+#endif /* _ISAL_COMP_PMD_PRIVATE_H_ */
drivers/compress/isal/meson.build +1 -1
index 4447e20..94c10fd 100644
@@ -7,7 +7,7 @@ if not dep.found()
 endif
 
 deps += 'bus_vdev'
-sources = files('isal_compress_pmd.c')
+sources = files('isal_compress_pmd.c', 'isal_compress_pmd_ops.c')
 ext_deps += dep
 pkgconfig_extra_libs += '-lisal'
 
Signature · 1 line
2.7.4
author
#50 lore raw

PATCH v5 3/10 compress/isal: add basic pmd ops

In reply to Lee Daly

Add basic device poll mode driver operations to give ability to start, stop, close device etc.

---
 drivers/compress/isal/isal_compress_pmd_ops.c     | 63 +++++++++++++++++++++--
 drivers/compress/isal/isal_compress_pmd_private.h | 12 +++++
 2 files changed, 70 insertions(+), 5 deletions(-)
drivers/compress/isal/isal_compress_pmd_ops.c +58 -5
index cff05b4..2f828c5 100644
@@ -2,18 +2,71 @@
  * Copyright(c) 2018 Intel Corporation
  */
 
+#include <rte_common.h>
 #include <rte_compressdev_pmd.h>
 
+#include "isal_compress_pmd_private.h"
+
+static const struct rte_compressdev_capabilities isal_pmd_capabilities[] = {
+	RTE_COMP_END_OF_CAPABILITIES_LIST()
+};
+
+/** Configure device */
+static int
+isal_comp_pmd_config(struct rte_compressdev *dev __rte_unused,
+		struct rte_compressdev_config *config __rte_unused)
+{
+	return 0;
+}
+
+/** Start device */
+static int
+isal_comp_pmd_start(__rte_unused struct rte_compressdev *dev)
+{
+	return 0;
+}
+
+/** Stop device */
+static void
+isal_comp_pmd_stop(__rte_unused struct rte_compressdev *dev)
+{
+}
+
+/** Close device */
+static int
+isal_comp_pmd_close(struct rte_compressdev *dev)
+{
+	/* Free private data */
+	struct isal_comp_private *internals = dev->data->dev_private;
+
+	rte_mempool_free(internals->priv_xform_mp);
+	return 0;
+}
+
+/** Get device info */
+static void
+isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
+		struct rte_compressdev_info *dev_info)
+{
+	if (dev_info != NULL) {
+		dev_info->capabilities = isal_pmd_capabilities;
+		dev_info->feature_flags = RTE_COMPDEV_FF_CPU_AVX512 |
+				RTE_COMPDEV_FF_CPU_AVX2 |
+				RTE_COMPDEV_FF_CPU_AVX |
+				RTE_COMPDEV_FF_CPU_SSE;
+	}
+}
+
 struct rte_compressdev_ops isal_pmd_ops = {
-		.dev_configure		= NULL,
-		.dev_start		= NULL,
-		.dev_stop		= NULL,
-		.dev_close		= NULL,
+		.dev_configure		= isal_comp_pmd_config,
+		.dev_start		= isal_comp_pmd_start,
+		.dev_stop		= isal_comp_pmd_stop,
+		.dev_close		= isal_comp_pmd_close,
 
 		.stats_get		= NULL,
 		.stats_reset		= NULL,
 
-		.dev_infos_get		= NULL,
+		.dev_infos_get		= isal_comp_pmd_info_get,
 
 		.queue_pair_setup	= NULL,
 		.queue_pair_release	= NULL,
drivers/compress/isal/isal_compress_pmd_private.h +12 -0
index 09ecfb7..efbe68b 100644
@@ -18,6 +18,18 @@ struct isal_comp_private {
 	struct rte_mempool *priv_xform_mp;
 };
 
+/** ISA-L queue pair */
+struct isal_comp_qp {
+	/* Queue Pair Identifier */
+	uint16_t id;
+	/* Unique Queue Pair Name */
+	char name[RTE_COMPRESSDEV_NAME_MAX_LEN];
+	/* Queue pair statistics */
+	struct rte_compressdev_stats qp_stats;
+	/* Number of free elements on ring */
+	uint16_t num_free_elements;
+} __rte_cache_aligned;
+
 /** device specific operations function pointer structure */
 extern struct rte_compressdev_ops *isal_compress_pmd_ops;
 
Signature · 1 line
2.7.4
author
#51 lore raw

PATCH v5 4/10 compress/isal: add private xform related ops

In reply to Lee Daly

This patch creates, configures and frees the private xform, taking the applications xform and using it to populate the PMDs own private xform with the information which will be required for the compress/decompress functions. This information includes the level, algorithm, type of huffman code, type of checksum etc.

---
 drivers/compress/isal/isal_compress_pmd.c         | 196 ++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c     | 103 +++++++++++-
 drivers/compress/isal/isal_compress_pmd_private.h |  15 ++
 3 files changed, 309 insertions(+), 5 deletions(-)
drivers/compress/isal/isal_compress_pmd.c +196 -0
index 5cc9409..3f3afda 100644
@@ -1,6 +1,7 @@
 /* SPDX-License-Identifier: BSD-3-Clause
  * Copyright(c) 2018 Intel Corporation
  */
+#include <isa-l.h>
 
 #include <rte_bus_vdev.h>
 #include <rte_common.h>
@@ -9,8 +10,203 @@
 
 #include "isal_compress_pmd_private.h"
 
+#define RTE_COMP_ISAL_WINDOW_SIZE 15
+#define RTE_COMP_ISAL_LEVEL_ZERO 0 /* ISA-L Level 0 used for fixed Huffman */
+#define RTE_COMP_ISAL_LEVEL_ONE 1
+#define RTE_COMP_ISAL_LEVEL_TWO 2
+#define RTE_COMP_ISAL_LEVEL_THREE 3 /* Optimised for AVX512 & AVX2 only */
+
 int isal_logtype_driver;
 
+/* Verify and set private xform parameters */
+int
+isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
+		const struct rte_comp_xform *xform)
+{
+	if (xform == NULL)
+		return -EINVAL;
+
+	/* Set compression private xform variables */
+	if (xform->type == RTE_COMP_COMPRESS) {
+		/* Set private xform type - COMPRESS/DECOMPRESS */
+		priv_xform->type = RTE_COMP_COMPRESS;
+
+		/* Set private xform algorithm */
+		if (xform->compress.algo != RTE_COMP_ALGO_DEFLATE) {
+			if (xform->compress.algo == RTE_COMP_ALGO_NULL) {
+				ISAL_PMD_LOG(ERR, "By-pass not supported\n");
+				return -ENOTSUP;
+			}
+			ISAL_PMD_LOG(ERR, "Algorithm not supported\n");
+			return -ENOTSUP;
+		}
+		priv_xform->compress.algo = RTE_COMP_ALGO_DEFLATE;
+
+		/* Set private xform checksum */
+		switch (xform->compress.chksum) {
+		case(RTE_COMP_CHECKSUM_NONE):
+			priv_xform->compress.chksum = RTE_COMP_CHECKSUM_NONE;
+			break;
+		case(RTE_COMP_CHECKSUM_ADLER32):
+			priv_xform->compress.chksum = IGZIP_ZLIB;
+			break;
+		case(RTE_COMP_CHECKSUM_CRC32):
+			priv_xform->compress.chksum = IGZIP_GZIP;
+			break;
+		default:
+			ISAL_PMD_LOG(ERR, "Checksum not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform window size, 32K supported */
+		if (xform->compress.window_size == RTE_COMP_ISAL_WINDOW_SIZE)
+			priv_xform->compress.window_size =
+					RTE_COMP_ISAL_WINDOW_SIZE;
+		else {
+			ISAL_PMD_LOG(ERR, "Window size not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform huffman type */
+		switch (xform->compress.deflate.huffman) {
+		case(RTE_COMP_HUFFMAN_DEFAULT):
+			priv_xform->compress.deflate.huffman =
+					RTE_COMP_HUFFMAN_DEFAULT;
+			break;
+		case(RTE_COMP_HUFFMAN_FIXED):
+			priv_xform->compress.deflate.huffman =
+					RTE_COMP_HUFFMAN_FIXED;
+			break;
+		case(RTE_COMP_HUFFMAN_DYNAMIC):
+			priv_xform->compress.deflate.huffman =
+					RTE_COMP_HUFFMAN_DYNAMIC;
+			break;
+		default:
+			ISAL_PMD_LOG(ERR, "Huffman code not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform level.
+		 * Checking compliance with compressdev API, -1 <= level => 9
+		 */
+		if (xform->compress.level < RTE_COMP_LEVEL_PMD_DEFAULT ||
+				xform->compress.level > RTE_COMP_LEVEL_MAX) {
+			ISAL_PMD_LOG(ERR, "Compression level out of range\n");
+			return -EINVAL;
+		}
+		/* Check for Compressdev API level 0, No compression
+		 * not supported in ISA-L
+		 */
+		else if (xform->compress.level == RTE_COMP_LEVEL_NONE) {
+			ISAL_PMD_LOG(ERR, "No Compression not supported\n");
+			return -ENOTSUP;
+		}
+		/* If using fixed huffman code, level must be 0 */
+		else if (priv_xform->compress.deflate.huffman ==
+				RTE_COMP_HUFFMAN_FIXED) {
+			ISAL_PMD_LOG(DEBUG, "ISA-L level 0 used due to a"
+					" fixed huffman code\n");
+			priv_xform->compress.level = RTE_COMP_ISAL_LEVEL_ZERO;
+			priv_xform->level_buffer_size =
+					ISAL_DEF_LVL0_DEFAULT;
+		} else {
+			/* Mapping API levels to ISA-L levels 1,2 & 3 */
+			switch (xform->compress.level) {
+			case RTE_COMP_LEVEL_PMD_DEFAULT:
+				/* Default is 1 if not using fixed huffman */
+				priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_ONE;
+				priv_xform->level_buffer_size =
+						ISAL_DEF_LVL1_DEFAULT;
+				break;
+			case RTE_COMP_LEVEL_MIN:
+				priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_ONE;
+				priv_xform->level_buffer_size =
+						ISAL_DEF_LVL1_DEFAULT;
+				break;
+			case RTE_COMP_ISAL_LEVEL_TWO:
+				priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_TWO;
+				priv_xform->level_buffer_size =
+						ISAL_DEF_LVL2_DEFAULT;
+				break;
+			/* Level 3 or higher requested */
+			default:
+				/* Check for AVX512, to use ISA-L level 3 */
+				if (rte_cpu_get_flag_enabled(
+						RTE_CPUFLAG_AVX512F)) {
+					priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_THREE;
+					priv_xform->level_buffer_size =
+						ISAL_DEF_LVL3_DEFAULT;
+				}
+				/* Check for AVX2, to use ISA-L level 3 */
+				else if (rte_cpu_get_flag_enabled(
+						RTE_CPUFLAG_AVX2)) {
+					priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_THREE;
+					priv_xform->level_buffer_size =
+						ISAL_DEF_LVL3_DEFAULT;
+				} else {
+					ISAL_PMD_LOG(DEBUG, "Requested ISA-L level"
+						" 3 or above; Level 3 optimized"
+						" for AVX512 & AVX2 only."
+						" level changed to 2.\n");
+					priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_TWO;
+					priv_xform->level_buffer_size =
+						ISAL_DEF_LVL2_DEFAULT;
+				}
+			}
+		}
+	}
+
+	/* Set decompression private xform variables */
+	else if (xform->type == RTE_COMP_DECOMPRESS) {
+
+		/* Set private xform type - COMPRESS/DECOMPRESS */
+		priv_xform->type = RTE_COMP_DECOMPRESS;
+
+		/* Set private xform algorithm */
+		if (xform->decompress.algo != RTE_COMP_ALGO_DEFLATE) {
+			if (xform->decompress.algo == RTE_COMP_ALGO_NULL) {
+				ISAL_PMD_LOG(ERR, "By pass not supported\n");
+				return -ENOTSUP;
+			}
+			ISAL_PMD_LOG(ERR, "Algorithm not supported\n");
+			return -ENOTSUP;
+		}
+		priv_xform->decompress.algo = RTE_COMP_ALGO_DEFLATE;
+
+		/* Set private xform checksum */
+		switch (xform->decompress.chksum) {
+		case(RTE_COMP_CHECKSUM_NONE):
+			priv_xform->decompress.chksum = RTE_COMP_CHECKSUM_NONE;
+			break;
+		case(RTE_COMP_CHECKSUM_ADLER32):
+			priv_xform->decompress.chksum = ISAL_ZLIB;
+			break;
+		case(RTE_COMP_CHECKSUM_CRC32):
+			priv_xform->decompress.chksum = ISAL_GZIP;
+			break;
+		default:
+			ISAL_PMD_LOG(ERR, "Checksum not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform window size, 32K supported */
+		if (xform->decompress.window_size == RTE_COMP_ISAL_WINDOW_SIZE)
+			priv_xform->decompress.window_size =
+					RTE_COMP_ISAL_WINDOW_SIZE;
+		else {
+			ISAL_PMD_LOG(ERR, "Window size not supported\n");
+			return -ENOTSUP;
+		}
+	}
+	return 0;
+}
+
 /* Create ISA-L compression device */
 static int
 compdev_isal_create(const char *name, struct rte_vdev_device *vdev,
drivers/compress/isal/isal_compress_pmd_ops.c +98 -5

122 lines not on this page · raw message

drivers/compress/isal/isal_compress_pmd_private.h +15 -0
index efbe68b..7e3b840 100644
@@ -30,6 +30,21 @@ struct isal_comp_qp {
 	uint16_t num_free_elements;
 } __rte_cache_aligned;
 
+/** ISA-L private xform structure */
+struct isal_priv_xform {
+	enum rte_comp_xform_type type;
+	union {
+		struct rte_comp_compress_xform compress;
+		struct rte_comp_decompress_xform decompress;
+	};
+	uint32_t level_buffer_size;
+} __rte_cache_aligned;
+
+/** Set and validate NULL comp private xform parameters */
+extern int
+isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
+			const struct rte_comp_xform *xform);
+
 /** device specific operations function pointer structure */
 extern struct rte_compressdev_ops *isal_compress_pmd_ops;
 
Signature · 1 line
2.7.4
author
#52 lore raw

PATCH v5 5/10 compress/isal: add queue pair related ops

In reply to Lee Daly

This adds queue pair operations such as setup and release.

---
 drivers/compress/isal/isal_compress_pmd_ops.c     | 111 +++++++++++++++++++++-
 drivers/compress/isal/isal_compress_pmd_private.h |   2 +
 2 files changed, 111 insertions(+), 2 deletions(-)
drivers/compress/isal/isal_compress_pmd_ops.c +109 -2
index 9c89df0..b0abf42 100644
@@ -4,6 +4,7 @@
 
 #include <rte_common.h>
 #include <rte_compressdev_pmd.h>
+#include <rte_malloc.h>
 
 #include "isal_compress_pmd_private.h"
 
@@ -106,6 +107,112 @@ isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
 	}
 }
 
+
+/** Release queue pair */
+static int
+isal_comp_pmd_qp_release(struct rte_compressdev *dev, uint16_t qp_id)
+{
+	struct isal_comp_qp *qp = dev->data->queue_pairs[qp_id];
+
+	if (qp == NULL)
+		return -EINVAL;
+
+	if (dev->data->queue_pairs[qp_id] != NULL)
+		rte_free(dev->data->queue_pairs[qp_id]);
+
+	return 0;
+}
+
+/** Create a ring to place process packets on */
+static struct rte_ring *
+isal_comp_pmd_qp_create_processed_pkts_ring(struct isal_comp_qp *qp,
+		unsigned int ring_size, int socket_id)
+{
+	struct rte_ring *r;
+
+	r = rte_ring_lookup(qp->name);
+	if (r) {
+		if (rte_ring_get_size(r) >= ring_size) {
+			ISAL_PMD_LOG(DEBUG,
+				"Reusing existing ring %s for processed packets",
+				qp->name);
+			return r;
+		}
+
+			ISAL_PMD_LOG(ERR,
+				"Unable to reuse existing ring %s"
+				" for processed packets",
+			 qp->name);
+		return NULL;
+	}
+
+	return rte_ring_create(qp->name, ring_size, socket_id,
+			RING_F_SP_ENQ | RING_F_SC_DEQ);
+}
+
+/** set a unique name for the queue pair based on its name, dev_id and qp_id */
+static int
+isal_comp_pmd_qp_set_unique_name(struct rte_compressdev *dev,
+struct isal_comp_qp *qp)
+{
+	unsigned int n = snprintf(qp->name, sizeof(qp->name),
+			"isal_compression_pmd_%u_qp_%u",
+			dev->data->dev_id, qp->id);
+
+	if (n >= sizeof(qp->name))
+		return -1;
+
+	return 0;
+}
+
+/* Setup a queue pair */
+static int
+isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
+		uint32_t max_inflight_ops, int socket_id)
+{
+	struct isal_comp_qp *qp = NULL;
+	int retval;
+
+	/* Free memory prior to re-allocation if needed. */
+	if (dev->data->queue_pairs[qp_id] != NULL)
+		isal_comp_pmd_qp_release(dev, qp_id);
+
+	/* Allocate the queue pair data structure. */
+	qp = rte_zmalloc_socket("Isa-l compression PMD Queue Pair", sizeof(*qp),
+					RTE_CACHE_LINE_SIZE, socket_id);
+	if (qp == NULL) {
+		ISAL_PMD_LOG(ERR, "Failed to allocate queue pair memory");
+		return (-ENOMEM);
+	}
+
+	qp->id = qp_id;
+	dev->data->queue_pairs[qp_id] = qp;
+
+	retval = isal_comp_pmd_qp_set_unique_name(dev, qp);
+	if (retval) {
+		ISAL_PMD_LOG(ERR, "Failed to create unique name for isal "
+				"compression device");
+		goto qp_setup_cleanup;
+	}
+
+	qp->processed_pkts = isal_comp_pmd_qp_create_processed_pkts_ring(qp,
+			max_inflight_ops, socket_id);
+	if (qp->processed_pkts == NULL) {
+		ISAL_PMD_LOG(ERR, "Failed to create unique name for isal "
+				"compression device");
+		goto qp_setup_cleanup;
+	}
+
+	memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
+	return 0;
+
+qp_setup_cleanup:
+	if (qp)
+		rte_free(qp);
+
+	return -1;
+}
+
 /** Set private xform data*/
 static int
 isal_comp_pmd_priv_xform_create(struct rte_compressdev *dev,
@@ -161,8 +268,8 @@ struct rte_compressdev_ops isal_pmd_ops = {
 
 		.dev_infos_get		= isal_comp_pmd_info_get,
 
-		.queue_pair_setup	= NULL,
-		.queue_pair_release	= NULL,
+		.queue_pair_setup	= isal_comp_pmd_qp_setup,
+		.queue_pair_release	= isal_comp_pmd_qp_release,
 
 		.private_xform_create	= isal_comp_pmd_priv_xform_create,
 		.private_xform_free	= isal_comp_pmd_priv_xform_free,
drivers/compress/isal/isal_compress_pmd_private.h +2 -0
index 7e3b840..7505c76 100644
@@ -24,6 +24,8 @@ struct isal_comp_qp {
 	uint16_t id;
 	/* Unique Queue Pair Name */
 	char name[RTE_COMPRESSDEV_NAME_MAX_LEN];
+	/* Ring for placing process packets */
+	struct rte_ring *processed_pkts;
 	/* Queue pair statistics */
 	struct rte_compressdev_stats qp_stats;
 	/* Number of free elements on ring */
Signature · 1 line
2.7.4
author
#53 lore raw

PATCH v5 6/10 compress/isal: support enqueue/dequeue api

In reply to Lee Daly

This patchs adds support for the compressdev enqueue_burst and dequeue_burst API operations.

---
 drivers/compress/isal/isal_compress_pmd.c     | 70 +++++++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c |  2 +
 2 files changed, 72 insertions(+)
drivers/compress/isal/isal_compress_pmd.c +70 -0
index 3f3afda..f43b3d9 100644
@@ -207,6 +207,72 @@ isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
 	return 0;
 }
 
+/* Process compression/decompression operation */
+static int
+process_op(struct isal_comp_qp *qp __rte_unused,
+		struct rte_comp_op *op __rte_unused,
+		struct isal_priv_xform *priv_xform)
+{
+	switch (priv_xform->type) {
+	case RTE_COMP_COMPRESS:
+		break;
+	case RTE_COMP_DECOMPRESS:
+		break;
+	default:
+		ISAL_PMD_LOG(ERR, "Operation Not Supported\n");
+		return -ENOTSUP;
+	}
+	return 0;
+}
+
+/* Enqueue burst */
+static uint16_t
+isal_comp_pmd_enqueue_burst(void *queue_pair, struct rte_comp_op **ops,
+			uint16_t nb_ops)
+{
+	struct isal_comp_qp *qp = queue_pair;
+	uint16_t i;
+	int retval;
+	int16_t num_enq = RTE_MIN(qp->num_free_elements, nb_ops);
+
+	for (i = 0; i < num_enq; i++) {
+		if (unlikely(ops[i]->op_type != RTE_COMP_OP_STATELESS)) {
+			ops[i]->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+			ISAL_PMD_LOG(ERR, "Stateful operation not Supported\n");
+			qp->qp_stats.enqueue_err_count++;
+			continue;
+		}
+		retval = process_op(qp, ops[i], ops[i]->private_xform);
+		if (unlikely(retval < 0) ||
+				ops[i]->status != RTE_COMP_OP_STATUS_SUCCESS) {
+			qp->qp_stats.enqueue_err_count++;
+		}
+	}
+
+	retval = rte_ring_enqueue_burst(qp->processed_pkts, (void *)ops,
+			num_enq, NULL);
+	qp->num_free_elements -= retval;
+	qp->qp_stats.enqueued_count += retval;
+
+	return retval;
+}
+
+/* Dequeue burst */
+static uint16_t
+isal_comp_pmd_dequeue_burst(void *queue_pair, struct rte_comp_op **ops,
+		uint16_t nb_ops)
+{
+	struct isal_comp_qp *qp = queue_pair;
+	uint16_t nb_dequeued;
+
+	nb_dequeued = rte_ring_dequeue_burst(qp->processed_pkts, (void **)ops,
+			nb_ops, NULL);
+	qp->num_free_elements += nb_dequeued;
+	qp->qp_stats.dequeued_count += nb_dequeued;
+
+	return nb_dequeued;
+}
+
 /* Create ISA-L compression device */
 static int
 compdev_isal_create(const char *name, struct rte_vdev_device *vdev,
@@ -223,6 +289,10 @@ compdev_isal_create(const char *name, struct rte_vdev_device *vdev,
 
 	dev->dev_ops = isal_compress_pmd_ops;
 
+	/* register rx/tx burst functions for data path */
+	dev->dequeue_burst = isal_comp_pmd_dequeue_burst;
+	dev->enqueue_burst = isal_comp_pmd_enqueue_burst;
+
 	return 0;
 }
 
drivers/compress/isal/isal_compress_pmd_ops.c +2 -0
index b0abf42..4033864 100644
@@ -203,6 +203,8 @@ isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
 		goto qp_setup_cleanup;
 	}
 
+	qp->num_free_elements = rte_ring_free_count(qp->processed_pkts);
+
 	memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
 	return 0;
 
Signature · 1 line
2.7.4
author
#54 lore raw

PATCH v5 7/10 compress/isal: add stats related ops

In reply to Lee Daly

Add functions for statistic retrieval and resetting.

---
 drivers/compress/isal/isal_compress_pmd_ops.c | 33 +++++++++++++++++++++++++--
 1 file changed, 31 insertions(+), 2 deletions(-)
drivers/compress/isal/isal_compress_pmd_ops.c +31 -2
index 4033864..dcf79bf 100644
@@ -93,6 +93,24 @@ isal_comp_pmd_close(struct rte_compressdev *dev)
 	return 0;
 }
 
+/** Get device statistics */
+static void
+isal_comp_pmd_stats_get(struct rte_compressdev *dev,
+		struct rte_compressdev_stats *stats)
+{
+	uint16_t qp_id;
+
+	for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+		struct isal_comp_qp *qp = dev->data->queue_pairs[qp_id];
+
+		stats->enqueued_count += qp->qp_stats.enqueued_count;
+		stats->dequeued_count += qp->qp_stats.dequeued_count;
+
+		stats->enqueue_err_count += qp->qp_stats.enqueue_err_count;
+		stats->dequeue_err_count += qp->qp_stats.dequeue_err_count;
+	}
+}
+
 /** Get device info */
 static void
 isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
@@ -107,6 +125,17 @@ isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
 	}
 }
 
+/** Reset device statistics */
+static void
+isal_comp_pmd_stats_reset(struct rte_compressdev *dev)
+{
+	uint16_t qp_id;
+
+	for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+		struct isal_comp_qp *qp = dev->data->queue_pairs[qp_id];
+		memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
+	}
+}
 
 /** Release queue pair */
 static int
@@ -265,8 +294,8 @@ struct rte_compressdev_ops isal_pmd_ops = {
 		.dev_stop		= isal_comp_pmd_stop,
 		.dev_close		= isal_comp_pmd_close,
 
-		.stats_get		= NULL,
-		.stats_reset		= NULL,
+		.stats_get		= isal_comp_pmd_stats_get,
+		.stats_reset		= isal_comp_pmd_stats_reset,
 
 		.dev_infos_get		= isal_comp_pmd_info_get,
 
Signature · 1 line
2.7.4
author
#55 lore raw

PATCH v5 8/10 compress/isal: add ISA-L compression functionality

In reply to Lee Daly

Adds compression functionality, this sets internal ISA-L structures, provides input & output mbuf addresses, executes compression, which ISA-L calls deflate, and finally error checks.

---
 drivers/compress/isal/isal_compress_pmd.c         | 86 ++++++++++++++++++++++-
 drivers/compress/isal/isal_compress_pmd_ops.c     | 28 ++++++++
 drivers/compress/isal/isal_compress_pmd_private.h |  2 +
 3 files changed, 114 insertions(+), 2 deletions(-)
drivers/compress/isal/isal_compress_pmd.c +84 -2
index f43b3d9..6b6d12c 100644
@@ -6,6 +6,7 @@
 #include <rte_bus_vdev.h>
 #include <rte_common.h>
 #include <rte_malloc.h>
+#include <rte_mbuf.h>
 #include <rte_compressdev_pmd.h>
 
 #include "isal_compress_pmd_private.h"
@@ -207,14 +208,95 @@ isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
 	return 0;
 }
 
+/* Stateless Compression Function */
+static int
+process_isal_deflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
+		struct isal_priv_xform *priv_xform)
+{
+	int ret = 0;
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;
+
+	/* Required due to init clearing level_buf */
+	uint8_t *temp_level_buf = qp->stream->level_buf;
+
+	/* Initialize compression stream */
+	isal_deflate_stateless_init(qp->stream);
+
+	qp->stream->level_buf = temp_level_buf;
+
+	/* Stateless operation, input will be consumed in one go */
+	qp->stream->flush = NO_FLUSH;
+
+	/* Set op checksum, none by default */
+	qp->stream->gzip_flag = priv_xform->compress.chksum;
+
+	/* set op level & intermediate level buffer */
+	qp->stream->level = priv_xform->compress.level;
+	qp->stream->level_buf_size = priv_xform->level_buffer_size;
+
+	/* Point compression stream structure to input/output buffers */
+	qp->stream->avail_in = op->src.length;
+	qp->stream->next_in = rte_pktmbuf_mtod(op->m_src, uint8_t *);
+	qp->stream->avail_out = op->m_dst->data_len;
+	qp->stream->next_out  = rte_pktmbuf_mtod(op->m_dst, uint8_t *);
+	qp->stream->end_of_stream = 1; /* All input consumed in one go */
+
+	if (unlikely(!qp->stream->next_in || !qp->stream->next_out)) {
+		ISAL_PMD_LOG(ERR, "Invalid source or destination buffers\n");
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		return -1;
+	}
+
+	/* Set op huffman code */
+	if (priv_xform->compress.deflate.huffman == RTE_COMP_HUFFMAN_FIXED)
+		isal_deflate_set_hufftables(qp->stream, NULL,
+				IGZIP_HUFFTABLE_STATIC);
+	else if (priv_xform->compress.deflate.huffman ==
+			RTE_COMP_HUFFMAN_DEFAULT)
+		isal_deflate_set_hufftables(qp->stream, NULL,
+			IGZIP_HUFFTABLE_DEFAULT);
+	/* Dynamically change the huffman code to suit the input data */
+	else if (priv_xform->compress.deflate.huffman ==
+			RTE_COMP_HUFFMAN_DYNAMIC)
+		isal_deflate_set_hufftables(qp->stream, NULL,
+				IGZIP_HUFFTABLE_CUSTOM);
+
+	/* Execute compression operation */
+	ret =  isal_deflate_stateless(qp->stream);
+
+	/* Check that output buffer did not run out of space */
+	if (ret == STATELESS_OVERFLOW) {
+		ISAL_PMD_LOG(ERR, "Output buffer not big enough\n");
+		op->status = RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED;
+		return ret;
+	}
+
+	/* Check that input buffer has been fully consumed */
+	if (qp->stream->avail_in != (uint32_t)0) {
+		ISAL_PMD_LOG(ERR, "Input buffer could not be read entirely\n");
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return -1;
+	}
+
+	if (ret != COMP_OK) {
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return ret;
+	}
+
+	op->consumed = qp->stream->total_in;
+	op->produced = qp->stream->total_out;
+
+	return ret;
+}
+
 /* Process compression/decompression operation */
 static int
-process_op(struct isal_comp_qp *qp __rte_unused,
-		struct rte_comp_op *op __rte_unused,
+process_op(struct isal_comp_qp *qp, struct rte_comp_op *op,
 		struct isal_priv_xform *priv_xform)
 {
 	switch (priv_xform->type) {
 	case RTE_COMP_COMPRESS:
+		process_isal_deflate(op, qp, priv_xform);
 		break;
 	case RTE_COMP_DECOMPRESS:
 		break;
drivers/compress/isal/isal_compress_pmd_ops.c +28 -0
index dcf79bf..a72a886 100644
@@ -1,6 +1,7 @@
 /* SPDX-License-Identifier: BSD-3-Clause
  * Copyright(c) 2018 Intel Corporation
  */
+#include <isa-l.h>
 
 #include <rte_common.h>
 #include <rte_compressdev_pmd.h>
@@ -9,6 +10,17 @@
 #include "isal_compress_pmd_private.h"
 
 static const struct rte_compressdev_capabilities isal_pmd_capabilities[] = {
+	{
+		.algo = RTE_COMP_ALGO_DEFLATE,
+		.comp_feature_flags =	RTE_COMP_FF_ADLER32_CHECKSUM |
+					RTE_COMP_FF_CRC32_CHECKSUM |
+					RTE_COMP_FF_SHAREABLE_PRIV_XFORM,
+		.window_size = {
+			.min = 15,
+			.max = 15,
+			.increment = 0
+		},
+	},
 	RTE_COMP_END_OF_CAPABILITIES_LIST()
 };
 
@@ -146,6 +158,12 @@ isal_comp_pmd_qp_release(struct rte_compressdev *dev, uint16_t qp_id)
 	if (qp == NULL)
 		return -EINVAL;
 
+	if (qp->stream != NULL)
+		rte_free(qp->stream);
+
+	if (qp->stream->level_buf != NULL)
+		rte_free(qp->stream->level_buf);
+
 	if (dev->data->queue_pairs[qp_id] != NULL)
 		rte_free(dev->data->queue_pairs[qp_id]);
 
@@ -214,6 +232,16 @@ isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
 		return (-ENOMEM);
 	}
 
+	/* Initialize memory for compression stream structure */
+	qp->stream = rte_zmalloc_socket("Isa-l compression stream ",
+			sizeof(struct isal_zstream),  RTE_CACHE_LINE_SIZE,
+			socket_id);
+
+	/* Initialize memory for compression level buffer */
+	qp->stream->level_buf = rte_zmalloc_socket("Isa-l compression lev_buf",
+			ISAL_DEF_LVL3_DEFAULT, RTE_CACHE_LINE_SIZE,
+			socket_id);
+
 	qp->id = qp_id;
 	dev->data->queue_pairs[qp_id] = qp;
 
drivers/compress/isal/isal_compress_pmd_private.h +2 -0
index 7505c76..6ba34a6 100644
@@ -28,6 +28,8 @@ struct isal_comp_qp {
 	struct rte_ring *processed_pkts;
 	/* Queue pair statistics */
 	struct rte_compressdev_stats qp_stats;
+	/* Compression stream information*/
+	struct isal_zstream *stream;
 	/* Number of free elements on ring */
 	uint16_t num_free_elements;
 } __rte_cache_aligned;
Signature · 1 line
2.7.4
author
#56 lore raw

PATCH v5 9/10 compress/isal: add ISA-L decomp functionality

In reply to Lee Daly

Adds decompression functionality, similar to compression, this sets internal ISA-L structures, provides input & output mbuf addresses, executes decompression, which ISA-L calls inflate and finally error checks.

---
 drivers/compress/isal/isal_compress_pmd.c         | 55 +++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c     |  8 ++++
 drivers/compress/isal/isal_compress_pmd_private.h |  2 +
 3 files changed, 65 insertions(+)
drivers/compress/isal/isal_compress_pmd.c +55 -0
index 6b6d12c..a0b4e39 100644
@@ -289,6 +289,60 @@ process_isal_deflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
 	return ret;
 }
 
+/* Stateless Decompression Function */
+static int
+process_isal_inflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
+		struct isal_priv_xform *priv_xform)
+{
+	int ret = 0;
+
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;
+
+	/* Initialize decompression state */
+	isal_inflate_init(qp->state);
+
+	/* Set op checksum, none by default */
+	qp->state->crc_flag = priv_xform->decompress.chksum;
+
+	/* Point decompression state structure to input/output buffers */
+	qp->state->avail_in = op->src.length;
+	qp->state->next_in = rte_pktmbuf_mtod(op->m_src, uint8_t *);
+	qp->state->avail_out = op->m_dst->data_len;
+	qp->state->next_out  = rte_pktmbuf_mtod(op->m_dst, uint8_t *);
+
+	if (unlikely(!qp->state->next_in || !qp->state->next_out)) {
+		ISAL_PMD_LOG(ERR, "Invalid source or destination buffers\n");
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		return -1;
+	}
+
+	/* Execute decompression operation */
+	ret = isal_inflate_stateless(qp->state);
+
+	if (ret == ISAL_OUT_OVERFLOW) {
+		ISAL_PMD_LOG(ERR, "Output buffer not big enough\n");
+		op->status = RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED;
+		return ret;
+	}
+
+	/* Check that input buffer has been fully consumed */
+	if (qp->state->avail_in != (uint32_t)0) {
+		ISAL_PMD_LOG(ERR, "Input buffer could not be read entirely\n");
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return -1;
+	}
+
+	if (ret != ISAL_DECOMP_OK) {
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return ret;
+	}
+
+	op->consumed = op->src.length - qp->state->avail_in;
+	op->produced = qp->state->total_out;
+
+return ret;
+}
+
 /* Process compression/decompression operation */
 static int
 process_op(struct isal_comp_qp *qp, struct rte_comp_op *op,
@@ -299,6 +353,7 @@ process_op(struct isal_comp_qp *qp, struct rte_comp_op *op,
 		process_isal_deflate(op, qp, priv_xform);
 		break;
 	case RTE_COMP_DECOMPRESS:
+		process_isal_inflate(op, qp, priv_xform);
 		break;
 	default:
 		ISAL_PMD_LOG(ERR, "Operation Not Supported\n");
drivers/compress/isal/isal_compress_pmd_ops.c +8 -0
index a72a886..e8c5298 100644
@@ -164,6 +164,9 @@ isal_comp_pmd_qp_release(struct rte_compressdev *dev, uint16_t qp_id)
 	if (qp->stream->level_buf != NULL)
 		rte_free(qp->stream->level_buf);
 
+	if (qp->state != NULL)
+		rte_free(qp->state);
+
 	if (dev->data->queue_pairs[qp_id] != NULL)
 		rte_free(dev->data->queue_pairs[qp_id]);
 
@@ -242,6 +245,11 @@ isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
 			ISAL_DEF_LVL3_DEFAULT, RTE_CACHE_LINE_SIZE,
 			socket_id);
 
+	/* Initialize memory for decompression state structure */
+	qp->state = rte_zmalloc_socket("Isa-l decompression state",
+			sizeof(struct inflate_state), RTE_CACHE_LINE_SIZE,
+			socket_id);
+
 	qp->id = qp_id;
 	dev->data->queue_pairs[qp_id] = qp;
 
drivers/compress/isal/isal_compress_pmd_private.h +2 -0
index 6ba34a6..46e9fcf 100644
@@ -30,6 +30,8 @@ struct isal_comp_qp {
 	struct rte_compressdev_stats qp_stats;
 	/* Compression stream information*/
 	struct isal_zstream *stream;
+	/* Decompression state information*/
+	struct inflate_state *state;
 	/* Number of free elements on ring */
 	uint16_t num_free_elements;
 } __rte_cache_aligned;
Signature · 1 line
2.7.4
author
#57 lore raw

PATCH v5 10/10 doc: add compression driver and ISA-L PMD docs

In reply to Lee Daly

This adds general compression drivers feature guide as well as the ISA-L PMD documentation and guide.

---
 .gitignore                                   |  1 +
 MAINTAINERS                                  | 12 +++++
 devtools/test-build.sh                       |  4 ++
 doc/guides/compressdevs/features/default.ini | 24 +++++++++
 doc/guides/compressdevs/features/isal.ini    | 22 ++++++++
 doc/guides/compressdevs/index.rst            | 13 +++++
 doc/guides/compressdevs/isal.rst             | 77 ++++++++++++++++++++++++++++
 doc/guides/compressdevs/overview.rst         | 12 +++++
 doc/guides/conf.py                           |  5 ++
 doc/guides/index.rst                         |  1 +
 doc/guides/rel_notes/release_18_05.rst       |  3 +-
 11 files changed, 173 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/compressdevs/features/default.ini
 create mode 100644 doc/guides/compressdevs/features/isal.ini
 create mode 100644 doc/guides/compressdevs/index.rst
 create mode 100644 doc/guides/compressdevs/isal.rst
 create mode 100644 doc/guides/compressdevs/overview.rst
.gitignore +1 -0
index 6df5ba0..9105e26 100644
@@ -3,6 +3,7 @@ doc/guides/cryptodevs/overview_feature_table.txt
 doc/guides/cryptodevs/overview_cipher_table.txt
 doc/guides/cryptodevs/overview_auth_table.txt
 doc/guides/cryptodevs/overview_aead_table.txt
+doc/guides/compressdevs/overview_feature_table.txt
 cscope.out.po
 cscope.out.in
 cscope.out
MAINTAINERS +12 -0
index d2dd61c..a22f37f 100644
@@ -766,6 +766,18 @@ F: doc/guides/cryptodevs/zuc.rst
 F: doc/guides/cryptodevs/features/zuc.ini
 
 
+Compression Drivers
+-------------------
+M: Pablo de Lara <[email protected]>
+T: git://dpdk.org/next/dpdk-next-crypto
+F: doc/guides/compressdevs/features/default.ini
+
+ISA-L PMD
+M: Lee Daly <[email protected]>
+F: drivers/compress/isal
+F: doc/guides/compressdevs/isal.rst
+F: doc/guides/compressdevs/features/isal.ini
+
 Eventdev Drivers
 ----------------
 M: Jerin Jacob <[email protected]>
devtools/test-build.sh +4 -0
index 3362edc..66f3ece 100755
@@ -45,6 +45,7 @@ default_path=$PATH
 # - DPDK_DEP_SSL (y/[n])
 # - DPDK_DEP_SZE (y/[n])
 # - DPDK_DEP_ZLIB (y/[n])
+# - DPDK_DEP_ISAL (y/[n])
 # - DPDK_MAKE_JOBS (int)
 # - DPDK_NOTIFY (notify-send)
 # - FLEXRAN_SDK
@@ -129,6 +130,7 @@ reset_env ()
 	unset DPDK_DEP_SSL
 	unset DPDK_DEP_SZE
 	unset DPDK_DEP_ZLIB
+	unset DPDK_DEP_ISAL
 	unset AESNI_MULTI_BUFFER_LIB_PATH
 	unset ARMV8_CRYPTO_LIB_PATH
 	unset FLEXRAN_SDK
@@ -178,6 +180,8 @@ config () # <directory> <target> <options>
 		test "$DPDK_DEP_ZLIB" != y || \
 		sed -ri          's,(BNX2X_PMD=)n,\1y,' $1/.config
 		sed -ri            's,(NFP_PMD=)n,\1y,' $1/.config
+		test "$DPDK_DEP_ISAL" != y || \
+		sed -ri          's,(ISAL_PMD=)n,\1y,' $1/.config
 		test "$DPDK_DEP_PCAP" != y || \
 		sed -ri               's,(PCAP=)n,\1y,' $1/.config
 		test -z "$ARMV8_CRYPTO_LIB_PATH" || \
doc/guides/compressdevs/features/default.ini new file +24 -0
new file mode 100644
index 0000000..795fc55
@@ -0,0 +1,24 @@
+;
+; Features of a default compression driver.
+;
+; This file defines the features that are valid for inclusion in
+; the other driver files and also the order that they appear in
+; the features table in the documentation.
+;
+[Features]
+HW Accelerated =
+CPU SSE        =
+CPU AVX        =
+CPU AVX2       =
+CPU AVX512     =
+CPU NEON       =
+Stateful       =
+By-Pass        =
+Chained mbufs  =
+Deflate        =
+LZS            =
+Adler32        =
+Crc32          =
+Adler32&Crc32  =
+Fixed          =
+Dynamic        =
doc/guides/compressdevs/features/isal.ini new file +22 -0
new file mode 100644
index 0000000..6a6ffb8
@@ -0,0 +1,22 @@
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; Supported features of 'ISA-L' compression driver.
+;
+[Features]
+HW Accelerated =
+CPU SSE        = Y
+CPU AVX        = Y
+CPU AVX2       = Y
+CPU AVX512     = Y
+CPU NEON       =
+Stateful       =
+By-Pass        =
+Chained mbufs  =
+Deflate        = Y
+LZS            =
+Adler32        = Y
+Crc32          = Y
+Adler32&Crc32  =
+Fixed          = Y
+Dynamic        = Y
doc/guides/compressdevs/index.rst new file +13 -0
new file mode 100644
index 0000000..bc59ce8
@@ -0,0 +1,13 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Compression Device Drivers
+==========================
+
+
+.. toctree::
+    :maxdepth: 2
+    :numbered:
+
+    overview
+    isal
doc/guides/compressdevs/isal.rst new file +77 -0

77 lines not on this page · raw message

doc/guides/compressdevs/overview.rst new file +12 -0
new file mode 100644
index 0000000..ca37de1
@@ -0,0 +1,12 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Compression Device Supported Functionality Matrices
+===================================================
+
+Supported Feature Flags
+-----------------------
+
+.. _table_compression_pmd_features:
+
+.. include:: overview_feature_table.txt
doc/guides/conf.py +5 -0
index 38de280..c883306 100644
@@ -388,6 +388,11 @@ def setup(app):
                             'AEAD',
                             'AEAD algorithms in crypto drivers',
                             'AEAD algorithm')
+    table_file = dirname(__file__) + '/compressdevs/overview_feature_table.txt'
+    generate_overview_table(table_file, 1,
+                            'Features',
+                            'Features availability in compression drivers',
+                            'Feature')
 
     if LooseVersion(sphinx_version) < LooseVersion('1.3.1'):
         print('Upgrade sphinx to version >= 1.3.1 for '
doc/guides/index.rst +1 -0
index d60529d..18fe0ec 100644
@@ -17,6 +17,7 @@ DPDK documentation
    nics/index
    bbdevs/index
    cryptodevs/index
+   compressdevs/index
    eventdevs/index
    mempool/index
    platform/index
doc/guides/rel_notes/release_18_05.rst +2 -1
index ce70d1f..d6a9d6f 100644
@@ -78,7 +78,8 @@ New Features
 * **Added a new compression poll mode driver using Intels ISA-L.**
 
    Added the new ``ISA-L`` compression driver, for compression and decompression
-   operations in software.
+   operations in software. See the :doc:`../compressdevs/isal` compression driver
+   guide for details on this new driver.
 
 
 API Changes
Signature · 1 line
2.7.4
author
#58 lore raw

PATCH v4 2/10 compress/isal: add pmd device init and de-init

In reply to Lee Daly

This patch adds device initialization functionality such as probe and create, also adding deinitialize functionality.
Dynamic logging component also added.

---
 drivers/compress/isal/Makefile                    |  1 +
 drivers/compress/isal/isal_compress_pmd.c         | 74 +++++++++++++++++++++--
 drivers/compress/isal/isal_compress_pmd_ops.c     | 25 ++++++++
 drivers/compress/isal/isal_compress_pmd_private.h | 24 ++++++++
 drivers/compress/isal/meson.build                 |  2 +-
 5 files changed, 121 insertions(+), 5 deletions(-)
 create mode 100644 drivers/compress/isal/isal_compress_pmd_ops.c
 create mode 100644 drivers/compress/isal/isal_compress_pmd_private.h
drivers/compress/isal/Makefile +1 -0
index 9b1d866..95904f6 100644
@@ -25,6 +25,7 @@ EXPORT_MAP := rte_pmd_isal_version.map
 
 # library source files
 SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal_compress_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal_compress_pmd_ops.c
 
 # export include files
 include $(RTE_SDK)/mk/rte.lib.mk
drivers/compress/isal/isal_compress_pmd.c +70 -4

93 lines not on this page · raw message

drivers/compress/isal/isal_compress_pmd_ops.c new file +25 -0
new file mode 100644
index 0000000..cff05b4
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include <rte_compressdev_pmd.h>
+
+struct rte_compressdev_ops isal_pmd_ops = {
+		.dev_configure		= NULL,
+		.dev_start		= NULL,
+		.dev_stop		= NULL,
+		.dev_close		= NULL,
+
+		.stats_get		= NULL,
+		.stats_reset		= NULL,
+
+		.dev_infos_get		= NULL,
+
+		.queue_pair_setup	= NULL,
+		.queue_pair_release	= NULL,
+
+		.private_xform_create	= NULL,
+		.private_xform_free	= NULL,
+};
+
+struct rte_compressdev_ops *isal_compress_pmd_ops = &isal_pmd_ops;
drivers/compress/isal/isal_compress_pmd_private.h new file +24 -0
new file mode 100644
index 0000000..09ecfb7
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#ifndef _ISAL_COMP_PMD_PRIVATE_H_
+#define _ISAL_COMP_PMD_PRIVATE_H_
+
+#define COMPDEV_NAME_ISAL_PMD		compress_isal
+/**< ISA-L comp PMD device name */
+
+extern int isal_logtype_driver;
+#define ISAL_PMD_LOG(level, fmt, args...) \
+	rte_log(RTE_LOG_ ## level, isal_logtype_driver, "%s(): "fmt "\n", \
+			__func__, ##args)
+
+/* private data structure for each ISA-L compression device */
+struct isal_comp_private {
+	struct rte_mempool *priv_xform_mp;
+};
+
+/** device specific operations function pointer structure */
+extern struct rte_compressdev_ops *isal_compress_pmd_ops;
+
+#endif /* _ISAL_COMP_PMD_PRIVATE_H_ */
drivers/compress/isal/meson.build +1 -1
index 4447e20..94c10fd 100644
@@ -7,7 +7,7 @@ if not dep.found()
 endif
 
 deps += 'bus_vdev'
-sources = files('isal_compress_pmd.c')
+sources = files('isal_compress_pmd.c', 'isal_compress_pmd_ops.c')
 ext_deps += dep
 pkgconfig_extra_libs += '-lisal'
 
Signature · 1 line
2.7.4
author
#59 lore raw

PATCH v4 3/10 compress/isal: add basic pmd ops

In reply to Lee Daly

Add basic device poll mode driver operations to give ability to start, stop, close device etc.

---
 drivers/compress/isal/isal_compress_pmd_ops.c     | 63 +++++++++++++++++++++--
 drivers/compress/isal/isal_compress_pmd_private.h | 12 +++++
 2 files changed, 70 insertions(+), 5 deletions(-)
drivers/compress/isal/isal_compress_pmd_ops.c +58 -5
index cff05b4..2f828c5 100644
@@ -2,18 +2,71 @@
  * Copyright(c) 2018 Intel Corporation
  */
 
+#include <rte_common.h>
 #include <rte_compressdev_pmd.h>
 
+#include "isal_compress_pmd_private.h"
+
+static const struct rte_compressdev_capabilities isal_pmd_capabilities[] = {
+	RTE_COMP_END_OF_CAPABILITIES_LIST()
+};
+
+/** Configure device */
+static int
+isal_comp_pmd_config(struct rte_compressdev *dev __rte_unused,
+		struct rte_compressdev_config *config __rte_unused)
+{
+	return 0;
+}
+
+/** Start device */
+static int
+isal_comp_pmd_start(__rte_unused struct rte_compressdev *dev)
+{
+	return 0;
+}
+
+/** Stop device */
+static void
+isal_comp_pmd_stop(__rte_unused struct rte_compressdev *dev)
+{
+}
+
+/** Close device */
+static int
+isal_comp_pmd_close(struct rte_compressdev *dev)
+{
+	/* Free private data */
+	struct isal_comp_private *internals = dev->data->dev_private;
+
+	rte_mempool_free(internals->priv_xform_mp);
+	return 0;
+}
+
+/** Get device info */
+static void
+isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
+		struct rte_compressdev_info *dev_info)
+{
+	if (dev_info != NULL) {
+		dev_info->capabilities = isal_pmd_capabilities;
+		dev_info->feature_flags = RTE_COMPDEV_FF_CPU_AVX512 |
+				RTE_COMPDEV_FF_CPU_AVX2 |
+				RTE_COMPDEV_FF_CPU_AVX |
+				RTE_COMPDEV_FF_CPU_SSE;
+	}
+}
+
 struct rte_compressdev_ops isal_pmd_ops = {
-		.dev_configure		= NULL,
-		.dev_start		= NULL,
-		.dev_stop		= NULL,
-		.dev_close		= NULL,
+		.dev_configure		= isal_comp_pmd_config,
+		.dev_start		= isal_comp_pmd_start,
+		.dev_stop		= isal_comp_pmd_stop,
+		.dev_close		= isal_comp_pmd_close,
 
 		.stats_get		= NULL,
 		.stats_reset		= NULL,
 
-		.dev_infos_get		= NULL,
+		.dev_infos_get		= isal_comp_pmd_info_get,
 
 		.queue_pair_setup	= NULL,
 		.queue_pair_release	= NULL,
drivers/compress/isal/isal_compress_pmd_private.h +12 -0
index 09ecfb7..efbe68b 100644
@@ -18,6 +18,18 @@ struct isal_comp_private {
 	struct rte_mempool *priv_xform_mp;
 };
 
+/** ISA-L queue pair */
+struct isal_comp_qp {
+	/* Queue Pair Identifier */
+	uint16_t id;
+	/* Unique Queue Pair Name */
+	char name[RTE_COMPRESSDEV_NAME_MAX_LEN];
+	/* Queue pair statistics */
+	struct rte_compressdev_stats qp_stats;
+	/* Number of free elements on ring */
+	uint16_t num_free_elements;
+} __rte_cache_aligned;
+
 /** device specific operations function pointer structure */
 extern struct rte_compressdev_ops *isal_compress_pmd_ops;
 
Signature · 1 line
2.7.4
author
#60 lore raw

PATCH v4 4/10 compress/isal: add private xform related ops

In reply to Lee Daly

This patch creates, configures and frees the private xform, taking the applications xform and using it to populate the PMDs own private xform with the information which will be required for the compress/decompress functions. This information includes the level, algorithm, type of huffman code, type of checksum etc.

---
 drivers/compress/isal/isal_compress_pmd.c         | 197 ++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c     | 103 ++++++++++-
 drivers/compress/isal/isal_compress_pmd_private.h |  15 ++
 3 files changed, 310 insertions(+), 5 deletions(-)
drivers/compress/isal/isal_compress_pmd.c +197 -0
index 5cc9409..325867b 100644
@@ -1,6 +1,7 @@
 /* SPDX-License-Identifier: BSD-3-Clause
  * Copyright(c) 2018 Intel Corporation
  */
+#include <isa-l.h>
 
 #include <rte_bus_vdev.h>
 #include <rte_common.h>
@@ -9,8 +10,204 @@
 
 #include "isal_compress_pmd_private.h"
 
+#define RTE_COMP_ISAL_WINDOW_SIZE 15
+#define RTE_COMP_ISAL_LEVEL_ZERO 0 /* ISA-L Level 0 used for fixed Huffman */
+#define RTE_COMP_ISAL_LEVEL_ONE 1
+#define RTE_COMP_ISAL_LEVEL_TWO 2
+#define RTE_COMP_ISAL_LEVEL_THREE 3 /* Optimised for AVX512 & AVX2 only */
+
 int isal_logtype_driver;
 
+/* Verify and set private xform parameters */
+int
+isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
+		const struct rte_comp_xform *xform)
+{
+	if (xform == NULL)
+		return -EINVAL;
+
+	/* Set compression private xform variables */
+	if (xform->type == RTE_COMP_COMPRESS) {
+		/* Set private xform type - COMPRESS/DECOMPRESS */
+		priv_xform->type = RTE_COMP_COMPRESS;
+
+		/* Set private xform algorithm */
+		if (xform->compress.algo != RTE_COMP_ALGO_DEFLATE) {
+			if (xform->compress.algo == RTE_COMP_ALGO_NULL) {
+				ISAL_PMD_LOG(ERR, "By-pass not supported\n");
+				return -ENOTSUP;
+			}
+			ISAL_PMD_LOG(ERR, "Algorithm not supported\n");
+			return -ENOTSUP;
+		}
+		priv_xform->compress.algo = RTE_COMP_ALGO_DEFLATE;
+
+		/* Set private xform checksum */
+		switch (xform->compress.chksum) {
+		case(RTE_COMP_CHECKSUM_NONE):
+			priv_xform->compress.chksum = RTE_COMP_CHECKSUM_NONE;
+			break;
+		case(RTE_COMP_CHECKSUM_ADLER32):
+			priv_xform->compress.chksum = IGZIP_ZLIB;
+			break;
+		case(RTE_COMP_CHECKSUM_CRC32):
+			priv_xform->compress.chksum = IGZIP_GZIP;
+			break;
+		default:
+			ISAL_PMD_LOG(ERR, "Checksum not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform window size, 32K supported */
+		if (xform->compress.window_size == RTE_COMP_ISAL_WINDOW_SIZE)
+			priv_xform->compress.window_size =
+					RTE_COMP_ISAL_WINDOW_SIZE;
+		else {
+			ISAL_PMD_LOG(ERR, "Window size not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform huffman type */
+		switch (xform->compress.deflate.huffman) {
+		case(RTE_COMP_HUFFMAN_DEFAULT):
+			priv_xform->compress.deflate.huffman =
+					RTE_COMP_HUFFMAN_DEFAULT;
+			break;
+		case(RTE_COMP_HUFFMAN_FIXED):
+			priv_xform->compress.deflate.huffman =
+					RTE_COMP_HUFFMAN_FIXED;
+			break;
+		case(RTE_COMP_HUFFMAN_DYNAMIC):
+			priv_xform->compress.deflate.huffman =
+					RTE_COMP_HUFFMAN_DYNAMIC;
+			break;
+		default:
+			ISAL_PMD_LOG(ERR, "Huffman code not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform level.
+		 * Checking compliance with compressdev API, -1 <= level => 9
+		 */
+		if (xform->compress.level < RTE_COMP_LEVEL_PMD_DEFAULT ||
+				xform->compress.level > RTE_COMP_LEVEL_MAX) {
+			ISAL_PMD_LOG(ERR, "Compression level out of range\n");
+			return -EINVAL;
+		}
+		/* Check for Compressdev API level 0, No compression
+		 * not supported in ISA-L
+		 */
+		else if (xform->compress.level == RTE_COMP_LEVEL_NONE) {
+			ISAL_PMD_LOG(ERR, "No Compression not supported\n");
+			return -ENOTSUP;
+		}
+		/* If using fixed huffman code, level must be 0 */
+		else if (priv_xform->compress.deflate.huffman ==
+				RTE_COMP_HUFFMAN_FIXED) {
+			ISAL_PMD_LOG(DEBUG, "ISA-L level 0 used due to a"
+					" fixed huffman code\n");
+			priv_xform->compress.level = RTE_COMP_ISAL_LEVEL_ZERO;
+			priv_xform->level_buffer_size =
+					ISAL_DEF_LVL0_DEFAULT;
+		} else {
+			/* Mapping API levels to ISA-L levels 1,2 & 3 */
+			switch (xform->compress.level) {
+			case RTE_COMP_LEVEL_PMD_DEFAULT:
+				/* Default is 1 if not using fixed huffman */
+				priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_ONE;
+				priv_xform->level_buffer_size =
+						ISAL_DEF_LVL1_DEFAULT;
+				break;
+			case RTE_COMP_LEVEL_MIN:
+				priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_ONE;
+				priv_xform->level_buffer_size =
+						ISAL_DEF_LVL1_DEFAULT;
+				break;
+			case RTE_COMP_ISAL_LEVEL_TWO:
+				priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_TWO;
+				priv_xform->level_buffer_size =
+						ISAL_DEF_LVL2_DEFAULT;
+				break;
+			/* Level 3 or higher requested */
+			default:
+				/* Check for AVX512, to use ISA-L level 3 */
+				if (rte_cpu_get_flag_enabled(
+						RTE_CPUFLAG_AVX512F)) {
+					priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_THREE;
+					priv_xform->level_buffer_size =
+						ISAL_DEF_LVL3_DEFAULT;
+				}
+				/* Check for AVX2, to use ISA-L level 3 */
+				else if (rte_cpu_get_flag_enabled(
+						RTE_CPUFLAG_AVX2)) {
+					priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_THREE;
+					priv_xform->level_buffer_size =
+						ISAL_DEF_LVL3_DEFAULT;
+				}
+				else{
+					ISAL_PMD_LOG(DEBUG, "Requested ISA-L level"
+						" 3 or above; Level 3 optimized"
+						" for AVX512 & AVX2 only."
+						" level changed to 2.\n");
+					priv_xform->compress.level =
+						RTE_COMP_ISAL_LEVEL_TWO;
+					priv_xform->level_buffer_size =
+						ISAL_DEF_LVL2_DEFAULT;
+				}
+			}
+		}
+	}
+
+	/* Set decompression private xform variables */
+	else if (xform->type == RTE_COMP_DECOMPRESS) {
+
+		/* Set private xform type - COMPRESS/DECOMPRESS */
+		priv_xform->type = RTE_COMP_DECOMPRESS;
+
+		/* Set private xform algorithm */
+		if (xform->decompress.algo != RTE_COMP_ALGO_DEFLATE) {
+			if (xform->decompress.algo == RTE_COMP_ALGO_NULL) {
+				ISAL_PMD_LOG(ERR, "By pass not supported\n");
+				return -ENOTSUP;
+			}
+			ISAL_PMD_LOG(ERR, "Algorithm not supported\n");
+			return -ENOTSUP;
+		}
+		priv_xform->decompress.algo = RTE_COMP_ALGO_DEFLATE;
+
+		/* Set private xform checksum */
+		switch (xform->decompress.chksum) {
+		case(RTE_COMP_CHECKSUM_NONE):
+			priv_xform->decompress.chksum = RTE_COMP_CHECKSUM_NONE;
+			break;
+		case(RTE_COMP_CHECKSUM_ADLER32):
+			priv_xform->decompress.chksum = ISAL_ZLIB;
+			break;
+		case(RTE_COMP_CHECKSUM_CRC32):
+			priv_xform->decompress.chksum = ISAL_GZIP;
+			break;
+		default:
+			ISAL_PMD_LOG(ERR, "Checksum not supported\n");
+			return -ENOTSUP;
+		}
+
+		/* Set private xform window size, 32K supported */
+		if (xform->decompress.window_size == RTE_COMP_ISAL_WINDOW_SIZE)
+			priv_xform->decompress.window_size =
+					RTE_COMP_ISAL_WINDOW_SIZE;
+		else {
+			ISAL_PMD_LOG(ERR, "Window size not supported\n");
+			return -ENOTSUP;
+		}
+	}
+	return 0;
+}
+
 /* Create ISA-L compression device */
 static int
 compdev_isal_create(const char *name, struct rte_vdev_device *vdev,
drivers/compress/isal/isal_compress_pmd_ops.c +98 -5

122 lines not on this page · raw message

drivers/compress/isal/isal_compress_pmd_private.h +15 -0
index efbe68b..7e3b840 100644
@@ -30,6 +30,21 @@ struct isal_comp_qp {
 	uint16_t num_free_elements;
 } __rte_cache_aligned;
 
+/** ISA-L private xform structure */
+struct isal_priv_xform {
+	enum rte_comp_xform_type type;
+	union {
+		struct rte_comp_compress_xform compress;
+		struct rte_comp_decompress_xform decompress;
+	};
+	uint32_t level_buffer_size;
+} __rte_cache_aligned;
+
+/** Set and validate NULL comp private xform parameters */
+extern int
+isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
+			const struct rte_comp_xform *xform);
+
 /** device specific operations function pointer structure */
 extern struct rte_compressdev_ops *isal_compress_pmd_ops;
 
Signature · 1 line
2.7.4
author
#61 lore raw

PATCH v4 5/10 compress/isal: add queue pair related ops

In reply to Lee Daly

This adds queue pair operations such as setup and release.

---
 drivers/compress/isal/isal_compress_pmd_ops.c     | 111 +++++++++++++++++++++-
 drivers/compress/isal/isal_compress_pmd_private.h |   2 +
 2 files changed, 111 insertions(+), 2 deletions(-)
drivers/compress/isal/isal_compress_pmd_ops.c +109 -2
index 9c89df0..b0abf42 100644
@@ -4,6 +4,7 @@
 
 #include <rte_common.h>
 #include <rte_compressdev_pmd.h>
+#include <rte_malloc.h>
 
 #include "isal_compress_pmd_private.h"
 
@@ -106,6 +107,112 @@ isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
 	}
 }
 
+
+/** Release queue pair */
+static int
+isal_comp_pmd_qp_release(struct rte_compressdev *dev, uint16_t qp_id)
+{
+	struct isal_comp_qp *qp = dev->data->queue_pairs[qp_id];
+
+	if (qp == NULL)
+		return -EINVAL;
+
+	if (dev->data->queue_pairs[qp_id] != NULL)
+		rte_free(dev->data->queue_pairs[qp_id]);
+
+	return 0;
+}
+
+/** Create a ring to place process packets on */
+static struct rte_ring *
+isal_comp_pmd_qp_create_processed_pkts_ring(struct isal_comp_qp *qp,
+		unsigned int ring_size, int socket_id)
+{
+	struct rte_ring *r;
+
+	r = rte_ring_lookup(qp->name);
+	if (r) {
+		if (rte_ring_get_size(r) >= ring_size) {
+			ISAL_PMD_LOG(DEBUG,
+				"Reusing existing ring %s for processed packets",
+				qp->name);
+			return r;
+		}
+
+			ISAL_PMD_LOG(ERR,
+				"Unable to reuse existing ring %s"
+				" for processed packets",
+			 qp->name);
+		return NULL;
+	}
+
+	return rte_ring_create(qp->name, ring_size, socket_id,
+			RING_F_SP_ENQ | RING_F_SC_DEQ);
+}
+
+/** set a unique name for the queue pair based on its name, dev_id and qp_id */
+static int
+isal_comp_pmd_qp_set_unique_name(struct rte_compressdev *dev,
+struct isal_comp_qp *qp)
+{
+	unsigned int n = snprintf(qp->name, sizeof(qp->name),
+			"isal_compression_pmd_%u_qp_%u",
+			dev->data->dev_id, qp->id);
+
+	if (n >= sizeof(qp->name))
+		return -1;
+
+	return 0;
+}
+
+/* Setup a queue pair */
+static int
+isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
+		uint32_t max_inflight_ops, int socket_id)
+{
+	struct isal_comp_qp *qp = NULL;
+	int retval;
+
+	/* Free memory prior to re-allocation if needed. */
+	if (dev->data->queue_pairs[qp_id] != NULL)
+		isal_comp_pmd_qp_release(dev, qp_id);
+
+	/* Allocate the queue pair data structure. */
+	qp = rte_zmalloc_socket("Isa-l compression PMD Queue Pair", sizeof(*qp),
+					RTE_CACHE_LINE_SIZE, socket_id);
+	if (qp == NULL) {
+		ISAL_PMD_LOG(ERR, "Failed to allocate queue pair memory");
+		return (-ENOMEM);
+	}
+
+	qp->id = qp_id;
+	dev->data->queue_pairs[qp_id] = qp;
+
+	retval = isal_comp_pmd_qp_set_unique_name(dev, qp);
+	if (retval) {
+		ISAL_PMD_LOG(ERR, "Failed to create unique name for isal "
+				"compression device");
+		goto qp_setup_cleanup;
+	}
+
+	qp->processed_pkts = isal_comp_pmd_qp_create_processed_pkts_ring(qp,
+			max_inflight_ops, socket_id);
+	if (qp->processed_pkts == NULL) {
+		ISAL_PMD_LOG(ERR, "Failed to create unique name for isal "
+				"compression device");
+		goto qp_setup_cleanup;
+	}
+
+	memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
+	return 0;
+
+qp_setup_cleanup:
+	if (qp)
+		rte_free(qp);
+
+	return -1;
+}
+
 /** Set private xform data*/
 static int
 isal_comp_pmd_priv_xform_create(struct rte_compressdev *dev,
@@ -161,8 +268,8 @@ struct rte_compressdev_ops isal_pmd_ops = {
 
 		.dev_infos_get		= isal_comp_pmd_info_get,
 
-		.queue_pair_setup	= NULL,
-		.queue_pair_release	= NULL,
+		.queue_pair_setup	= isal_comp_pmd_qp_setup,
+		.queue_pair_release	= isal_comp_pmd_qp_release,
 
 		.private_xform_create	= isal_comp_pmd_priv_xform_create,
 		.private_xform_free	= isal_comp_pmd_priv_xform_free,
drivers/compress/isal/isal_compress_pmd_private.h +2 -0
index 7e3b840..7505c76 100644
@@ -24,6 +24,8 @@ struct isal_comp_qp {
 	uint16_t id;
 	/* Unique Queue Pair Name */
 	char name[RTE_COMPRESSDEV_NAME_MAX_LEN];
+	/* Ring for placing process packets */
+	struct rte_ring *processed_pkts;
 	/* Queue pair statistics */
 	struct rte_compressdev_stats qp_stats;
 	/* Number of free elements on ring */
Signature · 1 line
2.7.4
author
#62 lore raw

PATCH v4 6/10 compress/isal: support enqueue/dequeue api

In reply to Lee Daly

This patchs adds support for the compressdev enqueue_burst and dequeue_burst API operations.

---
 drivers/compress/isal/isal_compress_pmd.c     | 64 +++++++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c |  2 +
 2 files changed, 66 insertions(+)
drivers/compress/isal/isal_compress_pmd.c +64 -0
index 325867b..b13822a 100644
@@ -208,6 +208,66 @@ isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
 	return 0;
 }
 
+/* Process compression/decompression operation */
+static int
+process_op(struct isal_comp_qp *qp __rte_unused,
+		struct rte_comp_op *op __rte_unused,
+		struct isal_priv_xform *priv_xform)
+{
+	switch (priv_xform->type) {
+	case RTE_COMP_COMPRESS:
+		break;
+	case RTE_COMP_DECOMPRESS:
+		break;
+	default:
+		ISAL_PMD_LOG(ERR, "Operation Not Supported\n");
+		return -ENOTSUP;
+	}
+	return 0;
+}
+
+/* Enqueue burst */
+static uint16_t
+isal_comp_pmd_enqueue_burst(void *queue_pair, struct rte_comp_op **ops,
+			uint16_t nb_ops)
+{
+	struct isal_comp_qp *qp = queue_pair;
+	uint16_t i;
+	int retval;
+	int16_t num_enq = RTE_MIN(qp->num_free_elements, nb_ops);
+
+	for (i = 0; i < num_enq; i++) {
+		retval = process_op(qp, ops[i], ops[i]->private_xform);
+		if (unlikely(retval < 0) ||
+				ops[i]->status != RTE_COMP_OP_STATUS_SUCCESS) {
+			qp->qp_stats.enqueue_err_count++;
+		}
+	}
+
+	retval = rte_ring_enqueue_burst(qp->processed_pkts, (void *)ops,
+			num_enq, NULL);
+	qp->num_free_elements -= retval;
+	qp->qp_stats.enqueued_count += retval;
+
+	return retval;
+}
+
+/* Dequeue burst */
+static uint16_t
+isal_comp_pmd_dequeue_burst(void *queue_pair, struct rte_comp_op **ops,
+		uint16_t nb_ops)
+{
+	struct isal_comp_qp *qp = queue_pair;
+	uint16_t nb_dequeued;
+
+	nb_dequeued = rte_ring_dequeue_burst(qp->processed_pkts, (void **)ops,
+			nb_ops, NULL);
+	qp->num_free_elements += nb_dequeued;
+	qp->qp_stats.dequeued_count += nb_dequeued;
+
+	return nb_dequeued;
+}
+
 /* Create ISA-L compression device */
 static int
 compdev_isal_create(const char *name, struct rte_vdev_device *vdev,
@@ -224,6 +284,10 @@ compdev_isal_create(const char *name, struct rte_vdev_device *vdev,
 
 	dev->dev_ops = isal_compress_pmd_ops;
 
+	/* register rx/tx burst functions for data path */
+	dev->dequeue_burst = isal_comp_pmd_dequeue_burst;
+	dev->enqueue_burst = isal_comp_pmd_enqueue_burst;
+
 	return 0;
 }
 
drivers/compress/isal/isal_compress_pmd_ops.c +2 -0
index b0abf42..4033864 100644
@@ -203,6 +203,8 @@ isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
 		goto qp_setup_cleanup;
 	}
 
+	qp->num_free_elements = rte_ring_free_count(qp->processed_pkts);
+
 	memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
 	return 0;
 
Signature · 1 line
2.7.4
#63 lore raw

Re: [PATCH v4 06/10] compress/isal: support enqueue/dequeue api

Quoted 49 lines

-----Original Message-----
From: Daly, Lee
Sent: Saturday, April 28, 2018 12:38 AM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v4 06/10] compress/isal: support enqueue/dequeue api

This patchs adds support for the compressdev enqueue_burst and dequeue_burst API operations.

---
 drivers/compress/isal/isal_compress_pmd.c     | 64
+++++++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c |  2 +
 2 files changed, 66 insertions(+)
drivers/compress/isal/isal_compress_pmd.c +18 -0
index 325867b..b13822a 100644
@@ -208,6 +208,66 @@ isal_comp_set_priv_xform_parameters(struct
 isal_priv_xform *priv_xform,
 	return 0;
 }
 
+/* Process compression/decompression operation */ static int
+process_op(struct isal_comp_qp *qp __rte_unused,
+		struct rte_comp_op *op __rte_unused,
+		struct isal_priv_xform *priv_xform)
+{
+	switch (priv_xform->type) {
+	case RTE_COMP_COMPRESS:
+		break;
+	case RTE_COMP_DECOMPRESS:
+		break;
+	default:
+		ISAL_PMD_LOG(ERR, "Operation Not Supported\n");
+		return -ENOTSUP;
+	}
+	return 0;
+}
+
+/* Enqueue burst */

drivers/compress/isal/isal_compress_pmd.c isal_comp_set_priv_xform_parameters(struct · line 239

Quoted 10 lines
drivers/compress/isal/isal_compress_pmd.c +10 -0
+static uint16_t
+isal_comp_pmd_enqueue_burst(void *queue_pair, struct rte_comp_op **ops,
+			uint16_t nb_ops)
+{
+	struct isal_comp_qp *qp = queue_pair;
+	uint16_t i;
+	int retval;
+	int16_t num_enq = RTE_MIN(qp->num_free_elements, nb_ops);
+
+	for (i = 0; i < num_enq; i++) {

Before calling process_op, check if the operation is a stateless op, since private_xform is only used with stateless ops and if not, set the status to error and continue with the next op.

Quoted 1 line
drivers/compress/isal/isal_compress_pmd.c +1 -0
+		retval = process_op(qp, ops[i], ops[i]->private_xform);
author
#64 lore raw

PATCH v4 7/10 compress/isal: add stats related ops

In reply to Lee Daly

Add functions for statistic retrieval and resetting.

---
 drivers/compress/isal/isal_compress_pmd_ops.c | 33 +++++++++++++++++++++++++--
 1 file changed, 31 insertions(+), 2 deletions(-)
drivers/compress/isal/isal_compress_pmd_ops.c +31 -2
index 4033864..dcf79bf 100644
@@ -93,6 +93,24 @@ isal_comp_pmd_close(struct rte_compressdev *dev)
 	return 0;
 }
 
+/** Get device statistics */
+static void
+isal_comp_pmd_stats_get(struct rte_compressdev *dev,
+		struct rte_compressdev_stats *stats)
+{
+	uint16_t qp_id;
+
+	for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+		struct isal_comp_qp *qp = dev->data->queue_pairs[qp_id];
+
+		stats->enqueued_count += qp->qp_stats.enqueued_count;
+		stats->dequeued_count += qp->qp_stats.dequeued_count;
+
+		stats->enqueue_err_count += qp->qp_stats.enqueue_err_count;
+		stats->dequeue_err_count += qp->qp_stats.dequeue_err_count;
+	}
+}
+
 /** Get device info */
 static void
 isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
@@ -107,6 +125,17 @@ isal_comp_pmd_info_get(struct rte_compressdev *dev __rte_unused,
 	}
 }
 
+/** Reset device statistics */
+static void
+isal_comp_pmd_stats_reset(struct rte_compressdev *dev)
+{
+	uint16_t qp_id;
+
+	for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+		struct isal_comp_qp *qp = dev->data->queue_pairs[qp_id];
+		memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
+	}
+}
 
 /** Release queue pair */
 static int
@@ -265,8 +294,8 @@ struct rte_compressdev_ops isal_pmd_ops = {
 		.dev_stop		= isal_comp_pmd_stop,
 		.dev_close		= isal_comp_pmd_close,
 
-		.stats_get		= NULL,
-		.stats_reset		= NULL,
+		.stats_get		= isal_comp_pmd_stats_get,
+		.stats_reset		= isal_comp_pmd_stats_reset,
 
 		.dev_infos_get		= isal_comp_pmd_info_get,
 
Signature · 1 line
2.7.4
author
#65 lore raw

PATCH v4 8/10 compress/isal: add ISA-L compression functionality

In reply to Lee Daly

Adds compression functionality, this sets internal ISA-L structures, provides input & output mbuf addresses, executes compression, which ISA-L calls deflate, and finally error checks.

---
 drivers/compress/isal/isal_compress_pmd.c         | 93 ++++++++++++++++++++++-
 drivers/compress/isal/isal_compress_pmd_ops.c     | 28 +++++++
 drivers/compress/isal/isal_compress_pmd_private.h |  2 +
 3 files changed, 121 insertions(+), 2 deletions(-)
drivers/compress/isal/isal_compress_pmd.c +91 -2
index b13822a..e09fd3f 100644
@@ -6,6 +6,7 @@
 #include <rte_bus_vdev.h>
 #include <rte_common.h>
 #include <rte_malloc.h>
+#include <rte_mbuf.h>
 #include <rte_compressdev_pmd.h>
 
 #include "isal_compress_pmd_private.h"
@@ -208,14 +209,102 @@ isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform,
 	return 0;
 }
 
+/* Stateless Compression Function */
+static int
+process_isal_deflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
+		struct isal_priv_xform *priv_xform)
+{
+	int ret = 0;
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;
+
+	/* Required due to init clearing level_buf */
+	uint8_t *temp_level_buf = qp->stream->level_buf;
+
+	/* Initialize compression stream */
+	isal_deflate_stateless_init(qp->stream);
+
+	qp->stream->level_buf = temp_level_buf;
+
+	qp->stream->flush = NO_FLUSH;
+
+	/* Set op checksum, none by default */
+	qp->stream->gzip_flag = priv_xform->compress.chksum;
+
+	/* set op level & intermediate level buffer */
+	qp->stream->level = priv_xform->compress.level;
+	qp->stream->level_buf_size = priv_xform->level_buffer_size;
+
+	/* Point compression stream structure to input/output buffers */
+	qp->stream->avail_in = op->src.length;
+	qp->stream->next_in = rte_pktmbuf_mtod(op->m_src, uint8_t *);
+	qp->stream->avail_out = op->m_dst->data_len;
+	qp->stream->next_out  = rte_pktmbuf_mtod(op->m_dst, uint8_t *);
+	qp->stream->end_of_stream = 1; /* All input consumed in one go */
+
+	if (unlikely(!qp->stream->next_in || !qp->stream->next_out)) {
+		ISAL_PMD_LOG(ERR, "Invalid source or destination buffers\n");
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		return -1;
+	}
+
+	/* Set op huffman code */
+	if (priv_xform->compress.deflate.huffman == RTE_COMP_HUFFMAN_FIXED)
+		isal_deflate_set_hufftables(qp->stream, NULL,
+				IGZIP_HUFFTABLE_STATIC);
+	else if (priv_xform->compress.deflate.huffman ==
+			RTE_COMP_HUFFMAN_DEFAULT)
+		isal_deflate_set_hufftables(qp->stream, NULL,
+			IGZIP_HUFFTABLE_DEFAULT);
+	/* Dynamically change the huffman code to suit the input data */
+	else if (priv_xform->compress.deflate.huffman ==
+			RTE_COMP_HUFFMAN_DYNAMIC) {
+		struct isal_hufftables     hufftable;
+		struct isal_huff_histogram histogram;
+		memset(&histogram, 0, sizeof(struct isal_huff_histogram));
+
+		isal_update_histogram(qp->stream->next_in, qp->stream->avail_in,
+				&histogram);
+		isal_create_hufftables(&hufftable, &histogram);
+		isal_deflate_set_hufftables(qp->stream, &hufftable,
+				IGZIP_HUFFTABLE_CUSTOM);
+	}
+
+	/* Execute compression operation */
+	ret =  isal_deflate_stateless(qp->stream);
+
+	/* Check that output buffer did not run out of space */
+	if (ret == STATELESS_OVERFLOW) {
+		ISAL_PMD_LOG(ERR, "Output buffer not big enough\n");
+		op->status = RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED;
+		return ret;
+	}
+
+	/* Check that input buffer has been fully consumed */
+	if (qp->stream->avail_in != (uint32_t)0) {
+		ISAL_PMD_LOG(ERR, "Input buffer could not be read entirely\n");
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return -1;
+	}
+
+	if (ret != COMP_OK) {
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return ret;
+	}
+
+	op->consumed = op->src.length - qp->stream->avail_in;
+	op->produced = qp->stream->total_out;
+
+	return ret;
+}
+
 /* Process compression/decompression operation */
 static int
-process_op(struct isal_comp_qp *qp __rte_unused,
-		struct rte_comp_op *op __rte_unused,
+process_op(struct isal_comp_qp *qp, struct rte_comp_op *op,
 		struct isal_priv_xform *priv_xform)
 {
 	switch (priv_xform->type) {
 	case RTE_COMP_COMPRESS:
+		process_isal_deflate(op, qp, priv_xform);
 		break;
 	case RTE_COMP_DECOMPRESS:
 		break;
drivers/compress/isal/isal_compress_pmd_ops.c +28 -0
index dcf79bf..a72a886 100644
@@ -1,6 +1,7 @@
 /* SPDX-License-Identifier: BSD-3-Clause
  * Copyright(c) 2018 Intel Corporation
  */
+#include <isa-l.h>
 
 #include <rte_common.h>
 #include <rte_compressdev_pmd.h>
@@ -9,6 +10,17 @@
 #include "isal_compress_pmd_private.h"
 
 static const struct rte_compressdev_capabilities isal_pmd_capabilities[] = {
+	{
+		.algo = RTE_COMP_ALGO_DEFLATE,
+		.comp_feature_flags =	RTE_COMP_FF_ADLER32_CHECKSUM |
+					RTE_COMP_FF_CRC32_CHECKSUM |
+					RTE_COMP_FF_SHAREABLE_PRIV_XFORM,
+		.window_size = {
+			.min = 15,
+			.max = 15,
+			.increment = 0
+		},
+	},
 	RTE_COMP_END_OF_CAPABILITIES_LIST()
 };
 
@@ -146,6 +158,12 @@ isal_comp_pmd_qp_release(struct rte_compressdev *dev, uint16_t qp_id)
 	if (qp == NULL)
 		return -EINVAL;
 
+	if (qp->stream != NULL)
+		rte_free(qp->stream);
+
+	if (qp->stream->level_buf != NULL)
+		rte_free(qp->stream->level_buf);
+
 	if (dev->data->queue_pairs[qp_id] != NULL)
 		rte_free(dev->data->queue_pairs[qp_id]);
 
@@ -214,6 +232,16 @@ isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
 		return (-ENOMEM);
 	}
 
+	/* Initialize memory for compression stream structure */
+	qp->stream = rte_zmalloc_socket("Isa-l compression stream ",
+			sizeof(struct isal_zstream),  RTE_CACHE_LINE_SIZE,
+			socket_id);
+
+	/* Initialize memory for compression level buffer */
+	qp->stream->level_buf = rte_zmalloc_socket("Isa-l compression lev_buf",
+			ISAL_DEF_LVL3_DEFAULT, RTE_CACHE_LINE_SIZE,
+			socket_id);
+
 	qp->id = qp_id;
 	dev->data->queue_pairs[qp_id] = qp;
 
drivers/compress/isal/isal_compress_pmd_private.h +2 -0
index 7505c76..6ba34a6 100644
@@ -28,6 +28,8 @@ struct isal_comp_qp {
 	struct rte_ring *processed_pkts;
 	/* Queue pair statistics */
 	struct rte_compressdev_stats qp_stats;
+	/* Compression stream information*/
+	struct isal_zstream *stream;
 	/* Number of free elements on ring */
 	uint16_t num_free_elements;
 } __rte_cache_aligned;
Signature · 1 line
2.7.4
#66 lore raw

Re: [PATCH v4 08/10] compress/isal: add ISA-L compression functionality

Hi Lee,

Quoted 46 lines

-----Original Message-----
From: Daly, Lee
Sent: Saturday, April 28, 2018 12:39 AM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v4 08/10] compress/isal: add ISA-L compression functionality

Adds compression functionality, this sets internal ISA-L structures, provides input & output mbuf addresses, executes compression, which ISA-L calls deflate, and finally error checks.

---
 drivers/compress/isal/isal_compress_pmd.c         | 93
++++++++++++++++++++++-
 drivers/compress/isal/isal_compress_pmd_ops.c     | 28 +++++++
 drivers/compress/isal/isal_compress_pmd_private.h |  2 +
 3 files changed, 121 insertions(+), 2 deletions(-)
drivers/compress/isal/isal_compress_pmd.c +8 -0
index b13822a..e09fd3f 100644
@@ -6,6 +6,7 @@
 #include <rte_bus_vdev.h>
 #include <rte_common.h>
 #include <rte_malloc.h>
+#include <rte_mbuf.h>
 #include <rte_compressdev_pmd.h>
 
 #include "isal_compress_pmd_private.h"
@@ -208,14 +209,102 @@ isal_comp_set_priv_xform_parameters(struct
 isal_priv_xform *priv_xform,
 	return 0;
 }
 
+/* Stateless Compression Function */
+static int
+process_isal_deflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
+		struct isal_priv_xform *priv_xform)
+{
+	int ret = 0;
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;

drivers/compress/isal/isal_compress_pmd.c isal_comp_set_priv_xform_parameters(struct · line 229

Quoted 10 lines
drivers/compress/isal/isal_compress_pmd.c +10 -0
+
+	/* Required due to init clearing level_buf */
+	uint8_t *temp_level_buf = qp->stream->level_buf;
+
+	/* Initialize compression stream */
+	isal_deflate_stateless_init(qp->stream);
+
+	qp->stream->level_buf = temp_level_buf;
+
+	qp->stream->flush = NO_FLUSH;

Worth putting a comment here saying why you are not using the flush flag from op.

Quoted 4 lines
drivers/compress/isal/isal_compress_pmd.c +4 -0
+
+	/* Set op checksum, none by default */
+	qp->stream->gzip_flag = priv_xform->compress.chksum;
+

...

drivers/compress/isal/isal_compress_pmd.c isal_comp_set_priv_xform_parameters(struct isal_priv_xform *priv_xform, · line 294

Quoted 8 lines
drivers/compress/isal/isal_compress_pmd.c +8 -0
+	}
+
+	if (ret != COMP_OK) {
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return ret;
+	}
+
+	op->consumed = op->src.length - qp->stream->avail_in;

Could you use qp->stream->total_in here? Looks simpler.

author
#67 lore raw

PATCH v4 9/10 compress/isal: add ISA-L decomp functionality

In reply to Lee Daly

Adds decompression functionality, similar to compression, this sets internal ISA-L structures, provides input & output mbuf addresses, executes decompression, which ISA-L calls inflate and finally error checks.

---
 drivers/compress/isal/isal_compress_pmd.c         | 55 +++++++++++++++++++++++
 drivers/compress/isal/isal_compress_pmd_ops.c     |  8 ++++
 drivers/compress/isal/isal_compress_pmd_private.h |  2 +
 3 files changed, 65 insertions(+)
drivers/compress/isal/isal_compress_pmd.c +55 -0
index e09fd3f..d52b076 100644
@@ -297,6 +297,60 @@ process_isal_deflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
 	return ret;
 }
 
+/* Stateless Decompression Function */
+static int
+process_isal_inflate(struct rte_comp_op *op, struct isal_comp_qp *qp,
+		struct isal_priv_xform *priv_xform)
+{
+	int ret = 0;
+
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;
+
+	/* Initialize decompression state */
+	isal_inflate_init(qp->state);
+
+	/* Set op checksum, none by default */
+	qp->state->crc_flag = priv_xform->decompress.chksum;
+
+	/* Point decompression state structure to input/output buffers */
+	qp->state->avail_in = op->src.length;
+	qp->state->next_in = rte_pktmbuf_mtod(op->m_src, uint8_t *);
+	qp->state->avail_out = op->m_dst->data_len;
+	qp->state->next_out  = rte_pktmbuf_mtod(op->m_dst, uint8_t *);
+
+	if (unlikely(!qp->state->next_in || !qp->state->next_out)) {
+		ISAL_PMD_LOG(ERR, "Invalid source or destination buffers\n");
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		return -1;
+	}
+
+	/* Execute decompression operation */
+	ret = isal_inflate_stateless(qp->state);
+
+	if (ret == ISAL_OUT_OVERFLOW) {
+		ISAL_PMD_LOG(ERR, "Output buffer not big enough\n");
+		op->status = RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED;
+		return ret;
+	}
+
+	/* Check that input buffer has been fully consumed */
+	if (qp->state->avail_in != (uint32_t)0) {
+		ISAL_PMD_LOG(ERR, "Input buffer could not be read entirely\n");
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return -1;
+	}
+
+	if (ret != ISAL_DECOMP_OK) {
+		op->status = RTE_COMP_OP_STATUS_ERROR;
+		return ret;
+	}
+
+	op->consumed = op->src.length - qp->state->avail_in;
+	op->produced = qp->state->total_out;
+
+return ret;
+}
+
 /* Process compression/decompression operation */
 static int
 process_op(struct isal_comp_qp *qp, struct rte_comp_op *op,
@@ -307,6 +361,7 @@ process_op(struct isal_comp_qp *qp, struct rte_comp_op *op,
 		process_isal_deflate(op, qp, priv_xform);
 		break;
 	case RTE_COMP_DECOMPRESS:
+		process_isal_inflate(op, qp, priv_xform);
 		break;
 	default:
 		ISAL_PMD_LOG(ERR, "Operation Not Supported\n");
drivers/compress/isal/isal_compress_pmd_ops.c +8 -0
index a72a886..e8c5298 100644
@@ -164,6 +164,9 @@ isal_comp_pmd_qp_release(struct rte_compressdev *dev, uint16_t qp_id)
 	if (qp->stream->level_buf != NULL)
 		rte_free(qp->stream->level_buf);
 
+	if (qp->state != NULL)
+		rte_free(qp->state);
+
 	if (dev->data->queue_pairs[qp_id] != NULL)
 		rte_free(dev->data->queue_pairs[qp_id]);
 
@@ -242,6 +245,11 @@ isal_comp_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
 			ISAL_DEF_LVL3_DEFAULT, RTE_CACHE_LINE_SIZE,
 			socket_id);
 
+	/* Initialize memory for decompression state structure */
+	qp->state = rte_zmalloc_socket("Isa-l decompression state",
+			sizeof(struct inflate_state), RTE_CACHE_LINE_SIZE,
+			socket_id);
+
 	qp->id = qp_id;
 	dev->data->queue_pairs[qp_id] = qp;
 
drivers/compress/isal/isal_compress_pmd_private.h +2 -0
index 6ba34a6..46e9fcf 100644
@@ -30,6 +30,8 @@ struct isal_comp_qp {
 	struct rte_compressdev_stats qp_stats;
 	/* Compression stream information*/
 	struct isal_zstream *stream;
+	/* Decompression state information*/
+	struct inflate_state *state;
 	/* Number of free elements on ring */
 	uint16_t num_free_elements;
 } __rte_cache_aligned;
Signature · 1 line
2.7.4
author
#68 lore raw

PATCH v4 10/10 doc: add compression driver and ISA-L PMD docs

In reply to Lee Daly

This adds general compression drivers feature guide as well as the ISA-L PMD documentation and guide.

---
 .gitignore                                         |  1 +
 MAINTAINERS                                        | 12 ++++
 devtools/test-build.sh                             |  4 ++
 doc/guides/compressdevs/features/default.ini       | 24 +++++++
 doc/guides/compressdevs/features/isal.ini          | 22 ++++++
 doc/guides/compressdevs/index.rst                  | 13 ++++
 doc/guides/compressdevs/isal.rst                   | 77 ++++++++++++++++++++
 doc/guides/compressdevs/overview.rst               | 12 ++++
 doc/guides/compressdevs/overview_feature_table.txt | 84 ++++++++++++++++++++++
 doc/guides/conf.py                                 |  5 ++
 doc/guides/index.rst                               |  1 +
 doc/guides/rel_notes/release_18_05.rst             |  3 +-
 12 files changed, 257 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/compressdevs/features/default.ini
 create mode 100644 doc/guides/compressdevs/features/isal.ini
 create mode 100644 doc/guides/compressdevs/index.rst
 create mode 100644 doc/guides/compressdevs/isal.rst
 create mode 100644 doc/guides/compressdevs/overview.rst
 create mode 100644 doc/guides/compressdevs/overview_feature_table.txt
.gitignore +1 -0
index 6df5ba0..9105e26 100644
@@ -3,6 +3,7 @@ doc/guides/cryptodevs/overview_feature_table.txt
 doc/guides/cryptodevs/overview_cipher_table.txt
 doc/guides/cryptodevs/overview_auth_table.txt
 doc/guides/cryptodevs/overview_aead_table.txt
+doc/guides/compressdevs/overview_feature_table.txt
 cscope.out.po
 cscope.out.in
 cscope.out
MAINTAINERS +12 -0
index d2dd61c..a22f37f 100644
@@ -766,6 +766,18 @@ F: doc/guides/cryptodevs/zuc.rst
 F: doc/guides/cryptodevs/features/zuc.ini
 
 
+Compression Drivers
+-------------------
+M: Pablo de Lara <[email protected]>
+T: git://dpdk.org/next/dpdk-next-crypto
+F: doc/guides/compressdevs/features/default.ini
+
+ISA-L PMD
+M: Lee Daly <[email protected]>
+F: drivers/compress/isal
+F: doc/guides/compressdevs/isal.rst
+F: doc/guides/compressdevs/features/isal.ini
+
 Eventdev Drivers
 ----------------
 M: Jerin Jacob <[email protected]>
devtools/test-build.sh +4 -0
index 3362edc..66f3ece 100755
@@ -45,6 +45,7 @@ default_path=$PATH
 # - DPDK_DEP_SSL (y/[n])
 # - DPDK_DEP_SZE (y/[n])
 # - DPDK_DEP_ZLIB (y/[n])
+# - DPDK_DEP_ISAL (y/[n])
 # - DPDK_MAKE_JOBS (int)
 # - DPDK_NOTIFY (notify-send)
 # - FLEXRAN_SDK
@@ -129,6 +130,7 @@ reset_env ()
 	unset DPDK_DEP_SSL
 	unset DPDK_DEP_SZE
 	unset DPDK_DEP_ZLIB
+	unset DPDK_DEP_ISAL
 	unset AESNI_MULTI_BUFFER_LIB_PATH
 	unset ARMV8_CRYPTO_LIB_PATH
 	unset FLEXRAN_SDK
@@ -178,6 +180,8 @@ config () # <directory> <target> <options>
 		test "$DPDK_DEP_ZLIB" != y || \
 		sed -ri          's,(BNX2X_PMD=)n,\1y,' $1/.config
 		sed -ri            's,(NFP_PMD=)n,\1y,' $1/.config
+		test "$DPDK_DEP_ISAL" != y || \
+		sed -ri          's,(ISAL_PMD=)n,\1y,' $1/.config
 		test "$DPDK_DEP_PCAP" != y || \
 		sed -ri               's,(PCAP=)n,\1y,' $1/.config
 		test -z "$ARMV8_CRYPTO_LIB_PATH" || \
doc/guides/compressdevs/features/default.ini new file +24 -0
new file mode 100644
index 0000000..795fc55
@@ -0,0 +1,24 @@
+;
+; Features of a default compression driver.
+;
+; This file defines the features that are valid for inclusion in
+; the other driver files and also the order that they appear in
+; the features table in the documentation.
+;
+[Features]
+HW Accelerated =
+CPU SSE        =
+CPU AVX        =
+CPU AVX2       =
+CPU AVX512     =
+CPU NEON       =
+Stateful       =
+By-Pass        =
+Chained mbufs  =
+Deflate        =
+LZS            =
+Adler32        =
+Crc32          =
+Adler32&Crc32  =
+Fixed          =
+Dynamic        =
doc/guides/compressdevs/features/isal.ini new file +22 -0
new file mode 100644
index 0000000..6a6ffb8
@@ -0,0 +1,22 @@
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; Supported features of 'ISA-L' compression driver.
+;
+[Features]
+HW Accelerated =
+CPU SSE        = Y
+CPU AVX        = Y
+CPU AVX2       = Y
+CPU AVX512     = Y
+CPU NEON       =
+Stateful       =
+By-Pass        =
+Chained mbufs  =
+Deflate        = Y
+LZS            =
+Adler32        = Y
+Crc32          = Y
+Adler32&Crc32  =
+Fixed          = Y
+Dynamic        = Y
doc/guides/compressdevs/index.rst new file +13 -0
new file mode 100644
index 0000000..bc59ce8
@@ -0,0 +1,13 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Compression Device Drivers
+==========================
+
+
+.. toctree::
+    :maxdepth: 2
+    :numbered:
+
+    overview
+    isal
doc/guides/compressdevs/isal.rst new file +77 -0

77 lines not on this page · raw message

doc/guides/compressdevs/overview.rst new file +12 -0
new file mode 100644
index 0000000..ca37de1
@@ -0,0 +1,12 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Compression Device Supported Functionality Matrices
+===================================================
+
+Supported Feature Flags
+-----------------------
+
+.. _table_compression_pmd_features:
+
+.. include:: overview_feature_table.txt
doc/guides/compressdevs/overview_feature_table.txt new file +84 -0

84 lines not on this page · raw message

doc/guides/conf.py +5 -0
index 38de280..c883306 100644
@@ -388,6 +388,11 @@ def setup(app):
                             'AEAD',
                             'AEAD algorithms in crypto drivers',
                             'AEAD algorithm')
+    table_file = dirname(__file__) + '/compressdevs/overview_feature_table.txt'
+    generate_overview_table(table_file, 1,
+                            'Features',
+                            'Features availability in compression drivers',
+                            'Feature')
 
     if LooseVersion(sphinx_version) < LooseVersion('1.3.1'):
         print('Upgrade sphinx to version >= 1.3.1 for '
doc/guides/index.rst +1 -0
index d60529d..18fe0ec 100644
@@ -17,6 +17,7 @@ DPDK documentation
    nics/index
    bbdevs/index
    cryptodevs/index
+   compressdevs/index
    eventdevs/index
    mempool/index
    platform/index
doc/guides/rel_notes/release_18_05.rst +2 -1
index ce70d1f..d6a9d6f 100644
@@ -78,7 +78,8 @@ New Features
 * **Added a new compression poll mode driver using Intels ISA-L.**
 
    Added the new ``ISA-L`` compression driver, for compression and decompression
-   operations in software.
+   operations in software. See the :doc:`../compressdevs/isal` compression driver
+   guide for details on this new driver.
 
 
 API Changes
Signature · 1 line
2.7.4
#69 lore raw

Re: [PATCH v4 10/10] doc: add compression driver and ISA-L PMD docs

Quoted 15 lines

-----Original Message-----
From: Daly, Lee
Sent: Saturday, April 28, 2018 12:39 AM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Tucker, Greg B <[email protected]>; Jain, Deepak K <[email protected]>; Trahe, Fiona <[email protected]>; Daly, Lee <[email protected]>
Subject: [PATCH v4 10/10] doc: add compression driver and ISA-L PMD docs

This adds general compression drivers feature guide as well as the ISA-L PMD documentation and guide.

Quoted 10 lines
---
 .gitignore                                         |  1 +
 MAINTAINERS                                        | 12 ++++
 devtools/test-build.sh                             |  4 ++
 doc/guides/compressdevs/features/default.ini       | 24 +++++++
 doc/guides/compressdevs/features/isal.ini          | 22 ++++++
 doc/guides/compressdevs/index.rst                  | 13 ++++
 doc/guides/compressdevs/isal.rst                   | 77 ++++++++++++++++++++
 doc/guides/compressdevs/overview.rst               | 12 ++++
 doc/guides/compressdevs/overview_feature_table.txt | 84

Overview_feature_table.txt is generated with conf.py, so it doesn't have to be upstreamed.