Skip to contentSkip to conversation

Introduce UCAP API and usage in mlx5

PATCH Applied Leon Romanovsky 3 participants 14 messages Mar 6, 2025 – Mar 8, 2025

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover Introduce UCAP API and usage in mlx5

Changelog:
v1:
* Used kref primitives instead of open-coded variant
* Check return value from dev_set_name()
* Added extra brackets around type in UCAP_ENABLED macro
v0: https://lore.kernel.org/all/[email protected]

--------------------------------------------------------------------------
This series introduces the User CAPability (UCAP) API that allows creating user contexts with various firmware privileges.

The UCAP API provides fine-grained control over specific firmware features by representing each capability as a character device with root read-write access. Root processes can grant users special privileges by allowing access to these character devices. User contexts created using a file descriptor of a UCAP will have specific UCAP privileges.

Two UCAP character devices are created for mlx5, and user contexts opened with at least one of these UCAPs are considered privileged. To ensure that privileged commands can always proceed, non-privileged commands are limited when a privileged user is present on the device.

Thanks

Chiara Meiohas (5):
RDMA/uverbs: Introduce UCAP (User CAPabilities) API RDMA/mlx5: Create UCAP char devices for supported device capabilities RDMA/uverbs: Add support for UCAPs in context creation RDMA/mlx5: Check enabled UCAPs when creating ucontext
docs: infiniband: document the UCAP API

Patrisious Haddad (1):
RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace

 Documentation/infiniband/index.rst            |   1 +
 Documentation/infiniband/ucaps.rst            |  71 +++++
 drivers/infiniband/core/Makefile              |   3 +-
 drivers/infiniband/core/ucaps.c               | 267 ++++++++++++++++++
 drivers/infiniband/core/uverbs_cmd.c          |  19 ++
 drivers/infiniband/core/uverbs_main.c         |   2 +
 .../infiniband/core/uverbs_std_types_device.c |   4 +
 drivers/infiniband/hw/mlx5/devx.c             |  31 +-
 drivers/infiniband/hw/mlx5/devx.h             |   5 +-
 drivers/infiniband/hw/mlx5/fs.c               | 154 +++++++++-
 drivers/infiniband/hw/mlx5/fs.h               |   2 +
 drivers/infiniband/hw/mlx5/main.c             |  77 ++++-
 drivers/infiniband/hw/mlx5/mlx5_ib.h          |   3 +
 include/rdma/ib_ucaps.h                       |  25 ++
 include/rdma/ib_verbs.h                       |   1 +
 include/uapi/rdma/ib_user_ioctl_cmds.h        |   1 +
 include/uapi/rdma/mlx5_user_ioctl_cmds.h      |   1 +
 include/uapi/rdma/mlx5_user_ioctl_verbs.h     |   2 +
 18 files changed, 647 insertions(+), 22 deletions(-)
 create mode 100644 Documentation/infiniband/ucaps.rst
 create mode 100644 drivers/infiniband/core/ucaps.c
 create mode 100644 include/rdma/ib_ucaps.h
Signature · 1 line
2.48.1
#1 lore raw

PATCH 1/6 RDMA/uverbs: Introduce UCAP (User CAPabilities) API

From: Chiara Meiohas <[email protected]>

Implement a new User CAPabilities (UCAP) API to provide fine-grained control over specific firmware features.

This approach offers more granular capabilities than the existing Linux capabilities, which may be too generic for certain FW features.

This mechanism represents each capability as a character device with root read-write access. Root processes can grant users special privileges by allowing access to these character devices (e.g., using chown).

UCAP character devices are located in /dev/infiniband and the class path is /sys/class/infiniband_ucaps.

---
 drivers/infiniband/core/Makefile      |   3 +-
 drivers/infiniband/core/ucaps.c       | 267 ++++++++++++++++++++++++++
 drivers/infiniband/core/uverbs_main.c |   2 +
 include/rdma/ib_ucaps.h               |  25 +++
 4 files changed, 296 insertions(+), 1 deletion(-)
 create mode 100644 drivers/infiniband/core/ucaps.c
 create mode 100644 include/rdma/ib_ucaps.h
drivers/infiniband/core/Makefile +2 -1
index 8ab4eea5a0a5..d49ded7e95f0 100644
@@ -39,6 +39,7 @@ ib_uverbs-y :=			uverbs_main.o uverbs_cmd.o uverbs_marshall.o \
 				uverbs_std_types_async_fd.o \
 				uverbs_std_types_srq.o \
 				uverbs_std_types_wq.o \
-				uverbs_std_types_qp.o
+				uverbs_std_types_qp.o \
+				ucaps.o
 ib_uverbs-$(CONFIG_INFINIBAND_USER_MEM) += umem.o umem_dmabuf.o
 ib_uverbs-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += umem_odp.o
drivers/infiniband/core/ucaps.c new file +267 -0

267 lines not on this page · raw message

drivers/infiniband/core/uverbs_main.c +2 -0
index 85cfc790a7bb..973fe2c7ef53 100644
@@ -52,6 +52,7 @@
 #include <rdma/ib.h>
 #include <rdma/uverbs_std_types.h>
 #include <rdma/rdma_netlink.h>
+#include <rdma/ib_ucaps.h>
 
 #include "uverbs.h"
 #include "core_priv.h"
@@ -1345,6 +1346,7 @@ static void __exit ib_uverbs_cleanup(void)
 				 IB_UVERBS_NUM_FIXED_MINOR);
 	unregister_chrdev_region(dynamic_uverbs_dev,
 				 IB_UVERBS_NUM_DYNAMIC_MINOR);
+	ib_cleanup_ucaps();
 	mmu_notifier_synchronize();
 }
 
include/rdma/ib_ucaps.h new file +25 -0
new file mode 100644
index 000000000000..8f0552a2b2b0
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/*
+ * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ */
+
+#ifndef _IB_UCAPS_H_
+#define _IB_UCAPS_H_
+
+#define UCAP_ENABLED(ucaps, type) (!!((ucaps) & (1U << (type))))
+
+enum rdma_user_cap {
+	RDMA_UCAP_MLX5_CTRL_LOCAL,
+	RDMA_UCAP_MLX5_CTRL_OTHER_VHCA,
+	RDMA_UCAP_MAX
+};
+
+void ib_cleanup_ucaps(void);
+
+int ib_create_ucap(enum rdma_user_cap type);
+
+void ib_remove_ucap(enum rdma_user_cap type);
+
+int ib_get_ucaps(int *fds, int fd_count, uint64_t *idx_mask);
+
+#endif /* _IB_UCAPS_H_ */
Signature · 1 line
2.48.1
Reviewed-by on v1 1/6
#2 lore raw

Re: [PATCH rdma-next v1 1/6] RDMA/uverbs: Introduce UCAP (User CAPabilities) API

Quoted 5 lines · Leon Romanovsky

From: Chiara Meiohas <[email protected]>

Implement a new User CAPabilities (UCAP) API to provide fine-grained control over specific firmware features.

Quoted 7 lines

This approach offers more granular capabilities than the existing Linux capabilities, which may be too generic for certain FW features.

This mechanism represents each capability as a character device with root read-write access. Root processes can grant users special privileges by allowing access to these character devices (e.g., using chown).

Hi, Chiara

I read this patch-set carefully. If I get this patch-set correctly, this patch-set introduces a new User CAPabilities API to control specific firmware feature. Do we have a user guide to use this UCAP? For example, we suspect that a Firmware problem will occur in production environment, how can we use this UCAP to debug this Firmware problem?

Thanks.

Zhu Yanjun

Quoted 353 lines

UCAP character devices are located in /dev/infiniband and the class path is /sys/class/infiniband_ucaps.

---
  drivers/infiniband/core/Makefile      |   3 +-
  drivers/infiniband/core/ucaps.c       | 267 ++++++++++++++++++++++++++
  drivers/infiniband/core/uverbs_main.c |   2 +
  include/rdma/ib_ucaps.h               |  25 +++
  4 files changed, 296 insertions(+), 1 deletion(-)

create mode 100644 drivers/infiniband/core/ucaps.c create mode 100644 include/rdma/ib_ucaps.h

drivers/infiniband/core/Makefile +2 -1
index 8ab4eea5a0a5..d49ded7e95f0 100644
@@ -39,6 +39,7 @@ ib_uverbs-y :=			uverbs_main.o uverbs_cmd.o uverbs_marshall.o \
  				uverbs_std_types_async_fd.o \
  				uverbs_std_types_srq.o \
  				uverbs_std_types_wq.o \
-				uverbs_std_types_qp.o
+				uverbs_std_types_qp.o \
+				ucaps.o
  ib_uverbs-$(CONFIG_INFINIBAND_USER_MEM) += umem.o umem_dmabuf.o
  ib_uverbs-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += umem_odp.o
drivers/infiniband/core/ucaps.c new file +267 -0
new file mode 100644
index 000000000000..6853c6d078f9
@@ -0,0 +1,267 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/*
+ * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ */
+
+#include <linux/kref.h>
+#include <linux/cdev.h>
+#include <linux/mutex.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <rdma/ib_ucaps.h>
+
+#define RDMA_UCAP_FIRST RDMA_UCAP_MLX5_CTRL_LOCAL
+
+static DEFINE_MUTEX(ucaps_mutex);
+static struct ib_ucap *ucaps_list[RDMA_UCAP_MAX];
+static bool ucaps_class_is_registered;
+static dev_t ucaps_base_dev;
+
+struct ib_ucap {
+	struct cdev cdev;
+	struct device dev;
+	struct kref ref;
+};
+
+static const char *ucap_names[RDMA_UCAP_MAX] = {
+	[RDMA_UCAP_MLX5_CTRL_LOCAL] = "mlx5_perm_ctrl_local",
+	[RDMA_UCAP_MLX5_CTRL_OTHER_VHCA] = "mlx5_perm_ctrl_other_vhca"
+};
+
+static char *ucaps_devnode(const struct device *dev, umode_t *mode)
+{
+	if (mode)
+		*mode = 0600;
+
+	return kasprintf(GFP_KERNEL, "infiniband/%s", dev_name(dev));
+}
+
+static const struct class ucaps_class = {
+	.name = "infiniband_ucaps",
+	.devnode = ucaps_devnode,
+};
+
+static const struct file_operations ucaps_cdev_fops = {
+	.owner = THIS_MODULE,
+	.open = simple_open,
+};
+
+/**
+ * ib_cleanup_ucaps - cleanup all API resources and class.
+ *
+ * This is called once, when removing the ib_uverbs module.
+ */
+void ib_cleanup_ucaps(void)
+{
+	mutex_lock(&ucaps_mutex);
+	if (!ucaps_class_is_registered) {
+		mutex_unlock(&ucaps_mutex);
+		return;
+	}
+
+	for (int i = RDMA_UCAP_FIRST; i < RDMA_UCAP_MAX; i++)
+		WARN_ON(ucaps_list[i]);
+
+	class_unregister(&ucaps_class);
+	ucaps_class_is_registered = false;
+	unregister_chrdev_region(ucaps_base_dev, RDMA_UCAP_MAX);
+	mutex_unlock(&ucaps_mutex);
+}
+
+static int get_ucap_from_devt(dev_t devt, u64 *idx_mask)
+{
+	for (int type = RDMA_UCAP_FIRST; type < RDMA_UCAP_MAX; type++) {
+		if (ucaps_list[type] && ucaps_list[type]->dev.devt == devt) {
+			*idx_mask |= 1 << type;
+			return 0;
+		}
+	}
+
+	return -EINVAL;
+}
+
+static int get_devt_from_fd(unsigned int fd, dev_t *ret_dev)
+{
+	struct file *file;
+
+	file = fget(fd);
+	if (!file)
+		return -EBADF;
+
+	*ret_dev = file_inode(file)->i_rdev;
+	fput(file);
+	return 0;
+}
+
+/**
+ * ib_ucaps_init - Initialization required before ucap creation.
+ *
+ * Return: 0 on success, or a negative errno value on failure
+ */
+static int ib_ucaps_init(void)
+{
+	int ret = 0;
+
+	if (ucaps_class_is_registered)
+		return ret;
+
+	ret = class_register(&ucaps_class);
+	if (ret)
+		return ret;
+
+	ret = alloc_chrdev_region(&ucaps_base_dev, 0, RDMA_UCAP_MAX,
+				  ucaps_class.name);
+	if (ret < 0) {
+		class_unregister(&ucaps_class);
+		return ret;
+	}
+
+	ucaps_class_is_registered = true;
+
+	return 0;
+}
+
+static void ucap_dev_release(struct device *device)
+{
+	struct ib_ucap *ucap = container_of(device, struct ib_ucap, dev);
+
+	kfree(ucap);
+}
+
+/**
+ * ib_create_ucap - Add a ucap character device
+ * @type: UCAP type
+ *
+ * Creates a ucap character device in the /dev/infiniband directory. By default,
+ * the device has root-only read-write access.
+ *
+ * A driver may call this multiple times with the same UCAP type. A reference
+ * count tracks creations and deletions.
+ *
+ * Return: 0 on success, or a negative errno value on failure
+ */
+int ib_create_ucap(enum rdma_user_cap type)
+{
+	struct ib_ucap *ucap;
+	int ret;
+
+	if (type >= RDMA_UCAP_MAX)
+		return -EINVAL;
+
+	mutex_lock(&ucaps_mutex);
+	ret = ib_ucaps_init();
+	if (ret)
+		goto unlock;
+
+	ucap = ucaps_list[type];
+	if (ucap) {
+		kref_get(&ucap->ref);
+		mutex_unlock(&ucaps_mutex);
+		return 0;
+	}
+
+	ucap = kzalloc(sizeof(*ucap), GFP_KERNEL);
+	if (!ucap) {
+		ret = -ENOMEM;
+		goto unlock;
+	}
+
+	device_initialize(&ucap->dev);
+	ucap->dev.class = &ucaps_class;
+	ucap->dev.devt = MKDEV(MAJOR(ucaps_base_dev), type);
+	ucap->dev.release = ucap_dev_release;
+	ret = dev_set_name(&ucap->dev, ucap_names[type]);
+	if (ret)
+		goto err_device;
+
+	cdev_init(&ucap->cdev, &ucaps_cdev_fops);
+	ucap->cdev.owner = THIS_MODULE;
+
+	ret = cdev_device_add(&ucap->cdev, &ucap->dev);
+	if (ret)
+		goto err_device;
+
+	kref_init(&ucap->ref);
+	ucaps_list[type] = ucap;
+	mutex_unlock(&ucaps_mutex);
+
+	return 0;
+
+err_device:
+	put_device(&ucap->dev);
+unlock:
+	mutex_unlock(&ucaps_mutex);
+	return ret;
+}
+EXPORT_SYMBOL(ib_create_ucap);
+
+static void ib_release_ucap(struct kref *ref)
+{
+	struct ib_ucap *ucap = container_of(ref, struct ib_ucap, ref);
+	enum rdma_user_cap type;
+
+	for (type = RDMA_UCAP_FIRST; type < RDMA_UCAP_MAX; type++) {
+		if (ucaps_list[type] == ucap)
+			break;
+	}
+	WARN_ON(type == RDMA_UCAP_MAX);
+
+	ucaps_list[type] = NULL;
+	cdev_device_del(&ucap->cdev, &ucap->dev);
+	put_device(&ucap->dev);
+}
+
+/**
+ * ib_remove_ucap - Remove a ucap character device
+ * @type: User cap type
+ *
+ * Removes the ucap character device according to type. The device is completely
+ * removed from the filesystem when its reference count reaches 0.
+ */
+void ib_remove_ucap(enum rdma_user_cap type)
+{
+	struct ib_ucap *ucap;
+
+	mutex_lock(&ucaps_mutex);
+	ucap = ucaps_list[type];
+	if (WARN_ON(!ucap))
+		goto end;
+
+	kref_put(&ucap->ref, ib_release_ucap);
+end:
+	mutex_unlock(&ucaps_mutex);
+}
+EXPORT_SYMBOL(ib_remove_ucap);
+
+/**
+ * ib_get_ucaps - Get bitmask of ucap types from file descriptors
+ * @fds: Array of file descriptors
+ * @fd_count: Number of file descriptors in the array
+ * @idx_mask: Bitmask to be updated based on the ucaps in the fd list
+ *
+ * Given an array of file descriptors, this function returns a bitmask of
+ * the ucaps where a bit is set if an FD for that ucap type was in the array.
+ *
+ * Return: 0 on success, or a negative errno value on failure
+ */
+int ib_get_ucaps(int *fds, int fd_count, uint64_t *idx_mask)
+{
+	int ret = 0;
+	dev_t dev;
+
+	*idx_mask = 0;
+	mutex_lock(&ucaps_mutex);
+	for (int i = 0; i < fd_count; i++) {
+		ret = get_devt_from_fd(fds[i], &dev);
+		if (ret)
+			goto end;
+
+		ret = get_ucap_from_devt(dev, idx_mask);
+		if (ret)
+			goto end;
+	}
+
+end:
+	mutex_unlock(&ucaps_mutex);
+	return ret;
+}
drivers/infiniband/core/uverbs_main.c +2 -0
index 85cfc790a7bb..973fe2c7ef53 100644
@@ -52,6 +52,7 @@
  #include <rdma/ib.h>
  #include <rdma/uverbs_std_types.h>
  #include <rdma/rdma_netlink.h>
+#include <rdma/ib_ucaps.h>
 
  #include "uverbs.h"
  #include "core_priv.h"
@@ -1345,6 +1346,7 @@ static void __exit ib_uverbs_cleanup(void)
  				 IB_UVERBS_NUM_FIXED_MINOR);
  	unregister_chrdev_region(dynamic_uverbs_dev,
  				 IB_UVERBS_NUM_DYNAMIC_MINOR);
+	ib_cleanup_ucaps();
  	mmu_notifier_synchronize();
  }
 
include/rdma/ib_ucaps.h new file +25 -0
new file mode 100644
index 000000000000..8f0552a2b2b0
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/*
+ * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ */
+
+#ifndef _IB_UCAPS_H_
+#define _IB_UCAPS_H_
+
+#define UCAP_ENABLED(ucaps, type) (!!((ucaps) & (1U << (type))))
+
+enum rdma_user_cap {
+	RDMA_UCAP_MLX5_CTRL_LOCAL,
+	RDMA_UCAP_MLX5_CTRL_OTHER_VHCA,
+	RDMA_UCAP_MAX
+};
+
+void ib_cleanup_ucaps(void);
+
+int ib_create_ucap(enum rdma_user_cap type);
+
+void ib_remove_ucap(enum rdma_user_cap type);
+
+int ib_get_ucaps(int *fds, int fd_count, uint64_t *idx_mask);
+
+#endif /* _IB_UCAPS_H_ */
#3 lore raw

Re: [PATCH rdma-next v1 1/6] RDMA/uverbs: Introduce UCAP (User CAPabilities) API

Quoted 14 lines · Zhu Yanjun
Quoted 5 lines · Leon Romanovsky

From: Chiara Meiohas <[email protected]>

Implement a new User CAPabilities (UCAP) API to provide fine-grained control over specific firmware features.

Quoted 7 lines

This approach offers more granular capabilities than the existing Linux capabilities, which may be too generic for certain FW features.

This mechanism represents each capability as a character device with root read-write access. Root processes can grant users special privileges by allowing access to these character devices (e.g., using chown).

Quoted 8 lines

Hi, Chiara

I read this patch-set carefully. If I get this patch-set correctly, this patch-set introduces a new User CAPabilities API to control specific firmware feature.
Do we have a user guide to use this UCAP? For example, we suspect that a Firmware problem will occur in production environment, how can we use this UCAP to debug this Firmware problem?

It is not helpful for debugging FW as these capabilities are needed when you are opening ucontext. Your FW shouldn't get an access from one ucontext access to others.

These UCAPs are used when administrator wants to control specific FW feature by separating between userA(containerA) and userB(containerB), which are running on same machine with same FW.

The better debug interface is provided by FWCTL, use it:
https://lore.kernel.org/linux-rdma/[email protected]/

Thanks

Quoted 359 lines

Thanks.

Zhu Yanjun

Quoted 351 lines

UCAP character devices are located in /dev/infiniband and the class path is /sys/class/infiniband_ucaps.

---
  drivers/infiniband/core/Makefile      |   3 +-
  drivers/infiniband/core/ucaps.c       | 267 ++++++++++++++++++++++++++
  drivers/infiniband/core/uverbs_main.c |   2 +
  include/rdma/ib_ucaps.h               |  25 +++
  4 files changed, 296 insertions(+), 1 deletion(-)

create mode 100644 drivers/infiniband/core/ucaps.c create mode 100644 include/rdma/ib_ucaps.h

drivers/infiniband/core/Makefile +2 -1
index 8ab4eea5a0a5..d49ded7e95f0 100644
@@ -39,6 +39,7 @@ ib_uverbs-y :=			uverbs_main.o uverbs_cmd.o uverbs_marshall.o \
  				uverbs_std_types_async_fd.o \
  				uverbs_std_types_srq.o \
  				uverbs_std_types_wq.o \
-				uverbs_std_types_qp.o
+				uverbs_std_types_qp.o \
+				ucaps.o
  ib_uverbs-$(CONFIG_INFINIBAND_USER_MEM) += umem.o umem_dmabuf.o
  ib_uverbs-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += umem_odp.o
drivers/infiniband/core/ucaps.c new file +267 -0
new file mode 100644
index 000000000000..6853c6d078f9
@@ -0,0 +1,267 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/*
+ * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ */
+
+#include <linux/kref.h>
+#include <linux/cdev.h>
+#include <linux/mutex.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <rdma/ib_ucaps.h>
+
+#define RDMA_UCAP_FIRST RDMA_UCAP_MLX5_CTRL_LOCAL
+
+static DEFINE_MUTEX(ucaps_mutex);
+static struct ib_ucap *ucaps_list[RDMA_UCAP_MAX];
+static bool ucaps_class_is_registered;
+static dev_t ucaps_base_dev;
+
+struct ib_ucap {
+	struct cdev cdev;
+	struct device dev;
+	struct kref ref;
+};
+
+static const char *ucap_names[RDMA_UCAP_MAX] = {
+	[RDMA_UCAP_MLX5_CTRL_LOCAL] = "mlx5_perm_ctrl_local",
+	[RDMA_UCAP_MLX5_CTRL_OTHER_VHCA] = "mlx5_perm_ctrl_other_vhca"
+};
+
+static char *ucaps_devnode(const struct device *dev, umode_t *mode)
+{
+	if (mode)
+		*mode = 0600;
+
+	return kasprintf(GFP_KERNEL, "infiniband/%s", dev_name(dev));
+}
+
+static const struct class ucaps_class = {
+	.name = "infiniband_ucaps",
+	.devnode = ucaps_devnode,
+};
+
+static const struct file_operations ucaps_cdev_fops = {
+	.owner = THIS_MODULE,
+	.open = simple_open,
+};
+
+/**
+ * ib_cleanup_ucaps - cleanup all API resources and class.
+ *
+ * This is called once, when removing the ib_uverbs module.
+ */
+void ib_cleanup_ucaps(void)
+{
+	mutex_lock(&ucaps_mutex);
+	if (!ucaps_class_is_registered) {
+		mutex_unlock(&ucaps_mutex);
+		return;
+	}
+
+	for (int i = RDMA_UCAP_FIRST; i < RDMA_UCAP_MAX; i++)
+		WARN_ON(ucaps_list[i]);
+
+	class_unregister(&ucaps_class);
+	ucaps_class_is_registered = false;
+	unregister_chrdev_region(ucaps_base_dev, RDMA_UCAP_MAX);
+	mutex_unlock(&ucaps_mutex);
+}
+
+static int get_ucap_from_devt(dev_t devt, u64 *idx_mask)
+{
+	for (int type = RDMA_UCAP_FIRST; type < RDMA_UCAP_MAX; type++) {
+		if (ucaps_list[type] && ucaps_list[type]->dev.devt == devt) {
+			*idx_mask |= 1 << type;
+			return 0;
+		}
+	}
+
+	return -EINVAL;
+}
+
+static int get_devt_from_fd(unsigned int fd, dev_t *ret_dev)
+{
+	struct file *file;
+
+	file = fget(fd);
+	if (!file)
+		return -EBADF;
+
+	*ret_dev = file_inode(file)->i_rdev;
+	fput(file);
+	return 0;
+}
+
+/**
+ * ib_ucaps_init - Initialization required before ucap creation.
+ *
+ * Return: 0 on success, or a negative errno value on failure
+ */
+static int ib_ucaps_init(void)
+{
+	int ret = 0;
+
+	if (ucaps_class_is_registered)
+		return ret;
+
+	ret = class_register(&ucaps_class);
+	if (ret)
+		return ret;
+
+	ret = alloc_chrdev_region(&ucaps_base_dev, 0, RDMA_UCAP_MAX,
+				  ucaps_class.name);
+	if (ret < 0) {
+		class_unregister(&ucaps_class);
+		return ret;
+	}
+
+	ucaps_class_is_registered = true;
+
+	return 0;
+}
+
+static void ucap_dev_release(struct device *device)
+{
+	struct ib_ucap *ucap = container_of(device, struct ib_ucap, dev);
+
+	kfree(ucap);
+}
+
+/**
+ * ib_create_ucap - Add a ucap character device
+ * @type: UCAP type
+ *
+ * Creates a ucap character device in the /dev/infiniband directory. By default,
+ * the device has root-only read-write access.
+ *
+ * A driver may call this multiple times with the same UCAP type. A reference
+ * count tracks creations and deletions.
+ *
+ * Return: 0 on success, or a negative errno value on failure
+ */
+int ib_create_ucap(enum rdma_user_cap type)
+{
+	struct ib_ucap *ucap;
+	int ret;
+
+	if (type >= RDMA_UCAP_MAX)
+		return -EINVAL;
+
+	mutex_lock(&ucaps_mutex);
+	ret = ib_ucaps_init();
+	if (ret)
+		goto unlock;
+
+	ucap = ucaps_list[type];
+	if (ucap) {
+		kref_get(&ucap->ref);
+		mutex_unlock(&ucaps_mutex);
+		return 0;
+	}
+
+	ucap = kzalloc(sizeof(*ucap), GFP_KERNEL);
+	if (!ucap) {
+		ret = -ENOMEM;
+		goto unlock;
+	}
+
+	device_initialize(&ucap->dev);
+	ucap->dev.class = &ucaps_class;
+	ucap->dev.devt = MKDEV(MAJOR(ucaps_base_dev), type);
+	ucap->dev.release = ucap_dev_release;
+	ret = dev_set_name(&ucap->dev, ucap_names[type]);
+	if (ret)
+		goto err_device;
+
+	cdev_init(&ucap->cdev, &ucaps_cdev_fops);
+	ucap->cdev.owner = THIS_MODULE;
+
+	ret = cdev_device_add(&ucap->cdev, &ucap->dev);
+	if (ret)
+		goto err_device;
+
+	kref_init(&ucap->ref);
+	ucaps_list[type] = ucap;
+	mutex_unlock(&ucaps_mutex);
+
+	return 0;
+
+err_device:
+	put_device(&ucap->dev);
+unlock:
+	mutex_unlock(&ucaps_mutex);
+	return ret;
+}
+EXPORT_SYMBOL(ib_create_ucap);
+
+static void ib_release_ucap(struct kref *ref)
+{
+	struct ib_ucap *ucap = container_of(ref, struct ib_ucap, ref);
+	enum rdma_user_cap type;
+
+	for (type = RDMA_UCAP_FIRST; type < RDMA_UCAP_MAX; type++) {
+		if (ucaps_list[type] == ucap)
+			break;
+	}
+	WARN_ON(type == RDMA_UCAP_MAX);
+
+	ucaps_list[type] = NULL;
+	cdev_device_del(&ucap->cdev, &ucap->dev);
+	put_device(&ucap->dev);
+}
+
+/**
+ * ib_remove_ucap - Remove a ucap character device
+ * @type: User cap type
+ *
+ * Removes the ucap character device according to type. The device is completely
+ * removed from the filesystem when its reference count reaches 0.
+ */
+void ib_remove_ucap(enum rdma_user_cap type)
+{
+	struct ib_ucap *ucap;
+
+	mutex_lock(&ucaps_mutex);
+	ucap = ucaps_list[type];
+	if (WARN_ON(!ucap))
+		goto end;
+
+	kref_put(&ucap->ref, ib_release_ucap);
+end:
+	mutex_unlock(&ucaps_mutex);
+}
+EXPORT_SYMBOL(ib_remove_ucap);
+
+/**
+ * ib_get_ucaps - Get bitmask of ucap types from file descriptors
+ * @fds: Array of file descriptors
+ * @fd_count: Number of file descriptors in the array
+ * @idx_mask: Bitmask to be updated based on the ucaps in the fd list
+ *
+ * Given an array of file descriptors, this function returns a bitmask of
+ * the ucaps where a bit is set if an FD for that ucap type was in the array.
+ *
+ * Return: 0 on success, or a negative errno value on failure
+ */
+int ib_get_ucaps(int *fds, int fd_count, uint64_t *idx_mask)
+{
+	int ret = 0;
+	dev_t dev;
+
+	*idx_mask = 0;
+	mutex_lock(&ucaps_mutex);
+	for (int i = 0; i < fd_count; i++) {
+		ret = get_devt_from_fd(fds[i], &dev);
+		if (ret)
+			goto end;
+
+		ret = get_ucap_from_devt(dev, idx_mask);
+		if (ret)
+			goto end;
+	}
+
+end:
+	mutex_unlock(&ucaps_mutex);
+	return ret;
+}
drivers/infiniband/core/uverbs_main.c +2 -0
index 85cfc790a7bb..973fe2c7ef53 100644
@@ -52,6 +52,7 @@
  #include <rdma/ib.h>
  #include <rdma/uverbs_std_types.h>
  #include <rdma/rdma_netlink.h>
+#include <rdma/ib_ucaps.h>
  #include "uverbs.h"
  #include "core_priv.h"
@@ -1345,6 +1346,7 @@ static void __exit ib_uverbs_cleanup(void)
  				 IB_UVERBS_NUM_FIXED_MINOR);
  	unregister_chrdev_region(dynamic_uverbs_dev,
  				 IB_UVERBS_NUM_DYNAMIC_MINOR);
+	ib_cleanup_ucaps();
  	mmu_notifier_synchronize();
  }
include/rdma/ib_ucaps.h new file +25 -0
new file mode 100644
index 000000000000..8f0552a2b2b0
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/*
+ * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ */
+
+#ifndef _IB_UCAPS_H_
+#define _IB_UCAPS_H_
+
+#define UCAP_ENABLED(ucaps, type) (!!((ucaps) & (1U << (type))))
+
+enum rdma_user_cap {
+	RDMA_UCAP_MLX5_CTRL_LOCAL,
+	RDMA_UCAP_MLX5_CTRL_OTHER_VHCA,
+	RDMA_UCAP_MAX
+};
+
+void ib_cleanup_ucaps(void);
+
+int ib_create_ucap(enum rdma_user_cap type);
+
+void ib_remove_ucap(enum rdma_user_cap type);
+
+int ib_get_ucaps(int *fds, int fd_count, uint64_t *idx_mask);
+
+#endif /* _IB_UCAPS_H_ */
#4 lore raw

PATCH 2/6 RDMA/mlx5: Create UCAP char devices for supported device capabilities

In reply to Leon Romanovsky

From: Chiara Meiohas <[email protected]>

Create UCAP character devices when probing an IB device with supported firmware capabilities.

If the RDMA_CTRL general object type is supported, check for specific UCTX capabilities:
Create /dev/infiniband/mlx5_perm_ctrl_local for RDMA_UCAP_MLX5_CTRL_LOCAL Create /dev/infiniband/mlx5_perm_ctrl_other_vhca for RDMA_UCAP_MLX5_CTRL_OTHER_VHCA

---
 drivers/infiniband/hw/mlx5/main.c | 47 +++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)
drivers/infiniband/hw/mlx5/main.c +47 -0
index 81849eb671a1..04b489a6a449 100644
@@ -47,6 +47,7 @@
 #include <rdma/uverbs_ioctl.h>
 #include <rdma/mlx5_user_ioctl_verbs.h>
 #include <rdma/mlx5_user_ioctl_cmds.h>
+#include <rdma/ib_ucaps.h>
 #include "macsec.h"
 #include "data_direct.h"
 
@@ -4201,8 +4202,47 @@ static int mlx5_ib_init_var_table(struct mlx5_ib_dev *dev)
 	return (var_table->bitmap) ? 0 : -ENOMEM;
 }
 
+static void mlx5_ib_cleanup_ucaps(struct mlx5_ib_dev *dev)
+{
+	if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
+		ib_remove_ucap(RDMA_UCAP_MLX5_CTRL_LOCAL);
+
+	if (MLX5_CAP_GEN(dev->mdev, uctx_cap) &
+	    MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA)
+		ib_remove_ucap(RDMA_UCAP_MLX5_CTRL_OTHER_VHCA);
+}
+
+static int mlx5_ib_init_ucaps(struct mlx5_ib_dev *dev)
+{
+	int ret;
+
+	if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL) {
+		ret = ib_create_ucap(RDMA_UCAP_MLX5_CTRL_LOCAL);
+		if (ret)
+			return ret;
+	}
+
+	if (MLX5_CAP_GEN(dev->mdev, uctx_cap) &
+	    MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA) {
+		ret = ib_create_ucap(RDMA_UCAP_MLX5_CTRL_OTHER_VHCA);
+		if (ret)
+			goto remove_local;
+	}
+
+	return 0;
+
+remove_local:
+	if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
+		ib_remove_ucap(RDMA_UCAP_MLX5_CTRL_LOCAL);
+	return ret;
+}
+
 static void mlx5_ib_stage_caps_cleanup(struct mlx5_ib_dev *dev)
 {
+	if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
+	    MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL)
+		mlx5_ib_cleanup_ucaps(dev);
+
 	bitmap_free(dev->var_table.bitmap);
 }
 
@@ -4253,6 +4293,13 @@ static int mlx5_ib_stage_caps_init(struct mlx5_ib_dev *dev)
 			return err;
 	}
 
+	if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
+	    MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL) {
+		err = mlx5_ib_init_ucaps(dev);
+		if (err)
+			return err;
+	}
+
 	dev->ib_dev.use_cq_dim = true;
 
 	return 0;
Signature · 1 line
2.48.1
#5 lore raw

Re: [PATCH rdma-next v1 2/6] RDMA/mlx5: Create UCAP char devices for supported device capabilities

kernel test robot noticed the following build errors:

Hi Leon,

kernel test robot noticed the following build errors:

[auto build test ERROR on rdma/for-next]
[also build test ERROR on linus/master v6.14-rc5 next-20250307] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Leon-Romanovsky/RDMA-uverbs-Introduce-UCAP-User-CAPabilities-API/20250306-195708
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git for-next
patch link:    https://lore.kernel.org/r/30ed40e7a12a694cf4ee257459ed61b145b7837d.1741261611.git.leon%40kernel.org
patch subject: [PATCH rdma-next v1 2/6] RDMA/mlx5: Create UCAP char devices for supported device capabilities
config: arm64-allmodconfig (https://download.01.org/0day-ci/archive/20250308/[email protected]/config)
compiler: clang version 18.1.8 (https://github.com/llvm/llvm-project 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250308/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

>> drivers/infiniband/hw/mlx5/main.c:4207:42: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL'; did you mean 'MLX5_UCTX_CAP_RAW_TX'?
    4207 |         if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
         |                                                 ^~~~~~~~~~~~~~~~~~~~~~~
         |                                                 MLX5_UCTX_CAP_RAW_TX
   include/linux/mlx5/mlx5_ifc.h:1571:2: note: 'MLX5_UCTX_CAP_RAW_TX' declared here
    1571 |         MLX5_UCTX_CAP_RAW_TX = 1UL << 0,
         |         ^
>> drivers/infiniband/hw/mlx5/main.c:4211:6: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA'; did you mean 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA'?
    4211 |             MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA)
         |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         |             RDMA_UCAP_MLX5_CTRL_OTHER_VHCA
   include/rdma/ib_ucaps.h:13:2: note: 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA' declared here
      13 |         RDMA_UCAP_MLX5_CTRL_OTHER_VHCA,
         |         ^
   drivers/infiniband/hw/mlx5/main.c:4219:42: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL'; did you mean 'MLX5_UCTX_CAP_RAW_TX'?
    4219 |         if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL) {
         |                                                 ^~~~~~~~~~~~~~~~~~~~~~~
         |                                                 MLX5_UCTX_CAP_RAW_TX
   include/linux/mlx5/mlx5_ifc.h:1571:2: note: 'MLX5_UCTX_CAP_RAW_TX' declared here
    1571 |         MLX5_UCTX_CAP_RAW_TX = 1UL << 0,
         |         ^
   drivers/infiniband/hw/mlx5/main.c:4226:6: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA'; did you mean 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA'?
    4226 |             MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA) {
         |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         |             RDMA_UCAP_MLX5_CTRL_OTHER_VHCA
   include/rdma/ib_ucaps.h:13:2: note: 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA' declared here
      13 |         RDMA_UCAP_MLX5_CTRL_OTHER_VHCA,
         |         ^
   drivers/infiniband/hw/mlx5/main.c:4235:42: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL'; did you mean 'MLX5_UCTX_CAP_RAW_TX'?
    4235 |         if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
         |                                                 ^~~~~~~~~~~~~~~~~~~~~~~
         |                                                 MLX5_UCTX_CAP_RAW_TX
   include/linux/mlx5/mlx5_ifc.h:1571:2: note: 'MLX5_UCTX_CAP_RAW_TX' declared here
    1571 |         MLX5_UCTX_CAP_RAW_TX = 1UL << 0,
         |         ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
    4242 |         if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
         |             ^                            ~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
    1299 |         MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
         |         ^                                                                  ~~~
   include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
     121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
         |                                                               ^                  ~~~
   include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
      55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
         |                                  ^                   ~~~
   note: (skipping 1 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   include/linux/stddef.h:16:32: note: expanded from macro 'offsetof'
      16 | #define offsetof(TYPE, MEMBER)  __builtin_offsetof(TYPE, MEMBER)
         |                                 ^                        ~~~~~~
   include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
      39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
         |                                                           ^
   include/uapi/linux/swab.h:131:31: note: expanded from macro '__swab64'
     131 |         (__u64)(__builtin_constant_p(x) ?       \
         |                                      ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
    4242 |         if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
         |             ^                            ~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
    1299 |         MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
         |         ^                                                                  ~~~
   include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
     121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
         |                                                               ^                  ~~~
   include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
      55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
         |                                  ^                   ~~~
   note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
      39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
         |                                                           ^
   include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
     132 |         ___constant_swab64(x) :                 \
         |                            ^
   include/uapi/linux/swab.h:25:12: note: expanded from macro '___constant_swab64'
      25 |         (((__u64)(x) & (__u64)0x00000000000000ffULL) << 56) |   \
         |                   ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
    4242 |         if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
         |             ^                            ~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
    1299 |         MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
         |         ^                                                                  ~~~
   include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
     121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
         |                                                               ^                  ~~~
   include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
      55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
         |                                  ^                   ~~~
   note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
      39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
         |                                                           ^
   include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
     132 |         ___constant_swab64(x) :                 \
         |                            ^
   include/uapi/linux/swab.h:26:12: note: expanded from macro '___constant_swab64'
      26 |         (((__u64)(x) & (__u64)0x000000000000ff00ULL) << 40) |   \
         |                   ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
    4242 |         if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
         |             ^                            ~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
    1299 |         MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
         |         ^                                                                  ~~~
   include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
     121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
         |                                                               ^                  ~~~
   include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
      55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
         |                                  ^                   ~~~
   note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
      39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
         |                                                           ^
   include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
     132 |         ___constant_swab64(x) :                 \
         |                            ^
   include/uapi/linux/swab.h:27:12: note: expanded from macro '___constant_swab64'
      27 |         (((__u64)(x) & (__u64)0x0000000000ff0000ULL) << 24) |   \
         |                   ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
    4242 |         if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
         |             ^                            ~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
    1299 |         MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
         |         ^                                                                  ~~~
   include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
     121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
         |                                                               ^                  ~~~
   include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
      55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
         |                                  ^                   ~~~
   note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
      39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
         |                                                           ^
   include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
     132 |         ___constant_swab64(x) :                 \
         |                            ^
   include/uapi/linux/swab.h:28:12: note: expanded from macro '___constant_swab64'
      28 |         (((__u64)(x) & (__u64)0x00000000ff000000ULL) <<  8) |   \
         |                   ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
    4242 |         if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
         |             ^                            ~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
    1299 |         MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
         |         ^                                                                  ~~~
   include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
     121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
         |                                                               ^                  ~~~
   include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
      55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
         |                                  ^                   ~~~
   note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
      39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
         |                                                           ^
   include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
     132 |         ___constant_swab64(x) :                 \
         |                            ^
   include/uapi/linux/swab.h:29:12: note: expanded from macro '___constant_swab64'
      29 |         (((__u64)(x) & (__u64)0x000000ff00000000ULL) >>  8) |   \
         |                   ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
    4242 |         if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
         |             ^                            ~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
    1299 |         MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
         |         ^                                                                  ~~~
   include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
     121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
         |                                                               ^                  ~~~
   include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
      55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
         |                                  ^                   ~~~
   note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
      39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
         |                                                           ^
   include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
     132 |         ___constant_swab64(x) :                 \
         |                            ^
   include/uapi/linux/swab.h:30:12: note: expanded from macro '___constant_swab64'
      30 |         (((__u64)(x) & (__u64)0x0000ff0000000000ULL) >> 24) |   \
         |                   ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
    4242 |         if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
         |             ^                            ~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
    1299 |         MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
         |         ^                                                                  ~~~
   include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
     121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
         |                                                               ^                  ~~~
   include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
      55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
         |                                  ^                   ~~~
   note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
      39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
         |                                                           ^
   include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
     132 |         ___constant_swab64(x) :                 \
         |                            ^
   include/uapi/linux/swab.h:31:12: note: expanded from macro '___constant_swab64'
      31 |         (((__u64)(x) & (__u64)0x00ff000000000000ULL) >> 40) |   \
         |                   ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
    4242 |         if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
         |             ^                            ~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
    1299 |         MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
         |         ^                                                                  ~~~
   include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
     121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
         |                                                               ^                  ~~~
   include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
      55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
         |                                  ^                   ~~~
   note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
      39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
         |                                                           ^
   include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
     132 |         ___constant_swab64(x) :                 \
         |                            ^
   include/uapi/linux/swab.h:32:12: note: expanded from macro '___constant_swab64'
      32 |         (((__u64)(x) & (__u64)0xff00000000000000ULL) >> 56)))
         |                   ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
    4242 |         if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
         |             ^                            ~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
    1299 |         MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
         |         ^                                                                  ~~~
   include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
     121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
         |                                                               ^                  ~~~
   include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
      55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
         |                                  ^                   ~~~
   note: (skipping 1 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   include/linux/stddef.h:16:32: note: expanded from macro 'offsetof'
      16 | #define offsetof(TYPE, MEMBER)  __builtin_offsetof(TYPE, MEMBER)
         |                                 ^                        ~~~~~~
   include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
      39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
         |                                                           ^
   include/uapi/linux/swab.h:133:12: note: expanded from macro '__swab64'
     133 |         __fswab64(x))
         |                   ^
>> drivers/infiniband/hw/mlx5/main.c:4243:6: error: use of undeclared identifier 'MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL'; did you mean 'MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER'?
    4243 |             MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL)
         |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         |             MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER
   include/linux/mlx5/mlx5_ifc.h:12493:2: note: 'MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER' declared here
    12493 |         MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER = BIT_ULL(0x20),
          |         ^
   drivers/infiniband/hw/mlx5/main.c:4296:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
    4296 |         if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
         |             ^                            ~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
    1299 |         MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
         |         ^                                                                  ~~~
   include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
     121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
         |                                                               ^                  ~~~
   include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
      55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
         |                                  ^                   ~~~
   note: (skipping 1 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   include/linux/stddef.h:16:32: note: expanded from macro 'offsetof'
      16 | #define offsetof(TYPE, MEMBER)  __builtin_offsetof(TYPE, MEMBER)
         |                                 ^                        ~~~~~~
   include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
      39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
         |                                                           ^
   include/uapi/linux/swab.h:131:31: note: expanded from macro '__swab64'
     131 |         (__u64)(__builtin_constant_p(x) ?       \
         |                                      ^
   drivers/infiniband/hw/mlx5/main.c:4296:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
    4296 |         if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
         |             ^                            ~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
    1299 |         MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
         |         ^                                                                  ~~~
   include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
     121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
         |                                                               ^                  ~~~
   include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
      55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
         |                                  ^                   ~~~
   note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
      39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
         |                                                           ^
   include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
     132 |         ___constant_swab64(x) :                 \
         |                            ^
   include/uapi/linux/swab.h:25:12: note: expanded from macro '___constant_swab64'
      25 |         (((__u64)(x) & (__u64)0x00000000000000ffULL) << 56) |   \
         |                   ^
   drivers/infiniband/hw/mlx5/main.c:4296:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
    4296 |         if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
         |             ^                            ~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
    1299 |         MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
         |         ^                                                                  ~~~
   include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
     121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
         |                                                               ^                  ~~~
   include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
      55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
         |                                  ^                   ~~~
   note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
      39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
         |                                                           ^
   include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
     132 |         ___constant_swab64(x) :                 \
         |                            ^
   include/uapi/linux/swab.h:26:12: note: expanded from macro '___constant_swab64'
      26 |         (((__u64)(x) & (__u64)0x000000000000ff00ULL) << 40) |   \
         |                   ^

fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated.

vim +4207 drivers/infiniband/hw/mlx5/main.c

4204
4205 static void mlx5_ib_cleanup_ucaps(struct mlx5_ib_dev *dev)
4206 {

> 4207		if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
  4208			ib_remove_ucap(RDMA_UCAP_MLX5_CTRL_LOCAL);
  4209	
  4210		if (MLX5_CAP_GEN(dev->mdev, uctx_cap) &
> 4211		    MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA)
  4212			ib_remove_ucap(RDMA_UCAP_MLX5_CTRL_OTHER_VHCA);
  4213	}
  4214	
  4215	static int mlx5_ib_init_ucaps(struct mlx5_ib_dev *dev)
  4216	{
  4217		int ret;
  4218	
  4219		if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL) {
  4220			ret = ib_create_ucap(RDMA_UCAP_MLX5_CTRL_LOCAL);
  4221			if (ret)
  4222				return ret;
  4223		}
  4224	
  4225		if (MLX5_CAP_GEN(dev->mdev, uctx_cap) &
  4226		    MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA) {
  4227			ret = ib_create_ucap(RDMA_UCAP_MLX5_CTRL_OTHER_VHCA);
  4228			if (ret)
  4229				goto remove_local;
  4230		}
  4231	
  4232		return 0;
  4233	
  4234	remove_local:
  4235		if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
  4236			ib_remove_ucap(RDMA_UCAP_MLX5_CTRL_LOCAL);
  4237		return ret;
  4238	}
  4239	
  4240	static void mlx5_ib_stage_caps_cleanup(struct mlx5_ib_dev *dev)
  4241	{
> 4242		if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
> 4243		    MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL)

4244 mlx5_ib_cleanup_ucaps(dev);
4245
4246 bitmap_free(dev->var_table.bitmap);
4247 }
4248

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#6 lore raw

PATCH 3/6 RDMA/uverbs: Add support for UCAPs in context creation

In reply to Leon Romanovsky

From: Chiara Meiohas <[email protected]>

Add support for file descriptor array attribute for GET_CONTEXT commands.

Check that the file descriptor (fd) array represents fds for valid UCAPs. Store the enabled UCAPs from the fd array as a bitmask in ib_ucontext.

---
 drivers/infiniband/core/uverbs_cmd.c          | 19 +++++++++++++++++++
 .../infiniband/core/uverbs_std_types_device.c |  4 ++++
 include/rdma/ib_verbs.h                       |  1 +
 include/uapi/rdma/ib_user_ioctl_cmds.h        |  1 +
 4 files changed, 25 insertions(+)
drivers/infiniband/core/uverbs_cmd.c +19 -0
index 5ad14c39d48c..96d639e1ffa0 100644
@@ -42,6 +42,7 @@
 
 #include <rdma/uverbs_types.h>
 #include <rdma/uverbs_std_types.h>
+#include <rdma/ib_ucaps.h>
 #include "rdma_core.h"
 
 #include "uverbs.h"
@@ -232,6 +233,8 @@ int ib_init_ucontext(struct uverbs_attr_bundle *attrs)
 {
 	struct ib_ucontext *ucontext = attrs->context;
 	struct ib_uverbs_file *file = attrs->ufile;
+	int *fd_array;
+	int fd_count;
 	int ret;
 
 	if (!down_read_trylock(&file->hw_destroy_rwsem))
@@ -247,6 +250,22 @@ int ib_init_ucontext(struct uverbs_attr_bundle *attrs)
 	if (ret)
 		goto err;
 
+	if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_GET_CONTEXT_FD_ARR)) {
+		fd_count = uverbs_attr_ptr_get_array_size(attrs,
+							  UVERBS_ATTR_GET_CONTEXT_FD_ARR,
+							  sizeof(int));
+		if (fd_count < 0) {
+			ret = fd_count;
+			goto err_uncharge;
+		}
+
+		fd_array = uverbs_attr_get_alloced_ptr(attrs,
+						       UVERBS_ATTR_GET_CONTEXT_FD_ARR);
+		ret = ib_get_ucaps(fd_array, fd_count, &ucontext->enabled_caps);
+		if (ret)
+			goto err_uncharge;
+	}
+
 	ret = ucontext->device->ops.alloc_ucontext(ucontext,
 						   &attrs->driver_udata);
 	if (ret)
drivers/infiniband/core/uverbs_std_types_device.c +4 -0
index fb0555647336..c0fd283d9d6c 100644
@@ -437,6 +437,10 @@ DECLARE_UVERBS_NAMED_METHOD(
 			    UVERBS_ATTR_TYPE(u32), UA_OPTIONAL),
 	UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_GET_CONTEXT_CORE_SUPPORT,
 			    UVERBS_ATTR_TYPE(u64), UA_OPTIONAL),
+	UVERBS_ATTR_PTR_IN(UVERBS_ATTR_GET_CONTEXT_FD_ARR,
+			   UVERBS_ATTR_MIN_SIZE(sizeof(int)),
+			   UA_OPTIONAL,
+			   UA_ALLOC_AND_COPY),
 	UVERBS_ATTR_UHW());
 
 DECLARE_UVERBS_NAMED_METHOD(
include/rdma/ib_verbs.h +1 -0
index a5761038935d..9941f4185c79 100644
@@ -1530,6 +1530,7 @@ struct ib_ucontext {
 	struct ib_uverbs_file  *ufile;
 
 	struct ib_rdmacg_object	cg_obj;
+	u64 enabled_caps;
 	/*
 	 * Implementation details of the RDMA core, don't use in drivers:
 	 */
include/uapi/rdma/ib_user_ioctl_cmds.h +1 -0
index ec719053aab9..ac7b162611ed 100644
@@ -88,6 +88,7 @@ enum uverbs_attrs_query_port_cmd_attr_ids {
 enum uverbs_attrs_get_context_attr_ids {
 	UVERBS_ATTR_GET_CONTEXT_NUM_COMP_VECTORS,
 	UVERBS_ATTR_GET_CONTEXT_CORE_SUPPORT,
+	UVERBS_ATTR_GET_CONTEXT_FD_ARR,
 };
 
 enum uverbs_attrs_query_context_attr_ids {
Signature · 1 line
2.48.1
#7 lore raw

PATCH 4/6 RDMA/mlx5: Check enabled UCAPs when creating ucontext

In reply to Leon Romanovsky

From: Chiara Meiohas <[email protected]>

Verify that the enabled UCAPs are supported by the device before creating the ucontext.
If supported, create the ucontext with the associated capabilities.

Store the privileged ucontext UID on creation and remove it when destroying the privileged ucontext. This allows the command interface to recognize privileged commands through its UID.

---
 drivers/infiniband/hw/mlx5/devx.c | 31 +++++++++++++++++++++++++++++--
 drivers/infiniband/hw/mlx5/devx.h |  5 +++--
 drivers/infiniband/hw/mlx5/main.c | 30 ++++++++++++++++++++++++++----
 3 files changed, 58 insertions(+), 8 deletions(-)
drivers/infiniband/hw/mlx5/devx.c +29 -2
index 39304cae5b10..2479da8620ca 100644
@@ -13,6 +13,7 @@
 #include <rdma/uverbs_std_types.h>
 #include <linux/mlx5/driver.h>
 #include <linux/mlx5/fs.h>
+#include <rdma/ib_ucaps.h>
 #include "mlx5_ib.h"
 #include "devx.h"
 #include "qp.h"
@@ -122,7 +123,27 @@ devx_ufile2uctx(const struct uverbs_attr_bundle *attrs)
 	return to_mucontext(ib_uverbs_get_ucontext(attrs));
 }
 
-int mlx5_ib_devx_create(struct mlx5_ib_dev *dev, bool is_user)
+static int set_uctx_ucaps(struct mlx5_ib_dev *dev, u64 req_ucaps, u32 *cap)
+{
+	if (UCAP_ENABLED(req_ucaps, RDMA_UCAP_MLX5_CTRL_LOCAL)) {
+		if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
+			*cap |= MLX5_UCTX_CAP_RDMA_CTRL;
+		else
+			return -EOPNOTSUPP;
+	}
+
+	if (UCAP_ENABLED(req_ucaps, RDMA_UCAP_MLX5_CTRL_OTHER_VHCA)) {
+		if (MLX5_CAP_GEN(dev->mdev, uctx_cap) &
+		    MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA)
+			*cap |= MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA;
+		else
+			return -EOPNOTSUPP;
+	}
+
+	return 0;
+}
+
+int mlx5_ib_devx_create(struct mlx5_ib_dev *dev, bool is_user, u64 req_ucaps)
 {
 	u32 in[MLX5_ST_SZ_DW(create_uctx_in)] = {};
 	u32 out[MLX5_ST_SZ_DW(create_uctx_out)] = {};
@@ -146,6 +167,12 @@ int mlx5_ib_devx_create(struct mlx5_ib_dev *dev, bool is_user)
 	    capable(CAP_SYS_RAWIO))
 		cap |= MLX5_UCTX_CAP_INTERNAL_DEV_RES;
 
+	if (req_ucaps) {
+		err = set_uctx_ucaps(dev, req_ucaps, &cap);
+		if (err)
+			return err;
+	}
+
 	MLX5_SET(create_uctx_in, in, opcode, MLX5_CMD_OP_CREATE_UCTX);
 	MLX5_SET(uctx, uctx, cap, cap);
 
@@ -2575,7 +2602,7 @@ int mlx5_ib_devx_init(struct mlx5_ib_dev *dev)
 	struct mlx5_devx_event_table *table = &dev->devx_event_table;
 	int uid;
 
-	uid = mlx5_ib_devx_create(dev, false);
+	uid = mlx5_ib_devx_create(dev, false, 0);
 	if (uid > 0) {
 		dev->devx_whitelist_uid = uid;
 		xa_init(&table->event_xa);
drivers/infiniband/hw/mlx5/devx.h +3 -2
index 1344bf4c9d21..ee9e7d3af93f 100644
@@ -24,13 +24,14 @@ struct devx_obj {
 	struct list_head event_sub; /* holds devx_event_subscription entries */
 };
 #if IS_ENABLED(CONFIG_INFINIBAND_USER_ACCESS)
-int mlx5_ib_devx_create(struct mlx5_ib_dev *dev, bool is_user);
+int mlx5_ib_devx_create(struct mlx5_ib_dev *dev, bool is_user, u64 req_ucaps);
 void mlx5_ib_devx_destroy(struct mlx5_ib_dev *dev, u16 uid);
 int mlx5_ib_devx_init(struct mlx5_ib_dev *dev);
 void mlx5_ib_devx_cleanup(struct mlx5_ib_dev *dev);
 void mlx5_ib_ufile_hw_cleanup(struct ib_uverbs_file *ufile);
 #else
-static inline int mlx5_ib_devx_create(struct mlx5_ib_dev *dev, bool is_user)
+static inline int mlx5_ib_devx_create(struct mlx5_ib_dev *dev, bool is_user,
+				      u64 req_ucaps)
 {
 	return -EOPNOTSUPP;
 }
drivers/infiniband/hw/mlx5/main.c +26 -4
index 04b489a6a449..d07cacaa0abd 100644
@@ -1935,6 +1935,12 @@ static int set_ucontext_resp(struct ib_ucontext *uctx,
 	return 0;
 }
 
+static bool uctx_rdma_ctrl_is_enabled(u64 enabled_caps)
+{
+	return UCAP_ENABLED(enabled_caps, RDMA_UCAP_MLX5_CTRL_LOCAL) ||
+	       UCAP_ENABLED(enabled_caps, RDMA_UCAP_MLX5_CTRL_OTHER_VHCA);
+}
+
 static int mlx5_ib_alloc_ucontext(struct ib_ucontext *uctx,
 				  struct ib_udata *udata)
 {
@@ -1977,10 +1983,17 @@ static int mlx5_ib_alloc_ucontext(struct ib_ucontext *uctx,
 		return -EINVAL;
 
 	if (req.flags & MLX5_IB_ALLOC_UCTX_DEVX) {
-		err = mlx5_ib_devx_create(dev, true);
+		err = mlx5_ib_devx_create(dev, true, uctx->enabled_caps);
 		if (err < 0)
 			goto out_ctx;
 		context->devx_uid = err;
+
+		if (uctx_rdma_ctrl_is_enabled(uctx->enabled_caps)) {
+			err = mlx5_cmd_add_privileged_uid(dev->mdev,
+							  context->devx_uid);
+			if (err)
+				goto out_devx;
+		}
 	}
 
 	lib_uar_4k = req.lib_caps & MLX5_LIB_CAP_4K_UAR;
@@ -1995,7 +2008,7 @@ static int mlx5_ib_alloc_ucontext(struct ib_ucontext *uctx,
 	/* updates req->total_num_bfregs */
 	err = calc_total_bfregs(dev, lib_uar_4k, &req, bfregi);
 	if (err)
-		goto out_devx;
+		goto out_ucap;
 
 	mutex_init(&bfregi->lock);
 	bfregi->lib_uar_4k = lib_uar_4k;
@@ -2003,7 +2016,7 @@ static int mlx5_ib_alloc_ucontext(struct ib_ucontext *uctx,
 				GFP_KERNEL);
 	if (!bfregi->count) {
 		err = -ENOMEM;
-		goto out_devx;
+		goto out_ucap;
 	}
 
 	bfregi->sys_pages = kcalloc(bfregi->num_sys_pages,
@@ -2067,6 +2080,11 @@ static int mlx5_ib_alloc_ucontext(struct ib_ucontext *uctx,
 out_count:
 	kfree(bfregi->count);
 
+out_ucap:
+	if (req.flags & MLX5_IB_ALLOC_UCTX_DEVX &&
+	    uctx_rdma_ctrl_is_enabled(uctx->enabled_caps))
+		mlx5_cmd_remove_privileged_uid(dev->mdev, context->devx_uid);
+
 out_devx:
 	if (req.flags & MLX5_IB_ALLOC_UCTX_DEVX)
 		mlx5_ib_devx_destroy(dev, context->devx_uid);
@@ -2111,8 +2129,12 @@ static void mlx5_ib_dealloc_ucontext(struct ib_ucontext *ibcontext)
 	kfree(bfregi->sys_pages);
 	kfree(bfregi->count);
 
-	if (context->devx_uid)
+	if (context->devx_uid) {
+		if (uctx_rdma_ctrl_is_enabled(ibcontext->enabled_caps))
+			mlx5_cmd_remove_privileged_uid(dev->mdev,
+						       context->devx_uid);
 		mlx5_ib_devx_destroy(dev, context->devx_uid);
+	}
 }
 
 static phys_addr_t uar_index2pfn(struct mlx5_ib_dev *dev,
Signature · 1 line
2.48.1
#8 lore raw

Re: [PATCH rdma-next v1 4/6] RDMA/mlx5: Check enabled UCAPs when creating ucontext

kernel test robot noticed the following build errors:

Hi Leon,

kernel test robot noticed the following build errors:

[auto build test ERROR on rdma/for-next]
[also build test ERROR on linus/master v6.14-rc5 next-20250307] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Leon-Romanovsky/RDMA-uverbs-Introduce-UCAP-User-CAPabilities-API/20250306-195708
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git for-next
patch link:    https://lore.kernel.org/r/8b180583a207cb30deb7a2967934079749cdcc44.1741261611.git.leon%40kernel.org
patch subject: [PATCH rdma-next v1 4/6] RDMA/mlx5: Check enabled UCAPs when creating ucontext
config: s390-defconfig (https://download.01.org/0day-ci/archive/20250308/[email protected]/config)
compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250308/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

>> drivers/infiniband/hw/mlx5/main.c:1992:10: error: call to undeclared function 'mlx5_cmd_add_privileged_uid'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
                           err = mlx5_cmd_add_privileged_uid(dev->mdev,
                                 ^
>> drivers/infiniband/hw/mlx5/main.c:2086:3: error: call to undeclared function 'mlx5_cmd_remove_privileged_uid'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
                   mlx5_cmd_remove_privileged_uid(dev->mdev, context->devx_uid);
                   ^
   drivers/infiniband/hw/mlx5/main.c:2134:4: error: call to undeclared function 'mlx5_cmd_remove_privileged_uid'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
                           mlx5_cmd_remove_privileged_uid(dev->mdev,
                           ^
   drivers/infiniband/hw/mlx5/main.c:4229:42: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL'; did you mean 'MLX5_UCTX_CAP_RAW_TX'?
           if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
                                                   ^~~~~~~~~~~~~~~~~~~~~~~
                                                   MLX5_UCTX_CAP_RAW_TX
   include/linux/mlx5/mlx5_ifc.h:1571:2: note: 'MLX5_UCTX_CAP_RAW_TX' declared here
           MLX5_UCTX_CAP_RAW_TX = 1UL << 0,
           ^
   drivers/infiniband/hw/mlx5/main.c:4233:6: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA'; did you mean 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA'?
               MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA)
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
               RDMA_UCAP_MLX5_CTRL_OTHER_VHCA
   include/rdma/ib_ucaps.h:13:2: note: 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA' declared here
           RDMA_UCAP_MLX5_CTRL_OTHER_VHCA,
           ^
   drivers/infiniband/hw/mlx5/main.c:4241:42: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL'; did you mean 'MLX5_UCTX_CAP_RAW_TX'?
           if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL) {
                                                   ^~~~~~~~~~~~~~~~~~~~~~~
                                                   MLX5_UCTX_CAP_RAW_TX
   include/linux/mlx5/mlx5_ifc.h:1571:2: note: 'MLX5_UCTX_CAP_RAW_TX' declared here
           MLX5_UCTX_CAP_RAW_TX = 1UL << 0,
           ^
   drivers/infiniband/hw/mlx5/main.c:4248:6: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA'; did you mean 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA'?
               MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA) {
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
               RDMA_UCAP_MLX5_CTRL_OTHER_VHCA
   include/rdma/ib_ucaps.h:13:2: note: 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA' declared here
           RDMA_UCAP_MLX5_CTRL_OTHER_VHCA,
           ^
   drivers/infiniband/hw/mlx5/main.c:4257:42: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL'; did you mean 'MLX5_UCTX_CAP_RAW_TX'?
           if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
                                                   ^~~~~~~~~~~~~~~~~~~~~~~
                                                   MLX5_UCTX_CAP_RAW_TX
   include/linux/mlx5/mlx5_ifc.h:1571:2: note: 'MLX5_UCTX_CAP_RAW_TX' declared here
           MLX5_UCTX_CAP_RAW_TX = 1UL << 0,
           ^
   drivers/infiniband/hw/mlx5/main.c:4264:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
           if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
               ^                            ~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
           MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
           ^                                                                  ~~~
   include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
   #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
                                                                 ^                  ~~~
   include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
   #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
                                    ^                   ~~~
   include/linux/mlx5/device.h:52:35: note: expanded from macro '__mlx5_bit_off'
   #define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
                                     ^                                      ~~~
   include/linux/stddef.h:16:32: note: expanded from macro 'offsetof'
   #define offsetof(TYPE, MEMBER)  __builtin_offsetof(TYPE, MEMBER)
                                   ^                        ~~~~~~
   include/uapi/linux/byteorder/big_endian.h:39:51: note: expanded from macro '__be64_to_cpu'
   #define __be64_to_cpu(x) ((__force __u64)(__be64)(x))
                                                     ^
   drivers/infiniband/hw/mlx5/main.c:4265:6: error: use of undeclared identifier 'MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL'; did you mean 'MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER'?
               MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL)
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
               MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER
   include/linux/mlx5/mlx5_ifc.h:12493:2: note: 'MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER' declared here
           MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER = BIT_ULL(0x20),
           ^
   drivers/infiniband/hw/mlx5/main.c:4318:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
           if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
               ^                            ~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
           MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
           ^                                                                  ~~~
   include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
   #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
                                                                 ^                  ~~~
   include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
   #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
                                    ^                   ~~~
   include/linux/mlx5/device.h:52:35: note: expanded from macro '__mlx5_bit_off'
   #define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
                                     ^                                      ~~~
   include/linux/stddef.h:16:32: note: expanded from macro 'offsetof'
   #define offsetof(TYPE, MEMBER)  __builtin_offsetof(TYPE, MEMBER)
                                   ^                        ~~~~~~
   include/uapi/linux/byteorder/big_endian.h:39:51: note: expanded from macro '__be64_to_cpu'
   #define __be64_to_cpu(x) ((__force __u64)(__be64)(x))
                                                     ^
   drivers/infiniband/hw/mlx5/main.c:4319:6: error: use of undeclared identifier 'MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL'; did you mean 'MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER'?
               MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL) {
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
               MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER
   include/linux/mlx5/mlx5_ifc.h:12493:2: note: 'MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER' declared here
           MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER = BIT_ULL(0x20),
           ^

12 errors generated.

vim +/mlx5_cmd_add_privileged_uid +1992 drivers/infiniband/hw/mlx5/main.c

1943
1944 static int mlx5_ib_alloc_ucontext(struct ib_ucontext *uctx,
1945 struct ib_udata *udata)
1946 {
1947 struct ib_device *ibdev = uctx->device;
1948 struct mlx5_ib_dev *dev = to_mdev(ibdev);
1949 struct mlx5_ib_alloc_ucontext_req_v2 req = {};
1950 struct mlx5_ib_alloc_ucontext_resp resp = {};
1951 struct mlx5_ib_ucontext *context = to_mucontext(uctx);
1952 struct mlx5_bfreg_info *bfregi;
1953 int ver;
1954 int err;
1955 size_t min_req_v2 = offsetof(struct mlx5_ib_alloc_ucontext_req_v2,
1956 max_cqe_version);
1957 bool lib_uar_4k;
1958 bool lib_uar_dyn;
1959
1960 if (!dev->ib_active)
1961 return -EAGAIN;
1962
1963 if (udata->inlen == sizeof(struct mlx5_ib_alloc_ucontext_req))
1964 ver = 0;
1965 else if (udata->inlen >= min_req_v2)
1966 ver = 2;
1967 else
1968 return -EINVAL;
1969
1970 err = ib_copy_from_udata(&req, udata, min(udata->inlen, sizeof(req)));
1971 if (err)
1972 return err;
1973
1974 if (req.flags & ~MLX5_IB_ALLOC_UCTX_DEVX)
1975 return -EOPNOTSUPP;
1976
1977 if (req.comp_mask || req.reserved0 || req.reserved1 || req.reserved2)
1978 return -EOPNOTSUPP;
1979
1980 req.total_num_bfregs = ALIGN(req.total_num_bfregs,
1981 MLX5_NON_FP_BFREGS_PER_UAR);
1982 if (req.num_low_latency_bfregs > req.total_num_bfregs - 1)
1983 return -EINVAL;
1984
1985 if (req.flags & MLX5_IB_ALLOC_UCTX_DEVX) {
1986 err = mlx5_ib_devx_create(dev, true, uctx->enabled_caps);
1987 if (err < 0)
1988 goto out_ctx;
1989 context->devx_uid = err;
1990
1991 if (uctx_rdma_ctrl_is_enabled(uctx->enabled_caps)) {

> 1992				err = mlx5_cmd_add_privileged_uid(dev->mdev,
  1993								  context->devx_uid);
  1994				if (err)
  1995					goto out_devx;
  1996			}
  1997		}
  1998	
  1999		lib_uar_4k = req.lib_caps & MLX5_LIB_CAP_4K_UAR;
  2000		lib_uar_dyn = req.lib_caps & MLX5_LIB_CAP_DYN_UAR;
  2001		bfregi = &context->bfregi;
  2002	
  2003		if (lib_uar_dyn) {
  2004			bfregi->lib_uar_dyn = lib_uar_dyn;
  2005			goto uar_done;
  2006		}
  2007	
  2008		/* updates req->total_num_bfregs */
  2009		err = calc_total_bfregs(dev, lib_uar_4k, &req, bfregi);
  2010		if (err)
  2011			goto out_ucap;
  2012	
  2013		mutex_init(&bfregi->lock);
  2014		bfregi->lib_uar_4k = lib_uar_4k;
  2015		bfregi->count = kcalloc(bfregi->total_num_bfregs, sizeof(*bfregi->count),
  2016					GFP_KERNEL);
  2017		if (!bfregi->count) {
  2018			err = -ENOMEM;
  2019			goto out_ucap;
  2020		}
  2021	
  2022		bfregi->sys_pages = kcalloc(bfregi->num_sys_pages,
  2023					    sizeof(*bfregi->sys_pages),
  2024					    GFP_KERNEL);
  2025		if (!bfregi->sys_pages) {
  2026			err = -ENOMEM;
  2027			goto out_count;
  2028		}
  2029	
  2030		err = allocate_uars(dev, context);
  2031		if (err)
  2032			goto out_sys_pages;
  2033	
  2034	uar_done:
  2035		err = mlx5_ib_alloc_transport_domain(dev, &context->tdn,
  2036						     context->devx_uid);
  2037		if (err)
  2038			goto out_uars;
  2039	
  2040		INIT_LIST_HEAD(&context->db_page_list);
  2041		mutex_init(&context->db_page_mutex);
  2042	
  2043		context->cqe_version = min_t(__u8,
  2044					 (__u8)MLX5_CAP_GEN(dev->mdev, cqe_version),
  2045					 req.max_cqe_version);
  2046	
  2047		err = set_ucontext_resp(uctx, &resp);
  2048		if (err)
  2049			goto out_mdev;
  2050	
  2051		resp.response_length = min(udata->outlen, sizeof(resp));
  2052		err = ib_copy_to_udata(udata, &resp, resp.response_length);
  2053		if (err)
  2054			goto out_mdev;
  2055	
  2056		bfregi->ver = ver;
  2057		bfregi->num_low_latency_bfregs = req.num_low_latency_bfregs;
  2058		context->lib_caps = req.lib_caps;
  2059		print_lib_caps(dev, context->lib_caps);
  2060	
  2061		if (mlx5_ib_lag_should_assign_affinity(dev)) {
  2062			u32 port = mlx5_core_native_port_num(dev->mdev) - 1;
  2063	
  2064			atomic_set(&context->tx_port_affinity,
  2065				   atomic_add_return(
  2066					   1, &dev->port[port].roce.tx_port_affinity));
  2067		}
  2068	
  2069		return 0;
  2070	
  2071	out_mdev:
  2072		mlx5_ib_dealloc_transport_domain(dev, context->tdn, context->devx_uid);
  2073	
  2074	out_uars:
  2075		deallocate_uars(dev, context);
  2076	
  2077	out_sys_pages:
  2078		kfree(bfregi->sys_pages);
  2079	
  2080	out_count:
  2081		kfree(bfregi->count);
  2082	
  2083	out_ucap:
  2084		if (req.flags & MLX5_IB_ALLOC_UCTX_DEVX &&
  2085		    uctx_rdma_ctrl_is_enabled(uctx->enabled_caps))
> 2086			mlx5_cmd_remove_privileged_uid(dev->mdev, context->devx_uid);

2087
2088 out_devx:
2089 if (req.flags & MLX5_IB_ALLOC_UCTX_DEVX)
2090 mlx5_ib_devx_destroy(dev, context->devx_uid);
2091
2092 out_ctx:
2093 return err;
2094 }
2095

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#9 lore raw

PATCH 5/6 RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace

In reply to Leon Romanovsky

From: Patrisious Haddad <[email protected]>

This patch adds RDMA_TRANSPORT_RX and RDMA_TRANSPORT_TX as a new flow table type for matcher creation.

---
 drivers/infiniband/hw/mlx5/fs.c           | 154 ++++++++++++++++++++--
 drivers/infiniband/hw/mlx5/fs.h           |   2 +
 drivers/infiniband/hw/mlx5/mlx5_ib.h      |   3 +
 include/uapi/rdma/mlx5_user_ioctl_cmds.h  |   1 +
 include/uapi/rdma/mlx5_user_ioctl_verbs.h |   2 +
 5 files changed, 149 insertions(+), 13 deletions(-)
drivers/infiniband/hw/mlx5/fs.c +141 -13
index 162814ae8cb4..6ae2801fa13f 100644
@@ -12,6 +12,7 @@
 #include <rdma/mlx5_user_ioctl_verbs.h>
 #include <rdma/ib_hdrs.h>
 #include <rdma/ib_umem.h>
+#include <rdma/ib_ucaps.h>
 #include <linux/mlx5/driver.h>
 #include <linux/mlx5/fs.h>
 #include <linux/mlx5/fs_helpers.h>
@@ -690,7 +691,7 @@ static struct mlx5_ib_flow_prio *_get_prio(struct mlx5_ib_dev *dev,
 					   struct mlx5_ib_flow_prio *prio,
 					   int priority,
 					   int num_entries, int num_groups,
-					   u32 flags)
+					   u32 flags, u16 vport)
 {
 	struct mlx5_flow_table_attr ft_attr = {};
 	struct mlx5_flow_table *ft;
@@ -698,6 +699,7 @@ static struct mlx5_ib_flow_prio *_get_prio(struct mlx5_ib_dev *dev,
 	ft_attr.prio = priority;
 	ft_attr.max_fte = num_entries;
 	ft_attr.flags = flags;
+	ft_attr.vport = vport;
 	ft_attr.autogroup.max_num_groups = num_groups;
 	ft = mlx5_create_auto_grouped_flow_table(ns, &ft_attr);
 	if (IS_ERR(ft))
@@ -792,7 +794,7 @@ static struct mlx5_ib_flow_prio *get_flow_table(struct mlx5_ib_dev *dev,
 	ft = prio->flow_table;
 	if (!ft)
 		return _get_prio(dev, ns, prio, priority, max_table_size,
-				 num_groups, flags);
+				 num_groups, flags, 0);
 
 	return prio;
 }
@@ -935,7 +937,7 @@ int mlx5_ib_fs_add_op_fc(struct mlx5_ib_dev *dev, u32 port_num,
 	prio = &dev->flow_db->opfcs[type];
 	if (!prio->flow_table) {
 		prio = _get_prio(dev, ns, prio, priority,
-				 dev->num_ports * MAX_OPFC_RULES, 1, 0);
+				 dev->num_ports * MAX_OPFC_RULES, 1, 0, 0);
 		if (IS_ERR(prio)) {
 			err = PTR_ERR(prio);
 			goto free;
@@ -1413,17 +1415,51 @@ static struct ib_flow *mlx5_ib_create_flow(struct ib_qp *qp,
 	return ERR_PTR(err);
 }
 
+static int mlx5_ib_fill_transport_ns_info(struct mlx5_ib_dev *dev,
+					  enum mlx5_flow_namespace_type type,
+					  u32 *flags, u16 *vport_idx,
+					  u16 *vport,
+					  struct mlx5_core_dev **ft_mdev,
+					  u32 ib_port)
+{
+	struct mlx5_core_dev *esw_mdev;
+
+	if (!is_mdev_switchdev_mode(dev->mdev))
+		return 0;
+
+	if (!MLX5_CAP_ADV_RDMA(dev->mdev, rdma_transport_manager))
+		return -EOPNOTSUPP;
+
+	if (!dev->port[ib_port - 1].rep)
+		return -EINVAL;
+
+	esw_mdev = mlx5_eswitch_get_core_dev(dev->port[ib_port - 1].rep->esw);
+	if (esw_mdev != dev->mdev)
+		return -EOPNOTSUPP;
+
+	*flags |= MLX5_FLOW_TABLE_OTHER_VPORT;
+	*ft_mdev = esw_mdev;
+	*vport = dev->port[ib_port - 1].rep->vport;
+	*vport_idx = dev->port[ib_port - 1].rep->vport_index;
+
+	return 0;
+}
+
 static struct mlx5_ib_flow_prio *
 _get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority,
 		enum mlx5_flow_namespace_type ns_type,
-		bool mcast)
+		bool mcast, u32 ib_port)
 {
+	struct mlx5_core_dev *ft_mdev = dev->mdev;
 	struct mlx5_flow_namespace *ns = NULL;
 	struct mlx5_ib_flow_prio *prio = NULL;
 	int max_table_size = 0;
+	u16 vport_idx = 0;
 	bool esw_encap;
 	u32 flags = 0;
+	u16 vport = 0;
 	int priority;
+	int ret;
 
 	if (mcast)
 		priority = MLX5_IB_FLOW_MCAST_PRIO;
@@ -1471,13 +1507,38 @@ _get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority,
 			MLX5_CAP_FLOWTABLE_RDMA_TX(dev->mdev, log_max_ft_size));
 		priority = user_priority;
 		break;
+	case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
+	case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
+		if (ib_port == 0 || user_priority > MLX5_RDMA_TRANSPORT_BYPASS_PRIO)
+			return ERR_PTR(-EINVAL);
+		ret = mlx5_ib_fill_transport_ns_info(dev, ns_type, &flags,
+						     &vport_idx, &vport,
+						     &ft_mdev, ib_port);
+		if (ret)
+			return ERR_PTR(ret);
+
+		if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX)
+			max_table_size =
+				BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_RX(
+					ft_mdev, log_max_ft_size));
+		else
+			max_table_size =
+				BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_TX(
+					ft_mdev, log_max_ft_size));
+		priority = user_priority;
+		break;
 	default:
 		break;
 	}
 
 	max_table_size = min_t(int, max_table_size, MLX5_FS_MAX_ENTRIES);
 
-	ns = mlx5_get_flow_namespace(dev->mdev, ns_type);
+	if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX ||
+	    ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX)
+		ns = mlx5_get_flow_vport_namespace(ft_mdev, ns_type, vport_idx);
+	else
+		ns = mlx5_get_flow_namespace(ft_mdev, ns_type);
+
 	if (!ns)
 		return ERR_PTR(-EOPNOTSUPP);
 
@@ -1497,6 +1558,12 @@ _get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority,
 	case MLX5_FLOW_NAMESPACE_RDMA_TX:
 		prio = &dev->flow_db->rdma_tx[priority];
 		break;
+	case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
+		prio = &dev->flow_db->rdma_transport_rx[ib_port - 1];
+		break;
+	case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
+		prio = &dev->flow_db->rdma_transport_tx[ib_port - 1];
+		break;
 	default: return ERR_PTR(-EINVAL);
 	}
 
@@ -1507,7 +1574,7 @@ _get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority,
 		return prio;
 
 	return _get_prio(dev, ns, prio, priority, max_table_size,
-			 MLX5_FS_MAX_TYPES, flags);
+			 MLX5_FS_MAX_TYPES, flags, vport);
 }
 
 static struct mlx5_ib_flow_handler *
@@ -1626,7 +1693,8 @@ static struct mlx5_ib_flow_handler *raw_fs_rule_add(
 	mutex_lock(&dev->flow_db->lock);
 
 	ft_prio = _get_flow_table(dev, fs_matcher->priority,
-				  fs_matcher->ns_type, mcast);
+				  fs_matcher->ns_type, mcast,
+				  fs_matcher->ib_port);
 	if (IS_ERR(ft_prio)) {
 		err = PTR_ERR(ft_prio);
 		goto unlock;
@@ -1742,6 +1810,12 @@ mlx5_ib_ft_type_to_namespace(enum mlx5_ib_uapi_flow_table_type table_type,
 	case MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_TX:
 		*namespace = MLX5_FLOW_NAMESPACE_RDMA_TX;
 		break;
+	case MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_TRANSPORT_RX:
+		*namespace = MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX;
+		break;
+	case MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_TRANSPORT_TX:
+		*namespace = MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX;
+		break;
 	default:
 		return -EINVAL;
 	}
@@ -1831,7 +1905,8 @@ static int get_dests(struct uverbs_attr_bundle *attrs,
 		return -EINVAL;
 
 	/* Allow only DEVX object or QP as dest when inserting to RDMA_RX */
-	if ((fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_RX) &&
+	if ((fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_RX ||
+	     fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX) &&
 	    ((!dest_devx && !dest_qp) || (dest_devx && dest_qp)))
 		return -EINVAL;
 
@@ -1848,7 +1923,8 @@ static int get_dests(struct uverbs_attr_bundle *attrs,
 			return -EINVAL;
 		/* Allow only flow table as dest when inserting to FDB or RDMA_RX */
 		if ((fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_FDB_BYPASS ||
-		     fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_RX) &&
+		     fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_RX ||
+		     fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX) &&
 		    *dest_type != MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE)
 			return -EINVAL;
 	} else if (dest_qp) {
@@ -1869,14 +1945,16 @@ static int get_dests(struct uverbs_attr_bundle *attrs,
 			*dest_id = mqp->raw_packet_qp.rq.tirn;
 		*dest_type = MLX5_FLOW_DESTINATION_TYPE_TIR;
 	} else if ((fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_EGRESS ||
-		    fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TX) &&
+		    fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TX ||
+		    fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX) &&
 		   !(*flags & MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP)) {
 		*dest_type = MLX5_FLOW_DESTINATION_TYPE_PORT;
 	}
 
 	if (*dest_type == MLX5_FLOW_DESTINATION_TYPE_TIR &&
 	    (fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_EGRESS ||
-	     fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TX))
+	     fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TX ||
+	     fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX))
 		return -EINVAL;
 
 	return 0;
@@ -2353,6 +2431,15 @@ static int mlx5_ib_matcher_ns(struct uverbs_attr_bundle *attrs,
 	return 0;
 }
 
+static bool verify_context_caps(struct mlx5_ib_dev *dev, u64 enabled_caps)
+{
+	if (is_mdev_switchdev_mode(dev->mdev))
+		return UCAP_ENABLED(enabled_caps,
+				    RDMA_UCAP_MLX5_CTRL_OTHER_VHCA);
+
+	return UCAP_ENABLED(enabled_caps, RDMA_UCAP_MLX5_CTRL_LOCAL);
+}
+
 static int UVERBS_HANDLER(MLX5_IB_METHOD_FLOW_MATCHER_CREATE)(
 	struct uverbs_attr_bundle *attrs)
 {
@@ -2401,6 +2488,26 @@ static int UVERBS_HANDLER(MLX5_IB_METHOD_FLOW_MATCHER_CREATE)(
 		goto end;
 	}
 
+	if (uverbs_attr_is_valid(attrs, MLX5_IB_ATTR_FLOW_MATCHER_IB_PORT)) {
+		err = uverbs_copy_from(&obj->ib_port, attrs,
+				       MLX5_IB_ATTR_FLOW_MATCHER_IB_PORT);
+		if (err)
+			goto end;
+		if (!rdma_is_port_valid(&dev->ib_dev, obj->ib_port)) {
+			err = -EINVAL;
+			goto end;
+		}
+		if (obj->ns_type != MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX &&
+		    obj->ns_type != MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX) {
+			err = -EINVAL;
+			goto end;
+		}
+		if (!verify_context_caps(dev, uobj->context->enabled_caps)) {
+			err = -EOPNOTSUPP;
+			goto end;
+		}
+	}
+
 	uobj->object = obj;
 	obj->mdev = dev->mdev;
 	atomic_set(&obj->usecnt, 0);
@@ -2448,7 +2555,7 @@ static int UVERBS_HANDLER(MLX5_IB_METHOD_STEERING_ANCHOR_CREATE)(
 
 	mutex_lock(&dev->flow_db->lock);
 
-	ft_prio = _get_flow_table(dev, priority, ns_type, 0);
+	ft_prio = _get_flow_table(dev, priority, ns_type, 0, 0);
 	if (IS_ERR(ft_prio)) {
 		err = PTR_ERR(ft_prio);
 		goto free_obj;
@@ -2834,7 +2941,10 @@ DECLARE_UVERBS_NAMED_METHOD(
 			     UA_OPTIONAL),
 	UVERBS_ATTR_CONST_IN(MLX5_IB_ATTR_FLOW_MATCHER_FT_TYPE,
 			     enum mlx5_ib_uapi_flow_table_type,
-			     UA_OPTIONAL));
+			     UA_OPTIONAL),
+	UVERBS_ATTR_PTR_IN(MLX5_IB_ATTR_FLOW_MATCHER_IB_PORT,
+			   UVERBS_ATTR_TYPE(u32),
+			   UA_OPTIONAL));
 
 DECLARE_UVERBS_NAMED_METHOD_DESTROY(
 	MLX5_IB_METHOD_FLOW_MATCHER_DESTROY,
@@ -2904,8 +3014,26 @@ int mlx5_ib_fs_init(struct mlx5_ib_dev *dev)
 	if (!dev->flow_db)
 		return -ENOMEM;
 
+	dev->flow_db->rdma_transport_rx = kcalloc(dev->num_ports,
+					sizeof(struct mlx5_ib_flow_prio),
+					GFP_KERNEL);
+	if (!dev->flow_db->rdma_transport_rx)
+		goto free_flow_db;
+
+	dev->flow_db->rdma_transport_tx = kcalloc(dev->num_ports,
+					sizeof(struct mlx5_ib_flow_prio),
+					GFP_KERNEL);
+	if (!dev->flow_db->rdma_transport_tx)
+		goto free_rdma_transport_rx;
+
 	mutex_init(&dev->flow_db->lock);
 
 	ib_set_device_ops(&dev->ib_dev, &flow_ops);
 	return 0;
+
+free_rdma_transport_rx:
+	kfree(dev->flow_db->rdma_transport_rx);
+free_flow_db:
+	kfree(dev->flow_db);
+	return -ENOMEM;
 }
drivers/infiniband/hw/mlx5/fs.h +2 -0
index b9734904f5f0..0516555eb1c1 100644
@@ -40,6 +40,8 @@ static inline void mlx5_ib_fs_cleanup(struct mlx5_ib_dev *dev)
 	 * is a safe assumption that all references are gone.
 	 */
 	mlx5_ib_fs_cleanup_anchor(dev);
+	kfree(dev->flow_db->rdma_transport_tx);
+	kfree(dev->flow_db->rdma_transport_rx);
 	kfree(dev->flow_db);
 }
 #endif /* _MLX5_IB_FS_H */
drivers/infiniband/hw/mlx5/mlx5_ib.h +3 -0
index 974a45c92fbb..ccaaef20f50d 100644
@@ -276,6 +276,7 @@ struct mlx5_ib_flow_matcher {
 	struct mlx5_core_dev	*mdev;
 	atomic_t		usecnt;
 	u8			match_criteria_enable;
+	u32			ib_port;
 };
 
 struct mlx5_ib_steering_anchor {
@@ -307,6 +308,8 @@ struct mlx5_ib_flow_db {
 	struct mlx5_ib_flow_prio	rdma_tx[MLX5_IB_NUM_FLOW_FT];
 	struct mlx5_ib_flow_prio	opfcs[MLX5_IB_OPCOUNTER_MAX];
 	struct mlx5_flow_table		*lag_demux_ft;
+	struct mlx5_ib_flow_prio        *rdma_transport_rx;
+	struct mlx5_ib_flow_prio        *rdma_transport_tx;
 	/* Protect flow steering bypass flow tables
 	 * when add/del flow rules.
 	 * only single add/removal of flow steering rule could be done
include/uapi/rdma/mlx5_user_ioctl_cmds.h +1 -0
index fd2e4a3a56b3..18f9fe070213 100644
@@ -239,6 +239,7 @@ enum mlx5_ib_flow_matcher_create_attrs {
 	MLX5_IB_ATTR_FLOW_MATCHER_MATCH_CRITERIA,
 	MLX5_IB_ATTR_FLOW_MATCHER_FLOW_FLAGS,
 	MLX5_IB_ATTR_FLOW_MATCHER_FT_TYPE,
+	MLX5_IB_ATTR_FLOW_MATCHER_IB_PORT,
 };
 
 enum mlx5_ib_flow_matcher_destroy_attrs {
include/uapi/rdma/mlx5_user_ioctl_verbs.h +2 -0
index 7c233df475e7..8f86e79d78a5 100644
@@ -45,6 +45,8 @@ enum mlx5_ib_uapi_flow_table_type {
 	MLX5_IB_UAPI_FLOW_TABLE_TYPE_FDB	= 0x2,
 	MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_RX	= 0x3,
 	MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_TX	= 0x4,
+	MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_TRANSPORT_RX	= 0x5,
+	MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_TRANSPORT_TX	= 0x6,
 };
 
 enum mlx5_ib_uapi_flow_action_packet_reformat_type {
Signature · 1 line
2.48.1
#10 lore raw

Re: [PATCH rdma-next v1 5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace

kernel test robot noticed the following build warnings:

Hi Leon,

kernel test robot noticed the following build warnings:

[auto build test WARNING on rdma/for-next]
[also build test WARNING on linus/master v6.14-rc5 next-20250306] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Leon-Romanovsky/RDMA-uverbs-Introduce-UCAP-User-CAPabilities-API/20250306-195708
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git for-next
patch link:    https://lore.kernel.org/r/2287d8c50483e880450c7e8e08d9de34cdec1b14.1741261611.git.leon%40kernel.org
patch subject: [PATCH rdma-next v1 5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace
config: arc-allyesconfig (https://download.01.org/0day-ci/archive/20250307/[email protected]/config)
compiler: arceb-elf-gcc (GCC) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250307/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

drivers/infiniband/hw/mlx5/fs.c: In function '_get_prio':
drivers/infiniband/hw/mlx5/fs.c:702:16: error: 'struct mlx5_flow_table_attr' has no member named 'vport'

     702 |         ft_attr.vport = vport;
         |                ^
   drivers/infiniband/hw/mlx5/fs.c: In function 'mlx5_ib_fill_transport_ns_info':
   drivers/infiniband/hw/mlx5/fs.c:1430:14: error: implicit declaration of function 'MLX5_CAP_ADV_RDMA'; did you mean 'MLX5_CAP_ODP_MAX'? [-Werror=implicit-function-declaration]
    1430 |         if (!MLX5_CAP_ADV_RDMA(dev->mdev, rdma_transport_manager))
         |              ^~~~~~~~~~~~~~~~~
         |              MLX5_CAP_ODP_MAX
   drivers/infiniband/hw/mlx5/fs.c:1430:43: error: 'rdma_transport_manager' undeclared (first use in this function); did you mean 'rdma_transport_type'?
    1430 |         if (!MLX5_CAP_ADV_RDMA(dev->mdev, rdma_transport_manager))
         |                                           ^~~~~~~~~~~~~~~~~~~~~~
         |                                           rdma_transport_type
   drivers/infiniband/hw/mlx5/fs.c:1430:43: note: each undeclared identifier is reported only once for each function it appears in
   drivers/infiniband/hw/mlx5/fs.c: In function '_get_flow_table':
   drivers/infiniband/hw/mlx5/fs.c:1510:14: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS'?
    1510 |         case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
         |              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         |              MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS
   drivers/infiniband/hw/mlx5/fs.c:1511:14: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC'?
    1511 |         case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
         |              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         |              MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC
   drivers/infiniband/hw/mlx5/fs.c:1512:53: error: 'MLX5_RDMA_TRANSPORT_BYPASS_PRIO' undeclared (first use in this function)
    1512 |                 if (ib_port == 0 || user_priority > MLX5_RDMA_TRANSPORT_BYPASS_PRIO)
         |                                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   In file included from include/linux/bits.h:6,
                    from include/linux/bitops.h:6,
                    from include/linux/bitmap.h:8,
                    from include/linux/ethtool.h:16,
                    from include/rdma/ib_verbs.h:15,
                    from drivers/infiniband/hw/mlx5/fs.c:7:
   drivers/infiniband/hw/mlx5/fs.c:1522:37: error: implicit declaration of function 'MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_RX'; did you mean 'MLX5_CAP_FLOWTABLE_RDMA_TX'? [-Werror=implicit-function-declaration]
    1522 |                                 BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_RX(
         |                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/vdso/bits.h:7:44: note: in definition of macro 'BIT'
       7 | #define BIT(nr)                 (UL(1) << (nr))
         |                                            ^~
   drivers/infiniband/hw/mlx5/fs.c:1523:50: error: 'log_max_ft_size' undeclared (first use in this function)
    1523 |                                         ft_mdev, log_max_ft_size));
         |                                                  ^~~~~~~~~~~~~~~
   include/vdso/bits.h:7:44: note: in definition of macro 'BIT'
       7 | #define BIT(nr)                 (UL(1) << (nr))
         |                                            ^~
   drivers/infiniband/hw/mlx5/fs.c:1526:37: error: implicit declaration of function 'MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_TX'; did you mean 'MLX5_CAP_FLOWTABLE_RDMA_TX'? [-Werror=implicit-function-declaration]
    1526 |                                 BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_TX(
         |                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/vdso/bits.h:7:44: note: in definition of macro 'BIT'
       7 | #define BIT(nr)                 (UL(1) << (nr))
         |                                            ^~
   drivers/infiniband/hw/mlx5/fs.c:1538:22: error: implicit declaration of function 'mlx5_get_flow_vport_namespace'; did you mean 'mlx5_get_flow_vport_acl_namespace'? [-Werror=implicit-function-declaration]
    1538 |                 ns = mlx5_get_flow_vport_namespace(ft_mdev, ns_type, vport_idx);
         |                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         |                      mlx5_get_flow_vport_acl_namespace
>> drivers/infiniband/hw/mlx5/fs.c:1538:20: warning: assignment to 'struct mlx5_flow_namespace *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
    1538 |                 ns = mlx5_get_flow_vport_namespace(ft_mdev, ns_type, vport_idx);
         |                    ^
   drivers/infiniband/hw/mlx5/fs.c: In function 'mlx5_ib_ft_type_to_namespace':
   drivers/infiniband/hw/mlx5/fs.c:1814:30: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS'?
    1814 |                 *namespace = MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX;
         |                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         |                              MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS
   drivers/infiniband/hw/mlx5/fs.c:1817:30: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC'?
    1817 |                 *namespace = MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX;
         |                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         |                              MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC
   drivers/infiniband/hw/mlx5/fs.c: In function 'get_dests':
   drivers/infiniband/hw/mlx5/fs.c:1909:37: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS'?
    1909 |              fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX) &&
         |                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         |                                     MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS
   drivers/infiniband/hw/mlx5/fs.c:1949:44: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC'?
    1949 |                     fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX) &&
         |                                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         |                                            MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC
   drivers/infiniband/hw/mlx5/fs.c: In function 'mlx5_ib_handler_MLX5_IB_METHOD_FLOW_MATCHER_CREATE':
   drivers/infiniband/hw/mlx5/fs.c:2500:37: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS'?
    2500 |                 if (obj->ns_type != MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX &&
         |                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         |                                     MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS
   drivers/infiniband/hw/mlx5/fs.c:2501:37: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC'?
    2501 |                     obj->ns_type != MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX) {
         |                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         |                                     MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC

cc1: some warnings being treated as errors

vim +1538 drivers/infiniband/hw/mlx5/fs.c

1447
1448 static struct mlx5_ib_flow_prio *
1449 _get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority,
1450 enum mlx5_flow_namespace_type ns_type,
1451 bool mcast, u32 ib_port)
1452 {
1453 struct mlx5_core_dev *ft_mdev = dev->mdev;
1454 struct mlx5_flow_namespace *ns = NULL;
1455 struct mlx5_ib_flow_prio *prio = NULL;
1456 int max_table_size = 0;
1457 u16 vport_idx = 0;
1458 bool esw_encap;
1459 u32 flags = 0;
1460 u16 vport = 0;
1461 int priority;
1462 int ret;
1463
1464 if (mcast)
1465 priority = MLX5_IB_FLOW_MCAST_PRIO;
1466 else
1467 priority = ib_prio_to_core_prio(user_priority, false);
1468
1469 esw_encap = mlx5_eswitch_get_encap_mode(dev->mdev) !=
1470 DEVLINK_ESWITCH_ENCAP_MODE_NONE;
1471 switch (ns_type) {
1472 case MLX5_FLOW_NAMESPACE_BYPASS:
1473 max_table_size = BIT(
1474 MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, log_max_ft_size));
1475 if (MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, decap) && !esw_encap)
1476 flags |= MLX5_FLOW_TABLE_TUNNEL_EN_DECAP;
1477 if (MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev,
1478 reformat_l3_tunnel_to_l2) &&
1479 !esw_encap)
1480 flags |= MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
1481 break;
1482 case MLX5_FLOW_NAMESPACE_EGRESS:
1483 max_table_size = BIT(
1484 MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, log_max_ft_size));
1485 if (MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, reformat) &&
1486 !esw_encap)
1487 flags |= MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
1488 break;
1489 case MLX5_FLOW_NAMESPACE_FDB_BYPASS:
1490 max_table_size = BIT(
1491 MLX5_CAP_ESW_FLOWTABLE_FDB(dev->mdev, log_max_ft_size));
1492 if (MLX5_CAP_ESW_FLOWTABLE_FDB(dev->mdev, decap) && esw_encap)
1493 flags |= MLX5_FLOW_TABLE_TUNNEL_EN_DECAP;
1494 if (MLX5_CAP_ESW_FLOWTABLE_FDB(dev->mdev,
1495 reformat_l3_tunnel_to_l2) &&
1496 esw_encap)
1497 flags |= MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
1498 priority = user_priority;
1499 break;
1500 case MLX5_FLOW_NAMESPACE_RDMA_RX:
1501 max_table_size = BIT(
1502 MLX5_CAP_FLOWTABLE_RDMA_RX(dev->mdev, log_max_ft_size));
1503 priority = user_priority;
1504 break;
1505 case MLX5_FLOW_NAMESPACE_RDMA_TX:
1506 max_table_size = BIT(
1507 MLX5_CAP_FLOWTABLE_RDMA_TX(dev->mdev, log_max_ft_size));
1508 priority = user_priority;
1509 break;
1510 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
1511 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
1512 if (ib_port == 0 || user_priority > MLX5_RDMA_TRANSPORT_BYPASS_PRIO)
1513 return ERR_PTR(-EINVAL);
1514 ret = mlx5_ib_fill_transport_ns_info(dev, ns_type, &flags,
1515 &vport_idx, &vport,
1516 &ft_mdev, ib_port);
1517 if (ret)
1518 return ERR_PTR(ret);
1519
1520 if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX)
1521 max_table_size =
1522 BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_RX(
1523 ft_mdev, log_max_ft_size));
1524 else
1525 max_table_size =
1526 BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_TX(
1527 ft_mdev, log_max_ft_size));
1528 priority = user_priority;
1529 break;
1530 default:
1531 break;
1532 }
1533
1534 max_table_size = min_t(int, max_table_size, MLX5_FS_MAX_ENTRIES);
1535
1536 if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX ||
1537 ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX)

> 1538			ns = mlx5_get_flow_vport_namespace(ft_mdev, ns_type, vport_idx);

1539 else
1540 ns = mlx5_get_flow_namespace(ft_mdev, ns_type);
1541
1542 if (!ns)
1543 return ERR_PTR(-EOPNOTSUPP);
1544
1545 switch (ns_type) {
1546 case MLX5_FLOW_NAMESPACE_BYPASS:
1547 prio = &dev->flow_db->prios[priority];
1548 break;
1549 case MLX5_FLOW_NAMESPACE_EGRESS:
1550 prio = &dev->flow_db->egress_prios[priority];
1551 break;
1552 case MLX5_FLOW_NAMESPACE_FDB_BYPASS:
1553 prio = &dev->flow_db->fdb[priority];
1554 break;
1555 case MLX5_FLOW_NAMESPACE_RDMA_RX:
1556 prio = &dev->flow_db->rdma_rx[priority];
1557 break;
1558 case MLX5_FLOW_NAMESPACE_RDMA_TX:
1559 prio = &dev->flow_db->rdma_tx[priority];
1560 break;
1561 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
1562 prio = &dev->flow_db->rdma_transport_rx[ib_port - 1];
1563 break;
1564 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
1565 prio = &dev->flow_db->rdma_transport_tx[ib_port - 1];
1566 break;
1567 default: return ERR_PTR(-EINVAL);
1568 }
1569
1570 if (!prio)
1571 return ERR_PTR(-EINVAL);
1572
1573 if (prio->flow_table)
1574 return prio;
1575
1576 return _get_prio(dev, ns, prio, priority, max_table_size,
1577 MLX5_FS_MAX_TYPES, flags, vport);
1578 }
1579

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#11 lore raw

Re: [PATCH rdma-next v1 5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace

In reply to Leon Romanovsky
kernel test robot noticed the following build errors:

Hi Leon,

kernel test robot noticed the following build errors:

[auto build test ERROR on rdma/for-next]
[also build test ERROR on linus/master v6.14-rc5 next-20250307] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Leon-Romanovsky/RDMA-uverbs-Introduce-UCAP-User-CAPabilities-API/20250306-195708
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git for-next
patch link:    https://lore.kernel.org/r/2287d8c50483e880450c7e8e08d9de34cdec1b14.1741261611.git.leon%40kernel.org
patch subject: [PATCH rdma-next v1 5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace
config: s390-defconfig (https://download.01.org/0day-ci/archive/20250308/[email protected]/config)
compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250308/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

drivers/infiniband/hw/mlx5/fs.c:702:10: error: no member named 'vport' in 'struct mlx5_flow_table_attr'

           ft_attr.vport = vport;
           ~~~~~~~ ^
>> drivers/infiniband/hw/mlx5/fs.c:1430:7: error: call to undeclared function 'MLX5_CAP_ADV_RDMA'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
           if (!MLX5_CAP_ADV_RDMA(dev->mdev, rdma_transport_manager))
                ^
   drivers/infiniband/hw/mlx5/fs.c:1430:36: error: use of undeclared identifier 'rdma_transport_manager'; did you mean 'skb_transport_header'?
           if (!MLX5_CAP_ADV_RDMA(dev->mdev, rdma_transport_manager))
                                             ^~~~~~~~~~~~~~~~~~~~~~
                                             skb_transport_header
   include/linux/skbuff.h:3018:30: note: 'skb_transport_header' declared here
   static inline unsigned char *skb_transport_header(const struct sk_buff *skb)
                                ^
   drivers/infiniband/hw/mlx5/fs.c:1510:7: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX'
           case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
                ^
   drivers/infiniband/hw/mlx5/fs.c:1511:7: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX'
           case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
                ^
   drivers/infiniband/hw/mlx5/fs.c:1512:39: error: use of undeclared identifier 'MLX5_RDMA_TRANSPORT_BYPASS_PRIO'
                   if (ib_port == 0 || user_priority > MLX5_RDMA_TRANSPORT_BYPASS_PRIO)
                                                       ^
   drivers/infiniband/hw/mlx5/fs.c:1520:18: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX'
                   if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX)
                                  ^
>> drivers/infiniband/hw/mlx5/fs.c:1522:9: error: call to undeclared function 'MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_RX'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
                                   BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_RX(
                                       ^
   drivers/infiniband/hw/mlx5/fs.c:1523:15: error: use of undeclared identifier 'log_max_ft_size'
                                           ft_mdev, log_max_ft_size));
                                                    ^
>> drivers/infiniband/hw/mlx5/fs.c:1526:9: error: call to undeclared function 'MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_TX'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
                                   BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_TX(
                                       ^
   drivers/infiniband/hw/mlx5/fs.c:1527:15: error: use of undeclared identifier 'log_max_ft_size'
                                           ft_mdev, log_max_ft_size));
                                                    ^
   drivers/infiniband/hw/mlx5/fs.c:1536:17: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX'
           if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX ||
                          ^
   drivers/infiniband/hw/mlx5/fs.c:1537:17: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX'
               ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX)
                          ^
>> drivers/infiniband/hw/mlx5/fs.c:1538:8: error: call to undeclared function 'mlx5_get_flow_vport_namespace'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
                   ns = mlx5_get_flow_vport_namespace(ft_mdev, ns_type, vport_idx);
                        ^
   drivers/infiniband/hw/mlx5/fs.c:1538:8: note: did you mean 'mlx5_get_flow_vport_acl_namespace'?
   include/linux/mlx5/fs.h:197:1: note: 'mlx5_get_flow_vport_acl_namespace' declared here
   mlx5_get_flow_vport_acl_namespace(struct mlx5_core_dev *dev,
   ^
   drivers/infiniband/hw/mlx5/fs.c:1538:6: error: incompatible integer to pointer conversion assigning to 'struct mlx5_flow_namespace *' from 'int' [-Wint-conversion]
                   ns = mlx5_get_flow_vport_namespace(ft_mdev, ns_type, vport_idx);
                      ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   drivers/infiniband/hw/mlx5/fs.c:1561:7: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX'
           case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
                ^
   drivers/infiniband/hw/mlx5/fs.c:1564:7: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX'
           case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
                ^
   drivers/infiniband/hw/mlx5/fs.c:1814:16: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX'
                   *namespace = MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX;
                                ^
   drivers/infiniband/hw/mlx5/fs.c:1817:16: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX'
                   *namespace = MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX;
                                ^

fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated.

vim +/MLX5_CAP_ADV_RDMA +1430 drivers/infiniband/hw/mlx5/fs.c

1417
1418 static int mlx5_ib_fill_transport_ns_info(struct mlx5_ib_dev *dev,

  1419						  enum mlx5_flow_namespace_type type,
  1420						  u32 *flags, u16 *vport_idx,
  1421						  u16 *vport,
  1422						  struct mlx5_core_dev **ft_mdev,
  1423						  u32 ib_port)
  1424	{
  1425		struct mlx5_core_dev *esw_mdev;
  1426	
  1427		if (!is_mdev_switchdev_mode(dev->mdev))
  1428			return 0;
  1429	
> 1430		if (!MLX5_CAP_ADV_RDMA(dev->mdev, rdma_transport_manager))
  1431			return -EOPNOTSUPP;
  1432	
  1433		if (!dev->port[ib_port - 1].rep)
  1434			return -EINVAL;
  1435	
  1436		esw_mdev = mlx5_eswitch_get_core_dev(dev->port[ib_port - 1].rep->esw);
  1437		if (esw_mdev != dev->mdev)
  1438			return -EOPNOTSUPP;
  1439	
  1440		*flags |= MLX5_FLOW_TABLE_OTHER_VPORT;
  1441		*ft_mdev = esw_mdev;
  1442		*vport = dev->port[ib_port - 1].rep->vport;
  1443		*vport_idx = dev->port[ib_port - 1].rep->vport_index;
  1444	
  1445		return 0;
  1446	}
  1447	
  1448	static struct mlx5_ib_flow_prio *
  1449	_get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority,
  1450			enum mlx5_flow_namespace_type ns_type,
  1451			bool mcast, u32 ib_port)
  1452	{
  1453		struct mlx5_core_dev *ft_mdev = dev->mdev;
  1454		struct mlx5_flow_namespace *ns = NULL;
  1455		struct mlx5_ib_flow_prio *prio = NULL;
  1456		int max_table_size = 0;
  1457		u16 vport_idx = 0;
  1458		bool esw_encap;
  1459		u32 flags = 0;
  1460		u16 vport = 0;
  1461		int priority;
  1462		int ret;
  1463	
  1464		if (mcast)
  1465			priority = MLX5_IB_FLOW_MCAST_PRIO;
  1466		else
  1467			priority = ib_prio_to_core_prio(user_priority, false);
  1468	
  1469		esw_encap = mlx5_eswitch_get_encap_mode(dev->mdev) !=
  1470			DEVLINK_ESWITCH_ENCAP_MODE_NONE;
  1471		switch (ns_type) {
  1472		case MLX5_FLOW_NAMESPACE_BYPASS:
  1473			max_table_size = BIT(
  1474				MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, log_max_ft_size));
  1475			if (MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, decap) && !esw_encap)
  1476				flags |= MLX5_FLOW_TABLE_TUNNEL_EN_DECAP;
  1477			if (MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev,
  1478						      reformat_l3_tunnel_to_l2) &&
  1479			    !esw_encap)
  1480				flags |= MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
  1481			break;
  1482		case MLX5_FLOW_NAMESPACE_EGRESS:
  1483			max_table_size = BIT(
  1484				MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, log_max_ft_size));
  1485			if (MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, reformat) &&
  1486			    !esw_encap)
  1487				flags |= MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
  1488			break;
  1489		case MLX5_FLOW_NAMESPACE_FDB_BYPASS:
  1490			max_table_size = BIT(
  1491				MLX5_CAP_ESW_FLOWTABLE_FDB(dev->mdev, log_max_ft_size));
  1492			if (MLX5_CAP_ESW_FLOWTABLE_FDB(dev->mdev, decap) && esw_encap)
  1493				flags |= MLX5_FLOW_TABLE_TUNNEL_EN_DECAP;
  1494			if (MLX5_CAP_ESW_FLOWTABLE_FDB(dev->mdev,
  1495						       reformat_l3_tunnel_to_l2) &&
  1496			    esw_encap)
  1497				flags |= MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
  1498			priority = user_priority;
  1499			break;
  1500		case MLX5_FLOW_NAMESPACE_RDMA_RX:
  1501			max_table_size = BIT(
  1502				MLX5_CAP_FLOWTABLE_RDMA_RX(dev->mdev, log_max_ft_size));
  1503			priority = user_priority;
  1504			break;
  1505		case MLX5_FLOW_NAMESPACE_RDMA_TX:
  1506			max_table_size = BIT(
  1507				MLX5_CAP_FLOWTABLE_RDMA_TX(dev->mdev, log_max_ft_size));
  1508			priority = user_priority;
  1509			break;
  1510		case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
  1511		case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
  1512			if (ib_port == 0 || user_priority > MLX5_RDMA_TRANSPORT_BYPASS_PRIO)
  1513				return ERR_PTR(-EINVAL);
  1514			ret = mlx5_ib_fill_transport_ns_info(dev, ns_type, &flags,
  1515							     &vport_idx, &vport,
  1516							     &ft_mdev, ib_port);
  1517			if (ret)
  1518				return ERR_PTR(ret);
  1519	
  1520			if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX)
  1521				max_table_size =
> 1522					BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_RX(
  1523						ft_mdev, log_max_ft_size));
  1524			else
  1525				max_table_size =
> 1526					BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_TX(
  1527						ft_mdev, log_max_ft_size));
  1528			priority = user_priority;
  1529			break;
  1530		default:
  1531			break;
  1532		}
  1533	
  1534		max_table_size = min_t(int, max_table_size, MLX5_FS_MAX_ENTRIES);
  1535	
  1536		if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX ||
  1537		    ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX)
> 1538			ns = mlx5_get_flow_vport_namespace(ft_mdev, ns_type, vport_idx);

1539 else
1540 ns = mlx5_get_flow_namespace(ft_mdev, ns_type);
1541
1542 if (!ns)
1543 return ERR_PTR(-EOPNOTSUPP);
1544
1545 switch (ns_type) {
1546 case MLX5_FLOW_NAMESPACE_BYPASS:
1547 prio = &dev->flow_db->prios[priority];
1548 break;
1549 case MLX5_FLOW_NAMESPACE_EGRESS:
1550 prio = &dev->flow_db->egress_prios[priority];
1551 break;
1552 case MLX5_FLOW_NAMESPACE_FDB_BYPASS:
1553 prio = &dev->flow_db->fdb[priority];
1554 break;
1555 case MLX5_FLOW_NAMESPACE_RDMA_RX:
1556 prio = &dev->flow_db->rdma_rx[priority];
1557 break;
1558 case MLX5_FLOW_NAMESPACE_RDMA_TX:
1559 prio = &dev->flow_db->rdma_tx[priority];
1560 break;
1561 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
1562 prio = &dev->flow_db->rdma_transport_rx[ib_port - 1];
1563 break;
1564 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
1565 prio = &dev->flow_db->rdma_transport_tx[ib_port - 1];
1566 break;
1567 default: return ERR_PTR(-EINVAL);
1568 }
1569
1570 if (!prio)
1571 return ERR_PTR(-EINVAL);
1572
1573 if (prio->flow_table)
1574 return prio;
1575
1576 return _get_prio(dev, ns, prio, priority, max_table_size,
1577 MLX5_FS_MAX_TYPES, flags, vport);
1578 }
1579

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#12 lore raw

PATCH 6/6 docs: infiniband: document the UCAP API

In reply to Leon Romanovsky

From: Chiara Meiohas <[email protected]>

Add an explanation on the newly added UCAP API.

---
 Documentation/infiniband/index.rst |  1 +
 Documentation/infiniband/ucaps.rst | 71 ++++++++++++++++++++++++++++++
 2 files changed, 72 insertions(+)
 create mode 100644 Documentation/infiniband/ucaps.rst
Documentation/infiniband/index.rst +1 -0
index 9cd7615438b9..5b4c24125f66 100644
@@ -12,6 +12,7 @@ InfiniBand
    opa_vnic
    sysfs
    tag_matching
+   ucaps
    user_mad
    user_verbs
 
Documentation/infiniband/ucaps.rst new file +71 -0

71 lines not on this page · raw message

Signature · 1 line
2.48.1
author Applied
#13 lore raw
In reply to Leon Romanovsky
Quoted 8 lines · Leon Romanovsky

Changelog:
v1:
* Used kref primitives instead of open-coded variant
* Check return value from dev_set_name()
* Added extra brackets around type in UCAP_ENABLED macro
v0: https://lore.kernel.org/all/[email protected]

[...]

Applied, thanks!

[1/6] RDMA/uverbs: Introduce UCAP (User CAPabilities) API

      https://git.kernel.org/rdma/rdma/c/054220ae51ceca
[2/6] RDMA/mlx5: Create UCAP char devices for supported device capabilities
      https://git.kernel.org/rdma/rdma/c/dc0633dbb2abcc
[3/6] RDMA/uverbs: Add support for UCAPs in context creation
      https://git.kernel.org/rdma/rdma/c/8108812389cf29
[4/6] RDMA/mlx5: Check enabled UCAPs when creating ucontext
      https://git.kernel.org/rdma/rdma/c/984ec408e39bf8
[5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace
      https://git.kernel.org/rdma/rdma/c/bec5f67b886710
[6/6] docs: infiniband: document the UCAP API
      https://git.kernel.org/rdma/rdma/c/850a5ea27782a5

Best regards,

Signature · 1 line
Leon Romanovsky <[email protected]>