Patch "Revert "mlx5: move affinity hints assignments to generic code"" has been added to the 4.14-stable tree
Subsystems (6)
This is a note to let you know that I've just added the patch titled
Revert "mlx5: move affinity hints assignments to generic code"
to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary
The filename of the patch is:
revert-mlx5-move-affinity-hints-assignments-to-generic-code.patch
and it can be found in the queue-4.14 subdirectory.
If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.
Quoted 1 line
From foo@baz Sun Dec 31 11:12:48 CET 2017
From: Saeed Mahameed <[email protected]>
Date: Fri, 10 Nov 2017 15:59:52 +0900
Subject: Revert "mlx5: move affinity hints assignments to generic code"
From: Saeed Mahameed <[email protected]>
[ Upstream commit 231243c82793428467524227ae02ca451e6a98e7 ]
Before the offending commit, mlx5 core did the IRQ affinity itself, and it seems that the new generic code have some drawbacks and one of them is the lack for user ability to modify irq affinity after the initial affinity values got assigned.
The issue is still being discussed and a solution in the new generic code is required, until then we need to revert this patch.
This fixes the following issue:
echo <new affinity> > /proc/irq/<x>/smp_affinity
fails with -EIO
This reverts commit a435393acafbf0ecff4deb3e3cb554b34f0d0664.
Note: kept mlx5_get_vector_affinity in include/linux/mlx5/driver.h since it is used in mlx5_ib driver.
--- drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 45 ++++++------- drivers/net/ethernet/mellanox/mlx5/core/main.c | 75 ++++++++++++++++++++-- include/linux/mlx5/driver.h | 1 4 files changed, 93 insertions(+), 29 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +1 -0
@@ -590,6 +590,7 @@ struct mlx5e_channel {struct mlx5_core_dev *mdev;struct mlx5e_tstamp *tstamp;int ix; +int cpu; };struct mlx5e_channels {
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +22 -23
144 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlx5/core/main.c +69 -6
112 lines not on this page · raw message
include/linux/mlx5/driver.h +1 -0
@@ -546,6 +546,7 @@ struct mlx5_core_sriov { };struct mlx5_irq_info { +cpumask_var_t mask;char name[MLX5_MAX_IRQ_NAME ]; };
Patches currently in stable-queue which might be from [email protected] are
queue-4.14/net-mlx5-fix-rate-limit-packet-pacing-naming-and-struct.patch queue-4.14/revert-mlx5-move-affinity-hints-assignments-to-generic-code.patch queue-4.14/net-mlx5-fpga-return-einval-if-size-is-zero.patch queue-4.14/net-mlx5e-fix-possible-deadlock-of-vxlan-lock.patch queue-4.14/net-mlx5e-prevent-possible-races-in-vxlan-control-flow.patch queue-4.14/net-mlx5-fix-error-flow-in-create_qp-command.patch queue-4.14/net-mlx5e-add-refcount-to-vxlan-structure.patch queue-4.14/net-mlx5e-fix-features-check-of-ipv6-traffic.patch
Quoted 8 lines
This is a note to let you know that I've just added the patch titled
Revert "mlx5: move affinity hints assignments to generic code"
to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary
Quoted 6 lines
The filename of the patch is:
revert-mlx5-move-affinity-hints-assignments-to-generic-code.patch
and it can be found in the queue-4.14 subdirectory.If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.
This patch in its current form, breaks nvme-rdma affinity settings. I've mentioned that in the patch review and Saeed promised to send a fix for it (which I haven't seen so far).
Saeed?
Greg KH
maintainerQuoted 12 lines · Sagi Grimberg
Quoted 12 lines
This is a note to let you know that I've just added the patch titled
Revert "mlx5: move affinity hints assignments to generic code"
to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summaryThe filename of the patch is:
revert-mlx5-move-affinity-hints-assignments-to-generic-code.patch
and it can be found in the queue-4.14 subdirectory.
Quoted 6 lines
Quoted 2 lines
If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.
This patch in its current form, breaks nvme-rdma affinity settings. I've mentioned that in the patch review and Saeed promised to send a fix for it (which I haven't seen so far).
So things are broken in Linus's tree?
Usually reverting is the safe thing to do, we should just be now back at the state the code was in before the patch, or has something else changed here?
thanks,
greg k-h
Quoted 2 lines
So things are broken in Linus's tree?
I wasn't aware that this landed in Linus tree.
CC'ing DaveM.
Quoted 3 lines
Usually reverting is the safe thing to do, we should just be now back at the state the code was in before the patch, or has something else changed here?
The patch title is misleading, its not a revert, but rather a partial unwind of code addition. The breakage lives on the RDMA interface of the driver (which has interfaces to net and infiniband) as some RDMA kernel consumers are relying on the affinity assignments done when passing PCI_IRQ_AFFINITY to pci_alloc_irq_vectors(). The fix is pretty simple, but as said, wasn't submitted yet.
I assume that this went in because the review came a after dave took it.
Greg KH
maintainerQuoted 4 lines · Sagi Grimberg
Quoted 2 lines
So things are broken in Linus's tree?
I wasn't aware that this landed in Linus tree.
That's the only way it would show up in the stable tree :)
Please work on resolving this upstream and I will be glad to take whatever patch happens there in the stable tree afterward.
thanks,
greg k-h
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
So things are broken in Linus's tree?
I wasn't aware that this landed in Linus tree.
That's the only way it would show up in the stable tree :)
I know, was just expressing my surprise..
Quoted 2 lines
Please work on resolving this upstream and I will be glad to take whatever patch happens there in the stable tree afterward.
Saeed, are you submitting a patch for this or should I?
Quoted 5 lines · Sagi Grimberg
Quoted 5 lines
Quoted 4 lines
Quoted 2 lines
So things are broken in Linus's tree?
I wasn't aware that this landed in Linus tree.
Quoted 8 lines
Quoted 1 line
That's the only way it would show up in the stable tree :)
I know, was just expressing my surprise..
Quoted 2 lines
Please work on resolving this upstream and I will be glad to take whatever patch happens there in the stable tree afterward.
Saeed, are you submitting a patch for this or should I?
Yes, I will submit the fix this week I will CC you and Greg, so you will ack it.
Thanks,
Saeed.
Quoted 5 lines · Sagi Grimberg
Quoted 2 lines
So things are broken in Linus's tree?
I wasn't aware that this landed in Linus tree.
Quoted 8 lines
CC'ing DaveM.
Quoted 3 lines
Usually reverting is the safe thing to do, we should just be now back at the state the code was in before the patch, or has something else changed here?
The patch title is misleading, its not a revert, but rather a partial unwind of code addition. The breakage lives on the RDMA interface of
Hi Sagi,
The commit message perfectly states that this revert is slightly modified to keep compilation work, it is not partial.
Quoted 6 lines
the driver (which has interfaces to net and infiniband) as some RDMA kernel consumers are relying on the affinity assignments done when passing
PCI_IRQ_AFFINITYtopci_alloc_irq_vectors(). The fix is pretty simple, but as said, wasn't submitted yet.I assume that this went in because the review came a after dave took it.
We had two options here
1. revert the whole series that introduced the vector affinity infrastructure for RDMA.
2. revert only mlx5 core patch "mlx5: move affinity hints assignments to generic code" with a little change to make it compile with the RDMA infrastructure.
I believe the required fix on top of this revert is to remove the RDMA get_vector_affinity callback from mlx5_ib ib_dev structurem, see below.
I can send it, All i need to know is to where ?
It should go through rdma-rc tree, if there is such thing ?
drivers/infiniband/hw/mlx5/main.c +0 -8
@@ -4256,14 +4256,6 @@ static void init_delay_drop(struct mlx5_ib_dev *dev)mlx5_ib_warn (dev,"Failed to init delay drop debugfs\n" ); } -static const struct cpumask * -mlx5_ib_get_vector_affinity (struct ib_device *ibdev,int comp_vector) -{ -struct mlx5_ib_dev *dev =to_mdev (ibdev); - -return mlx5_get_vector_affinity (dev->mdev, comp_vector); -} -/* The mlx5_ib_multiport_mutex should be held when calling this
function */
static void mlx5_ib_unbind_slave_port(struct mlx5_ib_dev *ibdev,
struct mlx5_ib_multiport_info *mpi)
@@ -4632,7 +4624,6 @@ static int mlx5_ib_stage_caps_init(struct mlx5_ib_dev *dev)
dev->ib_dev.check_mr_status = mlx5_ib_check_mr_status;
dev->ib_dev.get_port_immutable = mlx5_port_immutable;
dev->ib_dev.get_dev_fw_str = get_dev_fw_str;- dev->ib_dev.get_vector_affinity = mlx5_ib_get_vector_affinity;
if (MLX5_CAP_GEN(mdev, ipoib_enhanced_offloads))
dev->ib_dev.alloc_rdma_netdev =mlx5_ib_alloc_rdma_netdev;
Quoted 44 lines · Saeed Mahameed
Quoted 5 lines · Sagi Grimberg
Quoted 2 lines
So things are broken in Linus's tree?
I wasn't aware that this landed in Linus tree.
Quoted 8 lines
CC'ing DaveM.
Quoted 3 lines
Usually reverting is the safe thing to do, we should just be now back at the state the code was in before the patch, or has something else changed here?
The patch title is misleading, its not a revert, but rather a partial unwind of code addition. The breakage lives on the RDMA interface of
Hi Sagi,
The commit message perfectly states that this revert is slightly modified to keep compilation work, it is not partial.
Quoted 6 lines
the driver (which has interfaces to net and infiniband) as some RDMA kernel consumers are relying on the affinity assignments done when passing
PCI_IRQ_AFFINITYtopci_alloc_irq_vectors(). The fix is pretty simple, but as said, wasn't submitted yet.I assume that this went in because the review came a after dave took it.
We had two options here
1. revert the whole series that introduced the vector affinity infrastructure for RDMA.
2. revert only mlx5 core patch "mlx5: move affinity hints assignments to generic code" with a little change to make it compile with the RDMA infrastructure.I believe the required fix on top of this revert is to remove the RDMA
get_vector_affinitycallback frommlx5_ibib_devstructurem, see below.I can send it, All i need to know is to where ?
It should go through rdma-rc tree, if there is such thing ?drivers/infiniband/hw/mlx5/main.c +0
-0@@ -4256,14 +4256,6 @@ static void init_delay_drop(struct mlx5_ib_dev *dev) mlx5_ib_warn(dev, "Failed to init delay drop debugfs\n");
drivers/infiniband/hw/mlx5/main.c static void init_delay_drop(struct mlx5_ib_dev *dev) · line 4266
Quoted 10 lines
drivers/infiniband/hw/mlx5/main.c +0
-8} -static const struct cpumask * -mlx5_ib_get_vector_affinity (struct ib_device *ibdev,int comp_vector) -{ - struct mlx5_ib_dev *dev = to_mdev(ibdev); - - return mlx5_get_vector_affinity(dev->mdev, comp_vector); -} -
Oh, I just remembered that I had suggested to use cpumask_first(mdev->priv.irq_info[vector].mask); instead of the current broken implementation in mlx5, I see that you agreed to test it in another email thread.
I will send a patch very soon, but someone has to test it ASAP so we don't miss 4.15.
Quoted 14 lines
drivers/infiniband/hw/mlx5/main.c +0
-0/* The mlx5_ib_multiport_mutex should be held when calling thisfunction */
static voidmlx5_ib_unbind_slave_port(struct mlx5_ib_dev *ibdev, struct mlx5_ib_multiport_info *mpi)drivers/infiniband/hw/mlx5/main.c +0
-1@@ -4632,7 +4624,6 @@ static int mlx5_ib_stage_caps_init(struct mlx5_ib_dev *dev) dev->ib_dev.check_mr_status = mlx5_ib_check_mr_status; dev->ib_dev.get_port_immutable = mlx5_port_immutable; dev->ib_dev.get_dev_fw_str = get_dev_fw_str; - dev->ib_dev.get_vector_affinity = mlx5_ib_get_vector_affinity; if (MLX5_CAP_GEN(mdev, ipoib_enhanced_offloads)) dev->ib_dev.alloc_rdma_netdev =
mlx5_ib_alloc_rdma_netdev;