Skip to contentSkip to conversation

block: Use pci_enable_msix_range() instead of pci_enable_msix()

PATCH Superseded Alexander Gordeev 2 participants 16 messages Feb 19, 2014 – Feb 25, 2014

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover block: Use pci_enable_msix_range() instead of pci_enable_msix()

As result of deprecation of MSI-X/MSI enablement functions pci_enable_msix() and pci_enable_msi_block() all drivers using these two interfaces need to be updated to use the new pci_enable_msi_range() and pci_enable_msix_range() interfaces.

Cc Jens Axboe <[email protected]>
Cc Mike Miller <[email protected]>
Cc Asai Thambi S P <[email protected]>
Cc Joshua Morris <[email protected]>
Cc Philip Kelleher <[email protected]>
Cc Kyungmin Park <[email protected]>

Alexander Gordeev (9):
cciss: Fallback to MSI rather than to INTx if MSI-X failed
cciss: Use pci_enable_msix_range() instead of pci_enable_msix()
mtip32xx: Remove superfluous call to pci_disable_msi()
mtip32xx: Use pci_enable_msix_range() instead of pci_enable_msix()
rsxx: Use pci_enable_msix_range() instead of pci_enable_msix()
skd: Fix out of array boundary access
skd: Fix incomplete cleanup of MSI-X interrupt
skd: Use unified access to skdev->msix_entries throughout the code
skd: Use pci_enable_msix_range() instead of pci_enable_msix()

 drivers/block/cciss.c             |   14 +++-----
 drivers/block/mtip32xx/mtip32xx.c |    8 +++--
 drivers/block/rsxx/core.c         |    4 +-
 drivers/block/skd_main.c          |   67 +++++++++++++++----------------------
 4 files changed, 39 insertions(+), 54 deletions(-)
Signature · 1 line
1.7.7.6
#1 lore raw

PATCH 1/9 cciss: Fallback to MSI rather than to INTx if MSI-X failed

---
 drivers/block/cciss.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)
drivers/block/cciss.c +3 -2
index 036e8ab..fc5b0a4 100644
@@ -4092,13 +4092,14 @@ static void cciss_interrupt_mode(ctlr_info_t *h)
 		if (err > 0) {
 			dev_warn(&h->pdev->dev,
 				"only %d MSI-X vectors available\n", err);
-			goto default_int_mode;
+			goto single_msi_mode;
 		} else {
 			dev_warn(&h->pdev->dev,
 				"MSI-X init failed %d\n", err);
-			goto default_int_mode;
+			goto single_msi_mode;
 		}
 	}
+single_msi_mode:
 	if (pci_find_capability(h->pdev, PCI_CAP_ID_MSI)) {
 		if (!pci_enable_msi(h->pdev))
 			h->msi_vector = 1;
Signature · 1 line
1.7.7.6
#2 lore raw

PATCH 2/9 cciss: Use pci_enable_msix_range() instead of pci_enable_msix()

In reply to Alexander Gordeev

As result of deprecation of MSI-X/MSI enablement functions pci_enable_msix() and pci_enable_msi_block() all drivers using these two interfaces need to be updated to use the new pci_enable_msi_range() and pci_enable_msix_range() interfaces.

---
 drivers/block/cciss.c |   11 +++--------
 1 files changed, 3 insertions(+), 8 deletions(-)
drivers/block/cciss.c +3 -8
index fc5b0a4..5463cb5 100644
@@ -4080,19 +4080,14 @@ static void cciss_interrupt_mode(ctlr_info_t *h)
 		goto default_int_mode;
 
 	if (pci_find_capability(h->pdev, PCI_CAP_ID_MSIX)) {
-		err = pci_enable_msix(h->pdev, cciss_msix_entries, 4);
-		if (!err) {
+		err = pci_enable_msix_range(h->pdev, cciss_msix_entries, 4, 4);
+		if (err > 0) {
 			h->intr[0] = cciss_msix_entries[0].vector;
 			h->intr[1] = cciss_msix_entries[1].vector;
 			h->intr[2] = cciss_msix_entries[2].vector;
 			h->intr[3] = cciss_msix_entries[3].vector;
 			h->msix_vector = 1;
 			return;
-		}
-		if (err > 0) {
-			dev_warn(&h->pdev->dev,
-				"only %d MSI-X vectors available\n", err);
-			goto single_msi_mode;
 		} else {
 			dev_warn(&h->pdev->dev,
 				"MSI-X init failed %d\n", err);
@@ -4101,7 +4096,7 @@ static void cciss_interrupt_mode(ctlr_info_t *h)
 	}
 single_msi_mode:
 	if (pci_find_capability(h->pdev, PCI_CAP_ID_MSI)) {
-		if (!pci_enable_msi(h->pdev))
+		if (pci_enable_msi_range(h->pdev, 1, 1) > 0)
 			h->msi_vector = 1;
 		else
 			dev_warn(&h->pdev->dev, "MSI init failed\n");
Signature · 1 line
1.7.7.6
#3 lore raw

PATCH 3/9 mtip32xx: Remove superfluous call to pci_disable_msi()

In reply to Alexander Gordeev

There is no need to call pci_disable_msi() in case the previous call to pci_enable_msi() failed

---
 drivers/block/mtip32xx/mtip32xx.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)
drivers/block/mtip32xx/mtip32xx.c +3 -1
index 5160269..5299a65 100644
@@ -4615,7 +4615,7 @@ static int mtip_pci_probe(struct pci_dev *pdev,
 	if (rv) {
 		dev_warn(&pdev->dev,
 			"Unable to enable MSI interrupt.\n");
-		goto block_initialize_err;
+		goto msi_initialize_err;
 	}
 
 	/* Initialize the block layer. */
@@ -4645,6 +4645,8 @@ static int mtip_pci_probe(struct pci_dev *pdev,
 
 block_initialize_err:
 	pci_disable_msi(pdev);
+
+msi_initialize_err:
 	if (dd->isr_workq) {
 		flush_workqueue(dd->isr_workq);
 		destroy_workqueue(dd->isr_workq);
Signature · 1 line
1.7.7.6
#4 lore raw

PATCH 4/9 mtip32xx: Use pci_enable_msix_range() instead of pci_enable_msix()

In reply to Alexander Gordeev

As result of deprecation of MSI-X/MSI enablement functions pci_enable_msix() and pci_enable_msi_block() all drivers using these two interfaces need to be updated to use the new pci_enable_msi_range() and pci_enable_msix_range() interfaces.

---
 drivers/block/mtip32xx/mtip32xx.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
drivers/block/mtip32xx/mtip32xx.c +2 -2
index 5299a65..60e86e8 100644
@@ -4611,8 +4611,8 @@ static int mtip_pci_probe(struct pci_dev *pdev,
 	INIT_WORK(&dd->work[7].work, mtip_workq_sdbf7);
 
 	pci_set_master(pdev);
-	rv = pci_enable_msi(pdev);
-	if (rv) {
+	rv = pci_enable_msi_range(pdev, 1, 1);
+	if (rv < 0) {
 		dev_warn(&pdev->dev,
 			"Unable to enable MSI interrupt.\n");
 		goto msi_initialize_err;
Signature · 1 line
1.7.7.6
#5 lore raw

PATCH 5/9 rsxx: Use pci_enable_msix_range() instead of pci_enable_msix()

In reply to Alexander Gordeev

As result of deprecation of MSI-X/MSI enablement functions pci_enable_msix() and pci_enable_msi_block() all drivers using these two interfaces need to be updated to use the new pci_enable_msi_range() and pci_enable_msix_range() interfaces.

---
 drivers/block/rsxx/core.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
drivers/block/rsxx/core.c +2 -2
index a8de2ee..76c8124 100644
@@ -880,8 +880,8 @@ static int rsxx_pci_probe(struct pci_dev *dev,
 	spin_unlock_irq(&card->irq_lock);
 
 	if (!force_legacy) {
-		st = pci_enable_msi(dev);
-		if (st)
+		st = pci_enable_msi_range(dev, 1, 1);
+		if (st < 0)
 			dev_warn(CARD_TO_DEV(card),
 				"Failed to enable MSI\n");
 	}
Signature · 1 line
1.7.7.6
#6 lore raw

PATCH 6/9 skd: Fix out of array boundary access

In reply to Alexander Gordeev

When enabling MSI-X, interrupts are requested for SKD_MAX_MSIX_COUNT entries in skdev->msix_entries array, while the number of actually allocated entries is skdev->msix_count. This might lead to an out of boundary access in case number of allocated entries is less than SKD_MAX_MSIX_COUNT. This update fixes the described misbehaviour.

Cc Jens Axboe <[email protected]>
Cc Bartlomiej Zolnierkiewicz <[email protected]>
Cc Kyungmin Park <[email protected]>
---
 drivers/block/skd_main.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
drivers/block/skd_main.c +1 -1
index eb6e1e0..3fa0918 100644
@@ -3989,7 +3989,7 @@ static int skd_acquire_msix(struct skd_device *skdev)
 	}
 
 	/* Enable MSI-X vectors for the base queue */
-	for (i = 0; i < SKD_MAX_MSIX_COUNT; i++) {
+	for (i = 0; i < skdev->msix_count; i++) {
 		qentry = &skdev->msix_entries[i];
 		snprintf(qentry->isr_name, sizeof(qentry->isr_name),
 			 "%s%d-msix %s", DRV_NAME, skdev->devno,
Signature · 1 line
1.7.7.6
#7 lore raw

PATCH 7/9 skd: Fix incomplete cleanup of MSI-X interrupt

In reply to Alexander Gordeev

When enabling MSI-X interrupts fails due to lack of memory the call to pci_disable_msix() is missed and the device is left with MSI-X interrupts enabled while the driver assumes otherwise. This update fixes the described misbehaviour and cleans up the code of skd_release_msix() function.

Cc Jens Axboe <[email protected]>
Cc Bartlomiej Zolnierkiewicz <[email protected]>
Cc Kyungmin Park <[email protected]>
---
 drivers/block/skd_main.c |   50 +++++++++++++++++++++++----------------------
 1 files changed, 26 insertions(+), 24 deletions(-)
drivers/block/skd_main.c +26 -24
index 3fa0918..d0550ba 100644
@@ -3910,43 +3910,44 @@ static void skd_release_msix(struct skd_device *skdev)
 	struct skd_msix_entry *qentry;
 	int i;
 
-	if (skdev->msix_entries == NULL)
-		return;
-	for (i = 0; i < skdev->msix_count; i++) {
-		qentry = &skdev->msix_entries[i];
-		skdev = qentry->rsp;
+	if (skdev->msix_entries) {
+		for (i = 0; i < skdev->msix_count; i++) {
+			qentry = &skdev->msix_entries[i];
+			skdev = qentry->rsp;
+
+			if (qentry->have_irq)
+				devm_free_irq(&skdev->pdev->dev,
+					      qentry->vector, qentry->rsp);
+		}
 
-		if (qentry->have_irq)
-			devm_free_irq(&skdev->pdev->dev,
-				      qentry->vector, qentry->rsp);
+		kfree(skdev->msix_entries);
 	}
-	pci_disable_msix(skdev->pdev);
-	kfree(skdev->msix_entries);
+
+	if (skdev->msix_count)
+		pci_disable_msix(skdev->pdev);
+
 	skdev->msix_count = 0;
 	skdev->msix_entries = NULL;
 }
 
 static int skd_acquire_msix(struct skd_device *skdev)
 {
-	int i, rc;
-	struct pci_dev *pdev;
-	struct msix_entry *entries = NULL;
+	int i, rc, msix_count = SKD_MAX_MSIX_COUNT;
+	struct pci_dev *pdev = skdev->pdev;
+	struct msix_entry *entries;
 	struct skd_msix_entry *qentry;
 
-	pdev = skdev->pdev;
-	skdev->msix_count = SKD_MAX_MSIX_COUNT;
-	entries = kzalloc(sizeof(struct msix_entry) * SKD_MAX_MSIX_COUNT,
-			  GFP_KERNEL);
+	entries = kzalloc(sizeof(struct msix_entry) * msix_count, GFP_KERNEL);
 	if (!entries)
 		return -ENOMEM;
 
-	for (i = 0; i < SKD_MAX_MSIX_COUNT; i++)
+	for (i = 0; i < msix_count; i++)
 		entries[i].entry = i;
 
-	rc = pci_enable_msix(pdev, entries, SKD_MAX_MSIX_COUNT);
+	rc = pci_enable_msix(pdev, entries, msix_count);
 	if (rc < 0)
 		goto msix_out;
-	if (rc) {
+	else if (rc) {
 		if (rc < SKD_MIN_MSIX_COUNT) {
 			pr_err("(%s): failed to enable MSI-X %d\n",
 			       skd_name(skdev), rc);
@@ -3956,20 +3957,21 @@ static int skd_acquire_msix(struct skd_device *skdev)
 			 skdev->name, __func__, __LINE__,
 			 pci_name(pdev), skdev->name, rc);
 
-		skdev->msix_count = rc;
-		rc = pci_enable_msix(pdev, entries, skdev->msix_count);
+		msix_count = rc;
+		rc = pci_enable_msix(pdev, entries, msix_count);
 		if (rc) {
 			pr_err("(%s): failed to enable MSI-X "
 			       "support (%d) %d\n",
-			       skd_name(skdev), skdev->msix_count, rc);
+			       skd_name(skdev), msix_count, rc);
 			goto msix_out;
 		}
 	}
+
+	skdev->msix_count = msix_count;
 	skdev->msix_entries = kzalloc(sizeof(struct skd_msix_entry) *
 				      skdev->msix_count, GFP_KERNEL);
 	if (!skdev->msix_entries) {
 		rc = -ENOMEM;
-		skdev->msix_count = 0;
 		pr_err("(%s): msix table allocation error\n",
 		       skd_name(skdev));
 		goto msix_out;
Signature · 1 line
1.7.7.6
#8 lore raw

PATCH 8/9 skd: Use unified access to skdev->msix_entries throughout the code

In reply to Alexander Gordeev
Cc Jens Axboe <[email protected]>
Cc Bartlomiej Zolnierkiewicz <[email protected]>
Cc Kyungmin Park <[email protected]>
---
 drivers/block/skd_main.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
drivers/block/skd_main.c +1 -2
index d0550ba..1cd1095 100644
@@ -3977,8 +3977,8 @@ static int skd_acquire_msix(struct skd_device *skdev)
 		goto msix_out;
 	}
 
-	qentry = skdev->msix_entries;
 	for (i = 0; i < skdev->msix_count; i++) {
+		qentry = &skdev->msix_entries[i];
 		qentry->vector = entries[i].vector;
 		qentry->entry = entries[i].entry;
 		qentry->rsp = NULL;
@@ -3987,7 +3987,6 @@ static int skd_acquire_msix(struct skd_device *skdev)
 			 skdev->name, __func__, __LINE__,
 			 pci_name(pdev), skdev->name,
 			 i, qentry->vector, qentry->entry);
-		qentry++;
 	}
 
 	/* Enable MSI-X vectors for the base queue */
Signature · 1 line
1.7.7.6
#9 lore raw

PATCH 9/9 skd: Use pci_enable_msix_range() instead of pci_enable_msix()

In reply to Alexander Gordeev

As result of deprecation of MSI-X/MSI enablement functions pci_enable_msix() and pci_enable_msi_block() all drivers using these two interfaces need to be updated to use the new pci_enable_msi_range() and pci_enable_msix_range() interfaces.

Cc Jens Axboe <[email protected]>
Cc Bartlomiej Zolnierkiewicz <[email protected]>
Cc Kyungmin Park <[email protected]>
---
 drivers/block/skd_main.c |   38 ++++++++++++--------------------------
 1 files changed, 12 insertions(+), 26 deletions(-)
drivers/block/skd_main.c +12 -26
index 1cd1095..a69dd93 100644
@@ -3932,42 +3932,28 @@ static void skd_release_msix(struct skd_device *skdev)
 
 static int skd_acquire_msix(struct skd_device *skdev)
 {
-	int i, rc, msix_count = SKD_MAX_MSIX_COUNT;
+	int i, rc;
 	struct pci_dev *pdev = skdev->pdev;
 	struct msix_entry *entries;
 	struct skd_msix_entry *qentry;
 
-	entries = kzalloc(sizeof(struct msix_entry) * msix_count, GFP_KERNEL);
+	entries = kzalloc(sizeof(struct msix_entry) * SKD_MAX_MSIX_COUNT,
+			  GFP_KERNEL);
 	if (!entries)
 		return -ENOMEM;
 
-	for (i = 0; i < msix_count; i++)
+	for (i = 0; i < SKD_MAX_MSIX_COUNT; i++)
 		entries[i].entry = i;
 
-	rc = pci_enable_msix(pdev, entries, msix_count);
-	if (rc < 0)
+	rc = pci_enable_msix_range(pdev, entries,
+				   SKD_MIN_MSIX_COUNT, SKD_MAX_MSIX_COUNT);
+	if (rc < 0) {
+		pr_err("(%s): failed to enable MSI-X %d\n",
+		       skd_name(skdev), rc);
 		goto msix_out;
-	else if (rc) {
-		if (rc < SKD_MIN_MSIX_COUNT) {
-			pr_err("(%s): failed to enable MSI-X %d\n",
-			       skd_name(skdev), rc);
-			goto msix_out;
-		}
-		pr_debug("%s:%s:%d %s: <%s> allocated %d MSI-X vectors\n",
-			 skdev->name, __func__, __LINE__,
-			 pci_name(pdev), skdev->name, rc);
-
-		msix_count = rc;
-		rc = pci_enable_msix(pdev, entries, msix_count);
-		if (rc) {
-			pr_err("(%s): failed to enable MSI-X "
-			       "support (%d) %d\n",
-			       skd_name(skdev), msix_count, rc);
-			goto msix_out;
-		}
 	}
 
-	skdev->msix_count = msix_count;
+	skdev->msix_count = rc;
 	skdev->msix_entries = kzalloc(sizeof(struct skd_msix_entry) *
 				      skdev->msix_count, GFP_KERNEL);
 	if (!skdev->msix_entries) {
@@ -4046,8 +4032,8 @@ RETRY_IRQ_TYPE:
 	case SKD_IRQ_MSI:
 		snprintf(skdev->isr_name, sizeof(skdev->isr_name), "%s%d-msi",
 			 DRV_NAME, skdev->devno);
-		rc = pci_enable_msi(pdev);
-		if (!rc) {
+		rc = pci_enable_msi_range(pdev, 1, 1);
+		if (rc > 0) {
 			rc = devm_request_irq(&pdev->dev, pdev->irq, skd_isr, 0,
 					      skdev->isr_name, skdev);
 			if (rc) {
Signature · 1 line
1.7.7.6
maintainer
#10 lore raw
In reply to Alexander Gordeev
Quoted 5 lines · Alexander Gordeev

As result of deprecation of MSI-X/MSI enablement functions pci_enable_msix() and pci_enable_msi_block() all drivers using these two interfaces need to be updated to use the new pci_enable_msi_range() and pci_enable_msix_range() interfaces.

Thanks. Applied 3-4, and 6-9 (didn't get the others).

Signature · 1 line
Jens Axboe
#11 lore raw
Quoted 8 lines · Jens Axboe
Quoted 5 lines · Alexander Gordeev

As result of deprecation of MSI-X/MSI enablement functions pci_enable_msix() and pci_enable_msi_block() all drivers using these two interfaces need to be updated to use the new pci_enable_msi_range() and pci_enable_msix_range() interfaces.

Thanks. Applied 3-4, and 6-9 (didn't get the others).

Oh, I just realized I posted outdated series, sorry for that :( Out of applied patch 4 is not needed and patch 9 needs a rework.

Should I repost the whole series?

Quoted 2 lines
Signature · 1 line
Jens Axboe
Signature · 3 lines
Regards,
Alexander Gordeev
[email protected]
maintainer
#12 lore raw
Quoted 8 lines · Alexander Gordeev
Quoted 7 lines · Jens Axboe
Quoted 5 lines · Alexander Gordeev

As result of deprecation of MSI-X/MSI enablement functions pci_enable_msix() and pci_enable_msi_block() all drivers using these two interfaces need to be updated to use the new pci_enable_msi_range() and pci_enable_msix_range() interfaces.

Quoted 6 lines
Quoted 1 line

Thanks. Applied 3-4, and 6-9 (didn't get the others).

Oh, I just realized I posted outdated series, sorry for that :( Out of applied patch 4 is not needed and patch 9 needs a rework.

Should I repost the whole series?

Send a revised series against the previous, since they are already committed.

Signature · 1 line
Jens Axboe
#13 lore raw

[PATCH 10/9] skd: Use pci_enable_msix_exact() instead of pci_enable_msix_range()

In reply to Alexander Gordeev

Function pci_enable_msix_exact() is a variation of pci_enable_msix_range() that allows a device driver to request a particular number of MSI-X interrupts, rather than any number within a specified range.

Cc Jens Axboe <[email protected]>
Cc Bartlomiej Zolnierkiewicz <[email protected]>
Cc Kyungmin Park <[email protected]>
---
 drivers/block/skd_main.c |    7 +++----
 1 files changed, 3 insertions(+), 4 deletions(-)
drivers/block/skd_main.c +3 -4
index a69dd93..c244c5bc 100644
@@ -3945,15 +3945,14 @@ static int skd_acquire_msix(struct skd_device *skdev)
 	for (i = 0; i < SKD_MAX_MSIX_COUNT; i++)
 		entries[i].entry = i;
 
-	rc = pci_enable_msix_range(pdev, entries,
-				   SKD_MIN_MSIX_COUNT, SKD_MAX_MSIX_COUNT);
-	if (rc < 0) {
+	rc = pci_enable_msix_exact(pdev, entries, SKD_MAX_MSIX_COUNT);
+	if (rc) {
 		pr_err("(%s): failed to enable MSI-X %d\n",
 		       skd_name(skdev), rc);
 		goto msix_out;
 	}
 
-	skdev->msix_count = rc;
+	skdev->msix_count = SKD_MAX_MSIX_COUNT;
 	skdev->msix_entries = kzalloc(sizeof(struct skd_msix_entry) *
 				      skdev->msix_count, GFP_KERNEL);
 	if (!skdev->msix_entries) {
Signature · 6 lines
1.7.7.6

-- 
Regards,
Alexander Gordeev
[email protected]
#14 lore raw

[PATCH 11/9] mtip32xx: Use pci_enable_msi() instead of pci_enable_msi_range()

In reply to Alexander Gordeev

Commit "mtip32xx: Use pci_enable_msix_range() instead of pci_enable_msix()" was unnecessary, since pci_enable_msi() function is not deprecated and is still preferable for enabling the single MSI mode. This update reverts usage of pci_enable_msi() function.

Besides, the changelog for that commit was bogus, since mtip32xx driver uses MSI interrupt, not MSI-X.

Cc Jens Axboe <[email protected]>
Cc Asai Thambi S P <[email protected]>
---
 drivers/block/mtip32xx/mtip32xx.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
drivers/block/mtip32xx/mtip32xx.c +2 -2
index 60e86e8..5299a65 100644
@@ -4611,8 +4611,8 @@ static int mtip_pci_probe(struct pci_dev *pdev,
 	INIT_WORK(&dd->work[7].work, mtip_workq_sdbf7);
 
 	pci_set_master(pdev);
-	rv = pci_enable_msi_range(pdev, 1, 1);
-	if (rv < 0) {
+	rv = pci_enable_msi(pdev);
+	if (rv) {
 		dev_warn(&pdev->dev,
 			"Unable to enable MSI interrupt.\n");
 		goto msi_initialize_err;
Signature · 6 lines
1.7.7.6

-- 
Regards,
Alexander Gordeev
[email protected]
maintainer
#15 lore raw

Re: [PATCH 10/9] skd: Use pci_enable_msix_exact() instead of pci_enable_msix_range()

Quoted 4 lines · Alexander Gordeev

Function pci_enable_msix_exact() is a variation of pci_enable_msix_range() that allows a device driver to request a particular number of MSI-X interrupts, rather than any number within a specified range.

Applied 10-11, thanks.

Signature · 1 line
Jens Axboe