Skip to contentSkip to conversation

AHCI: Optimize interrupt processing

RFC PATCH Applied Alexander Gordeev 2 participants 4 messages Mar 6, 2013 – Apr 3, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover AHCI: Optimize interrupt processing

Hi Jeff,

As I am not sure about my reading of the statistics and the trade-off I identified (below), this patch a RFC.

The numbers are taken by running 'if=/dev/sd{a,b,c} of=/dev/null' All time values is in us.

Before this update host lock average holdtime was 3.266532061 and average waittime was 0.009832679 [1]. After the update average holdtime (slightly) rose up to 0.335267418 while average waittime decreased to 0.000320469 [2]. Which means host lock with local interrupt disabled is held roughly the same while the average waittime dropped 30 times.

After this update port events are handled with local interrupts enabled and compete on individual per-port locks with average holdtime 1.540987475 and average waittime 0.000714864 [3]. Comparing to [1], ata_scsi_queuecmd() holds port locks 2 times less and waits for locks 13 times less.

The downside of this change is introduction of a kernel thread and (supposedly) increased total average time of handling a AHCI interrupt - at most 1.5 times.

The upside is better access times from ata_scsi_queuecmd() to port locks and moving port interrupt handling out of the hardware interrupt context.

Thanks!

Lock usage statistics.

1. ahci_interrupt vs ata_scsi_queuecmd (host->lock)

Test holdtime-total waittime-total acquisitions holdtime-avg waittime-avg
#
01. 22732497.77 93399.89 06393367 3.555637862 0.014608874
02. 20358052.08 52869.72 06454133 3.154265969 0.008191607
03. 20322516.57 54981.40 06459318 3.146232554 0.008511951
04. 18558686.89 39178.05 06469468 2.868657344 0.006055838
05. 19069799.90 31961.00 06455953 2.953831897 0.004950625
06. 23783542.56 97159.79 06387322 3.723554654 0.015211350
07. 23889266.74 102625.45 06386666 3.740491007 0.016068705
08. 19284522.61 32655.91 06450568 2.989585198 0.005062486

							-----------	-----------

avg 3.266532061 0.009832679

2. ahci_single_irq_intr vs ahci_port_thread_fn (host->lock)

Test holdtime-total waittime-total acquisitions holdtime-avg waittime-avg
#
01. 7572820.95 08414.47 21614279 0.350361951 0.000389301
02. 7029942.02 02992.88 21680248 0.324255609 0.000138046
03. 7298791.32 05974.01 21701755 0.336322630 0.000275278
04. 7077621.26 04360.66 21745564 0.325474256 0.000275278
05. 7606619.38 12158.88 21832248 0.348412100 0.000556923
06. 7605174.70 12205.41 21828021 0.348413386 0.000559162
07. 7058514.97 03701.46 21740632 0.324669263 0.000170255
08. 7039641.91 04331.74 21711867 0.324230151 0.000199510

							-----------	-----------

avg 0.335267418 0.000320469

3. ahci_port_thread_fn vs ata_scsi_queuecmd (pp->lock)

Test holdtime-total waittime-total acquisitions holdtime-avg waittime-avg
#
01. 38531452.56 24236.62 21752844 1.771329421 0.001114182
02. 00024050.92 00005.84 00015999 1.503276455 0.000365023
03. 34071257.07 09699.50 21829593 1.560782973 0.000444328
04. 00019086.10 00001.37 00016054 1.188868818 0.000085337
05. 36535324.31 20676.43 21949439 1.664522009 0.000942003
06. 37433608.34 22942.34 21945295 1.705769202 0.001045433
07. 31456091.88 19895.22 21862170 1.438836670 0.000910030
08. 32628001.94 17739.99 21831844 1.494514249 0.000812574

							-----------	-----------

avg 1.540987475 0.000714864

Alexander Gordeev (1):
AHCI: Optimize interrupt processing

 drivers/ata/acard-ahci.c    |    8 ++---
 drivers/ata/ahci.c          |   54 ++++++++++++++++++-------------
 drivers/ata/ahci.h          |   10 +++--
 drivers/ata/ahci_platform.c |    3 +-
 drivers/ata/libahci.c       |   74 +++++++++++++++++++++++++------------------
 5 files changed, 85 insertions(+), 64 deletions(-)
Signature · 7 lines
1.7.7.6


-- 
Regards,
Alexander Gordeev
[email protected]
#1 lore raw

RFC PATCH 1/1 AHCI: Optimize interrupt processing

Split interrupt service routine into hardware context handler and threaded context handler. That allows to protect ports with individual locks rather than with a single host-wide lock, which results in better parallelism.

---
 drivers/ata/acard-ahci.c    |    8 ++---
 drivers/ata/ahci.c          |   54 ++++++++++++++++++-------------
 drivers/ata/ahci.h          |   10 +++--
 drivers/ata/ahci_platform.c |    3 +-
 drivers/ata/libahci.c       |   74 +++++++++++++++++++++++++------------------
 5 files changed, 85 insertions(+), 64 deletions(-)
drivers/ata/acard-ahci.c +3 -5
index 4e94ba2..e429225 100644
@@ -409,7 +409,7 @@ static int acard_ahci_init_one(struct pci_dev *pdev, const struct pci_device_id
 	struct device *dev = &pdev->dev;
 	struct ahci_host_priv *hpriv;
 	struct ata_host *host;
-	int n_ports, i, rc;
+	int n_ports, n_msis, i, rc;
 
 	VPRINTK("ENTER\n");
 
@@ -436,8 +436,7 @@ static int acard_ahci_init_one(struct pci_dev *pdev, const struct pci_device_id
 		return -ENOMEM;
 	hpriv->flags |= (unsigned long)pi.private_data;
 
-	if (!(hpriv->flags & AHCI_HFLAG_NO_MSI))
-		pci_enable_msi(pdev);
+	n_msis = ahci_init_interrupts(pdev, hpriv);
 
 	hpriv->mmio = pcim_iomap_table(pdev)[AHCI_PCI_BAR];
 
@@ -499,8 +498,7 @@ static int acard_ahci_init_one(struct pci_dev *pdev, const struct pci_device_id
 	acard_ahci_pci_print_info(host);
 
 	pci_set_master(pdev);
-	return ata_host_activate(host, pdev->irq, ahci_interrupt, IRQF_SHARED,
-				 &acard_ahci_sht);
+	return ahci_host_activate(host, pdev->irq, n_msis, &acard_ahci_sht);
 }
 
 module_pci_driver(acard_ahci_pci_driver);
drivers/ata/ahci.c +32 -22

108 lines not on this page · raw message

drivers/ata/ahci.h +6 -4
index 3ed76cc..0172bbd 100644
@@ -216,7 +216,6 @@ enum {
 	AHCI_HFLAG_DELAY_ENGINE		= (1 << 15), /* do not start engine on
 						        port start (wait until
 						        error-handling stage) */
-	AHCI_HFLAG_MULTI_MSI		= (1 << 16), /* multiple PCI MSIs */
 
 	/* ap->flags bits */
 
@@ -345,11 +344,14 @@ int ahci_port_resume(struct ata_port *ap);
 void ahci_set_em_messages(struct ahci_host_priv *hpriv,
 			  struct ata_port_info *pi);
 int ahci_reset_em(struct ata_host *host);
-irqreturn_t ahci_interrupt(int irq, void *dev_instance);
-irqreturn_t ahci_hw_interrupt(int irq, void *dev_instance);
+irqreturn_t ahci_single_irq_intr(int irq, void *dev_instance);
+irqreturn_t ahci_multi_irqs_intr(int irq, void *dev_instance);
 irqreturn_t ahci_thread_fn(int irq, void *dev_instance);
+irqreturn_t ahci_port_thread_fn(int irq, void *dev_instance);
 void ahci_print_info(struct ata_host *host, const char *scc_s);
-int ahci_host_activate(struct ata_host *host, int irq, unsigned int n_msis);
+int ahci_host_activate(struct ata_host *host, int irq, unsigned int n_msis,
+		       struct scsi_host_template *sht);
+int ahci_init_interrupts(struct pci_dev *pdev, struct ahci_host_priv *hpriv);
 
 static inline void __iomem *__ahci_port_base(struct ata_host *host,
 					     unsigned int port_no)
drivers/ata/ahci_platform.c +1 -2
index 09728e0..9c1f485 100644
@@ -188,8 +188,7 @@ static int __init ahci_probe(struct platform_device *pdev)
 	ahci_init_controller(host);
 	ahci_print_info(host, "platform");
 
-	rc = ata_host_activate(host, irq, ahci_interrupt, IRQF_SHARED,
-			       &ahci_platform_sht);
+	rc = ahci_host_activate(host, irq, 0, &ahci_platform_sht);
 	if (rc)
 		goto err0;
 
drivers/ata/libahci.c +43 -31

140 lines not on this page · raw message

Signature · 7 lines
1.7.7.6


-- 
Regards,
Alexander Gordeev
[email protected]
#2 lore raw
Quoted 4 lines · Alexander Gordeev

Split interrupt service routine into hardware context handler and threaded context handler. That allows to protect ports with individual locks rather than with a single host-wide lock, which results in better parallelism.

Hi Jeff,

Any comment on this change?
Thanks!

Signature · 3 lines
Regards,
Alexander Gordeev
[email protected]
Applied
#3 lore raw
In reply to Alexander Gordeev
Quoted 5 lines · Alexander Gordeev

Split interrupt service routine into hardware context handler and threaded context handler. That allows to protect ports with individual locks rather than with a single host-wide lock, which results in better parallelism.

Quoted 8 lines
---
  drivers/ata/acard-ahci.c    |    8 ++---
  drivers/ata/ahci.c          |   54 ++++++++++++++++++-------------
  drivers/ata/ahci.h          |   10 +++--
  drivers/ata/ahci_platform.c |    3 +-
  drivers/ata/libahci.c       |   74 +++++++++++++++++++++++++------------------
  5 files changed, 85 insertions(+), 64 deletions(-)

applied