Skip to contentSkip to conversation

: ipr: kexec boot speed improvements

PATCH Superseded Brian King 2 participants 3 messages Jul 28, 2014 – Nov 30, 2014

Subsystems (4)
SCSI Live update kexec handover (KHO) IBM Power Linux RAID adapter
lore
Share & tools
lore

Brian King

author
#0 lore raw

PATCH 3/5 : ipr: kexec boot speed improvements

Currently when performing a kexec boot with an ipr adapter, the adapter gets shutdown completely, flushing all write cache, as well as performing a full hardware reset of the card during the shutdown phase of the old kernel. This forces the new kernel to then wait for the adapter firmware to reinitialize, which slows down kexec boot. There is no need to do all of this on the newer ipr adapters, since they support the ability to cancel outstanding error buffers.

This patch adds a special case for kexec boot to simply cancel these outstanding error buffers, and as long as there is no other I/O outstanding to the adapter, which there shouldn't be, place the adapter into a state where it has no memory of any DMA addresses from the old kernel, then disable the device. This significantly speeds up kexec boot, particularly in configurations with multiple ipr adapters.

Signed-off-by Brian King <[email protected]>

---

 drivers/scsi/ipr.c |  158 ++++++++++++++++++++++++++++++++++++++++++++++++++---
 drivers/scsi/ipr.h |    6 +-
 2 files changed, 155 insertions(+), 9 deletions(-)
drivers/scsi/ipr.h renamed from drivers/scsi/ipr.h~ipr_cancel_hcams2 +5 -1
diff -puN drivers/scsi/ipr.h~ipr_cancel_hcams2 drivers/scsi/ipr.h
@@ -195,6 +195,8 @@
 /*
  * Adapter Commands
  */
+#define IPR_CANCEL_REQUEST				0xC0
+#define	IPR_CANCEL_64BIT_IOARCB			0x01
 #define IPR_QUERY_RSRC_STATE				0xC2
 #define IPR_RESET_DEVICE				0xC3
 #define	IPR_RESET_TYPE_SELECT				0x80

@@ -221,6 +223,7 @@
 #define IPR_ABBREV_SHUTDOWN_TIMEOUT		(10 * HZ)
 #define IPR_DUAL_IOA_ABBR_SHUTDOWN_TO	(2 * 60 * HZ)
 #define IPR_DEVICE_RESET_TIMEOUT		(ipr_fastfail ? 10 * HZ : 30 * HZ)
+#define IPR_CANCEL_TIMEOUT			(ipr_fastfail ? 10 * HZ : 30 * HZ)
 #define IPR_CANCEL_ALL_TIMEOUT		(ipr_fastfail ? 10 * HZ : 30 * HZ)
 #define IPR_ABORT_TASK_TIMEOUT		(ipr_fastfail ? 10 * HZ : 30 * HZ)
 #define IPR_INTERNAL_TIMEOUT			(ipr_fastfail ? 10 * HZ : 30 * HZ)

@@ -1401,7 +1404,8 @@ enum ipr_shutdown_type {
 	IPR_SHUTDOWN_NORMAL = 0x00,
 	IPR_SHUTDOWN_PREPARE_FOR_NORMAL = 0x40,
 	IPR_SHUTDOWN_ABBREV = 0x80,
-	IPR_SHUTDOWN_NONE = 0x100
+	IPR_SHUTDOWN_NONE = 0x100,
+	IPR_SHUTDOWN_KEXEC = 0x101,
 };
 
 struct ipr_trace_entry {
drivers/scsi/ipr.c renamed from drivers/scsi/ipr.c~ipr_cancel_hcams2 +150 -8

214 lines not on this page · raw message

_

Tested-by on v1 3/5
#1 lore raw

Hi Brian,

Quoted 18 lines

Currently when performing a kexec boot with an ipr adapter, the adapter gets shutdown completely, flushing all write cache, as well as performing a full hardware reset of the card during the shutdown phase of the old kernel. This forces the new kernel to then wait for the adapter firmware to reinitialize, which slows down kexec boot. There is no need to do all of this on the newer ipr adapters, since they support the ability to cancel outstanding error buffers.

This patch adds a special case for kexec boot to simply cancel these outstanding error buffers, and as long as there is no other I/O outstanding to the adapter, which there shouldn't be, place the adapter into a state where it has no memory of any DMA addresses from the old kernel, then disable the device. This significantly speeds up kexec boot, particularly in configurations with multiple ipr adapters.

Quoted 2 lines
Signed-off-by Brian King <[email protected]>

---

This reduces boot time on my box from 80 seconds to 34 seconds. ppc64 kernel developers (who boot boxes continually) thank you!

Tested-by Anton Blanchard <[email protected]>

Anton

#2 lore raw

Hi Brian,

Quoted 18 lines
Quoted 18 lines

Currently when performing a kexec boot with an ipr adapter, the adapter gets shutdown completely, flushing all write cache, as well as performing a full hardware reset of the card during the shutdown phase of the old kernel. This forces the new kernel to then wait for the adapter firmware to reinitialize, which slows down kexec boot. There is no need to do all of this on the newer ipr adapters, since they support the ability to cancel outstanding error buffers.

This patch adds a special case for kexec boot to simply cancel these outstanding error buffers, and as long as there is no other I/O outstanding to the adapter, which there shouldn't be, place the adapter into a state where it has no memory of any DMA addresses from the old kernel, then disable the device. This significantly speeds up kexec boot, particularly in configurations with multiple ipr adapters.

Quoted 7 lines
Quoted 2 lines
Signed-off-by Brian King <[email protected]>

---

This reduces boot time on my box from 80 seconds to 34 seconds. ppc64 kernel developers (who boot boxes continually) thank you!

Tested-by Anton Blanchard <[email protected]>

Just checking on this series. I'm very keen to see it go in since it is a huge improvement in boot time.

Anton