Skip to contentSkip to conversation

sparc64: sparse irq

PATCH Superseded Bob Picco 3 participants 5 messages Sep 7, 2014 – Sep 10, 2014 SPARC + UltraSPARC (sparc/sparc64)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Bob Picco

author
#0 lore raw

PATCH 1/1 sparc64: sparse irq

From: bob picco <[email protected]>

This patch attempts to do a few things. The highlights are: 1) provides the option to enable SPARSE_IRQ, 2) allocates ivector_table at boot time and
3) default to cookie only VIRQ mechanism for supported firmware. The first firmware with cookie only support for me appears on T5. You can optionally force the HV firmware to not cookie only mode which is the sysino support.

The sysino is a deprecated HV mechanism according to the most recent SPARC Virtual Machine Specification. HV_GRP_INTR is what controls the cookie/sysino firmware versioning. The history appears to be something like this:
major version 2 was dropped.
major version 3 is where a cookie only VIRQ
is possible. Using version 3 means ivector_table isn't required.

A new boot option is provided should cookie only HV support have issues. hvirq - this is the version for HV_GRP_INTR. This is related to HV API versioning. The code attempts major=3 first by default. The option can be used to override this default.

I've tested with SPARSE_IRQ on T5-8, M7-4 and T4-X and Ultra 45. I've also tested Ultra45 (Jalap?no) with !SPARSE_IRQ.

---
 arch/sparc/Kconfig              |    1 +
 arch/sparc/include/asm/irq_64.h |   22 ++-
 arch/sparc/kernel/irq_64.c      |  523 +++++++++++++++++++++++++++++++++------
 3 files changed, 464 insertions(+), 82 deletions(-)
arch/sparc/Kconfig +1 -0
index a537816..4ddb81e 100644
@@ -67,6 +67,7 @@ config SPARC64
 	select HAVE_SYSCALL_TRACEPOINTS
 	select HAVE_CONTEXT_TRACKING
 	select HAVE_DEBUG_KMEMLEAK
+	select MAY_HAVE_SPARSE_IRQ
 	select RTC_DRV_CMOS
 	select RTC_DRV_BQ4802
 	select RTC_DRV_SUN4V
arch/sparc/include/asm/irq_64.h +18 -4
index 91d2193..54c2622 100644
@@ -37,7 +37,17 @@
  *
  * ino_bucket->irq allocation is made during {sun4v_,}build_irq().
  */
-#define NR_IRQS    255
+#ifdef CONFIG_SPARSE_IRQ
+#if	NR_CPUS >	(1024)
+#define NR_IRQS		(8192)
+#elif	NR_CPUS >	(256)
+#define NR_IRQS		(4096)
+#else
+#define	NR_IRQS		(255)
+#endif
+#else
+#define NR_IRQS		(255)
+#endif
 
 void irq_install_pre_handler(int irq,
 			     void (*func)(unsigned int, void *, void *),
@@ -57,11 +67,15 @@ unsigned int sun4u_build_msi(u32 portid, unsigned int *irq_p,
 			     unsigned long iclr_base);
 void sun4u_destroy_msi(unsigned int irq);
 
-unsigned char irq_alloc(unsigned int dev_handle,
-			unsigned int dev_ino);
+#ifdef CONFIG_SPARSE_IRQ
+unsigned int irq_alloc(unsigned int dev_handle, unsigned int dev_ino);
+void irq_free(unsigned int irq);
+#else /* CONFIG_SPARSE_IRQ */
+unsigned char irq_alloc(unsigned int dev_handle, unsigned int dev_ino);
 #ifdef CONFIG_PCI_MSI
 void irq_free(unsigned int irq);
-#endif
+#endif /* CONFIG_PCI_MSI */
+#endif /* CONFIG_SPARSE_IRQ */
 
 void __init init_IRQ(void);
 void fixup_irqs(void);
arch/sparc/kernel/irq_64.c +444 -78

673 lines not on this page · raw message

Signature · 1 line
1.7.1
maintainer
#1 lore raw

From: Bob Picco <[email protected]>
Date: Sun, 7 Sep 2014 11:45:07 -0400

Quoted 22 lines

From: bob picco <[email protected]>

This patch attempts to do a few things. The highlights are: 1) provides the option to enable SPARSE_IRQ, 2) allocates ivector_table at boot time and
3) default to cookie only VIRQ mechanism for supported firmware. The first firmware with cookie only support for me appears on T5. You can optionally force the HV firmware to not cookie only mode which is the sysino support.

The sysino is a deprecated HV mechanism according to the most recent SPARC Virtual Machine Specification. HV_GRP_INTR is what controls the cookie/sysino firmware versioning. The history appears to be something like this:
major version 2 was dropped.
major version 3 is where a cookie only VIRQ
is possible. Using version 3 means ivector_table isn't required.

A new boot option is provided should cookie only HV support have issues. hvirq - this is the version for HV_GRP_INTR. This is related to HV API versioning. The code attempts major=3 first by default. The option can be used to override this default.

Quoted 5 lines

I've tested with SPARSE_IRQ on T5-8, M7-4 and T4-X and Ultra 45. I've also tested Ultra45 (Jalap?no) with !SPARSE_IRQ.

Let's just bite the bullet and enable SPARSE_IRQ unconditionally on SPARC64.

That will decrease the vmlinux image size and increase the testing coverage of SPARSE_IRQ.

#2 lore raw
In reply to Bob Picco

David Miller wrote: [Tue Sep 09 2014, 03:17:52PM EDT]

Quoted 25 lines

From: Bob Picco <[email protected]>
Date: Sun, 7 Sep 2014 11:45:07 -0400

Quoted 22 lines

From: bob picco <[email protected]>

This patch attempts to do a few things. The highlights are: 1) provides the option to enable SPARSE_IRQ, 2) allocates ivector_table at boot time and
3) default to cookie only VIRQ mechanism for supported firmware. The first firmware with cookie only support for me appears on T5. You can optionally force the HV firmware to not cookie only mode which is the sysino support.

The sysino is a deprecated HV mechanism according to the most recent SPARC Virtual Machine Specification. HV_GRP_INTR is what controls the cookie/sysino firmware versioning. The history appears to be something like this:
major version 2 was dropped.
major version 3 is where a cookie only VIRQ
is possible. Using version 3 means ivector_table isn't required.

A new boot option is provided should cookie only HV support have issues. hvirq - this is the version for HV_GRP_INTR. This is related to HV API versioning. The code attempts major=3 first by default. The option can be used to override this default.

Quoted 8 lines
Quoted 5 lines

I've tested with SPARSE_IRQ on T5-8, M7-4 and T4-X and Ultra 45. I've also tested Ultra45 (Jalap?no) with !SPARSE_IRQ.

Let's just bite the bullet and enable SPARSE_IRQ unconditionally on SPARC64.

Okay. Do you require more from me?

Quoted 3 lines

That will decrease the vmlinux image size and increase the testing coverage of SPARSE_IRQ.

Sounds good.

thanx,

bob

maintainer
#3 lore raw
In reply to Bob Picco

From: Bob Picco <[email protected]>
Date: Tue, 9 Sep 2014 15:34:24 -0400

Quoted 29 lines

David Miller wrote: [Tue Sep 09 2014, 03:17:52PM EDT]

Quoted 28 lines

From: Bob Picco <[email protected]>
Date: Sun, 7 Sep 2014 11:45:07 -0400

Quoted 25 lines

From: bob picco <[email protected]>

This patch attempts to do a few things. The highlights are: 1) provides the option to enable SPARSE_IRQ, 2) allocates ivector_table at boot time and
3) default to cookie only VIRQ mechanism for supported firmware. The first firmware with cookie only support for me appears on T5. You can optionally force the HV firmware to not cookie only mode which is the sysino support.

The sysino is a deprecated HV mechanism according to the most recent SPARC Virtual Machine Specification. HV_GRP_INTR is what controls the cookie/sysino firmware versioning. The history appears to be something like this:
major version 2 was dropped.
major version 3 is where a cookie only VIRQ
is possible. Using version 3 means ivector_table isn't required.

A new boot option is provided should cookie only HV support have issues. hvirq - this is the version for HV_GRP_INTR. This is related to HV API versioning. The code attempts major=3 first by default. The option can be used to override this default.

I've tested with SPARSE_IRQ on T5-8, M7-4 and T4-X and Ultra 45. I've also tested Ultra45 (Jalap?no) with !SPARSE_IRQ.

Quoted 6 lines
Quoted 5 lines
Quoted 2 lines

Let's just bite the bullet and enable SPARSE_IRQ unconditionally on SPARC64.

Okay. Do you require more from me?

I only require that you respin this patch to unconditionally enable SPARSE_IRQ.

Well, now that you are asking I do have a few more points I'm interested in :-)

In irq_alloc() you can probably just use irq_alloc_desc(), is IRQ 0 really special in some way that we cannot allocate it? That's the only reason I see for using __irq_alloc_descs().

Also, please use NUMA_NO_NODE for the node argument rather than -1. I am assuming that at some point you will perhaps add some logic to pick a specific node if appropriate.

Also, once you go to using SPARSE_IRQ always there is absolutely no reason to pick one NR_IRQS value over another. I would suggest therefore that you use the maximum possible value which, reading your patch, seems to be something like 8192.

If you still want the conditional NR_IRQS sizing in there, I want to see more details about what exactly determines the size you want to use. It's based upon NR_CPUS now but I can't understand that. What matters is how many virtual or real devices having INOs exist in the machine, and that seems like a purely platform property to me.

author
#4 lore raw
In reply to Bob Picco

David Miller wrote: [Tue Sep 09 2014, 05:40:46PM EDT]

Quoted 35 lines

From: Bob Picco <[email protected]>
Date: Tue, 9 Sep 2014 15:34:24 -0400

Quoted 32 lines

David Miller wrote: [Tue Sep 09 2014, 03:17:52PM EDT]

Quoted 31 lines

From: Bob Picco <[email protected]>
Date: Sun, 7 Sep 2014 11:45:07 -0400

Quoted 27 lines

From: bob picco <[email protected]>

This patch attempts to do a few things. The highlights are: 1) provides the option to enable SPARSE_IRQ, 2) allocates ivector_table at boot time and
3) default to cookie only VIRQ mechanism for supported firmware. The first firmware with cookie only support for me appears on T5. You can optionally force the HV firmware to not cookie only mode which is the sysino support.

The sysino is a deprecated HV mechanism according to the most recent SPARC Virtual Machine Specification. HV_GRP_INTR is what controls the cookie/sysino firmware versioning. The history appears to be something like this:
major version 2 was dropped.
major version 3 is where a cookie only VIRQ
is possible. Using version 3 means ivector_table isn't required.

A new boot option is provided should cookie only HV support have issues. hvirq - this is the version for HV_GRP_INTR. This is related to HV API versioning. The code attempts major=3 first by default. The option can be used to override this default.

I've tested with SPARSE_IRQ on T5-8, M7-4 and T4-X and Ultra 45. I've also tested Ultra45 (Jalap?no) with !SPARSE_IRQ.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

Let's just bite the bullet and enable SPARSE_IRQ unconditionally on SPARC64.

Okay. Do you require more from me?

I only require that you respin this patch to unconditionally enable SPARSE_IRQ.

Okay, MAY_HAVE_SPARSE_IRQ will become SPARSE_IRQ.

Quoted 3 lines

Well, now that you are asking I do have a few more points I'm interested in :-)

Ah, this is okay :)

Quoted 4 lines

In irq_alloc() you can probably just use irq_alloc_desc(), is IRQ 0 really special in some way that we cannot allocate it? That's the only reason I see for using __irq_alloc_descs().

My brain has atrophied for this reasoning - long time ago. I'll need to investigate should you feel it important? I think timer irq but unsure.

Quoted 2 lines

Also, please use NUMA_NO_NODE for the node argument rather than -1.

Okay.

Quoted 2 lines

I am assuming that at some point you will perhaps add some logic to pick a specific node if appropriate.

Yes.

Quoted 5 lines

Also, once you go to using SPARSE_IRQ always there is absolutely no reason to pick one NR_IRQS value over another. I would suggest therefore that you use the maximum possible value which, reading your patch, seems to be something like 8192.

No this is good. I didn't like this aspect. I'll leave it at 8192. I really have no clue what a good value will be for M7. The memory footprint is neglible. The boot time memory is probably comparable to !SPARSE_IRQ. So for 8192 we have a bitmap with 1UL << (13-3) bytes of bitmap.

Quoted 7 lines

If you still want the conditional NR_IRQS sizing in there, I want to see more details about what exactly determines the size you want to use. It's based upon NR_CPUS now but I can't understand that. What matters is how many virtual or real devices having INOs exist in the machine, and that seems like a purely platform property to me.

At one time it was astronomically large because of being pulled away for bugs. I was subjected to some of Bryce's wrath :)

thanx for the additional feedback,

bob

Quoted 4 lines

--
To unsubscribe from this list: send the line "unsubscribe sparclinux" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html