Skip to contentSkip to conversation

arch generic way to trigger unknown NMIs

RFC In review Don Zickus 4 participants 15 messages Oct 7, 2010 – Oct 17, 2010 Char and misc drivers

lore
Share & tools
lore

Don Zickus

author
#0 lore raw

RFC 1/1 arch generic way to trigger unknown NMIs

Hi,

I was trying to put some automation test scripts together and was looking for an arch neutral way to generate an unknown NMI. I know some arches do not support NMI. Any way I wanted to add the code to the lkdtm module, so I could more easily generate unknown NMIs without hacking up the code.

For x86, I have been using:

drivers/misc/lkdtm.c +6 -0
index ef34de7..c685242 100644
@@ -43,6 +43,7 @@
 #include <linux/slab.h>
 #include <scsi/scsi_cmnd.h>
 #include <linux/debugfs.h>
+#include <asm/apic.h>
 
 #ifdef CONFIG_IDE
 #include <linux/ide.h>
@@ -78,6 +79,7 @@ enum ctype {
 	SOFTLOCKUP,
 	HARDLOCKUP,
 	HUNG_TASK,
+	NMI,
 };
 
 static char* cp_name[] = {
@@ -105,6 +107,7 @@ static char* cp_type[] = {
 	"SOFTLOCKUP",
 	"HARDLOCKUP",
 	"HUNG_TASK",
+	"NMI",
 };
 
 static struct jprobe lkdtm;
@@ -340,6 +343,9 @@ static void lkdtm_do_action(enum ctype which)
 		set_current_state(TASK_UNINTERRUPTIBLE);
 		schedule();
 		break;
+	case NMI:
+		apic->send_IPI_allbutself(NMI_VECTOR);
+		break;
 	case NONE:
 	default:
 		break;

Anyone have any thoughts? Maybe there is an easier way?

Thanks,
Don

#1 lore raw

Having a regression test for this is good, but it would be also good if it wasn't a private one but in some public git repository.

Quoted 12 lines

static struct jprobe lkdtm;

drivers/misc/lkdtm.c +3 -0
@@ -340,6 +343,9 @@ static void lkdtm_do_action(enum ctype which)
 		set_current_state(TASK_UNINTERRUPTIBLE);
 		schedule();
 		break;
+	case NMI:
+		apic->send_IPI_allbutself(NMI_VECTOR);
+		break;
 	case NONE:
 	default:
 		break;
Quoted 1 line

Anyone have any thoughts? Maybe there is an easier way?

Do you really want the NMI on all CPUs - 1? Normally it's directed to a single one.

-Andi

author
#2 lore raw
Quoted 4 lines · Andi Kleen

Having a regression test for this is good, but it would be also good if it wasn't a private one but in some public git repository.

Yeah, I know. It isn't meant to be private, mostly glue logic to load this module using RedHat's internal test harness.

Is there a more public place to add a test like this? I guess that would be LTP. Though last time I looked at LTP, all the tests are written in 'C' whereas I just cobbled together some shell scripts to configure kdump, load the module, panic, process the resulting vmcore to verify it panic'd for the right reason.

Quoted 13 lines
Quoted 13 lines

static struct jprobe lkdtm;

drivers/misc/lkdtm.c +3 -0
@@ -340,6 +343,9 @@ static void lkdtm_do_action(enum ctype which)
 		set_current_state(TASK_UNINTERRUPTIBLE);
 		schedule();
 		break;
+	case NMI:
+		apic->send_IPI_allbutself(NMI_VECTOR);
+		break;
 	case NONE:
 	default:
 		break;
Quoted 4 lines
Quoted 1 line

Anyone have any thoughts? Maybe there is an easier way?

Do you really want the NMI on all CPUs - 1? Normally it's directed to a single one.

No I prefer a single one too, but there didn't seem to be a send_IPI_self() command, so I took the short route and sent it to everyone. :-(

Cheers,
Don

Quoted 2 lines

-Andi

#3 lore raw
Quoted 6 lines · Don Zickus
Quoted 4 lines · Andi Kleen

Having a regression test for this is good, but it would be also good if it wasn't a private one but in some public git repository.

Quoted 8 lines

Yeah, I know. It isn't meant to be private, mostly glue logic to load this module using RedHat's internal test harness.

Is there a more public place to add a test like this? I guess that would be LTP. Though last time I looked at LTP, all the tests are written in 'C' whereas I just cobbled together some shell scripts to configure kdump, load the module, panic, process the resulting vmcore to verify it panic'd for the right reason

One possible place would be mce-test, but LTP would work too I guess.

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

static struct jprobe lkdtm;

drivers/misc/lkdtm.c +3 -0
@@ -340,6 +343,9 @@ static void lkdtm_do_action(enum ctype which)
 		set_current_state(TASK_UNINTERRUPTIBLE);
 		schedule();
 		break;
+	case NMI:
+		apic->send_IPI_allbutself(NMI_VECTOR);
+		break;
 	case NONE:
 	default:
 		break;
Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Anyone have any thoughts? Maybe there is an easier way?

Do you really want the NMI on all CPUs - 1? Normally it's directed to a single one.

No I prefer a single one too, but there didn't seem to be a send_IPI_self() command, so I took the short route and sent it to everyone. :-(

You're not sending it to everyone, everyone but you.

Anyways for standard APIC send_IPI_cpu should be easy enough to add. Standard NMI is usually to CPU #0 only.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
author
#4 lore raw
Quoted 10 lines · Andi Kleen
Quoted 9 lines · Don Zickus
Quoted 4 lines · Andi Kleen

Having a regression test for this is good, but it would be also good if it wasn't a private one but in some public git repository.

Yeah, I know. It isn't meant to be private, mostly glue logic to load this module using RedHat's internal test harness.

Quoted 7 lines
Quoted 5 lines

Is there a more public place to add a test like this? I guess that would be LTP. Though last time I looked at LTP, all the tests are written in 'C' whereas I just cobbled together some shell scripts to configure kdump, load the module, panic, process the resulting vmcore to verify it panic'd for the right reason

One possible place would be mce-test, but LTP would work too I guess.

I forgot about mce-test. I can look at that too.

<snip>

Quoted 9 lines
Quoted 4 lines

No I prefer a single one too, but there didn't seem to be a send_IPI_self() command, so I took the short route and sent it to everyone. :-(

You're not sending it to everyone, everyone but you.

Anyways for standard APIC send_IPI_cpu should be easy enough to add. Standard NMI is usually to CPU #0 only.

Would that still be x86 specific though? I could probably code that up, though I wonder if it would be accepted just for a test case.

Cheers,
Don

#5 lore raw
Quoted 11 lines · Don Zickus
Quoted 8 lines
Quoted 3 lines

No I prefer a single one too, but there didn't seem to be a send_IPI_self() command, so I took the short route and sent it to everyone. :-(

You're not sending it to everyone, everyone but you.

Anyways for standard APIC send_IPI_cpu should be easy enough to add. Standard NMI is usually to CPU #0 only.

Would that still be x86 specific though? I could probably code that up, though I wonder if it would be accepted just for a test case.

The whole APIC interface is x86 specific.

"just a test case" is the wrong perspective.
Testing is important, if a feature is not tested it likely won't work. That's especially important for anything error handling related.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#6 lore raw
In reply to Don Zickus
Quoted 3 lines · Don Zickus

No I prefer a single one too, but there didn't seem to be a send_IPI_self() command, so I took the short route and sent it to everyone. :-(

Odd, APIC hardware does explicitly support such a mode (there are three
shorthands like this actually: all-but-self, all and self), so why don't just add a suitable wrapper?

Maciej

#7 lore raw
Quoted 6 lines · Maciej W. Rozycki
Quoted 3 lines · Don Zickus

No I prefer a single one too, but there didn't seem to be a send_IPI_self() command, so I took the short route and sent it to everyone. :-(

Quoted 6 lines

Odd, APIC hardware does explicitly support such a mode (there are three
shorthands like this actually: all-but-self, all and self), so why don't just add a suitable wrapper?

Maciej

We have send_IPI_self which can't be used for NMI delivery mode (ie with self shortland) but still can be used as say (didn't test)

	apic->send_IPI_mask(cpumask_of(cpu), NMI_VECTOR)

where cpu is a target.

Cyrill

#8 lore raw
Quoted 2 lines · Cyrill Gorcunov

We have send_IPI_self which can't be used for NMI delivery mode (ie with self shortland) but still can be used as say (didn't test)

So what's stopping send_IPI_self() from being used for an NMI? Can't the
function be updated/fixed to support these if there's a need?

Maciej

#9 lore raw
Quoted 10 lines · Maciej W. Rozycki
Quoted 2 lines · Cyrill Gorcunov

We have send_IPI_self which can't be used for NMI delivery mode (ie with self shortland) but still can be used as say (didn't test)

So what's stopping send_IPI_self() from being used for an NMI? Can't the
function be updated/fixed to support these if there's a need?

Maciej

Hi Maciej, the send_IPI_self could be modified to send NMI (at moment it
uses self shortcut with fixed delivery mode). The question is rather if we need it without a real caller yet. When Don's patch gets merged we will have a real caller then and could update send_IPI_self to support NMI delivery mode. Something like that :)

Cyrill

#10 lore raw
Quoted 5 lines · Cyrill Gorcunov

Hi Maciej, the send_IPI_self could be modified to send NMI (at moment it
uses self shortcut with fixed delivery mode). The question is rather if we need it without a real caller yet. When Don's patch gets merged we will have a real caller then and could update send_IPI_self to support NMI delivery mode. Something like that :)

Sounds backwards to me. My understanding is a need has just arisen, so
why not:

1. Update send_IPI_self().

2. Add code that makes use of the new functionality.

3. Submit all the changes as self-contained patches in a single series to be applied at the same time.

? That's what I'd imagine the most natural way of doing this would be.

Maciej

#11 lore raw
Quoted 13 lines · Maciej W. Rozycki
Quoted 5 lines · Cyrill Gorcunov

Hi Maciej, the send_IPI_self could be modified to send NMI (at moment it
uses self shortcut with fixed delivery mode). The question is rather if we need it without a real caller yet. When Don's patch gets merged we will have a real caller then and could update send_IPI_self to support NMI delivery mode. Something like that :)

Sounds backwards to me. My understanding is a need has just arisen, so
why not:

1. Update send_IPI_self().

Quoted 9 lines

2. Add code that makes use of the new functionality.

3. Submit all the changes as self-contained patches in a single series to be applied at the same time.

? That's what I'd imagine the most natural way of doing this would be.

Maciej

Well, Maciej I believe the problem is not in modifying send_IPI_self
but rather _how_ to make it more natural and do not introduce overhead. apic code is already weird enough :) Need to think.

(
btw, we will have to add additional flag which would check for NMI being generated by "NMI-tester" and make a second apic write to ICR to deassert level line, ie it could be something like

  apic->send_IPI_self(NMI_VECTOR)       ; with asserts level
   default_do_nmi()                     ; check for NMI being sent for testing purpose
     apic->send_IPI_self(NMI_VECTOR)    ; with deasserts level

iirc apic itself doesn't deassert nmi line on message with nmi deliver mode arrival
)

Cyrill

#12 lore raw
Quoted 27 lines · Cyrill Gorcunov
Quoted 9 lines
Quoted 5 lines

Hi Maciej, the send_IPI_self could be modified to send NMI (at moment it
uses self shortcut with fixed delivery mode). The question is rather if we need it without a real caller yet. When Don's patch gets merged we will have a real caller then and could update send_IPI_self to support NMI delivery mode. Something like that :)

Sounds backwards to me. My understanding is a need has just arisen, so
why not:

Quoted 8 lines

1. Update send_IPI_self().

2. Add code that makes use of the new functionality.

3. Submit all the changes as self-contained patches in a single series to be applied at the same time.

? That's what I'd imagine the most natural way of doing this would be.

Well, Maciej I believe the problem is not in modifying send_IPI_self
but rather _how_ to make it more natural and do not introduce overhead. apic code is already weird enough :) Need to think.

(
btw, we will have to add additional flag which would check for NMI being generated by "NMI-tester" and make a second apic write to ICR to deassert level line, ie it could be something like

Quoted 7 lines
  apic->send_IPI_self(NMI_VECTOR)       ; with asserts level
   default_do_nmi()                     ; check for NMI being sent for testing purpose
     apic->send_IPI_self(NMI_VECTOR)    ; with deasserts level

iirc apic itself doesn't deassert nmi line on message with nmi deliver mode arrival
)

How different is it to the other two send_IPI shorthand calls? Or the
fully-fledged one? I gather from this thread they already handle NMIs properly, so what is there within that cannot simply be copied over to this one?

Maciej

#13 lore raw
Quoted 37 lines · Maciej W. Rozycki
Quoted 27 lines · Cyrill Gorcunov
Quoted 9 lines
Quoted 5 lines

Hi Maciej, the send_IPI_self could be modified to send NMI (at moment it
uses self shortcut with fixed delivery mode). The question is rather if we need it without a real caller yet. When Don's patch gets merged we will have a real caller then and could update send_IPI_self to support NMI delivery mode. Something like that :)

Sounds backwards to me. My understanding is a need has just arisen, so
why not:

Quoted 8 lines

1. Update send_IPI_self().

2. Add code that makes use of the new functionality.

3. Submit all the changes as self-contained patches in a single series to be applied at the same time.

? That's what I'd imagine the most natural way of doing this would be.

Well, Maciej I believe the problem is not in modifying send_IPI_self
but rather _how_ to make it more natural and do not introduce overhead. apic code is already weird enough :) Need to think.

(
btw, we will have to add additional flag which would check for NMI being generated by "NMI-tester" and make a second apic write to ICR to deassert level line, ie it could be something like

Quoted 7 lines
  apic->send_IPI_self(NMI_VECTOR)       ; with asserts level
   default_do_nmi()                     ; check for NMI being sent for testing purpose
     apic->send_IPI_self(NMI_VECTOR)    ; with deasserts level

iirc apic itself doesn't deassert nmi line on message with nmi deliver mode arrival
)

Quoted 7 lines

How different is it to the other two send_IPI shorthand calls? Or the
fully-fledged one? I gather from this thread they already handle NMIs properly, so what is there within that cannot simply be copied over to this one?

Maciej

Hmm, good question ;) I thought about 82489dx which required level
trigger mode for nmi delivery, new (or present day) apics always use edge trigger mode so indeed there is no need for level assert/deassert.

Cyrill

#14 lore raw

On Sun, Oct 17, 2010 at 12:58:28PM +0400, Cyrill Gorcunov wrote:
...

Quoted 5 lines
Quoted 5 lines

How different is it to the other two send_IPI shorthand calls? Or the
fully-fledged one? I gather from this thread they already handle NMIs properly, so what is there within that cannot simply be copied over to this one?

Quoted 8 lines
Quoted 2 lines

Maciej

Hmm, good question ;) I thought about 82489dx which required level
trigger mode for nmi delivery, new (or present day) apics always use edge trigger mode so indeed there is no need for level assert/deassert.

Cyrill

I'll try to cook a patch for this.

Cyrill