Skip to contentSkip to conversation

x86, irq: get correct available vectors for cpu disable

PATCH In review Yinghai Lu 3 participants 8 messages Jan 28, 2014 – Mar 26, 2014 x86

lore
Share & tools
lore

Yinghai Lu

author
#0 lore raw

PATCH v3 1/1 x86, irq: get correct available vectors for cpu disable

used_vectors is a bitmap for vectors that are not tracked in per_cpu vector_irq.
used_vectors contains information on the first 32 exceptions, the system vectors. the IA32_SYSCALL_VECTOR (0x80), and the IRQ_MOVE_CLEANUP_VECTOR (0x20).

assign_irq_vectors() assigns vectors up to first_system_vector and it will not use vectors that are set used_vectors.

This patch modifies the code to scan up to first_system_vector and do a test on the used_vectors bitmap.

So count avaiable vectors correctly.

-v2: fix compiling problem.
-v3: update changelog and commets

---
 arch/x86/kernel/irq.c |   18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)
arch/x86/kernel/irq.c +16 -2
Index: linux-2.6/arch/x86/kernel/irq.c
===================================================================
@@ -17,6 +17,7 @@
 #include <asm/idle.h>
 #include <asm/mce.h>
 #include <asm/hw_irq.h>
+#include <asm/desc.h>
 
 #define CREATE_TRACE_POINTS
 #include <asm/trace/irq_vectors.h>
@@ -321,8 +322,21 @@ int check_irq_vectors_for_cpu_disable(vo
 	for_each_online_cpu(cpu) {
 		if (cpu == this_cpu)
 			continue;
-		for (vector = FIRST_EXTERNAL_VECTOR; vector < NR_VECTORS;
-		     vector++) {
+
+		/*
+		 * assign_irq_vector() only scan per_cpu vectors from
+		 * FIRST_EXTERNAL_VECTOR to first_system_vector.
+		 * It aslo skip vectors that are set in used_vectors bitmask.
+		 * used_vectors could have bits set for
+		 *	IA32_SYSCALL_VECTOR (0x80)
+		 *	IRQ_MOVE_CLEANUP_VECTOR (0x20)
+		 * Don't count those as available vectors.
+		 */
+		for (vector = FIRST_EXTERNAL_VECTOR;
+		     vector < first_system_vector; vector++) {
+			if (test_bit(vector, used_vectors))
+				continue;
+
 			if (per_cpu(vector_irq, cpu)[vector] < 0)
 				count++;
 		}
#1 lore raw
Quoted 14 lines · Yinghai Lu

used_vectors is a bitmap for vectors that are not tracked in per_cpu vector_irq.
used_vectors contains information on the first 32 exceptions, the system vectors. the IA32_SYSCALL_VECTOR (0x80), and the IRQ_MOVE_CLEANUP_VECTOR (0x20).

assign_irq_vectors() assigns vectors up to first_system_vector and it will not use vectors that are set used_vectors.

This patch modifies the code to scan up to first_system_vector and do a test on the used_vectors bitmap.

So count avaiable vectors correctly.

Review

Quoted 2 lines
(quoted diff) +0 -2
-v2: fix compiling problem.
-v3: update changelog and commets

Yinghai, I have not forgotten about this patch. I'm waiting for the system to become available so that I can test it. My apologies for the inconvenience,

P.

Quoted 42 lines
---
 arch/x86/kernel/irq.c |   18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

Index: linux-2.6/arch/x86/kernel/irq.c
===================================================================

arch/x86/kernel/irq.c +16 -2
@@ -17,6 +17,7 @@
 #include <asm/idle.h>
 #include <asm/mce.h>
 #include <asm/hw_irq.h>
+#include <asm/desc.h>
 
 #define CREATE_TRACE_POINTS
 #include <asm/trace/irq_vectors.h>
@@ -321,8 +322,21 @@ int check_irq_vectors_for_cpu_disable(vo
 	for_each_online_cpu(cpu) {
 		if (cpu == this_cpu)
 			continue;
-		for (vector = FIRST_EXTERNAL_VECTOR; vector < NR_VECTORS;
-		     vector++) {
+
+		/*
+		 * assign_irq_vector() only scan per_cpu vectors from
+		 * FIRST_EXTERNAL_VECTOR to first_system_vector.
+		 * It aslo skip vectors that are set in used_vectors bitmask.
+		 * used_vectors could have bits set for
+		 *	IA32_SYSCALL_VECTOR (0x80)
+		 *	IRQ_MOVE_CLEANUP_VECTOR (0x20)
+		 * Don't count those as available vectors.
+		 */
+		for (vector = FIRST_EXTERNAL_VECTOR;
+		     vector < first_system_vector; vector++) {
+			if (test_bit(vector, used_vectors))
+				continue;
+
 			if (per_cpu(vector_irq, cpu)[vector] < 0)
 				count++;
 		}
Reviewed-by Tested-by on v3 1/1
#2 lore raw
In reply to Yinghai Lu
Quoted 35 lines · Yinghai Lu

used_vectors is a bitmap for vectors that are not tracked in per_cpu vector_irq.
used_vectors contains information on the first 32 exceptions, the system vectors. the IA32_SYSCALL_VECTOR (0x80), and the IRQ_MOVE_CLEANUP_VECTOR (0x20).

assign_irq_vectors() assigns vectors up to first_system_vector and it will not use vectors that are set used_vectors.

This patch modifies the code to scan up to first_system_vector and do a test on the used_vectors bitmap.

So count avaiable vectors correctly.

(quoted diff) +0 -2
-v2: fix compiling problem.
-v3: update changelog and commets
---
 arch/x86/kernel/irq.c |   18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

Index: linux-2.6/arch/x86/kernel/irq.c
===================================================================

arch/x86/kernel/irq.c +1 -0
@@ -17,6 +17,7 @@
 #include <asm/idle.h>
 #include <asm/mce.h>
 #include <asm/hw_irq.h>
+#include <asm/desc.h>
 
 #define CREATE_TRACE_POINTS
 #include <asm/trace/irq_vectors.h>

arch/x86/kernel/irq.c int check_irq_vectors_for_cpu_disable(vo · line 329

Quoted 10 lines
arch/x86/kernel/irq.c +5 -2
@@ -321,8 +322,21 @@ int check_irq_vectors_for_cpu_disable(vo
 	for_each_online_cpu(cpu) {
 		if (cpu == this_cpu)
 			continue;
-		for (vector = FIRST_EXTERNAL_VECTOR; vector < NR_VECTORS;
-		     vector++) {
+
+		/*
+		 * assign_irq_vector() only scan per_cpu vectors from
+		 * FIRST_EXTERNAL_VECTOR to first_system_vector.
+		 * It aslo skip vectors that are set in used_vectors bitmask.

"also" ... but not a big deal IMO.

P.

Quoted 13 lines
arch/x86/kernel/irq.c +10 -0
+		 * used_vectors could have bits set for
+		 *	IA32_SYSCALL_VECTOR (0x80)
+		 *	IRQ_MOVE_CLEANUP_VECTOR (0x20)
+		 * Don't count those as available vectors.
+		 */
+		for (vector = FIRST_EXTERNAL_VECTOR;
+		     vector < first_system_vector; vector++) {
+			if (test_bit(vector, used_vectors))
+				continue;
+
 			if (per_cpu(vector_irq, cpu)[vector] < 0)
 				count++;
 		}
#3 lore raw
In reply to Yinghai Lu

Thanks for the patch.

Quoted 2 lines · Yinghai Lu

used_vectors is a bitmap for vectors that are not tracked in per_cpu vector_irq.

I feel like this comment (also in the code) could be misleading because vectors above first_system_vector are effectively not tracked in per_cpu vector_irq, but also may not have the bit set in used_vectors. For example, used_vectors from a system that I am looking at now:

first_system_vector

                239                  255
                  |                  |
                  10 01000 11111 11111

test_bit(240, used_vectors) does not return the correct answer to the question about whether the vector is tracked in per_cpu vector_irq. This leads to two meanings for the bitmap; for vectors less than first_system_vector whether or not they are tracked in per_cpu vector_irq, and for vectors above first_system_vector, whether or not they are in use:

static inline int is_per_cpu_vector(int vector) {
	return !test_bit(vector, used_vectors) &&
			vector < first_system_vector;
}

Thanks,
Linn

Quoted 56 lines

used_vectors contains information on the first 32 exceptions, the system vectors. the IA32_SYSCALL_VECTOR (0x80), and the IRQ_MOVE_CLEANUP_VECTOR (0x20).

assign_irq_vectors() assigns vectors up to first_system_vector and it will not use vectors that are set used_vectors.

This patch modifies the code to scan up to first_system_vector and do a test on the used_vectors bitmap.

So count avaiable vectors correctly.

(quoted diff) +0 -2
-v2: fix compiling problem.
-v3: update changelog and commets
---
 arch/x86/kernel/irq.c |   18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

Index: linux-2.6/arch/x86/kernel/irq.c
===================================================================

arch/x86/kernel/irq.c +16 -2
@@ -17,6 +17,7 @@
 #include <asm/idle.h>
 #include <asm/mce.h>
 #include <asm/hw_irq.h>
+#include <asm/desc.h>
 
 #define CREATE_TRACE_POINTS
 #include <asm/trace/irq_vectors.h>
@@ -321,8 +322,21 @@ int check_irq_vectors_for_cpu_disable(vo
 	for_each_online_cpu(cpu) {
 		if (cpu == this_cpu)
 			continue;
-		for (vector = FIRST_EXTERNAL_VECTOR; vector < NR_VECTORS;
-		     vector++) {
+
+		/*
+		 * assign_irq_vector() only scan per_cpu vectors from
+		 * FIRST_EXTERNAL_VECTOR to first_system_vector.
+		 * It aslo skip vectors that are set in used_vectors bitmask.
+		 * used_vectors could have bits set for
+		 *	IA32_SYSCALL_VECTOR (0x80)
+		 *	IRQ_MOVE_CLEANUP_VECTOR (0x20)
+		 * Don't count those as available vectors.
+		 */
+		for (vector = FIRST_EXTERNAL_VECTOR;
+		     vector < first_system_vector; vector++) {
+			if (test_bit(vector, used_vectors))
+				continue;
+
 			if (per_cpu(vector_irq, cpu)[vector] < 0)
 				count++;
 		}
author
#4 lore raw
Quoted 22 lines · Linn Crosetto

Thanks for the patch.

Quoted 2 lines · Yinghai Lu

used_vectors is a bitmap for vectors that are not tracked in per_cpu vector_irq.

I feel like this comment (also in the code) could be misleading because vectors above first_system_vector are effectively not tracked in per_cpu vector_irq, but also may not have the bit set in used_vectors. For example, used_vectors from a system that I am looking at now:

first_system_vector

                239                  255
                  |                  |
                  10 01000 11111 11111

test_bit(240, used_vectors) does not return the correct answer to the question about whether the vector is tracked in per_cpu vector_irq. This leads to two meanings for the bitmap; for vectors less than first_system_vector whether or not they are tracked in per_cpu vector_irq, and for vectors above first_system_vector, whether or not they are in use:

Quoted 4 lines
static inline int is_per_cpu_vector(int vector) {
        return !test_bit(vector, used_vectors) &&
                        vector < first_system_vector;
}

sorry, I can not catch what you want to say.

Do you mean the change log or comment in the patch is not right?

Thanks

Yinghai

#5 lore raw
Quoted 23 lines · Yinghai Lu
Quoted 22 lines · Linn Crosetto

Thanks for the patch.

Quoted 2 lines · Yinghai Lu

used_vectors is a bitmap for vectors that are not tracked in per_cpu vector_irq.

I feel like this comment (also in the code) could be misleading because vectors above first_system_vector are effectively not tracked in per_cpu vector_irq, but also may not have the bit set in used_vectors. For example, used_vectors from a system that I am looking at now:

first_system_vector

                239                  255
                  |                  |
                  10 01000 11111 11111

test_bit(240, used_vectors) does not return the correct answer to the question about whether the vector is tracked in per_cpu vector_irq. This leads to two meanings for the bitmap; for vectors less than first_system_vector whether or not they are tracked in per_cpu vector_irq, and for vectors above first_system_vector, whether or not they are in use:

Quoted 8 lines
Quoted 4 lines
static inline int is_per_cpu_vector(int vector) {
        return !test_bit(vector, used_vectors) &&
                        vector < first_system_vector;
}

sorry, I can not catch what you want to say.

Do you mean the change log or comment in the patch is not right?

Just noting that not all bits above first_system_vector are set in the bitmap, so the comment in asm/desc.h and the change log could be misleading:

/* used_vectors is BITMAP for irq is not managed by percpu vector_irq */

I have tested and the patch itself is good.

Thanks,
Linn

author
#6 lore raw
Quoted 7 lines · Linn Crosetto

On Tue, Mar 25, 2014 at 05:18:53PM -0700, Yinghai Lu wrote:

Just noting that not all bits above first_system_vector are set in the bitmap, so the comment in asm/desc.h and the change log could be misleading:

/* used_vectors is BITMAP for irq is not managed by percpu vector_irq */

Do you mean some vector that is neither used by percpu_vector_irq nor system_vector
after first_system_vector ?

Anyway if bit is set in used_vectors, that vector can not be used by percpu vector_irq.

so that statement looks still right.

Feel free to suggest right comment or changelog.

Quoted 1 line

I have tested and the patch itself is good.

Good.

Thanks

Yinghai

#7 lore raw
Quoted 6 lines · Yinghai Lu
Quoted 5 lines · Linn Crosetto

On Tue, Mar 25, 2014 at 05:18:53PM -0700, Yinghai Lu wrote:

Just noting that not all bits above first_system_vector are set in the bitmap, so the comment in asm/desc.h and the change log could be misleading:

Quoted 6 lines
Quoted 2 lines

/* used_vectors is BITMAP for irq is not managed by percpu vector_irq */

Do you mean some vector that is neither used by percpu_vector_irq nor system_vector
after first_system_vector ?

Yes, that is the case. In my previous example, first_system_vector is 239, but test_bit(240, used_vectors) would return unset, though it cannot be used by percpu vector_irq. If it marked all vectors unusable by percpu vector_irq, then all bits above first_system_vector would be set.

Quoted 6 lines

Anyway if bit is set in used_vectors, that vector can not be used by percpu vector_irq.

so that statement looks still right.

Feel free to suggest right comment or changelog.

The following might be more accurate:

/*
* The used_vectors BITMAP marks IRQs not managed by percpu vector_irq, below
* first_system_vector. Vectors above first_system_vector are not managed by
* percpu vector_irq.
*/