Skip to contentSkip to conversation

Drivers: hv: vmbus: Add cpu read lock

PATCH In review Saurabh Sengar 3 participants 4 messages Jun 9, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Drivers: hv: vmbus: Add cpu read lock

Add cpus_read_lock to prevent CPUs from going offline between query and actual use of cpumask. cpumask_of_node is first queried, and based on it used later, in case any CPU goes offline between these two events, it can potentially cause an infinite loop of retries.

---
 drivers/hv/channel_mgmt.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/hv/channel_mgmt.c +4 -0
index 85a2142..6a88b7e 100644
@@ -749,6 +749,9 @@ static void init_vp_index(struct vmbus_channel *channel)
 		return;
 	}
 
+	/* No CPUs should come up or down during this. */
+	cpus_read_lock();
+
 	for (i = 1; i <= ncpu + 1; i++) {
 		while (true) {
 			numa_node = next_numa_node_id++;
@@ -781,6 +784,7 @@ static void init_vp_index(struct vmbus_channel *channel)
 			break;
 	}
 
+	cpus_read_unlock();
 	channel->target_cpu = target_cpu;
 
 	free_cpumask_var(available_mask);
Signature · 1 line
1.8.3.1
#1 lore raw

From: Saurabh Sengar <[email protected]> Sent: Wednesday, June 8, 2022 10:27 PM

Quoted 34 lines

Add cpus_read_lock to prevent CPUs from going offline between query and actual use of cpumask. cpumask_of_node is first queried, and based on it used later, in case any CPU goes offline between these two events, it can potentially cause an infinite loop of retries.

---
 drivers/hv/channel_mgmt.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/hv/channel_mgmt.c +4 -1
index 85a2142..6a88b7e 100644
@@ -749,6 +749,9 @@ static void init_vp_index(struct vmbus_channel *channel)
 		return;
 	}
 
+	/* No CPUs should come up or down during this. */
+	cpus_read_lock();
+
 	for (i = 1; i <= ncpu + 1; i++) {
 		while (true) {
 			numa_node = next_numa_node_id++;
@@ -781,6 +784,7 @@ static void init_vp_index(struct vmbus_channel *channel)
 			break;
 	}
 
+	cpus_read_unlock();
 	channel->target_cpu = target_cpu;
 
 	free_cpumask_var(available_mask);
--
Quoted 1 line

1.8.3.1

This patch was motivated because I suggested a potential issue here during a separate conversation with Saurabh, but it turns out I was wrong. :-(

init_vp_index() is only called from vmbus_process_offer(), and the cpus_read_lock() is already held when init_vp_index() is called. So the issue doesn't exist, and this patch isn't needed.

However, looking at vmbus_process_offer(), there appears to be a different problem in that cpus_read_unlock() is not called when taking the error return because the sub_channel_index is zero.

Michael

maintainer
#2 lore raw
Quoted 59 lines

-----Original Message-----
From: Michael Kelley (LINUX) <[email protected]>
Sent: Thursday, June 9, 2022 9:51 AM
To: Saurabh Sengar <[email protected]>; KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; Dexuan Cui <[email protected]>; [email protected]; linux- [email protected]; Saurabh Singh Sengar <[email protected]>
Subject: RE: [PATCH] Drivers: hv: vmbus: Add cpu read lock

From: Saurabh Sengar <[email protected]> Sent: Wednesday, June 8, 2022 10:27 PM

drivers/hv/channel_mgmt.c static void init_vp_index(struct vmbus_channel

Quoted 16 lines

Add cpus_read_lock to prevent CPUs from going offline between query and actual use of cpumask. cpumask_of_node is first queried, and based on it used later, in case any CPU goes offline between these two events, it can potentially cause an infinite loop of retries.

---
 drivers/hv/channel_mgmt.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/hv/channel_mgmt.c +0 -0
index 85a2142..6a88b7e 100644
@@ -749,6 +749,9 @@ static void init_vp_index(struct vmbus_channel

*channel)

drivers/hv/channel_mgmt.c static void init_vp_index(struct vmbus_channel

Quoted 10 lines
drivers/hv/channel_mgmt.c +3 -0
 		return;
 	}
 
+	/* No CPUs should come up or down during this. */
+	cpus_read_lock();
+
 	for (i = 1; i <= ncpu + 1; i++) {
 		while (true) {
 			numa_node = next_numa_node_id++;
@@ -781,6 +784,7 @@ static void init_vp_index(struct vmbus_channel

*channel)

Quoted 9 lines
drivers/hv/channel_mgmt.c +1 -1
 			break;
 	}
 
+	cpus_read_unlock();
 	channel->target_cpu = target_cpu;
 
 	free_cpumask_var(available_mask);
--

1.8.3.1

This patch was motivated because I suggested a potential issue here during a separate conversation with Saurabh, but it turns out I was wrong. :-(

init_vp_index() is only called from vmbus_process_offer(), and the cpus_read_lock() is already held when init_vp_index() is called. So the issue doesn't exist, and this patch isn't needed.

Quoted 6 lines

However, looking at vmbus_process_offer(), there appears to be a different problem in that cpus_read_unlock() is not called when taking the error return because the sub_channel_index is zero.

Michael

        } else {
                /*
                 * Check to see if this is a valid sub-channel.
                 */
                if (newchannel->offermsg.offer.sub_channel_index == 0) {
                        mutex_unlock(&vmbus_connection.channel_mutex);
                        /*
                         * Don't call free_channel(), because newchannel->kobj
                         * is not initialized yet.
                         */
                        kfree(newchannel);
                        WARN_ON_ONCE(1);
                        return;
                }

If this happens, it should be a host bug. Yes, I also think the cpus_read_unlock() is missing in this error path.

Thanks,
- Haiyang

#3 lore raw
Quoted 81 lines · Haiyang Zhang
Quoted 59 lines

-----Original Message-----
From: Michael Kelley (LINUX) <[email protected]>
Sent: Thursday, June 9, 2022 9:51 AM
To: Saurabh Sengar <[email protected]>; KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; Dexuan Cui <[email protected]>; [email protected]; linux- [email protected]; Saurabh Singh Sengar <[email protected]>
Subject: RE: [PATCH] Drivers: hv: vmbus: Add cpu read lock

From: Saurabh Sengar <[email protected]> Sent: Wednesday, June 8, 2022 10:27 PM

drivers/hv/channel_mgmt.c static void init_vp_index(struct vmbus_channel

Quoted 16 lines

Add cpus_read_lock to prevent CPUs from going offline between query and actual use of cpumask. cpumask_of_node is first queried, and based on it used later, in case any CPU goes offline between these two events, it can potentially cause an infinite loop of retries.

---
 drivers/hv/channel_mgmt.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/hv/channel_mgmt.c +0 -0
index 85a2142..6a88b7e 100644
@@ -749,6 +749,9 @@ static void init_vp_index(struct vmbus_channel

*channel)

drivers/hv/channel_mgmt.c static void init_vp_index(struct vmbus_channel

Quoted 10 lines
drivers/hv/channel_mgmt.c +3 -0
 		return;
 	}
 
+	/* No CPUs should come up or down during this. */
+	cpus_read_lock();
+
 	for (i = 1; i <= ncpu + 1; i++) {
 		while (true) {
 			numa_node = next_numa_node_id++;
@@ -781,6 +784,7 @@ static void init_vp_index(struct vmbus_channel

*channel)

Quoted 9 lines
drivers/hv/channel_mgmt.c +1 -1
 			break;
 	}
 
+	cpus_read_unlock();
 	channel->target_cpu = target_cpu;
 
 	free_cpumask_var(available_mask);
--

1.8.3.1

This patch was motivated because I suggested a potential issue here during a separate conversation with Saurabh, but it turns out I was wrong. :-(

init_vp_index() is only called from vmbus_process_offer(), and the cpus_read_lock() is already held when init_vp_index() is called. So the issue doesn't exist, and this patch isn't needed.

Quoted 6 lines

However, looking at vmbus_process_offer(), there appears to be a different problem in that cpus_read_unlock() is not called when taking the error return because the sub_channel_index is zero.

Michael

        } else {
                /*
                 * Check to see if this is a valid sub-channel.
                 */
                if (newchannel->offermsg.offer.sub_channel_index == 0) {
                        mutex_unlock(&vmbus_connection.channel_mutex);
                        /*
                         * Don't call free_channel(), because newchannel->kobj
                         * is not initialized yet.
                         */
                        kfree(newchannel);
                        WARN_ON_ONCE(1);
                        return;
                }
Quoted 5 lines

If this happens, it should be a host bug. Yes, I also think the cpus_read_unlock() is missing in this error path.

Thanks,
- Haiyang

I see, will send another patch to fix this.

Regards,
Saurabh