Skip to contentSkip to conversation

Drivers: hv: vmbus: protect Offer/Rescind offer processing

PATCH Superseded Vitaly Kuznetsov 2 participants 8 messages Jan 19, 2015 – Jan 20, 2015 Hyper-V/Azure core and drivers

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover Drivers: hv: vmbus: protect Offer/Rescind offer processing

This patch series is a successor of "[PATCH] Drivers: hv: vmbus: serialize Offer and Rescind offer processing" and thereby have 'v2' in its name.

Changes from v1:
- Separate vmbus_device_create() return value check [K. Y. Srinivasan]
- Do not lose a rescind offer received during offer processing. Use renamed (in [PATCH v2 2/3]) spinlock to protect simulteneous test-and-set workflow for rescind and work fields. [K. Y. Srinivasan]

Vitaly Kuznetsov (3):
Drivers: hv: check vmbus_device_create() return value in
vmbus_process_offer()
Drivers: hv: rename sc_lock to the more generic lock
Drivers: hv: vmbus: serialize Offer and Rescind offer

 drivers/hv/channel.c      |  6 +++---
 drivers/hv/channel_mgmt.c | 45 ++++++++++++++++++++++++++++++++-------------
 include/linux/hyperv.h    |  7 ++++++-
 3 files changed, 41 insertions(+), 17 deletions(-)
Signature · 1 line
1.9.3
#1 lore raw

PATCH v2 1/3 Drivers: hv: check vmbus_device_create() return value in vmbus_process_offer()

vmbus_device_create() result is not being checked in vmbus_process_offer() and it can fail if kzalloc() fails. Add the check and do minor cleanup to avoid additional duplication of "free_channel(); return;" block.

---
 drivers/hv/channel_mgmt.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
drivers/hv/channel_mgmt.c +9 -5
index 2c59f03..80590c6 100644
@@ -341,11 +341,10 @@ static void vmbus_process_offer(struct work_struct *work)
 			if (channel->sc_creation_callback != NULL)
 				channel->sc_creation_callback(newchannel);
 
-			return;
+			goto out;
 		}
 
-		free_channel(newchannel);
-		return;
+		goto error;
 	}
 
 	/*
@@ -364,6 +363,8 @@ static void vmbus_process_offer(struct work_struct *work)
 		&newchannel->offermsg.offer.if_type,
 		&newchannel->offermsg.offer.if_instance,
 		newchannel);
+	if (!newchannel->device_obj)
+		goto error;
 
 	/*
 	 * Add the new device to the bus. This will kick off device-driver
@@ -379,9 +380,12 @@ static void vmbus_process_offer(struct work_struct *work)
 		list_del(&newchannel->listentry);
 		spin_unlock_irqrestore(&vmbus_connection.channel_lock, flags);
 		kfree(newchannel->device_obj);
-
-		free_channel(newchannel);
+		goto error;
 	}
+out:
+	return;
+error:
+	free_channel(newchannel);
 }
 
 enum {
Signature · 1 line
1.9.3
#2 lore raw

Re: [PATCH v2 1/3] Drivers: hv: check vmbus_device_create() return value in vmbus_process_offer()

Quoted 6 lines · Vitaly Kuznetsov

vmbus_device_create() result is not being checked in vmbus_process_offer() and it can fail if kzalloc() fails. Add the check and do minor cleanup to avoid additional duplication of "free_channel(); return;" block.

"out" is always a bad name for a label. It's too vague. It implies that the code uses "One Err" style error handling which is bug prone and I've ranted about that in the past so I won't here. This kind of coding is buggier than direct returns. But recently I've been looking at bugs where we return zero where the code should return a negative error code and, wow, do I hate "out" labels!

	if (function_whatever(xxx))
		goto out;

[ thousands of lines removed. ]

out:

	return ret;

Oh crap... Did the coder mean to return success or not???

If you use a direct return then the code looks like:

	if (function_whatever(xxx))
		return 0;

In that case, you can immediately see that the coder typed "0" deliberately. Direct returns are best. I guess that's not directly related to this code. But I didn't know that until I read to the bottom of the patch and I already had this rant prepared in my head ready to go...

"error" is a crap label name because it doesn't tell you what the code does. A better name is "err_free_chan" or something which talks about freeing the channel.

regards,
dan carpenter

#3 lore raw

Re: [PATCH v2 1/3] Drivers: hv: check vmbus_device_create() return value in vmbus_process_offer()

Quoted 3 lines · Dan Carpenter

"error" is a crap label name because it doesn't tell you what the code does. A better name is "err_free_chan" or something which talks about freeing the channel.

If you choose your label names correctly, then most of the time you can just read the code from top to bottom and understand it without skipping back and forth.

regards,
dan carpenter

#4 lore raw

Re: [PATCH v2 1/3] Drivers: hv: check vmbus_device_create() return value in vmbus_process_offer()

In reply to Dan Carpenter
Quoted 27 lines · Dan Carpenter
Quoted 6 lines · Vitaly Kuznetsov

vmbus_device_create() result is not being checked in vmbus_process_offer() and it can fail if kzalloc() fails. Add the check and do minor cleanup to avoid additional duplication of "free_channel(); return;" block.

"out" is always a bad name for a label. It's too vague. It implies that the code uses "One Err" style error handling which is bug prone and I've ranted about that in the past so I won't here. This kind of coding is buggier than direct returns. But recently I've been looking at bugs where we return zero where the code should return a negative error code and, wow, do I hate "out" labels!

	if (function_whatever(xxx))
		goto out;

[ thousands of lines removed. ]

out:

	return ret;

Oh crap... Did the coder mean to return success or not???

If you use a direct return then the code looks like:

Quoted 8 lines
	if (function_whatever(xxx))
		return 0;

In that case, you can immediately see that the coder typed "0" deliberately. Direct returns are best. I guess that's not directly related to this code. But I didn't know that until I read to the bottom of the patch and I already had this rant prepared in my head ready to go...

Thank you for your rant, Dan! It contains an explanation _why_ and so is useful. However ... :-)

1) vmbus_process_offer() returns void so we won't forget to set proper return code.
2) this patch is a preparation for the PATCH 3/3 where the label is being used to do some useful (non-trivial) work. "Direct returns" approach would require us to duplicate the code or move it to a function and call it from all return places. I consider adding "out" label being less evil.

Anyway, I can rename it to something less provocative in PATCH 3/3, e.g. init_rescind.

Quoted 4 lines

"error" is a crap label name because it doesn't tell you what the code does. A better name is "err_free_chan" or something which talks about freeing the channel.

And here I have to completely agree with you, I'll rename it in v3.

Quoted 3 lines

regards,
dan carpenter

Signature · 1 line
  Vitaly
#5 lore raw

Re: [PATCH v2 1/3] Drivers: hv: check vmbus_device_create() return value in vmbus_process_offer()

Fine. Fine. I saw the 3/3 patch after I wrote the email. Anyway I hope that my ranting has planted a seed of doubt and from now you will begin to view "out" labels with suspicion which will eventually flourish into the flower of hatred.

regards,
dan carpenter

#6 lore raw

PATCH v2 2/3 Drivers: hv: rename sc_lock to the more generic lock

In reply to Vitaly Kuznetsov

sc_lock spinlock in struct vmbus_channel is being used to not only protect the sc_list field, e.g. vmbus_open() function uses it to implement test-and-set access to the state field. Rename it to the more generic 'lock' and add the description.

---
 drivers/hv/channel.c      |  6 +++---
 drivers/hv/channel_mgmt.c | 10 +++++-----
 include/linux/hyperv.h    |  7 ++++++-
 3 files changed, 14 insertions(+), 9 deletions(-)
drivers/hv/channel.c +3 -3
index 433f72a..8608ed1 100644
@@ -73,14 +73,14 @@ int vmbus_open(struct vmbus_channel *newchannel, u32 send_ringbuffer_size,
 	unsigned long flags;
 	int ret, t, err = 0;
 
-	spin_lock_irqsave(&newchannel->sc_lock, flags);
+	spin_lock_irqsave(&newchannel->lock, flags);
 	if (newchannel->state == CHANNEL_OPEN_STATE) {
 		newchannel->state = CHANNEL_OPENING_STATE;
 	} else {
-		spin_unlock_irqrestore(&newchannel->sc_lock, flags);
+		spin_unlock_irqrestore(&newchannel->lock, flags);
 		return -EINVAL;
 	}
-	spin_unlock_irqrestore(&newchannel->sc_lock, flags);
+	spin_unlock_irqrestore(&newchannel->lock, flags);
 
 	newchannel->onchannel_callback = onchannelcallback;
 	newchannel->channel_callback_context = context;
drivers/hv/channel_mgmt.c +5 -5
index 80590c6..c526ed2 100644
@@ -146,7 +146,7 @@ static struct vmbus_channel *alloc_channel(void)
 		return NULL;
 
 	spin_lock_init(&channel->inbound_lock);
-	spin_lock_init(&channel->sc_lock);
+	spin_lock_init(&channel->lock);
 
 	INIT_LIST_HEAD(&channel->sc_list);
 	INIT_LIST_HEAD(&channel->percpu_list);
@@ -246,9 +246,9 @@ static void vmbus_process_rescind_offer(struct work_struct *work)
 		spin_unlock_irqrestore(&vmbus_connection.channel_lock, flags);
 	} else {
 		primary_channel = channel->primary_channel;
-		spin_lock_irqsave(&primary_channel->sc_lock, flags);
+		spin_lock_irqsave(&primary_channel->lock, flags);
 		list_del(&channel->sc_list);
-		spin_unlock_irqrestore(&primary_channel->sc_lock, flags);
+		spin_unlock_irqrestore(&primary_channel->lock, flags);
 	}
 	free_channel(channel);
 }
@@ -323,9 +323,9 @@ static void vmbus_process_offer(struct work_struct *work)
 			 * Process the sub-channel.
 			 */
 			newchannel->primary_channel = channel;
-			spin_lock_irqsave(&channel->sc_lock, flags);
+			spin_lock_irqsave(&channel->lock, flags);
 			list_add_tail(&newchannel->sc_list, &channel->sc_list);
-			spin_unlock_irqrestore(&channel->sc_lock, flags);
+			spin_unlock_irqrestore(&channel->lock, flags);
 
 			if (newchannel->target_cpu != get_cpu()) {
 				put_cpu();
include/linux/hyperv.h +6 -1
index 476c685..02dd978 100644
@@ -722,7 +722,12 @@ struct vmbus_channel {
 	 */
 	void (*sc_creation_callback)(struct vmbus_channel *new_sc);
 
-	spinlock_t sc_lock;
+	/*
+	 * The spinlock to protect the structure. It is being used to protect
+	 * test-and-set access to various attributes of the structure as well
+	 * as all sc_list operations.
+	 */
+	spinlock_t lock;
 	/*
 	 * All Sub-channels of a primary channel are linked here.
 	 */
Signature · 1 line
1.9.3
#7 lore raw

PATCH v2 3/3 Drivers: hv: vmbus: serialize Offer and Rescind offer

In reply to Vitaly Kuznetsov

Commit 4b2f9abea52a ("staging: hv: convert channel_mgmt.c to not call osd_schedule_callback")' was written under an assumption that we never receive Rescind offer while we're still processing the initial Offer request. However, the issue we fixed in 04a258c162a8 could be caused by this assumption not always being true.

In particular, we need to protect against the following:
1) Receiving a Rescind offer after we do queue_work() for processing an Offer request and before we actually enter vmbus_process_offer(). work.func points to vmbus_process_offer() at this moment and in vmbus_onoffer_rescind() we do another queue_work() without a check so we'll enter vmbus_process_offer() twice.
2) Receiving a Rescind offer after we enter vmbus_process_offer() and especially after we set >state = CHANNEL_OPEN_STATE. Many things can go wrong in that case, e.g. we can call free_channel() while we're still using it.

Implement the required protection by changing work->func at the very end of vmbus_process_offer() and checking work->func in vmbus_onoffer_rescind(). In case we receive rescind offer during or before vmbus_process_offer() is done we set rescind flag to true and we check it at the end of vmbus_process_offer() so such offer will not get lost.

---
 drivers/hv/channel_mgmt.c | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)
drivers/hv/channel_mgmt.c +18 -3
index c526ed2..512b74a 100644
@@ -279,9 +279,6 @@ static void vmbus_process_offer(struct work_struct *work)
 	int ret;
 	unsigned long flags;
 
-	/* The next possible work is rescind handling */
-	INIT_WORK(&newchannel->work, vmbus_process_rescind_offer);
-
 	/* Make sure this is a new offer */
 	spin_lock_irqsave(&vmbus_connection.channel_lock, flags);
 
@@ -383,6 +380,13 @@ static void vmbus_process_offer(struct work_struct *work)
 		goto error;
 	}
 out:
+	spin_lock_irqsave(&newchannel->lock, flags);
+	/* The next possible work is rescind handling */
+	INIT_WORK(&newchannel->work, vmbus_process_rescind_offer);
+	/* Check if rescind offer was already received */
+	if (newchannel->rescind)
+		queue_work(newchannel->controlwq, &newchannel->work);
+	spin_unlock_irqrestore(&newchannel->lock, flags);
 	return;
 error:
 	free_channel(newchannel);
@@ -520,6 +524,7 @@ static void vmbus_onoffer_rescind(struct vmbus_channel_message_header *hdr)
 {
 	struct vmbus_channel_rescind_offer *rescind;
 	struct vmbus_channel *channel;
+	unsigned long flags;
 
 	rescind = (struct vmbus_channel_rescind_offer *)hdr;
 	channel = relid2channel(rescind->child_relid);
@@ -528,11 +533,21 @@ static void vmbus_onoffer_rescind(struct vmbus_channel_message_header *hdr)
 		/* Just return here, no channel found */
 		return;
 
+	spin_lock_irqsave(&channel->lock, flags);
 	channel->rescind = true;
+	if (channel->work.func != vmbus_process_rescind_offer)
+		/*
+		 * We're still processing offer request, postponing rescind
+		 * offer processing. It will be done when vmbus_process_offer()
+		 * is done as it checks rescind flag.
+		 */
+		goto out;
 
 	/* work is initialized for vmbus_process_rescind_offer() from
 	 * vmbus_process_offer() where the channel got created */
 	queue_work(channel->controlwq, &channel->work);
+out:
+	spin_unlock_irqrestore(&channel->lock, flags);
 }
 
 /*
Signature · 1 line
1.9.3