Skip to contentSkip to conversation

hv: retry infinitely on hypercall transient failures

PATCH In review Long Li 3 participants 5 messages Jan 5, 2017 – Jan 7, 2017 Hyper-V/Azure core and drivers

lore
Share & tools
lore

Long Li

author
#0 lore raw

PATCH v2 1/1 hv: retry infinitely on hypercall transient failures

From: Long Li <[email protected]>

Hyper-v host guarantees that a hypercall will finish in reasonable time. Retry infinitely on transient failures to avoid returning error to upper layer.

---
 drivers/hv/connection.c | 30 ++++++++++++++----------------
 1 file changed, 14 insertions(+), 16 deletions(-)
drivers/hv/connection.c +14 -16
index 6ce8b87..4b3cfde 100644
@@ -438,46 +438,44 @@ void vmbus_on_event(unsigned long data)
 int vmbus_post_msg(void *buffer, size_t buflen)
 {
 	union hv_connection_id conn_id;
-	int ret = 0;
-	int retries = 0;
+	int ret;
 	u32 usec = 1;
 
 	conn_id.asu32 = 0;
 	conn_id.u.id = VMBUS_MESSAGE_CONNECTION_ID;
 
 	/*
-	 * hv_post_message() can have transient failures because of
-	 * insufficient resources. Retry the operation a couple of
-	 * times before giving up.
+	 * hv_post_message() can have transient failures. We retry infinitely
+	 * on these failures because host guarantees hypercall will finish.
 	 */
-	while (retries < 20) {
+	while (1) {
 		ret = hv_post_message(conn_id, 1, buffer, buflen);
 
 		switch (ret) {
+		/*
+		 * Retry on transient failures:
+		 * 1. HV_STATUS_INVALID_CONNECTION_ID:
+		 *    We send messages too frequently.
+		 *
+		 * 2. HV_STATUS_INSUFFICIENT_MEMORY and
+		 *    HV_STATUS_INSUFFICIENT_BUFFERS:
+		 *    The host is temporariliy running out of resources.
+		 */
 		case HV_STATUS_INVALID_CONNECTION_ID:
-			/*
-			 * We could get this if we send messages too
-			 * frequently.
-			 */
-			ret = -EAGAIN;
-			break;
 		case HV_STATUS_INSUFFICIENT_MEMORY:
 		case HV_STATUS_INSUFFICIENT_BUFFERS:
-			ret = -ENOMEM;
 			break;
 		case HV_STATUS_SUCCESS:
-			return ret;
+			return 0;
 		default:
 			pr_err("hv_post_msg() failed; error code:%d\n", ret);
 			return -EINVAL;
 		}
 
-		retries++;
 		udelay(usec);
 		if (usec < 2048)
 			usec *= 2;
 	}
-	return ret;
 }
 
 /*
Signature · 1 line
2.7.4
#1 lore raw
Quoted 4 lines · Long Li

From: Long Li <[email protected]>

Hyper-v host guarantees that a hypercall will finish in reasonable time. Retry infinitely on transient failures to avoid returning error to upper layer.

Again, never retry "forever", always have a way out, otherwise you will crash.

And again, why are you making this change? What problem does it solve?

greg k-h

#2 lore raw
Quoted 10 lines

-----Original Message-----
From: Greg KH [mailto:[email protected]]
Sent: Wednesday, January 04, 2017 11:48 PM
To: Long Li <[email protected]>
Cc: KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]; linux- [email protected]
Subject: Re: [PATCH v2] hv: retry infinitely on hypercall transient failures

On Wed, Jan 04, 2017 at 06:12:20PM -0800, Long Li wrote:

Quoted 8 lines
Quoted 4 lines

From: Long Li <[email protected]>

Hyper-v host guarantees that a hypercall will finish in reasonable time. Retry infinitely on transient failures to avoid returning error to upper layer.

Again, never retry "forever", always have a way out, otherwise you will crash.

And again, why are you making this change? What problem does it solve?

The problem it tries to solve is that in this code we are returning error prematurely on transient failures. The hypercall is used mostly in channel establishment. If we return a transient failure, the VM may not boot or not useful after boot due to some devices missing.

Another approach is to increase the number of retries. But we don't know how many retries is safe, and Windows host side expects the guest retry infinitely and not return error on transient failures.

Quoted 2 lines

greg k-h

#3 lore raw
Quoted 19 lines · Long Li
Quoted 10 lines

-----Original Message-----
From: Greg KH [mailto:[email protected]]
Sent: Wednesday, January 04, 2017 11:48 PM
To: Long Li <[email protected]>
Cc: KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]; linux- [email protected]
Subject: Re: [PATCH v2] hv: retry infinitely on hypercall transient failures

On Wed, Jan 04, 2017 at 06:12:20PM -0800, Long Li wrote:

Quoted 8 lines
Quoted 4 lines

From: Long Li <[email protected]>

Hyper-v host guarantees that a hypercall will finish in reasonable time. Retry infinitely on transient failures to avoid returning error to upper layer.

Again, never retry "forever", always have a way out, otherwise you will crash.

And again, why are you making this change? What problem does it solve?

Quoted 8 lines

The problem it tries to solve is that in this code we are returning error prematurely on transient failures. The hypercall is used mostly in channel establishment. If we return a transient failure, the VM may not boot or not useful after boot due to some devices missing.

Another approach is to increase the number of retries. But we don't know how many retries is safe, and Windows host side expects the guest retry infinitely and not return error on transient failures.

That implies a lot of trust in the host side, don't you think?

Worse case, make the delay a minute or so, but give the system a way out incase there's a bug in the host. As there will be bugs in the host, just like there are bugs in the client :)

thanks,

greg k-h

#4 lore raw
Quoted 43 lines

-----Original Message-----
From: Greg KH [mailto:[email protected]]
Sent: Friday, January 06, 2017 11:43 PM
To: Long Li <[email protected]>
Cc: KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]; linux- [email protected]
Subject: Re: [PATCH v2] hv: retry infinitely on hypercall transient failures

Quoted 10 lines · Long Li
Quoted 10 lines

-----Original Message-----
From: Greg KH [mailto:[email protected]]
Sent: Wednesday, January 04, 2017 11:48 PM
To: Long Li <[email protected]>
Cc: KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]; linux- [email protected]
Subject: Re: [PATCH v2] hv: retry infinitely on hypercall transient failures

Quoted 5 lines
Quoted 5 lines · Long Li
Quoted 4 lines

From: Long Li <[email protected]>

Hyper-v host guarantees that a hypercall will finish in reasonable time. Retry infinitely on transient failures to avoid returning error to upper

layer.

Quoted 2 lines
Quoted 2 lines

Again, never retry "forever", always have a way out, otherwise you will

crash.

Quoted 11 lines
Quoted 2 lines

And again, why are you making this change? What problem does it solve?

The problem it tries to solve is that in this code we are returning error prematurely on transient failures. The hypercall is used mostly in channel establishment. If we return a transient failure, the VM may not boot or not useful after boot due to some devices missing.

Another approach is to increase the number of retries. But we don't know how many retries is safe, and Windows host side expects the guest retry infinitely and not return error on transient failures.

Quoted 5 lines

That implies a lot of trust in the host side, don't you think?

Worse case, make the delay a minute or so, but give the system a way out incase there's a bug in the host. As there will be bugs in the host, just like there are bugs in the client :)

This makes sense. 1 minute is a long time for a hypercall. I will send V3.

Quoted 4 lines

thanks,

greg k-h