Skip to contentSkip to conversation

net/virtio: wait device ready in device reset

PATCH In review Xueming Li 4 participants 13 messages Aug 23, 2021 – Sep 28, 2021 Network drivers

lore
Share & tools
lore

Xueming Li

author
#0 lore raw

PATCH 1/1 net/virtio: wait device ready in device reset

According to virtio spec, the device MUST reset when 0 is written to device_status, and present a 0 in device_status once that is done.

This patch adds the missing part of waiting status 0 in reset function.

---
 drivers/net/virtio/virtio.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
drivers/net/virtio/virtio.c +5 -2
index 7e1e77797f..f003f612d6 100644
@@ -3,6 +3,8 @@
  * Copyright(c) 2020 Red Hat, Inc.
  */
 
+#include <unistd.h>
+
 #include "virtio.h"
 
 uint64_t
@@ -39,8 +41,9 @@ void
 virtio_reset(struct virtio_hw *hw)
 {
 	VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
-	/* flush status write */
-	VIRTIO_OPS(hw)->get_status(hw);
+	/* Flush status write and wait device ready. */
+	while (VIRTIO_OPS(hw)->get_status(hw) != VIRTIO_CONFIG_STATUS_RESET)
+		usleep(1000L);
 }
 
 void
Signature · 1 line
2.25.1
#1 lore raw
Quoted 24 lines · Xueming Li

According to virtio spec, the device MUST reset when 0 is written to device_status, and present a 0 in device_status once that is done.

This patch adds the missing part of waiting status 0 in reset function.

---
 drivers/net/virtio/virtio.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
drivers/net/virtio/virtio.c +2 -0
index 7e1e77797f..f003f612d6 100644
@@ -3,6 +3,8 @@
  * Copyright(c) 2020 Red Hat, Inc.
  */
 
+#include <unistd.h>
+
 #include "virtio.h"
 
 uint64_t

drivers/net/virtio/virtio.c void · line 46

Quoted 8 lines
drivers/net/virtio/virtio.c +3 -2
@@ -39,8 +41,9 @@ void
 virtio_reset(struct virtio_hw *hw)
 {
 	VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
-	/* flush status write */
-	VIRTIO_OPS(hw)->get_status(hw);
+	/* Flush status write and wait device ready. */
+	while (VIRTIO_OPS(hw)->get_status(hw) != VIRTIO_CONFIG_STATUS_RESET)
+		usleep(1000L);

Don't we need a protection against forever loop here?

Quoted 4 lines
drivers/net/virtio/virtio.c +0 -0
 }
 
 void
#2 lore raw
Quoted 32 lines

-----Original Message-----
From: Andrew Rybchenko <[email protected]>
Sent: Monday, August 23, 2021 5:57 PM
To: Xueming(Steven) Li <[email protected]>
Cc: [email protected]; Maxime Coquelin <[email protected]>; Chenbo Xia <[email protected]>
Subject: Re: [dpdk-dev] [PATCH] net/virtio: wait device ready in device reset

Quoted 22 lines · Xueming Li

According to virtio spec, the device MUST reset when 0 is written to device_status, and present a 0 in device_status once that is done.

This patch adds the missing part of waiting status 0 in reset function.

---
 drivers/net/virtio/virtio.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
drivers/net/virtio/virtio.c +2 -0
index 7e1e77797f..f003f612d6 100644
@@ -3,6 +3,8 @@
  * Copyright(c) 2020 Red Hat, Inc.
  */
 
+#include <unistd.h>
+
 #include "virtio.h"
 
Quoted 12 lines

drivers/net/virtio/virtio.c void · line 46

Quoted 10 lines
drivers/net/virtio/virtio.c +3 -2
 uint64_t
@@ -39,8 +41,9 @@ void
 virtio_reset(struct virtio_hw *hw)
 {
 	VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
-	/* flush status write */
-	VIRTIO_OPS(hw)->get_status(hw);
+	/* Flush status write and wait device ready. */
+	while (VIRTIO_OPS(hw)->get_status(hw) != VIRTIO_CONFIG_STATUS_RESET)
+		usleep(1000L);

Don't we need a protection against forever loop here?

Good question, ideally we need, kernel driver function vp_reset() seems to have same issue. How about leaving an error message before return?

Quoted 5 lines
Quoted 5 lines
drivers/net/virtio/virtio.c +0 -0
 }
 
 void
#3 lore raw
Quoted 45 lines · Xueming(Steven) Li
Quoted 32 lines

-----Original Message-----
From: Andrew Rybchenko <[email protected]>
Sent: Monday, August 23, 2021 5:57 PM
To: Xueming(Steven) Li <[email protected]>
Cc: [email protected]; Maxime Coquelin <[email protected]>; Chenbo Xia <[email protected]>
Subject: Re: [dpdk-dev] [PATCH] net/virtio: wait device ready in device reset

Quoted 22 lines · Xueming Li

According to virtio spec, the device MUST reset when 0 is written to device_status, and present a 0 in device_status once that is done.

This patch adds the missing part of waiting status 0 in reset function.

---
 drivers/net/virtio/virtio.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
drivers/net/virtio/virtio.c +2 -0
index 7e1e77797f..f003f612d6 100644
@@ -3,6 +3,8 @@
  * Copyright(c) 2020 Red Hat, Inc.
  */
 
+#include <unistd.h>
+
 #include "virtio.h"
 
Quoted 12 lines

drivers/net/virtio/virtio.c void · line 46

Quoted 10 lines
drivers/net/virtio/virtio.c +3 -2
 uint64_t
@@ -39,8 +41,9 @@ void
 virtio_reset(struct virtio_hw *hw)
 {
 	VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
-	/* flush status write */
-	VIRTIO_OPS(hw)->get_status(hw);
+	/* Flush status write and wait device ready. */
+	while (VIRTIO_OPS(hw)->get_status(hw) != VIRTIO_CONFIG_STATUS_RESET)
+		usleep(1000L);

Don't we need a protection against forever loop here?

Quoted 1 line

Good question, ideally we need, kernel driver function vp_reset() seems to have same issue.

Yes, I've seen it.

Quoted 1 line

How about leaving an error message before return?

@Maxime, @Chenbo, what do you think?

#4 lore raw

Hi Adrew & Xueming,

Quoted 56 lines

-----Original Message-----
From: Andrew Rybchenko <[email protected]>
Sent: Tuesday, August 24, 2021 11:41 PM
To: Xueming(Steven) Li <[email protected]>
Cc: [email protected]; Maxime Coquelin <[email protected]>; Xia, Chenbo <[email protected]>
Subject: Re: [dpdk-dev] [PATCH] net/virtio: wait device ready in device reset

Quoted 7 lines · Xueming(Steven) Li
Quoted 7 lines

-----Original Message-----
From: Andrew Rybchenko <[email protected]>
Sent: Monday, August 23, 2021 5:57 PM
To: Xueming(Steven) Li <[email protected]>
Cc: [email protected]; Maxime Coquelin <[email protected]>; Chenbo Xia

<[email protected]>

Quoted 1 line
Quoted 1 line

Subject: Re: [dpdk-dev] [PATCH] net/virtio: wait device ready in device

reset

Quoted 37 lines
Quoted 36 lines · Xueming Li
Quoted 24 lines

According to virtio spec, the device MUST reset when 0 is written to device_status, and present a 0 in device_status once that is done.

This patch adds the missing part of waiting status 0 in reset function.

---
 drivers/net/virtio/virtio.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
drivers/net/virtio/virtio.c +2 -0
index 7e1e77797f..f003f612d6 100644
@@ -3,6 +3,8 @@
  * Copyright(c) 2020 Red Hat, Inc.
  */
 
+#include <unistd.h>
+
 #include "virtio.h"
 
 uint64_t

drivers/net/virtio/virtio.c void · line 46

Quoted 8 lines
drivers/net/virtio/virtio.c +3 -2
@@ -39,8 +41,9 @@ void
 virtio_reset(struct virtio_hw *hw)
 {
 	VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
-	/* flush status write */
-	VIRTIO_OPS(hw)->get_status(hw);
+	/* Flush status write and wait device ready. */
+	while (VIRTIO_OPS(hw)->get_status(hw) != VIRTIO_CONFIG_STATUS_RESET)
+		usleep(1000L);

Don't we need a protection against forever loop here?

Quoted 8 lines
Quoted 1 line

Good question, ideally we need, kernel driver function vp_reset() seems to

have same issue.

Yes, I've seen it.

Quoted 1 line

How about leaving an error message before return?

@Maxime, @Chenbo, what do you think?

I would vote for waiting for some time before return rather than forever loop and error message is needed.

My understanding is for kernel, it's fine to sleep forever as kernel could schedule it but for DPDK, it will lead to main lcore unable to do other things but sleep forever. Meanwhile, users will see the app stuck but don't know what's wrong here.

Thanks,
Chenbo

#5 lore raw
Quoted 70 lines · Chenbo

Hi Adrew & Xueming,

Quoted 56 lines

-----Original Message-----
From: Andrew Rybchenko <[email protected]>
Sent: Tuesday, August 24, 2021 11:41 PM
To: Xueming(Steven) Li <[email protected]>
Cc: [email protected]; Maxime Coquelin <[email protected]>; Xia, Chenbo <[email protected]>
Subject: Re: [dpdk-dev] [PATCH] net/virtio: wait device ready in device reset

Quoted 7 lines · Xueming(Steven) Li
Quoted 7 lines

-----Original Message-----
From: Andrew Rybchenko <[email protected]>
Sent: Monday, August 23, 2021 5:57 PM
To: Xueming(Steven) Li <[email protected]>
Cc: [email protected]; Maxime Coquelin <[email protected]>; Chenbo Xia

<[email protected]>

Quoted 1 line
Quoted 1 line

Subject: Re: [dpdk-dev] [PATCH] net/virtio: wait device ready in device

reset

Quoted 37 lines
Quoted 36 lines · Xueming Li
Quoted 24 lines

According to virtio spec, the device MUST reset when 0 is written to device_status, and present a 0 in device_status once that is done.

This patch adds the missing part of waiting status 0 in reset function.

---
 drivers/net/virtio/virtio.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
drivers/net/virtio/virtio.c +2 -0
index 7e1e77797f..f003f612d6 100644
@@ -3,6 +3,8 @@
  * Copyright(c) 2020 Red Hat, Inc.
  */
 
+#include <unistd.h>
+
 #include "virtio.h"
 
 uint64_t

drivers/net/virtio/virtio.c void · line 46

Quoted 8 lines
drivers/net/virtio/virtio.c +3 -2
@@ -39,8 +41,9 @@ void
 virtio_reset(struct virtio_hw *hw)
 {
 	VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
-	/* flush status write */
-	VIRTIO_OPS(hw)->get_status(hw);
+	/* Flush status write and wait device ready. */
+	while (VIRTIO_OPS(hw)->get_status(hw) != VIRTIO_CONFIG_STATUS_RESET)
+		usleep(1000L);

Don't we need a protection against forever loop here?

Quoted 8 lines
Quoted 1 line

Good question, ideally we need, kernel driver function vp_reset() seems to

have same issue.

Yes, I've seen it.

Quoted 1 line

How about leaving an error message before return?

@Maxime, @Chenbo, what do you think?

I would vote for waiting for some time before return rather than forever loop and error message is needed.

Quoted 7 lines

My understanding is for kernel, it's fine to sleep forever as kernel could schedule it but for DPDK, it will lead to main lcore unable to do other things but sleep forever. Meanwhile, users will see the app stuck but don't know what's wrong here.

Thanks,
Chenbo

Hi all, thanks for you sugestion, new version posted:
https://mails.dpdk.org/archives/dev/2021-September/219866.html

Quoted 1 line
author
#6 lore raw

PATCH 1/1 net/virtio: wait device ready during reset

In reply to Xueming Li

According to virtio spec, the device MUST reset when 0 is written to device_status, and present 0 in device_status once reset is done.

This patch waits status value to be 0 during reset operation, if timeout in 3 seconds, log and continue.

Cc Andrew Rybchenko <[email protected]>
---
 drivers/net/virtio/virtio.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
drivers/net/virtio/virtio.c +13 -2
index 7e1e77797f..f865b27b65 100644
@@ -3,7 +3,10 @@
  * Copyright(c) 2020 Red Hat, Inc.
  */
 
+#include <unistd.h>
+
 #include "virtio.h"
+#include "virtio_logs.h"
 
 uint64_t
 virtio_negotiate_features(struct virtio_hw *hw, uint64_t host_features)
@@ -38,9 +41,17 @@ virtio_write_dev_config(struct virtio_hw *hw, size_t offset,
 void
 virtio_reset(struct virtio_hw *hw)
 {
+	uint32_t retry = 0;
+
 	VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
-	/* flush status write */
-	VIRTIO_OPS(hw)->get_status(hw);
+	/* Flush status write and wait device ready max 3 seconds. */
+	while (VIRTIO_OPS(hw)->get_status(hw) != VIRTIO_CONFIG_STATUS_RESET) {
+		if (retry++ > 3000) {
+			PMD_INIT_LOG(WARNING, "device reset timeout");
+			break;
+		}
+		usleep(1000L);
+	}
 }
 
 void
Signature · 1 line
2.33.0
#7 lore raw

Re: [dpdk-dev] [PATCH v1] net/virtio: wait device ready during reset

Quoted 5 lines · Xueming Li

According to virtio spec, the device MUST reset when 0 is written to device_status, and present 0 in device_status once reset is done.

This patch waits status value to be 0 during reset operation, if timeout in 3 seconds, log and continue.

I have no strong opinion on timeout.

Quoted 25 lines
Cc Andrew Rybchenko <[email protected]>
---
 drivers/net/virtio/virtio.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
drivers/net/virtio/virtio.c +3 -0
index 7e1e77797f..f865b27b65 100644
@@ -3,7 +3,10 @@
  * Copyright(c) 2020 Red Hat, Inc.
  */
 
+#include <unistd.h>
+
 #include "virtio.h"
+#include "virtio_logs.h"
 
 uint64_t
 virtio_negotiate_features(struct virtio_hw *hw, uint64_t host_features)
@@ -38,9 +41,17 @@ virtio_write_dev_config(struct virtio_hw *hw, size_t offset,
 void
 virtio_reset(struct virtio_hw *hw)

drivers/net/virtio/virtio.c virtio_write_dev_config(struct virtio_hw *hw, size_t offset, · line 50

Quoted 10 lines
drivers/net/virtio/virtio.c +6 -2
 {
+	uint32_t retry = 0;
+
 	VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
-	/* flush status write */
-	VIRTIO_OPS(hw)->get_status(hw);
+	/* Flush status write and wait device ready max 3 seconds. */
+	while (VIRTIO_OPS(hw)->get_status(hw) != VIRTIO_CONFIG_STATUS_RESET) {
+		if (retry++ > 3000) {
+			PMD_INIT_LOG(WARNING, "device reset timeout");

I think it would be very useful to log ethdev port ID here.

Quoted 8 lines
drivers/net/virtio/virtio.c +4 -0
+			break;
+		}
+		usleep(1000L);
+	}
 }
 
 void
#8 lore raw

Re: [dpdk-dev] [PATCH v1] net/virtio: wait device ready during reset

Quoted 38 lines · Andrew Rybchenko
Quoted 5 lines · Xueming Li

According to virtio spec, the device MUST reset when 0 is written to device_status, and present 0 in device_status once reset is done.

This patch waits status value to be 0 during reset operation, if timeout in 3 seconds, log and continue.

I have no strong opinion on timeout.

Quoted 29 lines
Cc Andrew Rybchenko <[email protected]>
---
 drivers/net/virtio/virtio.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
drivers/net/virtio/virtio.c +5 -0
index 7e1e77797f..f865b27b65 100644
@@ -3,7 +3,10 @@
  * Copyright(c) 2020 Red Hat, Inc.
  */
 
+#include <unistd.h>
+
 #include "virtio.h"
+#include "virtio_logs.h"
 
 uint64_t
 virtio_negotiate_features(struct virtio_hw *hw, uint64_t host_features)
@@ -38,9 +41,17 @@ virtio_write_dev_config(struct virtio_hw *hw, size_t offset,
 void
 virtio_reset(struct virtio_hw *hw)
 {
+	uint32_t retry = 0;
+
 	VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
Quoted 8 lines

drivers/net/virtio/virtio.c virtio_write_dev_config(struct virtio_hw *hw, size_t offset, · line 50

Quoted 6 lines
drivers/net/virtio/virtio.c +4 -2
-	/* flush status write */
-	VIRTIO_OPS(hw)->get_status(hw);
+	/* Flush status write and wait device ready max 3 seconds. */
+	while (VIRTIO_OPS(hw)->get_status(hw) != VIRTIO_CONFIG_STATUS_RESET) {
+		if (retry++ > 3000) {
+			PMD_INIT_LOG(WARNING, "device reset timeout");

I think it would be very useful to log ethdev port ID here.

Good suggesiton, v2 sent.

Quoted 10 lines
Quoted 9 lines
drivers/net/virtio/virtio.c +4 -0
+			break;
+		}
+		usleep(1000L);
+	}
 }
 
 void
author
#9 lore raw

PATCH v2 1/1 net/virtio: wait device ready during reset

In reply to Xueming Li

According to virtio spec, the device MUST reset when 0 is written to device_status, and present 0 in device_status once reset is done.

This patch waits status value to be 0 during reset operation, if timeout in 3 seconds, log and continue.

Cc Andrew Rybchenko <[email protected]>
---
 drivers/net/virtio/virtio.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
drivers/net/virtio/virtio.c +13 -2
index 7e1e77797f..d9e642f412 100644
@@ -3,7 +3,10 @@
  * Copyright(c) 2020 Red Hat, Inc.
  */
 
+#include <unistd.h>
+
 #include "virtio.h"
+#include "virtio_logs.h"
 
 uint64_t
 virtio_negotiate_features(struct virtio_hw *hw, uint64_t host_features)
@@ -38,9 +41,17 @@ virtio_write_dev_config(struct virtio_hw *hw, size_t offset,
 void
 virtio_reset(struct virtio_hw *hw)
 {
+	uint32_t retry = 0;
+
 	VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
-	/* flush status write */
-	VIRTIO_OPS(hw)->get_status(hw);
+	/* Flush status write and wait device ready max 3 seconds. */
+	while (VIRTIO_OPS(hw)->get_status(hw) != VIRTIO_CONFIG_STATUS_RESET) {
+		if (retry++ > 3000) {
+			PMD_INIT_LOG(WARNING, "port %u device reset timeout", hw->port_id);
+			break;
+		}
+		usleep(1000L);
+	}
 }
 
 void
Signature · 1 line
2.33.0
Acked-by on v2 1/1
#10 lore raw

Re: [dpdk-dev] [PATCH v2] net/virtio: wait device ready during reset

Quoted 8 lines · Xueming Li

According to virtio spec, the device MUST reset when 0 is written to device_status, and present 0 in device_status once reset is done.

This patch waits status value to be 0 during reset operation, if timeout in 3 seconds, log and continue.

Cc Andrew Rybchenko <[email protected]>
Reviewed-by on v2 1/1
#11 lore raw

Re: [dpdk-dev] [PATCH v2] net/virtio: wait device ready during reset

In reply to Xueming Li
Quoted 12 lines

-----Original Message-----
From: Xueming Li <[email protected]>
Sent: Wednesday, September 15, 2021 6:12 PM
To: [email protected]
Cc: [email protected]; Andrew Rybchenko <[email protected]>; Maxime Coquelin <[email protected]>; Xia, Chenbo <[email protected]>
Subject: [PATCH v2] net/virtio: wait device ready during reset

According to virtio spec, the device MUST reset when 0 is written to device_status, and present 0 in device_status once reset is done.

Quoted 7 lines

This patch waits status value to be 0 during reset operation, if timeout in 3 seconds, log and continue.

Cc Andrew Rybchenko <[email protected]>

--
2.33.0

#12 lore raw

Re: [dpdk-dev] [PATCH v2] net/virtio: wait device ready during reset

In reply to Xueming Li
Quoted 12 lines · Xueming Li

According to virtio spec, the device MUST reset when 0 is written to device_status, and present 0 in device_status once reset is done.

This patch waits status value to be 0 during reset operation, if timeout in 3 seconds, log and continue.

Cc Andrew Rybchenko <[email protected]>
---
  drivers/net/virtio/virtio.c | 15 +++++++++++++--
  1 file changed, 13 insertions(+), 2 deletions(-)

Applied to dpdk-next-virtio/main.

Thanks,
Maxime