Skip to contentSkip to conversation

Handle alternate miss-completions

PATCH Applied Jeroen de Borst 3 participants 5 messages Nov 17, 2022 – Nov 21, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v5 cover Handle alternate miss-completions

Some versions of the virtual NIC present miss-completions in an alternative way. Let the diver handle these alternate completions and announce this capability to the device.

The capability is announced uing a new AdminQ command that sends driver information to the device. The device can refuse a driver if it is lacking support for a capability, or it can adopt it's behavior to work around OS specific issues.

Changed in v5:
- Removed comments in fucntion calls
- Switched ENOTSUPP back to EOPNOTSUPP and made sure it gets passed Changed in v4:
- Clarified new AdminQ command in cover letter
- Changed EOPNOTSUPP to ENOTSUPP to match device's response Changed in v3:
- Rewording cover letter
- Added 'Reviewed-by: Jesse Brandeburg <[email protected]>' Changes in v2:
- Changed the subject to include 'gve:'

Jeroen de Borst (2):
Adding a new AdminQ command to verify driver
Handle alternate miss completions

 drivers/net/ethernet/google/gve/gve.h         |  1 +
 drivers/net/ethernet/google/gve/gve_adminq.c  | 21 +++++++-
 drivers/net/ethernet/google/gve/gve_adminq.h  | 51 ++++++++++++++++++
 .../net/ethernet/google/gve/gve_desc_dqo.h    |  5 ++
 drivers/net/ethernet/google/gve/gve_main.c    | 52 +++++++++++++++++++
 drivers/net/ethernet/google/gve/gve_tx_dqo.c  | 20 ++++---
 6 files changed, 142 insertions(+), 8 deletions(-)
Signature · 1 line
2.38.1.431.g37b22c650d-goog
#1 lore raw

PATCH v5 1/2 gve: Adding a new AdminQ command to verify driver

Check whether the driver is compatible with the device presented.

---
 drivers/net/ethernet/google/gve/gve.h        |  1 +
 drivers/net/ethernet/google/gve/gve_adminq.c | 21 +++++++-
 drivers/net/ethernet/google/gve/gve_adminq.h | 49 ++++++++++++++++++
 drivers/net/ethernet/google/gve/gve_main.c   | 52 ++++++++++++++++++++
 4 files changed, 122 insertions(+), 1 deletion(-)
drivers/net/ethernet/google/gve/gve.h +1 -0
index 5655da9cd236..64eb0442c82f 100644
@@ -563,6 +563,7 @@ struct gve_priv {
 	u32 adminq_report_stats_cnt;
 	u32 adminq_report_link_speed_cnt;
 	u32 adminq_get_ptype_map_cnt;
+	u32 adminq_verify_driver_compatibility_cnt;
 
 	/* Global stats */
 	u32 interface_up_cnt; /* count of times interface turned up since last reset */
drivers/net/ethernet/google/gve/gve_adminq.c +20 -1
index f7621ab672b9..60061288ad9d 100644
@@ -289,7 +289,7 @@ static int gve_adminq_parse_err(struct gve_priv *priv, u32 status)
 	case GVE_ADMINQ_COMMAND_ERROR_RESOURCE_EXHAUSTED:
 		return -ENOMEM;
 	case GVE_ADMINQ_COMMAND_ERROR_UNIMPLEMENTED:
-		return -ENOTSUPP;
+		return -EOPNOTSUPP;
 	default:
 		dev_err(&priv->pdev->dev, "parse_aq_err: unknown status code %d\n", status);
 		return -EINVAL;
@@ -407,6 +407,9 @@ static int gve_adminq_issue_cmd(struct gve_priv *priv,
 	case GVE_ADMINQ_GET_PTYPE_MAP:
 		priv->adminq_get_ptype_map_cnt++;
 		break;
+	case GVE_ADMINQ_VERIFY_DRIVER_COMPATIBILITY:
+		priv->adminq_verify_driver_compatibility_cnt++;
+		break;
 	default:
 		dev_err(&priv->pdev->dev, "unknown AQ command opcode %d\n", opcode);
 	}
@@ -878,6 +881,22 @@ int gve_adminq_report_stats(struct gve_priv *priv, u64 stats_report_len,
 	return gve_adminq_execute_cmd(priv, &cmd);
 }
 
+int gve_adminq_verify_driver_compatibility(struct gve_priv *priv,
+					   u64 driver_info_len,
+					   dma_addr_t driver_info_addr)
+{
+	union gve_adminq_command cmd;
+
+	memset(&cmd, 0, sizeof(cmd));
+	cmd.opcode = cpu_to_be32(GVE_ADMINQ_VERIFY_DRIVER_COMPATIBILITY);
+	cmd.verify_driver_compatibility = (struct gve_adminq_verify_driver_compatibility) {
+		.driver_info_len = cpu_to_be64(driver_info_len),
+		.driver_info_addr = cpu_to_be64(driver_info_addr),
+	};
+
+	return gve_adminq_execute_cmd(priv, &cmd);
+}
+
 int gve_adminq_report_link_speed(struct gve_priv *priv)
 {
 	union gve_adminq_command gvnic_cmd;
drivers/net/ethernet/google/gve/gve_adminq.h +49 -0

73 lines not on this page · raw message

drivers/net/ethernet/google/gve/gve_main.c +52 -0

70 lines not on this page · raw message

Signature · 1 line
2.38.1.431.g37b22c650d-goog
#2 lore raw

Re: [PATCH net-next v5 1/2] gve: Adding a new AdminQ command to verify driver

Quoted 11 lines · Jeroen de Borst

Check whether the driver is compatible with the device presented.

---
 drivers/net/ethernet/google/gve/gve.h        |  1 +
 drivers/net/ethernet/google/gve/gve_adminq.c | 21 +++++++-
 drivers/net/ethernet/google/gve/gve_adminq.h | 49 ++++++++++++++++++
 drivers/net/ethernet/google/gve/gve_main.c   | 52 ++++++++++++++++++++
 4 files changed, 122 insertions(+), 1 deletion(-)

<...>

Quoted 6 lines
drivers/net/ethernet/google/gve/gve_adminq.h +6 -0
+enum gve_driver_capbility {
+	gve_driver_capability_gqi_qpl = 0,
+	gve_driver_capability_gqi_rda = 1,
+	gve_driver_capability_dqo_qpl = 2, /* reserved for future use */
+	gve_driver_capability_dqo_rda = 3,
+};

drivers/net/ethernet/google/gve/gve_adminq.h enum gve_sup_feature_mask { · line 167

Quoted 10 lines
drivers/net/ethernet/google/gve/gve_adminq.h +10 -0
+
+#define GVE_CAP1(a) BIT((int)a)
+#define GVE_CAP2(a) BIT(((int)a) - 64)
+#define GVE_CAP3(a) BIT(((int)a) - 128)
+#define GVE_CAP4(a) BIT(((int)a) - 192)
+
+#define GVE_DRIVER_CAPABILITY_FLAGS1 \
+	(GVE_CAP1(gve_driver_capability_gqi_qpl) | \
+	 GVE_CAP1(gve_driver_capability_gqi_rda) | \
+	 GVE_CAP1(gve_driver_capability_dqo_rda))

I never understood it why people do it.

You created named enum gve_driver_capbility, but nothing in the code uses this name and you use the values as bits, which later you cast them to int.

It will be much saner, if you use anonymous enum, which is int in C-world and you won't need any (int) casting when you call to BIT().

BTW, you don't need this casting anyway and it will be much better if you use >> for bit operations and not "- 64|128|192".

Thanks

#3 lore raw

PATCH v5 2/2 gve: Handle alternate miss completions

In reply to Jeroen de Borst

The virtual NIC has 2 ways of indicating a miss-path completion. This handles the alternate.

---
 drivers/net/ethernet/google/gve/gve_adminq.h  |  4 +++-
 .../net/ethernet/google/gve/gve_desc_dqo.h    |  5 +++++
 drivers/net/ethernet/google/gve/gve_tx_dqo.c  | 20 ++++++++++++-------
 3 files changed, 21 insertions(+), 8 deletions(-)
drivers/net/ethernet/google/gve/gve_adminq.h +3 -1
index b9ee8be73f96..cf29662e6ad1 100644
@@ -154,6 +154,7 @@ enum gve_driver_capbility {
 	gve_driver_capability_gqi_rda = 1,
 	gve_driver_capability_dqo_qpl = 2, /* reserved for future use */
 	gve_driver_capability_dqo_rda = 3,
+	gve_driver_capability_alt_miss_compl = 4,
 };
 
 #define GVE_CAP1(a) BIT((int)a)
@@ -164,7 +165,8 @@ enum gve_driver_capbility {
 #define GVE_DRIVER_CAPABILITY_FLAGS1 \
 	(GVE_CAP1(gve_driver_capability_gqi_qpl) | \
 	 GVE_CAP1(gve_driver_capability_gqi_rda) | \
-	 GVE_CAP1(gve_driver_capability_dqo_rda))
+	 GVE_CAP1(gve_driver_capability_dqo_rda) | \
+	 GVE_CAP1(gve_driver_capability_alt_miss_compl))
 
 #define GVE_DRIVER_CAPABILITY_FLAGS2 0x0
 #define GVE_DRIVER_CAPABILITY_FLAGS3 0x0
drivers/net/ethernet/google/gve/gve_desc_dqo.h +5 -0
index e8fe9adef7f2..f79cd0591110 100644
@@ -176,6 +176,11 @@ static_assert(sizeof(struct gve_tx_compl_desc) == 8);
 #define GVE_COMPL_TYPE_DQO_MISS 0x1 /* Miss path completion */
 #define GVE_COMPL_TYPE_DQO_REINJECTION 0x3 /* Re-injection completion */
 
+/* The most significant bit in the completion tag can change the completion
+ * type from packet completion to miss path completion.
+ */
+#define GVE_ALT_MISS_COMPL_BIT BIT(15)
+
 /* Descriptor to post buffers to HW on buffer queue. */
 struct gve_rx_desc_dqo {
 	__le16 buf_id; /* ID returned in Rx completion descriptor */
drivers/net/ethernet/google/gve/gve_tx_dqo.c +13 -7
index 588d64819ed5..b76143bfd594 100644
@@ -953,12 +953,18 @@ int gve_clean_tx_done_dqo(struct gve_priv *priv, struct gve_tx_ring *tx,
 			atomic_set_release(&tx->dqo_compl.hw_tx_head, tx_head);
 		} else if (type == GVE_COMPL_TYPE_DQO_PKT) {
 			u16 compl_tag = le16_to_cpu(compl_desc->completion_tag);
-
-			gve_handle_packet_completion(priv, tx, !!napi,
-						     compl_tag,
-						     &pkt_compl_bytes,
-						     &pkt_compl_pkts,
-						     /*is_reinjection=*/false);
+			if (compl_tag & GVE_ALT_MISS_COMPL_BIT) {
+				compl_tag &= ~GVE_ALT_MISS_COMPL_BIT;
+				gve_handle_miss_completion(priv, tx, compl_tag,
+							   &miss_compl_bytes,
+							   &miss_compl_pkts);
+			} else {
+				gve_handle_packet_completion(priv, tx, !!napi,
+							     compl_tag,
+							     &pkt_compl_bytes,
+							     &pkt_compl_pkts,
+							     false);
+			}
 		} else if (type == GVE_COMPL_TYPE_DQO_MISS) {
 			u16 compl_tag = le16_to_cpu(compl_desc->completion_tag);
 
@@ -972,7 +978,7 @@ int gve_clean_tx_done_dqo(struct gve_priv *priv, struct gve_tx_ring *tx,
 						     compl_tag,
 						     &reinject_compl_bytes,
 						     &reinject_compl_pkts,
-						     /*is_reinjection=*/true);
+						     true);
 		}
 
 		tx->dqo_compl.head =
Signature · 1 line
2.38.1.431.g37b22c650d-goog
#4 lore raw
In reply to Jeroen de Borst
This series was applied to netdev/net-next.git (master) by David S. Miller <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (master) by David S. Miller <[email protected]>:

Quoted 10 lines · On Thu, 17 Nov 2022 08:26:59 -0800 you wrote:

Some versions of the virtual NIC present miss-completions in an alternative way. Let the diver handle these alternate completions and announce this capability to the device.

The capability is announced uing a new AdminQ command that sends driver information to the device. The device can refuse a driver if it is lacking support for a capability, or it can adopt it's behavior to work around OS specific issues.

[...]

Here is the summary with links:
- [net-next,v5,1/2] gve: Adding a new AdminQ command to verify driver

    https://git.kernel.org/netdev/net-next/c/c2a0c3ed5b64
  - [net-next,v5,2/2] gve: Handle alternate miss completions
    https://git.kernel.org/netdev/net-next/c/a5affbd8a73e

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html