Add gve driver
Subsystems (4)
v2 cover Add gve driver
This patch series adds the gve driver which will support the Compute Engine Virtual NIC that will be available in the future.
v2:
- Patch 1:
- Remove gve_size_assert.h and use static_assert instead.
- Loop forever instead of bugging if the device won't reset
- Use module_pci_driver
- Patch 2:
- Use be16_to_cpu in the RX Seq No define
- Remove unneeded ndo_change_mtu
- Patch 3:
- No Changes
- Patch 4:
- Instead of checking netif_carrier_ok in ethtool stats, just make sure the rings have valid pointers.
Catherine Sullivan (4):
gve: Add basic driver framework for Compute Engine Virtual NIC
gve: Add transmit and receive support
gve: Add workqueue and reset support
gve: Add ethtool support
.../networking/device_drivers/google/gve.rst | 123 ++ .../networking/device_drivers/index.rst | 1 + MAINTAINERS | 9 + drivers/net/ethernet/Kconfig | 1 + drivers/net/ethernet/Makefile | 1 + drivers/net/ethernet/google/Kconfig | 27 + drivers/net/ethernet/google/Makefile | 5 + drivers/net/ethernet/google/gve/Makefile | 4 + drivers/net/ethernet/google/gve/gve.h | 456 ++++++ drivers/net/ethernet/google/gve/gve_adminq.c | 388 ++++++ drivers/net/ethernet/google/gve/gve_adminq.h | 215 +++ drivers/net/ethernet/google/gve/gve_desc.h | 113 ++ drivers/net/ethernet/google/gve/gve_ethtool.c | 239 ++++ drivers/net/ethernet/google/gve/gve_main.c | 1217 +++++++++++++++++ .../net/ethernet/google/gve/gve_register.h | 27 + drivers/net/ethernet/google/gve/gve_rx.c | 445 ++++++ drivers/net/ethernet/google/gve/gve_tx.c | 584 ++++++++ 17 files changed, 3855 insertions(+) create mode 100644 Documentation/networking/device_drivers/google/gve.rst create mode 100644 drivers/net/ethernet/google/Kconfig create mode 100644 drivers/net/ethernet/google/Makefile create mode 100644 drivers/net/ethernet/google/gve/Makefile create mode 100644 drivers/net/ethernet/google/gve/gve.h create mode 100644 drivers/net/ethernet/google/gve/gve_adminq.c create mode 100644 drivers/net/ethernet/google/gve/gve_adminq.h create mode 100644 drivers/net/ethernet/google/gve/gve_desc.h create mode 100644 drivers/net/ethernet/google/gve/gve_ethtool.c create mode 100644 drivers/net/ethernet/google/gve/gve_main.c create mode 100644 drivers/net/ethernet/google/gve/gve_register.h create mode 100644 drivers/net/ethernet/google/gve/gve_rx.c create mode 100644 drivers/net/ethernet/google/gve/gve_tx.c
Signature · 1 line
2.22.0.410.gd8fdbe21b5-goog
PATCH v2 1/4 gve: Add basic driver framework for Compute Engine Virtual NIC
Add a driver framework for the Compute Engine Virtual NIC that will be available in the future.
At this point the only functionality is loading the driver.
--- .../networking/device_drivers/google/gve.rst | 82 ++++ .../networking/device_drivers/index.rst | 1 + MAINTAINERS | 9 + drivers/net/ethernet/Kconfig | 1 + drivers/net/ethernet/Makefile | 1 + drivers/net/ethernet/google/Kconfig | 27 ++ drivers/net/ethernet/google/Makefile | 5 + drivers/net/ethernet/google/gve/Makefile | 4 + drivers/net/ethernet/google/gve/gve.h | 135 ++++++ drivers/net/ethernet/google/gve/gve_adminq.c | 250 ++++++++++ drivers/net/ethernet/google/gve/gve_adminq.h | 133 ++++++ drivers/net/ethernet/google/gve/gve_main.c | 446 ++++++++++++++++++ .../net/ethernet/google/gve/gve_register.h | 27 ++ 13 files changed, 1121 insertions(+) create mode 100644 Documentation/networking/device_drivers/google/gve.rst create mode 100644 drivers/net/ethernet/google/Kconfig create mode 100644 drivers/net/ethernet/google/Makefile create mode 100644 drivers/net/ethernet/google/gve/Makefile create mode 100644 drivers/net/ethernet/google/gve/gve.h create mode 100644 drivers/net/ethernet/google/gve/gve_adminq.c create mode 100644 drivers/net/ethernet/google/gve/gve_adminq.h create mode 100644 drivers/net/ethernet/google/gve/gve_main.c create mode 100644 drivers/net/ethernet/google/gve/gve_register.h
Documentation/networking/device_drivers/google/gve.rst new file +82 -0
@@ -0,0 +1,82 @@ +.. SPDX-License-Identifier: GPL-2.0+ + +============================================================== +Linux kernel driver for Compute Engine Virtual Ethernet (gve): +============================================================== + +Supported Hardware +=================== +The GVE driver binds to a single PCI device id used by the virtual +Ethernet device found in some Compute Engine VMs. + ++--------------+----------+---------+ +|Field | Value | Comments| ++==============+==========+=========+ +|Vendor ID | `0x1AE0` | Google | ++--------------+----------+---------+ +|Device ID | `0x0042` | | ++--------------+----------+---------+ +|Sub-vendor ID | `0x1AE0` | Google | ++--------------+----------+---------+ +|Sub-device ID | `0x0058` | | ++--------------+----------+---------+ +|Revision ID | `0x0` | | ++--------------+----------+---------+ +|Device Class | `0x200` | Ethernet| ++--------------+----------+---------+ + +PCI Bars +======== +The gVNIC PCI device exposes three 32-bit memory BARS: +- Bar0 - Device configuration and status registers. +- Bar1 - MSI-X vector table +- Bar2 - IRQ, RX and TX doorbells + +Device Interactions +=================== +The driver interacts with the device in the following ways: + - Registers + - A block of MMIO registers + - See gve_register.h for more detail + - Admin Queue + - See description below + - Interrupts + - See supported interrupts below + +Registers +--------- +All registers are MMIO and big endian. + +The registers are used for initializing and configuring the device as well as +querying device status in response to management interrupts. + +Admin Queue (AQ) +---------------- +The Admin Queue is a PAGE_SIZE memory block, treated as an array of AQ +commands, used by the driver to issue commands to the device and set up +resources.The driver and the device maintain a count of how many commands +have been submitted and executed. To issue AQ commands, the driver must do +the following (with proper locking): + +1) Copy new commands into next available slots in the AQ array +2) Increment its counter by he number of new commands +3) Write the counter into the GVE_ADMIN_QUEUE_DOORBELL register +4) Poll the ADMIN_QUEUE_EVENT_COUNTER register until it equals + the value written to the doorbell, or until a timeout. + +The device will update the status field in each AQ command reported as +executed through the ADMIN_QUEUE_EVENT_COUNTER register. + +Interrupts +---------- +The following interrupts are supported by the driver: + +Management Interrupt +~~~~~~~~~~~~~~~~~~~~ +The management interrupt is used by the device to tell the driver to +look at the GVE_DEVICE_STATUS register. + +Notification Block Interrupts +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The notification block interrupts are used to tell the driver to poll +the queues associated with that interrupt.
Documentation/networking/device_drivers/index.rst +1 -0
@@ -21,6 +21,7 @@ Contents: intel/i40e intel/iavf intel/ice + google/gve mellanox/mlx5 .. only:: subproject
MAINTAINERS +9 -0
@@ -6720,6 +6720,15 @@ L: [email protected] S: Maintained F: drivers/input/touchscreen/goodix.c +GOOGLE ETHERNET DRIVERS +M: Catherine Sullivan <[email protected]> +R: Sagi Shahar <[email protected]> +R: Jon Olson <[email protected]> +L: [email protected] +S: Supported +F: Documentation/networking/device_drivers/google/gve.txt +F: drivers/net/ethernet/google + GPD POCKET FAN DRIVER M: Hans de Goede <[email protected]> L: [email protected]
drivers/net/ethernet/Kconfig +1 -0
@@ -76,6 +76,7 @@ source "drivers/net/ethernet/ezchip/Kconfig" source "drivers/net/ethernet/faraday/Kconfig" source "drivers/net/ethernet/freescale/Kconfig" source "drivers/net/ethernet/fujitsu/Kconfig" +source "drivers/net/ethernet/google/Kconfig" source "drivers/net/ethernet/hisilicon/Kconfig" source "drivers/net/ethernet/hp/Kconfig" source "drivers/net/ethernet/huawei/Kconfig"
drivers/net/ethernet/Makefile +1 -0
@@ -39,6 +39,7 @@ obj-$(CONFIG_NET_VENDOR_EZCHIP) += ezchip/ obj-$(CONFIG_NET_VENDOR_FARADAY) += faraday/ obj-$(CONFIG_NET_VENDOR_FREESCALE) += freescale/ obj-$(CONFIG_NET_VENDOR_FUJITSU) += fujitsu/ +obj-$(CONFIG_NET_VENDOR_GOOGLE) += google/ obj-$(CONFIG_NET_VENDOR_HISILICON) += hisilicon/ obj-$(CONFIG_NET_VENDOR_HP) += hp/ obj-$(CONFIG_NET_VENDOR_HUAWEI) += huawei/
drivers/net/ethernet/google/Kconfig new file +27 -0
@@ -0,0 +1,27 @@ +# +# Google network device configuration +# + +config NET_VENDOR_GOOGLE + bool "Google Devices" + default y + help + If you have a network (Ethernet) device belonging to this class, say Y. + + Note that the answer to this question doesn't directly affect the + kernel: saying N will just cause the configurator to skip all + the questions about Google devices. If you say Y, you will be asked + for your specific device in the following questions. + +if NET_VENDOR_GOOGLE + +config GVE + tristate "Google Virtual NIC (gVNIC) support" + depends on (PCI_MSI && X86) + help + This driver supports Google Virtual NIC (gVNIC)" + + To compile this driver as a module, choose M here. + The module will be called gve. + +endif #NET_VENDOR_GOOGLE
drivers/net/ethernet/google/Makefile new file +5 -0
@@ -0,0 +1,5 @@ +# +# Makefile for the Google network device drivers. +# + +obj-$(CONFIG_GVE) += gve/
drivers/net/ethernet/google/gve/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +# Makefile for the Google virtual Ethernet (gve) driver + +obj-$(CONFIG_GVE) += gve.o +gve-objs := gve_main.o gve_adminq.o
drivers/net/ethernet/google/gve/gve.h new file +135 -0
135 lines not on this page · raw message
drivers/net/ethernet/google/gve/gve_adminq.c new file +250 -0
250 lines not on this page · raw message
drivers/net/ethernet/google/gve/gve_adminq.h new file +133 -0
133 lines not on this page · raw message
drivers/net/ethernet/google/gve/gve_main.c new file +446 -0
446 lines not on this page · raw message
drivers/net/ethernet/google/gve/gve_register.h new file +27 -0
@@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: (GPL-2.0 OR MIT) +* Google virtual Ethernet (gve) driver +* +* Copyright (C) 2015-2019 Google, Inc. +*/ + +#ifndef _GVE_REGISTER_H_ +#define _GVE_REGISTER_H_ + +/* Fixed Configuration Registers */ +struct gve_registers { +__be32 device_status; +__be32 driver_status; +__be32 max_tx_queues; +__be32 max_rx_queues; +__be32 adminq_pfn; +__be32 adminq_doorbell; +__be32 adminq_event_counter; +u8 reserved[3 ]; +u8 driver_version; +}; + +enum gve_device_status_flags { +GVE_DEVICE_STATUS_RESET_MASK =BIT (1 ), +GVE_DEVICE_STATUS_LINK_STATUS_MASK =BIT (2 ), +}; +#endif /* _GVE_REGISTER_H_ */
Signature · 1 line
2.22.0.410.gd8fdbe21b5-goog
kbuild test robot
build botRe: [PATCH net-next v2 1/4] gve: Add basic driver framework for Compute Engine Virtual NIC
I love your patch! Yet something to improve:
Hi Catherine,
I love your patch! Yet something to improve:
[auto build test ERROR on net-next/master]
url: https://github.com/0day-ci/linux/commits/Catherine-Sullivan/Add-gve-driver/20190629-143743
config: i386-allyesconfig (attached as .config)
compiler: gcc-7 (Debian 7.4.0-9) 7.4.0
reproduce:
# save the attached .config to linux build tree
make ARCH=i386 If you fix the issue, kindly add following tag
All errors (new ones prefixed by >>):
In file included from drivers/net/ethernet/google/gve/gve_main.c:16:0:
>> drivers/net/ethernet/google/gve/gve_adminq.h:102:1: error: static assertion failed: "sizeof(struct gve_adminq_set_driver_parameter) == 16"
static_assert(sizeof(struct gve_adminq_set_driver_parameter) == 16);
^~~~~~~~~~~~~~vim +102 drivers/net/ethernet/google/gve/gve_adminq.h
101
> 102 static_assert(sizeof(struct gve_adminq_set_driver_parameter) == 16);
103
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
PATCH v2 2/4 gve: Add transmit and receive support
In reply to Catherine SullivanAdd support for passing traffic.
--- .../networking/device_drivers/google/gve.rst | 30 + drivers/net/ethernet/google/gve/Makefile | 2 +- drivers/net/ethernet/google/gve/gve.h | 257 +++++++- drivers/net/ethernet/google/gve/gve_adminq.c | 138 +++++ drivers/net/ethernet/google/gve/gve_adminq.h | 82 +++ drivers/net/ethernet/google/gve/gve_desc.h | 113 ++++ drivers/net/ethernet/google/gve/gve_main.c | 562 ++++++++++++++++- drivers/net/ethernet/google/gve/gve_rx.c | 442 +++++++++++++ drivers/net/ethernet/google/gve/gve_tx.c | 584 ++++++++++++++++++ 9 files changed, 2205 insertions(+), 5 deletions(-) create mode 100644 drivers/net/ethernet/google/gve/gve_desc.h create mode 100644 drivers/net/ethernet/google/gve/gve_rx.c create mode 100644 drivers/net/ethernet/google/gve/gve_tx.c
Documentation/networking/device_drivers/google/gve.rst +30 -0
@@ -42,6 +42,8 @@ The driver interacts with the device in the following ways: - See description below - Interrupts - See supported interrupts below + - Transmit and Receive Queues + - See description below Registers ---------
@@ -80,3 +82,31 @@ Notification Block Interrupts ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The notification block interrupts are used to tell the driver to poll the queues associated with that interrupt. + +The handler for these irqs schedule the napi for that block to run +and poll the queues. + +Traffic Queues +-------------- +gVNIC's queues are composed of a descriptor ring and a buffer and are +assigned to a notification block. + +The descriptor rings are power-of-two-sized ring buffers consisting of +fixed-size descriptors. They advance their head pointer using a __be32 +doorbell located in Bar2. The tail pointers are advanced by consuming +descriptors in-order and updating a __be32 counter. Both the doorbell +and the counter overflow to zero. + +Each queue's buffers must be registered in advance with the device as a +queue page list, and packet data can only be put in those pages. + +Transmit +~~~~~~~~ +gve maps the buffers for transmit rings into a FIFO and copies the packets +into the FIFO before sending them to the NIC. + +Receive +~~~~~~~ +The buffers for receive rings are put into a data ring that is the same +length as the descriptor ring and the head and tail pointers advance over +the rings together.
drivers/net/ethernet/google/gve/Makefile +1 -1
@@ -1,4 +1,4 @@ # Makefile for the Google virtual Ethernet (gve) driver obj-$(CONFIG_GVE) += gve.o -gve-objs := gve_main.o gve_adminq.o +gve-objs := gve_main.o gve_tx.o gve_rx.o gve_adminq.o
drivers/net/ethernet/google/gve/gve.h +256 -1
297 lines not on this page · raw message
drivers/net/ethernet/google/gve/gve_adminq.c +138 -0
162 lines not on this page · raw message
drivers/net/ethernet/google/gve/gve_adminq.h +82 -0
106 lines not on this page · raw message
drivers/net/ethernet/google/gve/gve_desc.h new file +113 -0
113 lines not on this page · raw message
drivers/net/ethernet/google/gve/gve_main.c +559 -3
632 lines not on this page · raw message
drivers/net/ethernet/google/gve/gve_rx.c new file +442 -0
442 lines not on this page · raw message
drivers/net/ethernet/google/gve/gve_tx.c new file +584 -0
584 lines not on this page · raw message
Signature · 1 line
2.22.0.410.gd8fdbe21b5-goog
kbuild test robot
build botRe: [PATCH net-next v2 2/4] gve: Add transmit and receive support
I love your patch! Perhaps something to improve:
Hi Catherine,
I love your patch! Perhaps something to improve:
[auto build test WARNING on net-next/master]
url: https://github.com/0day-ci/linux/commits/Catherine-Sullivan/Add-gve-driver/20190629-143743
reproduce:
# apt-get install sparse
# sparse version: v0.6.1-rc1-7-g2b96cd8-dirty
make ARCH=x86_64 allmodconfig
make C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__'If you fix the issue, kindly add following tag
sparse warnings: (new ones prefixed by >>)
drivers/net/ethernet/google/gve/gve_rx.c:11:6: sparse: sparse: symbol 'gve_rx_remove_from_block' was not declared. Should it be static?
drivers/net/ethernet/google/gve/gve_rx.c:217:16: sparse: sparse: incorrect type in argument 1 (different base types) @@ expected unsigned int val @@ got restricted __be3unsigned int val @@
drivers/net/ethernet/google/gve/gve_rx.c:217:16: sparse: expected unsigned int val
drivers/net/ethernet/google/gve/gve_rx.c:217:16: sparse: got restricted __be32 [usertype]
drivers/net/ethernet/google/gve/gve_rx.c:349:27: sparse: sparse: incorrect type in assignment (different base types) @@ expected restricted __wsum [usertype] csum @@ got restricted __wsum [usertype] csum @@
drivers/net/ethernet/google/gve/gve_rx.c:349:27: sparse: expected restricted __wsum [usertype] csum
drivers/net/ethernet/google/gve/gve_rx.c:349:27: sparse: got restricted __be16 [usertype] csum
drivers/net/ethernet/google/gve/gve_rx.c:374:19: sparse: sparse: incorrect type in assignment (different base types) @@ expected unsigned short [usertype] flags_seq @@ got resunsigned short [usertype] flags_seq @@
drivers/net/ethernet/google/gve/gve_rx.c:374:19: sparse: expected unsigned short [usertype] flags_seq
drivers/net/ethernet/google/gve/gve_rx.c:374:19: sparse: got restricted __be16 [usertype] flags_seq
>> drivers/net/ethernet/google/gve/gve_rx.c:378:17: sparse: sparse: cast to restricted __be16 >> drivers/net/ethernet/google/gve/gve_rx.c:378:17: sparse: sparse: cast to restricted __be16 >> drivers/net/ethernet/google/gve/gve_rx.c:378:17: sparse: sparse: cast to restricted __be16 >> drivers/net/ethernet/google/gve/gve_rx.c:378:17: sparse: sparse: cast to restricted __be16
vim +378 drivers/net/ethernet/google/gve/gve_rx.c
212
213 void gve_rx_write_doorbell(struct gve_priv *priv, struct gve_rx_ring *rx)
214 {
215 u32 db_idx = be32_to_cpu(rx->q_resources->db_index);
216
> 217 writel(cpu_to_be32(rx->desc.fill_cnt), &priv->db_bar2[db_idx]);
218 }
219
220 static enum pkt_hash_types gve_rss_type(__be16 pkt_flags)
221 {
222 if (likely(pkt_flags & (GVE_RXF_TCP | GVE_RXF_UDP)))
223 return PKT_HASH_TYPE_L4;
224 if (pkt_flags & (GVE_RXF_IPV4 | GVE_RXF_IPV6))
225 return PKT_HASH_TYPE_L3;
226 return PKT_HASH_TYPE_L2;
227 }
228
229 static struct sk_buff *gve_rx_copy(struct net_device *dev,
230 struct napi_struct *napi,
231 struct gve_rx_slot_page_info *page_info,
232 u16 len)
233 {
234 struct sk_buff *skb = napi_alloc_skb(napi, len);
235 void *va = page_info->page_address + GVE_RX_PAD +
236 page_info->page_offset;
237
238 if (unlikely(!skb))
239 return NULL;
240
241 __skb_put(skb, len);
242
243 skb_copy_to_linear_data(skb, va, len);
244
245 skb->protocol = eth_type_trans(skb, dev);
246 return skb;
247 }
248
249 static struct sk_buff *gve_rx_add_frags(struct net_device *dev,
250 struct napi_struct *napi,
251 struct gve_rx_slot_page_info *page_info,
252 u16 len)
253 {
254 struct sk_buff *skb = napi_get_frags(napi);
255
256 if (unlikely(!skb))
257 return NULL;
258
259 skb_add_rx_frag(skb, 0, page_info->page,
260 page_info->page_offset +
261 GVE_RX_PAD, len, PAGE_SIZE / 2);
262
263 return skb;
264 }
265
266 static void gve_rx_flip_buff(struct gve_rx_slot_page_info *page_info,
267 struct gve_rx_data_slot *data_ring)
268 {
269 u64 addr = be64_to_cpu(data_ring->qpl_offset);
270
271 page_info->page_offset ^= PAGE_SIZE / 2;
272 addr ^= PAGE_SIZE / 2;
273 data_ring->qpl_offset = cpu_to_be64(addr);
274 }
275
276 static bool gve_rx(struct gve_rx_ring *rx, struct gve_rx_desc *rx_desc,
277 netdev_features_t feat)
278 {
279 struct gve_rx_slot_page_info *page_info;
280 struct gve_priv *priv = rx->gve;
281 struct napi_struct *napi = &priv->ntfy_blocks[rx->ntfy_id].napi;
282 struct net_device *dev = priv->dev;
283 struct sk_buff *skb;
284 int pagecount;
285 u16 len;
286 u32 idx;
287
288 /* drop this packet */
289 if (unlikely(rx_desc->flags_seq & GVE_RXF_ERR))
290 return true;
291
292 len = be16_to_cpu(rx_desc->len) - GVE_RX_PAD;
293 idx = rx->data.cnt & rx->data.mask;
294 page_info = &rx->data.page_info[idx];
295
296 /* gvnic can only receive into registered segments. If the buffer
297 * can't be recycled, our only choice is to copy the data out of
298 * it so that we can return it to the device.
299 */
300
301 #if PAGE_SIZE == 4096
302 if (len <= priv->rx_copybreak) {
303 /* Just copy small packets */
304 skb = gve_rx_copy(dev, napi, page_info, len);
305 goto have_skb;
306 }
307 if (unlikely(!gve_can_recycle_pages(dev))) {
308 skb = gve_rx_copy(dev, napi, page_info, len);
309 goto have_skb;
310 }
311 pagecount = page_count(page_info->page);
312 if (pagecount == 1) {
313 /* No part of this page is used by any SKBs; we attach
314 * the page fragment to a new SKB and pass it up the
315 * stack.
316 */
317 skb = gve_rx_add_frags(dev, napi, page_info, len);
318 if (!skb)
319 return true;
320 /* Make sure the kernel stack can't release the page */
321 get_page(page_info->page);
322 /* "flip" to other packet buffer on this page */
323 gve_rx_flip_buff(page_info, &rx->data.data_ring[idx]);
324 } else if (pagecount >= 2) {
325 /* We have previously passed the other half of this
326 * page up the stack, but it has not yet been freed.
327 */
328 skb = gve_rx_copy(dev, napi, page_info, len);
329 } else {
330 WARN(pagecount < 1, "Pagecount should never be < 1");
331 return false;
332 }
333 #else
334 skb = gve_rx_copy(dev, napi, page_info, len);
335 #endif
336
337 have_skb:
338 if (!skb)
339 return true;
340
341 rx->data.cnt++;
342
343 if (likely(feat & NETIF_F_RXCSUM)) {
344 /* NIC passes up the partial sum */
345 if (rx_desc->csum)
346 skb->ip_summed = CHECKSUM_COMPLETE;
347 else
348 skb->ip_summed = CHECKSUM_NONE;
349 skb->csum = rx_desc->csum;
350 }
351
352 /* parse flags & pass relevant info up */
353 if (likely(feat & NETIF_F_RXHASH) &&
354 gve_needs_rss(rx_desc->flags_seq))
355 skb_set_hash(skb, be32_to_cpu(rx_desc->rss_hash),
356 gve_rss_type(rx_desc->flags_seq));
357
358 if (skb_is_nonlinear(skb))
359 napi_gro_frags(napi);
360 else
361 napi_gro_receive(napi, skb);
362 return true;
363 }
364
365 static bool gve_rx_work_pending(struct gve_rx_ring *rx)
366 {
367 struct gve_rx_desc *desc;
368 u16 flags_seq;
369 u32 next_idx;
370
371 next_idx = rx->desc.cnt & rx->desc.mask;
372 desc = rx->desc.desc_ring + next_idx;
373
374 flags_seq = desc->flags_seq;
375 /* Make sure we have synchronized the seq no with the device */
376 smp_rmb();
377
> 378 return (GVE_SEQNO(flags_seq) == rx->desc.seqno); 379 }
380
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
kbuild test robot
build botRe: [PATCH net-next v2 2/4] gve: Add transmit and receive support
In reply to Catherine SullivanI love your patch! Yet something to improve:
Hi Catherine,
I love your patch! Yet something to improve:
[auto build test ERROR on net-next/master]
url: https://github.com/0day-ci/linux/commits/Catherine-Sullivan/Add-gve-driver/20190629-143743
config: i386-allyesconfig (attached as .config)
compiler: gcc-7 (Debian 7.4.0-9) 7.4.0
reproduce:
# save the attached .config to linux build tree
make ARCH=i386 If you fix the issue, kindly add following tag
All errors (new ones prefixed by >>):
In file included from drivers/net/ethernet/google/gve/gve_main.c:16:0:
>> drivers/net/ethernet/google/gve/gve_adminq.h:134:1: error: static assertion failed: "sizeof(struct gve_adminq_create_rx_queue) == 48"
static_assert(sizeof(struct gve_adminq_create_rx_queue) == 48);
^~~~~~~~~~~~~~
drivers/net/ethernet/google/gve/gve_adminq.h:171:1: error: static assertion failed: "sizeof(struct gve_adminq_set_driver_parameter) == 16"
static_assert(sizeof(struct gve_adminq_set_driver_parameter) == 16);
^~~~~~~~~~~~~~vim +134 drivers/net/ethernet/google/gve/gve_adminq.h
133
> 134 static_assert(sizeof(struct gve_adminq_create_rx_queue) == 48);
135
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
PATCH v2 3/4 gve: Add workqueue and reset support
In reply to Catherine SullivanAdd support for the workqueue to handle management interrupts and support for resets.
--- .../networking/device_drivers/google/gve.rst | 11 + drivers/net/ethernet/google/gve/gve.h | 62 ++++++ drivers/net/ethernet/google/gve/gve_main.c | 190 +++++++++++++++++- drivers/net/ethernet/google/gve/gve_rx.c | 5 +- 4 files changed, 260 insertions(+), 8 deletions(-)
Documentation/networking/device_drivers/google/gve.rst +11 -0
@@ -40,6 +40,8 @@ The driver interacts with the device in the following ways: - See gve_register.h for more detail - Admin Queue - See description below + - Reset + - At any time the device can be reset - Interrupts - See supported interrupts below - Transmit and Receive Queues
@@ -69,6 +71,12 @@ the following (with proper locking): The device will update the status field in each AQ command reported as executed through the ADMIN_QUEUE_EVENT_COUNTER register. +Device Resets +------------- +A device reset is triggered by writing 0x0 to the AQ PFN register. +This causes the device to release all resources allocated by the +driver, including the AQ itself. + Interrupts ---------- The following interrupts are supported by the driver:
@@ -78,6 +86,9 @@ Management Interrupt The management interrupt is used by the device to tell the driver to look at the GVE_DEVICE_STATUS register. +The handler for the management irq simply queues the service task in +the workqueue to check the register and acks the irq. + Notification Block Interrupts ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The notification block interrupts are used to tell the driver to poll
drivers/net/ethernet/google/gve/gve.h +62 -0
81 lines not on this page · raw message
drivers/net/ethernet/google/gve/gve_main.c +183 -7
327 lines not on this page · raw message
drivers/net/ethernet/google/gve/gve_rx.c +4 -1
@@ -335,6 +335,9 @@ static bool gve_rx(struct gve_rx_ring *rx, struct gve_rx_desc *rx_desc,#endif have_skb: +/* We didn't manage to allocate an skb but we haven't had any +* reset worthy failures. +*/ if (!skb)return true;
@@ -400,7 +403,7 @@ bool gve_clean_rx_done(struct gve_rx_ring *rx, int budget, rx->desc.seqno); bytes +=be16_to_cpu (desc->len) -GVE_RX_PAD ;if (!gve_rx (rx, desc, feat)) -return false; +gve_schedule_reset (priv); cnt++; idx = cnt & rx->desc.mask; desc = rx->desc.desc_ring + idx;
Signature · 1 line
2.22.0.410.gd8fdbe21b5-goog
PATCH v2 4/4 gve: Add ethtool support
In reply to Catherine SullivanAdd support for the following ethtool commands:
ethtool -s|--change devname [msglvl N] [msglevel type on|off] ethtool -S|--statistics devname ethtool -i|--driver devname ethtool -l|--show-channels devname ethtool -L|--set-channels devname ethtool -g|--show-ring devname
ethtool --reset devname
--- drivers/net/ethernet/google/gve/Makefile | 2 +- drivers/net/ethernet/google/gve/gve.h | 4 + drivers/net/ethernet/google/gve/gve_ethtool.c | 239 ++++++++++++++++++ drivers/net/ethernet/google/gve/gve_main.c | 39 +++ 4 files changed, 283 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ethernet/google/gve/gve_ethtool.c
drivers/net/ethernet/google/gve/Makefile +1 -1
@@ -1,4 +1,4 @@ # Makefile for the Google virtual Ethernet (gve) driver obj-$(CONFIG_GVE) += gve.o -gve-objs := gve_main.o gve_tx.o gve_rx.o gve_adminq.o +gve-objs := gve_main.o gve_tx.o gve_rx.o gve_ethtool.o gve_adminq.o
drivers/net/ethernet/google/gve/gve.h +4 -0
@@ -449,4 +449,8 @@ int gve_reset(struct gve_priv *priv, bool attempt_teardown);int gve_adjust_queues (struct gve_priv *priv,struct gve_queue_config new_rx_config,struct gve_queue_config new_tx_config); +/* exported by ethtool.c */ +extern const struct ethtool_ops gve_ethtool_ops; +/* needed by ethtool */ +extern const char gve_version_str[];#endif /* _GVE_H_ */
drivers/net/ethernet/google/gve/gve_ethtool.c new file +239 -0
239 lines not on this page · raw message
drivers/net/ethernet/google/gve/gve_main.c +39 -0
@@ -735,6 +735,44 @@ static int gve_close(struct net_device *dev)return gve_reset_recovery (priv, false); } +int gve_adjust_queues (struct gve_priv *priv, +struct gve_queue_config new_rx_config, +struct gve_queue_config new_tx_config) +{ +int err; + +if (netif_carrier_ok (priv->dev)) { +/* To make this process as simple as possible we teardown the +* device, set the new configuration, and then bring the device +* up again. +*/ + err =gve_close (priv->dev); +/* we have already tried to reset in close, +* just fail at this point +*/ +if (err) +return err; + priv->tx_cfg = new_tx_config; + priv->rx_cfg = new_rx_config; + + err =gve_open (priv->dev); +if (err) + goto err; + +return 0 ; + } +/* Set the config for the next up. */ + priv->tx_cfg = new_tx_config; + priv->rx_cfg = new_rx_config; + +return 0 ; +err: +netif_err (priv, drv, priv->dev, +"Adjust queues failed! !!! DISABLING ALL QUEUES !!!\n" ); +gve_turndown (priv); +return err; +} +static void gve_turndown (struct gve_priv *priv) {int idx;
@@ -1071,6 +1109,7 @@ static int gve_probe(struct pci_dev *pdev, const struct pci_device_id *ent) }SET_NETDEV_DEV (dev, &pdev->dev);pci_set_drvdata (pdev, dev); + dev->ethtool_ops = &gve_ethtool_ops; dev->netdev_ops = &gve_netdev_ops;/* advertise features */ dev->hw_features =NETIF_F_HIGHDMA ;
Signature · 1 line
2.22.0.410.gd8fdbe21b5-goog
Re: [PATCH net-next v2 4/4] gve: Add ethtool support
Quoted 16 lines · On Fri, 28 Jun 2019 10:56:33 -0700 Catherine Sullivan <[email protected]> wrote:
drivers/net/ethernet/google/gve/gve_ethtool.c new file +16
-0+static void +gve_get_ethtool_stats (struct net_device *netdev, +struct ethtool_stats *stats,u64 *data) +{ +struct gve_priv *priv =netdev_priv (netdev); +u64 rx_pkts, rx_bytes, tx_pkts, tx_bytes; +int ring; +int i; + +ASSERT_RTNL (); + +for (rx_pkts =0 , rx_bytes =0 , ring =0 ; + ring < priv->rx_cfg.num_queues; ring++) { +if (priv->rx) { + rx_pkts += priv->rx[ring].rpackets; + rx_bytes += priv->rx[ring].rbytes;
drivers/net/ethernet/google/gve/gve_ethtool.c line 114
Quoted 10 lines
drivers/net/ethernet/google/gve/gve_ethtool.c new file +10
-0+ } + } +for (tx_pkts =0 , tx_bytes =0 , ring =0 ; + ring < priv->tx_cfg.num_queues; ring++) { +if (priv->tx) { + tx_pkts += priv->tx[ring].pkt_done; + tx_bytes += priv->tx[ring].bytes_done; + } + } +memset (data,0 ,GVE_MAIN_STATS_LEN *sizeof (*data));
memset here is unnecessary since ethtool_get_stats allocates and zeros the memory already.
Re: [PATCH net-next v2 4/4] gve: Add ethtool support
Quoted 25 lines · Stephen Hemminger
Quoted 21 lines · On Fri, 28 Jun 2019 10:56:33 -0700 Catherine Sullivan <[email protected]> wrote:
drivers/net/ethernet/google/gve/gve_ethtool.c new file +21
-0+static void +gve_get_ethtool_stats (struct net_device *netdev, +struct ethtool_stats *stats,u64 *data) +{ +struct gve_priv *priv =netdev_priv (netdev); +u64 rx_pkts, rx_bytes, tx_pkts, tx_bytes; +int ring; +int i; + +ASSERT_RTNL (); + +for (rx_pkts =0 , rx_bytes =0 , ring =0 ; + ring < priv->rx_cfg.num_queues; ring++) { +if (priv->rx) { + rx_pkts += priv->rx[ring].rpackets; + rx_bytes += priv->rx[ring].rbytes; + } + } +for (tx_pkts =0 , tx_bytes =0 , ring =0 ; + ring < priv->tx_cfg.num_queues; ring++) { +if (priv->tx) {
Quoted 8 lines
drivers/net/ethernet/google/gve/gve_ethtool.c line 114
Quoted 5 lines
drivers/net/ethernet/google/gve/gve_ethtool.c new file +5
-0+ tx_pkts += priv->tx[ring].pkt_done; + tx_bytes += priv->tx[ring].bytes_done; + } + } +memset (data,0 ,GVE_MAIN_STATS_LEN *sizeof (*data));memset here is unnecessary since
ethtool_get_statsallocates and zeros the memory already.
Ah, thanks, removed in v3.