Skip to contentSkip to conversation

virtio-net: Fix and update interrupt moderation

PATCH Superseded Heng Qi 3 participants 10 messages Sep 25, 2023 – Oct 8, 2023 1 message not in the archive

Subsystems (4)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Heng Qi

author
#0 lore raw

v2 cover virtio-net: Fix and update interrupt moderation

The setting of virtio coalescing parameters involves all-queues and per queue, so we must be careful to synchronize the two.

Regarding napi_tx switching, this patch set is not only compatible with the previous way of using tx-frames to switch napi_tx, but also improves the user experience when setting interrupt parameters.

This patch set has been tested and was part of the previous netdim patch set[1] and is now being split to be rolled out in steps.

[1] https://lore.kernel.org/all/[email protected]/

---
v1->v2:
1. Fix some minor comments and add ack tags.

Heng Qi (6):
virtio-net: initially change the value of tx-frames
virtio-net: fix mismatch of getting tx-frames
virtio-net: consistently save parameters for per-queue
virtio-net: fix per queue coalescing parameter setting
virtio-net: fix the vq coalescing setting for vq resize
virtio-net: a tiny comment update

 drivers/net/virtio_net.c | 104 ++++++++++++++++++++++++++++-----------
 1 file changed, 75 insertions(+), 29 deletions(-)
Signature · 1 line
2.19.1.6.gb485710b
author
#1 lore raw

PATCH v2 1/6 virtio-net: initially change the value of tx-frames

Background:
1. Commit 0c465be183c7 ("virtio_net: ethtool tx napi configuration") uses tx-frames to toggle napi_tx (0 off and 1 on) if notification coalescing is not supported.
2. Commit 31c03aef9bc2 ("virtio_net: enable napi_tx by default") enables napi_tx for all txqs by default.

Status:
When virtio-net supports notification coalescing, after initialization, tx-frames is 0 and napi_tx is true.

Problem:
When the user only wants to set rx coalescing params using
ethtool -C eth0 rx-usecs 10, or
ethtool -Q eth0 queue_mask 0x1 -C rx-usecs 10,
these cmds will carry tx-frames as 0, causing the napi_tx switching condition is satisfied. Then the user gets:
netlink error: Device or resource busy.

The same happens when trying to set rx-frames, adaptive_rx, adaptive_tx...

How to fix:
When notification coalescing feature is negotiated, initially make the value of tx-frames to be consistent with napi_tx.

For compatibility with the past, it is still supported to use tx-frames to toggle napi_tx.

---
 drivers/net/virtio_net.c | 28 +++++++++++++++++++++-------
 1 file changed, 21 insertions(+), 7 deletions(-)
drivers/net/virtio_net.c +21 -7
index fe7f314d65c9..10878c9d430a 100644
@@ -4442,13 +4442,6 @@ static int virtnet_probe(struct virtio_device *vdev)
 		dev->xdp_features |= NETDEV_XDP_ACT_RX_SG;
 	}
 
-	if (virtio_has_feature(vi->vdev, VIRTIO_NET_F_NOTF_COAL)) {
-		vi->intr_coal_rx.max_usecs = 0;
-		vi->intr_coal_tx.max_usecs = 0;
-		vi->intr_coal_tx.max_packets = 0;
-		vi->intr_coal_rx.max_packets = 0;
-	}
-
 	if (virtio_has_feature(vdev, VIRTIO_NET_F_HASH_REPORT))
 		vi->has_rss_hash_report = true;
 
@@ -4523,6 +4516,27 @@ static int virtnet_probe(struct virtio_device *vdev)
 	if (err)
 		goto free;
 
+	if (virtio_has_feature(vi->vdev, VIRTIO_NET_F_NOTF_COAL)) {
+		vi->intr_coal_rx.max_usecs = 0;
+		vi->intr_coal_tx.max_usecs = 0;
+		vi->intr_coal_rx.max_packets = 0;
+
+		/* Keep the default values of the coalescing parameters
+		 * aligned with the default napi_tx state.
+		 */
+		if (vi->sq[0].napi.weight)
+			vi->intr_coal_tx.max_packets = 1;
+		else
+			vi->intr_coal_tx.max_packets = 0;
+	}
+
+	if (virtio_has_feature(vi->vdev, VIRTIO_NET_F_VQ_NOTF_COAL)) {
+		/* The reason is the same as VIRTIO_NET_F_NOTF_COAL. */
+		for (i = 0; i < vi->max_queue_pairs; i++)
+			if (vi->sq[i].napi.weight)
+				vi->sq[i].intr_coal.max_packets = 1;
+	}
+
 #ifdef CONFIG_SYSFS
 	if (vi->mergeable_rx_bufs)
 		dev->sysfs_rx_queue_group = &virtio_net_mrg_rx_group;
Signature · 1 line
2.19.1.6.gb485710b
author
#2 lore raw

PATCH v2 2/6 virtio-net: fix mismatch of getting tx-frames

In reply to Heng Qi

Since virtio-net allows switching napi_tx for per txq, we have to get the specific txq's result now.

Fixes 394bd87764b6 ("virtio_net: support per queue interrupt coalesce command")
Cc Gavin Li <[email protected]>
---
 drivers/net/virtio_net.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/virtio_net.c +1 -1
index 10878c9d430a..15bac303e3ff 100644
@@ -3453,7 +3453,7 @@ static int virtnet_get_per_queue_coalesce(struct net_device *dev,
 	} else {
 		ec->rx_max_coalesced_frames = 1;
 
-		if (vi->sq[0].napi.weight)
+		if (vi->sq[queue].napi.weight)
 			ec->tx_max_coalesced_frames = 1;
 	}
 
Signature · 1 line
2.19.1.6.gb485710b
author
#3 lore raw

PATCH v2 3/6 virtio-net: consistently save parameters for per-queue

In reply to Heng Qi

When using .set_coalesce interface to set all queue coalescing parameters, we need to update both per-queue and global save values.

Fixes 394bd87764b6 ("virtio_net: support per queue interrupt coalesce command")
Cc Gavin Li <[email protected]>
---
 drivers/net/virtio_net.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/virtio_net.c +9 -0
index 15bac303e3ff..6120dd5343dd 100644
@@ -3233,6 +3233,7 @@ static int virtnet_send_notf_coal_cmds(struct virtnet_info *vi,
 				       struct ethtool_coalesce *ec)
 {
 	struct scatterlist sgs_tx, sgs_rx;
+	int i;
 
 	vi->ctrl->coal_tx.tx_usecs = cpu_to_le32(ec->tx_coalesce_usecs);
 	vi->ctrl->coal_tx.tx_max_packets = cpu_to_le32(ec->tx_max_coalesced_frames);
@@ -3246,6 +3247,10 @@ static int virtnet_send_notf_coal_cmds(struct virtnet_info *vi,
 	/* Save parameters */
 	vi->intr_coal_tx.max_usecs = ec->tx_coalesce_usecs;
 	vi->intr_coal_tx.max_packets = ec->tx_max_coalesced_frames;
+	for (i = 0; i < vi->max_queue_pairs; i++) {
+		vi->sq[i].intr_coal.max_usecs = ec->tx_coalesce_usecs;
+		vi->sq[i].intr_coal.max_packets = ec->tx_max_coalesced_frames;
+	}
 
 	vi->ctrl->coal_rx.rx_usecs = cpu_to_le32(ec->rx_coalesce_usecs);
 	vi->ctrl->coal_rx.rx_max_packets = cpu_to_le32(ec->rx_max_coalesced_frames);
@@ -3259,6 +3264,10 @@ static int virtnet_send_notf_coal_cmds(struct virtnet_info *vi,
 	/* Save parameters */
 	vi->intr_coal_rx.max_usecs = ec->rx_coalesce_usecs;
 	vi->intr_coal_rx.max_packets = ec->rx_max_coalesced_frames;
+	for (i = 0; i < vi->max_queue_pairs; i++) {
+		vi->rq[i].intr_coal.max_usecs = ec->rx_coalesce_usecs;
+		vi->rq[i].intr_coal.max_packets = ec->rx_max_coalesced_frames;
+	}
 
 	return 0;
 }
Signature · 1 line
2.19.1.6.gb485710b
author
#4 lore raw

PATCH v2 4/6 virtio-net: fix per queue coalescing parameter setting

In reply to Heng Qi

When the user sets a non-zero coalescing parameter to 0 for a specific virtqueue, it does not work as expected, so let's fix this.

Fixes 394bd87764b6 ("virtio_net: support per queue interrupt coalesce command")
Cc Gavin Li <[email protected]>

---
v1->v2:
1. Remove useless comments.

 drivers/net/virtio_net.c | 36 ++++++++++++++++--------------------
 1 file changed, 16 insertions(+), 20 deletions(-)
drivers/net/virtio_net.c +16 -20
index 6120dd5343dd..12ec3ae19b60 100644
@@ -3296,27 +3296,23 @@ static int virtnet_send_notf_coal_vq_cmds(struct virtnet_info *vi,
 {
 	int err;
 
-	if (ec->rx_coalesce_usecs || ec->rx_max_coalesced_frames) {
-		err = virtnet_send_ctrl_coal_vq_cmd(vi, rxq2vq(queue),
-						    ec->rx_coalesce_usecs,
-						    ec->rx_max_coalesced_frames);
-		if (err)
-			return err;
-		/* Save parameters */
-		vi->rq[queue].intr_coal.max_usecs = ec->rx_coalesce_usecs;
-		vi->rq[queue].intr_coal.max_packets = ec->rx_max_coalesced_frames;
-	}
+	err = virtnet_send_ctrl_coal_vq_cmd(vi, rxq2vq(queue),
+					    ec->rx_coalesce_usecs,
+					    ec->rx_max_coalesced_frames);
+	if (err)
+		return err;
 
-	if (ec->tx_coalesce_usecs || ec->tx_max_coalesced_frames) {
-		err = virtnet_send_ctrl_coal_vq_cmd(vi, txq2vq(queue),
-						    ec->tx_coalesce_usecs,
-						    ec->tx_max_coalesced_frames);
-		if (err)
-			return err;
-		/* Save parameters */
-		vi->sq[queue].intr_coal.max_usecs = ec->tx_coalesce_usecs;
-		vi->sq[queue].intr_coal.max_packets = ec->tx_max_coalesced_frames;
-	}
+	vi->rq[queue].intr_coal.max_usecs = ec->rx_coalesce_usecs;
+	vi->rq[queue].intr_coal.max_packets = ec->rx_max_coalesced_frames;
+
+	err = virtnet_send_ctrl_coal_vq_cmd(vi, txq2vq(queue),
+					    ec->tx_coalesce_usecs,
+					    ec->tx_max_coalesced_frames);
+	if (err)
+		return err;
+
+	vi->sq[queue].intr_coal.max_usecs = ec->tx_coalesce_usecs;
+	vi->sq[queue].intr_coal.max_packets = ec->tx_max_coalesced_frames;
 
 	return 0;
 }
Signature · 1 line
2.19.1.6.gb485710b
Acked-by on v2 4/6
#5 lore raw

Re: [PATCH net v2 4/6] virtio-net: fix per queue coalescing parameter setting

Quoted 8 lines · Heng Qi

When the user sets a non-zero coalescing parameter to 0 for a specific virtqueue, it does not work as expected, so let's fix this.

Fixes 394bd87764b6 ("virtio_net: support per queue interrupt coalesce command")
Cc Gavin Li <[email protected]>

Thanks

Quoted 59 lines

---
v1->v2:
1. Remove useless comments.

 drivers/net/virtio_net.c | 36 ++++++++++++++++--------------------
 1 file changed, 16 insertions(+), 20 deletions(-)
drivers/net/virtio_net.c +16 -21
index 6120dd5343dd..12ec3ae19b60 100644
@@ -3296,27 +3296,23 @@ static int virtnet_send_notf_coal_vq_cmds(struct virtnet_info *vi,
 {
        int err;
 
-       if (ec->rx_coalesce_usecs || ec->rx_max_coalesced_frames) {
-               err = virtnet_send_ctrl_coal_vq_cmd(vi, rxq2vq(queue),
-                                                   ec->rx_coalesce_usecs,
-                                                   ec->rx_max_coalesced_frames);
-               if (err)
-                       return err;
-               /* Save parameters */
-               vi->rq[queue].intr_coal.max_usecs = ec->rx_coalesce_usecs;
-               vi->rq[queue].intr_coal.max_packets = ec->rx_max_coalesced_frames;
-       }
+       err = virtnet_send_ctrl_coal_vq_cmd(vi, rxq2vq(queue),
+                                           ec->rx_coalesce_usecs,
+                                           ec->rx_max_coalesced_frames);
+       if (err)
+               return err;
 
-       if (ec->tx_coalesce_usecs || ec->tx_max_coalesced_frames) {
-               err = virtnet_send_ctrl_coal_vq_cmd(vi, txq2vq(queue),
-                                                   ec->tx_coalesce_usecs,
-                                                   ec->tx_max_coalesced_frames);
-               if (err)
-                       return err;
-               /* Save parameters */
-               vi->sq[queue].intr_coal.max_usecs = ec->tx_coalesce_usecs;
-               vi->sq[queue].intr_coal.max_packets = ec->tx_max_coalesced_frames;
-       }
+       vi->rq[queue].intr_coal.max_usecs = ec->rx_coalesce_usecs;
+       vi->rq[queue].intr_coal.max_packets = ec->rx_max_coalesced_frames;
+
+       err = virtnet_send_ctrl_coal_vq_cmd(vi, txq2vq(queue),
+                                           ec->tx_coalesce_usecs,
+                                           ec->tx_max_coalesced_frames);
+       if (err)
+               return err;
+
+       vi->sq[queue].intr_coal.max_usecs = ec->tx_coalesce_usecs;
+       vi->sq[queue].intr_coal.max_packets = ec->tx_max_coalesced_frames;
 
        return 0;
 }
--

2.19.1.6.gb485710b

author
#6 lore raw

PATCH v2 5/6 virtio-net: fix the vq coalescing setting for vq resize

In reply to Heng Qi

According to the definition of virtqueue coalescing spec[1]:

Upon disabling and re-enabling a transmit virtqueue, the device MUST set the coalescing parameters of the virtqueue to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command, or, if the driver did not set any TX coalescing parameters, to 0.

Upon disabling and re-enabling a receive virtqueue, the device MUST set the coalescing parameters of the virtqueue to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_RX_SET command, or, if the driver did not set any RX coalescing parameters, to 0.

We need to add this setting for vq resize (ethtool -G) where vq_reset happens.

[1] https://lists.oasis-open.org/archives/virtio-dev/202303/msg00415.html

Fixes 394bd87764b6 ("virtio_net: support per queue interrupt coalesce command")
Cc Gavin Li <[email protected]>
---
 drivers/net/virtio_net.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
drivers/net/virtio_net.c +27 -0
index 12ec3ae19b60..cb19b224419b 100644
@@ -2855,6 +2855,9 @@ static void virtnet_get_ringparam(struct net_device *dev,
 	ring->tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
 }
 
+static int virtnet_send_ctrl_coal_vq_cmd(struct virtnet_info *vi,
+					 u16 vqn, u32 max_usecs, u32 max_packets);
+
 static int virtnet_set_ringparam(struct net_device *dev,
 				 struct ethtool_ringparam *ring,
 				 struct kernel_ethtool_ringparam *kernel_ring,
@@ -2890,12 +2893,36 @@ static int virtnet_set_ringparam(struct net_device *dev,
 			err = virtnet_tx_resize(vi, sq, ring->tx_pending);
 			if (err)
 				return err;
+
+			/* Upon disabling and re-enabling a transmit virtqueue, the device must
+			 * set the coalescing parameters of the virtqueue to those configured
+			 * through the VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command, or, if the driver
+			 * did not set any TX coalescing parameters, to 0.
+			 */
+			err = virtnet_send_ctrl_coal_vq_cmd(vi, txq2vq(i),
+							    vi->intr_coal_tx.max_usecs,
+							    vi->intr_coal_tx.max_packets);
+			if (err)
+				return err;
+			/* Save parameters */
+			vi->sq[i].intr_coal.max_usecs = vi->intr_coal_tx.max_usecs;
+			vi->sq[i].intr_coal.max_packets = vi->intr_coal_tx.max_packets;
 		}
 
 		if (ring->rx_pending != rx_pending) {
 			err = virtnet_rx_resize(vi, rq, ring->rx_pending);
 			if (err)
 				return err;
+
+			/* The reason is same as the transmit virtqueue reset */
+			err = virtnet_send_ctrl_coal_vq_cmd(vi, rxq2vq(i),
+							    vi->intr_coal_rx.max_usecs,
+							    vi->intr_coal_rx.max_packets);
+			if (err)
+				return err;
+			/* Save parameters */
+			vi->rq[i].intr_coal.max_usecs = vi->intr_coal_rx.max_usecs;
+			vi->rq[i].intr_coal.max_packets = vi->intr_coal_rx.max_packets;
 		}
 	}
 
Signature · 1 line
2.19.1.6.gb485710b
maintainer
#7 lore raw

Re: [PATCH net v2 5/6] virtio-net: fix the vq coalescing setting for vq resize

Quoted 12 lines · Heng Qi

According to the definition of virtqueue coalescing spec[1]:

Upon disabling and re-enabling a transmit virtqueue, the device MUST set the coalescing parameters of the virtqueue to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command, or, if the driver did not set any TX coalescing parameters, to 0.

Upon disabling and re-enabling a receive virtqueue, the device MUST set the coalescing parameters of the virtqueue to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_RX_SET command, or, if the driver did not set any RX coalescing parameters, to 0.

Quoted 7 lines

We need to add this setting for vq resize (ethtool -G) where vq_reset happens.

[1] https://lists.oasis-open.org/archives/virtio-dev/202303/msg00415.html

Fixes 394bd87764b6 ("virtio_net: support per queue interrupt coalesce command")
Cc Gavin Li <[email protected]>

@Jason, since you commented on v1, waiting for your ack.

Quoted 24 lines
---
 drivers/net/virtio_net.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
drivers/net/virtio_net.c +5 -0
index 12ec3ae19b60..cb19b224419b 100644
@@ -2855,6 +2855,9 @@ static void virtnet_get_ringparam(struct net_device *dev,
 	ring->tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
 }
 
+static int virtnet_send_ctrl_coal_vq_cmd(struct virtnet_info *vi,
+					 u16 vqn, u32 max_usecs, u32 max_packets);
+
 static int virtnet_set_ringparam(struct net_device *dev,
 				 struct ethtool_ringparam *ring,
 				 struct kernel_ethtool_ringparam *kernel_ring,
@@ -2890,12 +2893,36 @@ static int virtnet_set_ringparam(struct net_device *dev,
 			err = virtnet_tx_resize(vi, sq, ring->tx_pending);
 			if (err)
 				return err;
+
+			/* Upon disabling and re-enabling a transmit virtqueue, the device must

drivers/net/virtio_net.c static int virtnet_set_ringparam(struct net_device *dev, · line 2907

Quoted 10 lines
drivers/net/virtio_net.c +10 -0
+			 * set the coalescing parameters of the virtqueue to those configured
+			 * through the VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command, or, if the driver
+			 * did not set any TX coalescing parameters, to 0.
+			 */
+			err = virtnet_send_ctrl_coal_vq_cmd(vi, txq2vq(i),
+							    vi->intr_coal_tx.max_usecs,
+							    vi->intr_coal_tx.max_packets);
+			if (err)
+				return err;
+			/* Save parameters */

As a very minor nit, I guess the comment could be dropped here (similar to patch 4/6). @Heng Qi: please don't repost just for this, let's wait for Jason' comments.

Cheers,

Paolo

A message that is not in the archive (sent off-list or not yet received): [email protected]

Acked-by
#9 lore raw

Re: [PATCH net v2 5/6] virtio-net: fix the vq coalescing setting for vq resize

Quoted 64 lines · Heng Qi
Quoted 51 lines · Paolo Abeni
Quoted 12 lines · Heng Qi

According to the definition of virtqueue coalescing spec[1]:

Upon disabling and re-enabling a transmit virtqueue, the device MUST set the coalescing parameters of the virtqueue to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command, or, if the driver did not set any TX coalescing parameters, to 0.

Upon disabling and re-enabling a receive virtqueue, the device MUST set the coalescing parameters of the virtqueue to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_RX_SET command, or, if the driver did not set any RX coalescing parameters, to 0.

Quoted 7 lines

We need to add this setting for vq resize (ethtool -G) where vq_reset happens.

[1] https://lists.oasis-open.org/archives/virtio-dev/202303/msg00415.html

Fixes 394bd87764b6 ("virtio_net: support per queue interrupt coalesce command")
Cc Gavin Li <[email protected]>

@Jason, since you commented on v1, waiting for your ack.

Quoted 29 lines
---
  drivers/net/virtio_net.c | 27 +++++++++++++++++++++++++++
  1 file changed, 27 insertions(+)
drivers/net/virtio_net.c +10 -0
index 12ec3ae19b60..cb19b224419b 100644
@@ -2855,6 +2855,9 @@ static void virtnet_get_ringparam(struct net_device *dev,
     ring->tx_pending = virtqueue_get_vring_size(vi->sq[0].vq);
  }
 
+static int virtnet_send_ctrl_coal_vq_cmd(struct virtnet_info *vi,
+                                     u16 vqn, u32 max_usecs, u32 max_packets);
+
  static int virtnet_set_ringparam(struct net_device *dev,
                              struct ethtool_ringparam *ring,
                              struct kernel_ethtool_ringparam *kernel_ring,
@@ -2890,12 +2893,36 @@ static int virtnet_set_ringparam(struct net_device *dev,
                     err = virtnet_tx_resize(vi, sq, ring->tx_pending);
                     if (err)
                             return err;
+
+                    /* Upon disabling and re-enabling a transmit virtqueue, the device must
+                     * set the coalescing parameters of the virtqueue to those configured
+                     * through the VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command, or, if the driver
+                     * did not set any TX coalescing parameters, to 0.
+                     */
+                    err = virtnet_send_ctrl_coal_vq_cmd(vi, txq2vq(i),
Quoted 8 lines

drivers/net/virtio_net.c static int virtnet_set_ringparam(struct net_device *dev, · line 2907

Quoted 5 lines
drivers/net/virtio_net.c +5 -0
+                                                        vi->intr_coal_tx.max_usecs,
+                                                        vi->intr_coal_tx.max_packets);
+                    if (err)
+                            return err;
+                    /* Save parameters */

As a very minor nit, I guess the comment could be dropped here (similar to patch 4/6). @Heng Qi: please don't repost just for this, let's wait for Jason' comments.

Quoted 2 lines

Ok. We are currently on the National Day holiday, Jason may reply in a few days, thanks!

With the comments removed.

Thanks

Quoted 5 lines
Quoted 4 lines

Cheers,

Paolo

author
#10 lore raw

PATCH v2 6/6 virtio-net: a tiny comment update

In reply to Heng Qi

Update a comment because virtio-net now supports both VIRTIO_NET_F_NOTF_COAL and VIRTIO_NET_F_VQ_NOTF_COAL.

---
 drivers/net/virtio_net.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/virtio_net.c +1 -1
index cb19b224419b..4d746bb3d0be 100644
@@ -3347,7 +3347,7 @@ static int virtnet_send_notf_coal_vq_cmds(struct virtnet_info *vi,
 static int virtnet_coal_params_supported(struct ethtool_coalesce *ec)
 {
 	/* usecs coalescing is supported only if VIRTIO_NET_F_NOTF_COAL
-	 * feature is negotiated.
+	 * or VIRTIO_NET_F_VQ_NOTF_COAL feature is negotiated.
 	 */
 	if (ec->rx_coalesce_usecs || ec->tx_coalesce_usecs)
 		return -EOPNOTSUPP;
Signature · 1 line
2.19.1.6.gb485710b