net/virtio: enable packet data prefetch on x86
PATCH 1/1 net/virtio: enable packet data prefetch on x86
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtio datapath utilized this feature for data access acceleration. As config RTE_PMD_PACKET_PREFETCH was discarded, now packet data prefetch is enabled based on architecture.
drivers/net/virtio/virtqueue.h +1 -1
@@ -106,7 +106,7 @@ virtqueue_store_flags_packed(struct vring_packed_desc *dp, dp->flags = flags; } } -#ifdef RTE_PMD_PACKET_PREFETCH +#if defined(RTE_ARCH_X86 )#define rte_packet_prefetch (p) rte_prefetch1(p)#else #define rte_packet_prefetch (p) do {} while(0)
Signature · 1 line
2.17.1
Quoted 5 lines · Marvin Liu
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtio datapath utilized this feature for data access acceleration. As config
RTE_PMD_PACKET_PREFETCHwas discarded, now packet data prefetch is enabled based on architecture.
IIUC, this config item was set for all architectures under make compilation.
$ git grep RTE_PMD_PACKET_PREFETCH v20.08 config/ v20.08:config/common_base:CONFIG_RTE_PMD_PACKET_PREFETCH=y
Now that we switched to meson, it got lost and this patch only restores it for x86.
Can other architectures check this?
Thanks.
Signature · 1 line
David Marchand
Quoted 7 lines · David Marchand
Quoted 5 lines · Marvin Liu
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtio datapath utilized this feature for data access acceleration. As config
RTE_PMD_PACKET_PREFETCHwas discarded, now packet data prefetch is enabled based on architecture.
Quoted 9 lines
IIUC, this config item was set for all architectures under make compilation.
$ git grep RTE_PMD_PACKET_PREFETCH v20.08 config/ v20.08:config/common_base:CONFIG_RTE_PMD_PACKET_PREFETCH=yNow that we switched to meson, it got lost and this patch only restores it for x86.
Can other architectures check this?
If it was globally enabled before, it probably should just be added to config/rte_config.h file.
/Bruce
Quoted 14 lines · Bruce Richardson
Quoted 12 lines · David Marchand
Quoted 5 lines · Marvin Liu
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtio datapath utilized this feature for data access acceleration. As config
RTE_PMD_PACKET_PREFETCHwas discarded, now packet data prefetch is enabled based on architecture.IIUC, this config item was set for all architectures under make compilation.
$ git grep RTE_PMD_PACKET_PREFETCH v20.08 config/ v20.08:config/common_base:CONFIG_RTE_PMD_PACKET_PREFETCH=y
Quoted 7 lines
Quoted 4 lines
Now that we switched to meson, it got lost and this patch only restores it for x86.
Can other architectures check this?If it was globally enabled before, it probably should just be added to
config/rte_config.hfile.
Restoring globally is the probably the best fix, yes.
I am still surprised nobody but x86 testers caught a perf regression.
Signature · 1 line
David marchand
Hi Marvin,
Quoted 13 lines · Marvin Liu
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtio datapath utilized this feature for data access acceleration. As config
RTE_PMD_PACKET_PREFETCHwas discarded, now packet data prefetch is enabled based on architecture.Signed-off-by Marvin Liu <[email protected]>drivers/net/virtio/virtqueue.h +0
-0
drivers/net/virtio/virtqueue.h virtqueue_store_flags_packed(struct vring_packed_desc *dp, · line 112
Quoted 8 lines
drivers/net/virtio/virtqueue.h +1
-1@@ -106,7 +106,7 @@ virtqueue_store_flags_packed(struct vring_packed_desc *dp, dp->flags = flags; } } -#ifdef RTE_PMD_PACKET_PREFETCH +#if defined(RTE_ARCH_X86 )#define rte_packet_prefetch (p) rte_prefetch1(p)#else #define rte_packet_prefetch (p) do {} while(0)
Thanks for catching this issue.
I agree it should be re-enabled by default, and not only on X86, not only on Virtio PMD.
AFAICS, prefetch was enabled for all platforms before the switch to Meson, so I see it as an involuntary change that needs to be reverted.
Then, I think having a possibility to disable it would be nice, so maybe we should add an option in our build system to do that.
Thanks,
Maxime
Quoted 27 lines · Maxime Coquelin
Quoted 13 lines · Marvin Liu
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtio datapath utilized this feature for data access acceleration. As config
RTE_PMD_PACKET_PREFETCHwas discarded, now packet data prefetch is enabled based on architecture.Signed-off-by Marvin Liu <[email protected]>drivers/net/virtio/virtqueue.h +0
-0drivers/net/virtio/virtqueue.h virtqueue_store_flags_packed(struct vring_packed_desc *dp, · line 112
Quoted 8 lines
drivers/net/virtio/virtqueue.h +1
-1@@ -106,7 +106,7 @@ virtqueue_store_flags_packed(struct vring_packed_desc *dp, dp->flags = flags; } } -#ifdef RTE_PMD_PACKET_PREFETCH +#if defined(RTE_ARCH_X86 )#define rte_packet_prefetch (p) rte_prefetch1(p)#else #define rte_packet_prefetch (p) do {} while(0)Thanks for catching this issue.
I agree it should be re-enabled by default, and not only on X86, not only on Virtio PMD.
Quoted 2 lines
AFAICS, prefetch was enabled for all platforms before the switch to Meson, so I see it as an involuntary change that needs to be reverted.
Yes and this will also solve https://patchwork.dpdk.org/patch/83468/.
Thanks.
Signature · 1 line
David Marchand
Quoted 43 lines
-----Original Message-----
From: David Marchand <[email protected]>
Sent: Thursday, November 12, 2020 4:58 PM
To: Liu, Yong <[email protected]>; Maxime Coquelin <[email protected]>
Cc: Xia, Chenbo <[email protected]>; dev <[email protected]>; Richardson, Bruce <[email protected]>
Subject: Re: [dpdk-dev] [PATCH] net/virtio: enable packet data prefetch on x86Quoted 9 lines · Maxime Coquelin
Review
Quoted 8 lines · Marvin Liu
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtio datapath utilized this feature for data access acceleration. As config
RTE_PMD_PACKET_PREFETCHwas discarded, now packet data prefetch is enabled based on architecture.Signed-off-by Marvin Liu <[email protected]>(quoted diff) +0
-0
b/drivers/net/virtio/virtqueue.hQuoted 4 lines
drivers/net/virtio/virtqueue.h virtqueue_store_flags_packed(struct
Quoted 4 lines
index
42c4c9882..0196290a5100644drivers/net/virtio/virtqueue.h +0
-0@@ -106,7 +106,7 @@ virtqueue_store_flags_packed(struct
vring_packed_desc*dp,Quoted 14 lines
drivers/net/virtio/virtqueue.h virtqueue_store_flags_packed(struct vring_packed_desc *dp, · line 112
Quoted 9 lines
drivers/net/virtio/virtqueue.h +1
-1dp->flags = flags; } } -#ifdef RTE_PMD_PACKET_PREFETCH +#if defined(RTE_ARCH_X86 )#define rte_packet_prefetch (p) rte_prefetch1(p)#else #define rte_packet_prefetch (p) do {} while(0)Thanks for catching this issue.
I agree it should be re-enabled by default, and not only on X86, not only on Virtio PMD.
Quoted 6 lines
Quoted 2 lines
AFAICS, prefetch was enabled for all platforms before the switch to Meson, so I see it as an involuntary change that needs to be reverted.
Yes and this will also solve https://patchwork.dpdk.org/patch/83468/.
Thanks.
Agreed, original patch was intended to recover prefetch configuration in meson build.
Please check http://patchwork.dpdk.org/patch/78451/.
And it leaded a discussion about how to utilize prefetch function optimally.
Due to no conclusion for current position is best for other platforms except x86, now only enable prefetch in virtio + x86.
Quoted 2 lines
--
David Marchand
Quoted 8 lines · Yong
Quoted 3 lines
Yes and this will also solve https://patchwork.dpdk.org/patch/83468/.
Thanks.Agreed, original patch was intended to recover prefetch configuration in meson build.
Please check http://patchwork.dpdk.org/patch/78451/.
And it leaded a discussion about how to utilize prefetch function optimally.
Due to no conclusion for current position is best for other platforms except x86, now only enable prefetch in virtio + x86.
I disagree.
No conclusion means the best is to restore the previous state, i.e. enable this option for all platforms.
If later other architectures want to change this, this can revisit.
Signature · 1 line
David Marchand
+ more people into this conversation. IMHO, restore to previous state is the best choice by now.
Quoted 32 lines
-----Original Message-----
From: David Marchand <[email protected]>
Sent: Friday, November 13, 2020 3:27 PM
To: Liu, Yong <[email protected]>
Cc: Maxime Coquelin <[email protected]>; Xia, Chenbo <[email protected]>; dev <[email protected]>; Richardson, Bruce <[email protected]>
Subject: Re: [dpdk-dev] [PATCH] net/virtio: enable packet data prefetch on x86Quoted 5 lines · Yong
Quoted 3 lines
Yes and this will also solve https://patchwork.dpdk.org/patch/83468/.
Thanks.Agreed, original patch was intended to recover prefetch configuration in
meson build.
Quoted 2 lines
Please check http://patchwork.dpdk.org/patch/78451/.
And it leaded a discussion about how to utilize prefetch functionoptimally.
Quoted 1 line
Due to no conclusion for current position is best for other platforms
except x86, now only enable prefetch in virtio + x86.
I disagree.
No conclusion means the best is to restore the previous state, i.e. enable this option for all platforms.If later other architectures want to change this, this can revisit.
--
David Marchand