build: enable packet data prefetch
Subsystems (2)
PATCH 1/1 build: enable packet data prefetch
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtualized data paths like virtio utilized this feature for acceleration. Since most modern cpus have support prefetch function, we can enable packet data prefetch as default.
config/meson.build +3 -0
@@ -109,6 +109,9 @@ if not is_windows add_project_link_arguments('-Wl,--no-as-needed', language: 'c') endif +# do prefetch of packet data +dpdk_conf.set('RTE_PMD_PACKET_PREFETCH', 1) + # use pthreads if available for the platform if not is_windows add_project_link_arguments('-pthread', language: 'c')
Signature · 1 line
2.17.1
Quoted 12 lines · On Tue, 22 Sep 2020 16:21:35 +0800 Marvin Liu <[email protected]> wrote:
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtualized data paths like virtio utilized this feature for acceleration. Since most modern cpus have support prefetch function, we can enable packet data prefetch as default.
Signed-off-by Marvin Liu <[email protected]>config/meson.build +0
-0
config/meson.build if not is_windows · line 117
Quoted 9 lines
config/meson.build +3
-0@@ -109,6 +109,9 @@ if not is_windows add_project_link_arguments('-Wl,--no-as-needed', language: 'c') endif +# do prefetch of packet data +dpdk_conf.set('RTE_PMD_PACKET_PREFETCH', 1) + # use pthreads if available for the platform if not is_windows add_project_link_arguments('-pthread', language: 'c')
With meson, the project has been using rte_config.h for this.
Quoted 17 lines
-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Tuesday, September 22, 2020 10:12 PM
To: Liu, Yong <[email protected]>
Cc: Richardson, Bruce <[email protected]>; [email protected]
Subject: Re: [dpdk-dev] [PATCH] build: enable packet data prefetchQuoted 5 lines · On Tue, 22 Sep 2020 16:21:35 +0800 Marvin Liu <[email protected]> wrote:
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtualized data paths like virtio utilized this feature for acceleration. Since most modern cpus have support prefetch function, we can enable packet data prefetch as default.
Quoted 4 lines
Quoted 2 lines
Signed-off-by Marvin Liu <[email protected]>With meson, the project has been using
rte_config.hfor this.
Thanks a lot, will send v2 for the change.
Regards,
Marvin
PATCH v2 1/1 config: enable packet data prefetch
In reply to Marvin LiuData prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtualized data paths like virtio utilized this feature for acceleration. Since most modern cpus have support prefetch function, we can enable packet data prefetch as default.
---
v2: move define from meson.build to rte_config.h
--- config/rte_config.h | 1 + 1 file changed, 1 insertion(+)
config/rte_config.h +1 -0
@@ -101,6 +101,7 @@#define RTE_LIBRTE_GRAPH_STATS 1/****** driver defines ********/ +#define RTE_PMD_PACKET_PREFETCH 1/* QuickAssist device */ /* Max. number of QuickAssist devices which can be attached */
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH v2] config: enable packet data prefetch
23/09/2020 03:51, Marvin Liu:
Quoted 8 lines
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtualized data paths like virtio utilized this feature for acceleration. Since most modern cpus have support prefetch function, we can enable packet data prefetch as default.
Signed-off-by Marvin Liu <[email protected]>---
+#defineRTE_PMD_PACKET_PREFETCH1
We could also remove the related #ifdefs.
What can be the drawback of always enable those prefetches?
Re: [dpdk-dev] [PATCH v2] config: enable packet data prefetch
Quoted 19 lines
-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Thursday, October 15, 2020 6:03 AM
To: Liu, Yong <[email protected]>
Cc: Richardson, Bruce <[email protected]>; [email protected]; [email protected]; [email protected]; Yigit, Ferruh <[email protected]>; [email protected]; [email protected]; David Christensen <[email protected]>; [email protected]
Subject: Re: [dpdk-dev] [PATCH v2] config: enable packet data prefetch23/09/2020 03:51, Marvin Liu:
Quoted 5 lines
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtualized data paths like virtio utilized this feature for acceleration. Since most modern cpus have support prefetch function, we can enable packet data prefetch as default.
Quoted 8 lines
Quoted 3 lines
Signed-off-by Marvin Liu <[email protected]>---
+#defineRTE_PMD_PACKET_PREFETCH1We could also remove the related #ifdefs.
What can be the drawback of always enable those prefetches?
Hi Thomas,
I think the potential drawback is that current prefetch location cannot guarantee the best performance across different platforms. Each developer has tuned the performance by adding prefetch instruction and verified the result on himself platform. So prefetch location is based on certain platform, also it will be hard for developer to compare the results across platforms.
Thanks,
Marvin
Re: [dpdk-dev] [PATCH v2] config: enable packet data prefetch
<snip>
Quoted 11 lines
Quoted 11 lines
23/09/2020 03:51, Marvin Liu:
Quoted 8 lines
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtualized data paths like virtio utilized this feature for acceleration. Since most modern cpus have support prefetch function, we can enable packet data prefetch as default.
Signed-off-by Marvin Liu <[email protected]>---
+#defineRTE_PMD_PACKET_PREFETCH1
Quoted 8 lines
Quoted 4 lines
We could also remove the related #ifdefs.
What can be the drawback of always enable those prefetches?
Hi Thomas,
I think the potential drawback is that current prefetch location cannot guarantee the best performance across different platforms.
Then, does it make sense to enable this by default?
Quoted 7 lines
Each developer has tuned the performance by adding prefetch instruction and verified the result on himself platform.
So prefetch location is based on certain platform, also it will be hard for developer to compare the results across platforms.Thanks,
Marvin
Re: [dpdk-dev] [PATCH v2] config: enable packet data prefetch
Quoted 31 lines
-----Original Message-----
From: Honnappa Nagarahalli <[email protected]>
Sent: Thursday, October 15, 2020 12:10 PM
To: Liu, Yong <[email protected]>; [email protected]
Cc: Richardson, Bruce <[email protected]>; [email protected]; [email protected]; [email protected]; Yigit, Ferruh <[email protected]>; [email protected]; David Christensen <[email protected]>; Ruifeng Wang <[email protected]>; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; nd <[email protected]>
Subject: RE: [dpdk-dev] [PATCH v2] config: enable packet data prefetch<snip>
Quoted 6 lines
Quoted 6 lines
23/09/2020 03:51, Marvin Liu:
Quoted 4 lines
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtualized data paths like virtio utilized this feature for acceleration. Since most modern cpus have support prefetch function, we can enable packet data prefetch as
default.
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Signed-off-by Marvin Liu <[email protected]>---
+#defineRTE_PMD_PACKET_PREFETCH1We could also remove the related #ifdefs.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
What can be the drawback of always enable those prefetches?
Hi Thomas,
I think the potential drawback is that current prefetch location cannot guarantee the best performance across different platforms.Then, does it make sense to enable this by default?
Now most of prefetch actions are placed after pointer of data is valid. I think this methodology can benefit all platforms. It's hard to say that it’s the best choice for all. But no more better solution in my mind.
At least, we need to allow user to enable packet data prefetch.
Regards,
Marvin
Quoted 7 lines
Quoted 7 lines
Each developer has tuned the performance by adding prefetch instruction and verified the result on himself platform.
So prefetch location is based on certain platform, also it will be hard for developer to compare the results across platforms.Thanks,
Marvin
Re: [dpdk-dev] [PATCH v2] config: enable packet data prefetch
15/10/2020 10:23, Liu, Yong:
Quoted 20 lines
From: Honnappa Nagarahalli <[email protected]>
Quoted 19 lines
Quoted 5 lines
Quoted 5 lines
23/09/2020 03:51, Marvin Liu:
Quoted 4 lines
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtualized data paths like virtio utilized this feature for acceleration. Since most modern cpus have support prefetch function, we can enable packet data prefetch as
default.
Quoted 13 lines
Quoted 9 lines
Quoted 4 lines
Signed-off-by Marvin Liu <[email protected]>---
+#defineRTE_PMD_PACKET_PREFETCH1We could also remove the related #ifdefs.
What can be the drawback of always enable those prefetches?
Hi Thomas,
I think the potential drawback is that current prefetch location cannot guarantee the best performance across different platforms.
Quoted 6 lines
Quoted 2 lines
Then, does it make sense to enable this by default?
Now most of prefetch actions are placed after pointer of data is valid. I think this methodology can benefit all platforms. It's hard to say that it’s the best choice for all. But no more better solution in my mind.
At least, we need to allow user to enable packet data prefetch.
In my opinion, it can be tested and measured.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Each developer has tuned the performance by adding prefetch instruction and verified the result on himself platform.
So prefetch location is based on certain platform, also it will be hard for developer to compare the results across platforms.
If it shows benefit on an architecture, then it should be enabled with #ifdef RTE_ARCH_XX
I am for removing the option RTE_PMD_PACKET_PREFETCH.
Re: [dpdk-dev] [PATCH v2] config: enable packet data prefetch
<snip>
Quoted 25 lines
15/10/2020 10:23, Liu, Yong:
Quoted 24 lines
From: Honnappa Nagarahalli <[email protected]>
Quoted 22 lines
Quoted 6 lines
Quoted 6 lines
23/09/2020 03:51, Marvin Liu:
Quoted 5 lines
Data prefetch instruction can preload data into cpu’s hierarchical cache before data access. Virtualized data paths like virtio utilized this feature for acceleration. Since most modern cpus have support prefetch function, we can enable packet data prefetch as
default.
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Signed-off-by Marvin Liu <[email protected]>---
+#defineRTE_PMD_PACKET_PREFETCH1Quoted 8 lines
Quoted 4 lines
We could also remove the related #ifdefs.
What can be the drawback of always enable those prefetches?
Hi Thomas,
I think the potential drawback is that current prefetch location cannot guarantee the best performance across different platforms.Then, does it make sense to enable this by default?
Quoted 7 lines
Quoted 1 line
Now most of prefetch actions are placed after pointer of data is valid. I
think this methodology can benefit all platforms.
Quoted 1 line
It's hard to say that it’s the best choice for all. But no more better solution
in my mind.
Quoted 1 line
At least, we need to allow user to enable packet data prefetch.
In my opinion, it can be tested and measured.
+ Joyce, to test this for VirtIO on Arm
Quoted 12 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Each developer has tuned the performance by adding prefetch instruction and verified the result on himself platform. So prefetch location is based on certain platform, also it will be hard for developer to compare the results across platforms.
If it shows benefit on an architecture, then it should be enabled with #ifdef
RTE_ARCH_XXI am for removing the option
RTE_PMD_PACKET_PREFETCH.
PATCH v3 1/1 drivers/net: enable packet data prefetch on x86
In reply to Marvin LiuData prefetch instruction can preload data into cpu’s hierarchical cache before data access. Data paths like e1000 and virtio utilized this feature for packet data access acceleration. Enabled packet data prefetch on x86 architecture, as prefetch instruction has been supported from very early generation.
---
v3: replace build config with pre-defined architecture macro
v2: move define from meson.build to rte_config.h
---
drivers/net/e1000/em_rxtx.c +1 -1
@@ -185,7 +185,7 @@ struct em_tx_queue {#define rte_em_prefetch (p) do {} while(0)#endif -#ifdef RTE_PMD_PACKET_PREFETCH +#if defined(RTE_ARCH_X86 )#define rte_packet_prefetch (p) rte_prefetch1(p)#else #define rte_packet_prefetch (p) do {} while(0)
drivers/net/e1000/igb_rxtx.c +1 -1
@@ -198,7 +198,7 @@ struct igb_tx_queue {#define rte_igb_prefetch (p) do {} while(0)#endif -#ifdef RTE_PMD_PACKET_PREFETCH +#if defined(RTE_ARCH_X86 )#define rte_packet_prefetch (p) rte_prefetch1(p)#else #define rte_packet_prefetch (p) do {} while(0)
drivers/net/enic/enic_rxtx.c +1 -1
@@ -25,7 +25,7 @@#define rte_enic_prefetch (p) do {} while (0)#endif -#ifdef RTE_PMD_PACKET_PREFETCH +#if defined(RTE_ARCH_X86 )#define rte_packet_prefetch (p) rte_prefetch1(p)#else #define rte_packet_prefetch (p) do {} while (0)
drivers/net/fm10k/fm10k_rxtx.c +1 -1
@@ -10,7 +10,7 @@#include "fm10k.h" #include "base/fm10k_type.h" -#ifdef RTE_PMD_PACKET_PREFETCH +#if defined(RTE_ARCH_X86 )#define rte_packet_prefetch (p) rte_prefetch1(p)#else #define rte_packet_prefetch (p) do {} while (0)
drivers/net/igc/igc_txrx.c +1 -1
@@ -16,7 +16,7 @@#define rte_igc_prefetch (p) do {} while (0)#endif -#ifdef RTE_PMD_PACKET_PREFETCH +#if defined(RTE_ARCH_X86 )#define rte_packet_prefetch (p) rte_prefetch1(p)#else #define rte_packet_prefetch (p) do {} while (0)
drivers/net/ixgbe/ixgbe_rxtx.h +1 -1
@@ -41,7 +41,7 @@#define RX_RING_SZ ((IXGBE_MAX_RING_DESC + RTE_PMD_IXGBE_RX_MAX_BURST) * \ sizeof(union ixgbe_adv_rx_desc)) -#ifdef RTE_PMD_PACKET_PREFETCH +#if defined(RTE_ARCH_X86 )#define rte_packet_prefetch (p) rte_prefetch1(p)#else #define rte_packet_prefetch (p) do {} while(0)
drivers/net/virtio/virtqueue.h +1 -1
@@ -106,7 +106,7 @@ virtqueue_store_flags_packed(struct vring_packed_desc *dp, dp->flags = flags; } } -#ifdef RTE_PMD_PACKET_PREFETCH +#if defined(RTE_ARCH_X86 )#define rte_packet_prefetch (p) rte_prefetch1(p)#else #define rte_packet_prefetch (p) do {} while(0)
Signature · 1 line
2.17.1