quick thread in DLB2
Hello Abdullah,
In the DLB2 code, I see a thread is created for a single operation:
In drivers/event/dlb2/pf/base/dlb2_resource.c pthread_create(&pthread, NULL, &dlb2_pp_profile_func, &dlb2_thread_data[i]); and just after:pthread_join(pthread, NULL);
Can we avoid creating this thread?
I guess no, because it must spawn on a specific CPU.
Hi Thomas,
That's right we need to create threads on specific CPUs.
________________________________
From: Thomas Monjalon <[email protected]>
Sent: Friday, September 1, 2023 7:09 AM
To: Sevincer, Abdullah <[email protected]>
Cc: [email protected] <[email protected]>; Tyler Retzlaff <[email protected]>
Subject: quick thread in DLB2
Hello Abdullah,
In the DLB2 code, I see a thread is created for a single operation:
In drivers/event/dlb2/pf/base/dlb2_resource.c pthread_create(&pthread, NULL, &dlb2_pp_profile_func, &dlb2_thread_data[i]); and just after:pthread_join(pthread, NULL);
Can we avoid creating this thread?
I guess no, because it must spawn on a specific CPU.
Quoted 12 lines · On Fri, 01 Sep 2023 16:08:48 +0200 Thomas Monjalon <[email protected]> wrote:
Hello Abdullah,
In the DLB2 code, I see a thread is created for a single operation:
Indrivers/event/dlb2/pf/base/dlb2_resource.cpthread_create(&pthread, NULL, &dlb2_pp_profile_func, &dlb2_thread_data[i]); and just after:pthread_join(pthread, NULL);Can we avoid creating this thread?
I guess no, because it must spawn on a specific CPU.
The per thread data seems to break lots of expectations in EAL. It all seems to be about capturing the number of cycles on different cores. Looks like a mess.
Hi Stephen,
It is probing ports for best CPU. Yes it collects cycles. We may rework in the future. Open to suggestions.
-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Wednesday, September 6, 2023 12:45 PM
To: Thomas Monjalon <[email protected]>
Cc: Sevincer, Abdullah <[email protected]>; [email protected]; Tyler Retzlaff <[email protected]>
Subject: Re: quick thread in DLB2
Quoted 12 lines · On Fri, 01 Sep 2023 16:08:48 +0200 Thomas Monjalon <[email protected]> wrote:
Hello Abdullah,
In the DLB2 code, I see a thread is created for a single operation:
Indrivers/event/dlb2/pf/base/dlb2_resource.cpthread_create(&pthread, NULL, &dlb2_pp_profile_func, &dlb2_thread_data[i]); and just after:pthread_join(pthread, NULL);Can we avoid creating this thread?
I guess no, because it must spawn on a specific CPU.
The per thread data seems to break lots of expectations in EAL. It all seems to be about capturing the number of cycles on different cores. Looks like a mess.
Quoted 3 lines · Abdullah
Hi Stephen,
It is probing ports for best CPU. Yes it collects cycles. We may rework in the future. Open to suggestions.
Why is this device a special snowflake? All threading should be using the existing infrastructure of service and control threads. Doing it this way looks like a special case optimization for testpmd. What if the user is using a multi-stage pipeline, flow rules, secondary processes, rings etc. Look at some of the real applications using DPDK. And talk to some of the network equipment vendors.
Quoted 2 lines · Abdullah
Hi Stephen,
It is probing ports for best CPU. Yes it collects cycles. We may rework in the future.
Best, in what sense? Is this some kind of topology exploration? One DLB port being closer to (cheaper to access for) certain cores?
Quoted 28 lines
Open to suggestions.
-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Wednesday, September 6, 2023 12:45 PM
To: Thomas Monjalon <[email protected]>
Cc: Sevincer, Abdullah <[email protected]>; [email protected]; Tyler Retzlaff <[email protected]>
Subject: Re: quick thread in DLB2Quoted 12 lines · On Fri, 01 Sep 2023 16:08:48 +0200 Thomas Monjalon <[email protected]> wrote:
Hello Abdullah,
In the DLB2 code, I see a thread is created for a single operation:
Indrivers/event/dlb2/pf/base/dlb2_resource.cpthread_create(&pthread, NULL, &dlb2_pp_profile_func, &dlb2_thread_data[i]); and just after:pthread_join(pthread, NULL);Can we avoid creating this thread?
I guess no, because it must spawn on a specific CPU.The per thread data seems to break lots of expectations in EAL. It all seems to be about capturing the number of cycles on different cores. Looks like a mess.
Mattias,
Yes that’s correct.
-----Original Message-----
From: Mattias Rönnblom <[email protected]>
Sent: Friday, September 8, 2023 12:28 AM
To: Sevincer, Abdullah <[email protected]>; Stephen Hemminger <[email protected]>; Thomas Monjalon <[email protected]>
Cc: [email protected]; Tyler Retzlaff <[email protected]>
Subject: Re: quick thread in DLB2
Quoted 2 lines · Abdullah
Hi Stephen,
It is probing ports for best CPU. Yes it collects cycles. We may rework in the future.
Best, in what sense? Is this some kind of topology exploration? One DLB port being closer to (cheaper to access for) certain cores?
Quoted 29 lines
Open to suggestions.
-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Wednesday, September 6, 2023 12:45 PM
To: Thomas Monjalon <[email protected]>
Cc: Sevincer, Abdullah <[email protected]>; [email protected]; Tyler Retzlaff <[email protected]>
Subject: Re: quick thread in DLB2Quoted 12 lines · On Fri, 01 Sep 2023 16:08:48 +0200 Thomas Monjalon <[email protected]> wrote:
Hello Abdullah,
In the DLB2 code, I see a thread is created for a single operation:
Indrivers/event/dlb2/pf/base/dlb2_resource.cpthread_create(&pthread, NULL, &dlb2_pp_profile_func, &dlb2_thread_data[i]); and just after:pthread_join(pthread, NULL);Can we avoid creating this thread?
I guess no, because it must spawn on a specific CPU.The per thread data seems to break lots of expectations in EAL. It all seems to be about capturing the number of cycles on different cores. Looks like a mess.
Quoted 4 lines · Abdullah
Mattias,
Yes that’s correct.
There is no way to cleaner and more robust way to achieve the same result? For example, by accessing /proc, or better, an DPDK abstraction of the same.
Quoted 42 lines
-----Original Message-----
From: Mattias Rönnblom <[email protected]>
Sent: Friday, September 8, 2023 12:28 AM
To: Sevincer, Abdullah <[email protected]>; Stephen Hemminger <[email protected]>; Thomas Monjalon <[email protected]>
Cc: [email protected]; Tyler Retzlaff <[email protected]>
Subject: Re: quick thread in DLB2Quoted 2 lines · Abdullah
Hi Stephen,
It is probing ports for best CPU. Yes it collects cycles. We may rework in the future.Best, in what sense? Is this some kind of topology exploration? One DLB port being closer to (cheaper to access for) certain cores?
Quoted 29 lines
Open to suggestions.
-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Wednesday, September 6, 2023 12:45 PM
To: Thomas Monjalon <[email protected]>
Cc: Sevincer, Abdullah <[email protected]>; [email protected]; Tyler Retzlaff <[email protected]>
Subject: Re: quick thread in DLB2Quoted 12 lines · On Fri, 01 Sep 2023 16:08:48 +0200 Thomas Monjalon <[email protected]> wrote:
Hello Abdullah,
In the DLB2 code, I see a thread is created for a single operation:
Indrivers/event/dlb2/pf/base/dlb2_resource.cpthread_create(&pthread, NULL, &dlb2_pp_profile_func, &dlb2_thread_data[i]); and just after:pthread_join(pthread, NULL);Can we avoid creating this thread?
I guess no, because it must spawn on a specific CPU.The per thread data seems to break lots of expectations in EAL. It all seems to be about capturing the number of cycles on different cores. Looks like a mess.
Quoted 10 lines
-----Original Message-----
From: Mattias Rönnblom <[email protected]>
Sent: Wednesday, September 13, 2023 10:48 AM
To: Sevincer, Abdullah <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]
Cc: [email protected]; Tyler Retzlaff <[email protected]>
Subject: Re: quick thread in DLB2
Quoted 8 lines
Quoted 4 lines · Abdullah
Mattias,
Yes that’s correct.There is no way to cleaner and more robust way to achieve the same result? For example, by accessing /proc, or better, an DPDK abstraction of the same.
There similar issues in other areas. For ex: the CPUs with large core count have larger interconnect. The SLC to CPU distance starts to matter and the memory latency increases. The distance of the cores on the interconnect also impacts lock behaviors. We probably need a common mechanism/library to export such details. Not sure how much of this would be a security risk.
Quoted 47 lines
Quoted 10 lines
-----Original Message-----
From: Mattias Rönnblom <[email protected]>
Sent: Friday, September 8, 2023 12:28 AM
To: Sevincer, Abdullah <[email protected]>; Stephen Hemminger <[email protected]>; Thomas Monjalon <[email protected]>
Cc: [email protected]; Tyler Retzlaff <[email protected]>
Subject: Re: quick thread in DLB2Quoted 3 lines
Quoted 2 lines · Abdullah
Hi Stephen,
It is probing ports for best CPU. Yes it collects cycles. We may rework in thefuture.
Quoted 2 lines
Best, in what sense? Is this some kind of topology exploration? One DLB
port being closer to (cheaper to access for) certain cores?
Quoted 30 lines
Quoted 30 lines
Open to suggestions.
-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Wednesday, September 6, 2023 12:45 PM
To: Thomas Monjalon <[email protected]>
Cc: Sevincer, Abdullah <[email protected]>; [email protected]; Tyler Retzlaff <[email protected]>
Subject: Re: quick thread in DLB2Quoted 12 lines · On Fri, 01 Sep 2023 16:08:48 +0200 Thomas Monjalon <[email protected]> wrote:
Hello Abdullah,
In the DLB2 code, I see a thread is created for a single operation:
Indrivers/event/dlb2/pf/base/dlb2_resource.cpthread_create(&pthread, NULL, &dlb2_pp_profile_func, &dlb2_thread_data[i]); and just after:pthread_join(pthread, NULL);Can we avoid creating this thread?
I guess no, because it must spawn on a specific CPU.The per thread data seems to break lots of expectations in EAL. It all seems to be about capturing the number of cycles on different cores. Looks like a mess.
Quoted 11 lines · Honnappa Nagarahalli
Quoted 11 lines
-----Original Message-----
From: Mattias Rönnblom <[email protected]>
Sent: Wednesday, September 13, 2023 10:48 AM
To: Sevincer, Abdullah <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]
Cc: [email protected]; Tyler Retzlaff <[email protected]>
Subject: Re: quick thread in DLB2On 2023-09-11 16:28, Sevincer, Abdullah wrote:
Quoted 8 lines
Quoted 7 lines
Quoted 4 lines
Mattias,
Yes that’s correct.There is no way to cleaner and more robust way to achieve the same result? For example, by accessing /proc, or better, an DPDK abstraction of the same.
There similar issues in other areas. For ex: the CPUs with large core count have larger interconnect. The SLC to CPU distance starts to matter and the memory latency increases. The distance of the cores on the interconnect also impacts lock behaviors. We probably need a common mechanism/library to export such details.
To make DSW (and other work schedulers) work better on systems with SMT, it would be useful to know which lcores are hardware thread siblings.
Topology related to CPU core capacity in heterogeneous system (e.g., big.LITTLE) could be used for similar purposes.
The list goes on but one wouldn't need to address all use cases in the v1 API.
Something like hwloc(7), but DPDK native.
Quoted 2 lines
Not sure how much of this would be a security risk.
What do have in mind? The DPDK library has no more privileges than the application running on top of it. As far as I see, what we are talking about here is mere convenience and portability, from a security point of view.
Quoted 47 lines
Quoted 47 lines
Quoted 10 lines
-----Original Message-----
From: Mattias Rönnblom <[email protected]>
Sent: Friday, September 8, 2023 12:28 AM
To: Sevincer, Abdullah <[email protected]>; Stephen Hemminger <[email protected]>; Thomas Monjalon <[email protected]>
Cc: [email protected]; Tyler Retzlaff <[email protected]>
Subject: Re: quick thread in DLB2Quoted 3 lines
Quoted 2 lines · Abdullah
Hi Stephen,
It is probing ports for best CPU. Yes it collects cycles. We may rework in thefuture.
Quoted 2 lines
Best, in what sense? Is this some kind of topology exploration? One DLB
port being closer to (cheaper to access for) certain cores?
Quoted 30 lines
Quoted 30 lines
Open to suggestions.
-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Wednesday, September 6, 2023 12:45 PM
To: Thomas Monjalon <[email protected]>
Cc: Sevincer, Abdullah <[email protected]>; [email protected]; Tyler Retzlaff <[email protected]>
Subject: Re: quick thread in DLB2Quoted 12 lines · On Fri, 01 Sep 2023 16:08:48 +0200 Thomas Monjalon <[email protected]> wrote:
Hello Abdullah,
In the DLB2 code, I see a thread is created for a single operation:
Indrivers/event/dlb2/pf/base/dlb2_resource.cpthread_create(&pthread, NULL, &dlb2_pp_profile_func, &dlb2_thread_data[i]); and just after:pthread_join(pthread, NULL);Can we avoid creating this thread?
I guess no, because it must spawn on a specific CPU.The per thread data seems to break lots of expectations in EAL. It all seems to be about capturing the number of cycles on different cores. Looks like a mess.