Skip to contentSkip to conversation

Windows DPDK real-time priority threads causing thread starvation

dpdk-dev [email protected] 1 participant 4 messages Dec 13, 2020 – Apr 1, 2021

lore
Share & tools
lore

[email protected]

author
#0 lore raw

https://bugs.dpdk.org/show_bug.cgi?id=600

            Bug ID: 600
           Summary: Windows DPDK real-time priority threads causing thread
                    starvation
           Product: DPDK
           Version: 20.08
          Hardware: All
                OS: Windows
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: core
          Assignee: [email protected]
          Reporter: [email protected]

Target Milestone: ---

During our verification tests on Windows DPDK we've noticed that DPDK polling threads, which run in REALTIME_PRIORITY_CLASS are causing starvation to other threads from the OS which need to change affinity and run in lower priority.

While running an application for a while we see the OS thread waits for 2:30 minutes and raises a bugcheck, see below example of such flow:

1) DPDK thread running on core-0 in real-time high priority(24) polling mode.
2) The thread is blocking the system function NtSetSystemInformation (ExpUpdateTimerConfiguration) in another thread from
switching to core-0 via KeSetSystemGroupAffinityThread since the calling
thread is priority 15.
3) NtSetSystemInformation exclusively acquired system-wide lock (ExpTimeRefreshLock) hence
it blocks other threads (e.g. calling NtQuerySystemInformation).

Call stack of the issue (from windbg):
# RetAddr : Args to Child
: Call Site
00 fffff8024f43abba : fffff8024f80bd20 fffffe0d9d18e838 fffffe0d00010224 fffff802`4f564050 : nt!ExpWaitForResource+0x1576d2
01 fffff8024fabbd66 : fffffe0d9d18eb80 0000000000000000 000000000000002c 00000040`738fc2f0 : nt!ExAcquireResourceExclusiveLite+0x18a
02 fffff8024f9bc3e9 : 0000000000000000 0000004000028000 0000017edc3be000 00000000`00000000 : nt!ExAcquireTimeRefreshLock+0x22
03 fffff8024f9bb83b : 0000000000000000 0000000000002001 0000017edc3a7d90 00000000`00000000 : nt!ExpQuerySystemInformation+0xa89
04 fffff8024f5ce605 : ffffae0f258382c0 0000017edc3699d0 fffffe0d9d18eb80 ffffae0f`2c96c2e0 : nt!NtQuerySystemInformation+0x2b
05 00007ffab0befe24 : 00007ffaacd300d1 00007ffa9fa5d858 0000017edbb3e130 00000000`0000004a : nt!KiSystemServiceCopyEnd+0x25
06 00007ffaacd300d1 : 00007ffa9fa5d858 0000017edbb3e130 000000000000004a 00007ffa`9f9d8269 : ntdll!NtQuerySystemInformation+0x14
07 00007ffa8a35dcc4 : 0000017edc5534d4 0000000000000000 0000000000000000 0000017e`dc3bd110 : KERNELBASE!GetTimeZoneInformation+0x51
08 00007ffa9e019abc : 0000017edc3814f0 00000040738fc670 00007ffa9e209540 0000017e`dc3814f0 : framedynos!CInstance::SetDateTime+0x264
09 00007ffa9e01a2f5 : 0000017edc3bd110 0000000000000000 00007ffa9e209540 0000017edc3814f0 : cimwin32!Process::LoadCheapPropertiesNT+0x45c 0a 00007ffa8a3559a2 : 0000000000000000 0000017edc380b70 0000000000000000 0000017edc383410 : cimwin32!Process::Enumerate+0x155
0b 00007ffa8a356f9d : 0000000000000000 00000040738fde09 0000017edc380b70 0000017e`00000000 : framedynos!Provider::ExecuteQuery+0x92
0c 00007ff6337527f5 : 0000017edbb3a380 0000017edbad5690 0000017edba750f8 0000017edbb3a370 : framedynos!CWbemProviderGlue::ExecQueryAsync+0x21d 0d 00007ff633752473 : 0000000000000000 0000000000000000 0000017edba75010 0000000000001000 :
wmiprvse!CInterceptor_IWbemSyncProvider::Helper_ExecQueryAsync+0x2b5 0e 00007ffaaee07853 : 0000017e00000000 0000017edbae5cc8 0000017edc5504c8 00000000`00000000 :
wmiprvse!CInterceptor_IWbemSyncProvider::ExecQueryAsync+0x143

Signature · 2 lines
You are receiving this mail because:
You are the assignee for the bug.
author
#1 lore raw

https://bugs.dpdk.org/show_bug.cgi?id=600

Tal Shnaiderman ([email protected]) changed:

What |Removed |Added
----------------------------------------------------------------------------

             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |FIXED
                 CC|                            |[email protected]

--- Comment #1 from Tal Shnaiderman ([email protected]) --- This was fixed in
http://patchwork.dpdk.org/project/dpdk/patch/[email protected]/

By lowering default thread priority

Signature · 2 lines
You are receiving this mail because:
You are the assignee for the bug.
author
#2 lore raw
In reply to [email protected]

https://bugs.dpdk.org/show_bug.cgi?id=600

[email protected] ([email protected]) changed:

What |Removed |Added
----------------------------------------------------------------------------

     Ever confirmed|0                           |1
             Status|RESOLVED                    |CONFIRMED
         Resolution|FIXED                       |---

--- Comment #2 from [email protected] ([email protected]) --- Fix tested and approved

Signature · 2 lines
You are receiving this mail because:
You are the assignee for the bug.
author
#3 lore raw
In reply to [email protected]

https://bugs.dpdk.org/show_bug.cgi?id=600

Thomas Monjalon ([email protected]) changed:

What |Removed |Added
----------------------------------------------------------------------------

             Status|CONFIRMED                   |RESOLVED
         Resolution|---                         |FIXED

--- Comment #3 from Thomas Monjalon ([email protected]) --- Resolved in http://git.dpdk.org/dpdk/commit/?id=16afcbfa30

Signature · 2 lines
You are receiving this mail because:
You are the assignee for the bug.