driver/pci: reduce the single block time in pci_read_config
PATCH 1/1 driver/pci: reduce the single block time in pci_read_config
From: Jiang Biao <[email protected]>
pci_read_config() could block several ms in kernel space, mainly caused by the while loop to call pci_user_read_config_dword(). Singel pci_user_read_config_dword() loop could consume 130us+,
| pci_user_read_config_dword() {
| _raw_spin_lock_irq() {
! 136.698 us | native_queued_spin_lock_slowpath();
! 137.582 us | }
| pci_read() {
| raw_pci_read() {
| pci_conf1_read() {
0.230 us | _raw_spin_lock_irqsave();
0.035 us | _raw_spin_unlock_irqrestore();
8.476 us | }
8.790 us | }
9.091 us | }
! 147.263 us | }and dozens of the loop could consume ms+.
If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.
Add scheduling chance in the loop to improve the latency.
--- drivers/pci/pci-sysfs.c | 1 + 1 file changed, 1 insertion(+)
drivers/pci/pci-sysfs.c +1 -0
@@ -708,6 +708,7 @@ static ssize_t pci_read_config(struct file *filp, struct kobject *kobj, data[off - init_off +3 ] = (val >>24 ) &0xff ; off +=4 ; size -=4 ; +cond_resched (); }if (size >=2 ) {
Signature · 1 line
1.8.3.1
kindly ping :)
Quoted 47 lines · Jiang Biao
From: Jiang Biao <[email protected]>
pci_read_config()could block several ms in kernel space, mainly caused by the while loop to callpci_user_read_config_dword(). Singelpci_user_read_config_dword()loop could consume 130us+,| pci_user_read_config_dword() { | _raw_spin_lock_irq() { ! 136.698 us | native_queued_spin_lock_slowpath(); ! 137.582 us | } | pci_read() { | raw_pci_read() { | pci_conf1_read() { 0.230 us | _raw_spin_lock_irqsave(); 0.035 us | _raw_spin_unlock_irqrestore(); 8.476 us | } 8.790 us | } 9.091 us | } ! 147.263 us | }and dozens of the loop could consume ms+.
If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.
Add scheduling chance in the loop to improve the latency.
Reported-by Bin Lai <[email protected]>Signed-off-by Jiang Biao <[email protected]>--- drivers/pci/pci-sysfs.c | 1 + 1 file changed, 1 insertion(+)drivers/pci/pci-sysfs.c +1
-1@@ -708,6 +708,7 @@ static ssize_t pci_read_config(struct file *filp, struct kobject *kobj, data[off - init_off +3 ] = (val >>24 ) &0xff ; off +=4 ; size -=4 ; +cond_resched (); }if (size >=2 ) { --1.8.3.1
Quoted 18 lines · Jiang Biao
From: Jiang Biao <[email protected]>
pci_read_config()could block several ms in kernel space, mainly caused by the while loop to callpci_user_read_config_dword(). Singelpci_user_read_config_dword()loop could consume 130us+,| pci_user_read_config_dword() { | _raw_spin_lock_irq() { ! 136.698 us | native_queued_spin_lock_slowpath(); ! 137.582 us | } | pci_read() { | raw_pci_read() { | pci_conf1_read() { 0.230 us | _raw_spin_lock_irqsave(); 0.035 us | _raw_spin_unlock_irqrestore(); 8.476 us | } 8.790 us | } 9.091 us | } ! 147.263 us | }
Quoted 6 lines
and dozens of the loop could consume ms+.
If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.
Add scheduling chance in the loop to improve the latency.
Thanks for the patch, this makes a lot of sense.
Shouldn't we do the same in pci_write_config()?
Quoted 21 lines
Reported-by Bin Lai <[email protected]>Signed-off-by Jiang Biao <[email protected]>--- drivers/pci/pci-sysfs.c | 1 + 1 file changed, 1 insertion(+)drivers/pci/pci-sysfs.c +1
-0@@ -708,6 +708,7 @@ static ssize_t pci_read_config(struct file *filp, struct kobject *kobj, data[off - init_off +3 ] = (val >>24 ) &0xff ; off +=4 ; size -=4 ; +cond_resched (); }if (size >=2 ) {Signature · 1 line
1.8.3.1
Hi,
Quoted 22 lines · Bjorn Helgaas
Quoted 20 lines · Jiang Biao
From: Jiang Biao <[email protected]>
pci_read_config()could block several ms in kernel space, mainly caused by the while loop to callpci_user_read_config_dword(). Singelpci_user_read_config_dword()loop could consume 130us+,| pci_user_read_config_dword() { | _raw_spin_lock_irq() { ! 136.698 us | native_queued_spin_lock_slowpath(); ! 137.582 us | } | pci_read() { | raw_pci_read() { | pci_conf1_read() { 0.230 us | _raw_spin_lock_irqsave(); 0.035 us | _raw_spin_unlock_irqrestore(); 8.476 us | } 8.790 us | } 9.091 us | } ! 147.263 us | }and dozens of the loop could consume ms+.
Quoted 8 lines
Quoted 4 lines
If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.
Add scheduling chance in the loop to improve the latency.
Thanks for the patch, this makes a lot of sense.
Shouldn't we do the same in
pci_write_config()?
Yes, IMHO, that could be helpful too.
I'll send v2 to include that. :)
Thanks a lot for your comment.
Regards,
Jiang
Quoted 28 lines · Jiang Biao
Hi,
Quoted 25 lines · Bjorn Helgaas
Quoted 23 lines · Jiang Biao
From: Jiang Biao <[email protected]>
pci_read_config()could block several ms in kernel space, mainly caused by the while loop to callpci_user_read_config_dword(). Singelpci_user_read_config_dword()loop could consume 130us+,| pci_user_read_config_dword() { | _raw_spin_lock_irq() { ! 136.698 us | native_queued_spin_lock_slowpath(); ! 137.582 us | } | pci_read() { | raw_pci_read() { | pci_conf1_read() { 0.230 us | _raw_spin_lock_irqsave(); 0.035 us | _raw_spin_unlock_irqrestore(); 8.476 us | } 8.790 us | } 9.091 us | } ! 147.263 us | }and dozens of the loop could consume ms+.
If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.
Quoted 6 lines
Quoted 5 lines
Quoted 1 line
Add scheduling chance in the loop to improve the latency.
Thanks for the patch, this makes a lot of sense.
Shouldn't we do the same in
pci_write_config()?Yes, IMHO, that could be helpful too.
If it's feasible, it would be nice to actually verify that it makes a difference. I know config writes should be faster than reads, but they're certainly not as fast as a CPU can pump out data, so there must be *some* mechanism that slows the CPU down.
Bjorn
Hi,
Quoted 34 lines · Bjorn Helgaas
Quoted 32 lines · Jiang Biao
Hi,
Quoted 29 lines · Bjorn Helgaas
Quoted 18 lines · Jiang Biao
From: Jiang Biao <[email protected]>
pci_read_config()could block several ms in kernel space, mainly caused by the while loop to callpci_user_read_config_dword(). Singelpci_user_read_config_dword()loop could consume 130us+,| pci_user_read_config_dword() { | _raw_spin_lock_irq() { ! 136.698 us | native_queued_spin_lock_slowpath(); ! 137.582 us | } | pci_read() { | raw_pci_read() { | pci_conf1_read() { 0.230 us | _raw_spin_lock_irqsave(); 0.035 us | _raw_spin_unlock_irqrestore(); 8.476 us | } 8.790 us | } 9.091 us | } ! 147.263 us | }Quoted 6 lines
and dozens of the loop could consume ms+.
If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.
Add scheduling chance in the loop to improve the latency.
Thanks for the patch, this makes a lot of sense.
Quoted 7 lines
Quoted 2 lines
Quoted 1 line
Shouldn't we do the same in
pci_write_config()?Yes, IMHO, that could be helpful too.
If it's feasible, it would be nice to actually verify that it makes a difference. I know config writes should be faster than reads, but they're certainly not as fast as a CPU can pump out data, so there must be *some* mechanism that slows the CPU down.
We did catch 5ms+ latency caused by pci_read_config() triggered by concurrent lspcis, and latency disappeared after this patch.
For pci_write_config path, we have not met the case actually.:) I'll have some tries to verify that, and would send another patch if verified.
Thx.
Regards,
Jiang
Hi, Bjorn
Quoted 35 lines · Bjorn Helgaas
Quoted 33 lines · Jiang Biao
Hi,
Quoted 30 lines · Bjorn Helgaas
Quoted 18 lines · Jiang Biao
From: Jiang Biao <[email protected]>
pci_read_config()could block several ms in kernel space, mainly caused by the while loop to callpci_user_read_config_dword(). Singelpci_user_read_config_dword()loop could consume 130us+,| pci_user_read_config_dword() { | _raw_spin_lock_irq() { ! 136.698 us | native_queued_spin_lock_slowpath(); ! 137.582 us | } | pci_read() { | raw_pci_read() { | pci_conf1_read() { 0.230 us | _raw_spin_lock_irqsave(); 0.035 us | _raw_spin_unlock_irqrestore(); 8.476 us | } 8.790 us | } 9.091 us | } ! 147.263 us | }Quoted 6 lines
and dozens of the loop could consume ms+.
If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.
Add scheduling chance in the loop to improve the latency.
Thanks for the patch, this makes a lot of sense.
Shouldn't we do the same in
pci_write_config()?
Quoted 8 lines
Quoted 1 line
Yes, IMHO, that could be helpful too.
If it's feasible, it would be nice to actually verify that it makes a difference. I know config writes should be faster than reads, but they're certainly not as fast as a CPU can pump out data, so there must be *some* mechanism that slows the CPU down.
Bjorn
We failed to build a test case to produce the latency by setpci command, AFAIU, setpci could be much less frequently realistically used than lspci. So, the latency from pci_write_config() path could not be verified for now, could we apply this patch alone to erase the verified latency introduced by pci_read_config() path? :)
Thanks a lot.
Regards,
Jiang
Quoted 40 lines · Jiang Biao
Quoted 30 lines · Bjorn Helgaas
Quoted 29 lines · Jiang Biao
Quoted 28 lines · Bjorn Helgaas
Quoted 18 lines · Jiang Biao
From: Jiang Biao <[email protected]>
pci_read_config()could block several ms in kernel space, mainly caused by the while loop to callpci_user_read_config_dword(). Singelpci_user_read_config_dword()loop could consume 130us+,| pci_user_read_config_dword() { | _raw_spin_lock_irq() { ! 136.698 us | native_queued_spin_lock_slowpath(); ! 137.582 us | } | pci_read() { | raw_pci_read() { | pci_conf1_read() { 0.230 us | _raw_spin_lock_irqsave(); 0.035 us | _raw_spin_unlock_irqrestore(); 8.476 us | } 8.790 us | } 9.091 us | } ! 147.263 us | }Quoted 6 lines
and dozens of the loop could consume ms+.
If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.
Add scheduling chance in the loop to improve the latency.
Thanks for the patch, this makes a lot of sense.
Quoted 9 lines
Quoted 3 lines
Quoted 1 line
Shouldn't we do the same in
pci_write_config()?Yes, IMHO, that could be helpful too.
If it's feasible, it would be nice to actually verify that it makes a difference. I know config writes should be faster than reads, but they're certainly not as fast as a CPU can pump out data, so there must be *some* mechanism that slows the CPU down.
Quoted 5 lines
We failed to build a test case to produce the latency by setpci command, AFAIU, setpci could be much less frequently realistically used than lspci. So, the latency from
pci_write_config()path could not be verified for now, could we apply this patch alone to erase the verified latency introduced bypci_read_config()path? :)
Thanks for trying! I'll apply the patch as-is. I'd like to include a note in the commit log about the user-visible effect of this. I looked through recent similar commits:
928da37a229f ("RDMA/umem: Add a schedule point in ib_umem_get()") 47aaabdedf36 ("fanotify: Avoid softlockups when reading many events") 9f47eb5461aa ("fs/btrfs: Add cond_resched() for try_release_extent_mapping() stalls") 0a3b3c253a1e ("mm/mmap.c: Add cond_resched() for exit_mmap() CPU stalls") b7e3debdd040 ("mm/memory_hotplug.c: fix false softlockup during pfn range removal") d35bd764e689 ("dm writecache: add cond_resched to loop in persistent_memory_claim()") da97f2d56bbd ("mm: call cond_resched() from deferred_init_memmap()") ab8b65be1831 ("KVM: PPC: Book3S: Fix some RCU-list locks")
48c963e31bc6 ("KVM: arm/arm64: Release kvm->mmu_lock in loop to prevent starvation") e84fe99b68ce ("mm/page_alloc: fix watchdog soft lockups during set_zone_contiguous()") 4005f5c3c9d0 ("wireguard: send/receive: cond_resched() when processing worker ringbuffers") 3fd44c86711f ("io_uring: use cond_resched() in io_ring_ctx_wait_and_kill()") 7979457b1d3a ("net: bridge: vlan: Add a schedule point during VLAN processing") 2ed6edd33a21 ("perf: Add cond_resched() to task_function_call()") 1edaa447d958 ("dm writecache: add cond_resched to avoid CPU hangs") ce9a4186f9ac ("macvlan: add cond_resched() during multicast processing") 7be1b9b8e9d1 ("drm/mm: Break long searches in fragmented address spaces") bb699a793110 ("drm/i915/gem: Break up long lists of object reclaim") 9424ef56e13a ("ext4: add cond_resched() to __ext4_find_entry()")
and many of them mention softlockups, RCU CPU stall warnings, or watchdogs triggering. Are you seeing one of those, or are you measuring latency some other way?
Bjorn
Hi,
Quoted 42 lines · Bjorn Helgaas
Quoted 40 lines · Jiang Biao
Quoted 39 lines · Bjorn Helgaas
Quoted 25 lines · Jiang Biao
Quoted 24 lines · Bjorn Helgaas
Quoted 23 lines · Jiang Biao
From: Jiang Biao <[email protected]>
pci_read_config()could block several ms in kernel space, mainly caused by the while loop to callpci_user_read_config_dword(). Singelpci_user_read_config_dword()loop could consume 130us+,| pci_user_read_config_dword() { | _raw_spin_lock_irq() { ! 136.698 us | native_queued_spin_lock_slowpath(); ! 137.582 us | } | pci_read() { | raw_pci_read() { | pci_conf1_read() { 0.230 us | _raw_spin_lock_irqsave(); 0.035 us | _raw_spin_unlock_irqrestore(); 8.476 us | } 8.790 us | } 9.091 us | } ! 147.263 us | }and dozens of the loop could consume ms+.
If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Add scheduling chance in the loop to improve the latency.
Thanks for the patch, this makes a lot of sense.
Shouldn't we do the same in
pci_write_config()?Yes, IMHO, that could be helpful too.
If it's feasible, it would be nice to actually verify that it makes a difference. I know config writes should be faster than reads, but they're certainly not as fast as a CPU can pump out data, so there must be *some* mechanism that slows the CPU down.
Quoted 7 lines
Quoted 5 lines
We failed to build a test case to produce the latency by setpci command, AFAIU, setpci could be much less frequently realistically used than lspci. So, the latency from
pci_write_config()path could not be verified for now, could we apply this patch alone to erase the verified latency introduced bypci_read_config()path? :)Thanks for trying! I'll apply the patch as-is. I'd like to include a
Thanks. :)
Quoted 23 lines
note in the commit log about the user-visible effect of this. I looked through recent similar commits:
928da37a229f("RDMA/umem: Add a schedule point inib_umem_get()")47aaabdedf36("fanotify: Avoid softlockups when reading many events")9f47eb5461aa("fs/btrfs: Addcond_resched()fortry_release_extent_mapping()stalls")0a3b3c253a1e("mm/mmap.c: Addcond_resched()forexit_mmap()CPU stalls")b7e3debdd040("mm/memory_hotplug.c: fix false softlockup during pfn range removal")d35bd764e689("dm writecache: addcond_reschedto loop inpersistent_memory_claim()")da97f2d56bbd("mm: callcond_resched()fromdeferred_init_memmap()")ab8b65be1831("KVM: PPC: Book3S: Fix some RCU-list locks")
48c963e31bc6("KVM: arm/arm64: Releasekvm->mmu_lockin loop to prevent starvation")e84fe99b68ce("mm/page_alloc: fix watchdog soft lockups duringset_zone_contiguous()")4005f5c3c9d0("wireguard: send/receive:cond_resched()when processing worker ringbuffers")3fd44c86711f("io_uring: usecond_resched()inio_ring_ctx_wait_and_kill()")7979457b1d3a("net: bridge: vlan: Add a schedule point during VLAN processing")2ed6edd33a21("perf: Addcond_resched()totask_function_call()")1edaa447d958("dm writecache: addcond_reschedto avoid CPU hangs")ce9a4186f9ac("macvlan: addcond_resched()during multicast processing")7be1b9b8e9d1("drm/mm: Break long searches in fragmented address spaces")bb699a793110("drm/i915/gem: Break up long lists of object reclaim")9424ef56e13a("ext4: addcond_resched()to__ext4_find_entry()")
Quoted 2 lines
and many of them mention softlockups, RCU CPU stall warnings, or watchdogs triggering. Are you seeing one of those, or are you
No softlockup or RCU stall warnings.
Quoted 1 line
measuring latency some other way?
We test the scheduling latency by cyclictest benchmark, the max latency could be more than 5ms without this patch. The ftrace log shows pci_read_config is the main cause, and the 5ms+ latency disappeared with this patch applied.
Thanks a lot.
Regards,
Jiang
Bjorn Helgaas
AppliedQuoted 20 lines · Jiang Biao
From: Jiang Biao <[email protected]>
pci_read_config()could block several ms in kernel space, mainly caused by the while loop to callpci_user_read_config_dword(). Singelpci_user_read_config_dword()loop could consume 130us+,| pci_user_read_config_dword() { | _raw_spin_lock_irq() { ! 136.698 us | native_queued_spin_lock_slowpath(); ! 137.582 us | } | pci_read() { | raw_pci_read() { | pci_conf1_read() { 0.230 us | _raw_spin_lock_irqsave(); 0.035 us | _raw_spin_unlock_irqrestore(); 8.476 us | } 8.790 us | } 9.091 us | } ! 147.263 us | }and dozens of the loop could consume ms+.
Quoted 7 lines
If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.
Add scheduling chance in the loop to improve the latency.
Reported-by Bin Lai <[email protected]>Signed-off-by Jiang Biao <[email protected]>
Applied to pci/enumeration for v5.10, thanks!
Quoted 19 lines
--- drivers/pci/pci-sysfs.c | 1 + 1 file changed, 1 insertion(+)drivers/pci/pci-sysfs.c +1
-0@@ -708,6 +708,7 @@ static ssize_t pci_read_config(struct file *filp, struct kobject *kobj, data[off - init_off +3 ] = (val >>24 ) &0xff ; off +=4 ; size -=4 ; +cond_resched (); }if (size >=2 ) {Signature · 1 line
1.8.3.1