Skip to contentSkip to conversation

driver/pci: reduce the single block time in pci_read_config

PATCH Applied Jiang Biao 2 participants 10 messages Aug 24, 2020 – Sep 17, 2020 PCI

lore
Share & tools
lore

Jiang Biao

author
#0 lore raw

PATCH 1/1 driver/pci: reduce the single block time in pci_read_config

From: Jiang Biao <[email protected]>

pci_read_config() could block several ms in kernel space, mainly caused by the while loop to call pci_user_read_config_dword(). Singel pci_user_read_config_dword() loop could consume 130us+,

              |    pci_user_read_config_dword() {
              |      _raw_spin_lock_irq() {
! 136.698 us  |        native_queued_spin_lock_slowpath();
! 137.582 us  |      }
              |      pci_read() {
              |        raw_pci_read() {
              |          pci_conf1_read() {
  0.230 us    |            _raw_spin_lock_irqsave();
  0.035 us    |            _raw_spin_unlock_irqrestore();
  8.476 us    |          }
  8.790 us    |        }
  9.091 us    |      }
! 147.263 us  |    }

and dozens of the loop could consume ms+.

If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.

Add scheduling chance in the loop to improve the latency.

---
 drivers/pci/pci-sysfs.c | 1 +
 1 file changed, 1 insertion(+)
drivers/pci/pci-sysfs.c +1 -0
index 6d78df9..3b9f63d 100644
@@ -708,6 +708,7 @@ static ssize_t pci_read_config(struct file *filp, struct kobject *kobj,
 		data[off - init_off + 3] = (val >> 24) & 0xff;
 		off += 4;
 		size -= 4;
+		cond_resched();
 	}
 
 	if (size >= 2) {
Signature · 1 line
1.8.3.1
author
#1 lore raw

kindly ping :)

Quoted 47 lines · Jiang Biao

From: Jiang Biao <[email protected]>

pci_read_config() could block several ms in kernel space, mainly caused by the while loop to call pci_user_read_config_dword(). Singel pci_user_read_config_dword() loop could consume 130us+,

              |    pci_user_read_config_dword() {
              |      _raw_spin_lock_irq() {
! 136.698 us  |        native_queued_spin_lock_slowpath();
! 137.582 us  |      }
              |      pci_read() {
              |        raw_pci_read() {
              |          pci_conf1_read() {
  0.230 us    |            _raw_spin_lock_irqsave();
  0.035 us    |            _raw_spin_unlock_irqrestore();
  8.476 us    |          }
  8.790 us    |        }
  9.091 us    |      }
! 147.263 us  |    }

and dozens of the loop could consume ms+.

If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.

Add scheduling chance in the loop to improve the latency.

---
 drivers/pci/pci-sysfs.c | 1 +
 1 file changed, 1 insertion(+)
drivers/pci/pci-sysfs.c +1 -1
index 6d78df9..3b9f63d 100644
@@ -708,6 +708,7 @@ static ssize_t pci_read_config(struct file *filp, struct kobject *kobj,
                data[off - init_off + 3] = (val >> 24) & 0xff;
                off += 4;
                size -= 4;
+               cond_resched();
        }
 
        if (size >= 2) {
--

1.8.3.1

#2 lore raw
In reply to Jiang Biao
Quoted 18 lines · Jiang Biao

From: Jiang Biao <[email protected]>

pci_read_config() could block several ms in kernel space, mainly caused by the while loop to call pci_user_read_config_dword(). Singel pci_user_read_config_dword() loop could consume 130us+,

              |    pci_user_read_config_dword() {
              |      _raw_spin_lock_irq() {
! 136.698 us  |        native_queued_spin_lock_slowpath();
! 137.582 us  |      }
              |      pci_read() {
              |        raw_pci_read() {
              |          pci_conf1_read() {
  0.230 us    |            _raw_spin_lock_irqsave();
  0.035 us    |            _raw_spin_unlock_irqrestore();
  8.476 us    |          }
  8.790 us    |        }
  9.091 us    |      }
! 147.263 us  |    }
Quoted 6 lines

and dozens of the loop could consume ms+.

If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.

Add scheduling chance in the loop to improve the latency.

Thanks for the patch, this makes a lot of sense.

Shouldn't we do the same in pci_write_config()?

Quoted 21 lines
---
 drivers/pci/pci-sysfs.c | 1 +
 1 file changed, 1 insertion(+)
drivers/pci/pci-sysfs.c +1 -0
index 6d78df9..3b9f63d 100644
@@ -708,6 +708,7 @@ static ssize_t pci_read_config(struct file *filp, struct kobject *kobj,
 		data[off - init_off + 3] = (val >> 24) & 0xff;
 		off += 4;
 		size -= 4;
+		cond_resched();
 	}
 
 	if (size >= 2) {
Signature · 1 line
1.8.3.1
author
#3 lore raw

Hi,

Quoted 22 lines · Bjorn Helgaas
Quoted 20 lines · Jiang Biao

From: Jiang Biao <[email protected]>

pci_read_config() could block several ms in kernel space, mainly caused by the while loop to call pci_user_read_config_dword(). Singel pci_user_read_config_dword() loop could consume 130us+,

              |    pci_user_read_config_dword() {
              |      _raw_spin_lock_irq() {
! 136.698 us  |        native_queued_spin_lock_slowpath();
! 137.582 us  |      }
              |      pci_read() {
              |        raw_pci_read() {
              |          pci_conf1_read() {
  0.230 us    |            _raw_spin_lock_irqsave();
  0.035 us    |            _raw_spin_unlock_irqrestore();
  8.476 us    |          }
  8.790 us    |        }
  9.091 us    |      }
! 147.263 us  |    }

and dozens of the loop could consume ms+.

Quoted 8 lines
Quoted 4 lines

If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.

Add scheduling chance in the loop to improve the latency.

Thanks for the patch, this makes a lot of sense.

Shouldn't we do the same in pci_write_config()?

Yes, IMHO, that could be helpful too.
I'll send v2 to include that. :)
Thanks a lot for your comment.

Regards,
Jiang

#4 lore raw
Quoted 28 lines · Jiang Biao

Hi,

Quoted 25 lines · Bjorn Helgaas
Quoted 23 lines · Jiang Biao

From: Jiang Biao <[email protected]>

pci_read_config() could block several ms in kernel space, mainly caused by the while loop to call pci_user_read_config_dword(). Singel pci_user_read_config_dword() loop could consume 130us+,

              |    pci_user_read_config_dword() {
              |      _raw_spin_lock_irq() {
! 136.698 us  |        native_queued_spin_lock_slowpath();
! 137.582 us  |      }
              |      pci_read() {
              |        raw_pci_read() {
              |          pci_conf1_read() {
  0.230 us    |            _raw_spin_lock_irqsave();
  0.035 us    |            _raw_spin_unlock_irqrestore();
  8.476 us    |          }
  8.790 us    |        }
  9.091 us    |      }
! 147.263 us  |    }

and dozens of the loop could consume ms+.

If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.

Quoted 6 lines
Quoted 5 lines
Quoted 1 line

Add scheduling chance in the loop to improve the latency.

Thanks for the patch, this makes a lot of sense.

Shouldn't we do the same in pci_write_config()?

Yes, IMHO, that could be helpful too.

If it's feasible, it would be nice to actually verify that it makes a difference. I know config writes should be faster than reads, but they're certainly not as fast as a CPU can pump out data, so there must be *some* mechanism that slows the CPU down.

Bjorn

author
#5 lore raw

Hi,

Quoted 34 lines · Bjorn Helgaas
Quoted 32 lines · Jiang Biao

Hi,

Quoted 29 lines · Bjorn Helgaas
Quoted 18 lines · Jiang Biao

From: Jiang Biao <[email protected]>

pci_read_config() could block several ms in kernel space, mainly caused by the while loop to call pci_user_read_config_dword(). Singel pci_user_read_config_dword() loop could consume 130us+,

              |    pci_user_read_config_dword() {
              |      _raw_spin_lock_irq() {
! 136.698 us  |        native_queued_spin_lock_slowpath();
! 137.582 us  |      }
              |      pci_read() {
              |        raw_pci_read() {
              |          pci_conf1_read() {
  0.230 us    |            _raw_spin_lock_irqsave();
  0.035 us    |            _raw_spin_unlock_irqrestore();
  8.476 us    |          }
  8.790 us    |        }
  9.091 us    |      }
! 147.263 us  |    }
Quoted 6 lines

and dozens of the loop could consume ms+.

If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.

Add scheduling chance in the loop to improve the latency.

Thanks for the patch, this makes a lot of sense.

Quoted 7 lines
Quoted 2 lines
Quoted 1 line

Shouldn't we do the same in pci_write_config()?

Yes, IMHO, that could be helpful too.

If it's feasible, it would be nice to actually verify that it makes a difference. I know config writes should be faster than reads, but they're certainly not as fast as a CPU can pump out data, so there must be *some* mechanism that slows the CPU down.

We did catch 5ms+ latency caused by pci_read_config() triggered by concurrent lspcis, and latency disappeared after this patch.
For pci_write_config path, we have not met the case actually.:) I'll have some tries to verify that, and would send another patch if verified.

Thx.
Regards,
Jiang

author
#6 lore raw
In reply to Bjorn Helgaas

Hi, Bjorn

Quoted 35 lines · Bjorn Helgaas
Quoted 33 lines · Jiang Biao

Hi,

Quoted 30 lines · Bjorn Helgaas
Quoted 18 lines · Jiang Biao

From: Jiang Biao <[email protected]>

pci_read_config() could block several ms in kernel space, mainly caused by the while loop to call pci_user_read_config_dword(). Singel pci_user_read_config_dword() loop could consume 130us+,

              |    pci_user_read_config_dword() {
              |      _raw_spin_lock_irq() {
! 136.698 us  |        native_queued_spin_lock_slowpath();
! 137.582 us  |      }
              |      pci_read() {
              |        raw_pci_read() {
              |          pci_conf1_read() {
  0.230 us    |            _raw_spin_lock_irqsave();
  0.035 us    |            _raw_spin_unlock_irqrestore();
  8.476 us    |          }
  8.790 us    |        }
  9.091 us    |      }
! 147.263 us  |    }
Quoted 6 lines

and dozens of the loop could consume ms+.

If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.

Add scheduling chance in the loop to improve the latency.

Thanks for the patch, this makes a lot of sense.

Shouldn't we do the same in pci_write_config()?

Quoted 8 lines
Quoted 1 line

Yes, IMHO, that could be helpful too.

If it's feasible, it would be nice to actually verify that it makes a difference. I know config writes should be faster than reads, but they're certainly not as fast as a CPU can pump out data, so there must be *some* mechanism that slows the CPU down.

Bjorn

We failed to build a test case to produce the latency by setpci command, AFAIU, setpci could be much less frequently realistically used than lspci. So, the latency from pci_write_config() path could not be verified for now, could we apply this patch alone to erase the verified latency introduced by pci_read_config() path? :)

Thanks a lot.
Regards,
Jiang

#7 lore raw
Quoted 40 lines · Jiang Biao
Quoted 30 lines · Bjorn Helgaas
Quoted 29 lines · Jiang Biao
Quoted 28 lines · Bjorn Helgaas
Quoted 18 lines · Jiang Biao

From: Jiang Biao <[email protected]>

pci_read_config() could block several ms in kernel space, mainly caused by the while loop to call pci_user_read_config_dword(). Singel pci_user_read_config_dword() loop could consume 130us+,

              |    pci_user_read_config_dword() {
              |      _raw_spin_lock_irq() {
! 136.698 us  |        native_queued_spin_lock_slowpath();
! 137.582 us  |      }
              |      pci_read() {
              |        raw_pci_read() {
              |          pci_conf1_read() {
  0.230 us    |            _raw_spin_lock_irqsave();
  0.035 us    |            _raw_spin_unlock_irqrestore();
  8.476 us    |          }
  8.790 us    |        }
  9.091 us    |      }
! 147.263 us  |    }
Quoted 6 lines

and dozens of the loop could consume ms+.

If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.

Add scheduling chance in the loop to improve the latency.

Thanks for the patch, this makes a lot of sense.

Quoted 9 lines
Quoted 3 lines
Quoted 1 line

Shouldn't we do the same in pci_write_config()?

Yes, IMHO, that could be helpful too.

If it's feasible, it would be nice to actually verify that it makes a difference. I know config writes should be faster than reads, but they're certainly not as fast as a CPU can pump out data, so there must be *some* mechanism that slows the CPU down.

Quoted 5 lines

We failed to build a test case to produce the latency by setpci command, AFAIU, setpci could be much less frequently realistically used than lspci. So, the latency from pci_write_config() path could not be verified for now, could we apply this patch alone to erase the verified latency introduced by pci_read_config() path? :)

Thanks for trying! I'll apply the patch as-is. I'd like to include a note in the commit log about the user-visible effect of this. I looked through recent similar commits:

928da37a229f ("RDMA/umem: Add a schedule point in ib_umem_get()") 47aaabdedf36 ("fanotify: Avoid softlockups when reading many events") 9f47eb5461aa ("fs/btrfs: Add cond_resched() for try_release_extent_mapping() stalls") 0a3b3c253a1e ("mm/mmap.c: Add cond_resched() for exit_mmap() CPU stalls") b7e3debdd040 ("mm/memory_hotplug.c: fix false softlockup during pfn range removal") d35bd764e689 ("dm writecache: add cond_resched to loop in persistent_memory_claim()") da97f2d56bbd ("mm: call cond_resched() from deferred_init_memmap()") ab8b65be1831 ("KVM: PPC: Book3S: Fix some RCU-list locks")
48c963e31bc6 ("KVM: arm/arm64: Release kvm->mmu_lock in loop to prevent starvation") e84fe99b68ce ("mm/page_alloc: fix watchdog soft lockups during set_zone_contiguous()") 4005f5c3c9d0 ("wireguard: send/receive: cond_resched() when processing worker ringbuffers") 3fd44c86711f ("io_uring: use cond_resched() in io_ring_ctx_wait_and_kill()") 7979457b1d3a ("net: bridge: vlan: Add a schedule point during VLAN processing") 2ed6edd33a21 ("perf: Add cond_resched() to task_function_call()") 1edaa447d958 ("dm writecache: add cond_resched to avoid CPU hangs") ce9a4186f9ac ("macvlan: add cond_resched() during multicast processing") 7be1b9b8e9d1 ("drm/mm: Break long searches in fragmented address spaces") bb699a793110 ("drm/i915/gem: Break up long lists of object reclaim") 9424ef56e13a ("ext4: add cond_resched() to __ext4_find_entry()")

and many of them mention softlockups, RCU CPU stall warnings, or watchdogs triggering. Are you seeing one of those, or are you measuring latency some other way?

Bjorn

author
#8 lore raw

Hi,

Quoted 42 lines · Bjorn Helgaas
Quoted 40 lines · Jiang Biao
Quoted 39 lines · Bjorn Helgaas
Quoted 25 lines · Jiang Biao
Quoted 24 lines · Bjorn Helgaas
Quoted 23 lines · Jiang Biao

From: Jiang Biao <[email protected]>

pci_read_config() could block several ms in kernel space, mainly caused by the while loop to call pci_user_read_config_dword(). Singel pci_user_read_config_dword() loop could consume 130us+,

              |    pci_user_read_config_dword() {
              |      _raw_spin_lock_irq() {
! 136.698 us  |        native_queued_spin_lock_slowpath();
! 137.582 us  |      }
              |      pci_read() {
              |        raw_pci_read() {
              |          pci_conf1_read() {
  0.230 us    |            _raw_spin_lock_irqsave();
  0.035 us    |            _raw_spin_unlock_irqrestore();
  8.476 us    |          }
  8.790 us    |        }
  9.091 us    |      }
! 147.263 us  |    }

and dozens of the loop could consume ms+.

If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

Add scheduling chance in the loop to improve the latency.

Thanks for the patch, this makes a lot of sense.

Shouldn't we do the same in pci_write_config()?

Yes, IMHO, that could be helpful too.

If it's feasible, it would be nice to actually verify that it makes a difference. I know config writes should be faster than reads, but they're certainly not as fast as a CPU can pump out data, so there must be *some* mechanism that slows the CPU down.

Quoted 7 lines
Quoted 5 lines

We failed to build a test case to produce the latency by setpci command, AFAIU, setpci could be much less frequently realistically used than lspci. So, the latency from pci_write_config() path could not be verified for now, could we apply this patch alone to erase the verified latency introduced by pci_read_config() path? :)

Thanks for trying! I'll apply the patch as-is. I'd like to include a

Thanks. :)

Quoted 23 lines

note in the commit log about the user-visible effect of this. I looked through recent similar commits:

928da37a229f ("RDMA/umem: Add a schedule point in ib_umem_get()") 47aaabdedf36 ("fanotify: Avoid softlockups when reading many events") 9f47eb5461aa ("fs/btrfs: Add cond_resched() for try_release_extent_mapping() stalls") 0a3b3c253a1e ("mm/mmap.c: Add cond_resched() for exit_mmap() CPU stalls") b7e3debdd040 ("mm/memory_hotplug.c: fix false softlockup during pfn range removal") d35bd764e689 ("dm writecache: add cond_resched to loop in persistent_memory_claim()") da97f2d56bbd ("mm: call cond_resched() from deferred_init_memmap()") ab8b65be1831 ("KVM: PPC: Book3S: Fix some RCU-list locks")
48c963e31bc6 ("KVM: arm/arm64: Release kvm->mmu_lock in loop to prevent starvation") e84fe99b68ce ("mm/page_alloc: fix watchdog soft lockups during set_zone_contiguous()") 4005f5c3c9d0 ("wireguard: send/receive: cond_resched() when processing worker ringbuffers") 3fd44c86711f ("io_uring: use cond_resched() in io_ring_ctx_wait_and_kill()") 7979457b1d3a ("net: bridge: vlan: Add a schedule point during VLAN processing") 2ed6edd33a21 ("perf: Add cond_resched() to task_function_call()") 1edaa447d958 ("dm writecache: add cond_resched to avoid CPU hangs") ce9a4186f9ac ("macvlan: add cond_resched() during multicast processing") 7be1b9b8e9d1 ("drm/mm: Break long searches in fragmented address spaces") bb699a793110 ("drm/i915/gem: Break up long lists of object reclaim") 9424ef56e13a ("ext4: add cond_resched() to __ext4_find_entry()")

Quoted 2 lines

and many of them mention softlockups, RCU CPU stall warnings, or watchdogs triggering. Are you seeing one of those, or are you

No softlockup or RCU stall warnings.

Quoted 1 line

measuring latency some other way?

We test the scheduling latency by cyclictest benchmark, the max latency could be more than 5ms without this patch. The ftrace log shows pci_read_config is the main cause, and the 5ms+ latency disappeared with this patch applied.

Thanks a lot.
Regards,
Jiang

#9 lore raw
In reply to Jiang Biao
Quoted 20 lines · Jiang Biao

From: Jiang Biao <[email protected]>

pci_read_config() could block several ms in kernel space, mainly caused by the while loop to call pci_user_read_config_dword(). Singel pci_user_read_config_dword() loop could consume 130us+,

              |    pci_user_read_config_dword() {
              |      _raw_spin_lock_irq() {
! 136.698 us  |        native_queued_spin_lock_slowpath();
! 137.582 us  |      }
              |      pci_read() {
              |        raw_pci_read() {
              |          pci_conf1_read() {
  0.230 us    |            _raw_spin_lock_irqsave();
  0.035 us    |            _raw_spin_unlock_irqrestore();
  8.476 us    |          }
  8.790 us    |        }
  9.091 us    |      }
! 147.263 us  |    }

and dozens of the loop could consume ms+.

Quoted 7 lines

If we execute some lspci commands concurrently, ms+ scheduling latency could be detected.

Add scheduling chance in the loop to improve the latency.

Applied to pci/enumeration for v5.10, thanks!

Quoted 19 lines
---
 drivers/pci/pci-sysfs.c | 1 +
 1 file changed, 1 insertion(+)
drivers/pci/pci-sysfs.c +1 -0
index 6d78df9..3b9f63d 100644
@@ -708,6 +708,7 @@ static ssize_t pci_read_config(struct file *filp, struct kobject *kobj,
 		data[off - init_off + 3] = (val >> 24) & 0xff;
 		off += 4;
 		size -= 4;
+		cond_resched();
 	}
 
 	if (size >= 2) {
Signature · 1 line
1.8.3.1