iocost_monitor: fix kernel queue kobj changes
PATCH 1/3 iocost_monitor: fix kernel queue kobj changes
From: Chengming Zhou <[email protected]>
When I use iocost_monitor on nvme0n1, this error shows up:
"Could not find ioc for nvme0n1"
There is no kobj in struct queue in recent kernel, it seems that the commit 2bd85221a625 ("block: untangle request_queue refcounting from sysfs") move the queue kobj to struct gendisk.
Fix it by using mq_kobj which is at the same level with queue kobj.
--- tools/cgroup/iocost_monitor.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/cgroup/iocost_monitor.py +1 -1
@@ -221,7 +221,7 @@ ioc = None for i, ptr in radix_tree_for_each(blkcg_root.blkg_tree.address_of_()): blkg = drgn.Object(prog, 'struct blkcg_gq', address=ptr) try: - if devname == blkg.q.kobj.parent.name.string_().decode('utf-8'): + if devname == blkg.q.mq_kobj.parent.name.string_().decode('utf-8'): q_id = blkg.q.id.value_() if blkg.pd[plid]: root_iocg = container_of(blkg.pd[plid], 'struct ioc_gq', 'pd')
Signature · 1 line
2.41.0
PATCH 2/3 iocost_monitor: print vrate inuse along with base_vrate
From: Chengming Zhou <[email protected]>
The real vrate iocost inuse is not base_vrate, but the atomic vtime_rate. We need iocost_monitor tool to display this real vrate that iocost use, to check if the boosted compensated vrate is normal.
Effect after change:
nvme0n1 RUN per=50.0ms cur_per=172116.580:v1040587.433 busy= +0 \
vrate=135.00%:270.00% params=ssd_dfl(CQ)
^
|
this is real vrate inuse--- tools/cgroup/iocost_monitor.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
tools/cgroup/iocost_monitor.py +5 -2
@@ -100,6 +100,7 @@ class IocStat: self.period_at = ioc.period_at.value_() / 1_000_000 self.vperiod_at = ioc.period_at_vtime.value_() / VTIME_PER_SEC self.vrate_pct = ioc.vtime_base_rate.value_() * 100 / VTIME_PER_USEC + self.ivrate_pct = ioc.vtime_rate.counter.value_() * 100 / VTIME_PER_USEC self.busy_level = ioc.busy_level.value_() self.autop_idx = ioc.autop_idx.value_() self.user_cost_model = ioc.user_cost_model.value_()
@@ -119,7 +120,9 @@ class IocStat: 'period_at' : self.period_at, 'period_vtime_at' : self.vperiod_at, 'busy_level' : self.busy_level, - 'vrate_pct' : self.vrate_pct, } + 'vrate_pct' : self.vrate_pct, + 'ivrate_pct' : self.ivrate_pct, + } def table_preamble_str(self): state = ('RUN' if self.running else 'IDLE') if self.enabled else 'OFF'
@@ -127,7 +130,7 @@ class IocStat: f'per={self.period_ms}ms ' \ f'cur_per={self.period_at:.3f}:v{self.vperiod_at:.3f} ' \ f'busy={self.busy_level:+3} ' \ - f'vrate={self.vrate_pct:6.2f}% ' \ + f'vrate={self.vrate_pct:6.2f}%:{self.ivrate_pct:6.2f}% ' \ f'params={self.autop_name}' if self.user_cost_model or self.user_qos_params: output += f'({"C" if self.user_cost_model else ""}{"Q" if self.user_qos_params else ""})'
Signature · 1 line
2.41.0
PATCH 3/3 iocost_monitor: improve it by adding iocg wait_ms
In reply to [email protected]From: Chengming Zhou <[email protected]>
The iocg can have three throttled metrics: wait, debt, delay. This patch add missing wait_ms to IocgStat to show the latest wait_ms of iocg.
As we are here, group iocg usage percents "inflt%" and "usage%" together, and group iocg throttled metrics "wait", "debt" and "delay" together.
Effect after changes:
nvme0n1 RUN per=50.0ms cur_per=177105.713:v1053528.587 busy= +0 vrate=135.00%:270.00% params=ssd_dfl(CQ)
active weight hweight% inflt% usage% wait debt delay InterfererGroup0 * 100/ 100 54.28/ 9.09 0.34 24.07 0.00 0.00 0.00 interfered * 84/ 1000 45.72/ 90.91 0.48 41.09 0.00 0.00 0.00
--- tools/cgroup/iocost_monitor.py | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
tools/cgroup/iocost_monitor.py +8 -4
@@ -138,7 +138,7 @@ class IocStat: def table_header_str(self): return f'{"":25} active {"weight":>9} {"hweight%":>13} {"inflt%":>6} ' \ - f'{"debt":>7} {"delay":>7} {"usage%"}' + f'{"usage%":>6} {"wait":>7} {"debt":>7} {"delay":>7}' class IocgStat: def __init__(self, iocg):
@@ -164,6 +164,8 @@ class IocgStat: self.usage = (100 * iocg.usage_delta_us.value_() / ioc.period_us.value_()) if self.active else 0 + self.wait_ms = (iocg.stat.wait_us.value_() - + iocg.last_stat.wait_us.value_()) / 1000 self.debt_ms = iocg.abs_vdebt.value_() / VTIME_PER_USEC / 1000 if blkg.use_delay.counter.value_() != 0: self.delay_ms = blkg.delay_nsec.counter.value_() / 1_000_000
@@ -180,9 +182,10 @@ class IocgStat: 'hweight_active_pct' : self.hwa_pct, 'hweight_inuse_pct' : self.hwi_pct, 'inflight_pct' : self.inflight_pct, + 'usage_pct' : self.usage, + 'wait_ms' : self.wait_ms, 'debt_ms' : self.debt_ms, 'delay_ms' : self.delay_ms, - 'usage_pct' : self.usage, 'address' : self.address } return out
@@ -192,9 +195,10 @@ class IocgStat: f'{round(self.inuse):5}/{round(self.active):5} ' \ f'{self.hwi_pct:6.2f}/{self.hwa_pct:6.2f} ' \ f'{self.inflight_pct:6.2f} ' \ + f'{min(self.usage, 999):6.2f} ' \ + f'{self.wait_ms:7.2f} ' \ f'{self.debt_ms:7.2f} ' \ - f'{self.delay_ms:7.2f} '\ - f'{min(self.usage, 999):6.2f}' + f'{self.delay_ms:7.2f}' out = out.rstrip(':') return out
Signature · 1 line
2.41.0
Re: [PATCH 3/3] iocost_monitor: improve it by adding iocg wait_ms
Quoted 8 lines · On Fri, Aug 04, 2023 at 02:50:39PM +0800, [email protected] wrote:
From: Chengming Zhou <[email protected]>
The iocg can have three throttled metrics: wait, debt, delay. This patch add missing
wait_msto IocgStat to show the latestwait_msof iocg.As we are here, group iocg usage percents "inflt%" and "usage%" together, and group iocg throttled metrics "wait", "debt" and "delay" together.
Quoted 8 lines
Effect after changes:
nvme0n1 RUN per=50.0ms cur_per=177105.713:v1053528.587 busy= +0 vrate=135.00%:270.00% params=ssd_dfl(CQ)
active weight hweight% inflt% usage% wait debt delay InterfererGroup0 * 100/ 100 54.28/ 9.09 0.34 24.07 0.00 0.00 0.00 interfered * 84/ 1000 45.72/ 90.91 0.48 41.09 0.00 0.00 0.00Signed-off-by Chengming Zhou <[email protected]>
For 1-3:
Thanks.
Signature · 1 line
tejun
Jens Axboe
AppliedQuoted 8 lines · On Fri, 04 Aug 2023 14:50:37 +0800, [email protected] wrote:
When I use
iocost_monitoron nvme0n1, this error shows up:
"Could not find ioc for nvme0n1"There is no kobj in struct queue in recent kernel, it seems that the commit
2bd85221a625("block: untanglerequest_queuerefcounting from sysfs") move the queue kobj to struct gendisk.[...]
Applied, thanks!
[1/3] iocost_monitor: fix kernel queue kobj changes
commit: 2eae9c4912b6cfdfadcd4fa8ac26879e18a504a1
[2/3] iocost_monitor: print vrate inuse along with base_vrate
commit: 8e93c1acd15e6a754c19ef12f6e69641f37e267a
[3/3] iocost_monitor: improve it by adding iocg wait_ms
commit: 68392b002023cb6dadd3d5044268470a7201b313Best regards,
Signature · 1 line
Jens Axboe