Skip to contentSkip to conversation

Question on IOStat accounting

DISCUSSION Indraneel Mukherjee 2 participants 4 messages Sep 17, 2014 – Sep 23, 2014 NVMe

lore
Share & tools
lore
#0 lore raw

In the current implementation of IO accounting in the nvme driver, accounting is started once IOD is successfully prepared. Thereafter 2 cases can happen, either IO gets successfully submitted to device or it can get queued up in the IOD list if there is a failure.

So what's the definition of iostat here? In-flight in the device or in-flight in the driver ? Is there a need to make this consistent?

-Indro

#1 lore raw
Quoted 7 lines · Indraneel Mukherjee

In the current implementation of IO accounting in the nvme driver, accounting is started once IOD is successfully prepared. Thereafter 2 cases can happen, either IO gets successfully submitted to device or it can get queued up in the IOD list if there is a failure.

So what's the definition of iostat here? In-flight in the device or in-flight in the driver ? Is there a need to make this consistent?

It seems consistent with other block drivers. Usually stats are handled above the device driver at the request layer, but bio-based drivers handle the stats on their own.

#2 lore raw

Re-posting as I think my mail was blocked due to company security policy banner in the previous mail :(

If nvme_submit_bio_queue() fails to queue the IOD, the BIO gets queued in the sq_cong BIO list.
But we don't start accounting for this immediately even though the BIO is now queued up with the Driver.
Any particular reason for this? Is it because it's a rare scenario we're talking about here?

Quoted 14 lines

-----Original Message-----
From: Linux-nvme [mailto:linux-nvme-bounces at lists.infradead.org] On Behalf Of Keith Busch
Sent: Wednesday, September 17, 2014 7:58 PM
To: Indraneel Mukherjee
Cc: linux-nvme at lists.infradead.org
Subject: Re: Question on IOStat accounting

Quoted 5 lines · Indraneel Mukherjee

In the current implementation of IO accounting in the nvme driver, accounting is started once IOD is successfully prepared. Thereafter 2 cases can happen, either IO gets successfully submitted to device or it can get queued up in the IOD list if there is a failure.

Quoted 4 lines
Quoted 2 lines

So what's the definition of iostat here? In-flight in the device or in-flight in the driver ? Is there a need to make this consistent?

It seems consistent with other block drivers. Usually stats are handled

above the

Quoted 1 line

device driver at the request layer, but bio-based drivers handle the stats

on their

Quoted 6 lines

own.

_______________________________________________
Linux-nvme mailing list
Linux-nvme at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

#3 lore raw
Quoted 6 lines · Indraneel Mukherjee

If nvme_submit_bio_queue() fails to queue the IOD, the BIO gets queued in the sq_cong BIO list.
But we don't start accounting for this immediately even though the BIO is now queued up with the Driver.
Any particular reason for this? Is it because it's a rare scenario we're talking about here?

Hi Indraneel,

This seems more-or-less consistent with request based block drivers. The main reason you'd get bios queued on the bio_list is if you're out of memory, and request based drivers wouldn't start accounting if it couldn't allocate memory either; if you're out of memory, you've probably bigger problems than iostats, so hopefully it's a rare scenario.

The other reason you'd get bio's queued is if we're splitting them, but then that kicks the kthread to immediatly send the two parts within a jiffie, so the accounting is probably not affected there.

Quoted 13 lines
Quoted 13 lines · Indraneel Mukherjee
Quoted 7 lines

In the current implementation of IO accounting in the nvme driver, accounting is started once IOD is successfully prepared. Thereafter 2 cases can happen, either IO gets successfully submitted to device or it can get queued up in the IOD list if there is a failure.

So what's the definition of iostat here? In-flight in the device or in-flight in the driver ? Is there a need to make this consistent?

It seems consistent with other block drivers. Usually stats are handled above the device driver at the request layer, but bio-based drivers handle the stats on their own.