Skip to contentSkip to conversation

block: fix -EAGAIN IOPOLL task/vm accounting

PATCH In review Jens Axboe 3 participants 10 messages Aug 29, 2020 – Sep 1, 2020 Block layer

lore
Share & tools
lore

Jens Axboe

author maintainer
#0 lore raw

PATCH 1/1 block: fix -EAGAIN IOPOLL task/vm accounting

We currently increment the task/vm counts when we first attempt to queue a bio. But this isn't necessarily correct - if the request allocation fails with -EAGAIN, for example, and the caller retries, then we'll over-account by as many retries as are done.

This can happen for polled IO, where we cannot wait for requests. Hence retries can get aggressive, if we're running out of requests. If this happens, then watching the IO rates in vmstat are incorrect as they count every issue attempt as successful and hence the stats are inflated by quite a lot potentially.

Add a bio flag to know if we've done accounting or not. This prevents the same bio from being accounted potentially many times, when retried.

---

block/blk-core.c +2 -1
index d9d632639bd1..ff562a8cd9c9 100644
@@ -1236,7 +1236,7 @@ blk_qc_t submit_bio(struct bio *bio)
 	 * If it's a regular read/write or a barrier with data attached,
 	 * go through the normal accounting stuff before submission.
 	 */
-	if (bio_has_data(bio)) {
+	if (bio_has_data(bio) && !bio_flagged(bio, BIO_ACCOUNTED)) {
 		unsigned int count;
 
 		if (unlikely(bio_op(bio) == REQ_OP_WRITE_SAME))
@@ -1259,6 +1259,7 @@ blk_qc_t submit_bio(struct bio *bio)
 				(unsigned long long)bio->bi_iter.bi_sector,
 				bio_devname(bio, b), count);
 		}
+		bio_set_flag(bio, BIO_ACCOUNTED);
 	}
 
 	/*
include/linux/blk_types.h +1 -0
index 63a39e47fc60..39bcc9326c7a 100644
@@ -266,6 +266,7 @@ enum {
 				 * of this bio. */
 	BIO_CGROUP_ACCT,	/* has been accounted to a cgroup */
 	BIO_TRACKED,		/* set if bio goes through the rq_qos path */
+	BIO_ACCOUNTED,		/* task/vm stats have been done */
 	BIO_FLAG_LAST
 };
 
Signature · 1 line
Jens Axboe
#1 lore raw
Quoted 5 lines · Jens Axboe

We currently increment the task/vm counts when we first attempt to queue a bio. But this isn't necessarily correct - if the request allocation fails with -EAGAIN, for example, and the caller retries, then we'll over-account by as many retries as are done.

Quoted 8 lines

This can happen for polled IO, where we cannot wait for requests. Hence retries can get aggressive, if we're running out of requests. If this happens, then watching the IO rates in vmstat are incorrect as they count every issue attempt as successful and hence the stats are inflated by quite a lot potentially.

Add a bio flag to know if we've done accounting or not. This prevents the same bio from being accounted potentially many times, when retried.

Can't the resubmitter just use submit_bio_noacct? What is the call stack here?

author maintainer
#2 lore raw
Quoted 12 lines · Christoph Hellwig
Quoted 11 lines · Jens Axboe

We currently increment the task/vm counts when we first attempt to queue a bio. But this isn't necessarily correct - if the request allocation fails with -EAGAIN, for example, and the caller retries, then we'll over-account by as many retries as are done.

This can happen for polled IO, where we cannot wait for requests. Hence retries can get aggressive, if we're running out of requests. If this happens, then watching the IO rates in vmstat are incorrect as they count every issue attempt as successful and hence the stats are inflated by quite a lot potentially.

Quoted 5 lines
Quoted 2 lines

Add a bio flag to know if we've done accounting or not. This prevents the same bio from being accounted potentially many times, when retried.

Can't the resubmitter just use submit_bio_noacct? What is the call stack here?

The resubmitter is way higher than that. You could potentially have that done in the block layer, but not higher up.

The use case is async submissions, going through ->read_iter() again. Or ->write_iter().

Signature · 1 line
Jens Axboe
#3 lore raw
Quoted 16 lines · Jens Axboe
Quoted 15 lines · Christoph Hellwig
Quoted 13 lines · Jens Axboe

We currently increment the task/vm counts when we first attempt to queue a bio. But this isn't necessarily correct - if the request allocation fails with -EAGAIN, for example, and the caller retries, then we'll over-account by as many retries as are done.

This can happen for polled IO, where we cannot wait for requests. Hence retries can get aggressive, if we're running out of requests. If this happens, then watching the IO rates in vmstat are incorrect as they count every issue attempt as successful and hence the stats are inflated by quite a lot potentially.

Add a bio flag to know if we've done accounting or not. This prevents the same bio from being accounted potentially many times, when retried.

Quoted 8 lines
Quoted 2 lines

Can't the resubmitter just use submit_bio_noacct? What is the call stack here?

The resubmitter is way higher than that. You could potentially have that done in the block layer, but not higher up.

The use case is async submissions, going through ->read_iter() again. Or ->write_iter().

But how does a bio flag help there? If we go through the file ops again the next submission will be a new bio structure.

#4 lore raw
Quoted 20 lines · Christoph Hellwig
Quoted 19 lines · Jens Axboe
Quoted 17 lines · Christoph Hellwig
Quoted 5 lines · Jens Axboe

We currently increment the task/vm counts when we first attempt to queue a bio. But this isn't necessarily correct - if the request allocation fails with -EAGAIN, for example, and the caller retries, then we'll over-account by as many retries as are done.

Quoted 8 lines

This can happen for polled IO, where we cannot wait for requests. Hence retries can get aggressive, if we're running out of requests. If this happens, then watching the IO rates in vmstat are incorrect as they count every issue attempt as successful and hence the stats are inflated by quite a lot potentially.

Add a bio flag to know if we've done accounting or not. This prevents the same bio from being accounted potentially many times, when retried.

Can't the resubmitter just use submit_bio_noacct? What is the call stack here?

Quoted 8 lines
Quoted 5 lines

The resubmitter is way higher than that. You could potentially have that done in the block layer, but not higher up.

The use case is async submissions, going through ->read_iter() again. Or ->write_iter().

But how does a bio flag help there? If we go through the file ops again the next submission will be a new bio structure.

Yeah, we also have use cases of stack bio variable.

Thanks,
Ming

author maintainer
#5 lore raw
In reply to Christoph Hellwig
Quoted 20 lines · Christoph Hellwig
Quoted 19 lines · Jens Axboe
Quoted 17 lines · Christoph Hellwig
Quoted 5 lines · Jens Axboe

We currently increment the task/vm counts when we first attempt to queue a bio. But this isn't necessarily correct - if the request allocation fails with -EAGAIN, for example, and the caller retries, then we'll over-account by as many retries as are done.

Quoted 8 lines

This can happen for polled IO, where we cannot wait for requests. Hence retries can get aggressive, if we're running out of requests. If this happens, then watching the IO rates in vmstat are incorrect as they count every issue attempt as successful and hence the stats are inflated by quite a lot potentially.

Add a bio flag to know if we've done accounting or not. This prevents the same bio from being accounted potentially many times, when retried.

Can't the resubmitter just use submit_bio_noacct? What is the call stack here?

Quoted 8 lines
Quoted 5 lines

The resubmitter is way higher than that. You could potentially have that done in the block layer, but not higher up.

The use case is async submissions, going through ->read_iter() again. Or ->write_iter().

But how does a bio flag help there? If we go through the file ops again the next submission will be a new bio structure.

Yeah the patch is garbage, can't work. The previous suggestion is here:

https://lore.kernel.org/linux-block/[email protected]/T/

which isn't super pretty either, but at least it works. Not sure there's a better solution, outside of marking the iocb as retry and then carrying that flag forward for the bio as well. And that seems a bit much for this case.

Signature · 1 line
Jens Axboe
#6 lore raw
Quoted 6 lines · Jens Axboe
Quoted 5 lines
Quoted 2 lines

The use case is async submissions, going through ->read_iter() again. Or ->write_iter().

But how does a bio flag help there? If we go through the file ops again the next submission will be a new bio structure.

Quoted 8 lines

Yeah the patch is garbage, can't work. The previous suggestion is here:

https://lore.kernel.org/linux-block/[email protected]/T/

which isn't super pretty either, but at least it works. Not sure there's a better solution, outside of marking the iocb as retry and then carrying that flag forward for the bio as well. And that seems a bit much for this case.

We'll still need a flag with the above to skip the submit_bio_noacct bios. But I think it is the right way to go. Eventually we'll also need to push the accounting down into the individual bio based drivers.

author maintainer
#7 lore raw
Quoted 11 lines · Christoph Hellwig
Quoted 10 lines · Jens Axboe
Quoted 5 lines
Quoted 2 lines

The use case is async submissions, going through ->read_iter() again. Or ->write_iter().

But how does a bio flag help there? If we go through the file ops again the next submission will be a new bio structure.

Yeah the patch is garbage, can't work. The previous suggestion is here:

https://lore.kernel.org/linux-block/[email protected]/T/

Quoted 8 lines
Quoted 4 lines

which isn't super pretty either, but at least it works. Not sure there's a better solution, outside of marking the iocb as retry and then carrying that flag forward for the bio as well. And that seems a bit much for this case.

We'll still need a flag with the above to skip the submit_bio_noacct bios. But I think it is the right way to go. Eventually we'll also need to push the accounting down into the individual bio based drivers.

For the iocb propagation, we'd really need the caller to mark the iocb as IOCB_ACCOUNTED (or whatever) if BIO_ACCOUNTED is set, since we can't do that further down the stack as we really don't know if we hit -EAGAIN before or after the bio was accounted... Which kind of sucks, as it'll be hard to contain in a generic fashion.

Signature · 1 line
Jens Axboe
#8 lore raw
Quoted 9 lines · Jens Axboe
Quoted 3 lines

We'll still need a flag with the above to skip the submit_bio_noacct bios. But I think it is the right way to go. Eventually we'll also need to push the accounting down into the individual bio based drivers.

For the iocb propagation, we'd really need the caller to mark the iocb as IOCB_ACCOUNTED (or whatever) if BIO_ACCOUNTED is set, since we can't do that further down the stack as we really don't know if we hit -EAGAIN before or after the bio was accounted... Which kind of sucks, as it'll be hard to contain in a generic fashion.

Well, that's why I think the only proper fix is to only account a bio when we know the driver is actually going to submit it.

author maintainer
#9 lore raw
Quoted 5 lines · Christoph Hellwig
Quoted 4 lines · Jens Axboe
Quoted 3 lines

We'll still need a flag with the above to skip the submit_bio_noacct bios. But I think it is the right way to go. Eventually we'll also need to push the accounting down into the individual bio based drivers.

Quoted 8 lines
Quoted 5 lines

For the iocb propagation, we'd really need the caller to mark the iocb as IOCB_ACCOUNTED (or whatever) if BIO_ACCOUNTED is set, since we can't do that further down the stack as we really don't know if we hit -EAGAIN before or after the bio was accounted... Which kind of sucks, as it'll be hard to contain in a generic fashion.

Well, that's why I think the only proper fix is to only account a bio when we know the driver is actually going to submit it.

Yeah I agree, it's a lot less code too. Which is basically back to my original RFC, I'll see if I can clean it up a bit.

Signature · 1 line
Jens Axboe