block: fix -EAGAIN IOPOLL task/vm accounting
Jens Axboe
maintainerPATCH 1/1 block: fix -EAGAIN IOPOLL task/vm accounting
We currently increment the task/vm counts when we first attempt to queue a bio. But this isn't necessarily correct - if the request allocation fails with -EAGAIN, for example, and the caller retries, then we'll over-account by as many retries as are done.
This can happen for polled IO, where we cannot wait for requests. Hence retries can get aggressive, if we're running out of requests. If this happens, then watching the IO rates in vmstat are incorrect as they count every issue attempt as successful and hence the stats are inflated by quite a lot potentially.
Add a bio flag to know if we've done accounting or not. This prevents the same bio from being accounted potentially many times, when retried.
---
block/blk-core.c +2 -1
@@ -1236,7 +1236,7 @@ blk_qc_t submit_bio(struct bio *bio)* If it's a regular read/write or a barrier with data attached, * go through the normal accounting stuff before submission. */ -if (bio_has_data (bio)) { +if (bio_has_data (bio) && !bio_flagged (bio,BIO_ACCOUNTED )) {unsigned int count;if (unlikely (bio_op (bio) ==REQ_OP_WRITE_SAME ))
@@ -1259,6 +1259,7 @@ blk_qc_t submit_bio(struct bio *bio) (unsigned long long )bio->bi_iter.bi_sector, bio_devname(bio, b), count); } +bio_set_flag (bio,BIO_ACCOUNTED ); }/*
include/linux/blk_types.h +1 -0
@@ -266,6 +266,7 @@ enum {* of this bio. */ BIO_CGROUP_ACCT ,/* has been accounted to a cgroup */ BIO_TRACKED ,/* set if bio goes through the rq_qos path */ +BIO_ACCOUNTED ,/* task/vm stats have been done */ BIO_FLAG_LAST };
Signature · 1 line
Jens Axboe
Quoted 5 lines · Jens Axboe
We currently increment the task/vm counts when we first attempt to queue a bio. But this isn't necessarily correct - if the request allocation fails with
-EAGAIN, for example, and the caller retries, then we'll over-account by as many retries as are done.
Quoted 8 lines
This can happen for polled IO, where we cannot wait for requests. Hence retries can get aggressive, if we're running out of requests. If this happens, then watching the IO rates in vmstat are incorrect as they count every issue attempt as successful and hence the stats are inflated by quite a lot potentially.
Add a bio flag to know if we've done accounting or not. This prevents the same bio from being accounted potentially many times, when retried.
Can't the resubmitter just use submit_bio_noacct? What is the call stack here?
Jens Axboe
maintainerQuoted 12 lines · Christoph Hellwig
Quoted 11 lines · Jens Axboe
We currently increment the task/vm counts when we first attempt to queue a bio. But this isn't necessarily correct - if the request allocation fails with
-EAGAIN, for example, and the caller retries, then we'll over-account by as many retries as are done.This can happen for polled IO, where we cannot wait for requests. Hence retries can get aggressive, if we're running out of requests. If this happens, then watching the IO rates in vmstat are incorrect as they count every issue attempt as successful and hence the stats are inflated by quite a lot potentially.
Quoted 5 lines
Quoted 2 lines
Add a bio flag to know if we've done accounting or not. This prevents the same bio from being accounted potentially many times, when retried.
Can't the resubmitter just use
submit_bio_noacct? What is the call stack here?
The resubmitter is way higher than that. You could potentially have that done in the block layer, but not higher up.
The use case is async submissions, going through ->read_iter() again. Or ->write_iter().
Signature · 1 line
Jens Axboe
Quoted 16 lines · Jens Axboe
Quoted 15 lines · Christoph Hellwig
Quoted 13 lines · Jens Axboe
We currently increment the task/vm counts when we first attempt to queue a bio. But this isn't necessarily correct - if the request allocation fails with
-EAGAIN, for example, and the caller retries, then we'll over-account by as many retries as are done.This can happen for polled IO, where we cannot wait for requests. Hence retries can get aggressive, if we're running out of requests. If this happens, then watching the IO rates in vmstat are incorrect as they count every issue attempt as successful and hence the stats are inflated by quite a lot potentially.
Add a bio flag to know if we've done accounting or not. This prevents the same bio from being accounted potentially many times, when retried.
Quoted 8 lines
Quoted 2 lines
Can't the resubmitter just use
submit_bio_noacct? What is the call stack here?The resubmitter is way higher than that. You could potentially have that done in the block layer, but not higher up.
The use case is async submissions, going through ->read_iter() again. Or ->write_iter().
But how does a bio flag help there? If we go through the file ops again the next submission will be a new bio structure.
Quoted 20 lines · Christoph Hellwig
Quoted 19 lines · Jens Axboe
Quoted 17 lines · Christoph Hellwig
Quoted 5 lines · Jens Axboe
We currently increment the task/vm counts when we first attempt to queue a bio. But this isn't necessarily correct - if the request allocation fails with
-EAGAIN, for example, and the caller retries, then we'll over-account by as many retries as are done.Quoted 8 lines
This can happen for polled IO, where we cannot wait for requests. Hence retries can get aggressive, if we're running out of requests. If this happens, then watching the IO rates in vmstat are incorrect as they count every issue attempt as successful and hence the stats are inflated by quite a lot potentially.
Add a bio flag to know if we've done accounting or not. This prevents the same bio from being accounted potentially many times, when retried.
Can't the resubmitter just use
submit_bio_noacct? What is the call stack here?
Quoted 8 lines
Quoted 5 lines
The resubmitter is way higher than that. You could potentially have that done in the block layer, but not higher up.
The use case is async submissions, going through ->read_iter() again. Or ->write_iter().
But how does a bio flag help there? If we go through the file ops again the next submission will be a new bio structure.
Yeah, we also have use cases of stack bio variable.
Thanks,
Ming
Jens Axboe
maintainerQuoted 20 lines · Christoph Hellwig
Quoted 19 lines · Jens Axboe
Quoted 17 lines · Christoph Hellwig
Quoted 5 lines · Jens Axboe
We currently increment the task/vm counts when we first attempt to queue a bio. But this isn't necessarily correct - if the request allocation fails with
-EAGAIN, for example, and the caller retries, then we'll over-account by as many retries as are done.Quoted 8 lines
This can happen for polled IO, where we cannot wait for requests. Hence retries can get aggressive, if we're running out of requests. If this happens, then watching the IO rates in vmstat are incorrect as they count every issue attempt as successful and hence the stats are inflated by quite a lot potentially.
Add a bio flag to know if we've done accounting or not. This prevents the same bio from being accounted potentially many times, when retried.
Can't the resubmitter just use
submit_bio_noacct? What is the call stack here?
Quoted 8 lines
Quoted 5 lines
The resubmitter is way higher than that. You could potentially have that done in the block layer, but not higher up.
The use case is async submissions, going through ->read_iter() again. Or ->write_iter().
But how does a bio flag help there? If we go through the file ops again the next submission will be a new bio structure.
Yeah the patch is garbage, can't work. The previous suggestion is here:
https://lore.kernel.org/linux-block/[email protected]/T/
which isn't super pretty either, but at least it works. Not sure there's a better solution, outside of marking the iocb as retry and then carrying that flag forward for the bio as well. And that seems a bit much for this case.
Signature · 1 line
Jens Axboe
Quoted 6 lines · Jens Axboe
Quoted 5 lines
Quoted 2 lines
The use case is async submissions, going through ->read_iter() again. Or ->write_iter().
But how does a bio flag help there? If we go through the file ops again the next submission will be a new bio structure.
Quoted 8 lines
Yeah the patch is garbage, can't work. The previous suggestion is here:
https://lore.kernel.org/linux-block/[email protected]/T/
which isn't super pretty either, but at least it works. Not sure there's a better solution, outside of marking the iocb as retry and then carrying that flag forward for the bio as well. And that seems a bit much for this case.
We'll still need a flag with the above to skip the submit_bio_noacct bios. But I think it is the right way to go. Eventually we'll also need to push the accounting down into the individual bio based drivers.
Jens Axboe
maintainerQuoted 11 lines · Christoph Hellwig
Quoted 10 lines · Jens Axboe
Quoted 5 lines
Quoted 2 lines
The use case is async submissions, going through ->read_iter() again. Or ->write_iter().
But how does a bio flag help there? If we go through the file ops again the next submission will be a new bio structure.
Yeah the patch is garbage, can't work. The previous suggestion is here:
Quoted 8 lines
Quoted 4 lines
which isn't super pretty either, but at least it works. Not sure there's a better solution, outside of marking the iocb as retry and then carrying that flag forward for the bio as well. And that seems a bit much for this case.
We'll still need a flag with the above to skip the
submit_bio_noacctbios. But I think it is the right way to go. Eventually we'll also need to push the accounting down into the individual bio based drivers.
For the iocb propagation, we'd really need the caller to mark the iocb as IOCB_ACCOUNTED (or whatever) if BIO_ACCOUNTED is set, since we can't do that further down the stack as we really don't know if we hit -EAGAIN before or after the bio was accounted... Which kind of sucks, as it'll be hard to contain in a generic fashion.
Signature · 1 line
Jens Axboe
Quoted 9 lines · Jens Axboe
Quoted 3 lines
We'll still need a flag with the above to skip the
submit_bio_noacctbios. But I think it is the right way to go. Eventually we'll also need to push the accounting down into the individual bio based drivers.For the iocb propagation, we'd really need the caller to mark the iocb as
IOCB_ACCOUNTED(or whatever) ifBIO_ACCOUNTEDis set, since we can't do that further down the stack as we really don't know if we hit-EAGAINbefore or after the bio was accounted... Which kind of sucks, as it'll be hard to contain in a generic fashion.
Well, that's why I think the only proper fix is to only account a bio when we know the driver is actually going to submit it.
Jens Axboe
maintainerQuoted 5 lines · Christoph Hellwig
Quoted 4 lines · Jens Axboe
Quoted 3 lines
We'll still need a flag with the above to skip the
submit_bio_noacctbios. But I think it is the right way to go. Eventually we'll also need to push the accounting down into the individual bio based drivers.
Quoted 8 lines
Quoted 5 lines
For the iocb propagation, we'd really need the caller to mark the iocb as
IOCB_ACCOUNTED(or whatever) ifBIO_ACCOUNTEDis set, since we can't do that further down the stack as we really don't know if we hit-EAGAINbefore or after the bio was accounted... Which kind of sucks, as it'll be hard to contain in a generic fashion.Well, that's why I think the only proper fix is to only account a bio when we know the driver is actually going to submit it.
Yeah I agree, it's a lot less code too. Which is basically back to my original RFC, I'll see if I can clean it up a bit.
Signature · 1 line
Jens Axboe