do_hashed and wq enqueue
Hi,
I've been looking for hashed io-wq enqueuing, but not fully understand the issue. As I remember, it was something about getting around blocking for buffered I/O. Is that so? Is there any write-up of a thread for this issue?
My case is 2-fd request, and I'm not sure how it should look. For splice() it's probably Ok to hash the non-pipe end (if any), but I wonder how it should work, if, for example, the restriction will be removed.
Signature · 1 line
Pavel Begunkov
Jens Axboe
maintainerQuoted 5 lines · Pavel Begunkov
Hi,
I've been looking for hashed io-wq enqueuing, but not fully understand the issue. As I remember, it was something about getting around blocking for buffered I/O. Is that so? Is there any write-up of a thread for this issue?
Not sure if there's a writeup, but the issue is that buffered writes all need to grab the per-inode mutex. Hence if you don't serialize these writes, you end up having potentially quite a few threads banging on the same mutex. This causes a high level of lock contention (and scheduling). By serializing by inode hash we avoid that, and yield the same performance.
Dave Chinner is working on lifting this restriction, at which point we'll have to gate the hashing based on whether or not the fs is smart or dumb when it comes to buffered writes and locking.
Quoted 3 lines
My case is 2-fd request, and I'm not sure how it should look. For
splice()it's probably Ok to hash the non-pipe end (if any), but I wonder how it should work, if, for example, the restriction will be removed.
Probably just do the same, but for the output fd only (and following the same restrictions in terms of file type).
Signature · 1 line
Jens Axboe
Quoted 13 lines · Jens Axboe
Quoted 5 lines · Pavel Begunkov
Hi,
I've been looking for hashed io-wq enqueuing, but not fully understand the issue. As I remember, it was something about getting around blocking for buffered I/O. Is that so? Is there any write-up of a thread for this issue?
Not sure if there's a writeup, but the issue is that buffered writes all need to grab the per-inode mutex. Hence if you don't serialize these writes, you end up having potentially quite a few threads banging on the same mutex. This causes a high level of lock contention (and scheduling). By serializing by inode hash we avoid that, and yield the same performance.
Quoted 3 lines
Dave Chinner is working on lifting this restriction, at which point we'll have to gate the hashing based on whether or not the fs is smart or dumb when it comes to buffered writes and locking.
Got it, thanks!
Quoted 7 lines
Quoted 3 lines
My case is 2-fd request, and I'm not sure how it should look. For
splice()it's probably Ok to hash the non-pipe end (if any), but I wonder how it should work, if, for example, the restriction will be removed.Probably just do the same, but for the output fd only (and following the same restrictions in terms of file type).
Signature · 1 line
Pavel Begunkov