Skip to content

RFC fixed worker: a new way to handle io works

Hao Xu io-uring 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest9 patches 4 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/9io-wq: decouple work_list protection from the big wqe->lockfs/io-wq.c+54 -350 replies
2/9io-wq: reduce acct->lock crossing functions lock/unlockfs/io-wq.c+9 -180 replies
3/9io-wq: update check condition for lockfs/io-wq.c+1 -42 replies
4/9io-wq: use IO_WQ_ACCT_NR rather than hardcoded numberfs/io-wq.c+2 -20 replies
5/9io-wq: move hash wait entry to io_wqe_acctfs/io-wq.c+24 -210 replies
6/9io-wq: add infra data structure for fixed workersfs/io-wq.c+45 -180 replies
7/9io-wq: implement fixed worker logicfs/io-wq.c+161 -250 replies
8/9io-wq: batch the handling of fixed worker private worksfs/io-wq.c fs/io-wq.h+37 -100 replies
9/9io-wq: small optimization for __io_worker_busy()fs/io-wq.c+6 -100 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC 0/9] fixed worker: a new way to handle io works

There is big contension in current io-wq implementation. Introduce a new type io-worker called fixed-worker to solve this problem. it is also a new way to handle works. In this new system, works are dispatched to different private queues rather than a long shared queue.

Detail introduction and data in 7/9.

To be done: 1) the hash optimization isn't applied yet
2) user interface
3) cannot ensure linear order for works of same reg file writing since we now have multiple work lists.
4) code clean

Sent this for suggestions.

The test program used in this patchset:
// nop_test.c
// remove some error handling, variable definition, header files etc. typedef long long ll;

ll usecs(struct timeval tv) {
    return tv.tv_sec*(ll)1000*1000+tv.tv_usec;
}

static int test_single_nop(struct io_uring *ring, int depth)

{
    for (i=0; i<depth; i++) {
        sqe = io_uring_get_sqe(ring);
        io_uring_prep_nop(sqe);
        sqe->flags |= IOSQE_ASYNC;
    }
    ret = io_uring_submit(ring);
    for(i=0; i<depth; i++) {
        ret = io_uring_wait_cqe(ring, &cqe);
        io_uring_cqe_seen(ring, cqe);
    }
    return 0;
}

int main(int argc, char *argv[])

{
    ll delta;
    struct io_uring ring;
    int ret, l, loop=4000000, depth = 10;
    struct timeval tv_begin, tv_end;
    struct timezone tz;

    ret = io_uring_queue_init(10010, &ring, 0);
    if (ret) {
        fprintf(stderr, "ring setup failed: %d\n", ret);
        return 1;
    }
    l = loop;
    gettimeofday(&tv_begin, &tz);
    while(loop--)
        test_single_nop(&ring, depth);
    gettimeofday(&tv_end, &tz);
    delta =  usecs(tv_end) - usecs(tv_begin);
    printf("time spent: %lld usecs\n", delta);
    printf("IOPS: %lld\n", (ll)l * depth * 1000000 / delta);

    return 0;
}

Hao Xu (9):
io-wq: decouple work_list protection from the big wqe->lock
io-wq: reduce acct->lock crossing functions lock/unlock
io-wq: update check condition for lock
io-wq: use IO_WQ_ACCT_NR rather than hardcoded number
io-wq: move hash wait entry to io_wqe_acct
io-wq: add infra data structure for fix workers
io-wq: implement fixed worker logic
io-wq: batch the handling of fixed worker private works
io-wq: small optimization for __io_worker_busy()

 fs/io-wq.c | 415 ++++++++++++++++++++++++++++++++++++++---------------
 fs/io-wq.h |   5 +
 2 files changed, 308 insertions(+), 112 deletions(-)
Signature · 1 line
2.24.4