fixed files
Hi,
As you remember, splice(2) needs two fds, and it's a bit of a pain finding a place for the second REQ_F_FIXED_FILE flag. So, I was thinking, can we use the last (i.e. sign) bit to mark an fd as fixed? A lot of userspace programs consider any negative result of open() as an error, so it's more or less safe to reuse it.
e.g.fill_sqe(fd) // is not fixedfill_sqe(buf_idx | LAST_BIT) // fixed file
Signature · 1 line
Pavel Begunkov
Jens Axboe
maintainerQuoted 11 lines · Pavel Begunkov
Hi,
As you remember, splice(2) needs two fds, and it's a bit of a pain finding a place for the second
REQ_F_FIXED_FILEflag. So, I was thinking, can we use the last (i.e. sign) bit to mark an fd as fixed? A lot of userspace programs consider any negative result ofopen()as an error, so it's more or less safe to reuse it.e.g.
fill_sqe(fd)// is not fixedfill_sqe(buf_idx | LAST_BIT)// fixed file
Right now we only support 1024 fixed buffers anyway, so we do have some space there. If we steal a bit, it'll still allow us to expand to 32K of fixed buffers in the future.
It's a bit iffy, but like you, I don't immediately see a better way to do this that doesn't include stealing an IOSQE bit or adding a special splice flag for it. Might still prefer the latter, to be honest...
Signature · 1 line
Jens Axboe
Quoted 13 lines · Jens Axboe
Quoted 11 lines · Pavel Begunkov
Hi,
As you remember, splice(2) needs two fds, and it's a bit of a pain finding a place for the second
REQ_F_FIXED_FILEflag. So, I was thinking, can we use the last (i.e. sign) bit to mark an fd as fixed? A lot of userspace programs consider any negative result ofopen()as an error, so it's more or less safe to reuse it.e.g.
fill_sqe(fd)// is not fixedfill_sqe(buf_idx | LAST_BIT)// fixed file
Quoted 7 lines
Right now we only support 1024 fixed buffers anyway, so we do have some space there. If we steal a bit, it'll still allow us to expand to 32K of fixed buffers in the future.
It's a bit iffy, but like you, I don't immediately see a better way to do this that doesn't include stealing an IOSQE bit or adding a special splice flag for it. Might still prefer the latter, to be honest...
"fixed" is clearly a per-{fd,buffer} attribute. If I'd now design it from the scratch, I would store fixed-resource index in the same field as fds and addr (but not separate @buf_index), and have per-resource switch-flag somewhere. And then I see 2 convenient ways:
1. encode the fixed bit into addr and fd, as supposed above.
2. Add N generic IOSQE_FIXED bits (i.e. IOSQE_FIXED_RESOURSE{1,2,...}), which correspond to resources (fd, buffer, etc) in order of occurrence in an sqe. I wouldn't expect having more than 3-4 flags.
And then IORING_OP_{READ,WRITE}_FIXED would have been the same opcode as the corresponding non-fixed version. But backward-compatibility is a pain.
Signature · 1 line
Pavel Begunkov
Jens Axboe
maintainerQuoted 27 lines · Pavel Begunkov
Quoted 13 lines · Jens Axboe
Quoted 11 lines · Pavel Begunkov
Hi,
As you remember, splice(2) needs two fds, and it's a bit of a pain finding a place for the second
REQ_F_FIXED_FILEflag. So, I was thinking, can we use the last (i.e. sign) bit to mark an fd as fixed? A lot of userspace programs consider any negative result ofopen()as an error, so it's more or less safe to reuse it.e.g.
fill_sqe(fd)// is not fixedfill_sqe(buf_idx | LAST_BIT)// fixed fileQuoted 7 lines
Right now we only support 1024 fixed buffers anyway, so we do have some space there. If we steal a bit, it'll still allow us to expand to 32K of fixed buffers in the future.
It's a bit iffy, but like you, I don't immediately see a better way to do this that doesn't include stealing an IOSQE bit or adding a special splice flag for it. Might still prefer the latter, to be honest...
"fixed" is clearly a per-{fd,buffer} attribute. If I'd now design it from the scratch, I would store fixed-resource index in the same field as fds and addr (but not separate @buf_index), and have per-resource switch-flag somewhere. And then I see 2 convenient ways:
Quoted 8 lines
1. encode the fixed bit into addr and fd, as supposed above.
2. Add N generic
IOSQE_FIXEDbits (i.e. IOSQE_FIXED_RESOURSE{1,2,...}), which correspond to resources (fd, buffer, etc) in order of occurrence in an sqe. I wouldn't expect having more than 3-4 flags.And then IORING_OP_{READ,WRITE}_FIXED would have been the same opcode as the corresponding non-fixed version. But backward-compatibility is a pain.
It's always much easier looking back, hindsight is much clearer. I'd also expand the sqe flags bits to 16 at least, but oh well.
I do think that for this particular case we add a SPLICE_F_FD1_FIXED and ditto for fd2, and just have the direct splice/vmsplice syscalls reject them as invalid. Both splice and vmsplice -EINVAL for unknown flags, which makes this possible.
That seems cleaner to me than trying to shoe-horn this information into the sqe itself, and it can easily be done as a prep patch to adding splice support.
Signature · 1 line
Jens Axboe
Quoted 34 lines · Jens Axboe
Quoted 33 lines · Pavel Begunkov
Quoted 13 lines · Jens Axboe
Quoted 11 lines · Pavel Begunkov
Hi,
As you remember, splice(2) needs two fds, and it's a bit of a pain finding a place for the second
REQ_F_FIXED_FILEflag. So, I was thinking, can we use the last (i.e. sign) bit to mark an fd as fixed? A lot of userspace programs consider any negative result ofopen()as an error, so it's more or less safe to reuse it.e.g.
fill_sqe(fd)// is not fixedfill_sqe(buf_idx | LAST_BIT)// fixed fileQuoted 7 lines
Right now we only support 1024 fixed buffers anyway, so we do have some space there. If we steal a bit, it'll still allow us to expand to 32K of fixed buffers in the future.
It's a bit iffy, but like you, I don't immediately see a better way to do this that doesn't include stealing an IOSQE bit or adding a special splice flag for it. Might still prefer the latter, to be honest...
"fixed" is clearly a per-{fd,buffer} attribute. If I'd now design it from the scratch, I would store fixed-resource index in the same field as fds and addr (but not separate @buf_index), and have per-resource switch-flag somewhere. And then I see 2 convenient ways:
1. encode the fixed bit into addr and fd, as supposed above.
2. Add N generic
IOSQE_FIXEDbits (i.e. IOSQE_FIXED_RESOURSE{1,2,...}), which correspond to resources (fd, buffer, etc) in order of occurrence in an sqe. I wouldn't expect having more than 3-4 flags.
Quoted 5 lines
Quoted 2 lines
And then IORING_OP_{READ,WRITE}_FIXED would have been the same opcode as the corresponding non-fixed version. But backward-compatibility is a pain.
It's always much easier looking back, hindsight is much clearer. I'd also expand the sqe flags bits to 16 at least, but oh well.
Totally agree. And I didn't meant I could have done better
BTW, we can extend sqe flags into free bits at the end of sqe, would need a bit of packing/unpacking though.
e.g. int flags = sqe1_flags | (sqe2_flags << 8)
Quoted 4 lines
I do think that for this particular case we add a
SPLICE_F_FD1_FIXEDand ditto for fd2, and just have the direct splice/vmsplice syscalls reject them as invalid. Both splice and vmsplice-EINVALfor unknown flags, which makes this possible.
Yes, I remember that from the splice thread.
Quoted 4 lines
That seems cleaner to me than trying to shoe-horn this information into the sqe itself, and it can easily be done as a prep patch to adding splice support.
Signature · 1 line
Pavel Begunkov