disk-based fds in select/poll
Pardon me if some parts of this seem clueless. While I'm no newbie in userland, kernelspace I don't play in very often...
It's fairly widely-known that select/poll returns immediately when testing a filesystem-based file descriptor for writability or readability.
On top of this, even when in non-blocking mode, read() could block if the pages needed aren't in core. sendfile() behaves in a similar way.
What would be needed to alleviate this?
I am thinking that a read() (or sendfile()) that would block because the pages aren't in core should instead post a request for the pages to be loaded (some kind of readahead mecanism?) and return immediately (maybe having given some data that *was* in core). A subsequent read() could have the data available, but not necessarily (again, it should give whatever it has in core, but return immediately).
sendfile() would be a lot more tricky to fix in that way I guess, but could still be possible (the destination fd would be unwritable for a while, until the transfer is finished). Also, the complexity would be higher (instead of simply causing readahead to happen (which might anyway), it would have to trigger the readahead, then get notification of when the pages are in core to send over, all the while preventing data from being written to the destination fd in some way).
In the mean time, I was also wondering if issuing smaller read() requests in a row might give me a better chance of success. I *know* read() will block, but if I only ask for, say, a page of data (rather than asking for the full data and relying on the non-blocking to return EAGAIN (like it should, IMHO!)), it shouldn't take too long, and could possibly trigger some readahead to be done by the kernel, right?
Or will the readahead be done "on my own time", read() only returning after the whole thing (my request + readahead) has been done?
I remember seeing a suggestion by Linus for an event-based I/O interface, similar to kqueue on FreeBSD but much simpler. I'd just say "I want it too!", ok? :-)
I know about the mincore() trick with mmap()'d files, but with small files, mmap()ing might not make sense (could be very often).
SGI's AIO might be a solution here, does it use threads? I'm trying to avoid context switching as much as possible, to keep the CPU cache as warm as possible.
Well, I might not have the choice to use threads, after all...
(sorry if this message got in twice, I used an NNTP gateway the previous time, I don't think it got through)
Signature · 1 line
Pierre Phaneuf
Quoted 4 lines
I am thinking that a
read()(orsendfile()) that would block because the pages aren't in core should instead post a request for the pages to be loaded (some kind of readahead mecanism?) and return immediately (maybe having given some data that *was* in core). A subsequentread()could
reads posts a readahead anyway so streaming reads tend not to block much
Quoted 3 lines
SGI's AIO might be a solution here, does it use threads? I'm trying to avoid context switching as much as possible, to keep the CPU cache as warm as possible.
glibc 2.2 does thread based aio_ and it will tend to avoid cache damage as the thread share the mm but on SMP its quite possible the read wil occur on the other CPU. Of course kernel based I/O might do the same too..
Quoted 6 lines · Alan Cox
Quoted 4 lines
I am thinking that a
read()(orsendfile()) that would block because the pages aren't in core should instead post a request for the pages to be loaded (some kind of readahead mecanism?) and return immediately (maybe having given some data that *was* in core). A subsequentread()couldreads posts a readahead anyway so streaming reads tend not to block much
Ok, so while knowing about select "lying" about readability of a file fd, if I would stick a file fd in my select-based loop anyway, but would only try to read a bit at a time (say, 4K or 8K) would trigger readahead, yet finish quickly enough that I can get back to processing other fds in my select loop?
Wouldn't that cause too many syscalls to be done? Or if this is actually the way to go without an actual thread, how should I go determining an optimal block size?
Was there anything new on the bind_event/get_events API idea that Linus proposed a while ago? That one had got me foaming at the mouth... :-)
Signature · 1 line
Pierre Phaneuf
Quoted 2 lines
Ok, so while knowing about select "lying" about readability of a file fd, if I would stick a file fd in my select-based loop anyway, but would
You could fix select to return when the page was cachied and return EWOULDBLOCK on reads if the page was not present to be honest. I don't think that would actually break any apps, and the specs seem to allow it
Quoted 3 lines
only try to read a bit at a time (say, 4K or 8K) would trigger readahead, yet finish quickly enough that I can get back to processing other fds in my select loop?
Probably
Quoted 3 lines
Wouldn't that cause too many syscalls to be done? Or if this is actually the way to go without an actual thread, how should I go determining an optimal block size?
fs block size I suspect or small multiple thereof