O_DIRECT, ext3fs, kernel 2.4.32... again
Hi all :)
I don't know if the patch to backport O_DIRECT support for ext3
under kernel 2.4.3x was finally accepted or not, but I'm having what I consider inconsistent behaviour due to O_DIRECT under ext3fs and kernel 2.4.32.
I can understand that ext3 doesn't support O_DIRECT, and that's
not a problem for me. In fact, if an app really needs O_DIRECT and the underlying filesystem doesn't support it, the app should fail, no more and no less.
The problem I'm having is with dvd+rw-tools. Apart from all the
problems regarding DVD writing, I have another problem: the open64 call with the O_DIRECT flag succeeds, but any subsequent read operation fails. IMHO, if the filesystem is going to return EINVAL for any read/write operation over an O_DIRECT'ed filehandle, it should return an error when opening, too.
The growisofs program tries to open a file using O_DIRECT and the
call succeeds, so it tries to read from that filehandle and the result is always EINVAL. I've tried a test program, just in case the problem was memory alignment of the buffer, but nothing is solved (I used posix_memalign and some recipe I found in this list, using the st_blksize and the st_size of the file). The problem seems to be in the O_DIRECT flag, because removing it from the open call makes all work.
Shouldn't ext3fs return an error when the O_DIRECT flag is used
in the open call? Is the open call userspace only and thus only libc can return such error? Am I misunderstanding the entire issue and this is a perfectly legal behaviour (allowing the open, failing in the read operation)?
Thanks a lot in advance :)))
Raúl Núñez de Arenas Coronado
Signature · 3 lines
Linux Registered User 88736 | http://www.dervishd.net http://www.pleyades.net & http://www.gotesdelluna.net It's my PC and I'll cry if I want to... RAmen!
Hi Raul,
Quoted 11 lines · DervishD
Hi all :)
I don't know if the patch to backport
O_DIRECTsupport for ext3
under kernel 2.4.3x was finally accepted or not, but I'm having what I consider inconsistent behaviour due toO_DIRECTunder ext3fs and kernel 2.4.32.I can understand that ext3 doesn't support
O_DIRECT, and that's
not a problem for me. In fact, if an app really needsO_DIRECTand the underlying filesystem doesn't support it, the app should fail, no more and no less.
On v2.4, nope it doesnt.
Quoted 17 lines
The problem I'm having is with dvd+rw-tools. Apart from all the
problems regarding DVD writing, I have another problem: the open64 call with theO_DIRECTflag succeeds, but any subsequent read operation fails. IMHO, if the filesystem is going to return EINVAL for any read/write operation over an O_DIRECT'ed filehandle, it should return an error when opening, too.The growisofs program tries to open a file using
O_DIRECTand the
call succeeds, so it tries to read from that filehandle and the result is always EINVAL.I've tried a test program, just in case the problem was memory alignment of the buffer, but nothing is solved (I used
posix_memalignand some recipe I found in this list, using thest_blksizeand thest_sizeof the file). The problem seems to be in theO_DIRECTflag, because removing it from the open call makes all work.
Quoted 5 lines
Shouldn't ext3fs return an error when the
O_DIRECTflag is used
in the open call? Is the open call userspace only and thus only libc can return such error? Am I misunderstanding the entire issue and this is a perfectly legal behaviour (allowing the open, failing in the read operation)?
Your interpretation is correct. It would be nicer for open() to fail on fs'es which don't support O_DIRECT, but v2.4 makes such check later at read/write unfortunately ;(
And its too late for changing that IMO...
Hi Marcelo :)
* Marcelo Tosatti <[email protected]> dixit:
Quoted 9 lines
Quoted 5 lines
Shouldn't ext3fs return an error when the
O_DIRECTflag is
used in the open call? Is the open call userspace only and thus only libc can return such error? Am I misunderstanding the entire issue and this is a perfectly legal behaviour (allowing the open, failing in the read operation)?Your interpretation is correct. It would be nicer for
open()to fail on fs'es which don't supportO_DIRECT, but v2.4 makes such check later at read/write unfortunately ;(
Oops :(
Quoted 1 line
And its too late for changing that IMO...
Probably. Anyway, since an userspace app shouldn't bother about
which underlying filesystem a file is under, ext3 should:
- fail in the open() call: OK, it's too late for that.
- don't check while in read()/write(): I'm not sure about this.
The problem I see is that I can't tell if (given that probably
the bug cannot be fixed right now) it's better to let the userspace app believe that O_DIRECT is honored but silently ignore it, or let the userspace believe that O_DIRECT was honored in the open() call and make all subsequent calls to read()/write() fail.
Myself, I would prefer to be deceived and have successful calls
even if the O_DIRECT flag was ignored instead of having successful calls to open(O_DIRECT) but failures on subsequent read()'s, but I must confess that I don't know what kind of scenarios need the use of O_DIRECT and I don't know if having O_DIRECT accepted but ignored is a good thing :(
I'm not familiar with the ext3 code, so I don't know if it's easy
to modify it so it will reject an open if O_DIRECT is specified :(((
Thanks for your answer, Marcelo :)
Raúl Núñez de Arenas Coronado
Signature · 3 lines
Linux Registered User 88736 | http://www.dervishd.net http://www.pleyades.net & http://www.gotesdelluna.net It's my PC and I'll cry if I want to... RAmen!
Quoted 9 lines · DervishD
Hi Marcelo :)
* Marcelo Tosatti <[email protected]> dixit:
Quoted 6 lines
Quoted 5 lines
Shouldn't ext3fs return an error when the
O_DIRECTflag is
used in the open call? Is the open call userspace only and thus only libc can return such error? Am I misunderstanding the entire issue and this is a perfectly legal behaviour (allowing the open, failing in the read operation)?
Quoted 5 lines
Quoted 3 lines
Your interpretation is correct. It would be nicer for
open()to fail on fs'es which don't supportO_DIRECT, but v2.4 makes such check later at read/write unfortunately ;(Oops :(
Nothing else really make sense due to fcntl...
fcntl(fd, F_SETFL, O_DIRECT);
...can happen at any time, to enable/disable direct I/O.
cheers.
Signature · 1 line
Nathan
Quoted 9 lines · Nathan Scott
Quoted 5 lines
Quoted 3 lines
Your interpretation is correct. It would be nicer for
open()to fail on fs'es which don't supportO_DIRECT, but v2.4 makes such check later at read/write unfortunately ;(Oops :(Nothing else really make sense due to fcntl...
fcntl(fd, F_SETFL, O_DIRECT);...can happen at any time, to enable/disable direct I/O.
Actually everytime the O_DIRECT feaure is enabled (open() or fctnl()) you could fail, why not?
Gruss
Bernd
Hi Nathan :)
* Nathan Scott <[email protected]> dixit:
Quoted 12 lines · DervishD
Quoted 11 lines
* Marcelo Tosatti <[email protected]> dixit:
Quoted 9 lines
Quoted 5 lines
Shouldn't ext3fs return an error when the
O_DIRECTflag is
used in the open call? Is the open call userspace only and thus only libc can return such error? Am I misunderstanding the entire issue and this is a perfectly legal behaviour (allowing the open, failing in the read operation)?Your interpretation is correct. It would be nicer for
open()to fail on fs'es which don't supportO_DIRECT, but v2.4 makes such check later at read/write unfortunately ;(
Quoted 5 lines
Quoted 1 line
Oops :(Nothing else really make sense due to fcntl...
fcntl(fd, F_SETFL, O_DIRECT);...can happen at any time, to enable/disable direct I/O.
I know, but that fcntl call should fail just like the open() one.
I mean, I don't find this very different, it's just another point where the flag can be activated and so it should fail if the underlying filesystem doesn't support it (and doesn't ignore it in read()/write()).
Raúl Núñez de Arenas Coronado
Signature · 3 lines
Linux Registered User 88736 | http://www.dervishd.net http://www.pleyades.net & http://www.gotesdelluna.net It's my PC and I'll cry if I want to... RAmen!
Quoted 1 line · DervishD
Hi Nathan :)
Hi there,
Quoted 13 lines
* Nathan Scott <[email protected]> dixit:
Quoted 11 lines · DervishD
Quoted 6 lines
* Marcelo Tosatti <[email protected]> dixit:
Quoted 3 lines
Your interpretation is correct. It would be nicer for
open()to fail on fs'es which don't supportO_DIRECT, but v2.4 makes such check later at read/write unfortunately ;(Oops :(Nothing else really make sense due to fcntl...
fcntl(fd, F_SETFL, O_DIRECT);...can happen at any time, to enable/disable direct I/O.
Quoted 5 lines
I know, but that fcntl call should fail just like the
open()one.
I mean, I don't find this very different, it's just another point where the flag can be activated and so it should fail if the underlying filesystem doesn't support it (and doesn't ignore it in read()/write()).
Problem is there is no way to know whether the underlying fs supports direct IO or not here (fcntl is implemented outside the filesystem, entirely). Which is not unfixable in itself (could use a superblock flag or something similar) but it's way out of scope for the sort of change going into 2.4 these days.
cheers.
Signature · 1 line
Nathan
Hi Nathan :)
* Nathan Scott <[email protected]> dixit:
Quoted 10 lines
Quoted 9 lines
Quoted 3 lines
Nothing else really make sense due to fcntl...
fcntl(fd, F_SETFL, O_DIRECT);...can happen at any time, to enable/disable direct I/O.
I know, but that fcntl call should fail just like the
open()one.
I mean, I don't find this very different, it's just another point where the flag can be activated and so it should fail if the underlying filesystem doesn't support it (and doesn't ignore it in read()/write()).
Quoted 3 lines
Problem is there is no way to know whether the underlying fs supports direct IO or not here (fcntl is implemented outside the filesystem, entirely).
I thought that it was implemented per filesystem.
Quoted 3 lines
Which is not unfixable in itself (could use a superblock flag or something similar) but it's way out of scope for the sort of change going into 2.4 these days.
Which approach does 2.6 kernel use? O_DIRECT is correctly handled
for ext3 there, AFAIK :? Are the differences too large?
I know that this change would be intrusive and probably large,
but IMHO is a quite important bug, because it prevents apps to selectively disable O_DIRECT (the flag is accepted by open(), so there's no reason the app should bother about which caused the read()/write() failures. In fact, is very difficult to know that those failures are caused by partial/buggy support of O_DIRECT flag).
Thanks for the information! :)
Raúl Núñez de Arenas Coronado
Signature · 3 lines
Linux Registered User 88736 | http://www.dervishd.net http://www.pleyades.net & http://www.gotesdelluna.net It's my PC and I'll cry if I want to... RAmen!
Quoted 2 lines · DervishD
...
Are the differences too large?
Yep.
Quoted 6 lines
I know that this change would be intrusive and probably large,
but IMHO is a quite important bug, because it prevents apps to selectively disableO_DIRECT(the flag is accepted byopen(), so there's no reason the app should bother about which caused the read()/write() failures. In fact, is very difficult to know that those failures are caused by partial/buggy support ofO_DIRECTflag).
You could open for direct, do a direct read, and see if it fails. If it fails, clear O_DIRECT on the fd via fcntl(F_SETFL) then do regular buffered IO instead... a bit hacky, but should work fine I think.
cheers.
Signature · 1 line
Nathan