fsync, rename, O_ATOMIC/O_PONIES
Hi XFS devs,
I'm a bit late to the party, but I'd like to respond to some comments at http://lwn.net/Articles/476263/#Comments
http://lwn.net/Articles/484709/
Quoted 5 lines
Could any of the fsync advocates post real code that does the atomic variant of open, write, close?
Hint: it's not possible without tons of regressions.
Linux devs should really provide a proper solution (like
O_ATOMIC) instead of blaming app devs for not doing the impossible.
I'd like to ask:
- Is there a tool to log all unsafe operations?
- What is the *right* way to update a file?
I've asked the last question multiple times, but nobody has been able to give me a proper answer.
Greetings,
Olaf
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 2 lines
I'd like to ask:
- Is there a tool to log all unsafe operations?
What is an unsafe operation? Anything that's dumb and might lose data?
The possibilities are sheer endless.
Quoted 1 line
- What is the *right* way to update a file?
fd = open(tmpfile, ...);
write(fd, ...); // or any other update
fdatasync(fd);
rename(tmpfile, realfile);
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 4 lines · Christoph Hellwig
Quoted 2 lines
I'd like to ask:
- Is there a tool to log all unsafe operations?What is an unsafe operation? Anything that's dumb and might lose data?
For example
Quoted 1 line
The possibilities are sheer endless.
And?
Quoted 6 lines
Quoted 1 line
- What is the *right* way to update a file?
fd = open(tmpfile, ...); write(fd, ...); // or any other update fdatasync(fd); rename(tmpfile, realfile);
Argh, come on.
That's not real and it's not complete. tmpfile is undefined, errors aren't handled and you have lots of unlisted assumptions or regressions.
Olaf
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 19 lines · Olaf van der Spek
Quoted 4 lines · Christoph Hellwig
Quoted 2 lines
I'd like to ask:
- Is there a tool to log all unsafe operations?What is an unsafe operation? Anything that's dumb and might lose data?
For example
Quoted 1 line
The possibilities are sheer endless.
And?
Quoted 6 lines
Quoted 1 line
- What is the *right* way to update a file?
fd =
open(tmpfile, ...);
write(fd, ...); // or any other update
fdatasync(fd);
rename(tmpfile, realfile);
Quoted 4 lines
Argh, come on.
That's not real and it's not complete. tmpfile is undefined, errors aren't handled and you have lots of unlisted assumptions or regressions.
The above is perfectly reasonable psuedo code for quickly describing how to safely overwriting a file. If you want to know about error handling and assumptions, read the man pages for operation.
To make it easy for you, I'll just point out this has already been dealt with early in in the LWN thread for that article, here's the link to the comment and the relevant LWN article providing you with all the information you want:
http://lwn.net/Articles/476606/
https://lwn.net/Articles/457667/
And that my stance on this atomic rename subject is simply this: If we want to change reanme behaviour, then the filesystem is not the right place to do it - atomic rename semantics need to be defined and enforced at the VFS. See here:
http://lwn.net/Articles/479152/
This has all been hashed out before - we don't need to revisit this discussion here. If you want to solve the problem once and for all, submit patches to LKML to implement the new rename semantics in a new syscall....
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 8 lines · Dave Chinner
Quoted 4 lines
Argh, come on.
That's not real and it's not complete. tmpfile is undefined, errors aren't handled and you have lots of unlisted assumptions or regressions.The above is perfectly reasonable psuedo code for quickly describing how to safely overwriting a file. If you want to know about error handling and assumptions, read the man pages for operation.
But I don't have a psuedo code compiler. Using psuedo code hides complexity and bugs. Even the code from Jeff Moyer in the article you're refering too contained bugs.
Don't you think it's quite strange there's no real code available to handle this widespread problem?
Quoted 7 lines
To make it easy for you, I'll just point out this has already been dealt with early in in the LWN thread for that article, here's the link to the comment and the relevant LWN article providing you with all the information you want:
http://lwn.net/Articles/476606/
https://lwn.net/Articles/457667/
It also doesn't handle the assumptions / regressions: http://lwn.net/Articles/485182/
Assuming the target is not a symlink to a different volume
Assuming you are allowed to create the tmp file
Assuming you are allowed to overwrite an existing file having the same name as your tmp file Assuming it's ok to reset meta-data, like file owner, permissions, acls, creation timestamp, etc. Assuming the performance regression due to fsync is ok (request was for atomic, not durable)
Quoted 6 lines
And that my stance on this atomic rename subject is simply this: If we want to change reanme behaviour, then the filesystem is not the right place to do it - atomic rename semantics need to be defined and enforced at the VFS. See here:
I don't think rename is part of the solution.
Olaf
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 10 lines · Olaf van der Spek
Quoted 8 lines · Dave Chinner
Quoted 4 lines
Argh, come on.
That's not real and it's not complete. tmpfile is undefined, errors aren't handled and you have lots of unlisted assumptions or regressions.The above is perfectly reasonable psuedo code for quickly describing how to safely overwriting a file. If you want to know about error handling and assumptions, read the man pages for operation.
Quoted 6 lines
But I don't have a psuedo code compiler. Using psuedo code hides complexity and bugs. Even the code from Jeff Moyer in the article you're refering too contained bugs.
Don't you think it's quite strange there's no real code available to handle this widespread problem?
What you are saying is this:
"I'm too lazy to understand data integrity principles and apply them to my application. Can someone please write some code perfect for my needs so I can copy and paste them into my application."
Stop wasting our time by being obtuse and repeating silly arguments as to why this isn't your problem and instead go read the man pages and modify the example code you've already been pointed at to do exactly what you need.
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs