o_sync in vfat driver
Subsystems (3)
*** Is that aim being achieved by the current policy? ***
As I understand it the old (<=2.6.11) sync model kept the data in sync without updating the FAT until later. This runs the risks of partial corruption of one or more files on pullout.
The new model attempts to be more rigourous by updating the FAT every time a block of data is written. Thus the "hammering" of the physical memory hosting the FAT record.
In view of the nature of flash memory this may actually be drastically increasing the chance that the whole FAT gets erased.
part IV (end of a sage)
If a pullout occurs during write , there is now a near 50% chance that this takes out the entire FAT.
It would seem that the main advantage of this scheme is that it is so slow that it encourages users to turn it off. Presumably in the process of coming to that conclusion they will become aware of the need to run umount or the sync command before doing removing the device.
= Danger of destroying hardware =
It seems that there are well documented cases of this abusive rewriting of the FAT causing rapid and total premature failure of what Alan Cox refers to as "ultra-crap devices".
There may be valid reasons of cost or miniaturisation that preclude the additional hardware found in more complex devices.
Even if better quality devices may have some sort of paging mechanism which makes them more resistant to this sort of abuse, it does not seem good engineering practice to dismiss those that fail as "shite".
There is nothing in the spec of vfat that suggests the FAT will be written 10.000 during the writing of one large file. Indeed it is hard to imagine that any other implementation on any other OS or any previous linux kernel behaves like that.
So should the hardware manufacturers have anticipated this particular driver implementation or should the kernel be more aware of the existing hardware that it purports to support.
= The way forward =
It would seem that the first step could be to revert to the 2.6.11 behaviour which was more appropriate and probably safer even from the data point of view.
I lack the knowlege and experience to produce reliable kernel code so I wont try. However, I have already seen a number of suggestions of how the old model could be improved. This post could be the starting point for a discussion of more robust techniques. In any case the coding is unlikely to be very complex given the existing , tested code base that is in place in 2.6.11
Any new technique should probably aim to be applicable to larger devices as well. The 2G limit is artificial and is a tacit recognition of the precarity of the current code. USB hard disks are just as prone to accidental cable pullout. Some periodic or per file sync should probably be envisaged for the VFAT sync mount option.
PS if anyone can tell me why I had to post this ten times and chop it into little bits it would be appreciated in not messing up the list in the future.
I spent an hour reading the faq and I dont see anything taboo here.
Quoted 5 lines · [email protected] wrote:
There is nothing in the spec of vfat that suggests the FAT will be written 10.000 during the writing of one large file. Indeed it is hard to imagine that any other implementation on any other OS or any previous linux kernel behaves like that.
We sync the file metadata once per write() syscall. If your app writes a large file in lots of little bits, it'll do a lot of syncs. Other implementations of fatfs will (must) do the same thing.
Quoted 3 lines
It would seem that the first step could be to revert to the 2.6.11 behaviour which was more appropriate and probably safer even from the data point of view.
fatfs used to be buggy - it didn't implement `-o sync'. Now it does, and what we're seeing is the fallout from the late fixing of that bug.
You're right - people need to understand what they're doing, make their own decision, then remove the `-o sync' option. There aren't any easy solutions.
Thanks for the reply.
Quoted 10 lines · Andrew Morton
Quoted 8 lines · [email protected] wrote:
There is nothing in the spec of vfat that suggests the FAT will be written
10.000 during the writing of one large file. Indeed it is hard to imagine
that any other implementation on any other OS or any previous linux kernel
behaves like that.
Quoted 3 lines
We sync the file metadata once per
write()syscall. If your app writes a large file in lots of little bits, it'll do a lot of syncs. Other implementations of fatfs will (must) do the same thing.
That would not seem to be the case at least on MS systems. I had a freind do some timings copying a large group of files to a 128M usb flash device. There was an arbitary mix of files including many small files and some larger files, one in excess of 50MB.
suse10 default 4m10
win2k 2m30
suse w/o sync 30s
The suse test was drag and drop in konqueror , the other dnd in windows explorer.
Quoted 9 lines
Quoted 5 lines
It would seem that the first step could be to revert to the 2.6.11 behaviour which was more appropriate and probably safer even from the data
point of view.fatfs used to be buggy - it didn't implement `-o sync'. Now it does, and what we're seeing is the fallout from the late fixing of that bug.
I just tested on my 2.6.11 kernel which would predate the change and there is a clear difference between mounting my usb device with and without sync option.
ls -ail /tmpd/mail*
239151 -rw-r--r-- 1 root root 8169540 2006-02-27 19:04 /tmpd/mail-bak.2006-02-28.bz2
bash-3.1#time cp !$ /mnt/usb
time cp /tmpd/mail* /mnt/usb
real 0m0.227s user 0m0.001s sys 0m0.070s
It returns immediately with no disk activity. About 30s later there was disk activity. Presumably some periodic flushing of IO buffers.
bash-3.1#umount /mnt/usb
bash-3.1#mount -o sync !$
bash-3.1#time cp /tmpd/mail* /mnt/usb
real 0m5.440s user 0m0.000s sys 0m0.143s
So the older model did seem to have some sync functionality , tho' presumably not the agressive one-for-one sync that is now being used.
Please correct me if my interpretation is flawed here:
flash has to be cleared before being written. If metadata is written with every block output with write(), the risk of erasing the FAT is now many times higher than with the old sync policy.
So the newer sync policy drastically _reduces_ the data security in the case of untimely disconnection despite the speed penalty and possible hardware damage it incurs.
A less rigourous sync policy may in fact be more appropriate than the current model.
Thanks again.
[Note: I am not subscribed to LKML, if you wish me to recieve any follow ups please BCC: col-pepper at piments point com . thx]
Quoted 6 lines · [email protected] wrote:
That would not seem to be the case at least on MS systems. I had a freind
do some timings copying a large group of files to a 128M usb flash device. There was an arbitary mix of files including many small files and some larger files, one in excess of 50MB.
Quoted 6 lines
suse10 default 4m10
win2k 2m30
suse w/o sync 30sThe suse test was drag and drop in konqueror , the other dnd in windows explorer.
We don't know that the same number of same-sized write()s were happening in each case.
There's been some talk about implementing fsync()-on-file-close for this problem, and some protopatches. But nothing final yet.
As Andrew suggested, Ogawa Hirofumi, FAT maintainer post following patch. I may help you. Please check it.
Message-Id: <[email protected]>
Subject: Re: [EXPERIMENT] Add new "flush" option
This adds new "flush" option for hotplug devices.
Current implementation of "flush" option does,
- synchronizing data pages at ->release() (last close(2))
- if user's work seems to be done (fs is not active), all metadata syncs by pdflush()
This option would provide kind of sane progress, and dirty buffers is flushed more frequently (if fs is not active). This option doesn't provide any robustness (robustness is provided by other options), but probably the option is proper for hotplug devices.
(Please don't assume that dirty buffers is synchronized at any point. This implementation will be changed easily.)
Quoted 80 lines · [email protected] wrote:
Thanks for the reply.
Quoted 9 lines · Andrew Morton
Quoted 5 lines · [email protected] wrote:
There is nothing in the spec of vfat that suggests the FAT will be written 10.000 during the writing of one large file. Indeed it is hard to imagine that any other implementation on any other OS or any previous linux kernel behaves like that.
Quoted 3 lines
We sync the file metadata once per
write()syscall. If your app writes a large file in lots of little bits, it'll do a lot of syncs. Other implementations of fatfs will (must) do the same thing.That would not seem to be the case at least on MS systems. I had a freind do some timings copying a large group of files to a 128M usb flash device. There was an arbitary mix of files including many small files and some larger files, one in excess of 50MB.
suse10 default 4m10
win2k 2m30
suse w/o sync 30sThe suse test was drag and drop in konqueror , the other dnd in windows explorer.
Quoted 9 lines
Quoted 4 lines
It would seem that the first step could be to revert to the 2.6.11 behaviour which was more appropriate and probably safer even from the data point of view.
fatfs used to be buggy - it didn't implement `-o sync'. Now it does, and what we're seeing is the fallout from the late fixing of that bug.
I just tested on my 2.6.11 kernel which would predate the change and there is a clear difference between mounting my usb device with and without sync option.
ls -ail /tmpd/mail*
239151 -rw-r--r-- 1 root root 8169540 2006-02-27 19:04 /tmpd/mail-bak.2006-02-28.bz2 bash-3.1#time cp !$/mnt/usb
time cp /tmpd/mail*/mnt/usbreal 0m0.227s user 0m0.001s sys 0m0.070sIt returns immediately with no disk activity. About 30s later there was disk activity. Presumably some periodic flushing of IO buffers.
bash-3.1#umount
/mnt/usb
bash-3.1#mount -o sync !$bash-3.1#time cp /tmpd/mail*
/mnt/usbreal 0m5.440s user 0m0.000s sys 0m0.143sSo the older model did seem to have some sync functionality , tho' presumably not the agressive one-for-one sync that is now being used.
Please correct me if my interpretation is flawed here:
flash has to be cleared before being written. If metadata is written with every block output with
write(), the risk of erasing the FAT is now many times higher than with the old sync policy.So the newer sync policy drastically _reduces_ the data security in the case of untimely disconnection despite the speed penalty and possible hardware damage it incurs.
A less rigourous sync policy may in fact be more appropriate than the current model.
Thanks again.
[Note: I am not subscribed to LKML, if you wish me to recieve any follow ups please BCC: col-pepper at piments point com . thx]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 2 lines
Hiroyuki Machida [email protected] SSW Dept. HENC, Sony Corp.
Quoted 5 lines · Andrew Morton
We don't know that the same number of same-sized write()s were happening in each case.
There's been some talk about implementing fsync()-on-file-close for this problem, and some protopatches. But nothing final yet.
Here's the patch I'm using in -suse right now. What I want to do is make a much more generic -o flush, but it'll still need a few bits in individual filesystem to kick off metadata writes quickly.
The basic goal behind the code is to trigger writes without waiting for both data and metadata. If the user is watching the memory stick, when the little light stops flashing all the data and metadata will be on disk.
It also generally throttles userland a little during file release. This could be changed to throttle for each page dirtied, but most users I asked liked the current setup better.
-chris
From: Chris Mason <[email protected]>
Subject: add -o flush for fat
Fat is commonly used on removable media, mounting with -o flush tells the FS to write things to disk as quickly as possible. It is like -o sync, but much faster (and not as safe).
fs/fat/file.c +19 -0
@@ -13,6 +13,7 @@#include <linux/smp_lock.h> #include <linux/buffer_head.h> #include <linux/writeback.h> +#include <linux/blkdev.h> int fat_generic_ioctl (struct inode *inode,struct file *filp,unsigned int cmd,unsigned long arg)
@@ -112,6 +113,19 @@ int fat_generic_ioctl(struct inode *inod } } +static int +fat_file_release (struct inode *inode,struct file *filp) +{ + +if ((filp->f_mode &FMODE_WRITE ) && +MSDOS_SB (inode->i_sb)->options.flush) { +writeback_inode (inode); +writeback_bdev (inode->i_sb); +blk_congestion_wait (WRITE ,HZ /10 ); + } +return 0 ; +} +struct file_operations fat_file_operations = { .llseek = generic_file_llseek, .read = do_sync_read,
@@ -121,6 +135,7 @@ struct file_operations fat_file_operatio .aio_read = generic_file_aio_read, .aio_write = generic_file_aio_write, .mmap = generic_file_mmap, + .release = fat_file_release, .ioctl = fat_generic_ioctl, .fsync = file_fsync, .sendfile = generic_file_sendfile,
@@ -293,6 +308,10 @@ void fat_truncate(struct inode *inode)lock_kernel ();fat_free (inode, nr_clusters);unlock_kernel (); +if (MSDOS_SB (inode->i_sb)->options.flush) { +writeback_inode (inode); +writeback_bdev (inode->i_sb); + } }struct inode_operations fat_file_inode_operations = {
fs/fat/inode.c +7 -2
@@ -24,6 +24,7 @@#include <linux/vfs.h> #include <linux/parser.h> #include <linux/uio.h> +#include <linux/writeback.h> #include <asm/unaligned.h> #ifndef CONFIG_FAT_DEFAULT_IOCHARSET
@@ -860,7 +861,7 @@ enum { Opt_charset, Opt_shortname_lower, Opt_shortname_win95, Opt_shortname_winnt, Opt_shortname_mixed, Opt_utf8_no, Opt_utf8_yes, Opt_uni_xl_no, Opt_uni_xl_yes, Opt_nonumtail_no, Opt_nonumtail_yes, - Opt_obsolate, Opt_err, + Opt_obsolate, Opt_flush, Opt_err, }; static match_table_t fat_tokens = {
@@ -892,7 +893,8 @@ static match_table_t fat_tokens = { {Opt_obsolate, "cvf_format=%20s"}, {Opt_obsolate, "cvf_options=%100s"}, {Opt_obsolate, "posix"}, - {Opt_err, NULL} + {Opt_flush, "flush"}, + {Opt_err, NULL}, };static match_table_t msdos_tokens = { {Opt_nodots, "nodots"},
@@ -1033,6 +1035,9 @@ static int parse_options(char *options,return 0 ; opts->codepage = option;break ; +case Opt_flush: + opts->flush =1 ; +break ;/* msdos specific */ case Opt_dots:
fs/fs-writeback.c +23 -0
@@ -390,6 +390,29 @@ sync_sb_inodes(struct super_block *sb, sreturn ;/* Leave any unwritten inodes on s_io */ } +void +writeback_bdev (struct super_block *sb) +{ +struct address_space *mapping = sb->s_bdev->bd_inode->i_mapping; +filemap_flush (mapping); +blk_run_address_space (mapping); +} +EXPORT_SYMBOL_GPL (writeback_bdev); + +void +writeback_inode (struct inode *inode) +{ + +struct address_space *mapping = inode->i_mapping; +struct writeback_control wbc = { + .sync_mode =WB_SYNC_NONE , + .nr_to_write =0 , + }; +sync_inode (inode, &wbc); +filemap_fdatawrite (mapping); +} +EXPORT_SYMBOL_GPL (writeback_inode); +/* * Start writeback of dirty pagecache data against all unlocked inodes. *
fs/msdos/namei.c +27 -1
@@ -11,6 +11,7 @@#include <linux/buffer_head.h> #include <linux/msdos_fs.h> #include <linux/smp_lock.h> +#include <linux/writeback.h> /* MS-DOS "device special files" */ static const unsigned char *reserved_names[] = {
@@ -293,7 +294,7 @@ static int msdos_create(struct inode *di struct nameidata *nd) {struct super_block *sb = dir->i_sb; -struct inode *inode; +struct inode *inode =NULL ;struct fat_slot_info sinfo;struct timespec ts;unsigned char msdos_name[MSDOS_NAME ];
@@ -329,6 +330,11 @@ static int msdos_create(struct inode *did_instantiate (dentry, inode); out:unlock_kernel (); +if (!err &&MSDOS_SB (sb)->options.flush) { +writeback_inode (dir); +writeback_inode (inode); +writeback_bdev (sb); + }return err; }
@@ -361,6 +367,11 @@ static int msdos_rmdir(struct inode *dirfat_detach (inode); out:unlock_kernel (); +if (!err &&MSDOS_SB (inode->i_sb)->options.flush) { +writeback_inode (dir); +writeback_inode (inode); +writeback_bdev (inode->i_sb); + }return err; }
@@ -414,6 +425,11 @@ static int msdos_mkdir(struct inode *dird_instantiate (dentry, inode);unlock_kernel (); +if (MSDOS_SB (sb)->options.flush) { +writeback_inode (dir); +writeback_inode (inode); +writeback_bdev (sb); + }return 0 ; out_free:
@@ -443,6 +459,11 @@ static int msdos_unlink(struct inode *difat_detach (inode); out:unlock_kernel (); +if (!err &&MSDOS_SB (inode->i_sb)->options.flush) { +writeback_inode (dir); +writeback_inode (inode); +writeback_bdev (inode->i_sb); + }return err; }
@@ -648,6 +669,11 @@ static int msdos_rename(struct inode *ol new_dir, new_msdos_name, new_dentry, is_hid); out:unlock_kernel (); +if (!err &&MSDOS_SB (old_dir->i_sb)->options.flush) { +writeback_inode (old_dir); +writeback_inode (new_dir); +writeback_bdev (old_dir->i_sb); + }return err; }
include/linux/msdos_fs.h +1 -0
@@ -203,6 +203,7 @@ struct fat_mount_options { unicode_xlate:1, /* create escape sequences for unhandled Unicode */ numtail:1, /* Does first alias have a numeric '~1' type tail? */ atari:1, /* Use Atari GEMDOS variation of MS-DOS fs */ + flush:1, /* write things quickly */ nocase:1; /* Does this need case conversion? 0=need case conversion*/ };
include/linux/writeback.h +2 -0
@@ -68,6 +68,8 @@ int inode_wait(void *);int inode_wait (void *);void sync_inodes_sb (struct super_block *,int wait);void sync_inodes (int wait); +void writeback_bdev (struct super_block *); +void writeback_inode (struct inode *);/* writeback.h requires fs.h; it, too, is not included from here. */ static inline void wait_on_inode (struct inode *inode)
Quoted 23 lines · Chris Mason
Quoted 5 lines · Andrew Morton
We don't know that the same number of same-sized write()s were happening in each case.
There's been some talk about implementing fsync()-on-file-close for this problem, and some protopatches. But nothing final yet.
Here's the patch I'm using in -suse right now. What I want to do is make a much more generic -o flush, but it'll still need a few bits in individual filesystem to kick off metadata writes quickly.
The basic goal behind the code is to trigger writes without waiting for both data and metadata. If the user is watching the memory stick, when the little light stops flashing all the data and metadata will be on disk.
It also generally throttles userland a little during file release. This could be changed to throttle for each page dirtied, but most users I asked liked the current setup better.
...
fs/fat/file.c int fat_generic_ioctl(struct inode *inod · line 117
Quoted 2 lines
fs/fat/file.c +2
-0+static int +fat_file_release (struct inode *inode,struct file *filp)
On a single line, please.
fs/fat/file.c void fat_truncate(struct inode *inode) · line 311
Quoted 1 line
fs/fat/file.c +1
-0+if (MSDOS_SB (inode->i_sb)->options.flush) {
Did you consider making `-o flush' a generic mount option rather than msdos-only?
I guess there isn't a lot of demand for this for other filesystems, and having an ignored option like this is a bit misleading...
fs/fs-writeback.c sync_sb_inodes(struct super_block *sb, s · line 412
Quoted 11 lines
fs/fs-writeback.c +11
-0+void +writeback_inode (struct inode *inode) +{ + +struct address_space *mapping = inode->i_mapping; +struct writeback_control wbc = { + .sync_mode =WB_SYNC_NONE , + .nr_to_write =0 , + }; +sync_inode (inode, &wbc); +filemap_fdatawrite (mapping);
I think that filemap_fdatawrite() will be a no-op?
Quoted 2 lines · Andrew Morton
On a single line, please.
Ack.
Quoted 4 lines
fs/fat/file.c void fat_truncate(struct inode *inode) · line 311
Quoted 1 line
fs/fat/file.c +1
-0+if (MSDOS_SB (inode->i_sb)->options.flush) {Did you consider making `-o flush' a generic mount option rather than msdos-only?
Yes, long term I think the generic option is better. I have three or so ideas for a generic patch:
1) When the block device leaves congestion, it asks for more io
2) pdflush operation that tries to constantly keep a given block device congested
3) my current patch aggregated to other filesystems that people want -o flush on.
I've made a few stabs at #1, but didn't like the end result. #2 seems like the best choice so far. If I got it working nicely I would add the generic option, otherwise with option #3 it's probably best to keep it per FS.
The main goal for my current patch was to find out if this functionality will actually make people happy (so far the beta testers like it). If the complaints are low, it's worth the time to add something generic.
Quoted 4 lines
I guess there isn't a lot of demand for this for other filesystems, and having an ignored option like this is a bit misleading...
Quoted 13 lines
fs/fs-writeback.c sync_sb_inodes(struct super_block *sb, s · line 412
Quoted 11 lines
fs/fs-writeback.c +11
-0+void +writeback_inode (struct inode *inode) +{ + +struct address_space *mapping = inode->i_mapping; +struct writeback_control wbc = { + .sync_mode =WB_SYNC_NONE , + .nr_to_write =0 , + }; +sync_inode (inode, &wbc); +filemap_fdatawrite (mapping);I think that
filemap_fdatawrite()will be a no-op?
This part is nasty, I want to write all of the file data pages and write the inode without waiting on it. The nr_to_write = 0 will make sure that sync_inode only writes the inode, and WB_SYNC_NONE makes sure it does not wait for that io to finish.
What I really want is WB_SYNC_NONE in mpage_writepages, but I don't want to trigger this code:
if (wbc->sync_mode == WB_SYNC_NONE) {
index = mapping->writeback_index; /* Start from prev offset */So, I use filemap_fdatawrite to make sure all of the data pages get written. It's not perfect, but I was going for minimal changes outside of fat.
-chris
Quoted 16 lines · OGAWA Hirofumi
fs/msdos/namei.c static int msdos_create(struct inode *di · line 339
Quoted 11 lines · Chris Mason
fs/msdos/namei.c +5
-0@@ -329,6 +330,11 @@ static int msdos_create(struct inode *did_instantiate (dentry, inode); out:unlock_kernel (); +if (!err &&MSDOS_SB (sb)->options.flush) { +writeback_inode (dir); +writeback_inode (inode); +writeback_bdev (sb); + }return err; }If buffers is already queued for I/O, and if you don't wait anything, the buffers wouldn't be (re-)submited, then those buffers will be flushing by normal periodically
wb_kupdate()after all.
Just to make sure we're using the same terms, do you mean the pages are marked dirty and on the SB's dirty list, or do you mean the page has been through writepage and is currently on its way to the disk?
Quoted 2 lines
Do you have any plan to address it? Or I'm just missing something?
If you mean the page is just dirty, it will get written by the filemap_fdatawrite calls. If you mean the page is PG_writeback, it is already on the way to the disk, so it passes the 'blinking light on the memory stick' rule.
-chris
Quoted 11 lines · OGAWA Hirofumi
Quoted 3 lines
Just to make sure we're using the same terms, do you mean the pages are marked dirty and on the SB's dirty list, or do you mean the page has been through writepage and is currently on its way to the disk?
The page is already on device's request queue, and the page is already marked a
PG_writeback. And that page is not processed by device yet.Then, you call next
filemap_fdatawrite(), it just re-dirty the page and queues tosb->s_dirty, because the page'sbuffer_headsis still locked. So, the re-dirtyed page is re-submited to device by periodicallywb_kupdate()?
filemap_fdatawrite() won't redirty the page. It will wait on the pending writeback.
-chris
OGAWA Hirofumi
maintainerQuoted 2 lines · Chris Mason
filemap_fdatawrite()won't redirty the page. It will wait on the pending writeback.
Umm... I'm looking the following code.
+ if (MSDOS_SB(sb)->options.flush) {
+ writeback_inode(dir);
+ writeback_inode(inode);
+ writeback_bdev(sb);
+ }
+void
+writeback_bdev(struct super_block *sb)
+{
+ struct address_space *mapping = sb->s_bdev->bd_inode->i_mapping;
+ filemap_flush(mapping);
+ blk_run_address_space(mapping);
+}
+EXPORT_SYMBOL_GPL(writeback_bdev);
filemap_flush() is using WB_SYNC_NONE.
in mpage_writepages()
if (wbc->sync_mode != WB_SYNC_NONE)
wait_on_page_writeback(page);
if (PageWriteback(page) ||
!clear_page_dirty_for_io(page)) {
unlock_page(page);
continue;
}Where does wait it?
--
OGAWA Hirofumi <[email protected]>
Quoted 15 lines · OGAWA Hirofumi
Quoted 2 lines · Chris Mason
filemap_fdatawrite()won't redirty the page. It will wait on the pending writeback.Umm... I'm looking the following code.
fs/fs-writeback.c +8
-0+void +writeback_bdev (struct super_block *sb) +{ +struct address_space *mapping = sb->s_bdev->bd_inode->i_mapping; +filemap_flush (mapping); +blk_run_address_space (mapping); +} +EXPORT_SYMBOL_GPL (writeback_bdev);
Quoted 2 lines
filemap_flush()is usingWB_SYNC_NONE.
Ok, I thought you were asking about the code that called filemap_fdatawrite, which does wait. filemap_flush is used on the underlying block device. In the case of a page that is already under IO, the io is not cancelled but allowed to continue.
This is the desired result. When you're doing a number of operations in sequence, each operation will start io on the block device. If they used filemap_fdatawrite instead of filemap_flush, they would end up being synchronous.
-chris
OGAWA Hirofumi
maintainerQuoted 9 lines · Chris Mason
Ok, I thought you were asking about the code that called
filemap_fdatawrite, which does wait.filemap_flushis used on the underlying block device. In the case of a page that is already under IO, the io is not cancelled but allowed to continue.This is the desired result. When you're doing a number of operations in sequence, each operation will start io on the block device. If they used
filemap_fdatawriteinstead offilemap_flush, they would end up being synchronous.
Of course, I know. Let's return to beginning of this thread, do you have any plan to address it?
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Hi all,
Quoted 9 lines · Chris Mason
Quoted 6 lines · Andrew Morton
We don't know that the same number of same-sized write()s were happening in each case.
There's been some talk about implementing fsync()-on-file-close for this problem, and some protopatches. But nothing final yet.
Quoted 7 lines
Here's the patch I'm using in -suse right now. What I want to do is make a much more generic -o flush, but it'll still need a few bits in individual filesystem to kick off metadata writes quickly.
The basic goal behind the code is to trigger writes without waiting for both
data and metadata. If the user is watching the memory stick, when the little light stops flashing all the data and metadata will be on disk.It also generally throttles userland a little during file release. This could be changed to throttle for each page dirtied, but most users I asked liked the current setup better.
I like the idea and would like to see something like this in mainline.
Here is some non-scientific benchmark done with 2.6.16, comparing default mount and flush mount of a USB2 stick:
///////////////////////////////////////////////////////////////////// Single File "Test": 43MB
$ time cp Test /media/usbdisk/test/ && time umount /media/usbdisk/ /////////////////////////////////////////////////////////////////////
VANILLA:
real 0m3.770s user 0m0.004s sys 0m0.308s
real 0m9.439s user 0m0.000s sys 0m0.040s
FLUSH:
real 0m6.000s user 0m0.012s sys 0m0.400s
real 0m3.668s user 0m0.000s sys 0m0.028s
REAL TIME RATIO (FLUSH/VANILLA):
9.6 / 13.1 = 0.73
///////////////////////////////////////////////////////////////////// Directory Tree "flushtest": 44MB (8866 files, 1820 dirs)
$ time cp -R flushtest/ /media/usbdisk/ && time umount /media/usbdisk/ /////////////////////////////////////////////////////////////////////
VANILLA:
real 0m0.966s user 0m0.024s sys 0m0.860s
real 1m11.962s user 0m0.004s sys 0m0.160s
FLUSH:
real 1m41.645s user 0m0.032s sys 0m1.112s
real 0m4.660s user 0m0.004s sys 0m0.068s
REAL TIME RATIO (FLUSH/VANILLA):
106.3 / 77.9 = 1.36
Quoted 82 lines · Mathis Ahrens
Hi all,
Quoted 15 lines · Chris Mason
Quoted 8 lines · Andrew Morton
We don't know that the same number of same-sized write()s were happening in
each case.There's been some talk about implementing fsync()-on-file-close for this
problem, and some protopatches. But nothing final yet.Here's the patch I'm using in -suse right now. What I want to do is make a much more generic -o flush, but it'll still need a few bits in individual filesystem to kick off metadata writes quickly.
Quoted 9 lines
The basic goal behind the code is to trigger writes without waiting for both
data and metadata. If the user is watching the memory stick, when the little light stops flashing all the data and metadata will be on disk.It also generally throttles userland a little during file release. This could be changed to throttle for each page dirtied, but most users I asked liked the current setup better.
I like the idea and would like to see something like this in mainline.
Here is some non-scientific benchmark done with 2.6.16, comparing default mount and flush mount of a USB2 stick:
///////////////////////////////////////////////////////////////////// Single File "Test": 43MB
$ time cp Test /media/usbdisk/test/ && time umount /media/usbdisk/ /////////////////////////////////////////////////////////////////////VANILLA:
real 0m3.770s user 0m0.004s sys 0m0.308sreal 0m9.439s user 0m0.000s sys 0m0.040sFLUSH:
real 0m6.000s user 0m0.012s sys 0m0.400sreal 0m3.668s user 0m0.000s sys 0m0.028sREAL TIME RATIO (FLUSH/VANILLA):
9.6 / 13.1 = 0.73///////////////////////////////////////////////////////////////////// Directory Tree "flushtest": 44MB (8866 files, 1820 dirs)
$ time cp -R flushtest/ /media/usbdisk/ && time umount /media/usbdisk/ /////////////////////////////////////////////////////////////////////VANILLA:
real 0m0.966s user 0m0.024s sys 0m0.860sreal 1m11.962s user 0m0.004s sys 0m0.160sFLUSH:
real 1m41.645s user 0m0.032s sys 0m1.112s
Quoted 8 lines
real 0m4.660s user 0m0.004s sys 0m0.068sREAL TIME RATIO (FLUSH/VANILLA):
106.3 / 77.9 = 1.36
That's interesting, albeit non-scientific, I think it is quite informative.
There are two basic problems with the current code: speed is down by around and order of magnitude compared to a non-synced write and the fact that the code is hammering the FAT. The two are obviously related.
Viewing the system globally rather than considering the details of the techniques used, it would seem that any algorithm that does not drastically reduce write times, at least on the one large file test , is missing the mark and presumably repeating the problem in a slightly different way.
Not knocking the efforts Chris has put in , it's great to see this is getting some attention, but I think viewing overall performance times as shown above gives a touchstone as to whether any particular proto is effective.
The fact that flush can be almost 40% slower in some cases is worrying.
Thanks for the info.