Samba strict allocate = yes stops btrfs compression working
Hi,
If i set strict allocate = yes in samba to speed up the transfer of a mssql database dump,
then btrfs does not compress the file.
I have tried it also by just copying a small file in Windows to the samba share and the same.
I have tried btrfs mount options autodefrag and then
btrfs fi defrag -c and the file still does not get compressed.
I have tried kernels 3.6.11, 3.8 and 3.10.7 on FC16 and FC18.
Any help, much appreciated.
Thanks.
Mark
Quoted 10 lines · Mark Ridley
Hi,
If i set strict allocate = yes in samba to speed up the transfer of a mssql database dump,
then btrfs does not compress the file.
I have tried it also by just copying a small file in Windows to the samba share and the same.
I have tried btrfs mount options autodefrag and then
btrfs fi defrag -c and the file still does not get compressed.
Quoted 4 lines
I have tried kernels 3.6.11, 3.8 and 3.10.7 on FC16 and FC18.
Any help, much appreciated.
Not sure what strict allocate = yes does, but I assume it probably does fallocate() in which case yeah we aren't going to compress, we'll just write into the preallocated space. We don't support compressed writes into preallocated space ATM, and I'm not sure we ever will. Thanks,
Josef
Quoted 5 lines · Josef Bacik
Not sure what strict allocate = yes does, but I assume it probably does
fallocate()in which case yeah we aren't going to compress, we'll just write
into the preallocated space. We don't support compressed writes into preallocated space ATM, and I'm not sure we ever will. Thanks,
Good to know, this renders btrfs as efficient storage backend for Windows file shares pretty useless. Does this also happen with compress-force?
As a work-around one could write a cronjob that regularly defrags all files changed since the last run with -c option...
Thanks,
Kai
Quoted 11 lines · Kai Krakow
Quoted 5 lines · Josef Bacik
Not sure what strict allocate = yes does, but I assume it probably does
fallocate()in which case yeah we aren't going to compress, we'll just write
into the preallocated space. We don't support compressed writes into preallocated space ATM, and I'm not sure we ever will. Thanks,Good to know, this renders btrfs as efficient storage backend for Windows file shares pretty useless. Does this also happen with compress-force?
Yeah the check is done in the same function, basically it goes
if (prealloc)
do prealloc
else if (compress && (!inode->no_compress || compress_force))
do compress
else
do normal cow
Quoted 3 lines
As a work-around one could write a cronjob that regularly defrags all files changed since the last run with -c option...
Yeah basically. The other option is to just not do strict allocate = yes. Now if it's _not_ doing prealloc then there's probably a problem somewhere, but I assume that is what its doing. Thanks,
Josef
I tried defrag -c and it does nothing to files that have come in with strict allocate = yes.
Quoted 22 lines · Kai Krakow
Quoted 5 lines · Josef Bacik
Not sure what strict allocate = yes does, but I assume it probably does
fallocate()in which case yeah we aren't going to compress, we'll just write
into the preallocated space. We don't support compressed writes into preallocated space ATM, and I'm not sure we ever will. Thanks,Good to know, this renders btrfs as efficient storage backend for Windows file shares pretty useless. Does this also happen with compress-force?
As a work-around one could write a cronjob that regularly defrags all files
changed since the last run with -c option...Thanks,
Kai--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 1 line · Josef Bacik
Not sure what strict allocate = yes does,
I've worked on SMB servers before and can answer that. Historically the way Windows apps (right back into the 16 bit days) have made sure there is space for a file about to be written is to ask the OS to allocate all the space for it. (Unix by default leaves holes making a sparse file.)
For example if a 10MB file is going to be written then an allocation will be done of 10MB. (The exact underlying protocol commands vary, but originally were similar to the Unix seek to end and write.) After that seeks and writes are done. Because the allocation succeeded the app knows that it won't get an out of space error.
Separately from that, it turns out that some filesystems do benefit from preallocating the file to the expected size, and then writing the contents in dribs and drabs into the allocated space.
Consequently Samba gives you the option of really allocating all the file, either for Windows semantics compatibility, or because it results in improved performance on the Unix filesystem.
However I can't see it being of any benefit on a COW filesystem like btrfs.
Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
iEYEARECAAYFAlIXFtsACgkQmOOfHg372QR7cwCggRyQxtxj9E7dNKV94M/Tv5o6 LC0AoN9icJNVxzkV0kDQSgf3Vt0N3g3V
=wBHz
-----END PGP SIGNATURE-----
The main reason I started using strict allocate = yes on samba was out of desperation/exasperation with BTRFS.
BTRFS stalls from time to time causing SAMBA and/or MSSQL to give up on the dump of a database.
Quoted 1 line
From what I have noticed, if for example you dump a 50GB database to samba
without strict allocate, then sometimes the file size gets set to 15GB (for example), MSSQL then fills it in and then the file size jumps up to a next amount and gets filled in.
But a lot of times, especially on a system that has been going for about 6 months and may be it is fragmented, but the next allocation of size just stalls for a while, you cannot even ls the directory.
Either Samba or MSSQL gets fed up of this and the dump fails with a write error.
I am seeing this on 3.3.2, 3.6.11 and even worse on 3.9 and 3.10.
I don't want to try nodatacow (which would probably fix the issue), but you lose compression on the whole filesystem, autodefrag doesn't fix it either.
Any suggestions?
Many thanks.
Quoted 44 lines · Roger Binns
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1Quoted 1 line · Josef Bacik
Not sure what strict allocate = yes does,
I've worked on SMB servers before and can answer that. Historically the way Windows apps (right back into the 16 bit days) have made sure there is space for a file about to be written is to ask the OS to allocate all the space for it. (Unix by default leaves holes making a sparse file.)
For example if a 10MB file is going to be written then an allocation will be done of 10MB. (The exact underlying protocol commands vary, but originally were similar to the Unix seek to end and write.) After that seeks and writes are done. Because the allocation succeeded the app knows that it won't get an out of space error.
Separately from that, it turns out that some filesystems do benefit from preallocating the file to the expected size, and then writing the contents in dribs and drabs into the allocated space.
Consequently Samba gives you the option of really allocating all the file, either for Windows semantics compatibility, or because it results in improved performance on the Unix filesystem.
However I can't see it being of any benefit on a COW filesystem like btrfs.
Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)iEYEARECAAYFAlIXFtsACgkQmOOfHg372QR7cwCggRyQxtxj9E7dNKV94M/Tv5o6 LC0AoN9icJNVxzkV0kDQSgf3Vt0N3g3V
=wBHz
-----END PGP SIGNATURE-------
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 2 lines · Mark Ridley
The main reason I started using strict allocate = yes on samba was out of desperation/exasperation with BTRFS.
The most effective performance option is to turn oplocks on. Opportunistic locks are granted to a client when it is the only one with a file open. That lets it do caching and not even tell the server about record locking. Once another client opens the same file then the first client has to flush all outstanding writes, its caches, record locking etc.
I don't know what Samba currently uses as the default value, but it is traditionally set to off because there are scenarios under which it isn't sufficiently robust (eg access on the Unix server side, a network break when there is outstanding data).
http://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/locking.html#id2616903
Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
iEYEARECAAYFAlIXuX0ACgkQmOOfHg372QSRwgCgwwwoo7QqRql8+CS5xggRBVRt krAAn1wFiIIliZJ7WbWh/oh8crEWLgfR
=HZG9
-----END PGP SIGNATURE-----
A message that is not in the archive (sent off-list or not yet received): CE3CD861.3C3F5%mark@backupsystems.
co.uk
Mark Ridley posted on Fri, 23 Aug 2013 09:20:04 +0100 as excerpted:
Quoted 3 lines
I don't want to try nodatacow (which would probably fix the issue), but you lose compression on the whole filesystem, autodefrag doesn't fix it either.
I don't do servantware (in the context of my sig) and thus don't do samba here, so I'll just black-box that side of things entirely.
But from the btrfs side, are you considering nodatacow at the filesystem or individual file level? It seems doing it at the individual file level might be what you need.
Note that nodatacow must be set on the file at creation in ordered to work, which is impractical to do directly. However, if you can arrange for the files in question to appear in a particular directory, you can set the nodatacow attribute on the directory, and files created within it will inherit that.
It seems to me that should do what you need, PROVIDED that you can arrange for the files to appear in a particular dir/dir-tree, not more or less randomly written throughout the entire (btrfs) filesystem.
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
That would be fine, but nodatacow (according to the btrfs wiki) stops compression, so I might as well get the speed benefits of 'strict allocate = yes' which also disables compression.
If you want to use BTRFS to store backups then compression has be turned on.
Database files like MSSQL usually compress down by 7-10 times, so its a shame to not get this benefit.
Quoted 32 lines · Duncan
Mark Ridley posted on Fri, 23 Aug 2013 09:20:04 +0100 as excerpted:
Quoted 3 lines
I don't want to try nodatacow (which would probably fix the issue), but you lose compression on the whole filesystem, autodefrag doesn't fix it either.
I don't do servantware (in the context of my sig) and thus don't do samba here, so I'll just black-box that side of things entirely.
But from the btrfs side, are you considering nodatacow at the filesystem or individual file level? It seems doing it at the individual file level might be what you need.
Note that nodatacow must be set on the file at creation in ordered to work, which is impractical to do directly. However, if you can arrange for the files in question to appear in a particular directory, you can set the nodatacow attribute on the directory, and files created within it will inherit that.
It seems to me that should do what you need, PROVIDED that you can arrange for the files to appear in a particular dir/dir-tree, not more or less randomly written throughout the entire (btrfs) filesystem.
Signature · 8 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi,
Quoted 3 lines
The speed improvement for dumping large databases through samba with strict allocate = yes to BTRFS was amazing. It reduced a 1 hour dump down to 20 minutes.
What you want btrfs to do is to allocate a file of fixed-size on disk in advance, without knowing how large the file will be after compression (which is the only thing of interest to btrfs). So the whole idea doesn't make a lot of sence.
Sure, you could generate a 50gb file filled with zeros, but once you put real data in it, it will fragment a lot - probably even more than the file which was created without strict allocate.
Regards
The speed improvement for dumping large databases through samba with strict allocate = yes to BTRFS was amazing. It reduced a 1 hour dump down to 20 minutes.
Quoted 44 lines · Roger Binns
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1Quoted 1 line · Josef Bacik
Not sure what strict allocate = yes does,
I've worked on SMB servers before and can answer that. Historically the way Windows apps (right back into the 16 bit days) have made sure there is space for a file about to be written is to ask the OS to allocate all the space for it. (Unix by default leaves holes making a sparse file.)
For example if a 10MB file is going to be written then an allocation will be done of 10MB. (The exact underlying protocol commands vary, but originally were similar to the Unix seek to end and write.) After that seeks and writes are done. Because the allocation succeeded the app knows that it won't get an out of space error.
Separately from that, it turns out that some filesystems do benefit from preallocating the file to the expected size, and then writing the contents in dribs and drabs into the allocated space.
Consequently Samba gives you the option of really allocating all the file, either for Windows semantics compatibility, or because it results in improved performance on the Unix filesystem.
However I can't see it being of any benefit on a COW filesystem like btrfs.
Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)iEYEARECAAYFAlIXFtsACgkQmOOfHg372QR7cwCggRyQxtxj9E7dNKV94M/Tv5o6 LC0AoN9icJNVxzkV0kDQSgf3Vt0N3g3V
=wBHz
-----END PGP SIGNATURE-------
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html