XFS Preallocate using ALLOCSP
Hi,
I wanted to preallocate the file using the ALLOCSP so that the preallocated space is zeroed out. Hence i used the xfsctl but the problem is that i cannot execute it as a non-root user. So i tried using fallocate but when i saw the xfs implementation of kernel it uses the RESVP cmd and since the unwritten flag is set i am assuming that the performance of writing to the preallocated space will suffer because of the metadata updates. So is there a way to preallocate using ALLOCSP as a non-root user or else i would be required to modify the kernel ??
TIA,
Smit
Signature · 7 lines
View this message in context: http://www.nabble.com/XFS-Preallocate-using-ALLOCSP-tp24042506p24042506.html Sent from the linux-xfs mailing list archive at Nabble.com. _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Ok i did the first part i.e preallocation using xfsctl() by using setresgid().
But still is there some way this option could be incorporated in the fallocate system call.
Quoted 15 lines · Smit Shah
Hi,
I wanted to preallocate the file using the ALLOCSP so that the preallocated space is zeroed out. Hence i used the xfsctl but the problem is that i cannot execute it as a non-root user. So i tried using fallocate but when i saw the xfs implementation of kernel it uses the RESVP cmd and since the unwritten flag is set i am assuming that the performance of writing to the preallocated space will suffer because of the metadata updates. So is there a way to preallocate using ALLOCSP as a non-root user or else i would be required to modify the kernel ??
TIA,
Smit
Signature · 7 lines
View this message in context: http://www.nabble.com/XFS-Preallocate-using-ALLOCSP-tp24042506p24046118.html Sent from the linux-xfs mailing list archive at Nabble.com. _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 11 lines · Smit Shah
Hi,
I wanted to preallocate the file using the ALLOCSP so that the preallocated space is zeroed out. Hence i used the xfsctl but the problem is that i cannot execute it as a non-root user. So i tried using fallocate but when i saw the xfs implementation of kernel it uses the RESVP cmd and since the unwritten flag is set i am assuming that the performance of writing to the preallocated space will suffer because of the metadata updates. So is there a way to preallocate using ALLOCSP as a non-root user or else i would be required to modify the kernel ??
Quoted 2 lines
TIA,
Smit
ALLOCSP simply writes 0s to a file.... you may as well do it using posix calls, as a normal user, I think.
-Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 14 lines · Eric Sandeen
Quoted 3 lines · Smit Shah
Hi,
I wanted to preallocate the file using the ALLOCSP so that the
preallocated
Quoted 2 lines
space is zeroed out. Hence i used the xfsctl but the problem is that i cannot execute it as a non-root user. So i tried using fallocate but
when
Quoted 1 line
i saw the xfs implementation of kernel it uses the RESVP cmd and since
the
Quoted 1 line
unwritten flag is set i am assuming that the performance of writing to
the
Quoted 1 line
preallocated space will suffer because of the metadata updates. So is
there
Quoted 9 lines
Quoted 5 lines
a way to preallocate using ALLOCSP as a non-root user or else i would be required to modify the kernel ??
TIA,
SmitALLOCSP simply writes 0s to a file.... you may as well do it using posix calls, as a normal user, I think.
Even the man page of fallocate says that it allocates and initializes to zero the disk space allocated but when i saw the code i did found out that it does not zero it out. Hence was a kindof confused. So posix_allocate is similar to ALLOCSP when falloc is not supported by the underlying filesystem
that is to ftruncate the file and zero it out. So all of them try to
allocate contiguous blocks but the only difference is when we use the fallocate in ext4/xfs it does not zero out the preallocated space. Am i right ?
But when i fallocate in ext4 i can see the write performance improvement but not in xfs and reason i found out in one of your previous comments is
because of the unwritten flag set in xfs. So how do we see if the unwritten
flag is set or not ? I did use xfs_info but it didnt show any such information.
Thanks,
Smit
Quoted 3 lines
-Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 28 lines · Smit Shah
Quoted 14 lines · Eric Sandeen
Quoted 3 lines · Smit Shah
Hi,
I wanted to preallocate the file using the ALLOCSP so that the
preallocated
Quoted 2 lines
space is zeroed out. Hence i used the xfsctl but the problem is that i cannot execute it as a non-root user. So i tried using fallocate but
when
Quoted 1 line
i saw the xfs implementation of kernel it uses the RESVP cmd and since
the
Quoted 1 line
unwritten flag is set i am assuming that the performance of writing to
the
Quoted 1 line
preallocated space will suffer because of the metadata updates. So is
there
Quoted 9 lines
Quoted 5 lines
a way to preallocate using ALLOCSP as a non-root user or else i would be required to modify the kernel ??
TIA,
SmitALLOCSP simply writes 0s to a file.... you may as well do it using posix calls, as a normal user, I think.
Quoted 9 lines
Even the man page of fallocate says that it allocates and initializes to zero the disk space allocated but when i saw the code i did found out that it does not zero it out. Hence was a kindof confused. So
posix_allocateis similar to ALLOCSP when falloc is not supported by the underlying filesystem
that is to ftruncate the file and zero it out. So all of them try to
allocate contiguous blocks but the only difference is when we use the fallocate in ext4/xfs it does not zero out the preallocated space. Am i right ?
I guess i am not right here ftruncate simply does a lseek and wirtes to it which might not be contiguous whereas fallocate tries to allocate contiguous block so as to reduce fragmentation and hence i thought to reduce fragmentation and for security reasons its better to use ALLOCSP rather than something like ftruncate /posix_fallocate or RSEVSP which kindof performs bad for writes with unwritten flag set and now there being a no direct way while creating the fs to disable unwritten.
Quoted 9 lines
But when i fallocate in ext4 i can see the write performance improvement but not in xfs and reason i found out in one of your previous comments is
because of the unwritten flag set in xfs. So how do we see if the unwritten
flag is set or not ? I did usexfs_infobut it didnt show any such information.
Quoted 9 lines
Thanks,
SmitQuoted 3 lines
-Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 2 lines · Smit Shah
Even the man page of fallocate says that it allocates and initializes to zero the disk space allocated
Bleah, so it does:
FALLOC_FL_KEEP_SIZE
This flag allocates and initializes to zero the disk space
well, that's misleading and/or wrong.
Quoted 7 lines
but when i saw the code i did found out that it does not zero it out. Hence was a kindof confused. So
posix_allocateis similar to ALLOCSP when falloc is not supported by the underlying filesystem that is to ftruncate the file and zero it out. So all of them try to allocate contiguous blocks but the only difference is when we use the fallocate in ext4/xfs it does not zero out the preallocated space. Am i right ?
fallocate / sys_fallocate marks the region as uninitialized so that you get back 0s when you read. It's implemented on xfs, ext4, ocfs2, and btrfs.
posix_fallocate manages to reach sys_fallocate when all the stars align:
kernel, glibc, and filesystgem. Otherwise it writes 0s.
Quoted 2 lines
But when i fallocate in ext4 i can see the write performance improvement but not in xfs
Testing how?
Quoted 4 lines
and reason i found out in one of your previous comments is because of the unwritten flag set in xfs. So how do we see if the unwritten flag is set or not ? I did use
xfs_infobut it didnt show any such information.
ext4 & xfs are doing the same basic thing, they must maintain the unwritten state on the preallocated extents, and manage that as it changes when portions are written with real data.
xfs_bmap -v -v -p on a file will show you extent state for xfs.
Quoted 1 line
I guess i am not right here ftruncate simply does a lseek and wirtes
ftruncate simply sets i_size, it does no data IO.
Quoted 2 lines
to it which might not be contiguous whereas fallocate tries to allocate contiguous block so as to reduce fragmentation
Actually fallocate's only official job is to reserve blocks so you don't get ENOSPC later. Because the request comes in all at once, you are very likely to get an optimal allocation, and that's a nice side effect, but it's not actually required by the interface.
Quoted 2 lines
and hence i
thought to reduce fragmentation and for security reasons
None of these normal interfaces poses any security risk. If you build xfs without the unwritten extent feature you could allocate w/o flagging uninitialized and expose stale data, but that's not a normal mode of operation.
Quoted 5 lines
its better
to use ALLOCSP rather than something like ftruncate /posix_fallocate or RSEVSP which kindof performs bad for writes with unwritten flag set and now there being a no direct way while creating the fs to disable unwritten.
In the end, there are only 2 ways to preallocate blocks: explicitly write 0s, or flag regions as unwritten (as xfs/ext4/... can do). (Ok, or a 3rd sorta-way, which is to reserve w/o flagging, maybe that's what you're looking for, but that's deprecated or not really available at this point).
Maybe I should ask what the end goal is here. :)
-Eric
Quoted 1 line
Thanks, Smit
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 32 lines · Eric Sandeen
Quoted 2 lines · Smit Shah
Even the man page of fallocate says that it allocates and initializes to zero the disk space allocated
Bleah, so it does:
FALLOC_FL_KEEP_SIZE
This flag allocates and initializes to zero the disk
spacewell, that's misleading and/or wrong.
Quoted 7 lines
but when i saw the code i did found out that it does not zero it out. Hence was a kindof confused. So
posix_allocateis similar to ALLOCSP when falloc is not supported by the underlying filesystem that is to ftruncate the file and zero it out. So all of them try to allocate contiguous blocks but the only difference is when we use the fallocate in ext4/xfs it does not zero out the preallocated space. Am i right ?fallocate /
sys_fallocatemarks the region as uninitialized so that you get back 0s when you read. It's implemented on xfs, ext4, ocfs2, and btrfs.
posix_fallocatemanages to reachsys_fallocatewhen all the stars align:
kernel, glibc, and filesystgem. Otherwise it writes 0s.Quoted 2 lines
But when i fallocate in ext4 i can see the write performance improvement but not in xfs
Testing how?
Quoted 8 lines
Quoted 4 lines
and reason i found out in one of your previous comments is because of the unwritten flag set in xfs. So how do we see if the unwritten flag is set or not ? I did use
xfs_infobut it didnt show any such information.ext4 & xfs are doing the same basic thing, they must maintain the unwritten state on the preallocated extents, and manage that as it changes when portions are written with real data.
Well, the difference in managing the unwritten state can theoretically result in different performance. Not that I'd expect ext4 being better than xfs in this respect. More data is needed here.
Quoted 5 lines
xfs_bmap-v -v -p on a file will show you extent state for xfs.Quoted 1 line
I guess i am not right here ftruncate simply does a lseek and wirtes
ftruncate simply sets
i_size, it does no data IO.
... and no block reservation/allocation either.
Quoted 8 lines
Quoted 2 lines
to it which might not be contiguous whereas fallocate tries to allocate contiguous block so as to reduce fragmentation
Actually fallocate's only official job is to reserve blocks so you don't get ENOSPC later. Because the request comes in all at once, you are very likely to get an optimal allocation, and that's a nice side effect, but it's not actually required by the interface.
Quoted 5 lines
Quoted 2 lines
and hence i
thought to reduce fragmentation and for security reasonsNone of these normal interfaces poses any security risk. If you build xfs without the unwritten extent feature
I don't think, it's possible. Not in any configurable way, at least.
Quoted 3 lines
you could allocate w/o flagging
uninitialized and expose stale data, but that's not a normal mode of operation.
That's was possible with mount option unwritten=0, but AFAIK, it's been recently completely removed from code.
Quoted 10 lines
Quoted 7 lines
its better
to use ALLOCSP rather than something like ftruncate /posix_fallocate or RSEVSP which kindof performs bad for writes with unwritten flag set and now there being a no direct way while creating the fs to disable unwritten.In the end, there are only 2 ways to preallocate blocks: explicitly write 0s, or flag regions as unwritten (as xfs/ext4/... can do).
Exactly.
That's a trade-off between spending time on setup or at the write time. And if explicit zeros are desirable for the former approach, it can be driven from the user space (after preallocation) rather than from the kernel with the exactly the same outcome.
Just restating the same what Eric already said :)
Felix
Quoted 7 lines
(Ok,
or a 3rd sorta-way, which is to reserve w/o flagging, maybe that's what you're looking for, but that's deprecated or not really available at this point).Maybe I should ask what the end goal is here. :)
Quoted 8 lines
-Eric
Quoted 1 line
Thanks, Smit
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 29 lines · Felix Blyakher
Quoted 26 lines · Eric Sandeen
Quoted 2 lines · Smit Shah
Even the man page of fallocate says that it allocates and initializes to zero the disk space allocated
Bleah, so it does:
FALLOC_FL_KEEP_SIZE
This flag allocates and initializes to zero the disk
spacewell, that's misleading and/or wrong.
Quoted 7 lines
but when i saw the code i did found out that it does not zero it out. Hence was a kindof confused. So
posix_allocateis similar to ALLOCSP when falloc is not supported by the underlying filesystem that is to ftruncate the file and zero it out. So all of them try to allocate contiguous blocks but the only difference is when we use the fallocate in ext4/xfs it does not zero out the preallocated space. Am i right ?fallocate /
sys_fallocatemarks the region as uninitialized so that you
get back 0s when you read. It's implemented on xfs, ext4, ocfs2, and btrfs.
Quoted 8 lines
Quoted 8 lines
posix_fallocatemanages to reachsys_fallocatewhen all the stars align:
kernel, glibc, and filesystgem. Otherwise it writes 0s.Quoted 2 lines
But when i fallocate in ext4 i can see the write performance improvement but not in xfs
Testing how?
I use IOmeter to test it.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
and reason i found out in one of your previous comments is because of the unwritten flag set in xfs. So how do we see if the unwritten flag is set or not ? I did use
xfs_infobut it didnt show any such information.
Quoted 8 lines
Quoted 3 lines
ext4 & xfs are doing the same basic thing, they must maintain the unwritten state on the preallocated extents, and manage that as it changes when portions are written with real data.
Well, the difference in managing the unwritten state can theoretically result in different performance. Not that I'd expect ext4 being better than xfs in this respect. More data is needed here.
Like when i do preallocation i see the thorughput for seq writes being the same in ext4 but for xfs its goes down by say 10Mps or so.
Quoted 3 lines
Quoted 3 lines
xfs_bmap-v -v -p on a file will show you extent state for xfs.
Thanks a lot.
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
I guess i am not right here ftruncate simply does a lseek and wirtes
ftruncate simply sets
i_size, it does no data IO.... and no block reservation/allocation either.
Yeah rite what i meant to say was that posix_fallocate uses ftruncate which in turn just updates the i_size and then posix_fallocate zeros out the whole thing
Quoted 44 lines
Quoted 10 lines
Quoted 2 lines
to it which might not be contiguous whereas fallocate tries to allocate contiguous block so as to reduce fragmentation
Actually fallocate's only official job is to reserve blocks so you don't
get ENOSPC later. Because the request comes in all at once, you are very likely to get an optimal allocation, and that's a nice side effect,
but it's not actually required by the interface.Quoted 5 lines
Quoted 2 lines
and hence i
thought to reduce fragmentation and for security reasonsNone of these normal interfaces poses any security risk. If you build xfs without the unwritten extent feature
I don't think, it's possible. Not in any configurable way, at least.
Quoted 3 lines
you could allocate w/o flagging
uninitialized and expose stale data, but that's not a normal mode of operation.That's was possible with mount option unwritten=0, but AFAIK, it's been recently completely removed from code.
Quoted 10 lines
Quoted 7 lines
its better
to use ALLOCSP rather than something like ftruncate /posix_fallocate or RSEVSP which kindof performs bad for writes with unwritten flag set and now there being a no direct way while creating the fs to disable unwritten.In the end, there are only 2 ways to preallocate blocks: explicitly write 0s, or flag regions as unwritten (as xfs/ext4/... can do).
Exactly.
That's a trade-off between spending time on setup or at the write time. And if explicit zeros are desirable for the former approach, it can be driven from the user space (after preallocation) rather than from the kernel with the exactly the same outcome.
Quoted 9 lines
Just restating the same what Eric already said :)
Felix
Quoted 5 lines
(Ok,
or a 3rd sorta-way, which is to reserve w/o flagging, maybe that's what
you're looking for, but that's deprecated or not really available at this point).
Yes that is what i was looking for :) but i guess its no more available through mkfs.xfs
and i will have to do it with xfs_db as stated in one of the Eric's replies in the previous posts
Quoted 3 lines
Quoted 3 lines
Maybe I should ask what the end goal is here. :)
Just to see if preallocation using fallocate helps reducing the fragmentation and increases the throughput. I guess it wll help reduce the fragmentation but the write performance is going to suffer.
Quoted 10 lines
Quoted 8 lines
-Eric
Quoted 1 line
Thanks, Smit
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 9 lines · Smit Shah
Quoted 5 lines · Felix Blyakher
Quoted 2 lines · Eric Sandeen
Maybe I should ask what the end goal is here. :)
Just to see if preallocation using fallocate helps reducing the fragmentation and increases the throughput. I guess it wll help reduce the fragmentation but the write performance is going to suffer.
Depends on the IO you do, I think. I'd be curious to know exactly how you're running preallocation & iometer so I could give it a shot, to see why xfs seems to be doing worse than ext4.
Thanks,
-Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 3 lines · Smit Shah
Yeah rite what i meant to say was that
posix_fallocateuses ftruncate which in turn just updates thei_sizeand thenposix_fallocatezeros out the whole thing
I didn't look at the libc implementation, but it seems strange to use ftruncate here at all. Is it more efficient to write zeros into a hole than appending to a file?
[snip]
Quoted 3 lines
Just to see if preallocation using fallocate helps reducing the fragmentation and increases the throughput. I guess it wll help reduce the fragmentation
... and as result read performance on such files.
Quoted 1 line
but the write performance is going to suffer.
It's not clear why it should. Not doing preallocation doesn't mean that there is no inode updates with every write. Why would extent conversion be more expensive that creating the space (extent) and updating the inode size for every write? It'd interesting to reproduce your results. Any details on your tests and the iometer usage?
Thanks,
Felix
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
In the vmware sparse disk, the chunks are allocated of the size of 64k, which kindof leads to fragmentation. Hence the sequential read throughput suffers. So thought it might be a good idea to preallocate some amount memory say 50Meg when it asks for allocating the chunk thus helps in reducing fragmentation and improves the throughput.
On linux i do preallocation using fallocate.
I have vmware workstation running Windows Vista and host is RHEL 5.3 with the guest having 1GB of memory and host having 2 Gig. And i have separate dedicated SATA disk on which i perform the tests. For testing, I create a new sparse vdisk and run a IOmeter sequential write on it.
For ext4 i tend to get say around 5-6 Mbps of improvement in write throughput sometimes but it never goes down. But for XFS i always see atleast
8-10 Mbps of downfall.
-Smit
Quoted 11 lines · Eric Sandeen
Quoted 9 lines · Smit Shah
Quoted 5 lines · Felix Blyakher
Quoted 2 lines · Eric Sandeen
Maybe I should ask what the end goal is here. :)
Just to see if preallocation using fallocate helps reducing the fragmentation and increases the throughput. I guess it wll help reduce the fragmentation but the write performance is going to suffer.
Quoted 9 lines
Depends on the IO you do, I think. I'd be curious to know exactly how you're running preallocation & iometer so I could give it a shot, to see why xfs seems to be doing worse than ext4.
Thanks,
-Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 10 lines · Felix Blyakher
Quoted 3 lines · Smit Shah
Yeah rite what i meant to say was that
posix_fallocateuses ftruncate which in turn just updates thei_sizeand thenposix_fallocatezeros out the whole thingI didn't look at the libc implementation, but it seems strange to use ftruncate here at all. Is it more efficient to write zeros into a hole than appending to a file?
I havent seen in detail but i had just scanned through the posix_fallocate code in libc sometime back and it seemed to be doing that but i can confirm that later.
Quoted 8 lines
[snip]
Quoted 3 lines
Just to see if preallocation using fallocate helps reducing the fragmentation and increases the throughput. I guess it wll help reduce the fragmentation
... and as result read performance on such files.
Quoted 8 lines
Quoted 1 line
but the write performance is going to suffer.
It's not clear why it should. Not doing preallocation doesn't mean that there is no inode updates with every write. Why would extent conversion be more expensive that creating the space (extent) and updating the inode size for every write? It'd interesting to reproduce your results. Any details on your tests and the iometer usage?
Since fallocate uses the RESVSP cmd for xfs. And as given given for RESVSP in man page for xfsctl
If the XFS filesystem is configured to flag unwritten file extents, performance will be negatively affected when writing to preallocated space, since extra filesystem transactions are required to convert extent flags on the range of the file written.
Quoted 5 lines
Thanks,
Felix
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 10 lines · Smit Shah
Quoted 7 lines · Felix Blyakher
Quoted 1 line
but the write performance is going to suffer.
It's not clear why it should. Not doing preallocation doesn't mean that there is no inode updates with every write. Why would extent conversion be more expensive that creating the space (extent) and updating the inode size for every write? It'd interesting to reproduce your results. Any details on your tests and the iometer usage?
Quoted 6 lines
Since fallocate uses the RESVSP cmd for xfs. And as given given for RESVSP in man page for xfsctl
If the XFS filesystem is configured to flag unwritten file extents, performance will be negatively affected when writing to preallocated space, since extra filesystem transactions are required to convert extent flags on the range of the file written.
And ext4 must do basically the same thing, as would any fs that flags unwritten extents.
ext4 may convert more at a time, though, rather than leaving "fragmented" written/unwritten/written/unwritten regions.
-Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 8 lines · Smit Shah
Quoted 7 lines · Felix Blyakher
Quoted 3 lines · Smit Shah
Yeah rite what i meant to say was that
posix_fallocateuses ftruncate which in turn just updates thei_sizeand thenposix_fallocatezeros out the whole thing
Quoted 7 lines
Quoted 3 lines
I didn't look at the libc implementation, but it seems strange to use ftruncate here at all. Is it more efficient to write zeros into a hole than appending to a file?
I havent seen in detail but i had just scanned through the
posix_fallocatecode in libc sometime back and it seemed to be doing that but i can confirm that later.
Yeah it simply write's zero's, i had seen the ftruncate used in the posix_fallocate implimentation and hence got confused. It is used when the prealloc length is zero and total file size is less than offset specified.
My bad.
Quoted 17 lines
Quoted 8 lines
[snip]
Quoted 3 lines
Just to see if preallocation using fallocate helps reducing the fragmentation and increases the throughput. I guess it wll help reduce the fragmentation
... and as result read performance on such files.
Quoted 8 lines
Quoted 1 line
but the write performance is going to suffer.
It's not clear why it should. Not doing preallocation doesn't mean that there is no inode updates with every write. Why would extent conversion be more expensive that creating the space (extent) and updating the inode size for every write? It'd interesting to reproduce your results. Any details on your tests and the iometer usage?
Quoted 13 lines
Since fallocate uses the RESVSP cmd for xfs. And as given given for RESVSP in man page for xfsctl
If the XFS filesystem is configured to flag unwritten file extents, performance will be negatively affected when writing to preallocated space, since extra filesystem transactions are required to convert extent flags on the range of the file written.Quoted 5 lines
Thanks,
Felix
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 12 lines · Smit Shah
Quoted 10 lines · Felix Blyakher
Quoted 1 line · Smit Shah
but the write performance is going to suffer.
It's not clear why it should. Not doing preallocation doesn't mean that there is no inode updates with every write. Why would extent conversion be more expensive that creating the space (extent) and updating the inode size for every write? It'd interesting to reproduce your results. Any details on your tests and the iometer usage?
Quoted 3 lines
Since fallocate uses the RESVSP cmd for xfs. And as given given for RESVSP in man page for xfsctl
If the XFS filesystem is configured to flag unwritten file extents,
Good point, we should update the man page, there is no 'if' any more.
Quoted 4 lines
performance will be negatively affected when writing to preallocated space, since extra filesystem transactions are required to convert extent flags on the range of the file written.
Yes, absolutely, there is overhead to handle unwritten extents, no doubt. My remark was wrt comparison between not doing preallocation at all and preallocation with unwritten extents. Both cases would incur the metadata updates with every write. And I doubt that write performance would suffer more in the preallocation case. Though, if you prefer to trade extra time spent at every write to setup time, you can/should manually fill the file with zeros.
Felix
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 9 lines · Eric Sandeen
Quoted 7 lines · Smit Shah
Since fallocate uses the RESVSP cmd for xfs. And as given given for RESVSP in man page for xfsctl
If the XFS filesystem is configured to flag unwritten file extents, performance will be negatively affected when writing to preallocated space, since extra filesystem transactions are required to convert extent flags on the range of the file written.
Quoted 5 lines
And ext4 must do basically the same thing, as would any fs that flags unwritten extents.
ext4 may convert more at a time, though, rather than leaving "fragmented" written/unwritten/written/unwritten regions.
Converting more at a time would require writing extra zeros to extra space converted, but not covered with write. Again, it's a trade off, and there is no clear winner. Well, at least not in my mind, until proven that some operations are more efficient than others.
Thanks,
Felix
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 1 line · On Dienstag 16 Juni 2009 Smit Shah wrote:
In the vmware sparse disk
Ah, virtualization again. Yes, that's no good for performance ;-) There's a "defragment disk" button in VMware which simply copies the flat file on the filesystem in order to remove fragmentation. I guess running xfs_fsr would help more in that case. Did you try with that?
Quoted 1 line
chunks are allocated of the size of 64k,
It's a pity that there's no option to define how large each chunk should be. Using 1-50MB of real disk space for each new chunk could help a lot to keep the file less fragmented. It get's really funny when you run "defrag" within the VM on a sparse vdisk.
mfg zmi
Signature · 10 lines
// Michael Monnerie, Ing.BSc ----- http://it-management.at // Tel: 0660 / 415 65 31 .network.your.ideas. // PGP Key: "curl -s http://zmi.at/zmi.asc | gpg --import" // Fingerprint: AC19 F9D5 36ED CD8A EF38 500E CE14 91F7 1C12 09B4 // Keyserver: wwwkeys.eu.pgp.net Key-ID: 1C1209B4 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs