Questions about XFS

Hi all,

I have a few questions about XFS that didn't make the XFS FAQ. I'm trying to get a feel for where I might want to use it on my servers (or at home). A mix of ext3 & ext4 has worked well for me. But I'd like to get to know XFS a bit better. The target OS would be RHEL6.

1. I don't have "lots and large". Why should I run XFS?

2. I don't have "lots and large". Why shouldn't I run XFS?

3. Why doesn't RHEL6 support XFS on root, when the XFS FAQ says XFS on root is fine? Is there some issue I should be aware of?

4. From the time I write() a bit of data, what's the maximum time before the data is actually committed to disk?

5. Ext4 provides some automatic fsync'ing to avoid the zero-length file issue for some common cases via the auto_da_alloc feature added in kernel 2.6.30. Does XFS have similar behavior?

6. RHEL6 Anaconda sets a RAID10 chunk size of 512K by default XFS complains and sets its log stripe down to 32k. Should I accept Anaconda's default? It knows I've requested XFS formatting before it sets the chunk size, after all.

8. Eric (and the XFS FAQ) have recommended just using the defaults for mkfs.xfs and mount. But I've also heard Dave say "Increase logbsize and use inode64; everybody does that, but we just haven't made it the default". I'm guessing it doesn't matter if one doesn't have large and lots?

I have a few questions about XFS that didn't make the XFS FAQ. I'm trying to get a feel for where I might want to use it on my servers (or at home). A mix of ext3 & ext4 has worked well for me. But I'd like to get to know XFS a bit better. The target OS would be RHEL6.

1. I don't have "lots and large". Why should I run XFS?

2. I don't have "lots and large". Why shouldn't I run XFS?

3. Why doesn't RHEL6 support XFS on root, when the XFS FAQ says XFS on root is fine? Is there some issue I should be aware of?

4. From the time I write() a bit of data, what's the maximum time before the data is actually committed to disk?

5. Ext4 provides some automatic fsync'ing to avoid the zero-length file issue for some common cases via the auto_da_alloc feature added in kernel 2.6.30. Does XFS have similar behavior?

Quoted 8 lines · Steve Bergman

4. From the time I write() a bit of data, what's the maximum time before the
data is actually committed to disk?

5. Ext4 provides some automatic fsync'ing to avoid the zero-length file issue for some common cases via the auto_da_alloc feature added in kernel 2.6.30. Does XFS have similar behavior?

I think that here you are talking more about ext3 than ext4.

The answer to both of these - even for ext4 or ext3 - is that unless your application and storage is all properly configured, you are effectively at risk indefinitely. Chris Mason did a study years ago where he was able to demonstrate that dirty data could get pinned in a disk cache effectively indefinitely. Only an fsync() would push that out.

Applications need to use the data integrity hooks in order to have a reliable promise that application data is crash safe. Jeff Moyer wrote up a really nice overview of this for lwn which you can find here:

http://lwn.net/Articles/457667

That said, if you have applications that do not do any of this, you can roll the dice and use a file system like ext3 that will periodically push data out of the page cache for you.

Note that without the barrier mount option, that is not sufficient to push data to platter, just moves it down the line to the next potentially volatile cache :) Even then, 4 out of every 5 seconds, your application will be certain to lose data if the box crashes while it is writing data. Lots of applications don't actually use the file system much (or write much), so ext3's sync behaviour helped mask poorly written applications pretty effectively for quite a while.

There really is no short cut to doing the job right - your applications need to use the correct calls and we all need to configure the file and storage stack correctly.

Thanks!

Ric

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

In #5 I was specifically talking about ext4. After the 2009 brouhaha over zero-length files in ext4 with delayed allocation turned on, Ted merged some patches into vanilla kernel 2,6,30 which mitigated the problem by recognizing certain common idioms and forcing automatically forcing an fsync. I'd heard the the XFS team modeled a set of XFS patches from them.

Regarding #4, I have 12 years experience with my workloads on ext3 and 3 yrs on ext4 and know what I have observed. As a practical matter, there are large differences between filesystem behaviors which aren't up for debate since I know my workloads' behavior in the real world far better than anyone else possibly could. (In fact, I'm not sure how anyone else could presume to know how my workloads and filesystems interact.) But if I understand correctly, ext4 at default settings journals metadata and commits it every 5s, while flushing data every 30s. Ext3 journals metadata, and commits it every 5 seconds, while effectively flushing data, *immediately before the metadata*, every 5 seconds. so the window in which data and metadata are not in sync is vanishingly small. Are you saying that with XFS there is no periodic flushing mechanism at all? And that unless there's an fsync/fdatasync/sync or the memory needs to be reclaimed, that it can sit in the page cache forever?

One thing is puzzling me. Everyone is telling me that I must ensure that fsync/fdatasync is used, even in environments where the concept doesn't exist. So I've gone to find good examples of how it it used. Since RHEL6 has been shipping with ext4 as the default for over 2.5 years, I figured it would be a great place to find examples. However, I've been unable to find examples of fsync or fdatasync being used, when using "strace -o file.out -f" on various system programs which one would very much expect to use it. We talked about some Python config utilities the other day. But now I've moved on to C and C++ code. e.g. "cupsd" copy/truncate/writes the config file "/etc/cups/printers.conf" quite frequently, all day long. But there is no sign whatsoever of any fsync or fdatasync when I grep the strace output file for those strings case insensitively. (And indeed, a complex printers.conf file turned up zero-length on one of my RHEL6.4 boxes last week.)

So I figured that when rpm installs a new vmlinuz, builds a new initramfs and puts it into place, and modifies grub.conf, that surely proper sync'ing must be done in this particularly critical case. But while I do see rpm fsync/fsync'ing its own database files, it never seems to fsync/fdatasync the critical system files it just installed and/or modified. Surely, after over 2 - 1/2 years of Red Hat shipping RHEL6 to customers, I must be mistaken in some way. Could you point me to an example in RHEL6.4 where I can see clearly how fsync is being properly used? In the mean time, I'll keep looking.

Thanks,
Steve

Quoted 47 lines · Ric Wheeler

Quoted 7 lines · Steve Bergman

4. From the time I write() a bit of data, what's the maximum time before the
data is actually committed to disk?

5. Ext4 provides some automatic fsync'ing to avoid the zero-length file issue for some common cases via the auto_da_alloc feature added in kernel 2.6.30. Does XFS have similar behavior?

I think that here you are talking more about ext3 than ext4.

The answer to both of these - even for ext4 or ext3 - is that unless your application and storage is all properly configured, you are effectively at risk indefinitely. Chris Mason did a study years ago where he was able to demonstrate that dirty data could get pinned in a disk cache effectively indefinitely. Only an fsync() would push that out.

Applications need to use the data integrity hooks in order to have a reliable promise that application data is crash safe. Jeff Moyer wrote up a really nice overview of this for lwn which you can find here:

http://lwn.net/Articles/457667

That said, if you have applications that do not do any of this, you can roll the dice and use a file system like ext3 that will periodically push data out of the page cache for you.

Note that without the barrier mount option, that is not sufficient to push data to platter, just moves it down the line to the next potentially volatile cache :) Even then, 4 out of every 5 seconds, your application will be certain to lose data if the box crashes while it is writing data. Lots of applications don't actually use the file system much (or write much), so ext3's sync behaviour helped mask poorly written applications pretty effectively for quite a while.

There really is no short cut to doing the job right - your applications need to use the correct calls and we all need to configure the file and storage stack correctly.

Thanks!

Ric

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Let's take a simple example - a database app that does say 30 transactions/sec.

In your example, you are extremely likely to lose up to just shy of 5 seconds of "committed" data - way over 100 transactions! That can be *really* serious amounts of data and translate into large financial loss.

In a second example, let's say you are copying data to disk (say a movie) at a rate of 50 MB/second. When the power cut hits at just the wrong time, you will have lost a large chunk of that data that has been "written" to disk (over 200MB).

Quoted 6 lines

Let's take a simple example - a database app that does say 30 transactions/sec.

In your example, you are extremely likely to lose up to just shy of 5 seconds of "committed" data - way over 100 transactions! That can be *really* serious amounts of data and translate into large financial loss.

Every database software will do the flushing correctly.

Quoted 5 lines

In a second example, let's say you are copying data to disk (say a movie) at a rate of 50 MB/second. When the power cut hits at just the wrong time, you will have lost a large chunk of that data that has been "written" to disk (over 200MB).

But why would anyone care about that? I know that the system went down while copying this large movie, so I'll just copy it again.

Quoted 1 line

Every database software will do the flushing correctly.

Stefan, you are making my point because every database will do the right thing, it won't rely on ext3's magic every 5 second fsync :)

Quoted 5 lines

Quoted 1 line

Every database software will do the flushing correctly.

Stefan, you are making my point because every database will do the right thing, it won't rely on ext3's magic every 5 second fsync :)

Ok, let us agree on this one then ;)

There is still kind of a dichotomy regarding the entire issue. While in many cases, noone will complain about a few seconds of lost data, it is an entirely different beast if you lose an entire file that may have been added to and maintained over month/years, just because the system went down at an unfortunate moment. In my opinion, having a consistent, yet possibly somewhat outdated file, is much more important than having the most recent view in an inconsistent/broken state.

From the LWN article we keep linking to ;)

Quoted 8 lines · Stefan Ring

Quoted 7 lines

Let's take a simple example - a database app that does say 30 transactions/sec.

In your example, you are extremely likely to lose up to just shy of 5 seconds of "committed" data - way over 100 transactions! That can be *really* serious amounts of data and translate into large financial loss.

Quoted 1 line

Every database software will do the flushing correctly.

Stefan, you are making my point because every database will do the right thing, it won't rely on ext3's magic every 5 second fsync :)

Ric

Quoted 10 lines

Quoted 7 lines

In a second example, let's say you are copying data to disk (say a movie) at
a rate of 50 MB/second. When the power cut hits at just the wrong time, you
will have lost a large chunk of that data that has been "written" to disk (over 200MB).

But why would anyone care about that? I know that the system went down while copying this large movie, so I'll just copy it again.

Thanks. But I'm specifically *not* talking about database apps, I'm talking about non-database applications written in languages that don't even have a concept of fsync.

I get it. You want a pony, and you don't want to pay anything for

Quoted 4 lines

In a second example, let's say you are copying data to disk (say a movie) at a rate of 50 MB/second. When the power cut hits at just the wrong time, you will have lost a large chunk of that data that has been "written" to disk (over 200MB).

But why would anyone care about that? I know that the system went down while copying this large movie, so I'll just copy it again.

In #5 I was specifically talking about ext4. After the 2009 brouhaha over zero-length files in ext4 with delayed allocation turned on, Ted merged some patches into vanilla kernel 2,6,30 which mitigated the problem by recognizing certain common idioms and forcing automatically forcing an fsync. I'd heard the the XFS team modeled a set of XFS patches from them.

Regarding #4, I have 12 years experience with my workloads on ext3 and 3 yrs on ext4 and know what I have observed. As a practical matter, there are large differences between filesystem behaviors which aren't up for debate since I know my workloads' behavior in the real world far better than anyone else possibly could. (In fact, I'm not sure how anyone else could presume to know how my workloads and filesystems interact.) But if I understand correctly, ext4 at default settings

journals metadata and commits it every 5s, while flushing data every 30s. Ext3 journals metadata, and commits it every 5 seconds, while effectively flushing data, *immediately before the metadata*, every 5 seconds. so the window in which data and metadata are not in sync is vanishingly small. Are you saying that with XFS there is no periodic flushing mechanism at all? And that unless there's an fsync/fdatasync/sync or the memory needs to be reclaimed, that it can sit in the page cache forever?

One thing is puzzling me. Everyone is telling me that I must ensure that fsync/fdatasync is used, even in environments where the concept doesn't exist. So I've gone to find good examples of how it it used. Since RHEL6 has been shipping with ext4 as the default for over 2.5 years, I figured it would be a great place to find examples. However, I've been unable to find examples of fsync or fdatasync being used, when using "strace -o file.out -f" on various system programs which one would very much expect to use it.

We talked about some Python
config utilities the other day. But now I've moved on to C and C++ code. e.g. "cupsd" copy/truncate/writes the config file "/etc/cups/printers.conf" quite frequently, all day long. But there is no sign whatsoever of any fsync or fdatasync when I grep the strace output file for those strings case insensitively. (And indeed, a complex printers.conf file turned up zero-length on one of my RHEL6.4 boxes last week.)

So I figured that when rpm installs a new vmlinuz, builds a new initramfs and puts it into place, and modifies grub.conf, that surely proper sync'ing must be done in this particularly critical case. But while I do see rpm fsync/fsync'ing its own database files, it never seems to fsync/fdatasync the critical system files it just installed and/or modified. Surely, after over 2 - 1/2 years of Red Hat shipping RHEL6 to customers, I must be mistaken in some way. Could you point me to an example in RHEL6.4 where I can see clearly how fsync is being properly used? In the mean time, I'll keep looking.

Thanks,
Steve

Quoted 41 lines · Ric Wheeler

Quoted 8 lines · Steve Bergman

4. From the time I write() a bit of data, what's the maximum time before the
data is actually committed to disk?

5. Ext4 provides some automatic fsync'ing to avoid the zero-length file issue for some common cases via the auto_da_alloc feature added in kernel 2.6.30. Does XFS have similar behavior?

I think that here you are talking more about ext3 than ext4.

The answer to both of these - even for ext4 or ext3 - is that unless your application and storage is all properly configured, you are effectively at risk indefinitely. Chris Mason did a study years ago where he was able to demonstrate that dirty data could get pinned in a disk cache effectively indefinitely. Only an fsync() would push that out.

Applications need to use the data integrity hooks in order to have a reliable promise that application data is crash safe. Jeff Moyer wrote up a really nice overview of this for lwn which you can find here:

http://lwn.net/Articles/457667

That said, if you have applications that do not do any of this, you can roll the dice and use a file system like ext3 that will periodically push data out of the page cache for you.

Note that without the barrier mount option, that is not sufficient to push data to platter, just moves it down the line to the next potentially volatile cache :) Even then, 4 out of every 5 seconds, your application will be certain to lose data if the box crashes while it is writing data. Lots of applications don't actually use the file system much (or write much), so ext3's sync behaviour helped mask poorly written applications pretty effectively for quite a while.

There really is no short cut to doing the job right - your applications need to use the correct calls and we all need to configure the file and storage stack correctly.

Quoted 8 lines

Thanks!

Ric

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Now about ext3 behavior. (Sorry, I know this is an XFS list, but I've been dying to have this clarified. And this does kind of relate to XFS.) Not only does data get flushed more frequently. But it gets careful treatment. It gets written out immediately before the metadata.

Quoted 6 lines · Steve Bergman

In #5 I was specifically talking about ext4. After the 2009 brouhaha over zero-length files in ext4 with delayed allocation turned on, Ted merged some patches into vanilla kernel 2,6,30 which mitigated the problem by recognizing certain common idioms and forcing automatically forcing an fsync. I'd heard the the XFS team modeled a set of XFS patches from them.

Assuming we're talking about the same behaviors, XFS resolved this issue in May 2007, in the 2.6.22 kernel, commit ba87ea6, over a year before ext4 even had delayed allocation working. ext4 added the flush-on-close heuristic in 2009, commit 7d8f9f7.

Quoted 7 lines

Regarding #4, I have 12 years experience with my workloads on ext3 and 3 yrs on ext4 and know what I have observed. As a practical matter, there are large differences between filesystem behaviors which aren't up for debate since I know my workloads' behavior in the real world far better than anyone else possibly could. (In fact, I'm not sure how anyone else could presume to know how my workloads and filesystems interact.) But if I understand correctly, ext4 at default settings

Quoted 8 lines

journals metadata and commits it every 5s, while flushing data every 30s. Ext3 journals metadata, and commits it every 5 seconds, while effectively flushing data, *immediately before the metadata*, every 5 seconds. so the window in which data and metadata are not in sync is vanishingly small. Are you saying that with XFS there is no periodic flushing mechanism at all? And that unless there's an fsync/fdatasync/sync or the memory needs to be reclaimed, that it can sit in the page cache forever?

No.

By and large, buffered IO in a filesystem is flushed out by the vm, due to either age or memory pressure. The filesystem then responds to these requests by the VM, writing data as requested.

You can read all about it in Documentation/sysctl/vm.txt but see dirty_expire_centisecs and dirty_writeback_centisecs - flushers wake up every 30s and push on data more than 5s old, by default.

ext3 is somewhat unique in data=ordered metadata logging driving data flushing, IMHO.

Quoted 8 lines

One thing is puzzling me. Everyone is telling me that I must ensure that fsync/fdatasync is used, even in environments where the concept doesn't exist. So I've gone to find good examples of how it it used. Since RHEL6 has been shipping with ext4 as the default for over 2.5 years, I figured it would be a great place to find examples. However, I've been unable to find examples of fsync or fdatasync being used, when using "strace -o file.out -f" on various system programs which one would very much expect to use it.

Whether or not an application *uses* fsync is orthogonal to whether or not it's *needed* to ensure persistence. Obviously you don't need to fsync every IO as soon as its issued. And there are buggy applications, yes. It's up to the app to decide what needs to be persistent and when. See the "When Should You Fsync?" section in the URL below.

Quoted 8 lines

We talked about some Python
config utilities the other day. But now I've moved on to C and C++ code. e.g. "cupsd" copy/truncate/writes the config file "/etc/cups/printers.conf" quite frequently, all day long. But there is no sign whatsoever of any fsync or fdatasync when I grep the strace output file for those strings case insensitively. (And indeed, a complex printers.conf file turned up zero-length on one of my RHEL6.4 boxes last week.)

I'd file a bug against cups, then.

Quoted 9 lines

So I figured that when rpm installs a new vmlinuz, builds a new initramfs and puts it into place, and modifies grub.conf, that surely proper sync'ing must be done in this particularly critical case. But while I do see rpm fsync/fsync'ing its own database files, it never seems to fsync/fdatasync the critical system files it just installed and/or modified. Surely, after over 2 - 1/2 years of Red Hat shipping RHEL6 to customers, I must be mistaken in some way. Could you point me to an example in RHEL6.4 where I can see clearly how fsync is being properly used? In the mean time, I'll keep looking.

database packages would get it right, I hope.

See also http://lwn.net/Articles/457667/, "Ensuring data reaches disk"

-Eric

Quoted 61 lines

Thanks,
Steve

Quoted 41 lines · Ric Wheeler

Quoted 8 lines · Steve Bergman

4. From the time I write() a bit of data, what's the maximum time before the
data is actually committed to disk?

5. Ext4 provides some automatic fsync'ing to avoid the zero-length file issue for some common cases via the auto_da_alloc feature added in kernel 2.6.30. Does XFS have similar behavior?

I think that here you are talking more about ext3 than ext4.

The answer to both of these - even for ext4 or ext3 - is that unless your application and storage is all properly configured, you are effectively at risk indefinitely. Chris Mason did a study years ago where he was able to demonstrate that dirty data could get pinned in a disk cache effectively indefinitely. Only an fsync() would push that out.

Applications need to use the data integrity hooks in order to have a reliable promise that application data is crash safe. Jeff Moyer wrote up a really nice overview of this for lwn which you can find here:

http://lwn.net/Articles/457667

That said, if you have applications that do not do any of this, you can roll the dice and use a file system like ext3 that will periodically push data out of the page cache for you.

Note that without the barrier mount option, that is not sufficient to push data to platter, just moves it down the line to the next potentially volatile cache :) Even then, 4 out of every 5 seconds, your application will be certain to lose data if the box crashes while it is writing data. Lots of applications don't actually use the file system much (or write much), so ext3's sync behaviour helped mask poorly written applications pretty effectively for quite a while.

There really is no short cut to doing the job right - your applications need to use the correct calls and we all need to configure the file and storage stack correctly.

Quoted 8 lines

Thanks!

Ric

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Are you saying that with XFS there is no periodic
flushing mechanism at all? And that unless there's an fsync/fdatasync/sync or the memory needs to be reclaimed, that it can sit in the page cache forever?

Quoted 4 lines · Steve Bergman

Are you saying that with XFS there is no periodic
flushing mechanism at all? And that unless there's an fsync/fdatasync/sync or the memory needs to be reclaimed, that it can sit in the page cache forever?

I read the later responses to this and they seemed to say that the data in the page cache should be written to the disk periodically. I am not meaning to hijack the thread. I just have a question directly related to this point.

I have an application that is streaming data to an XFS disk at a sustained 25 MB/sec. This is well below what the hardware supports. The application does fopen/fwrite/fclose (no active flushing or syncing).

I see that as my application writes data (the only process writing the only open file on the disk), the system cache grows and grows. Here is the unusual part: periodically, writes take some number of seconds to complete, rather than the typical <50 msecs). The increased time seems to correspond to the increasing size of the page cache.

If I do:

echo 1 > /proc/sys/vm/drop_caches

while the application is runnung, then the writes do not occasionally take longer. Until the cache grows again, and I do the echo again.

I am sure I must be misinterpreting what I see.

(on openSUSE 12.1. kernel 3.1.0)

--
Roger Oberholtzer

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Quoted 5 lines · Roger Oberholtzer

Quoted 4 lines · Steve Bergman

Are you saying that with XFS there is no periodic
flushing mechanism at all? And that unless there's an fsync/fdatasync/sync or the memory needs to be reclaimed, that it can sit in the page cache forever?

Quoted 4 lines

I read the later responses to this and they seemed to say that the data in the page cache should be written to the disk periodically. I am not meaning to hijack the thread. I just have a question directly related to this point.

You most likely need to adjust some of the vm tunings to cause the vm to kick out pages more evenly. Not sure what the opensuse crowd would suggest tweaking.

Quoted 3 lines

I have an application that is streaming data to an XFS disk at a sustained 25 MB/sec. This is well below what the hardware supports. The application does fopen/fwrite/fclose (no active flushing or syncing).

Sounds like this is more likely to be an application issue than a file system one. Can you push the IO write speed up with a simple "dd" test to a file?

/proc/sys/vm/drop_caches"

I have an application that is streaming data to an XFS disk at a sustained 25 MB/sec. This is well below what the hardware supports. The application does fopen/fwrite/fclose (no active flushing or syncing).

I see that as my application writes data (the only process writing the only open file on the disk), the system cache grows and grows. Here is the unusual part: periodically, writes take some number of seconds to complete, rather than the typical <50 msecs). The increased time seems to correspond to the increasing size of the page cache.

If I do:

echo 1 > /proc/sys/vm/drop_caches

while the application is runnung, then the writes do not occasionally take longer. Until the cache grows again, and I do the echo again.

I am sure I must be misinterpreting what I see.

On 6/12/2013 3:26 AM, Roger Oberholtzer wrote:
...

Quoted 3 lines

I have an application that is streaming data to an XFS disk at a sustained 25 MB/sec. This is well below what the hardware supports. The application does fopen/fwrite/fclose (no active flushing or syncing).

Buffered IO.

Quoted 5 lines

I see that as my application writes data (the only process writing the only open file on the disk), the system cache grows and grows. Here is the unusual part: periodically, writes take some number of seconds to complete, rather than the typical <50 msecs). The increased time seems to correspond to the increasing size of the page cache.

Standard Linux buffered IO behavior. Note this is not XFS specific.

Quoted 3 lines

If I do:

echo 1 > /proc/sys/vm/drop_caches

Dumps the page cache forcing your buffered writes to disk.

Quoted 2 lines

while the application is runnung, then the writes do not occasionally take longer. Until the cache grows again, and I do the echo again.

Which seems a bit laborious.

Quoted 1 line

I am sure I must be misinterpreting what I see.

Nope. The Linux virtual memory system has behaved this way for quite some time. You can teak how long IOs stay in cache. See dirty_* at
https://www.kernel.org/doc/Documentation/sysctl/vm.txt

Given the streaming nature you describe, have you looked at possibly using O_DIRECT?

Quoted 4 lines · Steve Bergman

Are you saying that with XFS there is no periodic
flushing mechanism at all? And that unless there's an fsync/fdatasync/sync or the memory needs to be reclaimed, that it can sit in the page cache forever?

I read the later responses to this and they seemed to say that the data in the page cache should be written to the disk periodically. I am not meaning to hijack the thread. I just have a question directly related to this point.

I have an application that is streaming data to an XFS disk at a sustained 25 MB/sec. This is well below what the hardware supports. The application does fopen/fwrite/fclose (no active flushing or syncing).

I have a few questions about XFS that didn't make the XFS FAQ. I'm trying to get a feel for where I might want to use it on my servers (or at home). A mix of ext3 & ext4 has worked well for me. But I'd like to get to know XFS a bit better. The target OS would be RHEL6.

9. I there something else I should have thought to ask?

I'm setting up a new server right now, and I'm breaking it up into a number of different LV's for flexibility. I've gone through a number of possibilities, and I think it's going to look something like this:

/ - Ext4
/root - Ext4
/home - Ext4
/var - Ext4
/usr/local - Ext4
/usr/local/worktmp - XFS (Very intensive & time-consuming random writes, here. But all files are temporary work files. XFS is performing well here in my testing.)

/usr/local/data - Ext3 (Cobol doesn't know about fsync. I need pony-magic in this LV.)

Backup on the /usr/local/worktmp is not critical, due to its nature. But can't I just lvm2-snapshot an XFS LV and rsync that to USB drive like I do everything else?

Hi Ben,

/usr/local/data - Ext3 (Cobol doesn't know about fsync. I need pony-magic in this LV.)