filesystem tuning hints?
Hello,
Please redirect me to an appropriate list if this is the wrong place -
This is perhaps a naive question, but please bear with me:
I recently had a chance to do some quick and dirty filesystem performance comparisons on a server here before putting it into production. I tested all the journaling filesystems available on stock suse linux enterprise server v9, using bonnie, tiobench, iozone, and dbench, which all showed similar trends - xfs tended to have steady performance and latency, jfs had low performance but low cpu usage, reiserfs got the best numbers in general, and ext3 results were all over the map. The dbench results are fairly indicative of the results as a whole.
BTW - the mount options were basically "-noatime" on all filesystems.
I also tested ext2 just out of curiosity, and it thrashed all the others by a large margin. Could I be doing something really really dumb here, or is this just the cost of journalling?
Are there any dynamic kernel parameters which could bring any of the journalled filesystems performance to a more respectable level?
Here are the dbench 3.04 results (MB/sec) plotted as nprocs vs fs type
n ext2 ext3 jfs reiser xfs
-------------------------------------------------------------------
1 239.45 180.94 35.30 209.02 154.44 2 438.83 287.87 36.34 324.25 157.31 4 807.57 389.64 35.81 475.24 154.95 8 1018.24 398.31 30.66 396.14 146.62 16 1003.61 354.79 27.10 403.79 139.17 32 1006.60 180.83 25.40 330.46 120.81 64 1007.61 117.39 24.88 107.89 79.18 128 1010.10 67.70 18.60 43.62 6.41 256 1005.33 26.55 4.10 34.98 7.27 512 973.30 18.00 2.97 29.61 5.34 1024 613.40 17.64 4.36 27.16 4.79 2048 84.05 13.53 16.37 23.29 3.84
Thanks & Regards,
Joel
In order to answer your question about tuning a filesystem, it's a requirement to know something about the workload, and you haven't provided that. In general if you haven't been using the latest e2fsprogs creating ext3 with a larger journal size will help --- unless you don't have the requisite memory to support having extra pinned buffers, since your application may end up getting pushed out to swap. Many journalling filesystems can get much better performance by putting the journal on a separate external device, especially if that device is a battery-backed non-volitile ramdisk. With ext3, depending on your workload (if it is fsync-intensive, for example), using data journalling with can help a lot, especially with an external journal device.
Finally, be warned that many filesystem benchmarks may be quite misleading, especially dbench, which doesn't match very many real-life workloads at all. It's designed to be half (the disk part) of an artificial web-based benchmark that was in itself a pretty bad benchmark that was easily subject to gaming and not very reflective of real-world workloads. Its main virtue is that it is easy to run and can be useful to developers as a good stress/smoke test to detect that new kernels haven't done something stupid, as long as you ignore its numbers. :-)
In many cases, using your actual workload will be the best benchmark.
Regards,
- Ted
Quoted 3 lines · joel
I also tested ext2 just out of curiosity, and it thrashed all the others by a large margin. Could I be doing something really really dumb here, or is this just the cost of journalling?
Journalling costs performance at the price of added I/O.
Quoted 2 lines
Are there any dynamic kernel parameters which could bring any of the journalled filesystems performance to a more respectable level?
Dynamic parameters? Probably not. If you *really* care about performance, you put the journal on a different physical drive (and maybe controller) than the actual filesystem itself. From 'man mkfs.ext3':
-J journal-options
...
device=external-journal
Attach the filesystem to the journal block device located on external-journal. The external journal must already have been created using the command
mke2fs -O journal_dev external-journal