LWN.net article: creating 1 billion files -> XFS looses
The subject is a bit harsh, but overall the article says:
XFS is slowest on creating and deleting a billion files XFS fsck needs 30GB RAM to fsck that 100TB filesystem.
http://lwn.net/SubscriberLink/400629/3fb4bc34d6223b32/
Signature · 13 lines
mit freundlichen Grüssen, Michael Monnerie, Ing. BSc it-management Internet Services http://proteger.at [gesprochen: Prot-e-schee] Tel: 0660 / 415 65 31 ****** Aktuelles Radiointerview! ****** http://www.it-podcast.at/aktuelle-sendung.html // Wir haben im Moment zwei Häuser zu verkaufen: // http://zmi.at/langegg/ // http://zmi.at/haus2009/
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 5 lines · Michael Monnerie
The subject is a bit harsh, but overall the article says:
XFS is slowest on creating and deleting a billion files XFS fsck needs 30GB RAM to fsck that 100TB filesystem.
The creation and deletion performance is a known issue, and too a large extent fixes by the new delaylog code. We're not quite as fast as ext4 yet, but it's getting close.
The repair result looks a lot like the pre-3.1.0 xfsprogs repair.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 5 lines · On Donnerstag, 19. August 2010 Christoph Hellwig wrote:
The creation and deletion performance is a known issue, and too a
large extent fixes by the new delaylog code. We're not quite as fast as ext4 yet, but it's getting close.The repair result looks a lot like the pre-3.1.0 xfsprogs repair.
Yes I know. I thought some XFS dev might contact the Author to do some re-testing, as a reputation is quickly destroyed by such articles and takes long to be returned. Just this week I had a friend in a FS discussion saying "ins't XFS destroying/zeroing files on power failure?". That information is ancient, but things like that stay in peoples brain for(almost)ever.
Signature · 13 lines
mit freundlichen Grüssen, Michael Monnerie, Ing. BSc it-management Internet Services http://proteger.at [gesprochen: Prot-e-schee] Tel: 0660 / 415 65 31 ****** Aktuelles Radiointerview! ****** http://www.it-podcast.at/aktuelle-sendung.html // Wir haben im Moment zwei Häuser zu verkaufen: // http://zmi.at/langegg/ // http://zmi.at/haus2009/
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Michael Monnerie put forth on 8/19/2010 7:45 AM:
Quoted 4 lines
Just this week I had a friend in a FS
discussion saying "ins't XFS destroying/zeroing files on power failure?". That information is ancient, but things like that stay in peoples brain for(almost)ever.
Had a similar lengthy discussion over on debian-users not more than a month or so ago. Same thing there. Of the 10 or so people active in the thread, I'd say 8 of them were anti-XFS because of the "corruption due to power failure" issue that they'd "read about" years before. Not a single one of them had ever used XFS. A couple of them considered it a "hobbyist quality" filesystem that might be ready for production use in a few years. Ahh the ignorance abounds in our world...
I did my best to educate them, sending them to the Wikipedia page on XFS and to the xfs.org site and specifically the relevant sections of the FAQ. Unfortunately there are some people who simply refuse to be educated. But those type of people aren't candidates for XFS anyway, thankfully. ;)
Signature · 6 lines
Stan _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 5 lines · Michael Monnerie
Quoted 4 lines · On Donnerstag, 19. August 2010 Christoph Hellwig wrote:
The creation and deletion performance is a known issue, and too a
large extent fixes by the new delaylog code. We're not quite as fast as ext4 yet, but it's getting close.
Quoted 8 lines
Quoted 1 line
The repair result looks a lot like the pre-3.1.0 xfsprogs repair.
Yes I know. I thought some XFS dev might contact the Author to do some re-testing, as a reputation is quickly destroyed by such articles and takes long to be returned. Just this week I had a friend in a FS discussion saying "ins't XFS destroying/zeroing files on power failure?". That information is ancient, but things like that stay in peoples brain for(almost)ever.
Don't worry too much - I have the details of the test that was run and alredy know why XFS appeared so slow: it was single threaded. Immediately, that means XFS will be slower to create 1b files regardless of any other detail.
Look at it this way - the initial numbers I'm seeing on my test rig are sustained create rates of about 8,000/s with default mkfs/mount options (i.e. no tuning, no delayed logging, 32k logbsize, etc) and it is burning exactly one of 8 CPUs in the VM. I know I can get an order of magnitude better performance out of XFS on this VM....
It'll take me a few days to run the numbers to be able to write a solid reply, but I have every confidence that a "create 1b inodes" benchmark tuned to XFS's strengths rather than one designed to avoid ext4's weaknesses will show very, very different results.
In the meantime, there is no need to start a flamewar. ;)
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Le Thu, 19 Aug 2010 13:12:45 +0200
Michael Monnerie <[email protected]> écrivait:
Quoted 3 lines
The subject is a bit harsh, but overall the article says:
XFS is slowest on creating and deleting a billion files XFS fsck needs 30GB RAM to fsck that 100TB filesystem.
Too bad I haven't got a 100 TB machine at hand. However I have a 24TB system dedicated to tests. I'm pretty sure we can do much better with XFS and the proper mount options :)
In fact, I have an unused 40 TB array too. See this one later...
Stay tuned :)
Signature · 6 lines
------------------------------------------------------------------------
Emmanuel Florac | Direction technique
| Intellique
| <[email protected]>
| +33 1 78 94 84 02
------------------------------------------------------------------------_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Le Thu, 19 Aug 2010 13:12:45 +0200
Michael Monnerie <[email protected]> écrivait:
Quoted 3 lines
The subject is a bit harsh, but overall the article says:
XFS is slowest on creating and deleting a billion files XFS fsck needs 30GB RAM to fsck that 100TB filesystem.
Just to go on this subject : a colleague (following my suggestion :) tried to create 1 billion files in the same XFS directory. Unfortunately the directories themselves don't scale well that far :
after 1 million files in the first 30 minutes, file creation slows down gradually, so after 100 hours we had about 230 million files. The directory size at that point was 5,3 GB.
Now we're starting afresh with 1000 directories with 1 million files each :)
(Kernel version used : vanilla 2.6.32.11 x86_64 smp)
Signature · 11 lines
------------------------------------------------------------------------
Emmanuel Florac | Direction technique
| Intellique
| <[email protected]>
| +33 1 78 94 84 02
------------------------------------------------------------------------
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfsQuoted 7 lines · Emmanuel Florac
Le Thu, 19 Aug 2010 13:12:45 +0200
Michael Monnerie <[email protected]> écrivait:Quoted 3 lines
The subject is a bit harsh, but overall the article says:
XFS is slowest on creating and deleting a billion files XFS fsck needs 30GB RAM to fsck that 100TB filesystem.
Quoted 6 lines
Just to go on this subject : a colleague (following my suggestion :) tried to create 1 billion files in the same XFS directory. Unfortunately the directories themselves don't scale well that far :
after 1 million files in the first 30 minutes, file creation slows down gradually, so after 100 hours we had about 230 million files. The directory size at that point was 5,3 GB.
Oh, that's larger than I've every run before ;)
Try using:
# mkfs.xfs -d size=64k
Will speed up large directory operations by at least an order of magnitude.
Quoted 2 lines
Now we're starting afresh with 1000 directories with 1 million files each :)
Which is exactly the test that was used to generate the numbers that were published.
Quoted 1 line
(Kernel version used : vanilla 2.6.32.11
x86_64smp)
Not much point in testing that kernel - delayed logging is where the future is for this sort of workload, which is what I'm testing.
FWIW, I'm able to create 50 million inodes in under 14 minutes with delayed logging and 8 threads using directories of 100k entries.
The run to 1 billion inodes that I started late last night (10 hours in) has just passed 700M inodes on a 16TB filesystem. It's running at about 25,000 creates/s, but it is limited by bad shrinker behaviour causing the dentry cache to be completely trashed causing ~3000 read iops to reload dentries that are still necessary for operation. It should be running about 3-4x faster than that.
FYI, The reason I'm taking a while to get numbers is that parallel create workloads of this scale are showing significant problems (VM livelocks, shrinker misbehaviour, lock contention in IO completion processing, buffer cache hash scaling issues, etc) and I'm trying to fix them as I go - these metadata workloads are completely unexplored territory....
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 4 lines · On Dienstag, 7. September 2010 Dave Chinner wrote:
# mkfs.xfs -d size=64kWill speed up large directory operations by at least an order of magnitude.
I've read the man page for mkfs.xfs, but I couldn't find out if using mkfs -d su=64k,sw=2
would be a redundant (and superior) option for that? I'd guess so, reading the description of sunit:
sunit=value
This is used to specify the stripe unit for a RAID device or a logical volume. The value has to be specified in 512-byte block units. Use the su suboption to specify the stripe unit size in bytes. This suboption ensures that data allocations will be stripe unit aligned when the current end of file is being extended and the file size is larger than 512KiB. Also inode allocations and the internal log will be stripe unit aligned.
Or would I still need to use size=64k?
Signature · 13 lines
mit freundlichen Grüssen, Michael Monnerie, Ing. BSc it-management Internet Services http://proteger.at [gesprochen: Prot-e-schee] Tel: 0660 / 415 65 31 ****** Aktuelles Radiointerview! ****** http://www.it-podcast.at/aktuelle-sendung.html // Wir haben im Moment zwei Häuser zu verkaufen: // http://zmi.at/langegg/ // http://zmi.at/haus2009/
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 8 lines · Michael Monnerie
Quoted 4 lines · On Dienstag, 7. September 2010 Dave Chinner wrote:
# mkfs.xfs -d size=64kWill speed up large directory operations by at least an order of magnitude.
I've read the man page for mkfs.xfs, but I couldn't find out if using mkfs -d su=64k,sw=2
Sorry, I screwed that up, it should have read:
# mkfs.xfs -n size=64k
(-n = naming = directories. -d = data != directories)
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 2 lines · On Dienstag, 7. September 2010 Dave Chinner wrote:
# mkfs.xfs -n size=64k (-n = naming = directories. -d = data != directories)
Thank you, Dave. Do I interpret that parameter right:
When a new directory is created, per default it would occupy only 4KB, with -n size=64k would be reserved. As the directory fills, space within that block will be used, so in the default case after 4KB (how many inodes would that be roughly? 256 Bytes/Inode, so 16 entries?) XFS would reserve the next block, but in your case 256 entries would fit.
That would keep dir fragmentation lower, and with todays disks, take a minimal more space, so it sounds very good to use that option. Especially with RAIDs, where stripes usually are 64KB or bigger. Or would the waste of space be so big that it could hurt?
Last question: Is there a way to set that option on a given XFS?
Signature · 13 lines
mit freundlichen Grüssen, Michael Monnerie, Ing. BSc it-management Internet Services http://proteger.at [gesprochen: Prot-e-schee] Tel: 0660 / 415 65 31 ****** Aktuelles Radiointerview! ****** http://www.it-podcast.at/aktuelle-sendung.html // Wir haben im Moment zwei Häuser zu verkaufen: // http://zmi.at/langegg/ // http://zmi.at/haus2009/
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 8 lines · Michael Monnerie
Quoted 2 lines · On Dienstag, 7. September 2010 Dave Chinner wrote:
# mkfs.xfs -n size=64k (-n = naming = directories. -d = data != directories)Thank you, Dave. Do I interpret that parameter right:
When a new directory is created, per default it would occupy only 4KB, with -n size=64k would be reserved.
No, it allocates 64k blocks for the directory instead of 4k blocks.
Quoted 4 lines
As the directory fills, space within
that block will be used, so in the default case after 4KB (how many inodes would that be roughly? 256 Bytes/Inode, so 16 entries?) XFS would reserve the next block, but in your case 256 entries would fit.
Inodes are not stored in the dirctory structure, only the directory entry name and the inode number. Hence the amount of space used by a directory entry is determined by the length of the name.
Quoted 4 lines
That would keep dir fragmentation lower, and with todays disks, take a minimal more space, so it sounds very good to use that option. Especially with RAIDs, where stripes usually are 64KB or bigger. Or would the waste of space be so big that it could hurt?
Well, there is extra overhead to allocate large directory blocks (16 pages instead of one, to begin with, then there's the vmap overhead, etc), so for small directories smaller block sizes are faster for create and unlink operations.
For empty directorys, operations on 4k block sized directories consume roughly 50% less CPU that 64k block size directories. The 4k block size directoeies consume less CPU out to roughly 1.5 million entries where the two are roughly equal. At directory sizes of 10 million entries, 64k directory block operations are consuming about 15% of the CPU that 4k directory block operations consume.
In terms of lookups, the 64k block directory will take less IO but consume more CPU for a given lookup. Hence it depends on your IO latency and whether directory readahead can hide that latency as to which will be faster. e.g. For SSDs, CPU usage might be the limiting factor, not the IO. Right now I don't have any numbers on what the difference might be - I'm getting 1B inode population issues worked out first before I start on measuring cold cache lookup times on 1B files....
Quoted 1 line
Last question: Is there a way to set that option on a given XFS?
No, it is a mkfs time parameter, though we have been discussing the possibility of being able to set it per-directory (at mkdir time when no blocks have been allocated).
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 1 line · On Dienstag, 7. September 2010 Dave Chinner wrote:
# mkfs.xfs -n size=64k
That explanation was worth a FAQ entry to keep it as a reference.
http://xfs.org/index.php/XFS_FAQ#Q:_Performance:_mkfs.xfs_-n_size.3D64k_option
Thanks, Dave.
Signature · 13 lines
mit freundlichen Grüssen, Michael Monnerie, Ing. BSc it-management Internet Services http://proteger.at [gesprochen: Prot-e-schee] Tel: 0660 / 415 65 31 ****** Aktuelles Radiointerview! ****** http://www.it-podcast.at/aktuelle-sendung.html // Wir haben im Moment zwei Häuser zu verkaufen: // http://zmi.at/langegg/ // http://zmi.at/haus2009/
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Le Tue, 7 Sep 2010 08:04:10 +1000 vous écriviez:
Quoted 1 line
Oh, that's larger than I've ever run before ;)
Excellent :) Still works fine afterwards; mount, umount, etc works flawlessly. Memory consumption though is huge :)
Quoted 7 lines
Try using:
# mkfs.xfs -d size=64kWill speed up large directory operations by at least an order of magnitude.
OK, we'll try that too :)
Quoted 10 lines
Quoted 2 lines
Now we're starting afresh with 1000 directories with 1 million files each :)
Which is exactly the test that was used to generate the numbers that were published.
Quoted 1 line
(Kernel version used : vanilla 2.6.32.11
x86_64smp)Not much point in testing that kernel - delayed logging is where the future is for this sort of workload, which is what I'm testing.
I'll compile a 2.6.36rc for comparison.
Signature · 11 lines
------------------------------------------------------------------------
Emmanuel Florac | Direction technique
| Intellique
| <[email protected]>
| +33 1 78 94 84 02
------------------------------------------------------------------------
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfsRe: LWN.net article: creating 1 billion files -> Tests we did
In reply to Michael MonnerieLe Thu, 19 Aug 2010 13:12:45 +0200
Michael Monnerie <[email protected]> écrivait:
Quoted 5 lines
The subject is a bit harsh, but overall the article says:
XFS is slowest on creating and deleting a billion files XFS fsck needs 30GB RAM to fsck that 100TB filesystem.
So We've made a test with 1KB files (space, space...) and a production kernel : 2.6.32.11 (yeah I know, 2.6.38 should be faster but you know, we upgrade our production kernels prudently :).
mk1BFiles will create and delete 1000000000 files with 32 threads
Version: v0.2.4-10-gf6decd3, build: Sep 7 2010 13:39:34
Creating 1000000000 files, started at 2010-09-07 13:45:16... Done, time spent: 89:35:12.262
Doing ls -R, started at 2010-09-11 07:20:28...
Stat: ls (pid: 18844) status: ok, returned value: 0 Cpu usage: user: 1:27:47.242, system: 20:18:21.689 Max rss: 229.01 MBytes, page fault: major: 4, minor: 58694
Compute size used by 1000000000 files, started at 2010-09-12 09:30:52... Size used by files: 11.1759 TBytes
Size used by directory: 32.897 GBytes
Size used (total): 11.2080 TBytes
Done, time spent: 25:50:32.355
Deleting 1000000000 files, started at 2010-09-13 11:21:24... Done, time spent: 68:37:38.117
Test run on a dual Opteron quad core, 16 GB RAM, kernel 2.6.32.11 x86_64...
Signature · 11 lines
------------------------------------------------------------------------
Emmanuel Florac | Direction technique
| Intellique
| <[email protected]>
| +33 1 78 94 84 02
------------------------------------------------------------------------
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfsRe: LWN.net article: creating 1 billion files -> Tests we did
Emmanuel Florac put forth on 9/16/2010 5:13 AM:
Quoted 2 lines
Test run on a dual Opteron quad core, 16 GB RAM, kernel 2.6.32.11
x86_64...
This is a test of storage system performance, and you left out the storage array specs? By doing so it seems you're stating the underlying storage is not relevant to the results.
So, are you saying I should be able to duplicate your results with that dual Opty system, but using an md RAID0 stripe over 8x2TB SATA disks connected to two $60 4 port SiI 3124 PCIe x1 cards?
Signature · 8 lines
Stan _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: LWN.net article: creating 1 billion files -> Tests we did
Quoted 3 lines · On Donnerstag, 16. September 2010 Stan Hoeppner wrote:
So, are you saying I should be able to duplicate your results with
that dual Opty system, but using an md RAID0 stripe over 8x2TB SATA disks connected to two $60 4 port SiI 3124 PCIe x1 cards?
According to Dave, with his patches you should even outperform that if you got faster CPUs :-)
Emmanuel, where is the "mk1BFiles" Benchmark? We're planning for a new hardware this year, so this would be a good time to run it. Could I have the script?
The output misses the time "ls" took, but I can one can calculate that from the next test start, so 9:30-7:20 = 2:10, is that correct? Two hours to just see all files, ugh. I guess it will take some years until we want to have such a filesystem. Either hardware must become quicker, or another wonderful new patch is needed.
Signature · 12 lines
// Michael Monnerie, Ing.BSc. ---------------------------------- Sorcerers have their magic wands: powerful, potentially dangerous tools with a life of their own. Witches have their familiars: creatures disguised as household beasts that could, if they choose, wreak the witches' havoc. Mystics have their golems: beings built of wood and tin brought to life to do their masters' bidding. I have Linux. ----------------------------------
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: LWN.net article: creating 1 billion files -> Tests we did
In reply to Stan Hoeppner[ ... useless run of something misrepresented as a test ... ]
Quoted 6 lines
Quoted 2 lines
Test run on a dual Opteron quad core, 16 GB RAM, kernel 2.6.32.11
x86_64...This is a test of storage system performance, and you left out the storage array specs? By doing so it seems you're stating the underlying storage is not relevant to the results. [ ... ]
This is only one of the several aspects of the waste of time that was misrepresented as a storage test, even if it is one of the funniest.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: LWN.net article: creating 1 billion files -> Tests we did
Le Fri, 17 Sep 2010 20:29:01 +0100 vous écriviez:
Quoted 1 line
[ ... useless run of something misrepresented as a test ... ]
I won't comment on the usefulness of your rant, though, because I'd rather stay amiable.
If you don't mind, I'll however post complementary results of the same test running on different filesystems on the very same hardware, because apparently you missed the fact that this was about tracking comparative xfs progress in some metadata intensive workloads.
Signature · 11 lines
------------------------------------------------------------------------
Emmanuel Florac | Direction technique
| Intellique
| <[email protected]>
| +33 1 78 94 84 02
------------------------------------------------------------------------
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfsRe: LWN.net article: creating 1 billion files -> Tests we did
In reply to Stan HoeppnerLe Thu, 16 Sep 2010 16:53:07 -0500 vous écriviez:
Quoted 3 lines
This is a test of storage system performance, and you left out the storage array specs? By doing so it seems you're stating the underlying storage is not relevant to the results.
Sorry :) The storage is a 24 2TB disks RAID-6 array on a 3W&re 9650. Not exactly stellar at IOPS performance.
Quoted 4 lines
So, are you saying I should be able to duplicate your results with that dual Opty system, but using an md RAID0 stripe over 8x2TB SATA disks connected to two $60 4 port SiI 3124 PCIe x1 cards?
Your setup may be slightly slower.
Signature · 11 lines
------------------------------------------------------------------------
Emmanuel Florac | Direction technique
| Intellique
| <[email protected]>
| +33 1 78 94 84 02
------------------------------------------------------------------------
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfsRe: LWN.net article: creating 1 billion files -> Tests we did
In reply to Emmanuel FloracQuoted 4 lines
Quoted 4 lines
The subject is a bit harsh, but overall the article says: XFS is slowest on creating and deleting a billion files XFS fsck needs 30GB RAM to fsck that 100TB filesystem.
Hahahaha. Very funny. So what?
Quoted 1 line
LWN is usually fairly decent, but I have noticed it does occasionally waste pixels on/bits things that the author(s) misrepresent as storage or file system tests.
However in this case the main takeaway of the presentation reported is that it is just a bad idea to assume that file systems can scale to large collections of small files as DBMSes designed for that purpose. So what?
Quoted 3 lines
So We've made a test with 1KB files (space, space...) and a production kernel : 2.6.32.11 (yeah I know, 2.6.38 should be faster but you know, we upgrade our production kernels prudently :).
Why is this a test of anything other than how to waste time?
Quoted 6 lines
mk1BFiles will create and delete 1000000000 files with 32 threads Version: v0.2.4-10-gf6decd3, build: Sep 7 2010 13:39:34
Creating 1000000000 files, started at 2010-09-07 13:45:16... Done, time spent: 89:35:12.262
Was there any intervening cache flush?
Quoted 4 lines
Doing
ls -R, started at 2010-09-11 07:20:28...
Stat: ls (pid: 18844) status: ok, returned value: 0 Cpu usage: user: 1:27:47.242, system: 20:18:21.689 Max rss: 229.01 MBytes, page fault: major: 4, minor: 58694
Was there any intervening cache flush?
Quoted 5 lines
Compute size used by 1000000000 files, started at 2010-09-12 09:30:52... Size used by files: 11.1759 TBytes
Size used by directory: 32.897 GBytes
Size used (total): 11.2080 TBytes
Done, time spent: 25:50:32.355
Was there any intervening cache flush?
Quoted 2 lines
Deleting 1000000000 files, started at 2010-09-13 11:21:24... Done, time spent: 68:37:38.117
Was there any intervening cache flush?
Why would anybody with even a little knowledge of computers and systems want to use a filesystem as database for small records?
Quoted 2 lines
Test run on a dual Opteron quad core, 16 GB RAM, kernel 2.6.32.11
x86_64...
So what?
Some of the most amusing quotes from the LWN article are from the comments.
"Recently I did similiar tests for determining how well PostgreSQL
would be able to deal with databases with potentially hundreds of thousands of tables. From what I found out, it's only limited by the file system's ability to work with that many files in a single directory."
HHAHAHAHAHAHAHA.
"> But in what situations will it make more sense to not group a
Quoted 1 line
billion of file items into logical groups?
Things like squid cache directories, git object directories, ccache cache directories, that hidden thumbnails directory in your $HOME... They all have in common that the files are named by a hash or something similar. There is no logical grouping at all here; it is a completely flat namespace."
AAAAAAGGGGGHHHHHHHHHHH.
But the original presentation has absolutely the funniest bit:
"Why Not Use a Database?
● Users and system administrators are familiar
with file systems
Backup, creation, etc are all well understood
● File systems handle partial failures pretty well
Being able to recover part of the stored data is useful for some applications
● File systems are “cheap” since they come with
your operating system!"
My evil translation of that is "because so many sysadms and programmers are incompetent and stupid and wish for ponies".
Of course the best bit is where someone :-) was quoted making sense:
“Millions of files may work; but 1 billion is an utter absurdity. A filesystem that can store reasonably 1 billion small files in 7TB is an unsolved research issue...,”
The stupidest bit of the presentation was part of the quoted reply:
“Strangely enough, I have been testing ext4 and stopped filling it at a bit over 1 billion 20KB files on Monday (with 60TB of storage). Running fsck on it took only 2.4 hours.”
Where the idea that the 'fsck' time that matters is that of a freshly created (and was the page cache flushed?), uncorrupted filesystem is intensely comical. "Possible" does not mean "reasonably". Just delirious.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: LWN.net article: creating 1 billion files -> Tests we did
Le Fri, 17 Sep 2010 20:57:48 +0100 vous écriviez:
Quoted 3 lines
LWN is usually fairly decent, but I have noticed it does occasionally waste pixels on/bits things that the author(s) misrepresent as storage or file system tests.
How unfortunate we missed your precious stance on that matter. Everybody knows that benchmarks are mostly useless /per se/, however it often occurs that comparative benchmarks may easily reveal some interesting differences.
As for the interest of an experiment of pushing something to the limit for the sake of it, it may equally reveal interesting bugs. The fact that all filesystems in this tests didn't simply *fail* under the load is by itself revealing of the overall robustness and stability of these, the VFS and Linux kernel.
As a side note, were you using a slightly less harsh tone, you'd probably be help people being less reluctant to discuss those points more deeply.
Signature · 11 lines
------------------------------------------------------------------------
Emmanuel Florac | Direction technique
| Intellique
| <[email protected]>
| +33 1 78 94 84 02
------------------------------------------------------------------------
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs