severe hardlink bug
I'm using btrfs with a kernel 2.6.32.2 (builtin) as the root file system of a Gentoo Linux installation.
While attempting to install the plt-scheme package a strange error about link counts occurred ([Error 31] Too many Links).
I was able to create a simple scenario to reproduce the error:
Create an empty directory and run the following script within the directory:
#!/bin/bash
echo "abc" >.tab-snip-class_scrbl.dep._portage_merge_.9766 for x in $(seq 80)
do echo $x; ln .tab-snip-class_scrbl.dep._portage_merge_.9766 \
.tab-snip-class_scrbl.dep._portage_merge_.9766.$x; done
(The names are taken from the original Gentoo package.)
When I run this script the first 66 links are created properly. All further link creations fail:
...
63
64
65
66
67
ln: creating hard link to `.tab-snip-class_scrbl.dep._portage_merge_.9766': Too many links 68
ln: creating hard link to `.tab-snip-class_scrbl.dep._portage_merge_.9766': Too many links 69
ln: creating hard link to `.tab-snip-class_scrbl.dep._portage_merge_.9766': Too many links 70
ln: creating hard link to `.tab-snip-class_scrbl.dep._portage_merge_.9766': Too many links
...
I was able to reproduce the bug independently on two different machines.
During earlier tests I determined that filename length has an effect as well as the link count.
Greetings, Michael
Quoted 1 line · Michael Niederle
I'm using btrfs with a kernel 2.6.32.2 (builtin) as the root file system of
a
Quoted 3 lines
Gentoo Linux installation.
While attempting to install the plt-scheme package a strange error about
link
Quoted 1 line
counts occurred ([Error 31] Too many Links).
See this thread:
Mass-Hardlinking Oops - http://thread.gmane.org/gmane.comp.file-
systems.btrfs/3427
There is a limit of the number of hardlink for a file. The maximum number of the link depends by name length.
BR
Goffredo
Signature · 2 lines
gpg key@ keyserver.linux.it: Goffredo Baroncelli (ghigo) <kreijackATinwind.it> Key fingerprint = 4769 7E51 5293 D36C 814E C054 BF04 F161 3DC5 0512
Quoted 9 lines · On Sun, 24 Jan 2010 09:09:44 +0100 Goffredo Baroncelli <[email protected]>…
Quoted 1 line · Michael Niederle
I'm using btrfs with a kernel 2.6.32.2 (builtin) as the root file system of
a
Quoted 3 lines
Gentoo Linux installation.
While attempting to install the plt-scheme package a strange error about
link
Quoted 1 line
counts occurred ([Error 31] Too many Links).
Quoted 7 lines
See this thread:
Mass-Hardlinking Oops - http://thread.gmane.org/gmane.comp.file-
systems.btrfs/3427There is a limit of the number of hardlink for a file. The maximum number of the link depends by name length.
Honestly, this dependency is braindead. How do the fs authors think an application programmer should judge how many hardlinks are possible for a certain fs ? This is a really bad design issue. Are we really in year 2010 ?
Quoted 2 lines
BR
Goffredo
Signature · 2 lines
Regards, Stephan
Quoted 17 lines · Stephan von Krawczynski
Quoted 14 lines · On Sun, 24 Jan 2010 09:09:44 +0100 Goffredo Baroncelli <[email protected]>…
Quoted 1 line · Michael Niederle
I'm using btrfs with a kernel 2.6.32.2 (builtin) as the root file system of
a
Quoted 3 lines
Gentoo Linux installation.
While attempting to install the plt-scheme package a strange error about
link
Quoted 1 line
counts occurred ([Error 31] Too many Links).
See this thread:
Mass-Hardlinking Oops - http://thread.gmane.org/gmane.comp.file-
systems.btrfs/3427
Quoted 6 lines
Quoted 2 lines
There is a limit of the number of hardlink for a file. The maximum number of the link depends by name length.
Honestly, this dependency is braindead. How do the fs authors think an application programmer should judge how many hardlinks are possible for a certain fs ? This is a really bad design issue. Are we really in year 2010 ?
Yes, this was a design trade off to maintain the back links for the file. Keep in mind the limit is only per-directory, as long as you're working between different directories you can have any many back links as you want.
But, it's on the list to get fixed once raid5/6 are out of the way.
-chris
Hi, Goffredo!
Quoted 4 lines
See this thread:
Mass-Hardlinking Oops - http://thread.gmane.org/gmane.comp.file-
systems.btrfs/3427
Sorry, I didn't find the thread myself - despite of massive googeling. :-(
The gentoo scenario however is a real world application - no artificial situation, so it would be really nice to fix the hard link problem somewhere in the future!
The problem will happen in all busybox-like scenarios - and long filenames are not so rarely used.
Greetings, Michael
Quoted 5 lines · you (Dipl.-Ing. Michael Niederle)
Hi, Goffredo!
Quoted 3 lines
See this thread:
Mass-Hardlinking Oops -
May be that the link was broken in two line in the previous mail. I copied again the ling. please give a try to see the thread.
http://thread.gmane.org/gmane.comp.file-systems.btrfs/3427
Quoted 5 lines
Sorry, I didn't find the thread myself - despite of massive googeling. :-(
The gentoo scenario however is a real world application - no artificial situation, so it would be really nice to fix the hard link problem somewhere
in
Quoted 1 line
the future!
At the time of the thread, some peoples raised concerns about this limit. But at the time nobody highlighted a real case which required a lot of hard links.
Pay attention that in order to reach the limits the hard links have to be in the same directory.
If there is a real case in gentoo/bosybox, please send all the info. But in order solve this problem, it was told that a disk format change is required.
BR
Goffredo
Quoted 1 line
The problem will happen in all busybox-like scenarios - and long filenames
are
Quoted 4 lines
not so rarely used.
Greetings, Michael
Signature · 2 lines
gpg key@ keyserver.linux.it: Goffredo Baroncelli (ghigo) <kreijackATinwind.it> Key fingerprint = 4769 7E51 5293 D36C 814E C054 BF04 F161 3DC5 0512
Hi, Goffredo!
I understood that a disk layout change will be necessary to accomodate for a bigger total filename-space per file. So it is best done at once when more issues arise that will need layout changes.
To recreate the error situation you need a Gentoo Linux with a btrfs root file system and then try to install the plt-scheme package:
Quoted 1 line
emerge plt-scheme
Compilation works quite fine, but the installation aborts with a "Too many links" error.
I reinstalled over 700 packages - plt-scheme beeing the only one failing due to the btrfs link restriction.
Greetings, Michael
Quoted 3 lines · Dipl.-Ing. Michael Niederle <mniederle <at> gmx.at> writes:
I reinstalled over 700 packages - plt-scheme beeing the only one failing due to the btrfs link restriction.
I have hit the same issue - tried to run BackupPC with a pool on btrfs filesystem. After some time the error of "too many links (31)" appeared to me. Now I'm forced to migrate to some other filesystem...
Quoted 9 lines · Konstantin Dmitriev
Quoted 3 lines · Dipl.-Ing. Michael Niederle <mniederle <at> gmx.at> writes:
I reinstalled over 700 packages - plt-scheme beeing the only one failing due to the btrfs link restriction.
I have hit the same issue - tried to run BackupPC with a pool on btrfs filesystem. After some time the error of "too many links (31)" appeared to me. Now I'm forced to migrate to some other filesystem...
btrfs only fails when you have hundreds of hardlinks to the same file in the *same* directory ... certainly not a standard use case.
use snapshots to your advantage:
- snap source
- rsync --inplace source to target (with some other opts that have
been discussed on list)
- snap target
- {rinse-and-repeat-in-24-hrs}
Signature · 1 line
C Anthony
Quoted 9 lines · C Anthony Risinger <anthony <at> xtfx.me> writes:
btrfs only fails when you have hundreds of hardlinks to the same file in the *same* directory ... certainly not a standard use case.
use snapshots to your advantage:
- snap source
- rsync --inplace source to target (with some other opts that have
been discussed on list)
- snap target
- {rinse-and-repeat-in-24-hrs}
I understand that the case is only for *same* directory.
You can claim that it's not a standard use case, but first Michael hit that, now me. There's at least one more case -
https://lists.samba.org/archive/rsync/2011-December/027117.html
The count of such cases will be increasing and the sooner it will be fixed - the less pain it will bring to the users. I know fixing that is a big structural change, but it will become worse with time.
If it's not going to be fixed - I don't care. Right now I'm forced to migrate to old mdadm raid-1 or ZFS. The sad thing is that
I really LOVED btrfs. Only that. ^__^
K.
Am 29.07.2012 21:13, schrieb C Anthony Risinger:
Quoted 8 lines
Quoted 6 lines · Konstantin Dmitriev
Quoted 3 lines · Dipl.-Ing. Michael Niederle <mniederle <at> gmx.at> writes:
I reinstalled over 700 packages - plt-scheme beeing the only one failing due to the btrfs link restriction.
Quoted 6 lines
Quoted 3 lines
I have hit the same issue - tried to run BackupPC with a pool on btrfs filesystem. After some time the error of "too many links (31)" appeared to me. Now I'm forced to migrate to some other filesystem...
btrfs only fails when you have hundreds of hardlinks to the same file in the *same* directory ... certainly not a standard use case.
Actually, "hundreds of hardlinks" is certainly over optimistic. In my testing 15 links in the same directory were enough to get the "Too many links" error. It depends on the length of the file name of the hardlinks.
Best regards
Arnd
Quoted 13 lines · Arnd Hannemann
Am 29.07.2012 21:13, schrieb C Anthony Risinger:
Quoted 12 lines
Quoted 9 lines · Konstantin Dmitriev
Quoted 3 lines · Dipl.-Ing. Michael Niederle <mniederle <at> gmx.at> writes:
I reinstalled over 700 packages - plt-scheme beeing the only one failing due to the btrfs link restriction.
I have hit the same issue - tried to run BackupPC with a pool on btrfs filesystem. After some time the error of "too many links (31)" appeared to me. Now I'm forced to migrate to some other filesystem...
Quoted 7 lines
Quoted 2 lines
btrfs only fails when you have hundreds of hardlinks to the same file in the *same* directory ... certainly not a standard use case.
Actually, "hundreds of hardlinks" is certainly over optimistic. In my testing 15 links in the same directory were enough to get the "Too many links" error. It depends on the length of the file name of the hardlinks.
Yes, per the linked patch it states 4k as the limit ... I thought I recalled a limit of 256 but it seems I may have been mistaken.
The purpose of my initial response was to suggest an alternative strategy -- one complementing btrfs's strengths -- a simple rsync + snapshot is much more effective than BackupPC IMO ... but then again, I'm bias, because I generally think BackupPC is junk.
--
C Anthony
Hallo, Arnd,
Du meintest am 30.07.12:
Quoted 7 lines
Quoted 2 lines
btrfs only fails when you have hundreds of hardlinks to the same file in the *same* directory ... certainly not a standard use case.
Actually, "hundreds of hardlinks" is certainly over optimistic. In my testing 15 links in the same directory were enough to get the "Too many links" error. It depends on the length of the file name of the hardlinks.
And then I may get problems with my favourite backup program "rsnapshot"; it uses hard links, and my backups often have more than 20 hard links to 1 file. Sometimes about 80 characters long.
Viele Gruesse!
Helmut
Hi Konstantin,
Quoted 9 lines · Konstantin Dmitriev
Quoted 3 lines · Dipl.-Ing. Michael Niederle <mniederle <at> gmx.at> writes:
I reinstalled over 700 packages - plt-scheme beeing the only one failing due to the btrfs link restriction.
I have hit the same issue - tried to run BackupPC with a pool on btrfs filesystem. After some time the error of "too many links (31)" appeared to me. Now I'm forced to migrate to some other filesystem...
Please give the patch set "btrfs: extended inode refs" by Mark Fasheh a try
(http://lwn.net/Articles/498226/). It eliminates the hard links per directory limit (introducing a rather random, artificial limit of 64k instead).
-Jan
Quoted 3 lines · Jan Schmidt <list.btrfs <at> jan-o-sch.net> writes:
Please give the patch set "btrfs: extended inode refs" by Mark Fasheh a try
(http://lwn.net/Articles/498226/). It eliminates the hard links per directory limit (introducing a rather random, artificial limit of 64k instead).
Hi, Jan!
I'm happy to see that there is something done on fixing that issue. Unfortunately I cannot afford to have unstable patched kernel on the production server. Probably I'll give another try to btrfs in 2013. ^__^
K.
Quoted 2 lines · Konstantin Dmitriev
Unfortunately I cannot afford to have unstable patched kernel on the production server.
I'd suggest that using btrfs for production servers is probably a bit brave (no matter what Oracle say), and what you've found illustrates the point rather well I'm afraid..
cheers,
Chris
Signature · 1 line
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
Am Sonntag, 24. Januar 2010 schrieb Michael Niederle:
Quoted 2 lines
I'm using btrfs with a kernel 2.6.32.2 (builtin) as the root file system of a Gentoo Linux installation.
Upgrade your kernel!
This kernel is waaaaaaaaaaaaaaaaaaaaaaaay to old for any production use of BTRFS. Heck, upstream still did not mark BTRFS as stable although Oracle and SUSE do support it. Current kernel have a many fixes.
Use at least 3.2. And have backups anyway.
The hard-link issue is known since ages and being worked on (see link from Jan in this thread). If you want to help, test the patch.
So, folks, can we get over it?
Thanks,
Signature · 2 lines
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7