ext3 duplicate entries with readdir
In a largish directory (21292 files) a scan using ext3 readdir found 21293 files, returning one entry twice.
Google "ext3 readdir twice" gives me a handful of identical complaints. I see reports from 2002-2008.
So, this is an old and well-known symptom but seemingly still not fixed, at least not in the kernel on this machine here.
This report is for Fedora release 8 (Werewolf) with kernel 2.6.24.5.
A 2002 post says "This is unfortunate, but the chances of our hitting this failure are relatively small".
In http://www.redhat.com/archives/ext3-users/2008-August/msg00009.html
tytso seems to understand what happens: a hash collision. But has this been fixed in the current tree?
Andries
Thomas Trauner gave a test program. It prints
expected 25989 files, but readdir reports 25990
So, yes, this is reproducible.
Quoted 9 lines · Andries E. Brouwer
In a largish directory (21292 files) a scan using ext3 readdir found 21293 files, returning one entry twice.
Google "ext3 readdir twice" gives me a handful of identical complaints. I see reports from 2002-2008.
So, this is an old and well-known symptom but seemingly still not fixed, at least not in the kernel on this machine here.
This report is for Fedora release 8 (Werewolf) with kernel 2.6.24.5.
Quoted 6 lines
A 2002 post says "This is unfortunate, but the chances of our hitting this failure are relatively small".
In http://www.redhat.com/archives/ext3-users/2008-August/msg00009.html
tytso seems to understand what happens: a hash collision. But has this been fixed in the current tree?
Yes, it has been fixed, please see the following git commits:6a897cf447a83c9c3fd1b85a1e525c02d6eada7d 8c9fa93d51123c5540762b1a9e1919d6f9c4af7c
Mike
Quoted 11 lines · Mike Snitzer
Quoted 4 lines · Andries E. Brouwer
In a largish directory (21292 files) a scan using ext3 readdir found 21293 files, returning one entry twice.
Has this been fixed in the current tree?
Yes, it has been fixed, please see the following git commits:
6a897cf447a83c9c3fd1b85a1e525c02d6eada7d8c9fa93d51123c5540762b1a9e1919d6f9c4af7c
Good, thanks!
Andries
[On another machine, Ubuntu with 2.6.27-7-server still fails:
% ../readdirtest
expected 61005 files, but readdir reports 61006
% rm *
rm: cannot remove `59992': No such file or directory (where the same name occurs twice in the expansion of *).]
Quoted 11 lines · Andries E. Brouwer
Quoted 4 lines
Yes, it has been fixed, please see the following git commits:
6a897cf447a83c9c3fd1b85a1e525c02d6eada7d8c9fa93d51123c5540762b1a9e1919d6f9c4af7c[On another machine, Ubuntu with 2.6.27-7-server still fails:
% ../readdirtest
expected 61005 files, but readdir reports 61006
% rm *
rm: cannot remove `59992': No such file or directory (where the same name occurs twice in the expansion of *).]
The patches were backported to 2.6.27 stable in 2.6.27.8. So the problem would expected to be still present in an 2.6.27.7 kernel.
- Ted
Quoted 13 lines · Theodore Tso
Quoted 11 lines · Andries E. Brouwer
Quoted 4 lines
Yes, it has been fixed, please see the following git commits:
6a897cf447a83c9c3fd1b85a1e525c02d6eada7d8c9fa93d51123c5540762b1a9e1919d6f9c4af7c[On another machine, Ubuntu with 2.6.27-7-server still fails:
% ../readdirtest
expected 61005 files, but readdir reports 61006
% rm *
rm: cannot remove `59992': No such file or directory (where the same name occurs twice in the expansion of *).]
Quoted 4 lines
The patches were backported to 2.6.27 stable in 2.6.27.8. So the problem would expected to be still present in an 2.6.27.7 kernel.
- Ted
Does the bug also affect xfs? Because I see the problem there.
MfG
Goswin
Quoted 13 lines · Goswin von Brederlow
Quoted 11 lines · Theodore Tso
Quoted 10 lines · Andries E. Brouwer
Quoted 4 lines
Yes, it has been fixed, please see the following git commits:
6a897cf447a83c9c3fd1b85a1e525c02d6eada7d8c9fa93d51123c5540762b1a9e1919d6f9c4af7c[On another machine, Ubuntu with 2.6.27-7-server still fails:
% ../readdirtest
expected 61005 files, but readdir reports 61006
% rm *
rm: cannot remove `59992': No such file or directory (where the same name occurs twice in the expansion of *).]
Quoted 6 lines
Quoted 4 lines
The patches were backported to 2.6.27 stable in 2.6.27.8. So the problem would expected to be still present in an 2.6.27.7 kernel.
- Ted
Does the bug also affect xfs? Because I see the problem there.
Completely different, and the wrong list for the question, really.
Please point me to the bug you filed for this problem on xfs, I'll look into it (and when you do so please cc: the xfs list, and trim off the current ext4-related recipients)
Thanks,
-Eric
Quoted 23 lines · Eric Sandeen
Quoted 13 lines · Goswin von Brederlow
Quoted 11 lines · Theodore Tso
Quoted 10 lines · Andries E. Brouwer
Quoted 4 lines
Yes, it has been fixed, please see the following git commits:
6a897cf447a83c9c3fd1b85a1e525c02d6eada7d8c9fa93d51123c5540762b1a9e1919d6f9c4af7c[On another machine, Ubuntu with 2.6.27-7-server still fails:
% ../readdirtest
expected 61005 files, but readdir reports 61006
% rm *
rm: cannot remove `59992': No such file or directory (where the same name occurs twice in the expansion of *).]Quoted 6 lines
Quoted 4 lines
The patches were backported to 2.6.27 stable in 2.6.27.8. So the problem would expected to be still present in an 2.6.27.7 kernel.
- Ted
Does the bug also affect xfs? Because I see the problem there.
Completely different, and the wrong list for the question, really.
Quoted 7 lines
Please point me to the bug you filed for this problem on xfs, I'll look into it (and when you do so please cc: the xfs list, and trim off the current ext4-related recipients)
Thanks,
-Eric
Haven't filed a bug and can't reproduce the problem but I had 2 cases that behaved the same way so I wondered if it was the same issue. Would have ment I could stop worrying about it and wouldn't have to try to reproduce it.
MfG
Goswin