Skip to contentSkip to conversation

Bonnie++ Burps on XFS

DISCUSSION Kurt Wall 3 participants 4 messages Apr 6, 2006

lore
Share & tools
lore

Kurt Wall

author
#0 lore raw

I've been using bonnie++ off and on for a long time. Suddenly, it has started failing when run against an XFS filesystem situated on a SATA drive. Here's the output of a run:

$ bench/sbin/bonnie++ -m "Luther 2.6.17-rc1 XFS SATA" \

Review

Quoted 3 lines
(quoted diff) +0 -3
-d /archive/bench \
-s 3008 \
-n 1024 >> bonnie.out 2>> bonnie.err

Version: 1.03
Writing with putc()...done
Writing intelligently...done
Rewriting...done
Reading with getc()...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...Bonnie: drastic I/O error (rmdir):
Directory not empty
Cleaning up test directory after error.

This is on a 2.6.17-rc1 Linux kernel. The interesting thing is that not even root cannot delete the directory in which bonnie++ is working:

[~]$ ls -ld /archive/bench/Bonnie.8426/
drwx------ 2 kwall users 4096 2006-04-05 09:06 /archive/bench/Bonnie.8426/ [~]$ ls -al /archive/bench/Bonnie.8426/
total 24
drwx------ 2 kwall users 4096 2006-04-05 09:06 ./
drwxr-xr-x 4 kwall users 42 2006-04-05 06:24 ../
[~]$ sudo rm -rf /archive/bench/Bonnie.8426
rm: cannot remove directory `/archive/bench/Bonnie.8426': Directory not empty

As you can see, the directory is empty, but a "rm" fails (as does "rmdir"). I'm retesting with a 2.6.16 kernel as I type. It isn't clear if the problems is XFS, some other piece of the kernel, or bonnie++. Neither dmesg nor syslog shows anything amiss. I suppose I could strace the run, but I'm not especially eager to deal with a multi-gigabyte output file if there is a more focused method to isolate the problem.

I've also written to bonnie++'s author.

Thanks for the help.

Best regards,

Kurt

Signature · 2 lines
The full impact of parenthood doesn't hit you until you multiply the
number of your kids by 32 teeth.
#1 lore raw
Quoted 3 lines · Kurt Wall

I've been using bonnie++ off and on for a long time. Suddenly, it has started failing when run against an XFS filesystem situated on a SATA drive. Here's the output of a run:

[ Please report these things to [email protected]... ]

Quoted 1 line

Delete files in sequential order...Bonnie: drastic I/O error (rmdir):

Anything in your system log?

Quoted 5 lines

clear if the problems is XFS, some other piece of the kernel, or bonnie++. Neither dmesg nor syslog shows anything amiss. I suppose I could strace the run, but I'm not especially eager to deal with a multi-gigabyte output file if there is a more focused method to isolate the problem.

See if 2.6.16.1 fails too for starters then we'll take it from there.

cheers.

Signature · 1 line
Nathan
#2 lore raw
Quoted 10 lines · Nathan Scott
Quoted 3 lines · Kurt Wall

I've been using bonnie++ off and on for a long time. Suddenly, it has started failing when run against an XFS filesystem situated on a SATA drive. Here's the output of a run:

[ Please report these things to [email protected]... ]

Quoted 1 line

Delete files in sequential order...Bonnie: drastic I/O error (rmdir):

Anything in your system log?

Lemme answer that for you - "no". I've reproduced the problem, I'll get back to you once I've nutted out whats gone wrong.

Thanks for reporting it.

cheers.

Signature · 1 line
Nathan
#3 lore raw
Quoted 22 lines · Nathan Scott
Quoted 8 lines · Nathan Scott
Quoted 5 lines · Kurt Wall

I've been using bonnie++ off and on for a long time. Suddenly, it has started failing when run against an XFS filesystem situated on a SATA drive. Here's the output of a run:

Quoted 10 lines

[ Please report these things to [email protected]... ]

Quoted 3 lines

Delete files in sequential order...Bonnie: drastic I/O error (rmdir):

Anything in your system log?

Quoted 9 lines

Lemme answer that for you - "no". I've reproduced the problem, I'll get back to you once I've nutted out whats gone wrong.

Thanks for reporting it.

cheers.

Me too.

2.6.16

I had filesystem corruption and needed ran xfs_repair.

After I finished I removed most of the lost and found bits but was left with:

haze:/scratch/lost+found# ll
total 28
drwxr-xr-x   4 root root   38 2006-04-02 08:29 ./
drwxrwxrwx  26 root root 4096 2006-04-04 18:25 ../
drwxrwxr-x   2 1046 1046 8192 2006-04-02 08:30 658616481/
drwxrwxr-x   2 1046 1046 8192 2006-04-02 08:22 823441168/
haze:/scratch/lost+found# rmdir *
rmdir: `658616481': Directory not empty
rmdir: `823441168': Directory not empty
haze:/scratch/lost+found# ll *
658616481:
total 12
drwxrwxr-x  2 1046 1046 8192 2006-04-02 08:30 ./
drwxr-xr-x  4 root root   38 2006-04-02 08:29 ../

823441168:
total 12
drwxrwxr-x 2 1046 1046 8192 2006-04-02 08:22 ./
drwxr-xr-x 4 root root 38 2006-04-02 08:29 ../

Obviously I had an fs corruption (due to raid failure due to some bogus libata errors) so this may not be the same thing.

David