Skip to contentSkip to conversation

"btrfs send" fails with having too many open fd's

DISCUSSION Robert Buhren 2 participants 3 messages Apr 7, 2013 – Apr 8, 2013 Btrfs file system

lore
Share & tools
lore
#0 lore raw

Hi there,

i'm trying to send/receive a snapshot of my home dir for some time now, but in the end it always fails because the maximum number of open fd's is exceeded.

Kern log says:

"6,7226,15012711248,-;VFS: file-max limit 799969 reached"

My setup is as follows:

Kernel: 3.9-rc5
btrfs-progs: compiled from git commit: commit 7854c8b667654502f69e05584729146a06827bc6

btrfs fi show:

Label: none uuid: ccedbef9-417b-454c-b1d4-8b4220111e6a
Total devices 1 FS bytes used 177.79GB
devid 1 size 210.85GB used 210.85GB path /dev/dm-0

Label: 'BACKUP' uuid: 019e75e4-aa2e-495d-9711-28ef7150b1f3

    Total devices 2 FS bytes used 137.35GB
    devid    2 size 298.09GB used 142.01GB path /dev/sdc1
    devid    1 size 298.09GB used 142.03GB path /dev/sdb1

The send fs is on a dm-crypt device and the receive on a btrfs raid1
A scrub for both fs ran fine without errors.I can however send/receive my root fs which is much smaller (~17GB).

The error occurs also if i just use "btrfs send" and pipe the output to a file.
It takes a very long time before my system crashes (several hours) so i wasn't able to monitor when exactly the fd's increase. In the beginning "btrfs send" just opens less than 10 fd's, but at some point "btrfs send" must open much more.

If further information is needed, i'd be happy to help. I'm also in #btrfs on freenode.

Regards,

Robert

PS: Please respond to my email, i'm not subscribed to the list.

#1 lore raw
Quoted 9 lines · Robert Buhren

"6,7226,15012711248,-;VFS: file-max limit 799969 reached"

Label: 'BACKUP' uuid: 019e75e4-aa2e-495d-9711-28ef7150b1f3
Total devices 2 FS bytes used 137.35GB

It takes a very long time before my system crashes (several hours) so i wasn't able to monitor when exactly the fd's increase.
In the beginning "btrfs send" just opens less than 10 fd's, but at some point "btrfs send" must open much more.

It could be a case of

1) an endless loop, the file limit you've set is quite high (799969), I doubt that so many open fd's are needed at any point.

2) missing file close after in some error path

The file's are open via
send_write
open_cur_inode_file
dentry_open

and closed from callbacks in close_cur_inode_file.

Case 2 looks more realistic, but I don't have an idea what could go wrong.

david

#2 lore raw
Quoted 20 lines · David Sterba
Quoted 7 lines · Robert Buhren

"6,7226,15012711248,-;VFS: file-max limit 799969 reached"
Label: 'BACKUP' uuid: 019e75e4-aa2e-495d-9711-28ef7150b1f3
Total devices 2 FS bytes used 137.35GB
It takes a very long time before my system crashes (several hours) so i wasn't able to monitor when exactly the fd's increase.
In the beginning "btrfs send" just opens less than 10 fd's, but at some point "btrfs send" must open much more.

It could be a case of

1) an endless loop, the file limit you've set is quite high (799969), I doubt that so many open fd's are needed at any point.

2) missing file close after in some error path

The file's are open via
send_write
open_cur_inode_file
dentry_open

Quoted 6 lines

and closed from callbacks in close_cur_inode_file.

Case 2 looks more realistic, but I don't have an idea what could go wrong.

david

Do you have a suggestion on how to debug it further?

Regards,

Robert