Regression in btrfs: properly set the termination value of ctx->pos in readdir
Hi,
the patch btrfs: properly set the termination value of ctx->pos in readdir introduces a regression to me.
A lot of stuff runs in "endless" or long running loops.
An example strace looks like this:
msgsnd(0, {1, "\3\0\0\0\247\r\0\0g8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56,
0) = 0msgrcv(32769, {1, "\3\0\0\0\247\r\0\0g8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56, 0, 0) = 56semop(98307, {{0, 1, SEM_UNDO}}, 1) = 0newfstatat(AT_FDCWD, "changelog", {st_mode=S_IFREG|0644, st_size=148, ...}, AT_SYMLINK_NOFOLLOW) = 0semop(98307, {{0, -1, SEM_UNDO}}, 1) = 0
msgsnd(0, {1, "\3\0\0\0\247\r\0\0h8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56,
0) = 0msgrcv(32769, {1, "\3\0\0\0\247\r\0\0h8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56, 0, 0) = 56semop(98307, {{0, 1, SEM_UNDO}}, 1) = 0newfstatat(AT_FDCWD, "changelog", {st_mode=S_IFREG|0644, st_size=148, ...}, AT_SYMLINK_NOFOLLOW) = 0semop(98307, {{0, -1, SEM_UNDO}}, 1) = 0
msgsnd(0, {1, "\3\0\0\0\247\r\0\0i8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56,
0) = 0msgrcv(32769, {1, "\3\0\0\0\247\r\0\0i8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56, 0, 0) = 56semop(98307, {{0, 1, SEM_UNDO}}, 1) = 0newfstatat(AT_FDCWD, "changelog", {st_mode=S_IFREG|0644, st_size=148, ...}, AT_SYMLINK_NOFOLLOW) = 0semop(98307, {{0, -1, SEM_UNDO}}, 1) = 0
msgsnd(0, {1, "\3\0\0\0\247\r\0\0j8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56,
0) = 0msgrcv(32769, {1, "\3\0\0\0\247\r\0\0j8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56, 0, 0) = 56semop(98307, {{0, 1, SEM_UNDO}}, 1) = 0newfstatat(AT_FDCWD, "changelog", {st_mode=S_IFREG|0644, st_size=148, ...}, AT_SYMLINK_NOFOLLOW) = 0semop(98307, {{0, -1, SEM_UNDO}}, 1) = 0
msgsnd(0, {1, "\3\0\0\0\247\r\0\0k8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56,
0) = 0msgrcv(32769, {1, "\3\0\0\0\247\r\0\0k8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56, 0, 0) = 56semop(98307, {{0, 1, SEM_UNDO}}, 1) = 0newfstatat(AT_FDCWD, "changelog", {st_mode=S_IFREG|0644, st_size=148, ...}, AT_SYMLINK_NOFOLLOW) = 0semop(98307, {{0, -1, SEM_UNDO}}, 1) = 0
msgsnd(0, {1, "\3\0\0\0\247\r\0\0l8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56,
0) = 0msgrcv(32769, {1, "\3\0\0\0\247\r\0\0l8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56, 0, 0) = 56semop(98307, {{0, 1, SEM_UNDO}}, 1) = 0newfstatat(AT_FDCWD, "changelog", {st_mode=S_IFREG|0644, st_size=148, ...}, AT_SYMLINK_NOFOLLOW) = 0semop(98307, {{0, -1, SEM_UNDO}}, 1) = 0
msgsnd(0, {1, "\3\0\0\0\247\r\0\0m8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56,
0) = 0msgrcv(32769, {1, "\3\0\0\0\247\r\0\0m8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56, 0, 0) = 56semop(98307, {{0, 1, SEM_UNDO}}, 1) = 0newfstatat(AT_FDCWD, "changelog", {st_mode=S_IFREG|0644, st_size=148, ...}, AT_SYMLINK_NOFOLLOW) = 0semop(98307, {{0, -1, SEM_UNDO}}, 1) = 0
msgsnd(0, {1, "\3\0\0\0\247\r\0\0n8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56,
0) = 0msgrcv(32769, {1, "\3\0\0\0\247\r\0\0n8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56, 0, 0) = 56semop(98307, {{0, 1, SEM_UNDO}}, 1) = 0newfstatat(AT_FDCWD, "changelog", {st_mode=S_IFREG|0644, st_size=148, ...}, AT_SYMLINK_NOFOLLOW) = 0semop(98307, {{0, -1, SEM_UNDO}}, 1) = 0
msgsnd(0, {1, "\3\0\0\0\247\r\0\0o8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56,
0) = 0msgrcv(32769, {1, "\3\0\0\0\247\r\0\0o8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56, 0, 0) = 56semop(98307, {{0, 1, SEM_UNDO}}, 1) = 0newfstatat(AT_FDCWD, "changelog", {st_mode=S_IFREG|0644, st_size=148, ...}, AT_SYMLINK_NOFOLLOW) = 0semop(98307, {{0, -1, SEM_UNDO}}, 1) = 0
msgsnd(0, {1, "\3\0\0\0\247\r\0\0p8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56,
0) = 0msgrcv(32769, {1, "\3\0\0\0\247\r\0\0p8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56, 0, 0) = 56semop(98307, {{0, 1, SEM_UNDO}}, 1) = 0newfstatat(AT_FDCWD, "changelog", {st_mode=S_IFREG|0644, st_size=148, ...}, AT_SYMLINK_NOFOLLOW) = 0semop(98307, {{0, -1, SEM_UNDO}}, 1) = 0
msgsnd(0, {1, "\3\0\0\0\247\r\0\0q8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56,
0) = 0msgrcv(32769, {1, "\3\0\0\0\247\r\0\0q8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56, 0, 0) = 56semop(98307, {{0, 1, SEM_UNDO}}, 1) = 0newfstatat(AT_FDCWD, "changelog", {st_mode=S_IFREG|0644, st_size=148, ...}, AT_SYMLINK_NOFOLLOW) = 0semop(98307, {{0, -1, SEM_UNDO}}, 1) = 0
msgsnd(0, {1, "\3\0\0\0\247\r\0\0r8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56,
0) = 0msgrcv(32769, {1, "\3\0\0\0\247\r\0\0r8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56, 0, 0) = 56semop(98307, {{0, 1, SEM_UNDO}}, 1) = 0newfstatat(AT_FDCWD, "changelog", {st_mode=S_IFREG|0644, st_size=148, ...}, AT_SYMLINK_NOFOLLOW) = 0semop(98307, {{0, -1, SEM_UNDO}}, 1) = 0
msgsnd(0, {1, "\3\0\0\0\247\r\0\0s8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56,
0) = 0msgrcv(32769, {1, "\3\0\0\0\247\r\0\0s8\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56, 0, 0) = 56semop(98307, {{0, 1, SEM_UNDO}}, 1) = 0
{st_mode=S_IFREG|0644, st_size=148, ...}, AT_SYMLINK_NOFOLLOW) = 0semop(98307, {{0, -1, SEM_UNDO}}, 1) = 0msgsnd(0, {1, "\3\0\0\0\247\r\0\0\2358\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56, 0) = 0msgrcv(32769, {1, "\3\0\0\0\247\r\0\0\2358\0\0\0\0\0\0\0\0\0\0\345<\1\0\0\0\0\0\35\0\0\0"...}, 56, 0, 0) = 56semop(98307, {{0, 1, SEM_UNDO}}, 1) = 0
^C <unfinished ...>
Process 3495 detached
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
36.28 0.076892 10 7717 semop 24.25 0.051399 13 3858 newfstatat 21.48 0.045530 12 3858 msgsnd 17.46 0.036994 10 3859 msgrcv 0.52 0.001110 24 46 write
------ ----------- ----------- --------- --------- ----------------
100.00 0.211925 19338 total
Stefan
Quoted 6 lines · Stefan Priebe - Profihost AG
Hi,
the patch btrfs: properly set the termination value of
ctx->posin readdir introduces a regression to me.A lot of stuff runs in "endless" or long running loops.
Just tested this and can confirm something is off.
In a directory with several files, create a new directory and move all files into the new subdir.
An immediately following ls will hang. The problem goes away after a manual sync.
-h
Quoted 11 lines · Holger Hoffstätte
Quoted 4 lines
the patch btrfs: properly set the termination value of
ctx->posin readdir introduces a regression to me.A lot of stuff runs in "endless" or long running loops.
Just tested this and can confirm something is off.
In a directory with several files, create a new directory and move all files into the new subdir.An immediately following ls will hang. The problem goes away after a manual sync.
I can reproduce the hang, thanks.
Quoted 6 lines · David Sterba
Quoted 5 lines · Holger Hoffstätte
Quoted 4 lines
the patch btrfs: properly set the termination value of
ctx->posin readdir introduces a regression to me.A lot of stuff runs in "endless" or long running loops.
Quoted 8 lines
Quoted 6 lines
Just tested this and can confirm something is off.
In a directory with several files, create a new directory and move all files into the new subdir.An immediately following ls will hang. The problem goes away after a manual sync.
I can reproduce the hang, thanks.
The hang was triggered by use of trace_printk and I'm seeing several strange bugs caused by added debugging, so I can't say I can reproduce it yet.
Quoted 11 lines · David Sterba
Quoted 10 lines · David Sterba
Quoted 9 lines · Holger Hoffstätte
Quoted 4 lines
the patch btrfs: properly set the termination value of
ctx->posin readdir introduces a regression to me.A lot of stuff runs in "endless" or long running loops.
Just tested this and can confirm something is off.
In a directory with several files, create a new directory and move all files into the new subdir.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
An immediately following ls will hang. The problem goes away after a manual sync.
I can reproduce the hang, thanks.
The hang was triggered by use of
trace_printkand I'm seeing several strange bugs caused by added debugging, so I can't say I can reproduce it yet.
Be that as it may, the procedure I described above immediately started to work again when I backed out this patch. I also didn't noctice anything wrong before Stefan's email, so it's apparently not easy to hit.
-h
Quoted 4 lines · Stefan Priebe - Profihost AG
the patch btrfs: properly set the termination value of
ctx->posin readdir introduces a regression to me.A lot of stuff runs in "endless" or long running loops.
This might be related to the readdir fix but I don't see it anywhere in the strace log. There don't seem to be any connection (fd, other structures) to the syscalls in the strace log. I'll investigate further.