Activating space_cache after read-only snapshots without space_cache have been taken
Hello everyone,
I've ran into problems with _very_ slow unmounting of my btrfs-formatted "backup" volume. I have a suspicion what might be the cause but maybe someone with more experience with the btrfs code could enlighten me whether it is actually correct.
The situation is the following: I have created a backup-volume to which I regularly rsync a backup of my system into a subvolume. After rsync'ing, I take a _read-only_ snapshot of that subvolume with a timestamp added to its name.
Now at the time I started using this backup volume, I was _not_ using the space_cache mount option and two read-only snapshots were taken during this time. Then I started using the space_cache option and continued doing snapshots.
A bit later, I started having very long lags when unmounting the backup volume (both during shutdown and when unmounting manually). I scrubbed and fsck'd the volume but this didn't show any errors. Defragmenting the root and subvolumes took a long time but didn't improve the situation much.
Now I started having the suspicion that maybe the space cache possibly couldn't be written to disk for the readonly subvolumes/snapshots that were created during the time when I wasn't using the space_cache option, forcing the cache to be rebuilt every time.
Clearing the cache didn't help. But when I deleted the two snapshots that I think were taken during the time without the mount option, the unmounting time seems to have improved considerably.
I will have to observe whether unmounting stays quick now but my question is whether it is possible that the read-only snapshots taken during the time when I wasn't using space_cache might actually have been the culprits.
Best,
Sebastian
Quoted 17 lines · Ochi
Hello everyone,
I've ran into problems with _very_ slow unmounting of my btrfs-formatted "backup" volume. I have a suspicion what might be the cause but maybe someone with more experience with the btrfs code could enlighten me whether it is actually correct.
The situation is the following: I have created a backup-volume to which I regularly rsync a backup of my system into a subvolume. After rsync'ing, I take a _read-only_ snapshot of that subvolume with a timestamp added to its name.
Now at the time I started using this backup volume, I was _not_ using the
space_cachemount option and two read-only snapshots were taken during this time. Then I started using thespace_cacheoption and continued doing snapshots.
Quoted 5 lines
A bit later, I started having very long lags when unmounting the backup volume (both during shutdown and when unmounting manually). I scrubbed and fsck'd the volume but this didn't show any errors. Defragmenting the root and subvolumes took a long time but didn't improve the situation much.
So are you using '-o nospace_cache' when creating two RO snapshots?
Quoted 10 lines
Now I started having the suspicion that maybe the space cache possibly couldn't be written to disk for the readonly subvolumes/snapshots that were created during the time when I wasn't using the
space_cacheoption, forcing the cache to be rebuilt every time.Clearing the cache didn't help. But when I deleted the two snapshots that I think were taken during the time without the mount option, the unmounting time seems to have improved considerably.
I don't know why this happens, but maybe you can observe the umount process's very slow behaviour by using 'cat /proc/{umount-pid}/stack' or 'perf top'.
thanks,
liubo
Quoted 12 lines
I will have to observe whether unmounting stays quick now but my question is whether it is possible that the read-only snapshots taken during the time when I wasn't using
space_cachemight actually have been the culprits.Best,
Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Liu Bo wrote (ao):
Quoted 11 lines · Ochi
Quoted 10 lines
The situation is the following: I have created a backup-volume to which I regularly rsync a backup of my system into a subvolume. After rsync'ing, I take a _read-only_ snapshot of that subvolume with a timestamp added to its name.
Now at the time I started using this backup volume, I was _not_ using the
space_cachemount option and two read-only snapshots were taken during this time. Then I started using thespace_cacheoption and continued doing snapshots.
Quoted 7 lines
Quoted 5 lines
A bit later, I started having very long lags when unmounting the backup volume (both during shutdown and when unmounting manually). I scrubbed and fsck'd the volume but this didn't show any errors. Defragmenting the root and subvolumes took a long time but didn't improve the situation much.
So are you using '-o
nospace_cache' when creating two RO snapshots?
No, he first created two ro snapshots, then (some time later) mounted with nospace_cache, and then continued to take ro snapshots.
Quoted 6 lines
Quoted 6 lines
Now I started having the suspicion that maybe the space cache possibly couldn't be written to disk for the readonly subvolumes/snapshots that were created during the time when I wasn't using the
space_cacheoption, forcing the cache to be rebuilt every time.
Quoted 7 lines
Quoted 3 lines
Clearing the cache didn't help. But when I deleted the two snapshots that I think were taken during the time without the mount option, the unmounting time seems to have improved considerably.
I don't know why this happens, but maybe you can observe the umount process's very slow behaviour by using 'cat
/proc/{umount-pid}/stack' or 'perf top'.
AFAIUI the problem is not there anymore, but this is a good tip for the future.
Sander
Quoted 18 lines · Sander
Liu Bo wrote (ao):
Quoted 17 lines · Ochi
Quoted 15 lines
The situation is the following: I have created a backup-volume to which I regularly rsync a backup of my system into a subvolume. After rsync'ing, I take a _read-only_ snapshot of that subvolume with a timestamp added to its name.
Now at the time I started using this backup volume, I was _not_ using the
space_cachemount option and two read-only snapshots were taken during this time. Then I started using thespace_cacheoption and continued doing snapshots.A bit later, I started having very long lags when unmounting the backup volume (both during shutdown and when unmounting manually). I scrubbed and fsck'd the volume but this didn't show any errors. Defragmenting the root and subvolumes took a long time but didn't improve the situation much.
Quoted 4 lines
Quoted 1 line
So are you using '-o
nospace_cache' when creating two RO snapshots?No, he first created two ro snapshots, then (some time later) mounted with
nospace_cache, and then continued to take ro snapshots.
I need to clarify this: The NOspace_cache option was never used, I just didn't explicitly activate space_cache in the beginning. However, I was not aware that space_cache is the default anyways (at least in Arch which is the distro I'm using). I reviewed old system logs and it actually looks like space caching was always being used right from the beginning, even when I didn't explicitly use the space_cache mount option. So I guess this wasn't the problem after all :\
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Now I started having the suspicion that maybe the space cache possibly couldn't be written to disk for the readonly subvolumes/snapshots that were created during the time when I wasn't using the
space_cacheoption, forcing the cache to be rebuilt every time.Clearing the cache didn't help. But when I deleted the two snapshots that I think were taken during the time without the mount option, the unmounting time seems to have improved considerably.
Quoted 8 lines
Quoted 3 lines
I don't know why this happens, but maybe you can observe the umount process's very slow behaviour by using 'cat
/proc/{umount-pid}/stack' or 'perf top'.AFAIUI the problem is not there anymore, but this is a good tip for the future.
Sander
That's correct, the problem has vanished after the deletion of the oldest two snapshots. Mounting and unmounting is reasonably fast now. I will just continue to use the volume normally (i.e. making regular backups and snapshotting) and report back if the problem appears again.
Just for the records: The btrfs volume and the first snapshots were originally created under kernel 3.7.10. I then updated to 3.8.3. I don't know if this information is useful - just in case... :)
Thanks,
Sebastian