Skip to contentSkip to conversation

btrfs: initialize periodic_reclaim_ready to true

PATCH In review Sun YangKai 2 participants 5 messages Jan 26 – Jan 30 Btrfs file system

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 btrfs: initialize periodic_reclaim_ready to true

The periodic_reclaim_ready flag determines whether the background reclaim worker should process a specific space_info. Previously, this flag defaulted to false because space_info structures are zero-initialized.

According to the original design, periodic reclaim should be active from the start and only disable itself (set to false) if it fails to find reclaimable block groups.

Now that the reclaim condition has been fixed in a previous patch to properly handle reclaim_bytes, it is necessary to enable this by default. This ensures background reclaim logic kicks in as soon as the thresholds are met after mount.

Fixes 862dd2fd93540 ("btrfs: fix periodic reclaim condition")
CC Boris Burkov <[email protected]>
---
 fs/btrfs/space-info.c | 1 +
 1 file changed, 1 insertion(+)
fs/btrfs/space-info.c +1 -0
index bb5aac7ee9d2..1530c30cacc9 100644
@@ -250,6 +250,7 @@ static void init_space_info(struct btrfs_fs_info *info,
 	space_info->clamp = 1;
 	btrfs_update_space_info_chunk_size(space_info, calc_chunk_size(info, flags));
 	space_info->subgroup_id = BTRFS_SUB_GROUP_PRIMARY;
+	space_info->periodic_reclaim_ready = true;
 
 	if (btrfs_is_zoned(info))
 		space_info->bg_reclaim_threshold = BTRFS_DEFAULT_ZONED_RECLAIM_THRESH;
Signature · 1 line
2.52.0
#1 lore raw
Quoted 4 lines · Sun YangKai

The periodic_reclaim_ready flag determines whether the background reclaim worker should process a specific space_info. Previously, this flag defaulted to false because space_info structures are zero-initialized.

Quoted 8 lines

According to the original design, periodic reclaim should be active from the start and only disable itself (set to false) if it fails to find reclaimable block groups.

Now that the reclaim condition has been fixed in a previous patch to properly handle reclaim_bytes, it is necessary to enable this by default. This ensures background reclaim logic kicks in as soon as the thresholds are met after mount.

Is this problem practical on a test/real workload or theoretical? If we never free net-1G, I don't know how much reclaim is gonna help anyway.

If the "net 1G freed" condition is not the actual condition that we want, maybe we should rethink that? We can enable it on 1G total freed, regardless of allocation to give it a chance to run even if the net free is 800M or something. I was worried about workloads that did actually allocate and fill in the gaps. Or we can just use the total available space as a proxy, like "if we do a free and the total free in the space_info is >1G, enable periodic reclaim". The reclaim loops aren't gonna be costly and we don't expect them to do anything when the fs isn't full anyway.

Either way, though, I don't think this is harmful, so we can probably put it in. Just curious what you think about the other ideas and why you decided this was needed.

Thanks,
Boris

Quoted 23 lines
Fixes 862dd2fd93540 ("btrfs: fix periodic reclaim condition")
CC Boris Burkov <[email protected]>
---
 fs/btrfs/space-info.c | 1 +
 1 file changed, 1 insertion(+)
fs/btrfs/space-info.c +1 -0
index bb5aac7ee9d2..1530c30cacc9 100644
@@ -250,6 +250,7 @@ static void init_space_info(struct btrfs_fs_info *info,
 	space_info->clamp = 1;
 	btrfs_update_space_info_chunk_size(space_info, calc_chunk_size(info, flags));
 	space_info->subgroup_id = BTRFS_SUB_GROUP_PRIMARY;
+	space_info->periodic_reclaim_ready = true;
 
 	if (btrfs_is_zoned(info))
 		space_info->bg_reclaim_threshold = BTRFS_DEFAULT_ZONED_RECLAIM_THRESH;
Signature · 1 line
2.52.0
#2 lore raw
Quoted 5 lines · Boris Burkov
Quoted 4 lines · Sun YangKai

The periodic_reclaim_ready flag determines whether the background reclaim worker should process a specific space_info. Previously, this flag defaulted to false because space_info structures are zero-initialized.

Quoted 8 lines
Quoted 8 lines

According to the original design, periodic reclaim should be active from the start and only disable itself (set to false) if it fails to find reclaimable block groups.

Now that the reclaim condition has been fixed in a previous patch to properly handle reclaim_bytes, it is necessary to enable this by default. This ensures background reclaim logic kicks in as soon as the thresholds are met after mount.

Hi Boris. Glad to receive your reply within hours :)

Quoted 2 lines

Is this problem practical on a test/real workload or theoretical? If we never free net-1G, I don't know how much reclaim is gonna help anyway.

Yes, actually we don't know how much space is enough currently :(. It would be a lot better of we could find it out when failed to reclaim a blockgroup ...

I have a test case for periodic reclaim like this:
1. mount the fs with >10G unallocated
2. filling the disk to nearly full (< 10G unallocated)
3. free up some space(>10G unallocated) and umount, preparing for the next test

I expect the periodic reclaim kick in when the disk is nearly full (during 2) instead of after freeing up some space(during 3). This let me started to think about this patch.

In real world workload, we may have a fs quite empty when mounted, and fill up to quite full and expect periodic reclaim will happen to get rid of running out of space for unallocated. But periodic reclaim will not work without this change.

And this patch is the simple and quick fix for that "edge case".

Quoted 5 lines

If the "net 1G freed" condition is not the actual condition that we want, maybe we should rethink that? We can enable it on 1G total freed, regardless of allocation to give it a chance to run even if the net free is 800M or something. I was worried about workloads that did actually allocate and fill in the gaps.

I agree. It's not reliable and may trigger too often than what we expect.

Quoted 5 lines

Or we can just use the total available space
as a proxy, like "if we do a free and the total free in the space_info is >1G, enable periodic reclaim". The reclaim loops aren't gonna be costly and we don't expect them to do anything when the fs isn't full anyway.

Comparing current free space with a target free space instead of tracing a reclaim_bytes is a good idea. At least we don't need to maintain an extra counter any more. But the fixed 1G target might cause some problems since a reclaim might fail with a >1G free space. With the fixed 1G target, we might trying the useless work again and again. I suggest to set a target larger than the free space we have when setting periodic_reclaim_ready to false so we'll not trigger periodic reclaim if there's no "enough" free space freed.

How much the target should be larger than the current free space? Still 1G, or maybe we could find a better value, taking more factors into consideration?

And at mount time we cannot find out a proper target value so just set periodic_reclaim_ready to true. (BTW, this is why I use "paused" instead of "ready" in the previous fixup patch: to make the default value fits the default logic, but just set it to true as default seems enough :)

So even with this redesign , I think this patch is also necessary, or at least no harm.

Thanks,
Sun YangKai

Quoted 6 lines

Either way, though, I don't think this is harmful, so we can probably put it in. Just curious what you think about the other ideas and why you decided this was needed.

Thanks,
Boris

#3 lore raw
Quoted 31 lines · Sun Yangkai
Quoted 5 lines · Boris Burkov
Quoted 4 lines · Sun YangKai

The periodic_reclaim_ready flag determines whether the background reclaim worker should process a specific space_info. Previously, this flag defaulted to false because space_info structures are zero-initialized.

Quoted 8 lines
Quoted 8 lines

According to the original design, periodic reclaim should be active from the start and only disable itself (set to false) if it fails to find reclaimable block groups.

Now that the reclaim condition has been fixed in a previous patch to properly handle reclaim_bytes, it is necessary to enable this by default. This ensures background reclaim logic kicks in as soon as the thresholds are met after mount.

Hi Boris. Glad to receive your reply within hours :)

Quoted 2 lines

Is this problem practical on a test/real workload or theoretical? If we never free net-1G, I don't know how much reclaim is gonna help anyway.

Yes, actually we don't know how much space is enough currently :(. It would be a lot better of we could find it out when failed to reclaim a blockgroup ...

I have a test case for periodic reclaim like this:
1. mount the fs with >10G unallocated
2. filling the disk to nearly full (< 10G unallocated)
3. free up some space(>10G unallocated) and umount, preparing for the next test

Quoted 3 lines

I expect the periodic reclaim kick in when the disk is nearly full (during 2) instead of after freeing up some space(during 3). This let me started to think about this patch.

If the filesystem has only ever allocated and never freed, then it cannot possibly make progress at 2.

If you have also freed a gig in smaller extents than later allocations but bigger extents than the ones in one of the block groups you are relocating, then it could succeed even with net allocation greater than -1GiB.

Quoted 8 lines

In real world workload, we may have a fs quite empty when mounted, and fill up to quite full and expect periodic reclaim will happen to get rid of running out of space for unallocated. But periodic reclaim will not work without this change.

And this patch is the simple and quick fix for that "edge case".

Except that it runs into any number of trivial edge cases itself. Like "never freeing" like I pointed out above, but also failing on the first try (setting back to false) then later having net free less than 1GiB but in a state that could make progress and not re-enabling.

Fundamentally the thing to work on is the enabling condition. Whether it's on at mount or not is kind of irrelevant, IMO, because it is essentially the same as the state after a relocation fails.

e.g., with your change, this is still possible and essentially not that different from starting at step 3 on a fresh mount.

T0: blank fs size 100G
T1: alloc 95G contiguous
T2: reclaim tries, fails; periodic_reclaim_ready = false
T3: concurrently free and allocate 10G while never seeing a net of -1G, but making real, big holes that could result in a successful relocation
T4: reclaim is still disabled forever

Quoted 14 lines
Quoted 5 lines

If the "net 1G freed" condition is not the actual condition that we want, maybe we should rethink that? We can enable it on 1G total freed, regardless of allocation to give it a chance to run even if the net free is 800M or something. I was worried about workloads that did actually allocate and fill in the gaps.

I agree. It's not reliable and may trigger too often than what we expect.

Quoted 5 lines

Or we can just use the total available space
as a proxy, like "if we do a free and the total free in the space_info is >1G, enable periodic reclaim". The reclaim loops aren't gonna be costly and we don't expect them to do anything when the fs isn't full anyway.

Quoted 7 lines

Comparing current free space with a target free space instead of tracing a reclaim_bytes is a good idea. At least we don't need to maintain an extra counter any more. But the fixed 1G target might cause some problems since a reclaim might fail with a >1G free space. With the fixed 1G target, we might trying the useless work again and again. I suggest to set a target larger than the free space we have when setting periodic_reclaim_ready to false so we'll not trigger periodic reclaim if there's no "enough" free space freed.

The current periodic+dynamic reclaim is already essentially saying it is targeting something like 2-10GiB unallocated. It simply goes away past 10G and gets quite aggressive as you get closer to 0. So more signals based on a target free doesn't make that much sense to me, unless I am misunderstanding your idea. Please feel free to explain in more detail :)

Here's how I was thinking when I came up with the current condition:
I was concerned about a filesystem that is in a relatively steady state and quite full, so in the <10G unallocated. If it is totally steady state, then nothing can ever make progress. So we should say we aren't ready until *something* changes. What could that something be? Some ideas:
1. any space at all has been freed
2. >THRESH space has been freed (ignoring concurrent allocations)
3. >THRESH net space has been freed
4. the largest available hole in a bg has changed (that means the bg is now "better" at receiving relocated extents)

So 1. will definitely catch every case where you could reclaim, but is also susceptible to getting stuck failing a lot with trivial amounts freed.

And 3. is maybe too "mean" and doesn't account for fragmentation.

I intuitively thought 2. was going to be too similar to 1., but maybe it's a good middle ground. If there is a high enough rate of freeing activity, since the last time we looked, then we should look again.

I think 4 is also interesting (perhaps in concert with 2 or 3) as it more directly accounts for the critical factor of the current fragmentation. But as written it is too similar to 1., small irrelevant changes might change the largest hole even if there is still no hope whatsoever.

Quoted 11 lines

How much the target should be larger than the current free space? Still 1G, or maybe we could find a better value, taking more factors into consideration?

And at mount time we cannot find out a proper target value so just set periodic_reclaim_ready to true. (BTW, this is why I use "paused" instead of "ready" in the previous fixup patch: to make the default value fits the default logic, but just set it to true as default seems enough :)

So even with this redesign , I think this patch is also necessary, or at least no harm.

Agreed that it's no harm.

Quoted 11 lines

Thanks,
Sun YangKai

Quoted 6 lines

Either way, though, I don't think this is harmful, so we can probably put it in. Just curious what you think about the other ideas and why you decided this was needed.

Thanks,
Boris

#4 lore raw
Quoted 53 lines · Boris Burkov
Quoted 31 lines · Sun Yangkai
Quoted 5 lines · Boris Burkov
Quoted 4 lines · Sun YangKai

The periodic_reclaim_ready flag determines whether the background reclaim worker should process a specific space_info. Previously, this flag defaulted to false because space_info structures are zero-initialized.

Quoted 8 lines
Quoted 8 lines

According to the original design, periodic reclaim should be active from the start and only disable itself (set to false) if it fails to find reclaimable block groups.

Now that the reclaim condition has been fixed in a previous patch to properly handle reclaim_bytes, it is necessary to enable this by default. This ensures background reclaim logic kicks in as soon as the thresholds are met after mount.

Hi Boris. Glad to receive your reply within hours :)

Quoted 2 lines

Is this problem practical on a test/real workload or theoretical? If we never free net-1G, I don't know how much reclaim is gonna help anyway.

Yes, actually we don't know how much space is enough currently :(. It would be a lot better of we could find it out when failed to reclaim a blockgroup ...

I have a test case for periodic reclaim like this:
1. mount the fs with >10G unallocated
2. filling the disk to nearly full (< 10G unallocated)
3. free up some space(>10G unallocated) and umount, preparing for the next test

Quoted 3 lines

I expect the periodic reclaim kick in when the disk is nearly full (during 2) instead of after freeing up some space(during 3). This let me started to think about this patch.

If the filesystem has only ever allocated and never freed, then it cannot possibly make progress at 2.

If you have also freed a gig in smaller extents than later allocations but bigger extents than the ones in one of the block groups you are relocating, then it could succeed even with net allocation greater than -1GiB.

Quoted 8 lines

In real world workload, we may have a fs quite empty when mounted, and fill up to quite full and expect periodic reclaim will happen to get rid of running out of space for unallocated. But periodic reclaim will not work without this change.

And this patch is the simple and quick fix for that "edge case".

Quoted 8 lines

Except that it runs into any number of trivial edge cases itself. Like "never freeing" like I pointed out above, but also failing on the first try (setting back to false) then later having net free less than 1GiB but in a state that could make progress and not re-enabling.

Fundamentally the thing to work on is the enabling condition. Whether it's on at mount or not is kind of irrelevant, IMO, because it is essentially the same as the state after a relocation fails.

Well... I agree with that the enabling condition is important. But I think mount time and relocation failure are different. Relocation failure will only happen when the disk is kind of "full" so we know we can do nothing helpful until there's enough net free. But at mount time it's not the case.

Quoted 32 lines

e.g., with your change, this is still possible and essentially not that different from starting at step 3 on a fresh mount.

T0: blank fs size 100G
T1: alloc 95G contiguous
T2: reclaim tries, fails; periodic_reclaim_ready = false
T3: concurrently free and allocate 10G while never seeing a net of -1G, but making real, big holes that could result in a successful relocation
T4: reclaim is still disabled forever

Quoted 14 lines
Quoted 5 lines

If the "net 1G freed" condition is not the actual condition that we want, maybe we should rethink that? We can enable it on 1G total freed, regardless of allocation to give it a chance to run even if the net free is 800M or something. I was worried about workloads that did actually allocate and fill in the gaps.

I agree. It's not reliable and may trigger too often than what we expect.

Quoted 5 lines

Or we can just use the total available space
as a proxy, like "if we do a free and the total free in the space_info is >1G, enable periodic reclaim". The reclaim loops aren't gonna be costly and we don't expect them to do anything when the fs isn't full anyway.

Quoted 7 lines

Comparing current free space with a target free space instead of tracing a reclaim_bytes is a good idea. At least we don't need to maintain an extra counter any more. But the fixed 1G target might cause some problems since a reclaim might fail with a >1G free space. With the fixed 1G target, we might trying the useless work again and again. I suggest to set a target larger than the free space we have when setting periodic_reclaim_ready to false so we'll not trigger periodic reclaim if there's no "enough" free space freed.

Quoted 6 lines

The current periodic+dynamic reclaim is already essentially saying it is targeting something like 2-10GiB unallocated. It simply goes away past 10G and gets quite aggressive as you get closer to 0. So more signals based on a target free doesn't make that much sense to me, unless I am misunderstanding your idea. Please feel free to explain in more detail :)

Maybe there's some misunderstanding here. Currently we maintain a separate counter called reclaimable_bytes to trace the net free/alloc and compare it with a fixed value(1G) as the enabling condition.
I want to use a target free value instead of the reclaimable_bytes counter and compare the free space with the target free as the enabling condition so we don't need to maintain a separate counter on every alloc/free code path when periodic_reclaim_ready is false. It's a micro optimization, totally get rid of the signed/unsigned comparing issue, and has nothing to do with the dynamic reclaim things:)

Quoted 11 lines

Here's how I was thinking when I came up with the current condition:
I was concerned about a filesystem that is in a relatively steady state and quite full, so in the <10G unallocated. If it is totally steady state, then nothing can ever make progress. So we should say we aren't ready until *something* changes. What could that something be? Some ideas:
1. any space at all has been freed
2. >THRESH space has been freed (ignoring concurrent allocations)
3. >THRESH net space has been freed
4. the largest available hole in a bg has changed (that means the bg is now "better" at receiving relocated extents)

Interesting. How can we detect the largest hole in the filesystem and what's the cost? I've never thought about that.

Quoted 10 lines

So 1. will definitely catch every case where you could reclaim, but is also susceptible to getting stuck failing a lot with trivial amounts freed.

And 3. is maybe too "mean" and doesn't account for fragmentation.

I intuitively thought 2. was going to be too similar to 1., but maybe it's a good middle ground. If there is a high enough rate of freeing activity, since the last time we looked, then we should look again.

Quoted 5 lines

I think 4 is also interesting (perhaps in concert with 2 or 3) as it more directly accounts for the critical factor of the current fragmentation. But as written it is too similar to 1., small irrelevant changes might change the largest hole even if there is still no hope whatsoever.

And with the "target free" method to detect net free, we can have a dynamic THRESH by setting the target free to "current_free - dyn_thresh". What do you think about using the min_used_bg->used as the dyn_thresh? I think it would be better than 1G.

Thanks,
Sun Yangkai

Quoted 25 lines
Quoted 11 lines

How much the target should be larger than the current free space? Still 1G, or maybe we could find a better value, taking more factors into consideration?

And at mount time we cannot find out a proper target value so just set periodic_reclaim_ready to true. (BTW, this is why I use "paused" instead of "ready" in the previous fixup patch: to make the default value fits the default logic, but just set it to true as default seems enough :)

So even with this redesign , I think this patch is also necessary, or at least no harm.

Agreed that it's no harm.

Quoted 11 lines

Thanks,
Sun YangKai

Quoted 6 lines

Either way, though, I don't think this is harmful, so we can probably put it in. Just curious what you think about the other ideas and why you decided this was needed.

Thanks,
Boris