Stupid (?) Idea about extent lifetimes.

It occurs to me that it would be desirable to mark extents as "least favoured nations" and so all new writes would like to not be written there and any data written there would have a desire to be somewhere else.

So lets say the wholly unallocated space has a natural status of 100.

Allocated blocks would normally have statuses less than that by a trivial amount, such as 99.

One could then marks blocks with higher numbers for being less favoured, or lower numbers for being more favoured as desired.

Basically this would create a gravity map of sorts that would be factored into allocation decisions.

So say you just converted an ext4 to btrfs. It's got all those oddly sized and placed extents. You could give them all higher numbers in hopes that the data would naturally migrate away. Say just number them really large with no two numbers the same. Now the largest number would naturally become vacant and likely to be freed.

Likewise you could weight your data to migrate spindle-ward or such in the weeks before a reorg.

Similarly changes in geometry could simply mark segments as ill-favoured where the old geometry doesn't match the new and data would migrate under pressure.

One could reverse the age induced entropy of a file system by just periodically increasing the disfavour values of all the blocks, causing the oldest blocks to be the least favoured of all, and so creating a slowly rolling pattern.

So say new blocks start life as 50s by default. And empty space is 100. And every so often every block gets an increment (say 3 so that 100 is naturally skipped over.

Young blocks are now very magnetic. As they age they lose favour. Eventually they pass the value for unallocated space. Then they start losing data and eventually, in a system with 100 percent turnover the blocks get deallocated.

defragging and occasional balancing take care of the files that "never" change.

Very high numbers could also be reserved for pinning. Specially flagged files would have reverse gravity. A desire to stay put. So say a NOCOW files or Swap Files might have reverse gravity and one could use some tool to allocate blocks at the cold end of the disk with those sorts of numbers. Effectively segregating the the static from the churning data.

Fresh files would thereby tend to vacate extents full of snapshot data and freeing static (Read only) snapshot data would tend to release contiguous space.

As the disk runs out of space the system naturally breaks down into the existing best-fit allocation.

It's less than a defrag or autodefrag, or balance, and it would tend to be more like digestive peristalsis, at the extreme end (where people are taking way too many snapshots) it becomes an elevator sort by extent age.

(If this is a tired idea, my apologies. I took cough medicine a little while ago and this thought that's been rattling around in my head for months bubbled out of its cauldron.)

Robert White posted on Tue, 15 Mar 2016 23:32:05 -0700 as excerpted:

I believe the word you wanted here instead of extents is "chunks".

Extents are parts of files, and btrfs is a COW (copy on write) filesystem, so writes modifying existing files already move out of the existing file extents.

But chunks, in btrfs context at least, are relatively large, nominally 1 GiB for data, 256 MiB for metadata (tho sizes can be larger on really large filesystems or smaller on real small ones), "chunks", that the filesystem allocates first as empty, and lets files (or metadata nodes) fill. It's these chunks that balance rewrites, and thus these chunks that would logically get your numeric "gravity" ratings.

With that change, it's an interesting concept. I'm not a dev and can't really guess if it's entirely practical to implement or not, but as a btrfs user and admin, I can certainly see practical uses, should it /be/ implemented.

Very creative concept and definitely worth discussion. Thanks for throwing it out here /for/ discussion. =:^)

Tho one caution. There's a lot more worthwhile ideas about what /could/ be done with btrfs than developers and developer time to implement them all, at least in anything like a reasonable time frame of say five years or less. So even if it's a pretty good idea, unless it happens to be practically usable to help implement some other project the devs are working on, simply because there's so many other pretty good ideas, it might take quite some time (over five years) to actually be implemented. So don't get your hopes up for anything too immediate, or even intermediate term (2-5 years out), unless you get extremely lucky and the devs see it as a way to implement something they already are working on or that's already a high priority "next" item on their roadmap.

Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman