Autodefrag option with more delay?
Hi!
On answering to the nocow file bit thread I wondered about the way autodefrag works. As I saw quite some time ago with a VM image on a 2,5 inch external eSATA harddisk it kicks in quite quickly and seems to cause a *lot* of additional writes.
I wonder about this case:
merkaba:/home/martin/.local/share/akonadi/db_data/akonadi> filefrag parttable.ibd
parttable.ibd: 8039 extents found
Akonadi stores metadata about parts, like mails, contacts and so on in there. Now the most regular workload for this with a mail setup like here is:
Add new data.
It may not change much of the data after all the filtering of the mail to their mailfolders have done. Similarily with VM images after installation, much of the image may not be touched anymore then.
I wonder how autodefrag would perform when it would take the *write hotness* of data into account. I.e. only kick in after data has cooled down a bit. That could help for any cases where applications only modify recently written data and leave older cold data alone. And I bet there may be quite some applications doing that. Including, but not limited to journald.
I bet it may do so already to some extent, but I also get the impression it probably doesn´t wait long enough for the data to cool down and kicks in too quickly. I bet it could be beneficial for it to be a bit more lazy.
In the end if could even be lazy enough to wait till data is actually read accessed in a way that causes delays. So it could postpone the defragmentation still someone actually makes use of the written data. Of course that would only optimize frequent accesses, first time accesses would be slower.
On the other hands more heuristics adds more complexity and may break stuff as well.
What do you think?
Ciao,
Signature · 2 lines
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
Am Samstag, 10. Januar 2015, 11:41:57 schrieb Martin Steigerwald:
Quoted 23 lines
Hi!
On answering to the nocow file bit thread I wondered about the way autodefrag works. As I saw quite some time ago with a VM image on a 2,5 inch external eSATA harddisk it kicks in quite quickly and seems to cause a *lot* of additional writes.
I wonder about this case:
merkaba:/home/martin/.local/share/akonadi/db_data/akonadi> filefrag parttable.ibd
parttable.ibd: 8039 extents foundAkonadi stores metadata about parts, like mails, contacts and so on in there. Now the most regular workload for this with a mail setup like here is:
Add new data.
It may not change much of the data after all the filtering of the mail to their mailfolders have done. Similarily with VM images after installation, much of the image may not be touched anymore then.
Quoted 5 lines
I wonder how autodefrag would perform when it would take the *write hotness* of data into account. I.e. only kick in after data has cooled down a bit. That could help for any cases where applications only modify recently written data and leave older cold data alone. And I bet there may be quite some applications doing that. Including, but not limited to journald.
Okay, nope. Not journald, as, as far as I understand it it would never leave the beginning of the file alone when it still writes to it. As MySQL would, I think, just write into certain parts of the database file, i.e. there where it would add the new records, as AFAIK MySQL stores the index seperate from the tables if you use innodb_file_per_table.
I have seen fluentd tail plugin using a position file to track the position in text based logfiles. It uses an external file as to not modify the logfile of the application or syslog daemon. But actually writing the position pointers into an external files IMHO makes a huge lot of sense for a copy on write filesystems like BTRFS.
Signature · 2 lines
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7