trim calls not occurring during device removal operation
I recently removed a device from a filesystem. All the devices in the file system were thin-provisioned LUNs. From the LUN manager, I was able to monitor the space allocated for each LUN. I noticed that during the operation, as the file system consumed increasing amounts of space from the other devices (from transferring the data off the one being removed), none of the deallocated space was freed on the backend for the device targeted for removal. Apparently, the file system was not calling a hardware trim operation for freed space.
For physical devices, such behavior might not be problematic. However, for logical devices backed on storage with usage approaching capacity, it is preferred to free space as soon as possible. If a single operation consumes increasing space on the backend but frees none, then the space on the backed may become depleted before the operation completes.
The file system had been mounted with the discard option set to async. In addition to noticing that the option was not causing trim calls to be made during the remove operation, I also noticed that calls to the fstrim system utility targeting the mounted file system did not cause any of the space to be trimmed.
In order to avoid overconsumption of space on the backend, it would be helpful to trim space as it is freed on a device targeted for an ongoing removal operation. Even if the backend space is not imediately needed, freeing the space as part of the removal operation would leave the device in a state that clearly signals to the LUN adminstrator that the LUN is no longer storing data, and so may be safely destroyed. It reduces the chance of data loss by human error if an administrator may understand that a LUN may be safely destroyed if and only if it is not showing as having allocated space.
Would you consider adding more robust support for trim operations as part of device removal?
All of my observations occurred under kernel 5.15.