ovl: Allow changing default fsync_mode
Subsystems (2)
PATCH 1/1 ovl: Allow changing default fsync_mode
We have enabled "volatile" fsync_mode on our Kubernetes production environment to prevent container exit from being blocked when there are many dirty pages to flush. This has worked well without introducing any issues.
However, on some of our production servers, upgrading the container runtime to support the "volatile" mount option is not straightforward [0]. To address this, we want to enable it by default within the kernel.
Add a Kconfig option to set the default fsync_mode at build time, and allow it to be overridden dynamically via a module parameter when loading overlayfs. This also aligns with how other features are configured.
--- fs/overlayfs/Kconfig | 15 +++++++++++++++ fs/overlayfs/params.c | 40 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 54 insertions(+), 1 deletion(-)
fs/overlayfs/Kconfig +15 -0
@@ -134,3 +134,18 @@ config OVERLAY_FS_DEBUG Say Y here to enable extra debugging checks in overlayfs. If unsure, say N. + +config OVERLAY_FS_FSYNC_MODE + int "Overlayfs: default fsync mode" + depends on OVERLAY_FS + range 0 2 + default 1 + help + Set the default fsync mode for Overlayfs. + + 0: volatile - Do not sync filesystem on fsync + 1: auto - Automatically detect sync behavior (default) + 2: strict - Always sync filesystem on fsync + + This can be overridden at runtime via the fsync_mode module + parameter or per-mount with the 'fsync=' mount option.
fs/overlayfs/params.c +39 -1
@@ -153,11 +153,49 @@ static const char *ovl_fsync_mode(struct ovl_config *config)return ovl_parameter_fsync[config->fsync_mode].name; } +static int ovl_default_fsync =CONFIG_OVERLAY_FS_FSYNC_MODE ;static int ovl_fsync_mode_def (void ) { -return OVL_FSYNC_AUTO ; +return ovl_default_fsync; } +static int ovl_fsync_mode_set (const char *val,const struct kernel_param *kp) +{ +int i; +int *p = kp->arg; + +if (!val) +return -EINVAL ; + +for (i =0 ; i <ARRAY_SIZE (ovl_parameter_fsync); i++) { +if (sysfs_streq (val, ovl_parameter_fsync[i].name)) { + *p = ovl_parameter_fsync[i].value; +return 0 ; + } + } +return -EINVAL ; +} + +static int ovl_fsync_mode_get (char *buffer,const struct kernel_param *kp) +{ +int val = *(int *)kp->arg; +int i; + +for (i =0 ; i <ARRAY_SIZE (ovl_parameter_fsync); i++) { +if (ovl_parameter_fsync[i].value == val) +return sysfs_emit (buffer,"%s\n" , ovl_parameter_fsync[i].name); + } +return sysfs_emit (buffer,"unknown\n" ); +} + +static const struct kernel_param_ops ovl_fsync_mode_ops = { + .set = ovl_fsync_mode_set, + .get = ovl_fsync_mode_get, +}; + +module_param_cb (fsync_mode, &ovl_fsync_mode_ops, &ovl_default_fsync,0644 ); +MODULE_PARM_DESC (fsync_mode,"fsync mode: auto, volatile, strict (default: auto)" ); +const struct fs_parameter_spec ovl_parameter_spec[] = {fsparam_string_empty ("lowerdir" , Opt_lowerdir),fsparam_file_or_string ("lowerdir+" , Opt_lowerdir_add),
Signature · 1 line
2.52.0
Quoted 8 lines · Yafang Shao
We have enabled "volatile"
fsync_modeon our Kubernetes production environment to prevent container exit from being blocked when there are many dirty pages to flush. This has worked well without introducing any issues.However, on some of our production servers, upgrading the container runtime to support the "volatile" mount option is not straightforward [0]. To address this, we want to enable it by default within the kernel.
Just a side note: "upgrade the container runtime is not straightforward", how? it seems that issue is already resolved and there is no more discussion.
Not quite sure applying a default volatile policy is quite feasible, especially the issue documented in
https://github.com/containerd/containerd/pull/10274/files#diff-9239161e2af83fd84df5792f9fe64701c517fe4598eae60d4d245d039955f46cR33
then userspace cannot drop volatile option as a somewhat workaround now.
Thanks,
Gao Xiang
Quoted 8 lines
Add a
Kconfigoption to set the defaultfsync_modeat build time, and allow it to be overridden dynamically via a module parameter when loading overlayfs. This also aligns with how other features are configured.Signed-off-by Yafang Shao <[email protected]>
Quoted 9 lines · Gao Xiang
Quoted 5 lines · Yafang Shao
We have enabled "volatile"
fsync_modeon our Kubernetes production environment to prevent container exit from being blocked when there are many dirty pages to flush. This has worked well without introducing any issues.
Quoted 7 lines
Quoted 3 lines
However, on some of our production servers, upgrading the container runtime to support the "volatile" mount option is not straightforward [0]. To address this, we want to enable it by default within the kernel.
Just a side note: "upgrade the container runtime is not straightforward", how? it seems that issue is already resolved and there is no more discussion.
We still have many production servers running Docker, while the "volatile" mount option is only supported by containerd. Upgrading from Docker to containerd is a difficult process.
Quoted 7 lines
Not quite sure applying a default volatile policy is quite feasible, especially the issue documented in
https://github.com/containerd/containerd/pull/10274/files#diff-9239161e2af83fd84df5792f9fe64701c517fe4598eae60d4d245d039955f46cR33then userspace cannot drop
volatileoption as a somewhat workaround now.
OS vendors can still set "auto" as the default config, while customers can override it dynamically via sysfs. We have been running with "volatile" on many production servers across different workloads for over a year, and it has worked as expected without any issues.
Signature · 2 lines
Regards Yafang
Quoted 14 lines · Yafang Shao
Quoted 13 lines · Gao Xiang
Quoted 8 lines · Yafang Shao
We have enabled "volatile"
fsync_modeon our Kubernetes production environment to prevent container exit from being blocked when there are many dirty pages to flush. This has worked well without introducing any issues.However, on some of our production servers, upgrading the container runtime to support the "volatile" mount option is not straightforward [0]. To address this, we want to enable it by default within the kernel.
Quoted 7 lines
Quoted 3 lines
Just a side note: "upgrade the container runtime is not straightforward", how? it seems that issue is already resolved and there is no more discussion.
We still have many production servers running Docker, while the "volatile" mount option is only supported by containerd. Upgrading from Docker to containerd is a difficult process.
But docker can be patched too: if upgrading the userspace is hard, why upgrading the linux kernel is easy?
Quoted 6 lines
Quoted 6 lines
Not quite sure applying a default volatile policy is quite feasible, especially the issue documented in
https://github.com/containerd/containerd/pull/10274/files#diff-9239161e2af83fd84df5792f9fe64701c517fe4598eae60d4d245d039955f46cR33
Quoted 7 lines
Quoted 2 lines
then userspace cannot drop
volatileoption as a somewhat workaround now.OS vendors can still set "auto" as the default config, while customers can override it dynamically via sysfs. We have been running with "volatile" on many production servers across different workloads for over a year, and it has worked as expected without any issues.
but sysfs setting still applies as system-wide, and there are some edge cases that we cannot apply volatile as default, that is my one concern.
The other concern is that since volatile omits fsync, so it's a posix violation (even that makes sense for container writable layers), not sure if we have to use it as the system-wide default configuration too.
Thanks,
Gao Xiang
Quoted 1 line
Quoted 22 lines · Gao Xiang
Quoted 18 lines · Yafang Shao
Quoted 16 lines · Gao Xiang
Quoted 8 lines · Yafang Shao
We have enabled "volatile"
fsync_modeon our Kubernetes production environment to prevent container exit from being blocked when there are many dirty pages to flush. This has worked well without introducing any issues.However, on some of our production servers, upgrading the container runtime to support the "volatile" mount option is not straightforward [0]. To address this, we want to enable it by default within the kernel.
Just a side note: "upgrade the container runtime is not straightforward", how? it seems that issue is already resolved and there is no more discussion.
Quoted 6 lines
Quoted 3 lines
We still have many production servers running Docker, while the "volatile" mount option is only supported by containerd. Upgrading from Docker to containerd is a difficult process.
But docker can be patched too: if upgrading the userspace is hard, why upgrading the linux kernel is easy?
It is quite easy since the kernel can be livepatched without rebooting. My employer is a heavy livepatch user. [1]
[1]. https://lore.kernel.org/live-patching/
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Not quite sure applying a default volatile policy is quite feasible, especially the issue documented in
https://github.com/containerd/containerd/pull/10274/files#diff-9239161e2af83fd84df5792f9fe64701c517fe4598eae60d4d245d039955f46cR33then userspace cannot drop
volatileoption as a somewhat workaround now.
Quoted 8 lines
Quoted 4 lines
OS vendors can still set "auto" as the default config, while customers can override it dynamically via sysfs. We have been running with "volatile" on many production servers across different workloads for over a year, and it has worked as expected without any issues.
but sysfs setting still applies as system-wide, and there are some edge cases that we cannot apply volatile as default, that is my one concern.
It is unclear whether there are mixed workloads on the same server that require both "volatile" and "strict" modes, but we have not encountered such use cases across our large fleet of servers.
Quoted 5 lines
The other concern is that since
volatileomits fsync, so it's a posix violation (even that makes sense for container writable layers), not sure if we have to use it as the system-wide default configuration too.
We have use cases for setting this as a system-wide configuration. It is unclear whether others have similar needs.
Signature · 2 lines
Regards Yafang
Quoted 27 lines · Yafang Shao
Quoted 26 lines · Gao Xiang
Quoted 21 lines · Yafang Shao
Quoted 9 lines · Gao Xiang
Quoted 5 lines · Yafang Shao
We have enabled "volatile"
fsync_modeon our Kubernetes production environment to prevent container exit from being blocked when there are many dirty pages to flush. This has worked well without introducing any issues.Quoted 7 lines
Quoted 3 lines
However, on some of our production servers, upgrading the container runtime to support the "volatile" mount option is not straightforward [0]. To address this, we want to enable it by default within the kernel.
Just a side note: "upgrade the container runtime is not straightforward", how? it seems that issue is already resolved and there is no more discussion.
We still have many production servers running Docker, while the "volatile" mount option is only supported by containerd. Upgrading from Docker to containerd is a difficult process.
Quoted 7 lines
Quoted 2 lines
But docker can be patched too: if upgrading the userspace is hard, why upgrading the linux kernel is easy?
It is quite easy since the kernel can be livepatched without rebooting. My employer is a heavy livepatch user. [1]
It's just a generic opinion, in general, docker can be live upgraded without pausing the containers, and upgrading userspace is easier / safer than patching the kernel.
Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
Quoted 10 lines
Not quite sure applying a default volatile policy is quite feasible, especially the issue documented in
https://github.com/containerd/containerd/pull/10274/files#diff-9239161e2af83fd84df5792f9fe64701c517fe4598eae60d4d245d039955f46cR33then userspace cannot drop
volatileoption as a somewhat workaround now.OS vendors can still set "auto" as the default config, while customers can override it dynamically via sysfs. We have been running with "volatile" on many production servers across different workloads for over a year, and it has worked as expected without any issues.
Quoted 7 lines
Quoted 3 lines
but sysfs setting still applies as system-wide, and there are some edge cases that we cannot apply volatile as default, that is my one concern.
It is unclear whether there are mixed workloads on the same server that require both "volatile" and "strict" modes, but we have not encountered such use cases across our large fleet of servers.
At least containerd needs to strip out volatile in some use cases, again see:
https://github.com/containerd/containerd/pull/9555
https://github.com/containerd/containerd/pull/10274/files#diff-9239161e2af83fd84df5792f9fe64701c517fe4598eae60d4d245d039955f46cR33
So set volatile as default will break userspace (containerd), and containerd needs to add another mount option to avoid the default volatile behavior, which is messy and makes the userspace more harder.
Quoted 9 lines
Quoted 6 lines
The other concern is that since
volatileomits fsync, so it's a posix violation (even that makes sense for container writable layers), not sure if we have to use it as the system-wide default configuration too.We have use cases for setting this as a system-wide configuration. It is unclear whether others have similar needs.
While I cannot speak out of overlayfs, but really it depends on if the use cases is generic.
Usually filesystems need to obey posix semantics as much as possible, and using specific mount option to relax (violate) some restriction, but it shouldn't be a system-wide stuff since otherwise user applications cannot know if they really live in the posix world.
Thanks,
Gao Xiang
Quoted 1 line
Quoted 34 lines · Gao Xiang
Quoted 30 lines · Yafang Shao
Quoted 28 lines · Gao Xiang
Quoted 14 lines · Yafang Shao
Quoted 13 lines · Gao Xiang
Quoted 8 lines · Yafang Shao
We have enabled "volatile"
fsync_modeon our Kubernetes production environment to prevent container exit from being blocked when there are many dirty pages to flush. This has worked well without introducing any issues.However, on some of our production servers, upgrading the container runtime to support the "volatile" mount option is not straightforward [0]. To address this, we want to enable it by default within the kernel.
Quoted 7 lines
Quoted 3 lines
Just a side note: "upgrade the container runtime is not straightforward", how? it seems that issue is already resolved and there is no more discussion.
We still have many production servers running Docker, while the "volatile" mount option is only supported by containerd. Upgrading from Docker to containerd is a difficult process.
But docker can be patched too: if upgrading the userspace is hard, why upgrading the linux kernel is easy?
Quoted 8 lines
Quoted 4 lines
It is quite easy since the kernel can be livepatched without rebooting. My employer is a heavy livepatch user. [1]
It's just a generic opinion, in general, docker can be live upgraded without pausing the containers, and upgrading userspace is easier / safer than patching the kernel.
Are you sure? How would one change an already mounted overlayfs that was not enabled with "volatile"?
Quoted 30 lines
Quoted 17 lines
Quoted 17 lines
Quoted 16 lines
Quoted 11 lines
Not quite sure applying a default volatile policy is quite feasible, especially the issue documented in
https://github.com/containerd/containerd/pull/10274/files#diff-9239161e2af83fd84df5792f9fe64701c517fe4598eae60d4d245d039955f46cR33then userspace cannot drop
volatileoption as a somewhat workaround now.OS vendors can still set "auto" as the default config, while customers can override it dynamically via sysfs. We have been running with "volatile" on many production servers across different workloads for over a year, and it has worked as expected without any issues.
Quoted 7 lines
Quoted 3 lines
but sysfs setting still applies as system-wide, and there are some edge cases that we cannot apply volatile as default, that is my one concern.
It is unclear whether there are mixed workloads on the same server that require both "volatile" and "strict" modes, but we have not encountered such use cases across our large fleet of servers.
At least containerd needs to strip out
volatilein some use cases, again see:
https://github.com/containerd/containerd/pull/9555
https://github.com/containerd/containerd/pull/10274/files#diff-9239161e2af83fd84df5792f9fe64701c517fe4598eae60d4d245d039955f46cR33
Quoted 4 lines
So set
volatileas default will break userspace (containerd), and containerd needs to add another mount option to avoid the defaultvolatilebehavior, which is messy and makes the userspace more harder.
However, our userspace is easier with it ;)
Quoted 11 lines
Quoted 10 lines
Quoted 7 lines
The other concern is that since
volatileomits fsync, so it's a posix violation (even that makes sense for container writable layers), not sure if we have to use it as the system-wide default configuration too.We have use cases for setting this as a system-wide configuration. It is unclear whether others have similar needs.
Quoted 8 lines
While I cannot speak out of overlayfs, but really it depends on if the use cases is generic.
Usually filesystems need to obey posix semantics as much as possible, and using specific mount option to relax (violate) some restriction, but it shouldn't be a system-wide stuff since otherwise user applications cannot know if they really live in the posix world.
AFAICS, the Linux kernel does not strictly follow POSIX in a number of areas.
BTW,
1. The "volatile" is not the default option.
2. Even when the system-wide default is set to "volatile", users can still change it per mount.
Signature · 2 lines
Regards Yafang
Quoted 77 lines · Yafang Shao
Quoted 34 lines · Gao Xiang
Quoted 30 lines · Yafang Shao
Quoted 28 lines · Gao Xiang
Quoted 14 lines · Yafang Shao
Quoted 13 lines · Gao Xiang
Quoted 8 lines · Yafang Shao
We have enabled "volatile"
fsync_modeon our Kubernetes production environment to prevent container exit from being blocked when there are many dirty pages to flush. This has worked well without introducing any issues.However, on some of our production servers, upgrading the container runtime to support the "volatile" mount option is not straightforward [0]. To address this, we want to enable it by default within the kernel.
Quoted 7 lines
Quoted 3 lines
Just a side note: "upgrade the container runtime is not straightforward", how? it seems that issue is already resolved and there is no more discussion.
We still have many production servers running Docker, while the "volatile" mount option is only supported by containerd. Upgrading from Docker to containerd is a difficult process.
But docker can be patched too: if upgrading the userspace is hard, why upgrading the linux kernel is easy?
Quoted 8 lines
Quoted 4 lines
It is quite easy since the kernel can be livepatched without rebooting. My employer is a heavy livepatch user. [1]
It's just a generic opinion, in general, docker can be live upgraded without pausing the containers, and upgrading userspace is easier / safer than patching the kernel.
Are you sure? How would one change an already mounted overlayfs that was not enabled with "volatile"?
Quoted 30 lines
Quoted 17 lines
Quoted 17 lines
Quoted 16 lines
Quoted 11 lines
Not quite sure applying a default volatile policy is quite feasible, especially the issue documented in
https://github.com/containerd/containerd/pull/10274/files#diff-9239161e2af83fd84df5792f9fe64701c517fe4598eae60d4d245d039955f46cR33then userspace cannot drop
volatileoption as a somewhat workaround now.OS vendors can still set "auto" as the default config, while customers can override it dynamically via sysfs. We have been running with "volatile" on many production servers across different workloads for over a year, and it has worked as expected without any issues.
Quoted 7 lines
Quoted 3 lines
but sysfs setting still applies as system-wide, and there are some edge cases that we cannot apply volatile as default, that is my one concern.
It is unclear whether there are mixed workloads on the same server that require both "volatile" and "strict" modes, but we have not encountered such use cases across our large fleet of servers.
At least containerd needs to strip out
volatilein some use cases, again see:
https://github.com/containerd/containerd/pull/9555
https://github.com/containerd/containerd/pull/10274/files#diff-9239161e2af83fd84df5792f9fe64701c517fe4598eae60d4d245d039955f46cR33
Quoted 6 lines
Quoted 4 lines
So set
volatileas default will break userspace (containerd), and containerd needs to add another mount option to avoid the defaultvolatilebehavior, which is messy and makes the userspace more harder.However, our userspace is easier with it ;)
It depends on how you define "our", if your docker can be upgraded, why bothering kernel?
As I said, common existing applications (like containerd) will break since they already assume "OVL_FSYNC_AUTO" is the default behavior, and explicitly use mount options to override the default "OVL_FSYNC_AUTO" behavior.
But with the patch, it just breaks the application assumption and userspace application needs to change again.
Quoted 21 lines
Quoted 12 lines
Quoted 11 lines
Quoted 8 lines
The other concern is that since
volatileomits fsync, so it's a posix violation (even that makes sense for container writable layers), not sure if we have to use it as the system-wide default configuration too.We have use cases for setting this as a system-wide configuration. It is unclear whether others have similar needs.
Quoted 8 lines
While I cannot speak out of overlayfs, but really it depends on if the use cases is generic.
Usually filesystems need to obey posix semantics as much as possible, and using specific mount option to relax (violate) some restriction, but it shouldn't be a system-wide stuff since otherwise user applications cannot know if they really live in the posix world.
Quoted 6 lines
AFAICS, the Linux kernel does not strictly follow POSIX in a number of areas.
BTW,
1. The "volatile" is not the default option.
2. Even when the system-wide default is set to "volatile", users can still change it per mount.
Here, I'm strongly against this, again: especially it just breaks application which implies "OVL_FSYNC_AUTO" is the default, and we shouldn't ask userspace to handle this to deal with the fact that the "volatile" could become the default again and again.
Thanks,
Gao Xiang
Quoted 1 line
Quoted 38 lines · Yafang Shao
Quoted 37 lines · Gao Xiang
Quoted 33 lines · Yafang Shao
Quoted 22 lines · Gao Xiang
Quoted 18 lines · Yafang Shao
Quoted 16 lines · Gao Xiang
Quoted 8 lines · Yafang Shao
We have enabled "volatile"
fsync_modeon our Kubernetes production environment to prevent container exit from being blocked when there are many dirty pages to flush. This has worked well without introducing any issues.However, on some of our production servers, upgrading the container runtime to support the "volatile" mount option is not straightforward [0]. To address this, we want to enable it by default within the kernel.
Just a side note: "upgrade the container runtime is not straightforward", how? it seems that issue is already resolved and there is no more discussion.
Quoted 6 lines
Quoted 3 lines
We still have many production servers running Docker, while the "volatile" mount option is only supported by containerd. Upgrading from Docker to containerd is a difficult process.
But docker can be patched too: if upgrading the userspace is hard, why upgrading the linux kernel is easy?
It is quite easy since the kernel can be livepatched without rebooting. My employer is a heavy livepatch user. [1]
Quoted 9 lines
Quoted 5 lines
Quoted 1 line
It's just a generic opinion, in general, docker can be live upgraded without pausing the containers, and upgrading userspace is easier / safer than patching the kernel.
Are you sure? How would one change an already mounted overlayfs that was not enabled with "volatile"?
I don't see how your patch can apply to already mounted overlayfs.
Again, I don't want such customized messy breaks userspace again; with that patch, container runtime needs to consider if volatile is the default which just breaks the existing containerd versions.
Thanks,
Gao Xiang
Quoted 51 lines · Gao Xiang
Quoted 38 lines · Yafang Shao
Quoted 37 lines · Gao Xiang
Quoted 33 lines · Yafang Shao
Quoted 22 lines · Gao Xiang
Quoted 18 lines · Yafang Shao
Quoted 16 lines · Gao Xiang
Quoted 8 lines · Yafang Shao
We have enabled "volatile"
fsync_modeon our Kubernetes production environment to prevent container exit from being blocked when there are many dirty pages to flush. This has worked well without introducing any issues.However, on some of our production servers, upgrading the container runtime to support the "volatile" mount option is not straightforward [0]. To address this, we want to enable it by default within the kernel.
Just a side note: "upgrade the container runtime is not straightforward", how? it seems that issue is already resolved and there is no more discussion.
Quoted 6 lines
Quoted 3 lines
We still have many production servers running Docker, while the "volatile" mount option is only supported by containerd. Upgrading from Docker to containerd is a difficult process.
But docker can be patched too: if upgrading the userspace is hard, why upgrading the linux kernel is easy?
It is quite easy since the kernel can be livepatched without rebooting. My employer is a heavy livepatch user. [1]
Quoted 9 lines
Quoted 5 lines
Quoted 1 line
It's just a generic opinion, in general, docker can be live upgraded without pausing the containers, and upgrading userspace is easier / safer than patching the kernel.
Are you sure? How would one change an already mounted overlayfs that was not enabled with "volatile"?
Quoted 7 lines
I don't see how your patch can apply to already mounted overlayfs.
Again, I don't want such customized messy breaks userspace again; with that patch, container runtime needs to consider if
volatileis the default which just breaks the existing containerd versions.
I'll leave this debate to the overlayfs maintainers ;)
Signature · 2 lines
Regards Yafang
Quoted 2 lines · Yafang Shao
Quoted 1 line · Gao Xiang
...
Quoted 7 lines
Quoted 5 lines
Again, I don't want such customized messy breaks userspace again; with that patch, container runtime needs to consider if
volatileis the default which just breaks the existing containerd versions.I'll leave this debate to the overlayfs maintainers ;)
On my own perspective and be responsible for common users (and as a containerd maintainer [1]), if you don't have a strong argument:
[1] https://github.com/containerd/.project/blob/main/MAINTAINERS#L36
And I don't want:
https://lore.kernel.org/r/CAOQ4uxhfwOU7O_vEQsAvRGV-v3_Dk1RcMnwGx-xOpx-FYjoKcg@mail.gmail.com
happens again.
Thanks,
Gao Xiang
Quoted 1 line
Quoted 15 lines · Gao Xiang
Quoted 2 lines · Yafang Shao
Quoted 1 line · Gao Xiang
...
Quoted 6 lines
Quoted 5 lines
Again, I don't want such customized messy breaks userspace again; with that patch, container runtime needs to consider if
volatileis the default which just breaks the existing containerd versions.
Quoted 4 lines
Quoted 1 line
I'll leave this debate to the overlayfs maintainers ;)
On my own perspective and be responsible for common users (and as a containerd maintainer [1]),
No wonder containerd is getting harder and harder to use ;)
Quoted 4 lines
if you don't have
a strong argument:NACKed-by Gao Xiang <[email protected]>
Feel free to do it however you like.
Signature · 2 lines
Regards Yafang
Quoted 16 lines · Yafang Shao
Quoted 15 lines · Gao Xiang
Quoted 2 lines · Yafang Shao
Quoted 1 line · Gao Xiang
...
Quoted 6 lines
Quoted 5 lines
Again, I don't want such customized messy breaks userspace again; with that patch, container runtime needs to consider if
volatileis the default which just breaks the existing containerd versions.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
I'll leave this debate to the overlayfs maintainers ;)
On my own perspective and be responsible for common users (and as a containerd maintainer [1]),
No wonder containerd is getting harder and harder to use ;)
What do you mean, can you explain exactly?
You're just adding a new way to break the existing applications, no? You just breaks previous shipped containerd.
Add a way to change the default behavior is fine, but the new default behavior should be worked with the same functionality and compatible, but switching to volatile feature is non-compatible and what is why containerd dropped volatile.
I've explained the technical reasons, can you also show your technical argument why you cannot patch docker with a very little change (if you can livepatch the kernel) and just restart the docker daemon, is that hard? -- Also, is that requirement common?
How pre-existing container runtime versions know their default option is not "auto", how do they add mount option "fsync=auto"? and how do they know there is a new sysfs knob to work around incompatible behavior? there are so many container runtime "docker", "containerd", "crio", how to make sure all these container runtime aware of the new default may not be "auto"?
Thanks,
Gao Xiang
Quoted 25 lines · Gao Xiang
Quoted 21 lines · Yafang Shao
Quoted 19 lines · Gao Xiang
Quoted 2 lines · Yafang Shao
Quoted 1 line · Gao Xiang
...
Quoted 7 lines
Quoted 5 lines
Again, I don't want such customized messy breaks userspace again; with that patch, container runtime needs to consider if
volatileis the default which just breaks the existing containerd versions.I'll leave this debate to the overlayfs maintainers ;)
On my own perspective and be responsible for common users (and as a containerd maintainer [1]),
Quoted 7 lines
Quoted 1 line
No wonder containerd is getting harder and harder to use ;)
What do you mean, can you explain exactly?
You're just adding a new way to break the existing applications, no? You just breaks previous shipped containerd.
It's your responsibility to handle the cases where "strict" is explicitly required. Please do your homework. It is not the kernel's fault.
Quoted 6 lines
Add a way to change the default behavior is fine, but the new default behavior should be worked with the same functionality and compatible, but switching to
volatilefeature is non-compatible and what is why containerd dropped volatile.
It only adds a dynamically changeable config. Why do you insist it breaks everything? Users can always change it whenever they need.
Quoted 7 lines
I've explained the technical reasons, can you also show your technical argument why you cannot patch docker with a very little change (if you can livepatch the kernel) and just restart the docker daemon, is that hard? -- Also, is that requirement common?
You'll need to restart the running services. When we applied "volatile" with a livepatch across our large fleet of production servers, everything worked fine without any breakage — we've done our homework.
Quoted 9 lines
How pre-existing container runtime versions know their default option is not "auto", how do they add mount option "fsync=auto"? and how do they know there is a new sysfs knob to work around incompatible behavior? there are so many container runtime "docker", "containerd", "crio", how to make sure all these container runtime aware of the new default may not be "auto"?
That's why it's a dynamically changeable config.
Again, pls do your homework.
Signature · 2 lines
Regards Yafang
(+try to cc more FS people for visibility.)
Quoted 30 lines · Yafang Shao
Quoted 29 lines · Gao Xiang
Quoted 16 lines · Yafang Shao
Quoted 15 lines · Gao Xiang
Quoted 2 lines · Yafang Shao
Quoted 1 line · Gao Xiang
...
Quoted 6 lines
Quoted 5 lines
Again, I don't want such customized messy breaks userspace again; with that patch, container runtime needs to consider if
volatileis the default which just breaks the existing containerd versions.Quoted 6 lines
Quoted 4 lines
Quoted 1 line
I'll leave this debate to the overlayfs maintainers ;)
On my own perspective and be responsible for common users (and as a containerd maintainer [1]),
No wonder containerd is getting harder and harder to use ;)
What do you mean, can you explain exactly?
Quoted 7 lines
Quoted 3 lines
You're just adding a new way to break the existing applications, no? You just breaks previous shipped containerd.
It's your responsibility to handle the cases where "strict" is explicitly required. Please do your homework. It is not the kernel's fault.
How do you modify the existing applications and scripts to adapt your incompatible new Kconfig?
Quoted 10 lines
Quoted 7 lines
Add a way to change the default behavior is fine, but the new default behavior should be worked with the same functionality and compatible, but switching to
volatilefeature is non-compatible and what is why containerd dropped volatile.It only adds a dynamically changeable config. Why do you insist it breaks everything? Users can always change it whenever they need.
Can you find any Kconfig option that changes user-visible default functionality and causes almost any user application that relies on remounting to fail to mount again? If so, I think we should Cc Linus now.
Thanks,
Gao Xiang
Quoted 39 lines · Gao Xiang
(+try to cc more FS people for visibility.)
Quoted 34 lines · Yafang Shao
Quoted 32 lines · Gao Xiang
Quoted 16 lines · Yafang Shao
Quoted 15 lines · Gao Xiang
Quoted 2 lines · Yafang Shao
Quoted 1 line · Gao Xiang
...
Quoted 6 lines
Quoted 5 lines
Again, I don't want such customized messy breaks userspace again; with that patch, container runtime needs to consider if
volatileis the default which just breaks the existing containerd versions.Quoted 6 lines
Quoted 4 lines
Quoted 1 line
I'll leave this debate to the overlayfs maintainers ;)
On my own perspective and be responsible for common users (and as a containerd maintainer [1]),
No wonder containerd is getting harder and harder to use ;)
What do you mean, can you explain exactly?
You're just adding a new way to break the existing applications, no? You just breaks previous shipped containerd.
Quoted 6 lines
Quoted 3 lines
It's your responsibility to handle the cases where "strict" is explicitly required. Please do your homework. It is not the kernel's fault.
How do you modify the existing applications and scripts to adapt your incompatible new
Kconfig?
Why do you still insist it's "incompatible"? As far as I can see, mounting on the same directory is a very rare case, especially in production environments. If your use case relies on "strict", then simply don't turn it on. Everything across our large fleet of servers works perfectly well with it.
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Add a way to change the default behavior is fine, but the new default behavior should be worked with the same functionality and compatible, but switching to
volatilefeature is non-compatible and what is why containerd dropped volatile.
Quoted 7 lines
Quoted 2 lines
It only adds a dynamically changeable config. Why do you insist it breaks everything? Users can always change it whenever they need.
Can you find any
Kconfigoption that changes user-visible default functionality and causes almost any user application that relies on remounting to fail to mount again? If so, I think we should Cc Linus now.
I can't get you.
Signature · 2 lines
Regards Yafang
Quoted 44 lines · Yafang Shao
Quoted 43 lines · Gao Xiang
(+try to cc more FS people for visibility.)
Quoted 37 lines · Yafang Shao
Quoted 25 lines · Gao Xiang
Quoted 21 lines · Yafang Shao
Quoted 19 lines · Gao Xiang
Quoted 2 lines · Yafang Shao
Quoted 1 line · Gao Xiang
...
Quoted 7 lines
Quoted 5 lines
Again, I don't want such customized messy breaks userspace again; with that patch, container runtime needs to consider if
volatileis the default which just breaks the existing containerd versions.I'll leave this debate to the overlayfs maintainers ;)
On my own perspective and be responsible for common users (and as a containerd maintainer [1]),
Quoted 7 lines
Quoted 1 line
No wonder containerd is getting harder and harder to use ;)
What do you mean, can you explain exactly?
You're just adding a new way to break the existing applications, no? You just breaks previous shipped containerd.
It's your responsibility to handle the cases where "strict" is explicitly required. Please do your homework. It is not the kernel's fault.
Quoted 8 lines
Quoted 2 lines
How do you modify the existing applications and scripts to adapt your incompatible new
Kconfig?Why do you still insist it's "incompatible"? As far as I can see, mounting on the same directory is a very rare case, especially in production environments. If your use case relies on "strict", then simply don't turn it on. Everything across our large fleet of servers works perfectly well with it.
I've listed my reasons since this patch is really a red line for me (otherwise I won't comment your patch, it's none of my business.) and I've said enough, and the upstream kernel + overlayfs does not work only for your company:
You should first ask and gather how many existing applications which can mount overlayfs will deal with your new Kconfig; if not, what is the target audience of your opt-in Kconfig other than your company as a way to workaround "docker" upgrade.
Or if you'd like to introduce a new incompatible "overlayfs" (an overlayfs cannot mount again by design + volatile by default), I think we should name it as another fstype in order to avoid breaking existing applications.
Quoted 20 lines
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Add a way to change the default behavior is fine, but the new default behavior should be worked with the same functionality and compatible, but switching to
volatilefeature is non-compatible and what is why containerd dropped volatile.Quoted 7 lines
Quoted 2 lines
It only adds a dynamically changeable config. Why do you insist it breaks everything? Users can always change it whenever they need.
Can you find any
Kconfigoption that changes user-visible default functionality and causes almost any user application that relies on remounting to fail to mount again? If so, I think we should Cc Linus now.I can't get you.
Quoted 69 lines · Gao Xiang
Quoted 44 lines · Yafang Shao
Quoted 43 lines · Gao Xiang
(+try to cc more FS people for visibility.)
Quoted 37 lines · Yafang Shao
Quoted 25 lines · Gao Xiang
Quoted 21 lines · Yafang Shao
Quoted 19 lines · Gao Xiang
Quoted 2 lines · Yafang Shao
Quoted 1 line · Gao Xiang
...
Quoted 7 lines
Quoted 5 lines
Again, I don't want such customized messy breaks userspace again; with that patch, container runtime needs to consider if
volatileis the default which just breaks the existing containerd versions.I'll leave this debate to the overlayfs maintainers ;)
On my own perspective and be responsible for common users (and as a containerd maintainer [1]),
Quoted 7 lines
Quoted 1 line
No wonder containerd is getting harder and harder to use ;)
What do you mean, can you explain exactly?
You're just adding a new way to break the existing applications, no? You just breaks previous shipped containerd.
It's your responsibility to handle the cases where "strict" is explicitly required. Please do your homework. It is not the kernel's fault.
Quoted 8 lines
Quoted 2 lines
How do you modify the existing applications and scripts to adapt your incompatible new
Kconfig?Why do you still insist it's "incompatible"? As far as I can see, mounting on the same directory is a very rare case, especially in production environments. If your use case relies on "strict", then simply don't turn it on. Everything across our large fleet of servers works perfectly well with it.
I've listed my reasons since this patch is really a red line for me (otherwise I won't comment your patch, it's none of my business.) and I've said enough, and the upstream kernel + overlayfs does not work only for your company:
You should first ask and gather how many existing applications which can mount overlayfs will deal with your new
Kconfig; if not, what is the target audience of your opt-inKconfigother than your company as a way to workaround "docker" upgrade.
Quoted 4 lines
Or if you'd like to introduce a new incompatible "overlayfs" (an overlayfs cannot mount again by design + volatile by default), I think we should name it as another fstype in order to avoid breaking existing applications.
I'm exhausted. We will keep this feature within our local kernel.
Signature · 2 lines
Regards Yafang
I have no idea where this coming from as I can't find an earlier version in the fsdevel archives. But changing user visible mount options through konfig options is simply bonkers.
I'd also like to not that the submitter does have a history of crazy patches including those to support proprietary modules and then attacking people criticizing those patches, so I can only suggest to every maintainer to ignore them for their own sanity.
Quoted 10 lines · Christoph Hellwig
I have no idea where this coming from as I can't find an earlier version in the fsdevel archives. But changing user visible mount options through konfig options is simply bonkers.
I'd also like to not that the submitter does have a history of crazy patches including those to support proprietary modules and then attacking people criticizing those patches, so I can only suggest to every maintainer to ignore them for their own sanity.
Honestly surprised you haven't made it to my spam list ;)
Signature · 2 lines
Regards Yafang
Hi Christoph,
Quoted 8 lines · Christoph Hellwig
I have no idea where this coming from as I can't find an earlier version in the fsdevel archives. But changing user visible mount options through konfig options is simply bonkers.
I'd also like to not that the submitter does have a history of crazy patches including those to support proprietary modules and then attacking people criticizing those patches, so I can only suggest to every maintainer to ignore them for their own sanity.
Sorry about that I didn't Cc the proper list at first, but it could be checked by using lore:
https://lore.kernel.org/linux-unionfs/[email protected]/T/#u
This topic is very specific to overlayfs details, I'm not sure how I could say the background in brief.
But almost every single container user uses overlayfs now, so in order to be responsible for end users, container runtimes and applications, I used some aggressive way this time.
Thanks,
Gao Xiang
Amir Goldstein
maintainerQuoted 17 lines · Gao Xiang
Hi Christoph,
Quoted 8 lines · Christoph Hellwig
I have no idea where this coming from as I can't find an earlier version in the fsdevel archives. But changing user visible mount options through konfig options is simply bonkers.
I'd also like to not that the submitter does have a history of crazy patches including those to support proprietary modules and then attacking people criticizing those patches, so I can only suggest to every maintainer to ignore them for their own sanity.
Sorry about that I didn't Cc the proper list at first, but it could be checked by using lore:
https://lore.kernel.org/linux-unionfs/[email protected]/T/#u
Quoted 7 lines
This topic is very specific to overlayfs details, I'm not sure how I could say the background in brief.
But almost every single container user uses overlayfs now, so in order to be responsible for end users, container runtimes and applications, I used some aggressive way this time.
Gao,
Thank you for holding the fort!
and for foreseeing this containerd regression.
I also agree with all your other arguments that "volatile" should never be the system/module default - it is too risky - one needs to know what they are doing when using "volatile".
I will respect your NACK.
Yafang,
I agree with you that your patch appears to follow precedents in overlayfs, but I also think that "volatile" is not a good candidate for this practice, mostly because it makes mount cycle fail.
Since it is so much easier for your employer to live patch the kernel than to upgrade/patch docker, how about applying this live patch to your kernels?
fs/overlayfs/params.c +1 -1
@@ -155,7 +155,7 @@ static const char *ovl_fsync_mode(struct ovl_config *config)static int ovl_fsync_mode_def (void ) { -return OVL_FSYNC_AUTO ; +return OVL_FSYNC_VOLATILE ; }
Thanks,
Amir.
Quoted 53 lines · Amir Goldstein
Quoted 17 lines · Gao Xiang
Hi Christoph,
Quoted 8 lines · Christoph Hellwig
I have no idea where this coming from as I can't find an earlier version in the fsdevel archives. But changing user visible mount options through konfig options is simply bonkers.
I'd also like to not that the submitter does have a history of crazy patches including those to support proprietary modules and then attacking people criticizing those patches, so I can only suggest to every maintainer to ignore them for their own sanity.
Sorry about that I didn't Cc the proper list at first, but it could be checked by using lore:
https://lore.kernel.org/linux-unionfs/[email protected]/T/#uQuoted 7 lines
This topic is very specific to overlayfs details, I'm not sure how I could say the background in brief.
But almost every single container user uses overlayfs now, so in order to be responsible for end users, container runtimes and applications, I used some aggressive way this time.
Gao,
Thank you for holding the fort!
and for foreseeing this containerd regression.I also agree with all your other arguments that "volatile" should never be the system/module default - it is too risky - one needs to know what they are doing when using "volatile".
I will respect your NACK.
Yafang,
I agree with you that your patch appears to follow precedents in overlayfs, but I also think that "volatile" is not a good candidate for this practice, mostly because it makes mount cycle fail.
Since it is so much easier for your employer to live patch the kernel than to upgrade/patch docker, how about applying this live patch to your kernels?
fs/overlayfs/params.c +0
-0
fs/overlayfs/params.c static const char *ovl_fsync_mode(struct ovl_config *config) · line 159
Quoted 6 lines
fs/overlayfs/params.c +1
-1@@ -155,7 +155,7 @@ static const char *ovl_fsync_mode(struct ovl_config *config)static int ovl_fsync_mode_def (void ) { -return OVL_FSYNC_AUTO ; +return OVL_FSYNC_VOLATILE ; }
Thanks for your guidance. This is similar to how we handle livepatching, and we also implement the "volatile" option for already-mounted systems.
Since the "volatile" option has been working fine for over a year across our large fleet of Kubernetes production servers, I submitted this change to upstream. In Kubernetes environments, there is no use case for remounting the workdir to the same directory, so the volatile file won't cause any issues — which is why the "volatile" setting has been working well there. But yeah, I respect your decision not to accept it.
Signature · 2 lines
Regards Yafang
Quoted 3 lines · Yafang Shao
Quoted 2 lines · Amir Goldstein
On Tue, Jun 23, 2026 at 3:47 PM Gao Xiang <[email protected]> wrote:
...
Quoted 7 lines
Since the "volatile" option has been working fine for over a year across our large fleet of Kubernetes production servers, I submitted this change to upstream. In Kubernetes environments, there is no use case for remounting the workdir to the same directory, so the volatile file won't cause any issues — which is why the "volatile" setting has been working well there.
That is not an excuse to introduce such new incompatible behavior to the linux upstream starting only for your customized fleet and bother every single opensource user/application to adapt this behavior which uses overlayfs now.
If you insist to work on a breaking change, please gather userspace feedback boardly first.
And here I just say you and your employer are lucky since I know it's also risky even on the generic kubernate perspective. Please just maintain your livepatch downstream, thanks.
No need to reply, and I won't reply this either, it's just waste of my time to explain a basic upstream principle again and again.
Thanks,
Gao Xiang
Quoted 25 lines · Gao Xiang
Quoted 3 lines · Yafang Shao
Quoted 2 lines · Amir Goldstein
On Tue, Jun 23, 2026 at 3:47 PM Gao Xiang <[email protected]> wrote:
...
Quoted 7 lines
Since the "volatile" option has been working fine for over a year across our large fleet of Kubernetes production servers, I submitted this change to upstream. In Kubernetes environments, there is no use case for remounting the workdir to the same directory, so the volatile file won't cause any issues — which is why the "volatile" setting has been working well there.
That is not an excuse to introduce such new incompatible behavior to the linux upstream starting only for your customized fleet and bother every single opensource user/application to adapt this behavior which uses overlayfs now.
If you insist to work on a breaking change, please gather userspace feedback boardly first.
Quoted 7 lines
And here I just say you and your employer are lucky since I know it's also risky even on the generic kubernate perspective. Please just maintain your livepatch downstream, thanks.
No need to reply, and I won't reply this either, it's just waste of my time to explain a basic upstream principle again and again.
I'll just go ahead and redirect you to /dev/null ;)
Signature · 2 lines
Regards Yafang
Hi,
Quoted 30 lines · Yafang Shao
Quoted 29 lines · Gao Xiang
Quoted 3 lines · Yafang Shao
Quoted 2 lines · Amir Goldstein
On Tue, Jun 23, 2026 at 3:47 PM Gao Xiang <[email protected]> wrote:
...
Quoted 7 lines
Since the "volatile" option has been working fine for over a year across our large fleet of Kubernetes production servers, I submitted this change to upstream. In Kubernetes environments, there is no use case for remounting the workdir to the same directory, so the volatile file won't cause any issues — which is why the "volatile" setting has been working well there.
That is not an excuse to introduce such new incompatible behavior to the linux upstream starting only for your customized fleet and bother every single opensource user/application to adapt this behavior which uses overlayfs now.
If you insist to work on a breaking change, please gather userspace feedback boardly first.
And here I just say you and your employer are lucky since I know it's also risky even on the generic kubernate perspective. Please just maintain your livepatch downstream, thanks.
Quoted 5 lines
Quoted 3 lines
No need to reply, and I won't reply this either, it's just waste of my time to explain a basic upstream principle again and again.
I'll just go ahead and redirect you to
/dev/null;)
The whole thread is:
https://lore.kernel.org/linux-unionfs/[email protected]/T/#u
Personally, I feel offensive due to Yafang's expression written above. I keep maintain my replies in technical discussion, but the replies above already include the following words:
"No wonder containerd is getting harder and harder to use ;)"
"Please do your homework."
"Again, pls do your homework."
and the emotional texts ;) make me feel uneasy over the time,
especially the final words "I'll just go ahead and redirect you to /dev/null ;)"
So I write an email as a public report to the CoC since I'm not sure how to proceed to comment the community patches because I could get offence as a result.
Thanks,
Gao Xiang
Quoted 57 lines · Gao Xiang
Hi,
Quoted 30 lines · Yafang Shao
Quoted 29 lines · Gao Xiang
Quoted 3 lines · Yafang Shao
Quoted 2 lines · Amir Goldstein
On Tue, Jun 23, 2026 at 3:47 PM Gao Xiang <[email protected]> wrote:
...
Quoted 7 lines
Since the "volatile" option has been working fine for over a year across our large fleet of Kubernetes production servers, I submitted this change to upstream. In Kubernetes environments, there is no use case for remounting the workdir to the same directory, so the volatile file won't cause any issues — which is why the "volatile" setting has been working well there.
That is not an excuse to introduce such new incompatible behavior to the linux upstream starting only for your customized fleet and bother every single opensource user/application to adapt this behavior which uses overlayfs now.
If you insist to work on a breaking change, please gather userspace feedback boardly first.
And here I just say you and your employer are lucky since I know it's also risky even on the generic kubernate perspective. Please just maintain your livepatch downstream, thanks.
Quoted 5 lines
Quoted 3 lines
No need to reply, and I won't reply this either, it's just waste of my time to explain a basic upstream principle again and again.
I'll just go ahead and redirect you to
/dev/null;)The whole thread is:
https://lore.kernel.org/linux-unionfs/[email protected]/T/#uPersonally, I feel offensive due to Yafang's expression written above. I keep maintain my replies in technical discussion, but the replies above already include the following words:
"No wonder containerd is getting harder and harder to use ;)"
"Please do your homework."
"Again, pls do your homework."
and the emotional texts
;)make me feel uneasy over the time,
Quoted 6 lines
especially the final words "I'll just go ahead and redirect you to
/dev/null;)"So I write an email as a public report to the CoC since I'm not sure how to proceed to comment the community patches because I could get offence as a result.
I believe you could learn a thing or two from Amir about how to comment on others' patches ;)
Signature · 2 lines
Regards Yafang
Quoted 61 lines · Yafang Shao
Quoted 60 lines · Gao Xiang
Hi,
Quoted 30 lines · Yafang Shao
Quoted 29 lines · Gao Xiang
Quoted 3 lines · Yafang Shao
Quoted 2 lines · Amir Goldstein
On Tue, Jun 23, 2026 at 3:47 PM Gao Xiang <[email protected]> wrote:
...
Quoted 7 lines
Since the "volatile" option has been working fine for over a year across our large fleet of Kubernetes production servers, I submitted this change to upstream. In Kubernetes environments, there is no use case for remounting the workdir to the same directory, so the volatile file won't cause any issues — which is why the "volatile" setting has been working well there.
That is not an excuse to introduce such new incompatible behavior to the linux upstream starting only for your customized fleet and bother every single opensource user/application to adapt this behavior which uses overlayfs now.
If you insist to work on a breaking change, please gather userspace feedback boardly first.
And here I just say you and your employer are lucky since I know it's also risky even on the generic kubernate perspective. Please just maintain your livepatch downstream, thanks.
Quoted 5 lines
Quoted 3 lines
No need to reply, and I won't reply this either, it's just waste of my time to explain a basic upstream principle again and again.
I'll just go ahead and redirect you to
/dev/null;)The whole thread is:
https://lore.kernel.org/linux-unionfs/[email protected]/T/#uPersonally, I feel offensive due to Yafang's expression written above. I keep maintain my replies in technical discussion, but the replies above already include the following words:
"No wonder containerd is getting harder and harder to use ;)"
"Please do your homework."
"Again, pls do your homework."
and the emotional texts
;)make me feel uneasy over the time,especially the final words "I'll just go ahead and redirect you to
/dev/null;)"
Quoted 6 lines
Quoted 3 lines
So I write an email as a public report to the CoC since I'm not sure how to proceed to comment the community patches because I could get offence as a result.
I believe you could learn a thing or two from Amir about how to comment on others' patches ;)
;) Please let me know why you commented:
"No wonder containerd is getting harder and harder to use ;)"
;)
Thanks,
Gao Xiang
Quoted 1 line
Quoted 69 lines · Gao Xiang
Quoted 65 lines · Yafang Shao
Quoted 63 lines · Gao Xiang
Hi,
Quoted 30 lines · Yafang Shao
Quoted 29 lines · Gao Xiang
Quoted 3 lines · Yafang Shao
Quoted 2 lines · Amir Goldstein
On Tue, Jun 23, 2026 at 3:47 PM Gao Xiang <[email protected]> wrote:
...
Quoted 7 lines
Since the "volatile" option has been working fine for over a year across our large fleet of Kubernetes production servers, I submitted this change to upstream. In Kubernetes environments, there is no use case for remounting the workdir to the same directory, so the volatile file won't cause any issues — which is why the "volatile" setting has been working well there.
That is not an excuse to introduce such new incompatible behavior to the linux upstream starting only for your customized fleet and bother every single opensource user/application to adapt this behavior which uses overlayfs now.
If you insist to work on a breaking change, please gather userspace feedback boardly first.
And here I just say you and your employer are lucky since I know it's also risky even on the generic kubernate perspective. Please just maintain your livepatch downstream, thanks.
Quoted 5 lines
Quoted 3 lines
No need to reply, and I won't reply this either, it's just waste of my time to explain a basic upstream principle again and again.
I'll just go ahead and redirect you to
/dev/null;)The whole thread is:
https://lore.kernel.org/linux-unionfs/[email protected]/T/#uPersonally, I feel offensive due to Yafang's expression written above. I keep maintain my replies in technical discussion, but the replies above already include the following words:
"No wonder containerd is getting harder and harder to use ;)"
"Please do your homework."
"Again, pls do your homework."
and the emotional texts
;)make me feel uneasy over the time,especially the final words "I'll just go ahead and redirect you to
/dev/null;)"So I write an email as a public report to the CoC since I'm not sure how to proceed to comment the community patches because I could get offence as a result.
Quoted 8 lines
Quoted 2 lines
I believe you could learn a thing or two from Amir about how to comment on others' patches ;)
;) Please let me know why you commented:
"No wonder containerd is getting harder and harder to use ;)"
;)
Because I find it harder to use than Docker ;)
Signature · 2 lines
Regards Yafang
Quoted 1 line · Yafang Shao
On Tue, Jun 23, 2026 at 5:49 PM Gao Xiang <[email protected]> wrote:
...
Quoted 21 lines
Quoted 12 lines
Quoted 11 lines
Quoted 8 lines
The other concern is that since
volatileomits fsync, so it's a posix violation (even that makes sense for container writable layers), not sure if we have to use it as the system-wide default configuration too.We have use cases for setting this as a system-wide configuration. It is unclear whether others have similar needs.
Quoted 8 lines
While I cannot speak out of overlayfs, but really it depends on if the use cases is generic.
Usually filesystems need to obey posix semantics as much as possible, and using specific mount option to relax (violate) some restriction, but it shouldn't be a system-wide stuff since otherwise user applications cannot know if they really live in the posix world.
Quoted 6 lines
AFAICS, the Linux kernel does not strictly follow POSIX in a number of areas.
BTW,
1. The "volatile" is not the default option.
2. Even when the system-wide default is set to "volatile", users can still change it per mount.
"
When overlay is mounted with "volatile" option, the directory "$workdir/work/incompat/volatile" is created. During next mount, overlay checks for this directory and refuses to mount if present. This is a strong indicator that the user should discard upper and work directories and create fresh ones.
"
When the system-wide default is set to "volatile", it will become a systemwide maintenance/compatiblity nightmare, at the very least, if any overlayfs mount that needs to be remountable on these systems (even something as small as a bash script that mounts an overlayfs which must be remountable) they have to add "fsync=", that needs application changes.
In my opinion, the argument "volatile is not the default option" is not valid, since any reasonably robust userspace application has to deal with _every visible Kconfig_ that could change the kernel's default incompatible behavior. Otherwise, users who run such applications on these kernels will inevitably complain.
On the other hand, I think the kernel itself should also be responsible for ensuring that a new opt-in default remains compatible with most existing userspace applications, regardless of whether it is gated behind a Kconfig.
I don't think exposing incompatible default behavior, even behind a new Kconfig, is okay without first reaching a sufficiently broad audience.
Thanks,
Gao Xiang