re-enable non-clustered mount & add MMP support
cover re-enable non-clustered mount & add MMP support
This serial patches re-enable ocfs2 non-clustered mount feature.
the previous patch c80af0c250c8 (Revert "ocfs2: mount shared volume without ha stack") revert Gang's non-clustered mount patch. This serial patches re-enable ocfs2 non-clustered mount.
the key different between local mount and non-clustered mount:
local mount feature (tunefs.ocfs2 --fs-features=[no]local) can't do convert job without ha stack. non-clustered mount feature can run totally without ha stack.
For avoiding data corruption when non-clustered & clustered mount are happening at same time, this serial patches also introduces MMP feature. MMP (Multiple Mount Protection) idea got from ext4 MMP (fs/ext4/mmp.c) which protects fs from being mounted more than once. For ocfs2 is a clustered fs and also for compatible with existing slotmap feature, I did some optimization and modification when porting from ext4 MMP to ocfs2.
The related userspace code for supporting MMP had been sent to github for reviewing:
- https://github.com/markfasheh/ocfs2-tools/pull/58
ocfs2-tools enable MMP and check status:
``` # enable MMP tunefs.ocfs2 --fs-feature=mmp /dev/vdb # check the command result tunefs.ocfs2 -Q "%H\n" /dev/vdb | grep MMP # active MMP on nocluster mount mount -t ocfs2 -o nocluster /dev/vdb /mnt # check slotmap info # echo slotmap | PAGER=cat debugfs.ocfs2 /dev/vdb ```
=== below are test cases for patches ====
<1> non-clustered mount vs local mount
1.1 tunefs.ocfs2 can't convert local/nolocal mount without ha stack.
``` (on ha stack env) mkfs.ocfs2 --cluster-stack=pcmk --cluster-name=hacluster -N 4 /dev/vdb tunefs.ocfs2 --fs-features=local /dev/vdb (<== success) tunefs.ocfs2 --fs-features=nolocal /dev/vdb (<== success) (on another node without ha stack) tunefs.ocfs2 --fs-features=local /dev/vdb (<== failure) ```
1.2 non-cluster feature can run without ha stack.
``` (on ha stack env) mkfs.ocfs2 --cluster-stack=pcmk --cluster-name=hacluster -N 4 /dev/vdb (on another node without ha stack) mount -t ocfs2 -o nocluster /dev/vdb /mnt (<== success) ```
<2> do clustered & non-clustered mount on same node
2.1 non-clustered mount => clustered mount
``` mkfs.ocfs2 --cluster-stack=pcmk --cluster-name=hacluster -N 4 /dev/vdb mount -t ocfs2 -o nocluster /dev/vdb /mnt mount -t ocfs2 /dev/vdb /mnt (<=== failure) ```
2.2 clustered mount => non-clustered mount
``` mkfs.ocfs2 --cluster-stack=pcmk --cluster-name=hacluster -N 4 /dev/vdb mount -t ocfs2 /dev/vdb /mnt mount -t ocfs2 -o nocluster /dev/vdb /mnt (<=== failure) ```
<3> one node does clustered mount, another does non-clustered mount
test rule: clustered mount and non-clustered mount can not exist at same time.
3.1 clustered mount @node1 => [no]clustered mount @node2
``` node1: mkfs.ocfs2 --cluster-stack=pcmk --cluster-name=hacluster -N 4 /dev/vdb mount -t ocfs2 /dev/vdb /mnt node2: mount -t ocfs2 /dev/vdb /mnt (<== success) umount /mnt mount -t ocfs2 -o nocluster /dev/vdb /mnt (<== failure) ```
3.2 enable mmp, repeate 3.1 case
``` node1: mkfs.ocfs2 --cluster-stack=pcmk --cluster-name=hacluster -N 4 /dev/vdb tunefs.ocfs2 --fs-features=mmp /dev/vdb (<== enable mmp) mount -t ocfs2 /dev/vdb /mnt node2: mount -t ocfs2 /dev/vdb /mnt (<== wait ~22s [*] for mmp, then success) umount /mnt mount -t ocfs2 -o nocluster /dev/vdb /mnt (<== failure) ```
[*] 22s:
(OCFS2_MMP_MIN_CHECK_INTERVAL * 2 + 1) * 2 times (calling schedule_timeout_interruptible)
3.3 noclustered mount @node1 => [no]clustered mount @node2
``` node1: mkfs.ocfs2 --cluster-stack=pcmk --cluster-name=hacluster -N 4 /dev/vdb mount -t ocfs2 -o nocluster /dev/vdb /mnt node2: mount -t ocfs2 /dev/vdb /mnt (<== failure) mount -t ocfs2 -o nocluster /dev/vdb /mnt (<== success, without mmp enable) umount /mnt (<== will ZERO out slotmap area while node1 still mounting) ```
3.4 enable mmp, repeate 3.3 case.
``` node1: mkfs.ocfs2 --cluster-stack=pcmk --cluster-name=hacluster -N 4 /dev/vdb tunefs.ocfs2 --fs-features=mmp /dev/vdb (<== enable mmp) mount -t ocfs2 -o nocluster /dev/vdb /mnt node2: mount -t ocfs2 /dev/vdb /mnt (<== failure) mount -t ocfs2 -o nocluster /dev/vdb /mnt (<== failure, denied by mmp) ```
<4> simulate mounting after machine crash
info:
- below all steps do on one node
- address 287387648 is the '//slot_map' extent address.
- test the rule: If last mount didn't do unmount, (eg: crash), the next mount MUST be same mount type.
4.0 how to calculate '//slot_map' extent address
```
# PAGER=cat debugfs.ocfs2 -R "stats" /dev/vdb | grep "Block Size Bits"
Block Size Bits: 12 Cluster Size Bits: 12
# PAGER=cat debugfs.ocfs2 -R "stat //slot_map" /dev/vdb | grep -A1
# "Block#"
## Offset Clusters Block# Flags
0 0 1 70163 0x0
```70163 * (1<<12) = 70163 * 4096 = 287387648
4.1 clustered mount => crash => non-clustered mount fails => clean slotmap => non-clustered mount succeeds
``` mkfs.ocfs2 --cluster-stack=pcmk --cluster-name=hacluster -N 4 /dev/vdb mount -t ocfs2 /dev/vdb /mnt dd if=/dev/vdb bs=1 count=32 skip=287387648 of=/root/slotmap.cluster.mnted (<== backup slot info) umount /mnt dd if=/root/slotmap.cluster.mnted of=/dev/vdb seek=287387648 bs=1 count=32 (<== overwrite) mount -t ocfs2 -o nocluster /dev/vdb /mnt <== failure mount -t ocfs2 /dev/vdb /mnt && umount /mnt <== clean slot 0 mount -t ocfs2 -o nocluster /dev/vdb /mnt <== success ```
4.2 non-clustered mount => crash => clustered mount fails => clean slotmap => clustered mount succeeds
``` mkfs.ocfs2 --cluster-stack=pcmk --cluster-name=hacluster -N 4 /dev/vdb mount -t ocfs2 -o nocluster /dev/vdb /mnt dd if=/dev/vdb bs=1 count=32 skip=287387648 of=/root/slotmap.nocluster.mnted umount /mnt dd if=/root/slotmap.nocluster.mnted of=/dev/vdb seek=287387648 bs=1 count=32 mount -t ocfs2 /dev/vdb /mnt <== failure mount -t ocfs2 -o nocluster /dev/vdb /mnt && umount /mnt <== clean slot 0 mount -t ocfs2 /dev/vdb /mnt <== success ```
<5> MMP test
5.1 node1 noclustered mount => node 2 noclustered mount
disable mmp
``` node1: mkfs.ocfs2 --cluster-stack=pcmk --cluster-name=hacluster -N 4 /dev/vdb mount -t ocfs2 -o nocluster /dev/vdb /mnt node2: mount -t ocfs2 -o nocluster /dev/vdb /mnt (<== success) ```
enable mmp
``` node1: mkfs.ocfs2 --cluster-stack=pcmk --cluster-name=hacluster -N 4 /dev/vdb tunefs.ocfs2 --fs-features=mmp /dev/vdb mount -t ocfs2 -o nocluster /dev/vdb /mnt node2: mount -t ocfs2 -o nocluster /dev/vdb /mnt (<== wait ~12s[*], failure by mmp) ```
[*] 12s:
sleep (OCFS2_MMP_MIN_CHECK_INTERVAL * 2 + 1) then detect mmp_seq was changed, then failed.
5.2 node1 clustered mount => node 2 clustered mount
see case 3.2
5.3 node1 noclustered mount => node 2 noclustered mount
see case 3.4
5.4 remount test
5.4.1 non-clustered mount (run commands on same node)
``` mkfs.ocfs2 --cluster-stack=pcmk --cluster-name=hacluster -N 4 /dev/vdb tunefs.ocfs2 --fs-features=mmp /dev/vdb mount -t ocfs2 -o nocluster /dev/vdb /mnt ps axj | grep kmmpd (<== will show kmmpd) PAGER=cat debugfs.ocfs2 -R "slotmap" /dev/vdb (<== show 'OCFS2_MMP_SEQ') mount -o remount,ro,nocluster /dev/vdb /mnt (<== kmmpd will stop) ps axj | grep kmmpd (<== won't show kmmpd) PAGER=cat debugfs.ocfs2 -R "slotmap" /dev/vdb (<== show 'OCFS2_MMP_SEQ_CLEAN') mount -o remount,rw,nocluster /dev/vdb /mnt (<== kmmpd will start) ps axj | grep kmmpd (<== will show kmmpd) PAGER=cat debugfs.ocfs2 -R "slotmap" /dev/vdb (<== show 'OCFS2_MMP_SEQ') ```
5.4.2 clustered mount
``` mkfs.ocfs2 --cluster-stack=pcmk --cluster-name=hacluster -N 4 /dev/vdb tunefs.ocfs2 --fs-features=mmp /dev/vdb mount -t ocfs2 /dev/vdb /mnt (<== clustered mount won't create kmmpd) PAGER=cat debugfs.ocfs2 -R "slotmap" /dev/vdb (<== show 'OCFS2_VALID_CLUSTER') mount -o remount,ro /dev/vdb /mnt PAGER=cat debugfs.ocfs2 -R "slotmap" /dev/vdb (<== show 'OCFS2_VALID_CLUSTER') mount -o remount,rw /dev/vdb /mnt (<== wait for ~22s by mmp start) PAGER=cat debugfs.ocfs2 -R "slotmap" /dev/vdb (<== show 'OCFS2_VALID_CLUSTER') ```
Heming Zhao (4):
ocfs2: Fix freeing uninitialized resource on ocfs2_dlm_shutdown
ocfs2: add mlog ML_WARNING support
re-enable "ocfs2: mount shared volume without ha stack"
ocfs2: introduce ext4 MMP feature
fs/ocfs2/cluster/masklog.c | 3 + fs/ocfs2/cluster/masklog.h | 9 +- fs/ocfs2/dlmglue.c | 3 + fs/ocfs2/ocfs2.h | 6 +- fs/ocfs2/ocfs2_fs.h | 13 +- fs/ocfs2/slot_map.c | 479 +++++++++++++++++++++++++++++++++++-- fs/ocfs2/slot_map.h | 3 + fs/ocfs2/super.c | 42 +++- 8 files changed, 527 insertions(+), 31 deletions(-)
Signature · 7 lines
2.37.1 _______________________________________________ Ocfs2-devel mailing list [email protected] https://oss.oracle.com/mailman/listinfo/ocfs2-devel
PATCH 1/4 ocfs2: Fix freeing uninitialized resource on ocfs2_dlm_shutdown
On local mount mode, there is no dlm resource initalized. If ocfs2_mount_volume() fails in ocfs2_find_slot(), error handling flow will call ocfs2_dlm_shutdown(), then does dlm resource cleanup job, which will trigger kernel crash.
return error")
--- fs/ocfs2/dlmglue.c | 3 +++ 1 file changed, 3 insertions(+)
fs/ocfs2/dlmglue.c +3 -0
@@ -3385,6 +3385,9 @@ int ocfs2_dlm_init(struct ocfs2_super *osb)void ocfs2_dlm_shutdown (struct ocfs2_super *osb,int hangup_pending) { +if (ocfs2_mount_local (osb)) +return ; +ocfs2_drop_osb_locks (osb);/*
Signature · 7 lines
2.37.1 _______________________________________________ Ocfs2-devel mailing list [email protected] https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 1/4] ocfs2: Fix freeing uninitialized resource on ocfs2_dlm_shutdown
Quoted 7 lines · Heming Zhao
On local mount mode, there is no dlm resource initalized. If
ocfs2_mount_volume()fails inocfs2_find_slot(), error handling flow will callocfs2_dlm_shutdown(), then does dlm resource cleanup job, which will trigger kernel crash.Fixes 0737e01de9c4 ("ocfs2: ocfs2_mount_volume does cleanup job beforereturn error")
Should be put at the same line.
Quoted 10 lines
Signed-off-by Heming Zhao <[email protected]>--- fs/ocfs2/dlmglue.c | 3 +++ 1 file changed, 3 insertions(+)fs/ocfs2/dlmglue.c +0
-0
fs/ocfs2/dlmglue.c int ocfs2_dlm_init(struct ocfs2_super *osb) · line 3390
Quoted 6 lines
fs/ocfs2/dlmglue.c +3
-0@@ -3385,6 +3385,9 @@ int ocfs2_dlm_init(struct ocfs2_super *osb)void ocfs2_dlm_shutdown (struct ocfs2_super *osb,int hangup_pending) { +if (ocfs2_mount_local (osb)) +return ; +
IMO, we have to do part of ocfs2_dlm_shutdown() jobs such as ocfs2_lock_res_free(), which will remove lockres from d_lockres_tracking added by ocfs2_xxx_lock_res_init().
Before commit 0737e01de9c4, it seems this issue also exists since osb->cconn is already set under local mount mode.
Thanks,
Joseph
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 1/4] ocfs2: Fix freeing uninitialized resource on ocfs2_dlm_shutdown
Quoted 8 lines · Joseph Qi
Quoted 5 lines · Heming Zhao
On local mount mode, there is no dlm resource initalized. If
ocfs2_mount_volume()fails inocfs2_find_slot(), error handling flow will callocfs2_dlm_shutdown(), then does dlm resource cleanup job, which will trigger kernel crash.
Quoted 4 lines
Quoted 2 lines
Fixes 0737e01de9c4 ("ocfs2: ocfs2_mount_volume does cleanup job beforereturn error")
Should be put at the same line.
OK
Quoted 6 lines
Quoted 6 lines
Signed-off-by Heming Zhao <[email protected]>--- fs/ocfs2/dlmglue.c | 3 +++ 1 file changed, 3 insertions(+)
Quoted 16 lines
fs/ocfs2/dlmglue.c int ocfs2_dlm_init(struct ocfs2_super *osb) · line 3390
Quoted 11 lines
fs/ocfs2/dlmglue.c +3
-0@@ -3385,6 +3385,9 @@ int ocfs2_dlm_init(struct ocfs2_super *osb)void ocfs2_dlm_shutdown (struct ocfs2_super *osb,int hangup_pending) { +if (ocfs2_mount_local (osb)) +return ; +IMO, we have to do part of
ocfs2_dlm_shutdown()jobs such asocfs2_lock_res_free(), which will remove lockres fromd_lockres_trackingadded byocfs2_xxx_lock_res_init().
ocfs2_dlm_shutdown does the cleanup job for ocfs2_dlm_init. This patch fixed crash in local mount error flow.
In local mount mode, ocfs2_dlm_init does nothing, which should make ocfs2_dlm_shutdown do nothing.
And I checked all calling ocfs2_dlm_shutdown cases:
1. mount flow:
ocfs2_fill_super
+ xxx =fails=> label:out_super (checked, work fine)
|
+ ocfs2_mount_volume =fails=> label:out_debugfs (checked, work fine) | |
| + xxx =fails=> cleanup everything before returning
|
+ xxx =fails=> label:out_dismout
At this time, dlm has been init successfully, we can call all lines of ocfs2_dlm_shutdown.
2. ocfs2_dismount_volume => 'osb->cconn' is true.
this MUST be dlm successfully init case. everything looks fine.
In previous mail/patch: [PATCH] test error handling during mount stage, I may forget to test local mount mode. So this crash didn't be triggered.
Quoted 2 lines
Before commit
0737e01de9c4, it seems this issue also exists sinceosb->cconnis already set under local mount mode.
Yes. The bug exists since local mount feature was introduced, commit number:c271c5c22b0a7ca45fda15f1f4d258bca36a5b94. I will change 'Fixes' on next version.
(Hope 'CC' list takes effect for this mail. -_-# )
Thanks,
Heming
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 1/4] ocfs2: Fix freeing uninitialized resource on ocfs2_dlm_shutdown
Quoted 38 lines · Heming Zhao
Quoted 8 lines · Joseph Qi
Quoted 5 lines · Heming Zhao
On local mount mode, there is no dlm resource initalized. If
ocfs2_mount_volume()fails inocfs2_find_slot(), error handling flow will callocfs2_dlm_shutdown(), then does dlm resource cleanup job, which will trigger kernel crash.Quoted 4 lines
Quoted 2 lines
Fixes 0737e01de9c4 ("ocfs2: ocfs2_mount_volume does cleanup job beforereturn error")
Should be put at the same line.
OK
Quoted 6 lines
Quoted 6 lines
Signed-off-by Heming Zhao <[email protected]>--- fs/ocfs2/dlmglue.c | 3 +++ 1 file changed, 3 insertions(+)Quoted 16 lines
fs/ocfs2/dlmglue.c int ocfs2_dlm_init(struct ocfs2_super *osb) · line 3390
Quoted 11 lines
fs/ocfs2/dlmglue.c +3
-0@@ -3385,6 +3385,9 @@ int ocfs2_dlm_init(struct ocfs2_super *osb)void ocfs2_dlm_shutdown (struct ocfs2_super *osb,int hangup_pending) { +if (ocfs2_mount_local (osb)) +return ; +IMO, we have to do part of
ocfs2_dlm_shutdown()jobs such asocfs2_lock_res_free(), which will remove lockres fromd_lockres_trackingadded byocfs2_xxx_lock_res_init().
Quoted 5 lines
ocfs2_dlm_shutdowndoes the cleanup job forocfs2_dlm_init. This patch fixed crash in local mount error flow.
In local mount mode,ocfs2_dlm_initdoes nothing, which should makeocfs2_dlm_shutdowndo nothing.
Not exactly, even in local mount we have to initialize lockres like super lock.
Quoted 18 lines
And I checked all calling
ocfs2_dlm_shutdowncases:
1. mount flow:
ocfs2_fill_super
+ xxx =fails=> label:out_super (checked, work fine)
|
+ocfs2_mount_volume=fails=> label:out_debugfs (checked, work fine) | |
| + xxx =fails=> cleanup everything before returning
|
+ xxx =fails=> label:out_dismout
At this time, dlm has been init successfully, we can call all lines ofocfs2_dlm_shutdown.2.
ocfs2_dismount_volume=> 'osb->cconn' is true.
this MUST be dlm successfully init case. everything looks fine.
Quoted 9 lines
In previous mail/patch: [PATCH] test error handling during mount stage, I may forget to test local mount mode. So this crash didn't be triggered.
Quoted 2 lines
Before commit
0737e01de9c4, it seems this issue also exists sinceosb->cconnis already set under local mount mode.Yes. The bug exists since local mount feature was introduced, commit number:
c271c5c22b0a7ca45fda15f1f4d258bca36a5b94. I will change 'Fixes' on next version.
This patch can be a separate fix and we don't have to bind with nocluster mount feature, even though you've triggered it by related local mount.
Thanks,
Joseph
Quoted 4 lines
(Hope 'CC' list takes effect for this mail. -_-# )
Thanks,
Heming
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 1/4] ocfs2: Fix freeing uninitialized resource on ocfs2_dlm_shutdown
Quoted 41 lines · Joseph Qi
Quoted 38 lines · Heming Zhao
Quoted 8 lines · Joseph Qi
Quoted 5 lines · Heming Zhao
On local mount mode, there is no dlm resource initalized. If
ocfs2_mount_volume()fails inocfs2_find_slot(), error handling flow will callocfs2_dlm_shutdown(), then does dlm resource cleanup job, which will trigger kernel crash.Quoted 4 lines
Quoted 2 lines
Fixes 0737e01de9c4 ("ocfs2: ocfs2_mount_volume does cleanup job beforereturn error")
Should be put at the same line.
OK
Quoted 22 lines
Quoted 11 lines
Signed-off-by Heming Zhao <[email protected]>--- fs/ocfs2/dlmglue.c | 3 +++ 1 file changed, 3 insertions(+)fs/ocfs2/dlmglue.c +0
-0fs/ocfs2/dlmglue.c int ocfs2_dlm_init(struct ocfs2_super *osb) · line 3390
Quoted 6 lines
fs/ocfs2/dlmglue.c +3
-0@@ -3385,6 +3385,9 @@ int ocfs2_dlm_init(struct ocfs2_super *osb)void ocfs2_dlm_shutdown (struct ocfs2_super *osb,int hangup_pending) { +if (ocfs2_mount_local (osb)) +return ; +IMO, we have to do part of
ocfs2_dlm_shutdown()jobs such asocfs2_lock_res_free(), which will remove lockres fromd_lockres_trackingadded byocfs2_xxx_lock_res_init().
Quoted 7 lines
Quoted 5 lines
ocfs2_dlm_shutdowndoes the cleanup job forocfs2_dlm_init. This patch fixed crash in local mount error flow.
In local mount mode,ocfs2_dlm_initdoes nothing, which should makeocfs2_dlm_shutdowndo nothing.Not exactly, even in local mount we have to initialize lockres like super lock.
Thank you for your patient explaination, I understood your meaning, will change it in next version.
I am very busy these days, it makes my brain stop working.
Quoted 26 lines
Quoted 26 lines
And I checked all calling
ocfs2_dlm_shutdowncases:
1. mount flow:
ocfs2_fill_super
+ xxx =fails=> label:out_super (checked, work fine)
|
+ocfs2_mount_volume=fails=> label:out_debugfs (checked, work fine) | |
| + xxx =fails=> cleanup everything before returning
|
+ xxx =fails=> label:out_dismout
At this time, dlm has been init successfully, we can call all lines ofocfs2_dlm_shutdown.2.
ocfs2_dismount_volume=> 'osb->cconn' is true.
this MUST be dlm successfully init case. everything looks fine.In previous mail/patch: [PATCH] test error handling during mount stage, I may forget to test local mount mode. So this crash didn't be triggered.
Quoted 2 lines
Before commit
0737e01de9c4, it seems this issue also exists sinceosb->cconnis already set under local mount mode.
Quoted 6 lines
Quoted 3 lines
Yes. The bug exists since local mount feature was introduced, commit number:
c271c5c22b0a7ca45fda15f1f4d258bca36a5b94. I will change 'Fixes' on next version.This patch can be a separate fix and we don't have to bind with nocluster mount feature, even though you've triggered it by related local mount.
OK.
Quoted 8 lines
Thanks,
JosephQuoted 4 lines
(Hope 'CC' list takes effect for this mail. -_-# )
Thanks,
Heming
Thanks,
Heming
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
PATCH 2/4 ocfs2: add mlog ML_WARNING support
In reply to Heming Zhao via Ocfs2-develThis commit gives new message type for ocfs2.
--- fs/ocfs2/cluster/masklog.c | 3 +++ fs/ocfs2/cluster/masklog.h | 9 +++++---- 2 files changed, 8 insertions(+), 4 deletions(-)
fs/ocfs2/cluster/masklog.c +3 -0
@@ -63,6 +63,9 @@ void __mlog_printk(const u64 *mask, const char *func, int line,if (*mask &ML_ERROR ) { level =KERN_ERR ; prefix ="ERROR: " ; + }else if (*mask &ML_WARNING ) { + level =KERN_WARNING ; + prefix ="WARNING: " ; }else if (*mask &ML_NOTICE ) { level =KERN_NOTICE ; }else {
fs/ocfs2/cluster/masklog.h +5 -4
@@ -86,10 +86,11 @@/* bits that are infrequently given and frequently matched in the high word */ #define ML_ERROR 0x1000000000000000ULL/* sent to KERN_ERR */ -#define ML_NOTICE 0x2000000000000000ULL/* setn to KERN_NOTICE */ -#define ML_KTHREAD 0x4000000000000000ULL/* kernel thread activity */ +#define ML_NOTICE 0x2000000000000000ULL/* sent to KERN_NOTICE */ +#define ML_WARNING 0x4000000000000000ULL/* sent to KERN_WARNING */ +#define ML_KTHREAD 0x8000000000000000ULL/* kernel thread activity */ -#define MLOG_INITIAL_AND_MASK (ML_ERROR|ML_NOTICE) +#define MLOG_INITIAL_AND_MASK (ML_ERROR|ML_WARNING|ML_NOTICE)#ifndef MLOG_MASK_PREFIX #define MLOG_MASK_PREFIX 0#endif
@@ -102,7 +103,7 @@#if defined(CONFIG_OCFS2_DEBUG_MASKLOG )#define ML_ALLOWED_BITS ~0#else -#define ML_ALLOWED_BITS (ML_ERROR|ML_NOTICE) +#define ML_ALLOWED_BITS (ML_ERROR|ML_WARNING|ML_NOTICE)#endif #define MLOG_MAX_BITS 64
Signature · 7 lines
2.37.1 _______________________________________________ Ocfs2-devel mailing list [email protected] https://oss.oracle.com/mailman/listinfo/ocfs2-devel
PATCH 3/4 re-enable "ocfs2: mount shared volume without ha stack"
In reply to Heming Zhao via Ocfs2-develthe key different between local mount and non-clustered mount:
local mount feature (tunefs.ocfs2 --fs-features=[no]local) can't do convert job without ha stack. non-clustered mount feature can run totally without ha stack.
commit 912f655d78c5 ("ocfs2: mount shared volume without ha stack") had bug, then commit c80af0c250c8f8a3c978aa5aafbe9c39b336b813 reverted it.
Let's give some explain for the issue mentioned by commit c80af0c250c8.
Under Junxiao's call trace, in __ocfs2_find_empty_slot(), the 'if' accessment is wrong. sl_node_num could be 0 at o2cb env.
with current information, the trigger flow (base on 912f655d78c5):
1>
nodeA with 'node_num = 0' for mounting. it will succeed. at this time, slotmap extent block will contains es_valid:1 & es_node_num:0 for nodeA
then ocfs2_update_disk_slot() will write back slotmap info to disk.
2>
then, nodeB with 'node_num = 1' for mounting
this time, osb->node_num is 1 (set by config file), osb->preferred is OCFS2_INVALID_SLOT (set by ocfs2_parse_options).
ocfs2_find_slot
+ ocfs2_update_slot_info //read slotmap info from disk
| + set si->si_slots[0].es_valid = 1 & si->si_slots[0].sl_node_num = 0 |
+ __ocfs2_node_num_to_slot //will return -ENOENT.
+ __ocfs2_find_empty_slot
+ if ((preferred >= 0) && (preferred < si->si_num_slots)) | fails enter this 'if' for preferred value is OCFS2_INVALID_SLOT |
+ 'for(i = 0; i < si->si_num_slots; i++)' search slot 0 successfully.
| 'si->si_slots[0].sl_node_num' is false. trigger 'break' condition. |
+ return slot 0.
it will cause nodeB grab nodeA journal dlm lock, then trigger hung.
How to do for this bug?
This commit re-enabled 912f655d78c5, next commit (add MMP support) will fix the issue.
--- fs/ocfs2/ocfs2.h | 4 +++- fs/ocfs2/slot_map.c | 46 ++++++++++++++++++++++++++------------------- fs/ocfs2/super.c | 21 +++++++++++++++++++++ 3 files changed, 51 insertions(+), 20 deletions(-)
fs/ocfs2/ocfs2.h +3 -1
@@ -277,6 +277,7 @@ enum ocfs2_mount_optionsOCFS2_MOUNT_JOURNAL_ASYNC_COMMIT =1 <<15 ,/* Journal Async Commit */ OCFS2_MOUNT_ERRORS_CONT =1 <<16 ,/* Return EIO to the calling process on error */ OCFS2_MOUNT_ERRORS_ROFS =1 <<17 ,/* Change filesystem to read-only on error */ + OCFS2_MOUNT_NOCLUSTER = 1 << 18, /* No cluster aware filesystem mount */ };#define OCFS2_OSB_SOFT_RO 0x0001
@@ -672,7 +673,8 @@ static inline int ocfs2_cluster_o2cb_global_heartbeat(struct ocfs2_super *osb)static inline int ocfs2_mount_local (struct ocfs2_super *osb) { -return (osb->s_feature_incompat &OCFS2_FEATURE_INCOMPAT_LOCAL_MOUNT ); +return ((osb->s_feature_incompat &OCFS2_FEATURE_INCOMPAT_LOCAL_MOUNT ) + || (osb->s_mount_opt &OCFS2_MOUNT_NOCLUSTER )); }static inline int ocfs2_uses_extended_slot_map (struct ocfs2_super *osb)
fs/ocfs2/slot_map.c +27 -19
65 lines not on this page · raw message
fs/ocfs2/super.c +21 -0
@@ -172,6 +172,7 @@ enum { Opt_dir_resv_level, Opt_journal_async_commit, Opt_err_cont, + Opt_nocluster, Opt_err, };
@@ -205,6 +206,7 @@ static const match_table_t tokens = { {Opt_dir_resv_level, "dir_resv_level=%u"}, {Opt_journal_async_commit, "journal_async_commit"}, {Opt_err_cont, "errors=continue"}, + {Opt_nocluster, "nocluster"}, {Opt_err, NULL} };
@@ -616,6 +618,13 @@ static int ocfs2_remount(struct super_block *sb, int *flags, char *data) goto out; } + tmp =OCFS2_MOUNT_NOCLUSTER ; +if ((osb->s_mount_opt & tmp) != (parsed_options.mount_opt & tmp)) { + ret = -EINVAL ; +mlog (ML_ERROR ,"Cannot change nocluster option on remount\n" ); + goto out; + } + tmp =OCFS2_MOUNT_HB_LOCAL |OCFS2_MOUNT_HB_GLOBAL |OCFS2_MOUNT_HB_NONE ;if ((osb->s_mount_opt & tmp) != (parsed_options.mount_opt & tmp)) {
@@ -856,6 +865,7 @@ static int ocfs2_verify_userspace_stack(struct ocfs2_super *osb, }if (ocfs2_userspace_stack (osb) && + !(osb->s_mount_opt &OCFS2_MOUNT_NOCLUSTER ) &&strncmp (osb->osb_cluster_stack, mopt->cluster_stack,OCFS2_STACK_LABEL_LEN )) { mlog(ML_ERROR,
@@ -1127,6 +1137,11 @@ static int ocfs2_fill_super(struct super_block *sb, void *data, int silent) osb->s_mount_opt &OCFS2_MOUNT_DATA_WRITEBACK ?"writeback" : "ordered"); +if ((osb->s_mount_opt &OCFS2_MOUNT_NOCLUSTER ) && + !(osb->s_feature_incompat &OCFS2_FEATURE_INCOMPAT_LOCAL_MOUNT )) +printk (KERN_NOTICE "ocfs2: The shared device (%s) is mounted " +"without cluster aware mode.\n" , osb->dev_str); +atomic_set (&osb->vol_state,VOLUME_MOUNTED );wake_up (&osb->osb_mount_event);
@@ -1437,6 +1452,9 @@ static int ocfs2_parse_options(struct super_block *sb,case Opt_journal_async_commit: mopt->mount_opt |=OCFS2_MOUNT_JOURNAL_ASYNC_COMMIT ;break ; +case Opt_nocluster: + mopt->mount_opt |=OCFS2_MOUNT_NOCLUSTER ; +break ;default :mlog (ML_ERROR , "Unrecognized mount option \"%s\" "
@@ -1548,6 +1566,9 @@ static int ocfs2_show_options(struct seq_file *s, struct dentry *root)if (opts &OCFS2_MOUNT_JOURNAL_ASYNC_COMMIT )seq_printf (s,",journal_async_commit" ); +if (opts &OCFS2_MOUNT_NOCLUSTER ) +seq_printf (s,",nocluster" ); +return 0 ; }
Signature · 7 lines
2.37.1 _______________________________________________ Ocfs2-devel mailing list [email protected] https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 3/4] re-enable "ocfs2: mount shared volume without ha stack"
Hi Heming,
Quoted 5 lines · On Fri, Jul 29, 2022 at 6:15 PM Heming Zhao via Ocfs2-devel…
the key different between local mount and non-clustered mount:
local mount feature (tunefs.ocfs2 --fs-features=[no]local) can't do convert job without ha stack. non-clustered mount feature can run totally without ha stack.
Can you please elaborate on this? Local mounts can run without a cluster stack so I don't see the difference there. We have tunefs.ocfs2 look for and join the cluster so as to avoid corrupting users data - that's a feature, not a bug. So what I'm seeing here is just opening us to potential corruptions. Is there a specific use case here that you're trying to account for? Are you fixing a particular bug?
Thanks,
--Mark
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 3/4] re-enable "ocfs2: mount shared volume without ha stack"
Hello Mark,
Quoted 5 lines · Mark Fasheh
Hi Heming,
Quoted 1 line · On Fri, Jul 29, 2022 at 6:15 PM Heming Zhao via Ocfs2-devel…
Quoted 7 lines
Quoted 4 lines
the key different between local mount and non-clustered mount:
local mount feature (tunefs.ocfs2 --fs-features=[no]local) can't do convert job without ha stack. non-clustered mount feature can run totally without ha stack.Can you please elaborate on this? Local mounts can run without a cluster stack so I don't see the difference there. We have
I am using pacemaker cluster stack. In my env, the trouble of the converting between local and clustered mounts are only happening on cluster stack.
the non-clustered mount feature (Gang He commit: 912f655d78c5) gave ocfs2 the ability to mount volume at any env (with/without cluster stack).
The 912f655d78c5 derived from SUSE customer complain: User wanted to fsck the backup ocfs2 volume in non-clustered env. They wanted to access the volume quickly and didn't want to take time/resource to set up HA stack. (by the way, pcmk stack at least needs two nodes to set up a cluster.)
Quoted 6 lines
tunefs.ocfs2 look for and join the cluster so as to avoid corrupting users data - that's a feature, not a bug. So what I'm seeing here is just opening us to potential corruptions. Is there a specific use case here that you're trying to account for? Are you fixing a particular bug?
Tunefs.ocfs2 still needs HA/dlm stack to protect joining action. commit 912f655d78c5 works on non-clustered env, which needs other tech (eg. MMP) to protect corrupting.
From my viewpoint, the non-clustered mount code is based on local mount code, which gives more flexible than local mount. non-clustered mount uses unify mount style align with clustered mount. I think users will like more to use non-clustered mount than using tunefs.ocfs2 to change mount type.
Thanks,
Heming
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 3/4] re-enable "ocfs2: mount shared volume without ha stack"
I found I didn't include Joseph in previous mail.
Quoted 19 lines · On 8/1/22 09:01, heming.zhao--- via Ocfs2-devel wrote:
Hello Mark,
Quoted 5 lines · Mark Fasheh
Hi Heming,
Quoted 1 line · On Fri, Jul 29, 2022 at 6:15 PM Heming Zhao via Ocfs2-devel…
Quoted 7 lines
Quoted 4 lines
the key different between local mount and non-clustered mount:
local mount feature (tunefs.ocfs2 --fs-features=[no]local) can't do convert job without ha stack. non-clustered mount feature can run totally without ha stack.Can you please elaborate on this? Local mounts can run without a cluster stack so I don't see the difference there. We have
I am using pacemaker cluster stack. In my env, the trouble of the converting between local and clustered mounts are only happening on cluster stack.
Quoted 6 lines
the non-clustered mount feature (Gang He commit:
912f655d78c5) gave ocfs2 the ability to mount volume at any env (with/without cluster stack).
The912f655d78c5derived from SUSE customer complain: User wanted to fsck the backup ocfs2 volume in non-clustered env. They wanted to access the volume quickly and didn't want to take time/resource to set up HA stack. (by the way, pcmk stack at least needs two nodes to set up a cluster.)
Ooh, above "needs two nodes to set up a cluster" is wrong. User could use a special corosync.conf to set up a single node cluster. But anyhow, the key is why we can't bypass the setup ha stack step.
Quoted 11 lines
Quoted 7 lines
tunefs.ocfs2 look for and join the cluster so as to avoid corrupting users data - that's a feature, not a bug. So what I'm seeing here is just opening us to potential corruptions. Is there a specific use case here that you're trying to account for? Are you fixing a particular bug?
Tunefs.ocfs2 still needs HA/dlm stack to protect joining action. commit
912f655d78c5works on non-clustered env, which needs other tech (eg. MMP) to protect corrupting.
Quoted 7 lines
From my viewpoint, the non-clustered mount code is based on local mount code,
which gives more flexible than local mount. non-clustered mount uses unify mount style align with clustered mount. I think users will like more to use non-clustered mount than using tunefs.ocfs2 to change mount type.Thanks,
Heming
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 3/4] re-enable "ocfs2: mount shared volume without ha stack"
In reply to heming.zhao--- via Ocfs2-develHi Heming,
Quoted 20 lines · On Sun, Jul 31, 2022 at 6:02 PM [email protected] <[email protected]>…
Hello Mark,
Quoted 5 lines · Mark Fasheh
Hi Heming,
Quoted 1 line · On Fri, Jul 29, 2022 at 6:15 PM Heming Zhao via Ocfs2-devel…
Quoted 7 lines
Quoted 4 lines
the key different between local mount and non-clustered mount:
local mount feature (tunefs.ocfs2 --fs-features=[no]local) can't do convert job without ha stack. non-clustered mount feature can run totally without ha stack.Can you please elaborate on this? Local mounts can run without a cluster stack so I don't see the difference there. We have
I am using pacemaker cluster stack. In my env, the trouble of the converting between local and clustered mounts are only happening on cluster stack.
Quoted 6 lines
the non-clustered mount feature (Gang He commit:
912f655d78c5) gave ocfs2 the ability to mount volume at any env (with/without cluster stack).
The912f655d78c5derived from SUSE customer complain: User wanted to fsck the backup ocfs2 volume in non-clustered env. They wanted to access the volume quickly and didn't want to take time/resource to set up HA stack. (by the way, pcmk stack at least needs two nodes to set up a cluster.)
Ok. I had some time to look over the ext4 mmp patches. I feel like there are two questions here:
1) Is MMP a useful feature for Ocfs2
My answer is yes absolutely, the user should have the option to enable this on local mounts.
2) Should we allow the user to bypass our cluster checks?
On this question I'm still a 'no'. I simply haven't seen enough evidence to warrant such a drastic change in policy. Allowing it via mount option too just feels extremely error-prone. I think we need to explore alternative avenues to help
ing the user out here. As you noted in your followup, a single node config is entirely possible in pacemaker (I've run that config myself). Why not provide an easy way for the user to drop down to that sort of a config? I know that's kind
of pushing responsibility for this to the cluster stack, but that's where it belongs in the first place.
Another option might be an 'observer mode' mount, where the node participates in the cluster (and the file system locking) but purely in a read-only fashion.
Quoted 9 lines
Quoted 6 lines
tunefs.ocfs2 look for and join the cluster so as to avoid corrupting users data - that's a feature, not a bug. So what I'm seeing here is just opening us to potential corruptions. Is there a specific use case here that you're trying to account for? Are you fixing a particular bug?
Tunefs.ocfs2 still needs HA/dlm stack to protect joining action. commit
912f655d78c5works on non-clustered env, which needs other tech (eg. MMP) to protect corrupting.
FWIW, I'm glad that we pulled commit 912f655d78c5 and I do not think we should go back to that paradigm.
Quoted 4 lines
From my viewpoint, the non-clustered mount code is based on local mount code,
which gives more flexible than local mount. non-clustered mount uses unify mount style align with clustered mount. I think users will like more to use non-clustered mount than using tunefs.ocfs2 to change mount type.
Can we get rid of the mount option, and make mmp something that users can turn on for Ocfs2 local mounts? I don't recall if we make a slot map for local mounts or not but it wouldn't be difficult to add that.
Btw, thank you very much for all of these patches, and also thank you for the very detailed test cases in your initial email. I'll try to give the actual code a review as well.
Thanks,
--Mark
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 3/4] re-enable "ocfs2: mount shared volume without ha stack"
Quoted 13 lines · Mark Fasheh
2) Should we allow the user to bypass our cluster checks?
On this question I'm still a 'no'. I simply haven't seen enough evidence to warrant such a drastic change in policy. Allowing it via mount option too just feels extremely error-prone. I think we need to explore alternative avenues to help
ing the user out here. As you noted in your followup, a single node config is entirely possible in pacemaker (I've run that config myself). Why not provide an easy way for the user to drop down to that sort of a config? I know that's kind
of pushing responsibility for this to the cluster stack, but that's where it belongs in the first place.
Quoted 3 lines
Another option might be an 'observer mode' mount, where the node participates in the cluster (and the file system locking) but purely in a read-only fashion.
Thinking about this some more... The only way that this works without potential corruptions is if we always write a periodic mmp sequence, even in clustered mode (which might mean each node writes to its own sector). That way tunefs can always check the disk for a mounted node, even without a cluster stack up. If tunefs sees anyone writing sequences to the disk, it can safely fail the operation. Tunefs also would have to be writing an mmp sequence once it has determined that the disk is not mounted. It could also write some flag alongisde the sequence that says 'tunefs is working on this disk'. If a cluster mount comes up and sees a live sequence with that flag, it will know to fail the mount request as the disk is being modified. Local mounts can also use this to ensure that they are the only mounted node.
As it turns out, we already do pretty much all of the sequence writing already for the o2cb cluster stack - check out cluseter/heartbeat.c. If memory serves, tunefs.ocfs2 has code to write to this heartbeat area as well. For o2cb, we use the disk heartbeat to detect node liveness, and to kill our local node if we see disk timeouts. For pcmk, we shouldn't take any of these actions as it is none of our responsibility. Under pcmk, the heartbeating would be purely for mount protection checks.
The downside to this is that all nodes would be heartbeating to the disk on a regular interval, not just one. To be fair, this is exactly how o2cb works and with the correct timeout choices, we were able to avoid a measurable performance impact, though in any case this might have to be a small price the user pays for cluster aware mount protection.
Let me know what you think.
Thanks,
--Mark
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 3/4] re-enable "ocfs2: mount shared volume without ha stack"
Hi Mark,
Quoted 18 lines · Mark Fasheh
Quoted 13 lines · Mark Fasheh
2) Should we allow the user to bypass our cluster checks?
On this question I'm still a 'no'. I simply haven't seen enough evidence to warrant such a drastic change in policy. Allowing it via mount option too just feels extremely error-prone. I think we need to explore alternative avenues to help
ing the user out here. As you noted in your followup, a single node config is entirely possible in pacemaker (I've run that config myself). Why not provide an easy way for the user to drop down to that sort of a config? I know that's kind
of pushing responsibility for this to the cluster stack, but that's where it belongs in the first place.Quoted 3 lines
Another option might be an 'observer mode' mount, where the node participates in the cluster (and the file system locking) but purely in a read-only fashion.
Quoted 13 lines
Thinking about this some more... The only way that this works without potential corruptions is if we always write a periodic mmp sequence, even in clustered mode (which might mean each node writes to its own sector). That way tunefs can always check the disk for a mounted node, even without a cluster stack up. If tunefs sees anyone writing sequences to the disk, it can safely fail the operation. Tunefs also would have to be writing an mmp sequence once it has determined that the disk is not mounted. It could also write some flag alongisde the sequence that says 'tunefs is working on this disk'. If a cluster mount comes up and sees a live sequence with that flag, it will know to fail the mount request as the disk is being modified. Local mounts can also use this to ensure that they are the only mounted node.
Above tunefs check & write seq steps are mmp feature work flow.
Your mentioned tunefs work flow matches my patch[4/4] idea, and this
patch does all the works in kernel ocfs2_find_slot().
Because sequences should be saved in SB, update it should grab ->osb_lock,
which influence the performance.
And for saving seqs, for saving space, we won't alloc different disk block for different node.
If multi-nodes share the same disk block (eg, keep using slot_map
for saving seqs),
the updating job will make IO performance issue.
For fixing performance issue, in my view, we should disable mmp sequences
updating when mounting mode is clustered. So I make a rule:
** If last mount didn't do unmount, (eg: crash), the next mount MUST be same mount type. **
above rule another meaning:
new coming node mounting type should same with exist mounting type.
and there is a basic knowledge:
current ocfs2 code under cluster stack already have the ability to prevent multiple mount when mounting type is clustered.
(from patch 4/4) there are mount labels:
+#define OCFS2_MMP_SEQ_CLEAN 0xFF4D4D50U /* mmp_seq value for clean unmount */
+#define OCFS2_MMP_SEQ_FSCK 0xE24D4D50U /* mmp_seq value when being fscked */
+#define OCFS2_MMP_SEQ_MAX 0xE24D4D4FU /* maximum valid mmp_seq value */
+#define OCFS2_MMP_SEQ_INIT 0x0 /* mmp_seq init value */
+#define OCFS2_VALID_CLUSTER 0xE24D4D55U /* value for clustered mount
+ under MMP disabled */
+#define OCFS2_VALID_NOCLUSTER 0xE24D4D5AU /* value for noclustered mount
+ under MMP disabled */
whenever mount successfully, there should be three types living labels
in slotmap area:
- OCFS2_MMP_SEQ_CLEAN, OCFS2_VALID_NOCLUSTER - for local/non-clustered mount
- OCFS2_VALID_CLUSTER - for clustered mount
new coming node will check if any slot contains living labels.
whenever unmount successfully, there should be two types left labels in slotmap
area:
-
OCFS2_MMP_SEQ_CLEAN or 0 (zero)
when a node does unmount, according to the mount type, it will clean (zeroed)
or write OCFS2_MMP_SEQ_CLEAN in the slot.
Quoted 9 lines
As it turns out, we already do pretty much all of the sequence writing already for the o2cb cluster stack - check out
cluseter/heartbeat.c. If memory serves, tunefs.ocfs2 has code to write to this heartbeat area as well. For o2cb, we use the disk heartbeat to detect node liveness, and to kill our local node if we see disk timeouts. For pcmk, we shouldn't take any of these actions as it is none of our responsibility. Under pcmk, the heartbeating would be purely for mount protection checks.
From my thinking, under cluster stack, there is enough protecting logic. For local/non-clustered mount, mmp will give the ability for detecting node liveness.
So I only enable kmmpd-<dev> kthread for local/non-clustered mount.
Quoted 6 lines
The downside to this is that all nodes would be heartbeating to the disk on a regular interval, not just one. To be fair, this is exactly how o2cb works and with the correct timeout choices, we were able to avoid a measurable performance impact, though in any case this might have to be a small price the user pays for cluster aware mount protection.
I already considered this performance issue in patch [4/4].
Quoted 5 lines
Let me know what you think.
Thanks,
--Mark
Thanks,
Heming
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 3/4] re-enable "ocfs2: mount shared volume without ha stack"
In reply to Mark Fasheh via Ocfs2-develHello Mark and Joseph,
please ignore my previous mess formated reply mail
Quoted 18 lines · Mark Fasheh
Quoted 13 lines · Mark Fasheh
2) Should we allow the user to bypass our cluster checks?
On this question I'm still a 'no'. I simply haven't seen enough evidence to warrant such a drastic change in policy. Allowing it via mount option too just feels extremely error-prone. I think we need to explore alternative avenues to help
ing the user out here. As you noted in your followup, a single node config is entirely possible in pacemaker (I've run that config myself). Why not provide an easy way for the user to drop down to that sort of a config? I know that's kind
of pushing responsibility for this to the cluster stack, but that's where it belongs in the first place.Quoted 3 lines
Another option might be an 'observer mode' mount, where the node participates in the cluster (and the file system locking) but purely in a read-only fashion.
Quoted 13 lines
Thinking about this some more... The only way that this works without potential corruptions is if we always write a periodic mmp sequence, even in clustered mode (which might mean each node writes to its own sector). That way tunefs can always check the disk for a mounted node, even without a cluster stack up. If tunefs sees anyone writing sequences to the disk, it can safely fail the operation. Tunefs also would have to be writing an mmp sequence once it has determined that the disk is not mounted. It could also write some flag alongisde the sequence that says 'tunefs is working on this disk'. If a cluster mount comes up and sees a live sequence with that flag, it will know to fail the mount request as the disk is being modified. Local mounts can also use this to ensure that they are the only mounted node.
Above tunefs check & write seq steps are mmp feature work flow.
Your mentioned tunefs work flow matches my patch[4/4] idea, and this patch does all the works in kernel ocfs2_find_slot().
Because sequences should be saved in SB, update it should grab ->osb_lock, which influence the performance.
And for saving seqs, for saving space, we won't alloc different disk block for different node. If multi-nodes share the same disk block (eg, keep using slot_map for saving seqs), the updating job will make IO performance issue.
For fixing performance issue, in my view, we should disable mmp sequences updating when mounting mode is clustered. So I make a rule:
** If last mount didn't do unmount, (eg: crash), the next mount MUST be same mount type. **
above rule another meaning:
new coming node mounting type should same with exist mounting type.
and there is a basic knowledge:
current ocfs2 code under cluster stack already have the ability to prevent multiple mount when mounting type is clustered.
(from patch 4/4) there are mount labels:
+#define OCFS2_MMP_SEQ_CLEAN 0xFF4D4D50U /* mmp_seq value for clean unmount */
+#define OCFS2_MMP_SEQ_FSCK 0xE24D4D50U /* mmp_seq value when being fscked */ +#define OCFS2_MMP_SEQ_MAX 0xE24D4D4FU /* maximum valid mmp_seq value */ +#define OCFS2_MMP_SEQ_INIT 0x0 /* mmp_seq init value */
+#define OCFS2_VALID_CLUSTER 0xE24D4D55U /* value for clustered mount
+ under MMP disabled */
+#define OCFS2_VALID_NOCLUSTER 0xE24D4D5AU /* value for noclustered mount
+ under MMP disabled */
whenever mount successfully, there should be three types living labels in slotmap area:
- OCFS2_MMP_SEQ_CLEAN, OCFS2_VALID_NOCLUSTER - for local/non-clustered mount
- OCFS2_VALID_CLUSTER - for clustered mount
new coming node will check if any slot contains living labels.
whenever unmount successfully, there should be two types left labels in slotmap area:
- OCFS2_MMP_SEQ_CLEAN or 0 (zero)
when a node does unmount, according to the mount type, it will clean (zeroed) or write OCFS2_MMP_SEQ_CLEAN in the slot.
Quoted 8 lines
As it turns out, we already do pretty much all of the sequence writing already for the o2cb cluster stack - check out
cluseter/heartbeat.c. If memory serves, tunefs.ocfs2 has code to write to this heartbeat area as well. For o2cb, we use the disk heartbeat to detect node liveness, and to kill our local node if we see disk timeouts. For pcmk, we shouldn't take any of these actions as it is none of our responsibility. Under pcmk, the heartbeating would be purely for mount protection checks.
From my thinking, under cluster stack, there is enough protecting logic.
For local/non-clustered mount, mmp will give the ability for detecting node liveness. So I only enable kmmpd-<dev> kthread for local/non-clustered mount.
Quoted 8 lines
The downside to this is that all nodes would be heartbeating to the disk on a regular interval, not just one. To be fair, this is exactly how o2cb works and with the correct timeout choices, we were able to avoid a measurable performance impact, though in any case this might have to be a small price the user pays for cluster aware mount protection.
I already considered this performance issue in patch [4/4].
Quoted 4 lines
Let me know what you think.
Thanks,
--Mark
Thanks,
Heming
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 3/4] re-enable "ocfs2: mount shared volume without ha stack"
In reply to Mark Fasheh via Ocfs2-develHi Mark,
Quoted 31 lines · Mark Fasheh
Hi Heming,
Quoted 20 lines · On Sun, Jul 31, 2022 at 6:02 PM [email protected] <[email protected]>…
Hello Mark,
Quoted 5 lines · Mark Fasheh
Hi Heming,
Quoted 1 line · On Fri, Jul 29, 2022 at 6:15 PM Heming Zhao via Ocfs2-devel…
Quoted 7 lines
Quoted 4 lines
the key different between local mount and non-clustered mount:
local mount feature (tunefs.ocfs2 --fs-features=[no]local) can't do convert job without ha stack. non-clustered mount feature can run totally without ha stack.Can you please elaborate on this? Local mounts can run without a cluster stack so I don't see the difference there. We have
I am using pacemaker cluster stack. In my env, the trouble of the converting between local and clustered mounts are only happening on cluster stack.
Quoted 6 lines
the non-clustered mount feature (Gang He commit:
912f655d78c5) gave ocfs2 the ability to mount volume at any env (with/without cluster stack).
The912f655d78c5derived from SUSE customer complain: User wanted to fsck the backup ocfs2 volume in non-clustered env. They wanted to access the volume quickly and didn't want to take time/resource to set up HA stack. (by the way, pcmk stack at least needs two nodes to set up a cluster.)
Quoted 8 lines
Ok. I had some time to look over the ext4 mmp patches. I feel like there are two questions here:
1) Is MMP a useful feature for Ocfs2
My answer is yes absolutely, the user should have the option to enable this on local mounts.
me too.
Quoted 14 lines
2) Should we allow the user to bypass our cluster checks?
On this question I'm still a 'no'. I simply haven't seen enough evidence to warrant such a drastic change in policy. Allowing it via mount option too just feels extremely error-prone. I think we need to explore alternative avenues to help
ing the user out here. As you noted in your followup, a single node config is entirely possible in pacemaker (I've run that config myself). Why not provide an easy way for the user to drop down to that sort of a config? I know that's kind
of pushing responsibility for this to the cluster stack, but that's where it belongs in the first place.
the reason for creating commit 912f655d78c5d4a is user didn't want to do any
set up job for cluster stack. So any HA solution (eg: automatically config
single node, auto install required software, create VM with already set up HA
stack, etc) is not remove the pain point. Base on bypass cluster setup, there only
left non-clustered mount. And in end user viewpoint, non-clustered mount is
also the most easy way to mount ocfs2 volume.
Quoted 4 lines
Another option might be an 'observer mode' mount, where the node participates in the cluster (and the file system locking) but purely in a read-only fashion.
In my feeling, the 'observer mode' is just read-only mount, we don't need to create it.
For 912f655d78c5d4a, user take a snapshot for ocfs2 volume, and do health-check
on a different env (not production env). there may belong to two different network.
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
tunefs.ocfs2 look for and join the cluster so as to avoid corrupting users data - that's a feature, not a bug. So what I'm seeing here is just opening us to potential corruptions. Is there a specific use case here that you're trying to account for? Are you fixing a particular bug?
Quoted 6 lines
Quoted 2 lines
Tunefs.ocfs2 still needs HA/dlm stack to protect joining action. commit
912f655d78c5works on non-clustered env, which needs other tech (eg. MMP) to protect corrupting.FWIW, I'm glad that we pulled commit
912f655d78c5and I do not think we should go back to that paradigm.
If you or other maintainers prefer to pull out non-clustered mount, I respect
the decision.
Quoted 10 lines
Quoted 5 lines
From my viewpoint, the non-clustered mount code is based on local mount code,
which gives more flexible than local mount. non-clustered mount uses unify mount style align with clustered mount. I think users will like more to use non-clustered mount than using tunefs.ocfs2 to change mount type.Can we get rid of the mount option, and make mmp something that users can turn on for Ocfs2 local mounts? I don't recall if we make a slot map for local mounts or not but it wouldn't be difficult to add that.
From tech side, it's possible to enable mmp in local mount.
Without 912f655d78c5d4a, local-mount uses slot 0 (the first available slot).
Quoted 6 lines
Btw, thank you very much for all of these patches, and also thank you for the very detailed test cases in your initial email. I'll try to give the actual code a review as well.
Thanks,
--Mark
Thanks,
Heming
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 3/4] re-enable "ocfs2: mount shared volume without ha stack"
In reply to Mark Fasheh via Ocfs2-develHello Mark and Joseph,
(please ignore my previous reply mail)
I may met thunderbird bug, I clearly remembered I added you (Joseph) in CC for this mail & previous mail. But I only see Mark and ocfs2-devel in the receiver list. and the mail format also mess, thunderbird replace '\n' to '\r\n'.
For fixing this problem, I change neomutt to resend these two mails
Quoted 31 lines · Mark Fasheh
Hi Heming,
Quoted 20 lines · On Sun, Jul 31, 2022 at 6:02 PM [email protected] <[email protected]>…
Hello Mark,
Quoted 5 lines · Mark Fasheh
Hi Heming,
Quoted 1 line · On Fri, Jul 29, 2022 at 6:15 PM Heming Zhao via Ocfs2-devel…
Quoted 7 lines
Quoted 4 lines
the key different between local mount and non-clustered mount:
local mount feature (tunefs.ocfs2 --fs-features=[no]local) can't do convert job without ha stack. non-clustered mount feature can run totally without ha stack.Can you please elaborate on this? Local mounts can run without a cluster stack so I don't see the difference there. We have
I am using pacemaker cluster stack. In my env, the trouble of the converting between local and clustered mounts are only happening on cluster stack.
Quoted 6 lines
the non-clustered mount feature (Gang He commit:
912f655d78c5) gave ocfs2 the ability to mount volume at any env (with/without cluster stack).
The912f655d78c5derived from SUSE customer complain: User wanted to fsck the backup ocfs2 volume in non-clustered env. They wanted to access the volume quickly and didn't want to take time/resource to set up HA stack. (by the way, pcmk stack at least needs two nodes to set up a cluster.)
Quoted 8 lines
Ok. I had some time to look over the ext4 mmp patches. I feel like there are two questions here:
1) Is MMP a useful feature for Ocfs2
My answer is yes absolutely, the user should have the option to enable this on local mounts.
me too.
Quoted 14 lines
2) Should we allow the user to bypass our cluster checks?
On this question I'm still a 'no'. I simply haven't seen enough evidence to warrant such a drastic change in policy. Allowing it via mount option too just feels extremely error-prone. I think we need to explore alternative avenues to help
ing the user out here. As you noted in your followup, a single node config is entirely possible in pacemaker (I've run that config myself). Why not provide an easy way for the user to drop down to that sort of a config? I know that's kind
of pushing responsibility for this to the cluster stack, but that's where it belongs in the first place.
the reason for creating commit 912f655d78c5d4a is user didn't want to do any set up job for cluster stack. So any HA solution (eg: automatically config single node, auto install required software, create VM with already set up HA stack, etc) is not remove the pain point. Base on bypass cluster setup, there only left non-clustered mount. And in end user viewpoint, non-clustered mount is also the most easy way to mount ocfs2 volume.
Quoted 4 lines
Another option might be an 'observer mode' mount, where the node participates in the cluster (and the file system locking) but purely in a read-only fashion.
In my feeling, the 'observer mode' is just read-only mount, we don't need to create it.
For 912f655d78c5d4a, user take a snapshot for ocfs2 volume, and do health-check
on a different env (not production env). there may belong to two different network.
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
tunefs.ocfs2 look for and join the cluster so as to avoid corrupting users data - that's a feature, not a bug. So what I'm seeing here is just opening us to potential corruptions. Is there a specific use case here that you're trying to account for? Are you fixing a particular bug?
Quoted 5 lines
Quoted 2 lines
Tunefs.ocfs2 still needs HA/dlm stack to protect joining action. commit
912f655d78c5works on non-clustered env, which needs other tech (eg. MMP) to protect corrupting.FWIW, I'm glad that we pulled commit
912f655d78c5and I do not think we should go back to that paradigm.
If you or other maintainers prefer to pull out non-clustered mount, I respect the decision.
Quoted 11 lines
Quoted 6 lines
From my viewpoint, the non-clustered mount code is based on local mount code,
which gives more flexible than local mount. non-clustered mount uses unify mount style align with clustered mount. I think users will like more to use non-clustered mount than using tunefs.ocfs2 to change mount type.Can we get rid of the mount option, and make mmp something that users can turn on for Ocfs2 local mounts? I don't recall if we make a slot map for local mounts or not but it wouldn't be difficult to add that.
From tech side, it's possible to enable mmp in local mount.
Without 912f655d78c5d4a, local-mount uses slot 0 (the first available slot).
Quoted 6 lines
Btw, thank you very much for all of these patches, and also thank you for the very detailed test cases in your initial email. I'll try to give the actual code a review as well.
Thanks,
--Mark
Thanks,
Heming
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
PATCH 4/4 ocfs2: introduce ext4 MMP feature
In reply to Heming Zhao via Ocfs2-develMMP (multiple mount protection) gives filesystem ability to prevent from being mounted multiple times.
For avoiding data corruption when non-clustered and/or clustered mount are happening at same time, this commit introduced MMP feature. MMP idea is from ext4 MMP (fs/ext4/mmp.c) code. For ocfs2 is a clustered fs and also for compatible with existing slotmap feature, I did some optimization and modification when porting from ext4 to ocfs2.
For optimization:
mmp has a kthread kmmpd-<dev>, which is only created in non-clustered mode.
We set a rule:
If last mount didn't do unmount, (eg: crash), the next mount MUST be same mount type.
At last, this commit also fix commit c80af0c250c8 ("Revert "ocfs2:
mount shared volume without ha stack") mentioned issue.
--- fs/ocfs2/ocfs2.h | 2 + fs/ocfs2/ocfs2_fs.h | 13 +- fs/ocfs2/slot_map.c | 459 ++++++++++++++++++++++++++++++++++++++++++-- fs/ocfs2/slot_map.h | 3 + fs/ocfs2/super.c | 23 ++- 5 files changed, 479 insertions(+), 21 deletions(-)
fs/ocfs2/ocfs2.h +2 -0
@@ -337,6 +337,8 @@ struct ocfs2_superunsigned int node_num;int slot_num;int preferred_slot; +u16 mmp_update_interval; +struct task_struct *mmp_task;int s_sectsize_bits;int s_clustersize;int s_clustersize_bits;
fs/ocfs2/ocfs2_fs.h +9 -4
@@ -87,7 +87,8 @@ | OCFS2_FEATURE_INCOMPAT_REFCOUNT_TREE \ | OCFS2_FEATURE_INCOMPAT_DISCONTIG_BG \ | OCFS2_FEATURE_INCOMPAT_CLUSTERINFO \ - | OCFS2_FEATURE_INCOMPAT_APPEND_DIO) + | OCFS2_FEATURE_INCOMPAT_APPEND_DIO \ + | OCFS2_FEATURE_INCOMPAT_MMP)#define OCFS2_FEATURE_RO_COMPAT_SUPP (OCFS2_FEATURE_RO_COMPAT_UNWRITTEN \ | OCFS2_FEATURE_RO_COMPAT_USRQUOTA \ | OCFS2_FEATURE_RO_COMPAT_GRPQUOTA)
@@ -167,6 +168,11 @@*/ #define OCFS2_FEATURE_INCOMPAT_APPEND_DIO 0x8000 +/* +* Multiple mount protection +*/ +#define OCFS2_FEATURE_INCOMPAT_MMP 0x10000 +/* * backup superblock flag is used to indicate that this volume * has backup superblocks.
@@ -535,8 +541,7 @@ struct ocfs2_slot_map { };struct ocfs2_extended_slot { -/*00*/ __u8 es_valid; -__u8 es_reserved1[3 ]; +/*00*/ __le32 es_valid;__le32 es_node_num;/*08*/ };
@@ -611,7 +616,7 @@ struct ocfs2_super_block {INCOMPAT flag set. */ /*B8*/ __le16 s_xattr_inline_size;/* extended attribute inline size for this fs*/ -__le16 s_reserved0; +__le16 s_mmp_update_interval;/* # seconds to wait in MMP checking */ __le32 s_dx_seed[3 ];/* seed[0-2] for dx dir hash. * s_uuid_hash serves as seed[3]. */ /*C0*/ __le64 s_reserved2[15 ];/* Fill out superblock */
fs/ocfs2/slot_map.c +444 -15
540 lines not on this page · raw message
fs/ocfs2/slot_map.h +3 -0
@@ -25,4 +25,7 @@ int ocfs2_slot_to_node_num_locked(struct ocfs2_super *osb, int slot_num,int ocfs2_clear_slot (struct ocfs2_super *osb,int slot_num); +int ocfs2_multi_mount_protect (struct ocfs2_super *osb,int noclustered); +void ocfs2_stop_mmpd (struct ocfs2_super *osb); +#endif
fs/ocfs2/super.c +21 -2
@@ -609,6 +609,7 @@ static int ocfs2_remount(struct super_block *sb, int *flags, char *data)struct mount_options parsed_options;struct ocfs2_super *osb =OCFS2_SB (sb);u32 tmp; +int noclustered;sync_filesystem (sb);
@@ -619,7 +620,8 @@ static int ocfs2_remount(struct super_block *sb, int *flags, char *data) } tmp =OCFS2_MOUNT_NOCLUSTER ; -if ((osb->s_mount_opt & tmp) != (parsed_options.mount_opt & tmp)) { + noclustered = osb->s_mount_opt & tmp; +if (noclustered != (parsed_options.mount_opt & tmp)) { ret = -EINVAL ;mlog (ML_ERROR ,"Cannot change nocluster option on remount\n" ); goto out;
@@ -686,10 +688,20 @@ static int ocfs2_remount(struct super_block *sb, int *flags, char *data) } sb->s_flags &= ~SB_RDONLY ; osb->osb_flags &= ~OCFS2_OSB_SOFT_RO ; +if (OCFS2_HAS_INCOMPAT_FEATURE (sb,OCFS2_FEATURE_INCOMPAT_MMP )) { +spin_unlock (&osb->osb_lock); +if (ocfs2_multi_mount_protect (osb, noclustered)) { +mlog (ML_ERROR ,"started MMP failed.\n" ); +ocfs2_stop_mmpd (osb); + ret = -EROFS ; + goto unlocked_osb; + } + } }trace_ocfs2_remount (sb->s_flags, osb->osb_flags, *flags); unlock_osb:spin_unlock (&osb->osb_lock); +unlocked_osb:/* Enable quota accounting after remounting RW */ if (!ret && !(*flags &SB_RDONLY )) {if (sb_any_quota_suspended (sb))
@@ -722,6 +734,8 @@ static int ocfs2_remount(struct super_block *sb, int *flags, char *data) sb->s_flags = (sb->s_flags & ~SB_POSIXACL ) | ((osb->s_mount_opt &OCFS2_MOUNT_POSIX_ACL ) ?SB_POSIXACL :0 ); +if (sb_rdonly (osb->sb)) +ocfs2_stop_mmpd (osb); } out:return ret;
@@ -1833,7 +1847,7 @@ static int ocfs2_mount_volume(struct super_block *sb) status =ocfs2_init_local_system_inodes (osb);if (status <0 ) {mlog_errno (status); - goto out_super_lock; + goto out_find_slot; } status =ocfs2_check_volume (osb);
@@ -1858,6 +1872,8 @@ static int ocfs2_mount_volume(struct super_block *sb)/* before journal shutdown, we should release slot_info */ ocfs2_free_slot_info (osb);ocfs2_journal_shutdown (osb); +out_find_slot: +ocfs2_stop_mmpd (osb); out_super_lock:ocfs2_super_unlock (osb,1 ); out_dlm:
@@ -1878,6 +1894,8 @@ static void ocfs2_dismount_volume(struct super_block *sb, int mnt_err) osb =OCFS2_SB (sb);BUG_ON (!osb); +ocfs2_stop_mmpd (osb); +/* Remove file check sysfs related directores/files, * and wait for the pending file check operations */ ocfs2_filecheck_remove_sysfs (osb);
@@ -2086,6 +2104,7 @@ static int ocfs2_initialize_super(struct super_block *sb,snprintf (osb->dev_str,sizeof (osb->dev_str),"%u,%u" ,MAJOR (osb->sb->s_dev),MINOR (osb->sb->s_dev)); + osb->mmp_update_interval =le16_to_cpu (di->id2.i_super.s_mmp_update_interval); osb->max_slots =le16_to_cpu (di->id2.i_super.s_max_slots);if (osb->max_slots >OCFS2_MAX_SLOTS || osb->max_slots ==0 ) { mlog(ML_ERROR, "Invalid number of node slots (%u)\n",
Signature · 7 lines
2.37.1 _______________________________________________ Ocfs2-devel mailing list [email protected] https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 4/4] ocfs2: introduce ext4 MMP feature
Quoted 49 lines · Heming Zhao
MMP (multiple mount protection) gives filesystem ability to prevent from being mounted multiple times.
For avoiding data corruption when non-clustered and/or clustered mount are happening at same time, this commit introduced MMP feature. MMP idea is from ext4 MMP (
fs/ext4/mmp.c) code. For ocfs2 is a clustered fs and also for compatible with existing slotmap feature, I did some optimization and modification when porting from ext4 to ocfs2.For optimization:
mmp has a kthread kmmpd-<dev>, which is only created in non-clustered mode.We set a rule:
If last mount didn't do unmount, (eg: crash), the next mount MUST be same mount type.At last, this commit also fix commit
c80af0c250c8("Revert "ocfs2:
mount shared volume without ha stack") mentioned issue.Signed-off-by Heming Zhao <[email protected]>--- fs/ocfs2/ocfs2.h | 2 + fs/ocfs2/ocfs2_fs.h | 13 +- fs/ocfs2/slot_map.c | 459 ++++++++++++++++++++++++++++++++++++++++++-- fs/ocfs2/slot_map.h | 3 + fs/ocfs2/super.c | 23 ++- 5 files changed, 479 insertions(+), 21 deletions(-)... ...
/*C0*/__le64s_reserved2[15]; /* Fill out superblock */fs/ocfs2/slot_map.c +14
-0+void ocfs2_stop_mmpd (struct ocfs2_super *osb) +{ +if (osb->mmp_task) { +kthread_stop (osb->mmp_task); + osb->mmp_task =NULL ; + } +} + +/* +* Protect the filesystem from being mounted more than once. +* +* This function was inspired by ext4 MMP feature. Because HA stack +* helps ocfs2 to manage nodes join/leave, so we only focus on MMP +* under nocluster mode.
fs/ocfs2/slot_map.c void ocfs2_free_slot_info(struct ocfs2_super *osb) · line 682
Quoted 10 lines
fs/ocfs2/slot_map.c +10
-0+* Another info is ocfs2 only uses slot 0 on nocuster mode. +* +* es_valid: +* 0: not available +* 1: valid, cluster mode +* 2: valid, nocluster mod> + * +* parameters: +* osb: the struct ocfs2_super +* noclustered: under noclustered mount +* slot: prefer slot number
Sorry, above es_valid & parameters comments are outdated. I will remove them on next version.
Quoted 2 lines
+ */
+intocfs2_multi_mount_protect(struct ocfs2_super *osb, int noclustered)
I forgot to run checkpatch.pl for all 4 patches. And checkpatch.pl reported many format warning. I prefer these patches as draft version, let's discuss the mmp+noncluster idea is workable/acceptable. If no one reject these code, I will file v1.
(For test cases, I put them in cover letter.)
Thanks,
Heming
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 4/4] ocfs2: introduce ext4 MMP feature
In reply to Heming Zhao via Ocfs2-develQuoted 13 lines · Heming Zhao
MMP (multiple mount protection) gives filesystem ability to prevent from being mounted multiple times.
For avoiding data corruption when non-clustered and/or clustered mount are happening at same time, this commit introduced MMP feature. MMP idea is from ext4 MMP (
fs/ext4/mmp.c) code. For ocfs2 is a clustered fs and also for compatible with existing slotmap feature, I did some optimization and modification when porting from ext4 to ocfs2.For optimization:
mmp has a kthread kmmpd-<dev>, which is only created in non-clustered mode.
Quoted 6 lines
We set a rule:
If last mount didn't do unmount, (eg: crash), the next mount MUST be same mount type.At last, this commit also fix commit
c80af0c250c8("Revert "ocfs2:
mount shared volume without ha stack") mentioned issue.
I suggest we re-split this series (especially patch 3 and 4), but not revive a buggy commit first and then another commit fixing it BTW.
Thanks,
Joseph
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 4/4] ocfs2: introduce ext4 MMP feature
Quoted 20 lines · Joseph Qi
Quoted 17 lines · Heming Zhao
MMP (multiple mount protection) gives filesystem ability to prevent from being mounted multiple times.
For avoiding data corruption when non-clustered and/or clustered mount are happening at same time, this commit introduced MMP feature. MMP idea is from ext4 MMP (
fs/ext4/mmp.c) code. For ocfs2 is a clustered fs and also for compatible with existing slotmap feature, I did some optimization and modification when porting from ext4 to ocfs2.For optimization:
mmp has a kthread kmmpd-<dev>, which is only created in non-clustered mode.We set a rule:
If last mount didn't do unmount, (eg: crash), the next mount MUST be same mount type.
Quoted 6 lines
Quoted 2 lines
At last, this commit also fix commit
c80af0c250c8("Revert "ocfs2:
mount shared volume without ha stack") mentioned issue.I suggest we re-split this series (especially patch 3 and 4), but not revive a buggy commit first and then another commit fixing it BTW.
No problem. I remembered this fixing style, and won't make this kind of mistake again.
Do we need to wait for Mark's further comment?
It looks Mark didn't like non-clustered mount at his first reply, but he showed another idea for non-clustered mount on the second reply. And I am sure what's his meaning/attitude for non-clustered mount.
Thanks,
Heming
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 4/4] ocfs2: introduce ext4 MMP feature
Very weird problem. I added Junxiao & Mark in 'CC' field, but there is only kept 'To' field. I tested to use CC to send mail for other mail address, it worked well.
Quoted 36 lines · On Mon, Aug 08, 2022 at 05:07:14PM +0800, Heming Zhao via Ocfs2-devel wrote:
Quoted 20 lines · Joseph Qi
Quoted 17 lines · Heming Zhao
MMP (multiple mount protection) gives filesystem ability to prevent from being mounted multiple times.
For avoiding data corruption when non-clustered and/or clustered mount are happening at same time, this commit introduced MMP feature. MMP idea is from ext4 MMP (
fs/ext4/mmp.c) code. For ocfs2 is a clustered fs and also for compatible with existing slotmap feature, I did some optimization and modification when porting from ext4 to ocfs2.For optimization:
mmp has a kthread kmmpd-<dev>, which is only created in non-clustered mode.We set a rule:
If last mount didn't do unmount, (eg: crash), the next mount MUST be same mount type.Quoted 6 lines
Quoted 2 lines
At last, this commit also fix commit
c80af0c250c8("Revert "ocfs2:
mount shared volume without ha stack") mentioned issue.I suggest we re-split this series (especially patch 3 and 4), but not revive a buggy commit first and then another commit fixing it BTW.
No problem. I remembered this fixing style, and won't make this kind of mistake again.
Do we need to wait for Mark's further comment?
It looks Mark didn't like non-clustered mount at his first reply, but he showed another idea for non-clustered mount on the second reply. And I am sure what's his meaning/attitude for non-clustered mount.
Quoted 7 lines
Thanks,
Heming_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
Re: [Ocfs2-devel] [PATCH 4/4] ocfs2: introduce ext4 MMP feature
In reply to Heming Zhao via Ocfs2-develQuoted 28 lines · Heming Zhao
Quoted 20 lines · Joseph Qi
Quoted 17 lines · Heming Zhao
MMP (multiple mount protection) gives filesystem ability to prevent from being mounted multiple times.
For avoiding data corruption when non-clustered and/or clustered mount are happening at same time, this commit introduced MMP feature. MMP idea is from ext4 MMP (
fs/ext4/mmp.c) code. For ocfs2 is a clustered fs and also for compatible with existing slotmap feature, I did some optimization and modification when porting from ext4 to ocfs2.For optimization:
mmp has a kthread kmmpd-<dev>, which is only created in non-clustered mode.We set a rule:
If last mount didn't do unmount, (eg: crash), the next mount MUST be same mount type.Quoted 6 lines
Quoted 2 lines
At last, this commit also fix commit
c80af0c250c8("Revert "ocfs2:
mount shared volume without ha stack") mentioned issue.I suggest we re-split this series (especially patch 3 and 4), but not revive a buggy commit first and then another commit fixing it BTW.
Quoted 8 lines
No problem. I remembered this fixing style, and won't make this kind of mistake again.
Do we need to wait for Mark's further comment?
It looks Mark didn't like non-clustered mount at his first reply, but he showed another idea for non-clustered mount on the second reply. And I am sure what's his meaning/attitude for non-clustered mount.
Yes, let's take Mark's input first for the next version.
Thanks,
Joseph
_______________________________________________
Ocfs2-devel mailing list
[email protected]
https://oss.oracle.com/mailman/listinfo/ocfs2-devel