Skip to contentSkip to conversation

nocontrold: Eliminating ocfs2_controld

PATCH New Goldwyn Rodrigues 3 participants 6 messages Sep 6, 2013 – Sep 26, 2013 OCFS2

lore
Share & tools
lore
#0 lore raw

cover nocontrold: Eliminating ocfs2_controld

Hi,

I am re-sending this patch series because I did not get a response for the previous set.

This is an effort of removing ocfs2_controld.pcmk and getting ocfs2 DLM handling up to the times with respect to DLM (>=4.0.1) and corosync (2.3.x). AFAIK, cman also is being phased out for a unified corosync cluster stack.

fs/dlm performs all the functions with respect to fencing and node management and provides the API's to do so for ocfs2. For all future references, DLM stands for fs/dlm code.

The advantages are:
+ No need to run an additional userspace daemon (ocfs2_controld)
+ No contrrold devince handling and controld protocol
+ Shifting responsibilities of node management to DLM layer
+ Huge reduction in source code, both in kernel and userspace

This feature requires modification in the userspace ocfs2-tools. The changes can be found at:
https://github.com/goldwynr/ocfs2-tools branch: nocontrold Currently, not many checks are present in the userspace code, but that would change soon.

These changes were developed on linux-stable 3.10.y, though they are applicable at the current upstream as well. If you want to give the entire kernel a spin, the link is:

https://github.com/goldwynr/linux-stable branch: nocontrold

Review comments/suggestions/criticism welcome.

Signature · 1 line
Goldwyn
#1 lore raw

cover nocontrold: Eliminating ocfs2_controld

On 2013-09-05T22:26:56, Goldwyn Rodrigues <[email protected]> wrote:

Hi Goldwyn,

thanks! This looks really good.

Quoted 4 lines

This is an effort of removing ocfs2_controld.pcmk and getting ocfs2 DLM handling up to the times with respect to DLM (>=4.0.1) and corosync (2.3.x). AFAIK, cman also is being phased out for a unified corosync cluster stack.

That's clearly necessary, also to bring OCFS2 more uptodate with the latest happenings in the GFS2 world; it'll allow both file systems to share exactly the same cluster stack.

Quoted 3 lines

https://github.com/goldwynr/ocfs2-tools branch: nocontrold Currently, not many checks are present in the userspace code, but that would change soon.

There's one question I have; how will this handle

- the "old" user-space code starting on a new kernel,
- or the "new" user-space code being run on an old kernel?

Is there anything we can do to at least provide a meaningful error message in the first case? The second should be easier to handle.

Regards,
Lars

Signature · 3 lines
Architect Storage/HA
SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imend?rffer, HRB 21284 (AG N?rnberg)
"Experience is the name everyone gives to their mistakes." -- Oscar Wilde
#2 lore raw

cover nocontrold: Eliminating ocfs2_controld

Hi Lars,

Quoted 15 lines · Lars Marowsky-Bree

On 2013-09-05T22:26:56, Goldwyn Rodrigues <[email protected]> wrote:

Hi Goldwyn,

thanks! This looks really good.

Quoted 4 lines

This is an effort of removing ocfs2_controld.pcmk and getting ocfs2 DLM handling up to the times with respect to DLM (>=4.0.1) and corosync (2.3.x). AFAIK, cman also is being phased out for a unified corosync cluster stack.

That's clearly necessary, also to bring OCFS2 more uptodate with the latest happenings in the GFS2 world; it'll allow both file systems to share exactly the same cluster stack.

Quoted 7 lines
Quoted 3 lines

https://github.com/goldwynr/ocfs2-tools branch: nocontrold Currently, not many checks are present in the userspace code, but that would change soon.

There's one question I have; how will this handle

- the "old" user-space code starting on a new kernel,

The ocfs2_controld.pcmk will refuse to start because of absence of the control device created by the kernel. Of course, this would deny mounts as well.

Quoted 1 line

- or the "new" user-space code being run on an old kernel?

The kernel code will fail citing the reason: The userspace daemon is not present.
The userspace complains (ESRCH):
mount.ocfs2: No such process while mounting /dev/sdc1 on /mnt. Check 'dmesg' for more information on this error.

Quoted 3 lines

Is there anything we can do to at least provide a meaningful error message in the first case? The second should be easier to handle.

Yes, we can capture the error code and ask the user to upgrade in the second case. However, for the first case mount.ocfs2 would give a cluster connect failure because ocfs2_controld is not present.

On a different note, we should consider increasing the kernel module version shown in dmesg to be in sync with the userspace tools and/or possibly increase the version number of both tools and kernel module.

Signature · 1 line
Goldwyn
#3 lore raw

cover nocontrold: Eliminating ocfs2_controld

Quoted 22 lines · Goldwyn Rodrigues

Hi Lars,

Quoted 19 lines · Lars Marowsky-Bree

On 2013-09-05T22:26:56, Goldwyn Rodrigues <[email protected]> wrote:

Hi Goldwyn,

thanks! This looks really good.

Quoted 4 lines

This is an effort of removing ocfs2_controld.pcmk and getting ocfs2 DLM handling up to the times with respect to DLM (>=4.0.1) and corosync (2.3.x). AFAIK, cman also is being phased out for a unified corosync cluster stack.

That's clearly necessary, also to bring OCFS2 more uptodate with the latest happenings in the GFS2 world; it'll allow both file systems to share exactly the same cluster stack.

Quoted 3 lines

https://github.com/goldwynr/ocfs2-tools branch: nocontrold Currently, not many checks are present in the userspace code, but that would change soon.

Quoted 7 lines
Quoted 3 lines

There's one question I have; how will this handle

- the "old" user-space code starting on a new kernel,

The ocfs2_controld.pcmk will refuse to start because of absence of the control device created by the kernel. Of course, this would deny mounts as well.

Do we know how the GFS2 project handled this case? It's going to be a major problem for people if a kernel update horks their cluster fs.

Quoted 10 lines
Quoted 2 lines

Is there anything we can do to at least provide a meaningful error message in the first case? The second should be easier to handle.

Yes, we can capture the error code and ask the user to upgrade in the second case. However, for the first case mount.ocfs2 would give a cluster connect failure because ocfs2_controld is not present.

On a different note, we should consider increasing the kernel module version shown in dmesg to be in sync with the userspace tools and/or possibly increase the version number of both tools and kernel module.

That shouldn't be a problem, the numbers are mostly there for us Ocfs2 devs.

	--Mark

--
Mark Fasheh

#4 lore raw

cover nocontrold: Eliminating ocfs2_controld

Hi Mark,

Thanks for the review so far. I was on vacation and could not get back to this earlier.

Quoted 25 lines · Mark Fasheh
Quoted 24 lines · Goldwyn Rodrigues

Hi Lars,

Quoted 21 lines · Lars Marowsky-Bree

On 2013-09-05T22:26:56, Goldwyn Rodrigues <[email protected]> wrote:

Hi Goldwyn,

thanks! This looks really good.

Quoted 4 lines

This is an effort of removing ocfs2_controld.pcmk and getting ocfs2 DLM handling up to the times with respect to DLM (>=4.0.1) and corosync (2.3.x). AFAIK, cman also is being phased out for a unified corosync cluster stack.

That's clearly necessary, also to bring OCFS2 more uptodate with the latest happenings in the GFS2 world; it'll allow both file systems to share exactly the same cluster stack.

Quoted 3 lines

https://github.com/goldwynr/ocfs2-tools branch: nocontrold Currently, not many checks are present in the userspace code, but that would change soon.

There's one question I have; how will this handle

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

- the "old" user-space code starting on a new kernel,

The ocfs2_controld.pcmk will refuse to start because of absence of the control device created by the kernel. Of course, this would deny mounts as well.

Do we know how the GFS2 project handled this case? It's going to be a major problem for people if a kernel update horks their cluster fs.

Okay, I have managed to work on this and can mount filesystems which are used with ocfs2_controld. So, we have backward compatibility.

The only downside is we will not have a code reduction :( I will post the patches soon for review.

Quoted 9 lines
Quoted 9 lines
Quoted 4 lines

Is there anything we can do to at least provide a meaningful error message in the first case? The second should be easier to handle.

Yes, we can capture the error code and ask the user to upgrade in the second case. However, for the first case mount.ocfs2 would give a cluster connect failure because ocfs2_controld is not present.

Quoted 5 lines
Quoted 3 lines

On a different note, we should consider increasing the kernel module version shown in dmesg to be in sync with the userspace tools and/or possibly increase the version number of both tools and kernel module.

That shouldn't be a problem, the numbers are mostly there for us Ocfs2 devs.

Understood. I hope the devs at Oracle does too :) especially from the tools POV.

Signature · 1 line
Goldwyn
#5 lore raw

cover nocontrold: Eliminating ocfs2_controld

In reply to Goldwyn Rodrigues

Firstly, thanks for developing this Goldwyn. I've been looking at the patches, and plan to review the series (in kernel) for you.

Quick question - do you have a pointer handy to the development stream on the dlm / gfs side of things? It would be instructive to see how it was done in more than one place.

	--Mark
Quoted 26 lines · Goldwyn Rodrigues

Hi,

I am re-sending this patch series because I did not get a response for the previous set.

This is an effort of removing ocfs2_controld.pcmk and getting ocfs2 DLM handling up to the times with respect to DLM (>=4.0.1) and corosync (2.3.x). AFAIK, cman also is being phased out for a unified corosync cluster stack.

fs/dlm performs all the functions with respect to fencing and node management and provides the API's to do so for ocfs2. For all future references, DLM stands for fs/dlm code.

The advantages are:
+ No need to run an additional userspace daemon (ocfs2_controld)
+ No contrrold devince handling and controld protocol
+ Shifting responsibilities of node management to DLM layer
+ Huge reduction in source code, both in kernel and userspace

This feature requires modification in the userspace ocfs2-tools. The changes can be found at:
https://github.com/goldwynr/ocfs2-tools branch: nocontrold Currently, not many checks are present in the userspace code, but that would change soon.

Quoted 10 lines

These changes were developed on linux-stable 3.10.y, though they are applicable at the current upstream as well. If you want to give the entire kernel a spin, the link is:

https://github.com/goldwynr/linux-stable branch: nocontrold

Review comments/suggestions/criticism welcome.

Signature · 1 line
Goldwyn

--
Mark Fasheh