What does rpc.mountd dlopen() libnfsjunct.so rather than libnfsjunct.so.0
Subsystems (2)
See $SUBJ
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
Both Fedora and openSUSE get upset about packaging a libFOO.so in a non "-devel" package and so trip over this library which clearly needs to be installed even if you aren't doing 'devel'opment.
I would like to change mountd as per the patch below to use the ".0" file. I believe this will not break any installation as the ".so" is installed as a symlink to the ".0" (or maybe ".0.0.0").
Would this be acceptable?
There is a bit of a discussion about this here:
https://bugzilla.redhat.com/show_bug.cgi?id=889174
but I either don't understand it or don't agree with it.
Thanks,
NeilBrown
utils/mountd/cache.c +1 -1
@@ -1139,7 +1139,7 @@ static struct exportent *lookup_junction(char *dom, const char *pathname,struct link_map *map;void *handle; - handle =dlopen ("libnfsjunct.so" ,RTLD_NOW ); + handle =dlopen ("libnfsjunct.so.0" ,RTLD_NOW );if (handle ==NULL ) {xlog (D_GENERAL ,"%s: dlopen: %s" , __func__,dlerror ());return NULL ;
Quoted 12 lines · NeilBrown
See $SUBJ
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
There is an API version field that mountd can examine before using the plug-in. .so symlinks to whichever library version is the blessed one.
Quoted 4 lines
Both Fedora and openSUSE get upset about packaging a libFOO.so in a non "-devel" package and so trip over this library which clearly needs to be installed even if you aren't doing 'devel'opment.
I've seen the rpmlint warning for some time. Never been sure exactly what to do about it, or whether it matters.
Quoted 6 lines
I would like to change mountd as per the patch below to use the ".0" file. I believe this will not break any installation as the ".so" is installed as a symlink to the ".0" (or maybe ".0.0.0").
Would this be acceptable?
That dlopen(3) should follow whatever packaging guidelines are set out by distributors. Do what you guys agree is correct.
Quoted 23 lines
There is a bit of a discussion about this here:
https://bugzilla.redhat.com/show_bug.cgi?id=889174but I either don't understand it or don't agree with it.
Thanks,
NeilBrownutils/mountd/cache.c +1
-1@@ -1139,7 +1139,7 @@ static struct exportent *lookup_junction(char *dom, const char *pathname,struct link_map *map;void *handle; - handle =dlopen ("libnfsjunct.so" ,RTLD_NOW ); + handle =dlopen ("libnfsjunct.so.0" ,RTLD_NOW );if (handle ==NULL ) {xlog (D_GENERAL ,"%s: dlopen: %s" , __func__,dlerror ());return NULL ;
Quoted 7 lines · On Tue, 25 Feb 2014 22:06:58 -0800 Chuck Lever <[email protected]> wrote:
Quoted 7 lines · NeilBrown
See $SUBJ
Quoted 7 lines
Quoted 5 lines
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
There is an API version field that mountd can examine before using the plug-in. .so symlinks to whichever library version is the blessed one.
What exactly does "blessed one" mean...
Maybe the question I want to ask is:
Is libnfsjunct.so something that only rpc.mountd would ever use and it could not possibly make any sense for anything else to ever use it?
If so, then we probably don't want to install the .so.0 or .so.0.0.0 files and maybe should give libnfsjunct.so a different suffix because it is not "shared" in any sense (I wanted to suggest ".lo" for loadable object, but I think that is already used).
If not, then you could conceivably have two tool installed which use different version, so it doesn't make sense for either to be "blessed". The bare ".so" name would be "the version that should be used when building anything from source".
Does the question have a simple answer?
Thanks,
NeilBrown
Quoted 28 lines · NeilBrown
Quoted 7 lines · On Tue, 25 Feb 2014 22:06:58 -0800 Chuck Lever <[email protected]> wrote:
Quoted 7 lines · NeilBrown
See $SUBJ
Quoted 7 lines
Quoted 5 lines
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
There is an API version field that mountd can examine before using the plug-in. .so symlinks to whichever library version is the blessed one.
What exactly does "blessed one" mean...
Maybe the question I want to ask is:
Is libnfsjunct.so something that only rpc.mountd would ever use and it could not possibly make any sense for anything else to ever use it?If so, then we probably don't want to install the .so.0 or .so.0.0.0 files and maybe should give libnfsjunct.so a different suffix because it is not "shared" in any sense (I wanted to suggest ".lo" for loadable object, but I think that is already used).
Quoted 6 lines
If not, then you could conceivably have two tool installed which use different version, so it doesn't make sense for either to be "blessed". The bare ".so" name would be "the version that should be used when building anything from source".
Does the question have a simple answer?
Shared objects are always called .so, however private modules SHOULD be installed in a package private directory and no in /usr/lib[64]
Simo.
Signature · 1 line
Simo Sorce * Red Hat, Inc * New York
Quoted 16 lines · NeilBrown
Quoted 13 lines · On Tue, 25 Feb 2014 22:06:58 -0800 Chuck Lever <[email protected]> wrote:
Quoted 11 lines · NeilBrown
See $SUBJ
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
There is an API version field that mountd can examine before using the plug-in. .so symlinks to whichever library version is the blessed one.
What exactly does "blessed one" mean...
It means the one the local administrator chooses to use. The only reason to have more than one plug-in version installed is when testing an upgrade.
Quoted 4 lines
Maybe the question I want to ask is:
Is libnfsjunct.so something that only rpc.mountd would ever use and it could not possibly make any sense for anything else to ever use it?
There is a public header in /usr/include that contains the API definition. In the future the plug-in might also be able to perform administrative operations on junctions, and other applications could invoke this functionality via the plug-in.
I don't think it's applicability is narrow, even though mountd is its only user today.
Quoted 10 lines
If so, then we probably don't want to install the .so.0 or .so.0.0.0 files and maybe should give libnfsjunct.so a different suffix because it is not "shared" in any sense (I wanted to suggest ".lo" for loadable object, but I think that is already used).
If not, then you could conceivably have two tool installed which use different version, so it doesn't make sense for either to be "blessed". The bare ".so" name would be "the version that should be used when building anything from source".
I don't understand this last bit. The plug-in can get bug fixes and new features and mountd does not need to be rebuilt from source to use them.
Quoted 2 lines
Does the question have a simple answer?
I'm afraid I don't yet understand the problem you are trying to solve.
Quoted 3 lines
Thanks,
NeilBrown
Quoted 11 lines · NeilBrown
See $SUBJ
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
Both Fedora and openSUSE get upset about packaging a libFOO.so in a non "-devel" package and so trip over this library which clearly needs to be installed even if you aren't doing 'devel'opment.
Keep in mind this rule is there only for real shared libraries that are loaded by the the system loader.
however it is waived for 'modules' that are opened dynamically but are private to the application.
Quoted 5 lines
I would like to change mountd as per the patch below to use the ".0" file. I believe this will not break any installation as the ".so" is installed as a symlink to the ".0" (or maybe ".0.0.0").
Would this be acceptable?
It looks to me like this is an internal module for mountd that is not for use by other apps (which is why it is not versioned and can be changed at will as it is deployed at the same time mountd is ?
Or am I wrong here ?
If I am not wrong I would be against this change personally and would rather move the .so file in a private library dir (if it is not already there) to make it clear it is a private module.
Quoted 4 lines
There is a bit of a discussion about this here:
https://bugzilla.redhat.com/show_bug.cgi?id=889174but I either don't understand it or don't agree with it.
Which part of it, there are different opinions stated in the bugzilla at different times.
Simo.
Signature · 1 line
Simo Sorce * Red Hat, Inc * New York
Quoted 23 lines · Simo Sorce
Quoted 11 lines · NeilBrown
See $SUBJ
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
Both Fedora and openSUSE get upset about packaging a libFOO.so in a non "-devel" package and so trip over this library which clearly needs to be installed even if you aren't doing 'devel'opment.
Keep in mind this rule is there only for real shared libraries that are loaded by the the system loader.
however it is waived for 'modules' that are opened dynamically but are private to the application.
Quoted 4 lines
I would like to change mountd as per the patch below to use the ".0" file. I believe this will not break any installation as the ".so" is installed as a symlink to the ".0" (or maybe ".0.0.0").
Quoted 5 lines
Quoted 1 line
Would this be acceptable?
It looks to me like this is an internal module for mountd that is not for use by other apps (which is why it is not versioned and can be changed at will as it is deployed at the same time mountd is ?
The plug-in API is versioned internally, but maybe I got that wrong, and should remove the API version field in favor of having consumers load via a specific .so number.
Quoted 5 lines
Or am I wrong here ?
If I am not wrong I would be against this change personally and would rather move the .so file in a private library dir (if it is not already there) to make it clear it is a private module.
rpc.mountd is the only user currently, but it’s not necessarily private to mountd. A generic storage manager tool might use it to resolve NFS and FedFS referrals for display, for example. We could add plug-in API functions for creating and removing referrals to enable generic tools to perform these operations.
A separate directory makes sense if there’s more than one thing to put in it. Right now we just have the plug-in library, and no plans to add more.
I took an expedient approach when implementing the plug-in, and could have gotten it wrong. I’m open to make this mechanism fit packaging guidelines and requirements.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Quoted 27 lines · Chuck Lever
Quoted 25 lines · Simo Sorce
Quoted 11 lines · NeilBrown
See $SUBJ
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
Both Fedora and openSUSE get upset about packaging a libFOO.so in a non "-devel" package and so trip over this library which clearly needs to be installed even if you aren't doing 'devel'opment.
Keep in mind this rule is there only for real shared libraries that are loaded by the the system loader.
however it is waived for 'modules' that are opened dynamically but are private to the application.
Quoted 5 lines
I would like to change mountd as per the patch below to use the ".0" file. I believe this will not break any installation as the ".so" is installed as a symlink to the ".0" (or maybe ".0.0.0").
Would this be acceptable?
Quoted 7 lines
Quoted 3 lines
It looks to me like this is an internal module for mountd that is not for use by other apps (which is why it is not versioned and can be changed at will as it is deployed at the same time mountd is ?
The plug-in API is versioned internally, but maybe I got that wrong, and should remove the API version field in favor of having consumers load via a specific .so number.
Either way works the same, it just changes what component makes the determination (app code vs linker)
Quoted 11 lines
Quoted 5 lines
Or am I wrong here ?
If I am not wrong I would be against this change personally and would rather move the .so file in a private library dir (if it is not already there) to make it clear it is a private module.
rpc.mountd is the only user currently, but it’s not necessarily private to mountd. A generic storage manager tool might use it to resolve NFS and FedFS referrals for display, for example. We could add plug-in API functions for creating and removing referrals to enable generic tools to perform these operations.
If it is a generic library why is it dlopened() instead of being simply linked in at build time ?
Quoted 3 lines
A separate directory makes sense if there’s more than one thing to put in it. Right now we just have the plug-in library, and no plans to add more.
directories are cheap, don't fear them :)
Quoted 3 lines
I took an expedient approach when implementing the plug-in, and could have gotten it wrong. I’m open to make this mechanism fit packaging guidelines and requirements.
Packaging guidelines vary depending on whether the library is public or private and therefore you need to guarantee ABI compatibility or not.
I think you need to make that determination first.
Simo.
Signature · 1 line
Simo Sorce * Red Hat, Inc * New York
Quoted 45 lines · Simo Sorce
Quoted 27 lines · Chuck Lever
Quoted 25 lines · Simo Sorce
Quoted 11 lines · NeilBrown
See $SUBJ
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
Both Fedora and openSUSE get upset about packaging a libFOO.so in a non "-devel" package and so trip over this library which clearly needs to be installed even if you aren't doing 'devel'opment.
Keep in mind this rule is there only for real shared libraries that are loaded by the the system loader.
however it is waived for 'modules' that are opened dynamically but are private to the application.
Quoted 5 lines
I would like to change mountd as per the patch below to use the ".0" file. I believe this will not break any installation as the ".so" is installed as a symlink to the ".0" (or maybe ".0.0.0").
Would this be acceptable?
Quoted 7 lines
Quoted 3 lines
It looks to me like this is an internal module for mountd that is not for use by other apps (which is why it is not versioned and can be changed at will as it is deployed at the same time mountd is ?
The plug-in API is versioned internally, but maybe I got that wrong, and should remove the API version field in favor of having consumers load via a specific .so number.
Either way works the same, it just changes what component makes the determination (app code vs linker)
Quoted 6 lines
Quoted 5 lines
Or am I wrong here ?
If I am not wrong I would be against this change personally and would rather move the .so file in a private library dir (if it is not already there) to make it clear it is a private module.
Quoted 8 lines
Quoted 5 lines
rpc.mountd is the only user currently, but it’s not necessarily private to mountd. A generic storage manager tool might use it to resolve NFS and FedFS referrals for display, for example. We could add plug-in API functions for creating and removing referrals to enable generic tools to perform these operations.
If it is a generic library why is it
dlopened()instead of being simply linked in at build time ?
Handling NFS and FedFS junctions requires support for sqlite3, LDAP, and XML, among others. The maintainer of nfs-utils preferred to add zero new build dependencies when we introduced this functionality. The design we came up with was to dlopen() a library that would pull in everything that was needed at run time.
If the plug-in is not installed, mountd simply skips trying to resolve junctions. This would be the case for embedded NFS servers, for example.
Quoted 7 lines
Quoted 4 lines
A separate directory makes sense if there’s more than one thing to put in it. Right now we just have the plug-in library, and no plans to add more.
directories are cheap, don't fear them :)
Quoted 8 lines
Quoted 3 lines
I took an expedient approach when implementing the plug-in, and could have gotten it wrong. I’m open to make this mechanism fit packaging guidelines and requirements.
Packaging guidelines vary depending on whether the library is public or private and therefore you need to guarantee ABI compatibility or not.
I think you need to make that determination first.
I attempted to guarantee API compatibility using the API version field and by publishing the API definition in a header under /usr/include. By that definition it is a public API that happens to have only one current user.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Quoted 62 lines · Chuck Lever
Quoted 45 lines · Simo Sorce
Quoted 27 lines · Chuck Lever
Quoted 25 lines · Simo Sorce
Quoted 11 lines · NeilBrown
See $SUBJ
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
Both Fedora and openSUSE get upset about packaging a libFOO.so in a non "-devel" package and so trip over this library which clearly needs to be installed even if you aren't doing 'devel'opment.
Keep in mind this rule is there only for real shared libraries that are loaded by the the system loader.
however it is waived for 'modules' that are opened dynamically but are private to the application.
Quoted 5 lines
I would like to change mountd as per the patch below to use the ".0" file. I believe this will not break any installation as the ".so" is installed as a symlink to the ".0" (or maybe ".0.0.0").
Would this be acceptable?
Quoted 7 lines
Quoted 3 lines
It looks to me like this is an internal module for mountd that is not for use by other apps (which is why it is not versioned and can be changed at will as it is deployed at the same time mountd is ?
The plug-in API is versioned internally, but maybe I got that wrong, and should remove the API version field in favor of having consumers load via a specific .so number.
Either way works the same, it just changes what component makes the determination (app code vs linker)
Quoted 6 lines
Quoted 5 lines
Or am I wrong here ?
If I am not wrong I would be against this change personally and would rather move the .so file in a private library dir (if it is not already there) to make it clear it is a private module.
Quoted 8 lines
Quoted 5 lines
rpc.mountd is the only user currently, but it’s not necessarily private to mountd. A generic storage manager tool might use it to resolve NFS and FedFS referrals for display, for example. We could add plug-in API functions for creating and removing referrals to enable generic tools to perform these operations.
If it is a generic library why is it
dlopened()instead of being simply linked in at build time ?Handling NFS and FedFS junctions requires support for sqlite3, LDAP, and XML, among others. The maintainer of nfs-utils preferred to add zero new build dependencies when we introduced this functionality. The design we came up with was to
dlopen()a library that would pull in everything that was needed at run time.
Quoted 4 lines
If the plug-in is not installed, mountd simply skips trying to resolve junctions. This would be the case for embedded NFS servers, for example.
Therefore this is an intimate library and the separation is a mere exercise in keeping the ability to not drag in dependencies in some install scenarios.
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
A separate directory makes sense if there’s more than one thing to put in it. Right now we just have the plug-in library, and no plans to add more.
directories are cheap, don't fear them :)
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
I took an expedient approach when implementing the plug-in, and could have gotten it wrong. I’m open to make this mechanism fit packaging guidelines and requirements.
Packaging guidelines vary depending on whether the library is public or private and therefore you need to guarantee ABI compatibility or not.
I think you need to make that determination first.
I think based on the above that we are looking at a library that is currently just a private plugin. The best course of action IMHO is to move it to /usr/lib[64]/nfs-mountd or something so that it is clear that it is a private plugin. At least until mountd is the only user.
Quoted 4 lines
I attempted to guarantee API compatibility using the API version field and by publishing the API definition in a header under
/usr/include. By that definition it is a public API that happens to have only one current user.
API compatibility is all you need for a private plugin indeed, and perhaps not even that.
However for a public library what would matter is ABI compatibility, not API compatibility.
Given it is a lot of effort to guarantee a public API and that there really is no other user on the horizon I would recommend to consider this code a private plugin and treat it as such.
That is:
1. consider it tightly integrated with rpc.mountd and to be installed in lockstep
2. consider it's API/ABI not stable and a private contract within rpc.mountd
3. package it accordingly
Simo.
Signature · 1 line
Simo Sorce * Red Hat, Inc * New York
Quoted 106 lines · Simo Sorce
Quoted 62 lines · Chuck Lever
Quoted 45 lines · Simo Sorce
Quoted 27 lines · Chuck Lever
Quoted 25 lines · Simo Sorce
Quoted 11 lines · NeilBrown
See $SUBJ
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
Both Fedora and openSUSE get upset about packaging a libFOO.so in a non "-devel" package and so trip over this library which clearly needs to be installed even if you aren't doing 'devel'opment.
Keep in mind this rule is there only for real shared libraries that are loaded by the the system loader.
however it is waived for 'modules' that are opened dynamically but are private to the application.
Quoted 5 lines
I would like to change mountd as per the patch below to use the ".0" file. I believe this will not break any installation as the ".so" is installed as a symlink to the ".0" (or maybe ".0.0.0").
Would this be acceptable?
Quoted 7 lines
Quoted 3 lines
It looks to me like this is an internal module for mountd that is not for use by other apps (which is why it is not versioned and can be changed at will as it is deployed at the same time mountd is ?
The plug-in API is versioned internally, but maybe I got that wrong, and should remove the API version field in favor of having consumers load via a specific .so number.
Either way works the same, it just changes what component makes the determination (app code vs linker)
Quoted 6 lines
Quoted 5 lines
Or am I wrong here ?
If I am not wrong I would be against this change personally and would rather move the .so file in a private library dir (if it is not already there) to make it clear it is a private module.
Quoted 8 lines
Quoted 5 lines
rpc.mountd is the only user currently, but it’s not necessarily private to mountd. A generic storage manager tool might use it to resolve NFS and FedFS referrals for display, for example. We could add plug-in API functions for creating and removing referrals to enable generic tools to perform these operations.
If it is a generic library why is it
dlopened()instead of being simply linked in at build time ?Handling NFS and FedFS junctions requires support for sqlite3, LDAP, and XML, among others. The maintainer of nfs-utils preferred to add zero new build dependencies when we introduced this functionality. The design we came up with was to
dlopen()a library that would pull in everything that was needed at run time.Quoted 4 lines
If the plug-in is not installed, mountd simply skips trying to resolve junctions. This would be the case for embedded NFS servers, for example.
Therefore this is an intimate library and the separation is a mere exercise in keeping the ability to not drag in dependencies in some install scenarios.
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
A separate directory makes sense if there’s more than one thing to put in it. Right now we just have the plug-in library, and no plans to add more.
directories are cheap, don't fear them :)
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
I took an expedient approach when implementing the plug-in, and could have gotten it wrong. I’m open to make this mechanism fit packaging guidelines and requirements.
Packaging guidelines vary depending on whether the library is public or private and therefore you need to guarantee ABI compatibility or not.
I think you need to make that determination first.
I think based on the above that we are looking at a library that is currently just a private plugin. The best course of action IMHO is to move it to /usr/lib[64]/nfs-mountd or something so that it is clear that it is a private plugin. At least until mountd is the only user.
Quoted 4 lines
I attempted to guarantee API compatibility using the API version field and by publishing the API definition in a header under
/usr/include. By that definition it is a public API that happens to have only one current user.API compatibility is all you need for a private plugin indeed, and perhaps not even that.
However for a public library what would matter is ABI compatibility, not API compatibility.
Given it is a lot of effort to guarantee a public API and that there really is no other user on the horizon I would recommend to consider this code a private plugin and treat it as such.
Quoted 6 lines
That is:
1. consider it tightly integrated with rpc.mountd and to be installed in lockstep
2. consider it's API/ABI not stable and a private contract within rpc.mountd
3. package it accordingly
No... I do not want to make rpc.mount and the libnfsjunct.so tightly coupled... Again, the use of it... if it exists... paradigm is good...
steved.
Quoted 3 lines
Simo.
Sorry for getting into this so late...
Quoted 54 lines · Chuck Lever
Quoted 51 lines · Simo Sorce
Quoted 27 lines · Chuck Lever
Quoted 25 lines · Simo Sorce
Quoted 11 lines · NeilBrown
See $SUBJ
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
Both Fedora and openSUSE get upset about packaging a libFOO.so in a non "-devel" package and so trip over this library which clearly needs to be installed even if you aren't doing 'devel'opment.
Keep in mind this rule is there only for real shared libraries that are loaded by the the system loader.
however it is waived for 'modules' that are opened dynamically but are private to the application.
Quoted 5 lines
I would like to change mountd as per the patch below to use the ".0" file. I believe this will not break any installation as the ".so" is installed as a symlink to the ".0" (or maybe ".0.0.0").
Would this be acceptable?
Quoted 7 lines
Quoted 3 lines
It looks to me like this is an internal module for mountd that is not for use by other apps (which is why it is not versioned and can be changed at will as it is deployed at the same time mountd is ?
The plug-in API is versioned internally, but maybe I got that wrong, and should remove the API version field in favor of having consumers load via a specific .so number.
Either way works the same, it just changes what component makes the determination (app code vs linker)
Quoted 11 lines
Quoted 5 lines
Or am I wrong here ?
If I am not wrong I would be against this change personally and would rather move the .so file in a private library dir (if it is not already there) to make it clear it is a private module.
rpc.mountd is the only user currently, but it’s not necessarily private to mountd. A generic storage manager tool might use it to resolve NFS and FedFS referrals for display, for example. We could add plug-in API functions for creating and removing referrals to enable generic tools to perform these operations.
Quoted 8 lines
Quoted 2 lines
If it is a generic library why is it
dlopened()instead of being simply linked in at build time ?Handling NFS and FedFS junctions requires support for sqlite3, LDAP, and XML, among others. The maintainer of nfs-utils preferred to add zero new build dependencies when we introduced this functionality. The design we came up with was to
dlopen()a library that would pull in everything that was needed at run time.
Right... The dependency list is way too long now and the last thing I wanted was make nfs-utils depend on things it will probably never use...
Quoted 3 lines
If the plug-in is not installed, mountd simply skips trying to resolve junctions. This would be the case for embedded NFS servers, for example.
I think this works well...
I guess I don't care about the version-ing or where the library lives... but I don't want rpc.mountd to be depending on libnfsjunct.so just use it when it exists.
steved.
Quoted 25 lines
Quoted 8 lines
Quoted 5 lines
A separate directory makes sense if there’s more than one thing to put in it. Right now we just have the plug-in library, and no plans to add more.
directories are cheap, don't fear them :)
Quoted 8 lines
Quoted 3 lines
I took an expedient approach when implementing the plug-in, and could have gotten it wrong. I’m open to make this mechanism fit packaging guidelines and requirements.
Packaging guidelines vary depending on whether the library is public or private and therefore you need to guarantee ABI compatibility or not.
I think you need to make that determination first.
I attempted to guarantee API compatibility using the API version field and by publishing the API definition in a header under
/usr/include. By that definition it is a public API that happens to have only one current user.--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Quoted 26 lines · On Wed, 26 Feb 2014 08:02:42 -0800 Chuck Lever <[email protected]> wrote:
Quoted 23 lines · Simo Sorce
Quoted 11 lines · NeilBrown
See $SUBJ
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
Both Fedora and openSUSE get upset about packaging a libFOO.so in a non "-devel" package and so trip over this library which clearly needs to be installed even if you aren't doing 'devel'opment.
Keep in mind this rule is there only for real shared libraries that are loaded by the the system loader.
however it is waived for 'modules' that are opened dynamically but are private to the application.
Quoted 4 lines
I would like to change mountd as per the patch below to use the ".0" file. I believe this will not break any installation as the ".so" is installed as a symlink to the ".0" (or maybe ".0.0.0").
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Would this be acceptable?
It looks to me like this is an internal module for mountd that is not for use by other apps (which is why it is not versioned and can be changed at will as it is deployed at the same time mountd is ?
The plug-in API is versioned internally, but maybe I got that wrong, and should remove the API version field in favor of having consumers load via a specific .so number.
The problem I see with using the internal versioning is that if the version is wrong, mountd fails to provide the required service.
So while I don't object to storing the version and performing the test, we should design work-flows so that the test can only fail if there is a serious configuration error, not just during a software upgrade.
Quoted 8 lines
Quoted 6 lines
Or am I wrong here ?
If I am not wrong I would be against this change personally and would rather move the .so file in a private library dir (if it is not already there) to make it clear it is a private module.
rpc.mountd is the only user currently, but it’s not necessarily private to mountd. A generic storage manager tool might use it to resolve NFS and FedFS referrals for display, for example. We could add plug-in API functions for creating and removing referrals to enable generic tools to perform these operations.
This is the answer I was looking for to the question I asked earlier - thanks. (So this is not an 'intimate library' to use Simo's term - it is truly a shared library).
If, one day, an incompatible ABI change was needed then we could have an rpc.mountd installed (or still running) which requires one ABI, and a generic storage manager tool which requires the other.
So we really need them to be stored in two different files. e.g. libnfsjunct.so.0 and libnfsjunct.so.1
To put it another way... libnfsjunct really is a shared library. The *only* reason that rpc.mountd treats it differently to other shared libraries is so that it can fail gracefully if the library isn't available (thus removing hard dependencies) - a difference that I am very comfortable with.
In every other way it should be treated like a shared library
- it should live in the standard /lib64 or whatever
- each application determines at compile-time what version it needs and finds it by appending the version number to the base file name
- the "libfoo.so" file should live in the "-devel" package along with the include file(s)
So rather than dlopening "libnfsjunct.so.0" rpc.mountd should probably use a library name provided by the include file
Thanks,
NeilBrown
utils/mountd/cache.c +5 -1
@@ -1139,7 +1139,11 @@ static struct exportent *lookup_junction(char *dom, const char *pathname,struct link_map *map;void *handle; - handle =dlopen ("libnfsjunct.so" ,RTLD_NOW ); +#ifdef JP_LIB_NAME + handle =dlopen (JP_LIB_NAME ,RTLD_NOW ); +#else + handle =dlopen ("libnfsjunct.so.0" ,RTLD_NOW ); +#endif if (handle ==NULL ) {xlog (D_GENERAL ,"%s: dlopen: %s" , __func__,dlerror ());return NULL ;
Quoted 55 lines · NeilBrown
Quoted 26 lines · On Wed, 26 Feb 2014 08:02:42 -0800 Chuck Lever <[email protected]> wrote:
Quoted 23 lines · Simo Sorce
Quoted 11 lines · NeilBrown
See $SUBJ
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
Both Fedora and openSUSE get upset about packaging a libFOO.so in a non "-devel" package and so trip over this library which clearly needs to be installed even if you aren't doing 'devel'opment.
Keep in mind this rule is there only for real shared libraries that are loaded by the the system loader.
however it is waived for 'modules' that are opened dynamically but are private to the application.
Quoted 4 lines
I would like to change mountd as per the patch below to use the ".0" file. I believe this will not break any installation as the ".so" is installed as a symlink to the ".0" (or maybe ".0.0.0").
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Would this be acceptable?
It looks to me like this is an internal module for mountd that is not for use by other apps (which is why it is not versioned and can be changed at will as it is deployed at the same time mountd is ?
The plug-in API is versioned internally, but maybe I got that wrong, and should remove the API version field in favor of having consumers load via a specific .so number.
The problem I see with using the internal versioning is that if the version is wrong, mountd fails to provide the required service.
So while I don't object to storing the version and performing the test, we should design work-flows so that the test can only fail if there is a serious configuration error, not just during a software upgrade.Quoted 8 lines
Quoted 6 lines
Or am I wrong here ?
If I am not wrong I would be against this change personally and would rather move the .so file in a private library dir (if it is not already there) to make it clear it is a private module.
rpc.mountd is the only user currently, but it’s not necessarily private to mountd. A generic storage manager tool might use it to resolve NFS and FedFS referrals for display, for example. We could add plug-in API functions for creating and removing referrals to enable generic tools to perform these operations.
This is the answer I was looking for to the question I asked earlier - thanks. (So this is not an 'intimate library' to use Simo's term - it is truly a shared library).
Quoted 5 lines
If, one day, an incompatible ABI change was needed then we could have an rpc.mountd installed (or still running) which requires one ABI, and a generic storage manager tool which requires the other.
So we really need them to be stored in two different files. e.g. libnfsjunct.so.0 and libnfsjunct.so.1
I was hoping this would never happen. One plug-in library should be able to serve mountd or any other tool that might need to play with junctions.
Only a crazy developer like me would ever need to have more than one library version at a time, and even then, it’s pretty simple to build what I need and reinstall, rather than having more than one installed at a time.
Quoted 13 lines
To put it another way... libnfsjunct really is a shared library. The *only* reason that rpc.mountd treats it differently to other shared libraries is so that it can fail gracefully if the library isn't available (thus removing hard dependencies) - a difference that I am very comfortable with.
In every other way it should be treated like a shared library
- it should live in the standard /lib64 or whatever
- each application determines at compile-time what version it needs and finds it by appending the version number to the base file name
- the "libfoo.so" file should live in the "-devel" package along with the include file(s)
Quoted 2 lines
So rather than dlopening "libnfsjunct.so.0" rpc.mountd should probably use a library name provided by the include file
I’m dense, I still don’t see why this makes a difference. I’ll admit that linker fu is something I’ve left to others, so don’t be afraid to spell it out slowly for me.
Quoted 10 lines
Thanks,
NeilBrownutils/mountd/cache.c +0
-0@@ -1139,7 +1139,11 @@ static struct exportent *lookup_junction(char *dom, const char *pathname,struct link_map *map;void *handle;
utils/mountd/cache.c static struct exportent *lookup_junction(char *dom, const char *pathname, · line 1149
Quoted 10 lines
utils/mountd/cache.c +5
-1- handle =dlopen ("libnfsjunct.so" ,RTLD_NOW ); +#ifdef JP_LIB_NAME + handle =dlopen (JP_LIB_NAME ,RTLD_NOW ); +#else + handle =dlopen ("libnfsjunct.so.0" ,RTLD_NOW ); +#endif if (handle ==NULL ) {xlog (D_GENERAL ,"%s: dlopen: %s" , __func__,dlerror ());return NULL ;
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Quoted 58 lines · On Thu, 27 Feb 2014 08:57:56 -0800 Chuck Lever <[email protected]> wrote:
Quoted 55 lines · NeilBrown
Quoted 26 lines · On Wed, 26 Feb 2014 08:02:42 -0800 Chuck Lever <[email protected]> wrote:
Quoted 23 lines · Simo Sorce
Quoted 11 lines · NeilBrown
See $SUBJ
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
Both Fedora and openSUSE get upset about packaging a libFOO.so in a non "-devel" package and so trip over this library which clearly needs to be installed even if you aren't doing 'devel'opment.
Keep in mind this rule is there only for real shared libraries that are loaded by the the system loader.
however it is waived for 'modules' that are opened dynamically but are private to the application.
Quoted 4 lines
I would like to change mountd as per the patch below to use the ".0" file. I believe this will not break any installation as the ".so" is installed as a symlink to the ".0" (or maybe ".0.0.0").
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Would this be acceptable?
It looks to me like this is an internal module for mountd that is not for use by other apps (which is why it is not versioned and can be changed at will as it is deployed at the same time mountd is ?
The plug-in API is versioned internally, but maybe I got that wrong, and should remove the API version field in favor of having consumers load via a specific .so number.
The problem I see with using the internal versioning is that if the version is wrong, mountd fails to provide the required service.
So while I don't object to storing the version and performing the test, we should design work-flows so that the test can only fail if there is a serious configuration error, not just during a software upgrade.Quoted 8 lines
Quoted 6 lines
Or am I wrong here ?
If I am not wrong I would be against this change personally and would rather move the .so file in a private library dir (if it is not already there) to make it clear it is a private module.
rpc.mountd is the only user currently, but it’s not necessarily private to mountd. A generic storage manager tool might use it to resolve NFS and FedFS referrals for display, for example. We could add plug-in API functions for creating and removing referrals to enable generic tools to perform these operations.
This is the answer I was looking for to the question I asked earlier - thanks. (So this is not an 'intimate library' to use Simo's term - it is truly a shared library).
Quoted 7 lines
Quoted 5 lines
If, one day, an incompatible ABI change was needed then we could have an rpc.mountd installed (or still running) which requires one ABI, and a generic storage manager tool which requires the other.
So we really need them to be stored in two different files. e.g. libnfsjunct.so.0 and libnfsjunct.so.1I was hoping this would never happen. One plug-in library should be able to serve mountd or any other tool that might need to play with junctions.
Certainly that is the hope. I think everyone who writes a shared library hopes they will get it right first time, and that if a change is ever needed then all users can be upgraded simultaneously.
$ ls -l /lib64/lib*.so.1 | grep -c '^-' 4 $ ls -l /lib64/lib*.so.1.* | grep -c '^-' 17 $ ls -l /lib64/lib*.so.[2-9]* | grep -c '^-' 20
That seems to happen often, but not always. That is why we have shared library versioning.
Quoted 16 lines
Only a crazy developer like me would ever need to have more than one library version at a time, and even then, it’s pretty simple to build what I need and reinstall, rather than having more than one installed at a time.
Quoted 13 lines
To put it another way... libnfsjunct really is a shared library. The *only* reason that rpc.mountd treats it differently to other shared libraries is so that it can fail gracefully if the library isn't available (thus removing hard dependencies) - a difference that I am very comfortable with.
In every other way it should be treated like a shared library
- it should live in the standard /lib64 or whatever
- each application determines at compile-time what version it needs and finds it by appending the version number to the base file name
- the "libfoo.so" file should live in the "-devel" package along with the include file(s)
Quoted 4 lines
Quoted 2 lines
So rather than dlopening "libnfsjunct.so.0" rpc.mountd should probably use a library name provided by the include file
I’m dense, I still don’t see why this makes a difference. I’ll admit that linker fu is something I’ve left to others, so don’t be afraid to spell it out slowly for me.
I'll try (might make sure I understand it too). The following is based in part on section 3.1.1 of
http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html
A shared library (like a cat) has three different names.
1/ The file name. This is normally /$LIBDIR/libFOO.so.maj.min.release
(e.g. /usr/lib/libnfsjunct.so.0.0.0), though it can be almost whatever you
like. It is used by installers to install the library, and by ldconfig.
ldconfig only wants it to start "lib" or "ld-" and to have ".so" somewhere in the name.
2/ The "soname". This is /$LIBDIR/libFOO.so.maj (i.e. only major number).
ldconfig will create a symlink from this name to the "most recent" library found with that SONAME (a field in the shared library:
objdump -x $LIBRARY | grep SONAME
).
An application which needs to be linked will contain the "soname" of each library that it wants to use. "ldd" lists these and the matching filename for each. ld.so effective calls "dlopen" on each "soname".
3/ The "linker name". This is the name that is used when you compile code.
You typically specify "-lFOO" and the linker interprets that at "$LIBPATH/libFOO.so" and finds a shared library. It extracts the SONAME from this library and stores that in that generated binary. Naturally the library version found at the "linker name" must match the include files describing data structures etc in the library.
To follow this pattern as closely as possible, and yet allow rpc.mountd to use dlopen() to load the library:
- the "soname" should be passed to dlopen(). (That is what ld.so does)
- that name should be determined from the compile-time environment. (that is what 'ld' does).
i.e. we should pass "libnfsjunct.so.0" to dlopen() (if the current fedfs-utils provides the compile-time environment). We could determine that string with a little script which runs
objdump -x /lib64/libnfsjunct.so | sed -n -e 's/^ *SONAME *//p'
or we could simply keep it in the include file (which must be in-sync with the .so).
Doing this
1/ ensures that we have the full flexibility of shared libraries should we
ever need that.
2/ makes the nfsjunct library look just like any other shared library and so
avoids confusion for package checkers.
Does that clarify at all?
Thanks,
NeilBrown
Quoted 24 lines
Quoted 8 lines
utils/mountd/cache.c +0
-0@@ -1139,7 +1139,11 @@ static struct exportent *lookup_junction(char *dom, const char *pathname,struct link_map *map;void *handle;utils/mountd/cache.c static struct exportent *lookup_junction(char *dom, const char *pathname, · line 1149
Quoted 10 lines
utils/mountd/cache.c +5
-1- handle =dlopen ("libnfsjunct.so" ,RTLD_NOW ); +#ifdef JP_LIB_NAME + handle =dlopen (JP_LIB_NAME ,RTLD_NOW ); +#else + handle =dlopen ("libnfsjunct.so.0" ,RTLD_NOW ); +#endif if (handle ==NULL ) {xlog (D_GENERAL ,"%s: dlopen: %s" , __func__,dlerror ());return NULL ;--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Quoted 146 lines · NeilBrown
Quoted 58 lines · On Thu, 27 Feb 2014 08:57:56 -0800 Chuck Lever <[email protected]> wrote:
Quoted 55 lines · NeilBrown
Quoted 26 lines · On Wed, 26 Feb 2014 08:02:42 -0800 Chuck Lever <[email protected]> wrote:
Quoted 23 lines · Simo Sorce
Quoted 11 lines · NeilBrown
See $SUBJ
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
Both Fedora and openSUSE get upset about packaging a libFOO.so in a non "-devel" package and so trip over this library which clearly needs to be installed even if you aren't doing 'devel'opment.
Keep in mind this rule is there only for real shared libraries that are loaded by the the system loader.
however it is waived for 'modules' that are opened dynamically but are private to the application.
Quoted 4 lines
I would like to change mountd as per the patch below to use the ".0" file. I believe this will not break any installation as the ".so" is installed as a symlink to the ".0" (or maybe ".0.0.0").
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Would this be acceptable?
It looks to me like this is an internal module for mountd that is not for use by other apps (which is why it is not versioned and can be changed at will as it is deployed at the same time mountd is ?
The plug-in API is versioned internally, but maybe I got that wrong, and should remove the API version field in favor of having consumers load via a specific .so number.
The problem I see with using the internal versioning is that if the version is wrong, mountd fails to provide the required service.
So while I don't object to storing the version and performing the test, we should design work-flows so that the test can only fail if there is a serious configuration error, not just during a software upgrade.Quoted 8 lines
Quoted 6 lines
Or am I wrong here ?
If I am not wrong I would be against this change personally and would rather move the .so file in a private library dir (if it is not already there) to make it clear it is a private module.
rpc.mountd is the only user currently, but it’s not necessarily private to mountd. A generic storage manager tool might use it to resolve NFS and FedFS referrals for display, for example. We could add plug-in API functions for creating and removing referrals to enable generic tools to perform these operations.
This is the answer I was looking for to the question I asked earlier - thanks. (So this is not an 'intimate library' to use Simo's term - it is truly a shared library).
Quoted 7 lines
Quoted 5 lines
If, one day, an incompatible ABI change was needed then we could have an rpc.mountd installed (or still running) which requires one ABI, and a generic storage manager tool which requires the other.
So we really need them to be stored in two different files. e.g. libnfsjunct.so.0 and libnfsjunct.so.1I was hoping this would never happen. One plug-in library should be able to serve mountd or any other tool that might need to play with junctions.
Certainly that is the hope. I think everyone who writes a shared library hopes they will get it right first time, and that if a change is ever needed then all users can be upgraded simultaneously.
$ ls -l /lib64/lib*.so.1 | grep -c '^-' 4 $ ls -l /lib64/lib*.so.1.* | grep -c '^-' 17 $ ls -l /lib64/lib*.so.[2-9]* | grep -c '^-' 20That seems to happen often, but not always. That is why we have shared library versioning.
Quoted 16 lines
Only a crazy developer like me would ever need to have more than one library version at a time, and even then, it’s pretty simple to build what I need and reinstall, rather than having more than one installed at a time.
Quoted 13 lines
To put it another way... libnfsjunct really is a shared library. The *only* reason that rpc.mountd treats it differently to other shared libraries is so that it can fail gracefully if the library isn't available (thus removing hard dependencies) - a difference that I am very comfortable with.
In every other way it should be treated like a shared library
- it should live in the standard /lib64 or whatever
- each application determines at compile-time what version it needs and finds it by appending the version number to the base file name
- the "libfoo.so" file should live in the "-devel" package along with the include file(s)Quoted 4 lines
Quoted 2 lines
So rather than dlopening "libnfsjunct.so.0" rpc.mountd should probably use a library name provided by the include file
I’m dense, I still don’t see why this makes a difference. I’ll admit that linker fu is something I’ve left to others, so don’t be afraid to spell it out slowly for me.
I'll try (might make sure I understand it too). The following is based in part on section 3.1.1 of
http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.htmlA shared library (like a cat) has three different names.
1/ The file name. This is normally /$LIBDIR/libFOO.so.maj.min.release
(e.g./usr/lib/libnfsjunct.so.0.0.0), though it can be almost whatever you
like. It is used by installers to install the library, and by ldconfig.
ldconfig only wants it to start "lib" or "ld-" and to have ".so" somewhere in the name.2/ The "soname". This is /$LIBDIR/libFOO.so.maj (i.e. only major number).
ldconfig will create a symlink from this name to the "most recent" library found with that SONAME (a field in the shared library:
objdump -x $LIBRARY | grep SONAME
).
An application which needs to be linked will contain the "soname" of each library that it wants to use. "ldd" lists these and the matching filename for each. ld.so effective calls "dlopen" on each "soname".3/ The "linker name". This is the name that is used when you compile code.
You typically specify "-lFOO" and the linker interprets that at "$LIBPATH/libFOO.so" and finds a shared library. It extracts the SONAME from this library and stores that in that generated binary. Naturally the library version found at the "linker name" must match the include files describing data structures etc in the library.To follow this pattern as closely as possible, and yet allow rpc.mountd to use
dlopen()to load the library:
- the "soname" should be passed todlopen(). (That is what ld.so does)
- that name should be determined from the compile-time environment. (that is what 'ld' does).i.e. we should pass "libnfsjunct.so.0" to
dlopen()(if the current fedfs-utils provides the compile-time environment). We could determine that string with a little script which runsobjdump -x
/lib64/libnfsjunct.so| sed -n -e 's/^ *SONAME *//p'or we could simply keep it in the include file (which must be in-sync with the .so).
Quoted 7 lines
Doing this
1/ ensures that we have the full flexibility of shared libraries should we
ever need that.
2/ makes the nfsjunct library look just like any other shared library and so
avoids confusion for package checkers.Does that clarify at all?
Thank you Neil, it’s coming into focus for me.
We had some conversation about this at Connectathon last week. It seems like a better design would look like:
o A separate directory under /usr/lib{64} where fedfs-utils would install its plug-ins
o Plug-in consumers would dlopen() via the plug-in library's soname to guarantee ABI compatibility
o The API version field would be deprecated
o We didn’t discuss how consumers discover the plug-in soname, but if the API is defined in the header and the soname has to match, maybe that’s the way to go
I don’t think any of these changes would alter the “loose-ness” of current coupling between rpc.mountd and the plug-in library (to address Steve’s concern), but they would make a better guarantee that mountd was loading the correct plug-in library version.
I’m not sure exactly how to get from point A to point B. Probably fedfs-utils would have to package the plug-in library in the old and new places until all distributed versions of mountd was changed to find the plug-ins in the right place. That would have to be the case to allow nfs-utils downgrades for a particular distribution.
Quoted 30 lines
Thanks,
NeilBrownQuoted 24 lines
Quoted 8 lines
utils/mountd/cache.c +0
-0@@ -1139,7 +1139,11 @@ static struct exportent *lookup_junction(char *dom, const char *pathname,struct link_map *map;void *handle;utils/mountd/cache.c static struct exportent *lookup_junction(char *dom, const char *pathname, · line 1149
Quoted 10 lines
utils/mountd/cache.c +5
-1- handle =dlopen ("libnfsjunct.so" ,RTLD_NOW ); +#ifdef JP_LIB_NAME + handle =dlopen (JP_LIB_NAME ,RTLD_NOW ); +#else + handle =dlopen ("libnfsjunct.so.0" ,RTLD_NOW ); +#endif if (handle ==NULL ) {xlog (D_GENERAL ,"%s: dlopen: %s" , __func__,dlerror ());return NULL ;--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Quoted 155 lines · On Mon, 3 Mar 2014 12:45:55 -0500 Chuck Lever <[email protected]> wrote:
Quoted 152 lines · NeilBrown
Quoted 58 lines · On Thu, 27 Feb 2014 08:57:56 -0800 Chuck Lever <[email protected]> wrote:
Quoted 55 lines · NeilBrown
Quoted 26 lines · On Wed, 26 Feb 2014 08:02:42 -0800 Chuck Lever <[email protected]> wrote:
Quoted 23 lines · Simo Sorce
Quoted 11 lines · NeilBrown
See $SUBJ
Shared libraries are usually versioned so you can release a new version with an incompatible API and gradually transition to it.
A rpc.mountd dlopens libnfsjunct.so with no version it is effectively prohibited from ever changing the API in an incompatible way.
Both Fedora and openSUSE get upset about packaging a libFOO.so in a non "-devel" package and so trip over this library which clearly needs to be installed even if you aren't doing 'devel'opment.
Keep in mind this rule is there only for real shared libraries that are loaded by the the system loader.
however it is waived for 'modules' that are opened dynamically but are private to the application.
Quoted 4 lines
I would like to change mountd as per the patch below to use the ".0" file. I believe this will not break any installation as the ".so" is installed as a symlink to the ".0" (or maybe ".0.0.0").
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Would this be acceptable?
It looks to me like this is an internal module for mountd that is not for use by other apps (which is why it is not versioned and can be changed at will as it is deployed at the same time mountd is ?
The plug-in API is versioned internally, but maybe I got that wrong, and should remove the API version field in favor of having consumers load via a specific .so number.
The problem I see with using the internal versioning is that if the version is wrong, mountd fails to provide the required service.
So while I don't object to storing the version and performing the test, we should design work-flows so that the test can only fail if there is a serious configuration error, not just during a software upgrade.Quoted 8 lines
Quoted 6 lines
Or am I wrong here ?
If I am not wrong I would be against this change personally and would rather move the .so file in a private library dir (if it is not already there) to make it clear it is a private module.
rpc.mountd is the only user currently, but it’s not necessarily private to mountd. A generic storage manager tool might use it to resolve NFS and FedFS referrals for display, for example. We could add plug-in API functions for creating and removing referrals to enable generic tools to perform these operations.
This is the answer I was looking for to the question I asked earlier - thanks. (So this is not an 'intimate library' to use Simo's term - it is truly a shared library).
Quoted 7 lines
Quoted 5 lines
If, one day, an incompatible ABI change was needed then we could have an rpc.mountd installed (or still running) which requires one ABI, and a generic storage manager tool which requires the other.
So we really need them to be stored in two different files. e.g. libnfsjunct.so.0 and libnfsjunct.so.1I was hoping this would never happen. One plug-in library should be able to serve mountd or any other tool that might need to play with junctions.
Certainly that is the hope. I think everyone who writes a shared library hopes they will get it right first time, and that if a change is ever needed then all users can be upgraded simultaneously.
$ ls -l /lib64/lib*.so.1 | grep -c '^-' 4 $ ls -l /lib64/lib*.so.1.* | grep -c '^-' 17 $ ls -l /lib64/lib*.so.[2-9]* | grep -c '^-' 20That seems to happen often, but not always. That is why we have shared library versioning.
Quoted 16 lines
Only a crazy developer like me would ever need to have more than one library version at a time, and even then, it’s pretty simple to build what I need and reinstall, rather than having more than one installed at a time.
Quoted 13 lines
To put it another way... libnfsjunct really is a shared library. The *only* reason that rpc.mountd treats it differently to other shared libraries is so that it can fail gracefully if the library isn't available (thus removing hard dependencies) - a difference that I am very comfortable with.
In every other way it should be treated like a shared library
- it should live in the standard /lib64 or whatever
- each application determines at compile-time what version it needs and finds it by appending the version number to the base file name
- the "libfoo.so" file should live in the "-devel" package along with the include file(s)Quoted 4 lines
Quoted 2 lines
So rather than dlopening "libnfsjunct.so.0" rpc.mountd should probably use a library name provided by the include file
I’m dense, I still don’t see why this makes a difference. I’ll admit that linker fu is something I’ve left to others, so don’t be afraid to spell it out slowly for me.
I'll try (might make sure I understand it too). The following is based in part on section 3.1.1 of
http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.htmlA shared library (like a cat) has three different names.
1/ The file name. This is normally /$LIBDIR/libFOO.so.maj.min.release
(e.g./usr/lib/libnfsjunct.so.0.0.0), though it can be almost whatever you
like. It is used by installers to install the library, and by ldconfig.
ldconfig only wants it to start "lib" or "ld-" and to have ".so" somewhere in the name.2/ The "soname". This is /$LIBDIR/libFOO.so.maj (i.e. only major number).
ldconfig will create a symlink from this name to the "most recent" library found with that SONAME (a field in the shared library:
objdump -x $LIBRARY | grep SONAME
).
An application which needs to be linked will contain the "soname" of each library that it wants to use. "ldd" lists these and the matching filename for each. ld.so effective calls "dlopen" on each "soname".3/ The "linker name". This is the name that is used when you compile code.
You typically specify "-lFOO" and the linker interprets that at "$LIBPATH/libFOO.so" and finds a shared library. It extracts the SONAME from this library and stores that in that generated binary. Naturally the library version found at the "linker name" must match the include files describing data structures etc in the library.To follow this pattern as closely as possible, and yet allow rpc.mountd to use
dlopen()to load the library:
- the "soname" should be passed todlopen(). (That is what ld.so does)
- that name should be determined from the compile-time environment. (that is what 'ld' does).i.e. we should pass "libnfsjunct.so.0" to
dlopen()(if the current fedfs-utils provides the compile-time environment). We could determine that string with a little script which runsobjdump -x
/lib64/libnfsjunct.so| sed -n -e 's/^ *SONAME *//p'or we could simply keep it in the include file (which must be in-sync with the .so).
Doing this
1/ ensures that we have the full flexibility of shared libraries should we
ever need that.
2/ makes the nfsjunct library look just like any other shared library and so
avoids confusion for package checkers.
Quoted 7 lines
Quoted 1 line
Does that clarify at all?
Thank you Neil, it’s coming into focus for me.
We had some conversation about this at Connectathon last week. It seems like a better design would look like:
o A separate directory under
/usr/lib{64}where fedfs-utils would install its plug-ins
While I don't object to this I wonder if it is worth the effort. Using a subdirectory would require rpc.mountd to know exactly what the full path was, and so would need to know if "64" was needed etc.
dlopen("fedfs-plugin/libfoo.so.1")
will not follow the standard search path (no search happens at all if a '/' is present), and
dlopen("libfoo.so.1")
does not search subdirectories.
When I look in /lib64 on my machine I find, for example
libnss_compat-2.18.so libnss_files.so.2 libnss_mdns6_minimal.so.2 libnss_compat.so.2 libnss_hesiod-2.18.so libnss_mdns_minimal.so.2 libnss_db-2.18.so libnss_hesiod.so.2 libnss_nis-2.18.so libnss_db.so.2 libnss_mdns.so.2 libnss_nis.so.2 libnss_dns-2.18.so libnss_mdns4.so.2 libnss_nisplus-2.18.so libnss_dns.so.2 libnss_mdns4_minimal.so.2 libnss_nisplus.so.2 libnss_files-2.18.so libnss_mdns6.so.2
which are all 'nss' plugins which a dlopen()ed by nsswitch. There is also
libdevmapper-event-*.so*
which are plugins loaded as needed by dmeventd. So there is clear precedent for pluggins living directly in /lib64 (or similar).
There are a few directories in /lib64:
32/. ast/. device-mapper/. engines/. ksh/. multipath/. security/.
Of these only 32, ast, and ksh (which is a symlink to ast) contain files with "soname" names.
/usr/lib contains a few more directories with soname files:
sane sasl2 qtcreator
being the largest.
So there is also some precedent for putting plugins in sub-directories.
I had a look at the code for sane, and it duplicates the searching of LD_LIBRARY_PATH (if set) from ld.so, and requires 'configure' to work out the correct libdir, to which it appends "/sane".
It looks like a lot of complexity that I would rather avoid myself....
Quoted 8 lines
o Plug-in consumers would
dlopen()via the plug-in library's soname to guarantee ABI compatibilityo The API version field would be deprecated
o We didn’t discuss how consumers discover the plug-in soname, but if the API is defined in the header and the soname has to match, maybe that’s the way to go
I don’t think any of these changes would alter the “loose-ness” of current coupling between rpc.mountd and the plug-in library (to address Steve’s concern), but they would make a better guarantee that mountd was loading the correct plug-in library version.
All the rest I completely agree with.
Quoted 3 lines
I’m not sure exactly how to get from point A to point B. Probably fedfs-utils would have to package the plug-in library in the old and new places until all distributed versions of mountd was changed to find the plug-ins in the right place. That would have to be the case to allow nfs-utils downgrades for a particular distribution.
One option would before the next fedfs release to update the major version of libnfsjunct to '1' and to discard the API version field and include the soname in the include file.
Then nfs-utils can determine at build time whether .0 or .1 is present (probably via some #define in the include file) and load the appropriate one. Then distros can install the .0 shared library where it is expected, and the .1 shared library where that is expected.
There would be no need to install the same version at two different locations. (Distros are already, presumably, quite capable of install multiple versions of shared libraries).
It would require having two source packages for fedfs, so maybe it would end up a bit awkward ... not sure.
Thanks,
NeilBrown
Quoted 39 lines
Quoted 4 lines
Quoted 29 lines
Thanks,
NeilBrownQuoted 24 lines
Quoted 8 lines
utils/mountd/cache.c +0
-0@@ -1139,7 +1139,11 @@ static struct exportent *lookup_junction(char *dom, const char *pathname,struct link_map *map;void *handle;utils/mountd/cache.c static struct exportent *lookup_junction(char *dom, const char *pathname, · line 1149
Quoted 10 lines
utils/mountd/cache.c +5
-1- handle =dlopen ("libnfsjunct.so" ,RTLD_NOW ); +#ifdef JP_LIB_NAME + handle =dlopen (JP_LIB_NAME ,RTLD_NOW ); +#else + handle =dlopen ("libnfsjunct.so.0" ,RTLD_NOW ); +#endif if (handle ==NULL ) {xlog (D_GENERAL ,"%s: dlopen: %s" , __func__,dlerror ());return NULL ;--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
On Feb 26, 2014, at 5:58 PM, NeilBrown <[email protected]> wrote:
[ … rationale removed … ]
Quoted 11 lines
So rather than dlopening "libnfsjunct.so.0" rpc.mountd should probably use a library name provided by the include file
Thanks,
NeilBrownutils/mountd/cache.c +0
-0
utils/mountd/cache.c static struct exportent *lookup_junction(char *dom, const char *pathname, · line 1143
Quoted 6 lines
utils/mountd/cache.c +2
-1@@ -1139,7 +1139,11 @@ static struct exportent *lookup_junction(char *dom, const char *pathname,struct link_map *map;void *handle; - handle =dlopen ("libnfsjunct.so" ,RTLD_NOW ); +#ifdef JP_LIB_NAME + handle =dlopen (JP_LIB_NAME ,RTLD_NOW );
What I can do now for fedfs-utils 0.10.1 is add a macro to the header file that defines the library name. I'd like to go with
#define JP_NFSPLUGIN_SONAME “libnfsjunct.so.0”
unless there are objections.
The mountd patch could also remove the API version check in invoke_junction_ops() when this macro is present.
utils/mountd/cache.c static struct exportent *lookup_junction(char *dom, const char *pathname, · line 1149
Quoted 6 lines
utils/mountd/cache.c +3
-0+#else + handle =dlopen ("libnfsjunct.so.0" ,RTLD_NOW ); +#endif if (handle ==NULL ) {xlog (D_GENERAL ,"%s: dlopen: %s" , __func__,dlerror ());return NULL ;
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Quoted 25 lines · On Tue, 4 Mar 2014 13:35:09 -0500 Chuck Lever <[email protected]> wrote:
On Feb 26, 2014, at 5:58 PM, NeilBrown <[email protected]> wrote:
[ … rationale removed … ]
Quoted 11 lines
So rather than dlopening "libnfsjunct.so.0" rpc.mountd should probably use a library name provided by the include file
Thanks,
NeilBrownutils/mountd/cache.c +0
-0utils/mountd/cache.c static struct exportent *lookup_junction(char *dom, const char *pathname, · line 1143
Quoted 6 lines
utils/mountd/cache.c +2
-1@@ -1139,7 +1139,11 @@ static struct exportent *lookup_junction(char *dom, const char *pathname,struct link_map *map;void *handle; - handle =dlopen ("libnfsjunct.so" ,RTLD_NOW ); +#ifdef JP_LIB_NAME + handle =dlopen (JP_LIB_NAME ,RTLD_NOW );What I can do now for fedfs-utils 0.10.1 is add a macro to the header file that defines the library name. I'd like to go with
Quoted 3 lines
#define
JP_NFSPLUGIN_SONAME“libnfsjunct.so.0”unless there are objections.
Perfectly happy with that, thanks.
Quoted 2 lines
The mountd patch could also remove the API version check in
invoke_junction_ops()when this macro is present.
I'll send a patch to Steve.
Thanks,
NeilBrown
Quoted 13 lines
utils/mountd/cache.c static struct exportent *lookup_junction(char *dom, const char *pathname, · line 1149
Quoted 7 lines
utils/mountd/cache.c +3
-0+#else + handle =dlopen ("libnfsjunct.so.0" ,RTLD_NOW ); +#endif if (handle ==NULL ) {xlog (D_GENERAL ,"%s: dlopen: %s" , __func__,dlerror ());return NULL ;--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
PATCH 1/1 mountd: use SONAME fir libnfsjunct when loading with dlopen.
The standard for loading shared libraries is to identify them by their "soname" (Which "objdump -x $BINARY | grep SONAME" will report). However mountd currently loads using the "linker name" which should only be used when building new code.
Future releases of fedfs-utils will define the soname in the include file, so if that is defined, use it. If not, use the soname of the first version: "libnfsjunct.so.0".
This is a slight behavioural change. However all distros known to package fedfs-utils will install "libnfsjunct.so.0" whenever they install the old name of "libnfsjunct.so", and "make install" will install both. So it should not be a noticeable change.
Also only test the JP_API_VERSION if it is defined. As the version is embedded in the soname, a secondary test is not needed.
utils/mountd/cache.c +7 -2
@@ -1087,12 +1087,13 @@ static struct exportent *invoke_junction_ops(void *handle, char *dom, __func__, error);return NULL ; } +#ifdef JP_API_VERSION if (ops->jp_api_version !=JP_API_VERSION ) {xlog (D_GENERAL ,"%s: unrecognized junction API version: %u" , __func__, ops->jp_api_version);return NULL ; } - +#endif status = ops->jp_init (false);if (status !=JP_OK ) { xlog(D_GENERAL, "%s: failed to resolve %s: %s",
@@ -1139,7 +1140,11 @@ static struct exportent *lookup_junction(char *dom, const char *pathname,struct link_map *map;void *handle; - handle =dlopen ("libnfsjunct.so" ,RTLD_NOW ); +#ifdef JP_NFSPLUGIN_SONAME + handle =dlopen (JP_NFSPLUGIN_SONAME ,RTLD_NOW ); +#else + handle =dlopen ("libnfsjunct.so.0" ,RTLD_NOW ); +#endif if (handle ==NULL ) {xlog (D_GENERAL ,"%s: dlopen: %s" , __func__,dlerror ());return NULL ;
Re: [PATCH: nfs-utils] mountd: use SONAME fir libnfsjunct when loading with dlopen.
Quoted 16 lines · NeilBrown
The standard for loading shared libraries is to identify them by their "soname" (Which "objdump -x $BINARY | grep SONAME" will report). However mountd currently loads using the "linker name" which should only be used when building new code.
Future releases of fedfs-utils will define the soname in the include file, so if that is defined, use it. If not, use the soname of the first version: "libnfsjunct.so.0".
This is a slight behavioural change. However all distros known to package fedfs-utils will install "libnfsjunct.so.0" whenever they install the old name of "libnfsjunct.so", and "make install" will install both. So it should not be a noticeable change.
Quoted 5 lines
Also only test the
JP_API_VERSIONif it is defined. As the version is embedded in the soname, a secondary test is not needed.Cc Chuck Lever <[email protected]>Signed-off-by NeilBrown <[email protected]>
Committed...
steved.
Quoted 34 lines
utils/mountd/cache.c +7
-2@@ -1087,12 +1087,13 @@ static struct exportent *invoke_junction_ops(void *handle, char *dom, __func__, error);return NULL ; } +#ifdef JP_API_VERSION if (ops->jp_api_version !=JP_API_VERSION ) {xlog (D_GENERAL ,"%s: unrecognized junction API version: %u" , __func__, ops->jp_api_version);return NULL ; } - +#endif status = ops->jp_init (false);if (status !=JP_OK ) { xlog(D_GENERAL, "%s: failed to resolve %s: %s",@@ -1139,7 +1140,11 @@ static struct exportent *lookup_junction(char *dom, const char *pathname,struct link_map *map;void *handle; - handle =dlopen ("libnfsjunct.so" ,RTLD_NOW ); +#ifdef JP_NFSPLUGIN_SONAME + handle =dlopen (JP_NFSPLUGIN_SONAME ,RTLD_NOW ); +#else + handle =dlopen ("libnfsjunct.so.0" ,RTLD_NOW ); +#endif if (handle ==NULL ) {xlog (D_GENERAL ,"%s: dlopen: %s" , __func__,dlerror ());return NULL ;