Skip to contentSkip to conversation

Test for ldap connectivity during lookup_ldap initialization

PATCH In review Oscar Salvador 2 participants 12 messages Aug 10, 2017 – Aug 18, 2017 Kernel automounter

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Test for ldap connectivity during lookup_ldap initialization

Right now we are just checking and parsing the ldap's configuration in lookup_init() function, and if everything goes well we return a zero value indicating that everything worked fine.

This has a problem, because in the case we set ldap method in /etc/nsswitch.conf, configuration parsing goes fine but ldap is not working, we will return a zero value in lookup_init() function, so autofs will move on, but then we will return NSS_STATUS_UNAVAIL in lookup_read_master() function once do_reconnect() fails, and because of this autofs will not be able to reread the maps.

NIS already handles this by testing for server connectivity during initialization. This patch does the same thing for LDAP.

---
 modules/lookup_ldap.c | 7 +++++++
 1 file changed, 7 insertions(+)
modules/lookup_ldap.c +7 -0
index 98701e5..29a67a5 100644
@@ -1681,6 +1681,7 @@ static int do_init(const char *mapfmt,
 		   int argc, const char *const *argv,
 		   struct lookup_context *ctxt, unsigned int reinit)
 {
+	struct ldap_conn conn;
 	unsigned int is_amd_format;
 	int ret;
 
@@ -1810,6 +1811,12 @@ static int do_init(const char *mapfmt,
 		}
 	}
 
+	memset(&conn, 0, sizeof(struct ldap_conn));
+	ret = do_reconnect (LOGOPT_ANY, &conn, ctxt);
+	if (ret != NSS_STATUS_SUCCESS)
+		return 1;
+
+	unbind_ldap_connection(LOGOPT_ANY, &conn, ctxt);
 	return ret;
 }
 
Signature · 4 lines
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe autofs" in
maintainer
#1 lore raw
Quoted 10 lines · Oscar Salvador

Right now we are just checking and parsing the ldap's configuration in lookup_init() function, and if everything goes well we return a zero value indicating that everything worked fine.

This has a problem, because in the case we set ldap method in /etc/nsswitch.conf, configuration parsing goes fine but ldap is not working, we will return a zero value in lookup_init() function, so autofs will move on, but then we will return NSS_STATUS_UNAVAIL in lookup_read_master() function once do_reconnect() fails, and because of this autofs will not be able to reread the maps.

Umm ... I can't see a check in the current upstream NIS code?

Quoted 26 lines

NIS already handles this by testing for server connectivity during initialization. This patch does the same thing for LDAP.

---
 modules/lookup_ldap.c | 7 +++++++
 1 file changed, 7 insertions(+)
modules/lookup_ldap.c +1 -0
index 98701e5..29a67a5 100644
@@ -1681,6 +1681,7 @@ static int do_init(const char *mapfmt,
 		   int argc, const char *const *argv,
 		   struct lookup_context *ctxt, unsigned int reinit)
 {
+	struct ldap_conn conn;
 	unsigned int is_amd_format;
 	int ret;
 
@@ -1810,6 +1811,12 @@ static int do_init(const char *mapfmt,
 		}
 	}

modules/lookup_ldap.c static int do_init(const char *mapfmt, · line 1822

Quoted 10 lines
modules/lookup_ldap.c +6 -0
 
+	memset(&conn, 0, sizeof(struct ldap_conn));
+	ret = do_reconnect (LOGOPT_ANY, &conn, ctxt);
+	if (ret != NSS_STATUS_SUCCESS)
+		return 1;
+
+	unbind_ldap_connection(LOGOPT_ANY, &conn, ctxt);
 	return ret;
 }
 

--
To unsubscribe from this list: send the line "unsubscribe autofs" in

maintainer
#2 lore raw
Quoted 5 lines · Ian Kent
Quoted 4 lines · Oscar Salvador

Right now we are just checking and parsing the ldap's configuration in lookup_init() function, and if everything goes well we return a zero value indicating that everything worked fine.

Quoted 8 lines
Quoted 6 lines

This has a problem, because in the case we set ldap method in /etc/nsswitch.conf, configuration parsing goes fine but ldap is not working, we will return a zero value in lookup_init() function, so autofs will move on, but then we will return NSS_STATUS_UNAVAIL in lookup_read_master() function once do_reconnect() fails, and because of this autofs will not be able to reread the maps.

Umm ... I can't see a check in the current upstream NIS code?

I also don't understand the problem in the upstream code your trying to resolve. Can you try and describe it again please?

Quoted 27 lines
Quoted 27 lines

NIS already handles this by testing for server connectivity during initialization. This patch does the same thing for LDAP.

---
 modules/lookup_ldap.c | 7 +++++++
 1 file changed, 7 insertions(+)
modules/lookup_ldap.c +1 -0
index 98701e5..29a67a5 100644
@@ -1681,6 +1681,7 @@ static int do_init(const char *mapfmt,
 		   int argc, const char *const *argv,
 		   struct lookup_context *ctxt, unsigned int reinit)
 {
+	struct ldap_conn conn;
 	unsigned int is_amd_format;
 	int ret;
 
@@ -1810,6 +1811,12 @@ static int do_init(const char *mapfmt,
 		}
 	}
 
Quoted 14 lines

modules/lookup_ldap.c static int do_init(const char *mapfmt, · line 1822

Quoted 10 lines
modules/lookup_ldap.c +6 -0
+	memset(&conn, 0, sizeof(struct ldap_conn));
+	ret = do_reconnect (LOGOPT_ANY, &conn, ctxt);
+	if (ret != NSS_STATUS_SUCCESS)
+		return 1;
+
+	unbind_ldap_connection(LOGOPT_ANY, &conn, ctxt);
 	return ret;
 }
 

--
To unsubscribe from this list: send the line "unsubscribe autofs" in

--
To unsubscribe from this list: send the line "unsubscribe autofs" in

#3 lore raw
Quoted 13 lines · Ian Kent
Quoted 12 lines · Ian Kent
Quoted 10 lines · Oscar Salvador

Right now we are just checking and parsing the ldap's configuration in lookup_init() function, and if everything goes well we return a zero value indicating that everything worked fine.

This has a problem, because in the case we set ldap method in /etc/nsswitch.conf, configuration parsing goes fine but ldap is not working, we will return a zero value in lookup_init() function, so autofs will move on, but then we will return NSS_STATUS_UNAVAIL in lookup_read_master() function once do_reconnect() fails, and because of this autofs will not be able to reread the maps.

Quoted 4 lines
Quoted 1 line

Umm ... I can't see a check in the current upstream NIS code?

I also don't understand the problem in the upstream code your trying to resolve. Can you try and describe it again please?

Sure, I'm going to try to explain it, hopefully better:

Let's say we set "files" and "nis" method in /etc/nsswitch.conf, just like this:

# grep auto /etc/nsswitch.conf
automount:	files nis

And then let's say we have this configuration:

# grep -v '^#' /etc/auto.master
/- auto.host
+auto.master
# cat /etc/auto.host
/data1  xx.xx.xx.xx:/mnt/export1
/data2  xx.xx.xx.xx:/mnt/export2
/data3	xx.xx.xx.xx:/mnt/export3

Once we start autofs, we should have the mounts:

auto.host on /data3 type autofs (rw,relatime,fd=7,pgrp=2128,timeout=300,minproto=5,maxproto=5,direct) auto.host on /data2 type autofs (rw,relatime,fd=7,pgrp=2128,timeout=300,minproto=5,maxproto=5,direct) auto.host on /data1 type autofs (rw,relatime,fd=7,pgrp=2128,timeout=300,minproto=5,maxproto=5,direct)

Then, let's say we want to remove one mount, so we comment one of the mounts in /etc/auto.host, and then we send the -HUP signal to the autofs process.

The result of signaling autofs -HUP is that it re-reads the config.

So, we call:

lookup_nss_read_master() -> do_read_master() -> open_lookup() ->

fails?

     |->yes: return NSS_STATUS_NOTFOUND
     |-> no: lookup->lookup_read_master()

The point of open_lookup() is to try to initialize the library by calling mod->lookup_init() (nis, ldap, ...), and in case it fails, we return NSS_STATUS_NOTFOUND, and do_read_master() returns NSS_STATUS_NOTFOUND too.

This is possible because in lookup_init() for lookup_yp.c we check if we are bound to a domain:

	if (!ctxt->domainname) {
                char *domainname;
                /* This should, but doesn't, take a const char ** */
                err = yp_get_default_domain(&domainname);
                if (err) {
                        logerr(MODPREFIX
                              "map %s: %s", ctxt->mapname, 
yperr_string(err));
                        ret = 1;
                        goto out;
                }
		...
		...
	}

In my case it fails because I'm not bound to any nis domain, so since mod->lookup_init() fails, open_lookup() returns NSS_STATUS_NOTFOUND and we just return instead of moving on.

Now, this doesn't work for ldap.
If we replace "nis" for "ldap" in /etc/nsswitch.conf, like:

# grep auto /etc/nsswitch.conf
automount:	files ldap

And then we comment one mount and try to reload autofs with HUP signal, the process is going to be different.

The point with ldap is that we don't really check if the configuration for ldap works or not, we just try to initialize it, but we don't check if we can connect to the ldap server and do a bind operation, so even if our ldap is broken lookup_init() from ldap is going to return success, letting us move on and call lookup->lookup_read_master(), which is going to fail hard in the first call to do_reconnect() in lookup_read_master() function.

The problem with this is that there is a big difference in failing in open_lookup() or failing in lookup->lookup_read_master(), and that's it if we return NSS_STATUS_NOTFOUND or NSS_STATUS_UNAVAIL.
In case we return the last one, lookup_nss_read_master() is going to do:

	if (result == NSS_STATUS_UNAVAIL)
		master->read_fail = 1;

and this will not let autofs re-read its configuration and remove the mounts we commented.

Given that said, what I wanted to achieve with my patch is to really check if we got a working ldap setup by trying to call do_reconnect() in do_init(), because it tries to connect to the ldap server found in the config and do a bind operation, which should be enough to test if our ldap works or not.

In my case, without the patch, if I try to reload the autofs config with the -HUP signal, autofs won't do it. With the patch, since I return NSS_STATUS_NOTFOUND if lookup_init() fails, autofs is able to re-read the config.

I hope I was able to explain it better this time.

This being said, I'm not sure if we way I chose to fix this is the best one.

Thanks
--
To unsubscribe from this list: send the line "unsubscribe autofs" in

maintainer
#4 lore raw
Quoted 66 lines · Oscar Salvador
Quoted 13 lines · Ian Kent
Quoted 12 lines · Ian Kent
Quoted 10 lines · Oscar Salvador

Right now we are just checking and parsing the ldap's configuration in lookup_init() function, and if everything goes well we return a zero value indicating that everything worked fine.

This has a problem, because in the case we set ldap method in /etc/nsswitch.conf, configuration parsing goes fine but ldap is not working, we will return a zero value in lookup_init() function, so autofs will move on, but then we will return NSS_STATUS_UNAVAIL in lookup_read_master() function once do_reconnect() fails, and because of this autofs will not be able to reread the maps.

Quoted 4 lines
Quoted 1 line

Umm ... I can't see a check in the current upstream NIS code?

I also don't understand the problem in the upstream code your trying to resolve. Can you try and describe it again please?

Sure, I'm going to try to explain it, hopefully better:

Let's say we set "files" and "nis" method in /etc/nsswitch.conf, just like this:

# grep auto /etc/nsswitch.conf
automount:    files nis

And then let's say we have this configuration:

# grep -v '^#' /etc/auto.master
/- auto.host
+auto.master
# cat /etc/auto.host
/data1  xx.xx.xx.xx:/mnt/export1
/data2  xx.xx.xx.xx:/mnt/export2
/data3    xx.xx.xx.xx:/mnt/export3

Once we start autofs, we should have the mounts:

auto.host on /data3 type autofs (rw,relatime,fd=7,pgrp=2128,timeout=300,minproto=5,maxproto=5,direct) auto.host on /data2 type autofs (rw,relatime,fd=7,pgrp=2128,timeout=300,minproto=5,maxproto=5,direct) auto.host on /data1 type autofs (rw,relatime,fd=7,pgrp=2128,timeout=300,minproto=5,maxproto=5,direct)

Then, let's say we want to remove one mount, so we comment one of the mounts in /etc/auto.host, and then we send the -HUP signal to the autofs process.

The result of signaling autofs -HUP is that it re-reads the config.

So, we call:

lookup_nss_read_master() -> do_read_master() -> open_lookup() ->

fails?

     |->yes: return NSS_STATUS_NOTFOUND
     |-> no: lookup->lookup_read_master()

The point of open_lookup() is to try to initialize the library by calling mod->lookup_init() (nis, ldap, ...), and in case it fails, we return NSS_STATUS_NOTFOUND, and do_read_master() returns NSS_STATUS_NOTFOUND too.

Quoted 6 lines

This is possible because in lookup_init() for lookup_yp.c we check if we are bound to a domain:

    if (!ctxt->domainname) {
                char *domainname;
                /* This should, but doesn't, take a const char ** */
                err = yp_get_default_domain(&domainname);

I didn't check if that returns bound domain of configured domain. I'll do that.

Quoted 40 lines
                if (err) {
                        logerr(MODPREFIX
                              "map %s: %s", ctxt->mapname, yperr_string(err));
                        ret = 1;
                        goto out;
                }
        ...
        ...
    }

In my case it fails because I'm not bound to any nis domain, so since mod->lookup_init() fails, open_lookup() returns NSS_STATUS_NOTFOUND and we just return instead of moving on.

Now, this doesn't work for ldap.
If we replace "nis" for "ldap" in /etc/nsswitch.conf, like:

# grep auto /etc/nsswitch.conf
automount:    files ldap

And then we comment one mount and try to reload autofs with HUP signal, the process is going to be different.

The point with ldap is that we don't really check if the configuration for ldap works or not, we just try to initialize it, but we don't check if we can connect to the ldap server and do a bind operation, so even if our ldap is broken lookup_init() from ldap is going to return success, letting us move on and call lookup->lookup_read_master(), which is going to fail hard in the first call to do_reconnect() in lookup_read_master() function.

The problem with this is that there is a big difference in failing in open_lookup() or failing in lookup->lookup_read_master(), and that's it if we return NSS_STATUS_NOTFOUND or NSS_STATUS_UNAVAIL.
In case we return the last one, lookup_nss_read_master() is going to do:

    if (result == NSS_STATUS_UNAVAIL)
        master->read_fail = 1;

and this will not let autofs re-read its configuration and remove the mounts we commented.

Quoted 8 lines

Given that said, what I wanted to achieve with my patch is to really check if we got a working ldap setup by trying to call do_reconnect() in do_init(), because it tries to connect to the ldap server found in the config and do a bind operation, which should be enough to test if our ldap works or not.

In my case, without the patch, if I try to reload the autofs config with the -HUP signal, autofs won't do it. With the patch, since I return NSS_STATUS_NOTFOUND if lookup_init() fails, autofs is able to re-read the config.

I hope I was able to explain it better this time.

Yes, this is what I was hoping for, thanks.

Quoted 2 lines

This being said, I'm not sure if we way I chose to fix this is the best one.

I need to check because there has been quite a bit of recent change in this area.

The unfortunate thing is lookup_init() probably should be returning NSS_STATUS_UNAVAIL for the overall semantics of how this is meant to work.

That it returns NSS_STATUS_NOTFOUND is not strictly right because lookup_init() can't know if the master map being looked for exists or not so an NSS_STATUS_UNAVAIL should allow it to continue on.

What's worse is the semantics change a bit between startup and map re-read and a change to one case can break the other.

Let me have a look around.

Ian
--
To unsubscribe from this list: send the line "unsubscribe autofs" in

maintainer
#5 lore raw
Quoted 69 lines · Ian Kent
Quoted 68 lines · Oscar Salvador
Quoted 13 lines · Ian Kent
Quoted 12 lines · Ian Kent
Quoted 10 lines · Oscar Salvador

Right now we are just checking and parsing the ldap's configuration in lookup_init() function, and if everything goes well we return a zero value indicating that everything worked fine.

This has a problem, because in the case we set ldap method in /etc/nsswitch.conf, configuration parsing goes fine but ldap is not working, we will return a zero value in lookup_init() function, so autofs will move on, but then we will return NSS_STATUS_UNAVAIL in lookup_read_master() function once do_reconnect() fails, and because of this autofs will not be able to reread the maps.

Quoted 4 lines
Quoted 1 line

Umm ... I can't see a check in the current upstream NIS code?

I also don't understand the problem in the upstream code your trying to resolve. Can you try and describe it again please?

Sure, I'm going to try to explain it, hopefully better:

Let's say we set "files" and "nis" method in /etc/nsswitch.conf, just like this:

# grep auto /etc/nsswitch.conf
automount:    files nis

And then let's say we have this configuration:

# grep -v '^#' /etc/auto.master
/- auto.host
+auto.master
# cat /etc/auto.host
/data1  xx.xx.xx.xx:/mnt/export1
/data2  xx.xx.xx.xx:/mnt/export2
/data3    xx.xx.xx.xx:/mnt/export3

Once we start autofs, we should have the mounts:

auto.host on /data3 type autofs (rw,relatime,fd=7,pgrp=2128,timeout=300,minproto=5,maxproto=5,direct) auto.host on /data2 type autofs (rw,relatime,fd=7,pgrp=2128,timeout=300,minproto=5,maxproto=5,direct) auto.host on /data1 type autofs (rw,relatime,fd=7,pgrp=2128,timeout=300,minproto=5,maxproto=5,direct)

Then, let's say we want to remove one mount, so we comment one of the mounts in /etc/auto.host, and then we send the -HUP signal to the autofs process.

The result of signaling autofs -HUP is that it re-reads the config.

So, we call:

lookup_nss_read_master() -> do_read_master() -> open_lookup() ->

fails?

     |->yes: return NSS_STATUS_NOTFOUND
     |-> no: lookup->lookup_read_master()

The point of open_lookup() is to try to initialize the library by calling mod->lookup_init() (nis, ldap, ...), and in case it fails, we return NSS_STATUS_NOTFOUND, and do_read_master() returns NSS_STATUS_NOTFOUND too.

This is possible because in lookup_init() for lookup_yp.c we check if we are bound to a domain:

Quoted 7 lines
Quoted 4 lines
    if (!ctxt->domainname) {
                char *domainname;
                /* This should, but doesn't, take a const char ** */
                err = yp_get_default_domain(&domainname);

I didn't check if that returns bound domain of configured domain. I'll do that.

yp_get_default_domain() doesn't do a network connection, it just gets the system configured NIS domain name, it can't be relied upon to check connectivity.

Quoted 31 lines
Quoted 31 lines
                if (err) {
                        logerr(MODPREFIX
                              "map %s: %s", ctxt->mapname, yperr_string(err));
                        ret = 1;
                        goto out;
                }
        ...
        ...
    }

In my case it fails because I'm not bound to any nis domain, so since mod->lookup_init() fails, open_lookup() returns NSS_STATUS_NOTFOUND and we just return instead of moving on.

Now, this doesn't work for ldap.
If we replace "nis" for "ldap" in /etc/nsswitch.conf, like:

# grep auto /etc/nsswitch.conf
automount:    files ldap

And then we comment one mount and try to reload autofs with HUP signal, the process is going to be different.

The point with ldap is that we don't really check if the configuration for ldap works or not, we just try to initialize it, but we don't check if we can connect to the ldap server and do a bind operation, so even if our ldap is broken lookup_init() from ldap is going to return success, letting us move on and call lookup->lookup_read_master(), which is going to fail hard in the first call to do_reconnect() in lookup_read_master() function.

Quoted 8 lines
Quoted 8 lines

The problem with this is that there is a big difference in failing in open_lookup() or failing in lookup->lookup_read_master(), and that's it if we return NSS_STATUS_NOTFOUND or NSS_STATUS_UNAVAIL.
In case we return the last one, lookup_nss_read_master() is going to do:

    if (result == NSS_STATUS_UNAVAIL)
        master->read_fail = 1;

and this will not let autofs re-read its configuration and remove the mounts we commented.

I can't duplicate this with the current upstream source but I do think there was a problem with HUP signal processing recently.

In any case, for the HUP signal processing master->read_fail = 1 shouldn't prevent the mounted mounts update ....

       lookup_nss_read_master(master, age);
       ...

       if (!master->read_fail)
                master_mount_mounts(master, age, readall);
        else {
                master->read_fail = 0;
                /* HUP signal sets readall == 1 only */   <== see here.
                if (!readall)
                        return 0;
                else
                        master_mount_mounts(master, age, readall);
        }
Quoted 16 lines
Quoted 10 lines

Given that said, what I wanted to achieve with my patch is to really check if we got a working ldap setup by trying to call do_reconnect() in do_init(), because it tries to connect to the ldap server found in the config and do a bind operation, which should be enough to test if our ldap works or not.

In my case, without the patch, if I try to reload the autofs config with the -HUP signal, autofs won't do it. With the patch, since I return NSS_STATUS_NOTFOUND if lookup_init() fails, autofs is able to re-read the config.

I hope I was able to explain it better this time.

Yes, this is what I was hoping for, thanks.

Quoted 2 lines

This being said, I'm not sure if we way I chose to fix this is the best one.

Quoted 8 lines

I need to check because there has been quite a bit of recent change in this area.

The unfortunate thing is lookup_init() probably should be returning NSS_STATUS_UNAVAIL for the overall semantics of how this is meant to work.

That it returns NSS_STATUS_NOTFOUND is not strictly right because lookup_init() can't know if the master map being looked for exists or not so an NSS_STATUS_UNAVAIL should allow it to continue on.

I think this return isn't right, it probably should be returning NSS_STATUS_UNAVAIL but I'd rather not change that right now either.

Quoted 5 lines

What's worse is the semantics change a bit between startup and map re-read and a change to one case can break the other.

Let me have a look around.

Could you have a look at:
autofs-5.1.2-wait-for-master-map-available-at-start.patch autofs-5.1.2-add-master-read-wait-option.patch autofs-5.1.2-work-around-sss-startup-delay.patch autofs-5.1.2-add-sss-master-map-wait-config-option.patch autofs-5.1.2-fix-included-master-map-not-found-return.patch autofs-5.1.2-dont-fail-on-master-map-read-fail-timeout.patch autofs-5.1.2-set-sane-default-master-read-wait-timeout.patch autofs-5.1.2-dont-return-until-after-master-map-retry-read.patch autofs-5.1.2-make-lookup_nss_read_master-return-nss-status.patch

found at https://www.kernel.org/pub/linux/daemons/autofs/v5/patches-5.1.3/.

It's true they are mostly related to startup problems when the network is not yet available and they have an annoying side effect of being unforgiving of mis-configured nsswitch.conf (a file that has a broken configuration by default and which autofs doesn't own and probably won't change to fix it) but overall I think they are an improvement.

Do you have these in your test package?

I can't reproduce it with a daemon built with these and they also fixed a lookup failure for nsswitch source ordering although it wasn't originally intended to do so.

Ian
--
To unsubscribe from this list: send the line "unsubscribe autofs" in

maintainer
#6 lore raw
Quoted 12 lines · Ian Kent

Could you have a look at:
autofs-5.1.2-wait-for-master-map-available-at-start.patch autofs-5.1.2-add-master-read-wait-option.patch autofs-5.1.2-work-around-sss-startup-delay.patch autofs-5.1.2-add-sss-master-map-wait-config-option.patch autofs-5.1.2-fix-included-master-map-not-found-return.patch autofs-5.1.2-dont-fail-on-master-map-read-fail-timeout.patch autofs-5.1.2-set-sane-default-master-read-wait-timeout.patch autofs-5.1.2-dont-return-until-after-master-map-retry-read.patch autofs-5.1.2-make-lookup_nss_read_master-return-nss-status.patch

Quoted 1 line

found at https://www.kernel.org/pub/linux/daemons/autofs/v5/patches-5.1.3/.

I think there might have been a couple of other patches that went in quite a while ago (during a false start on this) but I can't remember what they were now.

I'll look around further if you do decide to work with the patch series.

Ian
--
To unsubscribe from this list: send the line "unsubscribe autofs" in

maintainer
#7 lore raw
Quoted 13 lines · Ian Kent
Quoted 12 lines · Ian Kent

Could you have a look at:
autofs-5.1.2-wait-for-master-map-available-at-start.patch autofs-5.1.2-add-master-read-wait-option.patch autofs-5.1.2-work-around-sss-startup-delay.patch autofs-5.1.2-add-sss-master-map-wait-config-option.patch autofs-5.1.2-fix-included-master-map-not-found-return.patch autofs-5.1.2-dont-fail-on-master-map-read-fail-timeout.patch autofs-5.1.2-set-sane-default-master-read-wait-timeout.patch autofs-5.1.2-dont-return-until-after-master-map-retry-read.patch autofs-5.1.2-make-lookup_nss_read_master-return-nss-status.patch

Quoted 1 line
Quoted 1 line

found at https://www.kernel.org/pub/linux/daemons/autofs/v5/patches-5.1.3/.

And there's this one too. autofs-5.1.2-fix-work-around-sss-startup-delay.patch

Ian
--
To unsubscribe from this list: send the line "unsubscribe autofs" in

#8 lore raw
Quoted 14 lines · Ian Kent
Quoted 13 lines · Ian Kent
Quoted 12 lines · Ian Kent

Could you have a look at:
autofs-5.1.2-wait-for-master-map-available-at-start.patch autofs-5.1.2-add-master-read-wait-option.patch autofs-5.1.2-work-around-sss-startup-delay.patch autofs-5.1.2-add-sss-master-map-wait-config-option.patch autofs-5.1.2-fix-included-master-map-not-found-return.patch autofs-5.1.2-dont-fail-on-master-map-read-fail-timeout.patch autofs-5.1.2-set-sane-default-master-read-wait-timeout.patch autofs-5.1.2-dont-return-until-after-master-map-retry-read.patch autofs-5.1.2-make-lookup_nss_read_master-return-nss-status.patch

Quoted 4 lines
Quoted 1 line
Quoted 1 line

found at https://www.kernel.org/pub/linux/daemons/autofs/v5/patches-5.1.3/.

And there's this one too. autofs-5.1.2-fix-work-around-sss-startup-delay.patch

I can reproduce this with the latest version of autofs from git.

My setup is quite simple:

# git clone https://git.kernel.org/pub/scm/linux/storage/autofs/autofs.git
# ./configure && make && make install
# vim /etc/auto.master
/- auto.host
+auto.master
# vim /etc/auto.host
/data1  xx.xx.xx.xx:/mnt/export1
/data2  xx.xx.xx.xx:/mnt/export2
/data3	xx.xx.xx.xx:/mnt/export3
# grep autom /etc/nsswitch.conf
automount:	files ldap
# automount -d -v -f
# output of mounts (other terminal)
auto.host on /data3 type autofs (rw,relatime,fd=7,pgrp=29843,timeout=300,minproto=5,maxproto=5,direct)
auto.host on /data2 type autofs (rw,relatime,fd=7,pgrp=29843,timeout=300,minproto=5,maxproto=5,direct)
auto.host on /data1 type autofs (rw,relatime,fd=7,pgrp=29843,timeout=300,minproto=5,maxproto=5,direct)
# comment one mount in /etc/auto.host
# kill -HUP $(pidof automount)

Outcome:

do_hup_signal() -> do_read_master()[readall = 1] -> master_read_master()

in master_read_master we set master->read_fail to 0 again and we call master_mount_mounts()

master_mount_mounts () -> check_update_map_sources()

in check_update_map_sources() we set map_stale and source->stale to 1:

	if (readall)
                map_stale = 1;

	...
	...

	while (source) {
                if (readall)
                        source->stale = 1;
		...
	}

Then we add a new task to re-read the map:

	if (map_stale)
                st_add_task(ap, ST_READMAP);

st_readmap() -> do_readmap() -> lookup_nss_read_map()

Within lookup_nss_read_map(), we call read_map_source():

	list_for_each(p, head) {
                 this = list_entry(p, struct nss_source, list);
                  ...
                  ...
                  result = read_map_source(this, ap, map, age);
                  ...
         }

read_map_source() is being called for "files" and "ldap"

in the case with "files" everything goes fine, and back to lookup_nss_read_map() we set at_least_one to 1

	if(result == NSS_STATUS_SUCCESS) {
		at_least_one = 1;
		result = NSS_STATUS_TRYAGAIN;
	}

in the case with "ldap":

read_map_source()-> read_source_instance() -> do_read_map() -> open_lookup()

since open_lookup() for ldaps succeeds, we move on, and we call:

status = lookup->lookup_read_map(ap, age, lookup->context);

lookup->lookup_read_map() returns NSS_STATUS_UNAVAIL and we set map->stale to 0:

	if (status != NSS_STATUS_SUCCESS)
                map->stale = 0;

back to lookup_nss_read_map(), since result was NSS_STATUS_UNAVAIL, we set map->stale = 0 again

	if (result == NSS_STATUS_UNAVAIL)
		map->stale = 0;

back in do_readmap() we check if the map got stale or not, and in case it got, we do a readmap again:

	while (map) {
		if (!map->stale) {
			map = map->next;
			continue;
		}

		...
		...
		while (me) {
			do_readmap_mount(ap, mnts, map, me, now);
			me = cache_enumerate(mc, me);
		}
	}

but since we zeroed map->stale before, we bypass it and we don't do that.
Another fixup that doesn't not imply messing with return values of open_lookup() from ldap module could be this one:

daemon/lookup.c +5 -0
index 583d3d3..82f3a58 100644
@@ -550,6 +550,7 @@ int lookup_nss_read_map(struct autofs_point *ap, struct map_source *source, time
        struct map_source *map;
        enum nsswitch_status status;
        unsigned int at_least_one = 0;
+       unsigned int old_stale_val;
        int result = 0;
 
        /*
@@ -615,6 +616,7 @@ int lookup_nss_read_map(struct autofs_point *ap, struct map_source *source, time
 
                pthread_cleanup_push(nsslist_cleanup, &nsslist);
                head = &nsslist;
+               old_stale_val = map->stale;
                list_for_each(p, head) {
                        this = list_entry(p, struct nss_source, list);
 
@@ -654,6 +656,9 @@ int lookup_nss_read_map(struct autofs_point *ap, struct map_source *source, time
                if (!map)
                        break;
 
+               if (at_least_one)
+                       map->stale = old_stale_val;
+
                map = map->next;
        }
 

I tested it and works for me.

So in case that we succeed at least with one lookup method, we put back the original stale value (that gets zeroed in do_read_map() if lookup->lookup_read_map() returns something different than NSS_STATUS_SUCCESS)

In this case we don't skip the map in do_readmap()

        while (map) {
                if (!map->stale) {			<-- here
                        map = map->next;
                        continue;
                }

                ...
                ...
                while (me) {
                        do_readmap_mount(ap, mnts, map, me, now);
                        me = cache_enumerate(mc, me);
                }
        }

Thanks
--
To unsubscribe from this list: send the line "unsubscribe autofs" in

maintainer
#9 lore raw
Quoted 17 lines · Oscar Salvador
Quoted 14 lines · Ian Kent
Quoted 13 lines · Ian Kent
Quoted 12 lines · Ian Kent

Could you have a look at:
autofs-5.1.2-wait-for-master-map-available-at-start.patch autofs-5.1.2-add-master-read-wait-option.patch autofs-5.1.2-work-around-sss-startup-delay.patch autofs-5.1.2-add-sss-master-map-wait-config-option.patch autofs-5.1.2-fix-included-master-map-not-found-return.patch autofs-5.1.2-dont-fail-on-master-map-read-fail-timeout.patch autofs-5.1.2-set-sane-default-master-read-wait-timeout.patch autofs-5.1.2-dont-return-until-after-master-map-retry-read.patch autofs-5.1.2-make-lookup_nss_read_master-return-nss-status.patch

Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line

found at https://www.kernel.org/pub/linux/daemons/autofs/v5/patches-5.1.3/.

And there's this one too. autofs-5.1.2-fix-work-around-sss-startup-delay.patch

I can reproduce this with the latest version of autofs from git.

Sounds like my understanding of the problem is not right.

But it could also be I didn't actually use LDAP, I used NIS, I'll have to try harder to replicate it.

Quoted 27 lines

My setup is quite simple:

# git clone https://git.kernel.org/pub/scm/linux/storage/autofs/autofs.git
# ./configure && make && make install
# vim /etc/auto.master
/- auto.host
+auto.master
# vim /etc/auto.host
/data1  xx.xx.xx.xx:/mnt/export1
/data2  xx.xx.xx.xx:/mnt/export2
/data3    xx.xx.xx.xx:/mnt/export3
# grep autom /etc/nsswitch.conf
automount:    files ldap
# automount -d -v -f
# output of mounts (other terminal)
auto.host on /data3 type autofs (rw,relatime,fd=7,pgrp=29843,timeout=300,minproto=5,maxproto=5,direct)
auto.host on /data2 type autofs (rw,relatime,fd=7,pgrp=29843,timeout=300,minproto=5,maxproto=5,direct)
auto.host on /data1 type autofs (rw,relatime,fd=7,pgrp=29843,timeout=300,minproto=5,maxproto=5,direct)
Quoted 8 lines
# comment one mount in /etc/auto.host
# kill -HUP $(pidof automount)

Outcome:

do_hup_signal() -> do_read_master()[readall = 1] -> master_read_master()

I've only looked at it up to this point because I thought update meant removed mount entry not being umounted but I think your actually saying subsequent map not being re-read and so not being updated.

Is that what you mean?

Ian

--
To unsubscribe from this list: send the line "unsubscribe autofs" in

maintainer
#10 lore raw
Quoted 24 lines · Ian Kent
Quoted 23 lines · Oscar Salvador
Quoted 18 lines · Ian Kent
Quoted 13 lines · Ian Kent
Quoted 12 lines · Ian Kent

Could you have a look at:
autofs-5.1.2-wait-for-master-map-available-at-start.patch autofs-5.1.2-add-master-read-wait-option.patch autofs-5.1.2-work-around-sss-startup-delay.patch autofs-5.1.2-add-sss-master-map-wait-config-option.patch autofs-5.1.2-fix-included-master-map-not-found-return.patch autofs-5.1.2-dont-fail-on-master-map-read-fail-timeout.patch autofs-5.1.2-set-sane-default-master-read-wait-timeout.patch autofs-5.1.2-dont-return-until-after-master-map-retry-read.patch autofs-5.1.2-make-lookup_nss_read_master-return-nss-status.patch

Quoted 1 line
Quoted 1 line

found at https://www.kernel.org/pub/linux/daemons/autofs/v5/patches-5.1.3/.

And there's this one too. autofs-5.1.2-fix-work-around-sss-startup-delay.patch

Quoted 6 lines
Quoted 1 line

I can reproduce this with the latest version of autofs from git.

Sounds like my understanding of the problem is not right.

But it could also be I didn't actually use LDAP, I used NIS, I'll have to try harder to replicate it.

Success, I have now been able to reproduce this, give me some time to work out what's going on.

Quoted 41 lines
Quoted 28 lines

My setup is quite simple:

# git clone https://git.kernel.org/pub/scm/linux/storage/autofs/autofs.git
# ./configure && make && make install
# vim /etc/auto.master
/- auto.host
+auto.master
# vim /etc/auto.host
/data1  xx.xx.xx.xx:/mnt/export1
/data2  xx.xx.xx.xx:/mnt/export2
/data3    xx.xx.xx.xx:/mnt/export3
# grep autom /etc/nsswitch.conf
automount:    files ldap
# automount -d -v -f
# output of mounts (other terminal)
auto.host on /data3 type autofs (rw,relatime,fd=7,pgrp=29843,timeout=300,minproto=5,maxproto=5,direct)
auto.host on /data2 type autofs (rw,relatime,fd=7,pgrp=29843,timeout=300,minproto=5,maxproto=5,direct)
auto.host on /data1 type autofs (rw,relatime,fd=7,pgrp=29843,timeout=300,minproto=5,maxproto=5,direct)
Quoted 8 lines
# comment one mount in /etc/auto.host
# kill -HUP $(pidof automount)

Outcome:

do_hup_signal() -> do_read_master()[readall = 1] -> master_read_master()

I've only looked at it up to this point because I thought update meant removed mount entry not being umounted but I think your actually saying subsequent map not being re-read and so not being updated.

Quoted 7 lines

Is that what you mean?

Ian

--
To unsubscribe from this list: send the line "unsubscribe autofs" in

--
To unsubscribe from this list: send the line "unsubscribe autofs" in

#11 lore raw
In reply to Ian Kent
Quoted 24 lines · Ian Kent
Quoted 23 lines · Oscar Salvador
Quoted 18 lines · Ian Kent
Quoted 13 lines · Ian Kent
Quoted 12 lines · Ian Kent

Could you have a look at:
autofs-5.1.2-wait-for-master-map-available-at-start.patch autofs-5.1.2-add-master-read-wait-option.patch autofs-5.1.2-work-around-sss-startup-delay.patch autofs-5.1.2-add-sss-master-map-wait-config-option.patch autofs-5.1.2-fix-included-master-map-not-found-return.patch autofs-5.1.2-dont-fail-on-master-map-read-fail-timeout.patch autofs-5.1.2-set-sane-default-master-read-wait-timeout.patch autofs-5.1.2-dont-return-until-after-master-map-retry-read.patch autofs-5.1.2-make-lookup_nss_read_master-return-nss-status.patch

Quoted 1 line
Quoted 1 line

found at https://www.kernel.org/pub/linux/daemons/autofs/v5/patches-5.1.3/.

And there's this one too. autofs-5.1.2-fix-work-around-sss-startup-delay.patch

Quoted 6 lines
Quoted 1 line

I can reproduce this with the latest version of autofs from git.

Sounds like my understanding of the problem is not right.

But it could also be I didn't actually use LDAP, I used NIS, I'll have to try harder to replicate it.

With NIS you will not have any problem, it just works because open_lookup() returns NSS_STATUS_NOTFOUND in case it fails. But if you replace "nis" with "ldap" in /etc/nsswitch.conf, then you should be able to see the issue.

Quoted 34 lines
Quoted 34 lines

My setup is quite simple:

# git clone https://git.kernel.org/pub/scm/linux/storage/autofs/autofs.git
# ./configure && make && make install
# vim /etc/auto.master
/- auto.host
+auto.master
# vim /etc/auto.host
/data1  xx.xx.xx.xx:/mnt/export1
/data2  xx.xx.xx.xx:/mnt/export2
/data3    xx.xx.xx.xx:/mnt/export3
# grep autom /etc/nsswitch.conf
automount:    files ldap
# automount -d -v -f
# output of mounts (other terminal)
auto.host on /data3 type autofs (rw,relatime,fd=7,pgrp=29843,timeout=300,minproto=5,maxproto=5,direct)
auto.host on /data2 type autofs (rw,relatime,fd=7,pgrp=29843,timeout=300,minproto=5,maxproto=5,direct)
auto.host on /data1 type autofs (rw,relatime,fd=7,pgrp=29843,timeout=300,minproto=5,maxproto=5,direct)
# comment one mount in /etc/auto.host
# kill -HUP $(pidof automount)

Outcome:

Quoted 7 lines
Quoted 1 line

do_hup_signal() -> do_read_master()[readall = 1] -> master_read_master()

I've only looked at it up to this point because I thought update meant removed mount entry not being umounted but I think your actually saying subsequent map not being re-read and so not being updated.

Is that what you mean?

No, sorry, maybe I didn't explain it correctly.
I'm not talking about mountpoints not being unmounted, but about maps not being updated.

f.i:

If I have

auto.host on /data3 type autofs (rw,relatime,fd=7,pgrp=29843,timeout=300,minproto=5,maxproto=5,direct) auto.host on /data2 type autofs (rw,relatime,fd=7,pgrp=29843,timeout=300,minproto=5,maxproto=5,direct) auto.host on /data1 type autofs (rw,relatime,fd=7,pgrp=29843,timeout=300,minproto=5,maxproto=5,direct)

then I comment /data3 in /etc/auto.host, and then I signal autofs with -HUP.
autofs will re-read the maps and update them, so /data3 will not appear anymore when executing "mount"

This works unless you have two lookup methods in /etc/nsswitch.conf, one of them being ldap and having ldap not being able to connect to its server.

This is why I came up with the latest patch from my previous comment.
In case you have two lookup methods, and one of them is working and the other not, you still should be able to update the maps since you at least got one method working.

Thanks
Oscar
--
To unsubscribe from this list: send the line "unsubscribe autofs" in