Skip to contentSkip to conversation

Update to utils_expand_path as realpath(3) for non-existing directories

PATCH New Raphaël Beamonte 3 participants 14 messages Nov 13, 2013 – Nov 14, 2013 9 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): 20131112195134.GB12183@thessa

#2 lore raw

cover Update to utils_expand_path as realpath(3) for non-existing directories

David,

Please find here an updated series of patches following your previous comments. These patches introduce updated versions of the utils_resolve_relative and utils_expand_path functions, as well as unit tests to verify their behavior.

Thanks,
Raphaël

Raphaël Beamonte (4):
Introduce a new utils_resolve_relative function
Tests: Add test_utils_resolve_relative to unit tests Correct the behavior of the utils_expand_path function
Tests: Add test_utils_expand_path to unit tests

 .gitignore                               |    2 +
 src/common/utils.c                       |  154 +++++++++++++++++++++++----
 src/common/utils.h                       |    1 +
 tests/unit/Makefile.am                   |   20 +++-
 tests/unit/test_utils_expand_path.c      |  170 ++++++++++++++++++++++++++++++
 tests/unit/test_utils_resolve_relative.c |   98 +++++++++++++++++
 tests/unit_tests                         |    2 +
 7 files changed, 421 insertions(+), 26 deletions(-)
 create mode 100644 tests/unit/test_utils_expand_path.c
 create mode 100644 tests/unit/test_utils_resolve_relative.c
Signature · 7 lines
1.7.10.4


_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw

Re: [lttng-tools PATCH 3/4] Correct the behavior of the utils_expand_path function

----- Original Message -----

Quoted 6 lines

From: "Raphaël Beamonte" <[email protected]>
To: [email protected]
Cc: "Raphaël Beamonte" <[email protected]>, [email protected]
Sent: Wednesday, November 13, 2013 12:34:37 AM
Subject: [lttng-dev] [lttng-tools PATCH 3/4] Correct the behavior of the utils_expand_path function

Quoted 6 lines

Even if the utils_expand_path function was intended to allow to use unexistent directory paths, it was in fact only working for some kind of arguments. Paths like "foo", "bar/" or "bar/foo" when the "bar" directory does not exist wasn't working. This patch introduce a new way to expand paths in this function that also works properly for these kind of arguments.

I might be missing something, but how about, instead:

1) if path start with /, directly pass it to realpath()
2) if path does not start with /, prepend getcwd() to it, and then call realpath() on the result.

If we can do the same result as this patch without reimplementing tricky string handling, we should really aim for re-using existing libraries.

Thoughts ?

Thanks,

Mathieu

Quoted 131 lines
---
 src/common/utils.c |   86
 ++++++++++++++++++++++++++++++++++++++--------------
 1 file changed, 64 insertions(+), 22 deletions(-)
src/common/utils.c +64 -23
index 4ccf36a..6938a5a 100644
@@ -112,7 +112,7 @@ error:
 LTTNG_HIDDEN
 char *utils_expand_path(const char *path)
 {
-	const char *end_path = path;
+	const char *end_path = NULL;
 	char *next, *cut_path = NULL, *expanded_path = NULL;
 
 	/* Safety net */
@@ -120,38 +120,80 @@ char *utils_expand_path(const char *path)
 		goto error;
 	}
 
-	/* Find last token delimited by '/' */
-	while ((next = strpbrk(end_path + 1, "/"))) {
-		end_path = next;
-	}
-
-	/* Cut last token from original path */
-	cut_path = strndup(path, end_path - path);
-
+	/* Allocate memory for the expanded path */
 	expanded_path = zmalloc(PATH_MAX);
 	if (expanded_path == NULL) {
 		PERROR("zmalloc expand path");
 		goto error;
 	}
 
-	expanded_path = realpath((char *)cut_path, expanded_path);
-	if (expanded_path == NULL) {
-		switch (errno) {
-		case ENOENT:
-			ERR("%s: No such file or directory", cut_path);
-			break;
-		default:
-			PERROR("realpath utils expand path");
-			break;
+	/* If given path is already absolute */
+	if (*path == '/') {
+		strncpy(expanded_path, path, PATH_MAX);
+	/* Else, we have some work to do */
+	} else {
+		/* Pointer to the last char of the path */
+		const char *last_char = path + strlen(path) - 1;
+
+		end_path = path;
+
+		/* Split part that will be resolved by realpath (relative path from
+		 * current directory using ./ or ../ only) and part that could not
+		 * (directory names)
+		 */
+		while ((next = strpbrk(end_path, "/")) && (next != last_char)) {
+			end_path = next + 1;
+			if (strncmp(end_path, "./", 2) != 0 &&
+					strncmp(end_path, "../", 3) != 0) {
+				break;
+			}
+		}
+
+		/* If this is the end of the string, and we still can resolve it */
+		if (strncmp(end_path, "..\0", 3) == 0 ||
+				strncmp(end_path, ".\0", 2) == 0) {
+			end_path += strlen(end_path);
+		}
+
+		/* If the end part is the whole path, we are in the current dir */
+		if (end_path == path) {
+			cut_path = strdup(".");
+		/* Else, cut the resolvable part from original path */
+		} else {
+			cut_path = strndup(path, end_path - path);
+		}
+
+		/* Resolve the canonical path of the first part of the path */
+		expanded_path = realpath((char *)cut_path, expanded_path);
+		if (expanded_path == NULL) {
+			switch (errno) {
+			case ENOENT:
+				ERR("%s: No such file or directory", cut_path);
+				break;
+			default:
+				PERROR("realpath utils expand path");
+				break;
+			}
+			goto error;
+		}
+
+		/* Add end part to expanded path if not empty */
+		if (*end_path != 0) {
+			strncat(expanded_path, "/", PATH_MAX - strlen(expanded_path) - 1);
+			strncat(expanded_path, end_path,
+					PATH_MAX - strlen(expanded_path) - 1);
 		}
-		goto error;
 	}
 
-	/* Add end part to expanded path */
-	strncat(expanded_path, end_path, PATH_MAX - strlen(expanded_path) - 1);
+	/* Resolve the internal './' and '../' strings */
+	next = utils_resolve_relative(expanded_path);
+	if (next == NULL) {
+		goto error;
+	}
 
+	free(expanded_path);
 	free(cut_path);
-	return expanded_path;
+	return next;
 
 error:
 	free(expanded_path);
--

1.7.10.4

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Signature · 8 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): [email protected]

#6 lore raw

Re: [lttng-tools PATCH 3/4] Correct the behavior of the utils_expand_path function

Quoted 8 lines

I might be missing something, but how about, instead:

1) if path start with /, directly pass it to realpath()
2) if path does not start with /, prepend getcwd() to it, and then call realpath() on the result.

If we can do the same result as this patch without reimplementing tricky string handling, we should really aim for re-using existing libraries.

Thoughts ?

Hello Mathieu,

Thing is realpath only works if the path exists (realpath(3)). I perhaps want to give as argument an absolute path that does not exist, or a path containing strings such as '../' or './'. In such cases, realpath would do no good, and if we don't accept to use unexisting paths, then there is no point in using a recursive mkdir. If fact, if realpath would work in such cases, using an 'if' would not even be necessary, we could just call realpath and let it resolve the current directory if necessary.

Hope it helps to understand,

Raphaël

A message that is not in the archive (sent off-list or not yet received): CAE_Gge2rcE7-pH9KBh9WAkfcQYM-aNw1iAb2brUeyjVvA=Nn3Q@mail.gmail.com

#8 lore raw

Re: [lttng-tools PATCH 3/4] Correct the behavior of the utils_expand_path function

----- Original Message -----

Quoted 28 lines

From: "Raphaël Beamonte" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>
Cc: "David Goulet" <[email protected]>, [email protected]
Sent: Wednesday, November 13, 2013 10:27:03 PM
Subject: Re: [lttng-dev] [lttng-tools PATCH 3/4] Correct the behavior of the utils_expand_path function

Quoted 10 lines

I might be missing something, but how about, instead:

1) if path start with /, directly pass it to realpath()
2) if path does not start with /, prepend getcwd() to it, and then call realpath() on the result.

If we can do the same result as this patch without reimplementing tricky string handling, we should really aim for re-using existing libraries.

Thoughts ?

Hello Mathieu,

Thing is realpath only works if the path exists (realpath(3)). I perhaps want to give as argument an absolute path that does not exist, or a path containing strings such as '../' or './'. In such cases, realpath would do no good, and if we don't accept to use unexisting paths, then there is no point in using a recursive mkdir. If fact, if realpath would work in such cases, using an 'if' would not even be necessary, we could just call realpath and let it resolve the current directory if necessary.

Quoted 1 line

Hope it helps to understand,

Yes, it does, thank you!

Mathieu

Quoted 3 lines

Raphaël

Signature · 8 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
#9 lore raw

Re: [lttng-tools PATCH 3/4] Correct the behavior of the utils_expand_path function

In reply to a message not in the archive

Here are comments about the patch, even though it has been merged.

----- Original Message -----

Quoted 61 lines

From: "Raphaël Beamonte" <[email protected]>
To: [email protected]
Cc: "Raphaël Beamonte" <[email protected]>, [email protected]
Sent: Wednesday, November 13, 2013 12:34:37 AM
Subject: [lttng-dev] [lttng-tools PATCH 3/4] Correct the behavior of the utils_expand_path function

Even if the utils_expand_path function was intended to allow to use unexistent directory paths, it was in fact only working for some kind of arguments. Paths like "foo", "bar/" or "bar/foo" when the "bar" directory does not exist wasn't working. This patch introduce a new way to expand paths in this function that also works properly for these kind of arguments.

---
 src/common/utils.c |   86
 ++++++++++++++++++++++++++++++++++++++--------------
 1 file changed, 64 insertions(+), 22 deletions(-)
src/common/utils.c +3 -18
index 4ccf36a..6938a5a 100644
@@ -112,7 +112,7 @@ error:
 LTTNG_HIDDEN
 char *utils_expand_path(const char *path)
 {
-	const char *end_path = path;
+	const char *end_path = NULL;
 	char *next, *cut_path = NULL, *expanded_path = NULL;
 
 	/* Safety net */
@@ -120,38 +120,80 @@ char *utils_expand_path(const char *path)
 		goto error;
 	}
 
-	/* Find last token delimited by '/' */
-	while ((next = strpbrk(end_path + 1, "/"))) {
-		end_path = next;
-	}
-
-	/* Cut last token from original path */
-	cut_path = strndup(path, end_path - path);
-
+	/* Allocate memory for the expanded path */
 	expanded_path = zmalloc(PATH_MAX);
 	if (expanded_path == NULL) {
 		PERROR("zmalloc expand path");
 		goto error;
 	}
 
-	expanded_path = realpath((char *)cut_path, expanded_path);
-	if (expanded_path == NULL) {
-		switch (errno) {
-		case ENOENT:
-			ERR("%s: No such file or directory", cut_path);
-			break;
-		default:
-			PERROR("realpath utils expand path");
-			break;
+	/* If given path is already absolute */

src/common/utils.c char *utils_expand_path(const char *path) · line 140

Quoted 10 lines
src/common/utils.c +10 -0
+	if (*path == '/') {
+		strncpy(expanded_path, path, PATH_MAX);
+	/* Else, we have some work to do */
+	} else {
+		/* Pointer to the last char of the path */
+		const char *last_char = path + strlen(path) - 1;
+
+		end_path = path;
+
+		/* Split part that will be resolved by realpath (relative path from

Comments should start with a:

/*
* Then start of comment...

src/common/utils.c char *utils_expand_path(const char *path) · line 153

Quoted 13 lines
src/common/utils.c +13 -0
+		 * current directory using ./ or ../ only) and part that could not
+		 * (directory names)
+		 */
+		while ((next = strpbrk(end_path, "/")) && (next != last_char)) {
+			end_path = next + 1;
+			if (strncmp(end_path, "./", 2) != 0 &&
+					strncmp(end_path, "../", 3) != 0) {
+				break;
+			}
+		}
+
+		/* If this is the end of the string, and we still can resolve it */
+		if (strncmp(end_path, "..\0", 3) == 0 ||

There is no point in ending a "" with \0, it implicitly ends with a \0 already.

src/common/utils.c char *utils_expand_path(const char *path) · line 154

Quoted 1 line
src/common/utils.c +1 -0
+				strncmp(end_path, ".\0", 2) == 0) {

Same here.

src/common/utils.c char *utils_expand_path(const char *path) · line 158

Quoted 4 lines
src/common/utils.c +4 -0
+			end_path += strlen(end_path);
+		}
+
+		/* If the end part is the whole path, we are in the current dir */

Not necessarily. What happens in the unlikely case someone passes a path with simply "/" ?

src/common/utils.c char *utils_expand_path(const char *path) · line 167

Quoted 9 lines
src/common/utils.c +9 -0
+		if (end_path == path) {
+			cut_path = strdup(".");
+		/* Else, cut the resolvable part from original path */
+		} else {
+			cut_path = strndup(path, end_path - path);
+		}
+
+		/* Resolve the canonical path of the first part of the path */
+		expanded_path = realpath((char *)cut_path, expanded_path);

So let's say we have:

"../blah/../blah"

so only "../" passed to realpath, or the entire "../blah/../blah" ? (let's suppose blah/ exists), right ?

Quoted 15 lines
src/common/utils.c +15 -0
+		if (expanded_path == NULL) {
+			switch (errno) {
+			case ENOENT:
+				ERR("%s: No such file or directory", cut_path);
+				break;
+			default:
+				PERROR("realpath utils expand path");
+				break;
+			}
+			goto error;
+		}
+
+		/* Add end part to expanded path if not empty */
+		if (*end_path != 0) {
+			strncat(expanded_path, "/", PATH_MAX - strlen(expanded_path) - 1);

src/common/utils.c char *utils_expand_path(const char *path) · line 189

Quoted 10 lines
src/common/utils.c +4 -3
+			strncat(expanded_path, end_path,
+					PATH_MAX - strlen(expanded_path) - 1);
 		}
-		goto error;
 	}
 
-	/* Add end part to expanded path */
-	strncat(expanded_path, end_path, PATH_MAX - strlen(expanded_path) - 1);
+	/* Resolve the internal './' and '../' strings */
+	next = utils_resolve_relative(expanded_path);

In the "../blah/../blah" case, here the other ../ would be dealt with by utils_resolve_relative, is that it ?

Thanks,

Mathieu

Quoted 20 lines
src/common/utils.c +5 -2
+	if (next == NULL) {
+		goto error;
+	}
 
+	free(expanded_path);
 	free(cut_path);
-	return expanded_path;
+	return next;
 
 error:
 	free(expanded_path);
--

1.7.10.4

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Signature · 8 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): [email protected]

#11 lore raw

Re: [lttng-tools PATCH 3/4] Correct the behavior of the utils_expand_path function

2013/11/14 Mathieu Desnoyers <[email protected]>:

Quoted 4 lines

Comments should start with a:

/*
* Then start of comment...

Right. Should I submit a new patch for that ?

Quoted 5 lines

There is no point in ending a "" with \0, it implicitly ends with a \0 already.

Quoted 1 line

+ strncmp(end_path, ".\0", 2) == 0) {

Same here.

Was done like that for clarity, as "implicit" is not "clear". Would it be preferable to hide it but still compare the first 2 chars of a 1 char string ? (respectively 3 chars of a 2 chars string)

Quoted 1 line

Not necessarily. What happens in the unlikely case someone passes a path with simply "/" ?

Then we're not in this arm of the "if". If the path starts by (or is) '/', it is considered as a "absolute path", and we're just in the arm of the "if" that starts with the comment "/* If given path is already absolute */"

Quoted 5 lines

So let's say we have:

"../blah/../blah"

so only "../" passed to realpath, or the entire "../blah/../blah" ? (let's suppose blah/ exists), right ?

Perhaps "blah/" exists, perhaps "blah/" doesn't exist. In both cases, it should not matter for our function. As we don't have any easy way to know that directly, we are not considering it for the realpath call, but only the "relative paths" that we can evaluate ("./" or "../").
You can look at the loop that starts with the comment "/* Split part that will be resolved by realpath", and see that in this case, only "../" will be passed to realpath.

Quoted 1 line

In the "../blah/../blah" case, here the other ../ would be dealt with by utils_resolve_relative, is that it ?

Exactly, as we can't pass it to realpath in case the directory does not exist.

Raphaël

A message that is not in the archive (sent off-list or not yet received): CAE_Gge30J9GEssQoKSO-KL=gm22HRhfCXJXFBorqYSHekEfVqQ@mail.gmail.com

#13 lore raw

Re: [lttng-tools PATCH 3/4] Correct the behavior of the utils_expand_path function

----- Original Message -----

Quoted 6 lines

From: "Raphaël Beamonte" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>
Cc: "David Goulet" <[email protected]>, [email protected]
Sent: Thursday, November 14, 2013 11:00:12 AM
Subject: Re: [lttng-dev] [lttng-tools PATCH 3/4] Correct the behavior of the utils_expand_path function

Quoted 7 lines

2013/11/14 Mathieu Desnoyers <[email protected]>:

Quoted 4 lines

Comments should start with a:

/*
* Then start of comment...

Right. Should I submit a new patch for that ?

A patch that apply on top of lttng-tools master.

Quoted 9 lines
Quoted 7 lines

There is no point in ending a "" with \0, it implicitly ends with a \0 already.

Quoted 1 line

+ strncmp(end_path, ".\0", 2) == 0) {

Same here.

Was done like that for clarity, as "implicit" is not "clear".

"" with \0 is just sign that someone does not understand what he is doing. It's not clearer.

Quoted 3 lines

Would it
be preferable to hide it but still compare the first 2 chars of a 1 char string ? (respectively 3 chars of a 2 chars string)

I think you want to keep the comparison with 3 chars, but you might want to test it further to confirm. The comparison between the implied \0 and a character in the other string should do what you are looking for.

Quoted 25 lines
Quoted 3 lines

Not necessarily. What happens in the unlikely case someone passes a path with simply "/" ?

Then we're not in this arm of the "if". If the path starts by (or is) '/', it is considered as a "absolute path", and we're just in the arm of the "if" that starts with the comment "/* If given path is already absolute */"

Quoted 6 lines

So let's say we have:

"../blah/../blah"

so only "../" passed to realpath, or the entire "../blah/../blah" ? (let's suppose blah/ exists), right ?

Perhaps "blah/" exists, perhaps "blah/" doesn't exist. In both cases, it should not matter for our function. As we don't have any easy way to know that directly, we are not considering it for the realpath call, but only the "relative paths" that we can evaluate ("./" or "../").
You can look at the loop that starts with the comment "/* Split part that will be resolved by realpath", and see that in this case, only "../" will be passed to realpath.

Quoted 5 lines
Quoted 2 lines

In the "../blah/../blah" case, here the other ../ would be dealt with by utils_resolve_relative, is that it ?

Exactly, as we can't pass it to realpath in case the directory does not exist.

OK. I look forward to seeing your reply for the other patch.

Thanks,

Mathieu

Quoted 3 lines

Raphaël

Signature · 8 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): [email protected]

#15 lore raw

Re: [lttng-tools PATCH 1/4] Introduce a new utils_resolve_relative function

Some comments...

----- Original Message -----

Quoted 23 lines

From: "Raphaël Beamonte" <[email protected]>
To: [email protected]
Cc: "Raphaël Beamonte" <[email protected]>, [email protected]
Sent: Wednesday, November 13, 2013 12:34:35 AM
Subject: [lttng-dev] [lttng-tools PATCH 1/4] Introduce a new utils_resolve_relative function

This function aims to resolve relative path such as './' and '../' in the middle of a path string. This allows to use paths such as '~/../test' that are received as '/home/x/../test' for instance.

---
 src/common/utils.c |   68
 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 src/common/utils.h |    1 +
 2 files changed, 69 insertions(+)
src/common/utils.c +0 -0
index da4c036..4ccf36a 100644

src/common/utils.c line 41

Quoted 6 lines
src/common/utils.c +3 -0
@@ -36,6 +36,74 @@
 #include "defaults.h"
 
 /*
+ * Resolve the './' and '../' strings in the middle of a path using
+ * our very own way to do it, so that it works even if the directory
+ * does not exist

We should document that this returns allocated memory which needs to be freed by the caller.

Also, we should clearly state that the only reason why we implement this is that realpath() only works on existing paths, which does not cover our use-case of having to resolve partially unexisting paths.

Quoted 45 lines
src/common/utils.c +45 -0
+ */
+LTTNG_HIDDEN
+char *utils_resolve_relative(const char *path)
+{
+	char *next, *previous, *slash, *start_path, *absolute_path = NULL;
+
+	/* Safety net */
+	if (path == NULL) {
+		goto error;
+	}
+
+	/* Allocate memory for the absolute path */
+	absolute_path = zmalloc(PATH_MAX);
+	if (absolute_path == NULL) {
+		PERROR("zmalloc expand path");
+		goto error;
+	}
+
+	/* Copy the path in the absolute path */
+	strncpy(absolute_path, path, PATH_MAX);
+
+	/* As long as we find '/./' in the path string */
+	while ((next = strstr(absolute_path, "/./"))) {
+
+		/* We prepare the start_path not containing it */
+		start_path = strndup(absolute_path, next - absolute_path);
+
+		/* And we concatenate it with the part after this string */
+		snprintf(absolute_path, PATH_MAX, "%s%s", start_path, next + 2);
+
+		free(start_path);
+	}
+
+	/* As long as we find '/../' in the path string */
+	while ((next = strstr(absolute_path, "/../"))) {
+		/* If the path starts with '/../', there's a problem */
+		if (next == absolute_path) {
+			ERR("%s: Path cannot be resolved", path);
+			goto error;
+		}
+
+		/* We find the last level of directory */
+		previous = absolute_path;
+		while ((slash = strpbrk(previous + 1, "/")) && slash != next) {
+			previous = slash;

src/common/utils.c line 96

Quoted 10 lines
src/common/utils.c +10 -0
+		}
+
+		/* Then we prepare the start_path not containing it */
+		start_path = strndup(absolute_path, previous - absolute_path);
+
+		/* And we concatenate it with the part after the '/../' */
+		snprintf(absolute_path, PATH_MAX, "%s%s", start_path, next + 3);
+
+		free(start_path);
+	}

I might be missing something, but how can this work on paths like this ?

"./a/../b/./c/../d/./e" ?

At first glance, it looks like the only cases that work correctly are those in the test case, but there are various variants that won't.

One possible alternative approach to all this path mangling would be to use realpath() on sections of paths (using / as separator, from the start of path), until realpath() fails to lookup the final paths. Maybe then could we simply append the paths as is without resolving. Users would have to know that they should not put "." or ".." in the unexisting part of their paths. Thoughts ?

Thanks,

Mathieu

Quoted 36 lines
src/common/utils.c +10 -0
+
+	return absolute_path;
+
+error:
+	free(absolute_path);
+	return NULL;
+}
+
+
+/*
  * Return the realpath(3) of the path even if the last directory token does
  not
  * exist. For example, with /tmp/test1/test2, if test2/ does not exist but

the
* /tmp/test1 does, the real path is returned. In normal time, realpath(3)

src/common/utils.h +1 -1
index 52f2798..c56942f 100644
@@ -26,6 +26,7 @@
 #define MEBI_LOG2 20
 #define GIBI_LOG2 30
 
+char *utils_resolve_relative(const char *path);
 char *utils_expand_path(const char *path);
 int utils_create_pipe(int *dst);
 int utils_create_pipe_cloexec(int *dst);
--

1.7.10.4

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Signature · 8 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): [email protected]

#17 lore raw

Re: [lttng-tools PATCH 1/4] Introduce a new utils_resolve_relative function

2013/11/14 Mathieu Desnoyers <[email protected]>:

Quoted 1 line

We should document that this returns allocated memory which needs to be freed by the caller.

Right.

Quoted 1 line

Also, we should clearly state that the only reason why we implement this is that realpath() only works on existing paths, which does not cover our use-case of having to resolve partially unexisting paths.

... or fully unexisting paths ;)
Didn't add the comment here as it is already precised for the utils_expand_path function that is the one made for that. This utils_resolve_relative does not aim to replace realpath by itself, but just take care of resolving relative paths such as "./" and "../". If we wanted to resolve such paths but not using an absolute path as argument, it would be useful too.
Yeah, in this very case, it is used by utils_expand_path to replace realpath. But I think this comment is not appropriate for this function, that would lead to miscomprehension. Perhaps the comment about "so that it works even if the directory does not exist" isn't appropriate either.

Quoted 3 lines

I might be missing something, but how can this work on paths like this ?

"./a/../b/./c/../d/./e" ?

The first while will replace the /./ strings:
- ./a/../b/c/../d/e
The second while will replace the /../ strings:
- ./b/d/e
Just tried to add it in the unit test :
- ok 10 - valid test case: ./a/../b/./c/../d/./e

Yeah, there's still the starting "./", but the starting relative paths are not in the scope of this function, as it is stated in the descriptive comment ("in the middle of the path"). It is thus a working case.

Quoted 1 line

At first glance, it looks like the only cases that work correctly are those in the test case, but there are various variants that won't.

Can you give me examples ?
I can see one only case that's not working, it's when the path starts with "../../", as the "/../" will be treated as if it is in the middle of the string. I'll fix that and send the patch. But if you have any other examples that would not work, I would gladly use them to verify that the function works properly, it's sometimes difficult to think about all the different cases to test (most of my ideas were put in the unit test).

Quoted 1 line

One possible alternative approach to all this path mangling would be to use realpath() on sections of paths (using / as separator, from the start of path), until realpath() fails to lookup the final paths. Maybe then could we simply append the paths as is without resolving. Users would have to know that they should not put "." or ".." in the unexisting part of their paths. Thoughts ?

I think that it would be using many calls to realpath for some case of "try and fail", when we can do something that directly works (one call to realpath). That's not so many analysis to do on the developer side, and that's a better experience on the user side. Moreover, this function does not aim to replace realpath, that's the other one (but yeah, the other one is using this one).

Thanks,
Raphaël

#18 lore raw

PATCH 1/4 Introduce a new utils_resolve_relative function

In reply to a message not in the archive

This function aims to resolve relative path such as './' and '../' in the middle of a path string. This allows to use paths such as '~/../test' that are received as '/home/x/../test' for instance.

---
 src/common/utils.c |   68 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 src/common/utils.h |    1 +
 2 files changed, 69 insertions(+)
src/common/utils.c +68 -0
index da4c036..4ccf36a 100644
@@ -36,6 +36,74 @@
 #include "defaults.h"
 
 /*
+ * Resolve the './' and '../' strings in the middle of a path using
+ * our very own way to do it, so that it works even if the directory
+ * does not exist
+ */
+LTTNG_HIDDEN
+char *utils_resolve_relative(const char *path)
+{
+	char *next, *previous, *slash, *start_path, *absolute_path = NULL;
+
+	/* Safety net */
+	if (path == NULL) {
+		goto error;
+	}
+
+	/* Allocate memory for the absolute path */
+	absolute_path = zmalloc(PATH_MAX);
+	if (absolute_path == NULL) {
+		PERROR("zmalloc expand path");
+		goto error;
+	}
+
+	/* Copy the path in the absolute path */
+	strncpy(absolute_path, path, PATH_MAX);
+
+	/* As long as we find '/./' in the path string */
+	while ((next = strstr(absolute_path, "/./"))) {
+
+		/* We prepare the start_path not containing it */
+		start_path = strndup(absolute_path, next - absolute_path);
+
+		/* And we concatenate it with the part after this string */
+		snprintf(absolute_path, PATH_MAX, "%s%s", start_path, next + 2);
+
+		free(start_path);
+	}
+
+	/* As long as we find '/../' in the path string */
+	while ((next = strstr(absolute_path, "/../"))) {
+		/* If the path starts with '/../', there's a problem */
+		if (next == absolute_path) {
+			ERR("%s: Path cannot be resolved", path);
+			goto error;
+		}
+
+		/* We find the last level of directory */
+		previous = absolute_path;
+		while ((slash = strpbrk(previous + 1, "/")) && slash != next) {
+			previous = slash;
+		}
+
+		/* Then we prepare the start_path not containing it */
+		start_path = strndup(absolute_path, previous - absolute_path);
+
+		/* And we concatenate it with the part after the '/../' */
+		snprintf(absolute_path, PATH_MAX, "%s%s", start_path, next + 3);
+
+		free(start_path);
+	}
+
+	return absolute_path;
+
+error:
+	free(absolute_path);
+	return NULL;
+}
+
+
+/*
  * Return the realpath(3) of the path even if the last directory token does not
  * exist. For example, with /tmp/test1/test2, if test2/ does not exist but the
  * /tmp/test1 does, the real path is returned. In normal time, realpath(3)
src/common/utils.h +1 -0
index 52f2798..c56942f 100644
@@ -26,6 +26,7 @@
 #define MEBI_LOG2 20
 #define GIBI_LOG2 30
 
+char *utils_resolve_relative(const char *path);
 char *utils_expand_path(const char *path);
 int utils_create_pipe(int *dst);
 int utils_create_pipe_cloexec(int *dst);
Signature · 7 lines
1.7.10.4


_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
#19 lore raw

PATCH 2/4 Tests: Add test_utils_resolve_relative to unit tests

In reply to a message not in the archive
---
 .gitignore                               |    1 +
 tests/unit/Makefile.am                   |   15 +++--
 tests/unit/test_utils_resolve_relative.c |   98 ++++++++++++++++++++++++++++++
 tests/unit_tests                         |    1 +
 4 files changed, 111 insertions(+), 4 deletions(-)
 create mode 100644 tests/unit/test_utils_resolve_relative.c
.gitignore +1 -0
index 5bacea4..2d779e4 100644
@@ -56,6 +56,7 @@ tests/unit/test_session
 tests/unit/test_uri
 tests/unit/test_ust_data
 tests/unit/test_utils_parse_size_suffix
+tests/unit/test_utils_resolve_relative
 kernel_all_events_basic
 kernel_event_basic
 ust_global_event_wildcard
tests/unit/Makefile.am +11 -4
index a8dec2b..82f6508 100644
@@ -16,7 +16,8 @@ LIBHASHTABLE=$(top_builddir)/src/common/hashtable/libhashtable.la
 LIBRELAYD=$(top_builddir)/src/common/relayd/librelayd.la
 
 # Define test programs
-noinst_PROGRAMS = test_uri test_session	test_kernel_data test_utils_parse_size_suffix
+noinst_PROGRAMS = test_uri test_session test_kernel_data
+noinst_PROGRAMS += test_utils_parse_size_suffix test_utils_resolve_relative
 
 if HAVE_LIBLTTNG_UST_CTL
 noinst_PROGRAMS += test_ust_data
@@ -82,10 +83,16 @@ test_kernel_data_LDADD = $(LIBTAP) $(LIBCOMMON) $(LIBRELAYD) $(LIBSESSIOND_COMM)
 			 $(LIBHASHTABLE) -lrt
 test_kernel_data_LDADD += $(KERN_DATA_TRACE)
 
-# parse_size_suffix unit test
-UTILS_PARSE_SIZE_SUFFIX=$(top_builddir)/src/common/.libs/utils.o \
+# utils suffix for unit test
+UTILS_SUFFIX=$(top_builddir)/src/common/.libs/utils.o \
 		$(top_builddir)/src/common/.libs/runas.o
 
+# parse_size_suffix unit test
 test_utils_parse_size_suffix_SOURCES = test_utils_parse_size_suffix.c
 test_utils_parse_size_suffix_LDADD = $(LIBTAP) $(LIBHASHTABLE) $(LIBCOMMON)
-test_utils_parse_size_suffix_LDADD += $(UTILS_PARSE_SIZE_SUFFIX)
+test_utils_parse_size_suffix_LDADD += $(UTILS_SUFFIX)
+
+# resolve_relative unit test
+test_utils_resolve_relative_SOURCES = test_utils_resolve_relative.c
+test_utils_resolve_relative_LDADD = $(LIBTAP) $(LIBHASHTABLE) $(LIBCOMMON)
+test_utils_resolve_relative_LDADD += $(UTILS_SUFFIX)
tests/unit/test_utils_resolve_relative.c new file +98 -0

98 lines not on this page · raw message

tests/unit_tests +1 -0
index 6bf33cc..5d0f291 100644
@@ -3,3 +3,4 @@ unit/test_session
 unit/test_uri
 unit/test_ust_data
 unit/test_utils_parse_size_suffix
+unit/test_utils_resolve_relative
Signature · 7 lines
1.7.10.4


_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
#20 lore raw

PATCH 3/4 Correct the behavior of the utils_expand_path function

In reply to a message not in the archive

Even if the utils_expand_path function was intended to allow to use unexistent directory paths, it was in fact only working for some kind of arguments. Paths like "foo", "bar/" or "bar/foo" when the "bar" directory does not exist wasn't working. This patch introduce a new way to expand paths in this function that also works properly for these kind of arguments.

---
 src/common/utils.c |   86 ++++++++++++++++++++++++++++++++++++++--------------
 1 file changed, 64 insertions(+), 22 deletions(-)
src/common/utils.c +64 -22
index 4ccf36a..6938a5a 100644
@@ -112,7 +112,7 @@ error:
 LTTNG_HIDDEN
 char *utils_expand_path(const char *path)
 {
-	const char *end_path = path;
+	const char *end_path = NULL;
 	char *next, *cut_path = NULL, *expanded_path = NULL;
 
 	/* Safety net */
@@ -120,38 +120,80 @@ char *utils_expand_path(const char *path)
 		goto error;
 	}
 
-	/* Find last token delimited by '/' */
-	while ((next = strpbrk(end_path + 1, "/"))) {
-		end_path = next;
-	}
-
-	/* Cut last token from original path */
-	cut_path = strndup(path, end_path - path);
-
+	/* Allocate memory for the expanded path */
 	expanded_path = zmalloc(PATH_MAX);
 	if (expanded_path == NULL) {
 		PERROR("zmalloc expand path");
 		goto error;
 	}
 
-	expanded_path = realpath((char *)cut_path, expanded_path);
-	if (expanded_path == NULL) {
-		switch (errno) {
-		case ENOENT:
-			ERR("%s: No such file or directory", cut_path);
-			break;
-		default:
-			PERROR("realpath utils expand path");
-			break;
+	/* If given path is already absolute */
+	if (*path == '/') {
+		strncpy(expanded_path, path, PATH_MAX);
+	/* Else, we have some work to do */
+	} else {
+		/* Pointer to the last char of the path */
+		const char *last_char = path + strlen(path) - 1;
+
+		end_path = path;
+
+		/* Split part that will be resolved by realpath (relative path from
+		 * current directory using ./ or ../ only) and part that could not
+		 * (directory names)
+		 */
+		while ((next = strpbrk(end_path, "/")) && (next != last_char)) {
+			end_path = next + 1;
+			if (strncmp(end_path, "./", 2) != 0 &&
+					strncmp(end_path, "../", 3) != 0) {
+				break;
+			}
+		}
+
+		/* If this is the end of the string, and we still can resolve it */
+		if (strncmp(end_path, "..\0", 3) == 0 ||
+				strncmp(end_path, ".\0", 2) == 0) {
+			end_path += strlen(end_path);
+		}
+
+		/* If the end part is the whole path, we are in the current dir */
+		if (end_path == path) {
+			cut_path = strdup(".");
+		/* Else, cut the resolvable part from original path */
+		} else {
+			cut_path = strndup(path, end_path - path);
+		}
+
+		/* Resolve the canonical path of the first part of the path */
+		expanded_path = realpath((char *)cut_path, expanded_path);
+		if (expanded_path == NULL) {
+			switch (errno) {
+			case ENOENT:
+				ERR("%s: No such file or directory", cut_path);
+				break;
+			default:
+				PERROR("realpath utils expand path");
+				break;
+			}
+			goto error;
+		}
+
+		/* Add end part to expanded path if not empty */
+		if (*end_path != 0) {
+			strncat(expanded_path, "/", PATH_MAX - strlen(expanded_path) - 1);
+			strncat(expanded_path, end_path,
+					PATH_MAX - strlen(expanded_path) - 1);
 		}
-		goto error;
 	}
 
-	/* Add end part to expanded path */
-	strncat(expanded_path, end_path, PATH_MAX - strlen(expanded_path) - 1);
+	/* Resolve the internal './' and '../' strings */
+	next = utils_resolve_relative(expanded_path);
+	if (next == NULL) {
+		goto error;
+	}
 
+	free(expanded_path);
 	free(cut_path);
-	return expanded_path;
+	return next;
 
 error:
 	free(expanded_path);
Signature · 7 lines
1.7.10.4


_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
#21 lore raw

PATCH 4/4 Tests: Add test_utils_expand_path to unit tests

In reply to a message not in the archive
---
 .gitignore                          |    1 +
 tests/unit/Makefile.am              |    7 +-
 tests/unit/test_utils_expand_path.c |  170 +++++++++++++++++++++++++++++++++++
 tests/unit_tests                    |    1 +
 4 files changed, 178 insertions(+), 1 deletion(-)
 create mode 100644 tests/unit/test_utils_expand_path.c
.gitignore +1 -0
index 2d779e4..7041d37 100644
@@ -57,6 +57,7 @@ tests/unit/test_uri
 tests/unit/test_ust_data
 tests/unit/test_utils_parse_size_suffix
 tests/unit/test_utils_resolve_relative
+tests/unit/test_utils_expand_path
 kernel_all_events_basic
 kernel_event_basic
 ust_global_event_wildcard
tests/unit/Makefile.am +6 -1
index 82f6508..fa9c6a8 100644
@@ -17,7 +17,7 @@ LIBRELAYD=$(top_builddir)/src/common/relayd/librelayd.la
 
 # Define test programs
 noinst_PROGRAMS = test_uri test_session test_kernel_data
-noinst_PROGRAMS += test_utils_parse_size_suffix test_utils_resolve_relative
+noinst_PROGRAMS += test_utils_parse_size_suffix test_utils_resolve_relative test_utils_expand_path
 
 if HAVE_LIBLTTNG_UST_CTL
 noinst_PROGRAMS += test_ust_data
@@ -96,3 +96,8 @@ test_utils_parse_size_suffix_LDADD += $(UTILS_SUFFIX)
 test_utils_resolve_relative_SOURCES = test_utils_resolve_relative.c
 test_utils_resolve_relative_LDADD = $(LIBTAP) $(LIBHASHTABLE) $(LIBCOMMON)
 test_utils_resolve_relative_LDADD += $(UTILS_SUFFIX)
+
+# expand_path unit test
+test_utils_expand_path_SOURCES = test_utils_expand_path.c
+test_utils_expand_path_LDADD = $(LIBTAP) $(LIBHASHTABLE) $(LIBCOMMON)
+test_utils_expand_path_LDADD += $(UTILS_SUFFIX)
tests/unit/test_utils_expand_path.c new file +170 -0

170 lines not on this page · raw message

tests/unit_tests +1 -0
index 5d0f291..35b31a4 100644
@@ -4,3 +4,4 @@ unit/test_uri
 unit/test_ust_data
 unit/test_utils_parse_size_suffix
 unit/test_utils_resolve_relative
+unit/test_utils_expand_path
Signature · 7 lines
1.7.10.4


_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
#22 lore raw
In reply to a message not in the archive

Merged! Big thanks Raphaël especially for the two unit tests!

David

Quoted 17 lines · Raphaël Beamonte

David,

Please find here an updated series of patches following your previous comments. These patches introduce updated versions of the utils_resolve_relative and utils_expand_path functions, as well as unit tests to verify their behavior.

Thanks,
Raphaël

Raphaël Beamonte (4):
Introduce a new utils_resolve_relative function
Tests: Add test_utils_resolve_relative to unit tests Correct the behavior of the utils_expand_path function
Tests: Add test_utils_expand_path to unit tests

Quoted 14 lines
 .gitignore                               |    2 +
 src/common/utils.c                       |  154 +++++++++++++++++++++++----
 src/common/utils.h                       |    1 +
 tests/unit/Makefile.am                   |   20 +++-
 tests/unit/test_utils_expand_path.c      |  170 ++++++++++++++++++++++++++++++
 tests/unit/test_utils_resolve_relative.c |   98 +++++++++++++++++
 tests/unit_tests                         |    2 +
 7 files changed, 421 insertions(+), 26 deletions(-)
 create mode 100644 tests/unit/test_utils_expand_path.c
 create mode 100644 tests/unit/test_utils_resolve_relative.c
Signature · 1 line
1.7.10.4

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev