Skip to contentSkip to conversation

save_restore: Check whether path is writable

PATCH In review Martin Doucha 4 participants 9 messages Oct 21, 2022 – Oct 26, 2022 Library code

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 save_restore: Check whether path is writable

Tests using the .save_restore functionality currently cannot run without root privileges at all because the test will write into the path at least at the end and trigger error, even when the config paths are flagged as optional. Check whether .save_restore paths are writable and handle negative result the same way as if the path does not exist.

---

This is the first part of sysfile handling fixes to allow running some tests without root privileges again. I think this is a good enough solution for the save_restore part but we should discuss a few open questions first:

1) Is it OK to fail early during sysfile save when the test would otherwise run fine but throw TWARN at the end because the sysfile is read-only?
2) Should the '?' flag skip read-only files as if they don't exist? Alternatively, we could still let the '?' flag fail trying to write into read-only sysfiles and instead introduce a new flag for cases where read-only file should be skipped.

 doc/c-test-api.txt | 11 +++++------
 lib/tst_sys_conf.c | 32 ++++++++++++++++++++++----------
 2 files changed, 27 insertions(+), 16 deletions(-)
doc/c-test-api.txt +5 -6
index 64ee3397f..0f36b5a67 100644
@@ -1601,13 +1601,12 @@ If non-NULL value is passed it is written to the respective file at
 the beginning of the test. Only the first line of a specified file
 is saved and restored.
 
-Pathnames can be optionally prefixed to specify how strictly (during
-'store') are handled errors:
+Pathnames can be optionally prefixed to specify how to handle missing or
+read-only files:
 
-* (no prefix) - test ends with 'TCONF', if file doesn't exist
-* '?'         - test prints info message and continues,
-                if file doesn't exist or open/read fails
-* '!'         - test ends with 'TBROK', if file doesn't exist
+* (no prefix) - test ends with 'TCONF'
+* '?'         - test prints info message and continues, even on read error
+* '!'         - test ends with 'TBROK'
 
 'restore' is always strict and will TWARN if it encounters any error.
 
lib/tst_sys_conf.c +22 -10
index 003698825..1e381a249 100644
@@ -20,6 +20,22 @@ struct tst_sys_conf {
 
 static struct tst_sys_conf *save_restore_data;
 
+static void print_access_error(char flag, const char *err, const char *path)
+{
+	switch (flag) {
+	case '?':
+		tst_res(TINFO, "%s: '%s'", err, path);
+		break;
+
+	case '!':
+		tst_brk(TBROK|TERRNO, "%s: '%s'", err, path);
+		break;
+
+	default:
+		tst_brk(TCONF|TERRNO, "%s: '%s'", err, path);
+	}
+}
+
 void tst_sys_conf_dump(void)
 {
 	struct tst_sys_conf *i;
@@ -59,16 +75,12 @@ int tst_sys_conf_save(const char *path)
 		path++;
 
 	if (access(path, F_OK) != 0) {
-		switch (flag) {
-		case '?':
-			tst_res(TINFO, "Path not found: '%s'", path);
-			break;
-		case '!':
-			tst_brk(TBROK|TERRNO, "Path not found: '%s'", path);
-			break;
-		default:
-			tst_brk(TCONF|TERRNO, "Path not found: '%s'", path);
-		}
+		print_access_error(flag, "Path not found", path);
+		return 1;
+	}
+
+	if (access(path, W_OK) != 0) {
+		print_access_error(flag, "Path is not writable", path);
 		return 1;
 	}
 
Signature · 5 lines
2.37.3


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#1 lore raw

Hi Martin,

[ Cc Jan, who implemented the original behavior ]

Quoted 6 lines

Tests using the .save_restore functionality currently cannot run without root privileges at all because the test will write into the path at least at the end and trigger error, even when the config paths are flagged as optional. Check whether .save_restore paths are writable and handle negative result the same way as if the path does not exist.

Thanks for this effort!

Quoted 9 lines

---

This is the first part of sysfile handling fixes to allow running some tests without root privileges again. I think this is a good enough solution for the save_restore part but we should discuss a few open questions first:

1) Is it OK to fail early during sysfile save when the test would otherwise run fine but throw TWARN at the end because the sysfile is read-only?

I don't think that would be a good change.

Quoted 4 lines

2) Should the '?' flag skip read-only files as if they don't exist? Alternatively, we could still let the '?' flag fail trying to write into read-only sysfiles and instead introduce a new flag for cases where read-only file should be skipped.

Looking at files which use '?', some of them (mostly network related, I guess written/rewritten by Martin) use SAFE_TRY_FILE_PRINTF() on /proc/sys/user/max_user_namespaces. It looks to me these need to to skip read-only files, i.e. define new flag with this behavior.

Kind regards,
Petr

Quoted 76 lines
 doc/c-test-api.txt | 11 +++++------
 lib/tst_sys_conf.c | 32 ++++++++++++++++++++++----------
 2 files changed, 27 insertions(+), 16 deletions(-)
doc/c-test-api.txt +5 -6
index 64ee3397f..0f36b5a67 100644
@@ -1601,13 +1601,12 @@ If non-NULL value is passed it is written to the respective file at
 the beginning of the test. Only the first line of a specified file
 is saved and restored.
 
-Pathnames can be optionally prefixed to specify how strictly (during
-'store') are handled errors:
+Pathnames can be optionally prefixed to specify how to handle missing or
+read-only files:
 
-* (no prefix) - test ends with 'TCONF', if file doesn't exist
-* '?'         - test prints info message and continues,
-                if file doesn't exist or open/read fails
-* '!'         - test ends with 'TBROK', if file doesn't exist
+* (no prefix) - test ends with 'TCONF'
+* '?'         - test prints info message and continues, even on read error
+* '!'         - test ends with 'TBROK'
 
 'restore' is always strict and will TWARN if it encounters any error.
 
lib/tst_sys_conf.c +22 -10
index 003698825..1e381a249 100644
@@ -20,6 +20,22 @@ struct tst_sys_conf {
 
 static struct tst_sys_conf *save_restore_data;
 
+static void print_access_error(char flag, const char *err, const char *path)
+{
+	switch (flag) {
+	case '?':
+		tst_res(TINFO, "%s: '%s'", err, path);
+		break;
+
+	case '!':
+		tst_brk(TBROK|TERRNO, "%s: '%s'", err, path);
+		break;
+
+	default:
+		tst_brk(TCONF|TERRNO, "%s: '%s'", err, path);
+	}
+}
+
 void tst_sys_conf_dump(void)
 {
 	struct tst_sys_conf *i;
@@ -59,16 +75,12 @@ int tst_sys_conf_save(const char *path)
 		path++;
 
 	if (access(path, F_OK) != 0) {
-		switch (flag) {
-		case '?':
-			tst_res(TINFO, "Path not found: '%s'", path);
-			break;
-		case '!':
-			tst_brk(TBROK|TERRNO, "Path not found: '%s'", path);
-			break;
-		default:
-			tst_brk(TCONF|TERRNO, "Path not found: '%s'", path);
-		}
+		print_access_error(flag, "Path not found", path);
+		return 1;
+	}
+
+	if (access(path, W_OK) != 0) {
+		print_access_error(flag, "Path is not writable", path);
 		return 1;
 	}
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw
Quoted 9 lines · Petr Vorel

Hi Martin,

[ Cc Jan, who implemented the original behavior ]

Quoted 4 lines

Tests using the .save_restore functionality currently cannot run without root privileges at all because the test will write into the path at least at the end and trigger error, even when the config paths are flagged as optional.

Problem description makes it sound like this issue affects all 3 types of config options. Isn't the problem affecting only optional config paths?

Having entry with "(no prefix)" or "!" in save_restore implies that test wants to write to that path - if we TCONF on root privileges or read/write access probably doesn't make much difference - we can't continue.

For "?" prefix, I agree that since its optional, test should be able to run cleanly without root privileges.

Quoted 3 lines

Check whether .save_restore

Quoted 2 lines

paths are writable and handle negative result the same way as if the path does not exist.

Do you mean for "?" prefix only?

Quoted 11 lines
Quoted 10 lines

---

This is the first part of sysfile handling fixes to allow running some tests without root privileges again. I think this is a good enough solution for the save_restore part but we should discuss a few open questions first:

1) Is it OK to fail early during sysfile save when the test would otherwise run fine but throw TWARN at the end because the sysfile is read-only?

I don't think that would be a good change.

For optional path, if test can't read/write it (b/o of no root privileges), I think library shouldn't try to save it - then that would also skip attempt to restore it.

Quoted 2 lines
Quoted 2 lines

2) Should the '?' flag skip read-only files as if they don't exist?

I think yes. If we can't restore those files, that means test shouldn't be able to change those and we don't need to worry about saving them.

Quoted 89 lines
Quoted 3 lines

Alternatively, we could still let the '?' flag fail trying to write into read-only sysfiles and instead introduce a new flag for cases where read-only file should be skipped.

Looking at files which use '?', some of them (mostly network related, I guess written/rewritten by Martin) use SAFE_TRY_FILE_PRINTF() on /proc/sys/user/max_user_namespaces. It looks to me these need to to skip read-only files, i.e. define new flag with this behavior.

Kind regards,
Petr

Quoted 76 lines
 doc/c-test-api.txt | 11 +++++------
 lib/tst_sys_conf.c | 32 ++++++++++++++++++++++----------
 2 files changed, 27 insertions(+), 16 deletions(-)
doc/c-test-api.txt +5 -6
index 64ee3397f..0f36b5a67 100644
@@ -1601,13 +1601,12 @@ If non-NULL value is passed it is written to the respective file at
 the beginning of the test. Only the first line of a specified file
 is saved and restored.
 
-Pathnames can be optionally prefixed to specify how strictly (during
-'store') are handled errors:
+Pathnames can be optionally prefixed to specify how to handle missing or
+read-only files:
 
-* (no prefix) - test ends with 'TCONF', if file doesn't exist
-* '?'         - test prints info message and continues,
-                if file doesn't exist or open/read fails
-* '!'         - test ends with 'TBROK', if file doesn't exist
+* (no prefix) - test ends with 'TCONF'
+* '?'         - test prints info message and continues, even on read error
+* '!'         - test ends with 'TBROK'
 
 'restore' is always strict and will TWARN if it encounters any error.
 
lib/tst_sys_conf.c +22 -10
index 003698825..1e381a249 100644
@@ -20,6 +20,22 @@ struct tst_sys_conf {
 
 static struct tst_sys_conf *save_restore_data;
 
+static void print_access_error(char flag, const char *err, const char *path)
+{
+     switch (flag) {
+     case '?':
+             tst_res(TINFO, "%s: '%s'", err, path);
+             break;
+
+     case '!':
+             tst_brk(TBROK|TERRNO, "%s: '%s'", err, path);
+             break;
+
+     default:
+             tst_brk(TCONF|TERRNO, "%s: '%s'", err, path);
+     }
+}
+
 void tst_sys_conf_dump(void)
 {
      struct tst_sys_conf *i;
@@ -59,16 +75,12 @@ int tst_sys_conf_save(const char *path)
              path++;
 
      if (access(path, F_OK) != 0) {
-             switch (flag) {
-             case '?':
-                     tst_res(TINFO, "Path not found: '%s'", path);
-                     break;
-             case '!':
-                     tst_brk(TBROK|TERRNO, "Path not found: '%s'", path);
-                     break;
-             default:
-                     tst_brk(TCONF|TERRNO, "Path not found: '%s'", path);
-             }
+             print_access_error(flag, "Path not found", path);
+             return 1;
+     }
+
+     if (access(path, W_OK) != 0) {
+             print_access_error(flag, "Path is not writable", path);
              return 1;
      }
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#3 lore raw
Quoted 11 lines · Jan Stancek

On Fri, Oct 21, 2022 at 10:35 PM Petr Vorel <[email protected]> wrote:
Problem description makes it sound like this issue affects all 3 types of config options. Isn't the problem affecting only optional config paths?

Having entry with "(no prefix)" or "!" in save_restore implies that test wants to write to that path - if we TCONF on root privileges or read/write access probably doesn't make much difference - we can't continue.

For "?" prefix, I agree that since its optional, test should be able to run cleanly without root privileges.

It does affect all 3 but slightly differently, depending on the "val" field of the respective .save_restore item. The current implementation behaves like this without root privileges:
- (no prefix): If val is NULL, the test will save the data, run the test and trigger TWARN at the end. If val is not NULL, the test will fail immediately after saving sysfile data because it'll try to write into a read-only file. We'd want TCONF instead in the latter case.
- '!': Same behavior as with no prefix but we want to keep it.
- '?': Same behavior as with no prefix. We want either TCONF or to ignore the sysfile entirely with a TINFO message.

Quoted 3 lines

For optional path, if test can't read/write it (b/o of no root privileges), I think library shouldn't try to save it - then that would also skip attempt to restore it.

There are be two different kinds of optional paths, though:
1) paths that sometimes don't exist but must be written to if they do
2) paths that may be left alone if they exist and already contain the right value (otherwise TCONF)

So the question is whether I should steal the '?' prefix for type #2 and we'll introduce a new prefix later if needed, or whether we'll reserve the '?' prefix for type #1 according to current behavior and introduce the new prefix now.

Quoted 4 lines · Petr Vorel

Looking at files which use '?', some of them (mostly network related, I guess written/rewritten by Martin) use SAFE_TRY_FILE_PRINTF() on /proc/sys/user/max_user_namespaces. It looks to me these need to to skip read-only files, i.e. define new flag with this behavior.

All those SAFE_TRY_FILE_PRINTF() calls are writing a constant so they can be eliminated by filling the second field of the .save_restore struct. I'll do that in the follow-up patchset when we agree how to implement this part.

Signature · 11 lines
Martin Doucha   [email protected]
QA Engineer for Software Maintenance
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#4 lore raw
Quoted 11 lines · Martin Doucha
Quoted 9 lines · Jan Stancek

On Fri, Oct 21, 2022 at 10:35 PM Petr Vorel <[email protected]> wrote:
Problem description makes it sound like this issue affects all 3 types of config options. Isn't the problem affecting only optional config paths?

Having entry with "(no prefix)" or "!" in save_restore implies that test wants to write to that path - if we TCONF on root privileges or read/write access probably doesn't make much difference - we can't continue.

Quoted 8 lines
Quoted 2 lines

For "?" prefix, I agree that since its optional, test should be able to run cleanly without root privileges.

It does affect all 3 but slightly differently, depending on the "val" field of the respective .save_restore item. The current implementation behaves like this without root privileges:
- (no prefix): If val is NULL, the test will save the data, run the test and trigger TWARN at the end.

Is this real scenario? Why is test saving sysfs value, which is then never changed?
I would expect that in this case, you could drop save_restore entirely.

Quoted 18 lines

If val is not NULL, the test will fail
immediately after saving sysfile data because it'll try to write into a read-only file.

We'd want TCONF instead in the latter case.
- '!': Same behavior as with no prefix but we want to keep it.
- '?': Same behavior as with no prefix. We want either TCONF or to ignore the sysfile entirely with a TINFO message.

Quoted 3 lines

For optional path, if test can't read/write it (b/o of no root privileges), I think library shouldn't try to save it - then that would also skip attempt to restore it.

There are be two different kinds of optional paths, though:
1) paths that sometimes don't exist but must be written to if they do
2) paths that may be left alone if they exist and already contain the right value (otherwise TCONF)

Quoted 4 lines

So the question is whether I should steal the '?' prefix for type #2 and we'll introduce a new prefix later if needed, or whether we'll reserve the '?' prefix for type #1 according to current behavior and introduce the new prefix now.

The case 2) looks like it could apply to non-optional paths too. So maybe best option would be to drop "!" and "?" prefixes and turn them into flags/enums which can be then combined together.

"/proc/sys/kernel/pid_max", 0, val // TCONF if path doesn't exist "/proc/sys/kernel/pid_max", SR_MUST_EXIST, val // TBROK if path doesn't exist "/proc/sys/kernel/pid_max", SR_MAY_EXIST, val // if exists, save it "/proc/sys/kernel/pid_max", SR_CONST_VAL, val // if already has val, skip saving it
"/proc/sys/kernel/pid_max", SR_MAY_EXIST | SR_CONST_VAL, val // if exists check it already has val, otherwise save it

What do you think? Would that make it easier to represent/implement all cases?

Quoted 21 lines · Petr Vorel
Quoted 4 lines

Looking at files which use '?', some of them (mostly network related, I guess written/rewritten by Martin) use SAFE_TRY_FILE_PRINTF() on /proc/sys/user/max_user_namespaces. It looks to me these need to to skip read-only files, i.e. define new flag with this behavior.

All those SAFE_TRY_FILE_PRINTF() calls are writing a constant so they can be eliminated by filling the second field of the .save_restore struct. I'll do that in the follow-up patchset when we agree how to implement this part.

--
Martin Doucha [email protected]
QA Engineer for Software Maintenance
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#5 lore raw

Hi!

Quoted 12 lines

The case 2) looks like it could apply to non-optional paths too. So maybe best option would be to drop "!" and "?" prefixes and turn them into flags/enums which can be then combined together.

"/proc/sys/kernel/pid_max", 0, val // TCONF if path doesn't exist "/proc/sys/kernel/pid_max", SR_MUST_EXIST, val // TBROK if path doesn't exist "/proc/sys/kernel/pid_max", SR_MAY_EXIST, val // if exists, save it "/proc/sys/kernel/pid_max", SR_CONST_VAL, val // if already has val, skip saving it
"/proc/sys/kernel/pid_max", SR_MAY_EXIST | SR_CONST_VAL, val // if exists check it already has val, otherwise save it

Quoted 1 line

What do you think? Would that make it easier to represent/implement all cases?

Ah Jan already proposed something similar to what I had in mind. I agree with moving the attributes from the path to a separate field too.

Signature · 5 lines
Cyril Hrubis
[email protected]

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#6 lore raw
In reply to Jan Stancek
Quoted 10 lines · Jan Stancek
Quoted 5 lines · Martin Doucha

It does affect all 3 but slightly differently, depending on the "val" field of the respective .save_restore item. The current implementation behaves like this without root privileges:
- (no prefix): If val is NULL, the test will save the data, run the test and trigger TWARN at the end.

Is this real scenario? Why is test saving sysfs value, which is then never changed?
I would expect that in this case, you could drop save_restore entirely.

Some tests use simplified sysfile handling and only write a string constant passed in the .save_restore array. These have problem with lack of root privileges because the write happens in library code. Other tests write something non-constant in setup() or run(), these can handle read-only sysfiles themselves.

Quoted 12 lines

The case 2) looks like it could apply to non-optional paths too. So maybe best option would be to drop "!" and "?" prefixes and turn them into flags/enums which can be then combined together.

"/proc/sys/kernel/pid_max", 0, val // TCONF if path doesn't exist "/proc/sys/kernel/pid_max", SR_MUST_EXIST, val // TBROK if path doesn't exist "/proc/sys/kernel/pid_max", SR_MAY_EXIST, val // if exists, save it "/proc/sys/kernel/pid_max", SR_CONST_VAL, val // if already has val, skip saving it
"/proc/sys/kernel/pid_max", SR_MAY_EXIST | SR_CONST_VAL, val // if exists check it already has val, otherwise save it

Quoted 1 line

What do you think? Would that make it easier to represent/implement all cases?

Sounds good, I'll prepare a patchset for that.

Signature · 11 lines
Martin Doucha   [email protected]
QA Engineer for Software Maintenance
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#7 lore raw
In reply to Martin Doucha

Hi!

Quoted 8 lines
Quoted 3 lines

For optional path, if test can't read/write it (b/o of no root privileges), I think library shouldn't try to save it - then that would also skip attempt to restore it.

There are be two different kinds of optional paths, though:
1) paths that sometimes don't exist but must be written to if they do
2) paths that may be left alone if they exist and already contain the right value (otherwise TCONF)

Alternatively we can cleanup the interface, we moved from a single string to a structure so we can add more fields, what about adding flags that would describe one single attribute of the file instead of clobbering several different characteristics of the file into a single character?

We can then do something as:

struct tst_save_restore {
	const char *path;
	const char *val;
	/* the test needs the file to exist -> TCONF on missing */
	int required:1;
	/* write the value even if the file already contains it */
	int rewrite:1;
	...
};

This makes the inteface orthogonal and much easier to reason about.

Signature · 5 lines
Cyril Hrubis
[email protected]

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#8 lore raw
Quoted 28 lines

Hi!

Quoted 8 lines
Quoted 3 lines

For optional path, if test can't read/write it (b/o of no root privileges), I think library shouldn't try to save it - then that would also skip attempt to restore it.

There are be two different kinds of optional paths, though:
1) paths that sometimes don't exist but must be written to if they do
2) paths that may be left alone if they exist and already contain the right value (otherwise TCONF)

Alternatively we can cleanup the interface, we moved from a single string to a structure so we can add more fields, what about adding flags that would describe one single attribute of the file instead of clobbering several different characteristics of the file into a single character?

We can then do something as:

struct tst_save_restore {
	const char *path;
	const char *val;
	/* the test needs the file to exist -> TCONF on missing */
	int required:1;
	/* write the value even if the file already contains it */
	int rewrite:1;
	...
};
Quoted 1 line

This makes the inteface orthogonal and much easier to reason about.

Sounds reasonable approach to me.
Martin, Jan, WDYT?

Kind regards,
Petr

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp