syscalls/msgstress03: fix fork failure on small memory systems
PATCH 1/2 syscalls/msgstress03: fix fork failure on small memory systems
Running syscalls/msgstress03 on a system with less than ~4 GB of RAM fails:
msgstress03 1 TFAIL : msgstress03.c:155: Fork failed (may be OK if under stress)
In dmesg:
LTP: starting msgstress03
cgroup: fork rejected by pids controller in /user.slice/user-1000.slice/session-1.scope
The reason is cgroups pid limit set by systemd user.slice. The limit is set for login session, also for root user. For example on 2 GB RAM machine it is set as:
/sys/fs/cgroup/pids/user.slice/user-0.slice/pids.max:5207
Read the maximum number of pids and adjust the test limit. For 2 GB RAM machine with systemd this will result in:
msgstress03 0 TINFO : Found limit of processes 5056 (from /sys/fs/cgroup/pids/user.slice/user-1000.slice/pids.max)
msgstress03 0 TINFO : Requested number of processes higher than user session limit (10000 > 4556), setting to 4556--- include/ipcmsg.h | 2 + libs/libltpipc/libipc.c | 58 +++++++++++++++++++ .../syscalls/ipc/msgstress/msgstress03.c | 15 ++++- 3 files changed, 74 insertions(+), 1 deletion(-)
include/ipcmsg.h +2 -0
@@ -61,8 +61,10 @@ void rm_queue(int);key_t getipckey ();int getuserid (char *); +int get_session_uid (void );int get_max_msgqueues (void );int get_used_msgqueues (void ); +int get_pids_limit (void );#endif /* ipcmsg.h */
libs/libltpipc/libipc.c +58 -0
67 lines not on this page · raw message
testcases/kernel/syscalls/ipc/msgstress/msgstress03.c +14 -1
@@ -78,7 +78,7 @@ static void usage(void)int main (int argc,char **argv) { -int i, j, ok, pid; +int i, j, ok, pid, max_session_pids;int count, status;struct sigaction act;
@@ -109,6 +109,19 @@ int main(int argc, char **argv) } } + max_session_pids =get_pids_limit (); +if (max_session_pids >0 ) { +/* Clamp number of processes to session limit with some buffer for OS */ + max_session_pids = (max_session_pids >500 ? max_session_pids -500 :0 ); +if (nprocs >= max_session_pids) { +tst_resm (TINFO , +"Requested number of processes higher than user session limit (%d > %d), " +"setting to %d" , nprocs, max_session_pids, + max_session_pids); + nprocs = max_session_pids; + } + } +srand (getpid ()); tid =-1 ;
Signature · 1 line
2.27.0
PATCH 2/2 syscalls/msgstress04: fix fork failure on small memory systems
Running syscalls/msgstress04 on a system with less than ~4 GB of RAM fails:
Fork failure in the first child of child group 4396 Fork failure in the second child of child group 4413
msgstress04 1 TFAIL : msgstress04.c:222: Child exit status = 1
The reason is cgroups pid limit set by systemd user.slice. The limit is set for login session, also for root user. For example on 2 GB RAM machine it is set as:
/sys/fs/cgroup/pids/user.slice/user-0.slice/pids.max:5207
Read the maximum number of pids and adjust the test limit.
--- testcases/kernel/syscalls/ipc/msgstress/msgstress04.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
testcases/kernel/syscalls/ipc/msgstress/msgstress04.c +8 -1
@@ -387,7 +387,7 @@ static void term(int sig LTP_ATTRIBUTE_UNUSED)void setup (void ) { -int nr_msgqs, free_pids; +int nr_msgqs, free_pids, max_session_pids;tst_tmpdir ();/* You will want to enable some signal handling so you can capture
@@ -423,6 +423,13 @@ void setup(void)* For each child we fork up to 2*maxnkids grandchildren. */ maxnprocs = (free_pids /2 ) / (1 +2 * maxnkids); + max_session_pids =get_pids_limit (); +if ((max_session_pids >0 ) && (max_session_pids < free_pids)) { +/* Clamp number of processes to session limit with some buffer for OS */ + max_session_pids = (max_session_pids >500 ? max_session_pids -500 :0 ); + maxnprocs = (max_session_pids /2 ) / (1 +2 * maxnkids); + } +if (!maxnprocs)tst_brkm (TBROK , cleanup,"Not enough free pids" );
Signature · 1 line
2.27.0
PATCH 1/2 syscalls/msgstress03: fix fork failure on small memory systems
In reply to Krzysztof KozlowskiHi Krzysztof
Quoted 6 lines
Running syscalls/msgstress03 on a system with less than ~4 GB of RAM fails:
msgstress03 1 TFAIL : msgstress03.c:155: Fork failed (may be OK if under stress)
In dmesg:
Quoted 7 lines
LTP: starting msgstress03
cgroup: fork rejected by pids controller in /user.slice/user-1000.slice/session-1.scopeThe reason is cgroups pid limit set by systemd user.slice. The limit is set for login session, also for root user. For example on 2 GB RAM machine it is set as:
/sys/fs/cgroup/pids/user.slice/user-0.slice/pids.max:5207
This file only exists on cgroupv1 and cgroupv2 should use /sys/fs/cgroup/user.slice/user-0.slice/pids.max.
Also if some embedded system doesn't have systemd, do we have a generic way or fallback way to calculate the pid max?
For this problem, I have a calculate way in my patch[1] at this time last year.
[1]https://patchwork.ozlabs.org/project/ltp/patch/[email protected]/
Best Regards
Yang Xu
Quoted 135 lines
Read the maximum number of pids and adjust the test limit. For 2 GB RAM machine with systemd this will result in:
msgstress03 0 TINFO : Found limit of processes 5056 (from /sys/fs/cgroup/pids/user.slice/user-1000.slice/pids.max) msgstress03 0 TINFO : Requested number of processes higher than user session limit (10000> 4556), setting to 4556Signed-off-by Krzysztof Kozlowski<[email protected]>--- include/ipcmsg.h | 2 + libs/libltpipc/libipc.c | 58 +++++++++++++++++++ .../syscalls/ipc/msgstress/msgstress03.c | 15 ++++- 3 files changed, 74 insertions(+), 1 deletion(-)include/ipcmsg.h +2
-0@@ -61,8 +61,10 @@ void rm_queue(int);key_t getipckey ();int getuserid (char *); +int get_session_uid (void );int get_max_msgqueues (void );int get_used_msgqueues (void ); +int get_pids_limit (void );#endif /* ipcmsg.h */ libs/libltpipc/libipc.c +58
-0@@ -151,6 +151,31 @@ int getuserid(char *user)return (ent->pw_uid); } +/* +* Get the effective session UID - either one invoking current test via sudo +* or the real UID. +*/ +int get_session_uid (void ) +{ +const char *sudo_uid; + + sudo_uid =getenv ("SUDO_UID" ); +if (sudo_uid) { +int real_uid; + +TEST (sscanf (sudo_uid,"%u" ,&real_uid)); +if (TEST_RETURN !=1 ) { +#ifdef DEBUG +tst_resm (TINFO ,"No SUDO_UID from env" ); +#endif + }else { +return real_uid; + } + } + +return getuid (); +} +/* * rm_shm() - removes a shared memory segment. */ @@ -218,3 +243,36 @@ int get_max_msgqueues(void)fclose (f);return atoi (buff); } + +/* +* Get the limit of processes for current session configured by systemd user.slice. +* This also applies to root user. +*/ +int get_pids_limit (void ) +{ +int real_uid, ret; +char path[PATH_MAX ]; +long unsigned int max_pids; + + real_uid =get_session_uid (); +if (TEST_RETURN !=1 ) { +tst_resm (TINFO ,"Cannot get UID" ); +return -1 ; + } + + ret =snprintf (path,sizeof (path), +"/sys/fs/cgroup/pids/user.slice/user-%d.slice/pids.max" , + real_uid); +if (ret<0 || (size_t )ret>=sizeof (path)) +return -1 ; + +if (access (path,R_OK ) !=0 ) { +tst_resm (TINFO ,"Cannot read session user limits from '%s'" , path); +return -1 ; + } + +SAFE_FILE_SCANF (cleanup, path,"%lu" ,&max_pids); +tst_resm (TINFO ,"Found limit of processes %lu (from %s)" , max_pids, path); + +return max_pids; +}testcases/kernel/syscalls/ipc/msgstress/msgstress03.c +14
-1@@ -78,7 +78,7 @@ static void usage(void)int main (int argc,char **argv) { -int i, j, ok, pid; +int i, j, ok, pid, max_session_pids;int count, status;struct sigaction act;@@ -109,6 +109,19 @@ int main(int argc, char **argv) } } + max_session_pids =get_pids_limit (); +if (max_session_pids>0 ) { +/* Clamp number of processes to session limit with some buffer for OS */ + max_session_pids = (max_session_pids>500 ? max_session_pids -500 :0 ); +if (nprocs>= max_session_pids) { +tst_resm (TINFO , +"Requested number of processes higher than user session limit (%d> %d), " +"setting to %d" , nprocs, max_session_pids, + max_session_pids); + nprocs = max_session_pids; + } + } +srand (getpid ()); tid =-1 ;
PATCH 1/2 syscalls/msgstress03: fix fork failure on small memory systems
Quoted 10 lines · On 23/06/2021 11:28, [email protected] wrote:
Hi Krzysztof
Quoted 9 lines
Running syscalls/msgstress03 on a system with less than ~4 GB of RAM fails:
msgstress03 1 TFAIL : msgstress03.c:155: Fork failed (may be OK if under stress)
In dmesg:
LTP: starting msgstress03
cgroup: fork rejected by pids controller in /user.slice/user-1000.slice/session-1.scope
Quoted 6 lines
Quoted 4 lines
The reason is cgroups pid limit set by systemd user.slice. The limit is set for login session, also for root user. For example on 2 GB RAM machine it is set as:
/sys/fs/cgroup/pids/user.slice/user-0.slice/pids.max:5207This file only exists on cgroupv1 and cgroupv2 should use
/sys/fs/cgroup/user.slice/user-0.slice/pids.max.
I can use both paths.
Quoted 3 lines
Also if some embedded system doesn't have systemd, do we have a generic way or fallback way to calculate the pid max?
This slice is set by systemd, so no-systemd systems are out of scope because:
1. They will not be affected by this.
2. The limits there might be set by other init manager so other patch should come in. Making a generic solution for unknown-init-managers is a much bigger task and not necessarily needed,
Best regards,
Krzysztof
PATCH 1/2 syscalls/msgstress03: fix fork failure on small memory systems
In reply to Krzysztof KozlowskiHi!
Quoted 55 lines
libs/libltpipc/libipc.c +48
-0+/* +* Get the effective session UID - either one invoking current test via sudo +* or the real UID. +*/ +int get_session_uid (void ) +{ +const char *sudo_uid; + + sudo_uid =getenv ("SUDO_UID" ); +if (sudo_uid) { +int real_uid; + +TEST (sscanf (sudo_uid,"%u" , &real_uid)); +if (TEST_RETURN !=1 ) { +#ifdef DEBUG +tst_resm (TINFO ,"No SUDO_UID from env" ); +#endif + }else { +return real_uid; + } + } + +return getuid (); +} +/* * rm_shm() - removes a shared memory segment. */ @@ -218,3 +243,36 @@ int get_max_msgqueues(void)fclose (f);return atoi (buff); } + +/* +* Get the limit of processes for current session configured by systemd user.slice. +* This also applies to root user. +*/ +int get_pids_limit (void ) +{ +int real_uid, ret; +char path[PATH_MAX ]; +long unsigned int max_pids; + + real_uid =get_session_uid (); +if (TEST_RETURN !=1 ) { +tst_resm (TINFO ,"Cannot get UID" ); +return -1 ; + } + + ret =snprintf (path,sizeof (path), +"/sys/fs/cgroup/pids/user.slice/user-%d.slice/pids.max" , + real_uid); +if (ret <0 || (size_t )ret >=sizeof (path)) +return -1 ; +
libs/libltpipc/libipc.c int get_max_msgqueues(void) · line 278
Quoted 10 lines
libs/libltpipc/libipc.c +10
-0+if (access (path,R_OK ) !=0 ) { +tst_resm (TINFO ,"Cannot read session user limits from '%s'" , path); +return -1 ; + } + +SAFE_FILE_SCANF (cleanup, path,"%lu" , &max_pids); +tst_resm (TINFO ,"Found limit of processes %lu (from %s)" , max_pids, path); + +return max_pids; +}
This is quite generic functionality so we may as well put it into the include/tst_pid.h and lib/tst_pid.c as tst_get_pids_limit().
And we do already have tst_get_free_pids_() in there so we can substract that as well if applicable to make it a bit more precise.
Also I think that it may make sense to put the part that substract some constant to leave room for the rest of the system to the library as well. There is no point in having this heuristic in each test.
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH 1/2 syscalls/msgstress03: fix fork failure on small memory systems
Quoted 61 lines · Cyril Hrubis
Hi!
Quoted 60 lines
libs/libltpipc/libipc.c +53
-0+/* +* Get the effective session UID - either one invoking current test via sudo +* or the real UID. +*/ +int get_session_uid (void ) +{ +const char *sudo_uid; + + sudo_uid =getenv ("SUDO_UID" ); +if (sudo_uid) { +int real_uid; + +TEST (sscanf (sudo_uid,"%u" , &real_uid)); +if (TEST_RETURN !=1 ) { +#ifdef DEBUG +tst_resm (TINFO ,"No SUDO_UID from env" ); +#endif + }else { +return real_uid; + } + } + +return getuid (); +} +/* * rm_shm() - removes a shared memory segment. */ @@ -218,3 +243,36 @@ int get_max_msgqueues(void)fclose (f);return atoi (buff); } + +/* +* Get the limit of processes for current session configured by systemd user.slice. +* This also applies to root user. +*/ +int get_pids_limit (void ) +{ +int real_uid, ret; +char path[PATH_MAX ]; +long unsigned int max_pids; + + real_uid =get_session_uid (); +if (TEST_RETURN !=1 ) { +tst_resm (TINFO ,"Cannot get UID" ); +return -1 ; + } + + ret =snprintf (path,sizeof (path), +"/sys/fs/cgroup/pids/user.slice/user-%d.slice/pids.max" , + real_uid); +if (ret <0 || (size_t )ret >=sizeof (path)) +return -1 ; + +if (access (path,R_OK ) !=0 ) { +tst_resm (TINFO ,"Cannot read session user limits from '%s'" , path); +return -1 ; + } +
Quoted 8 lines
libs/libltpipc/libipc.c int get_max_msgqueues(void) · line 278
Quoted 5 lines
libs/libltpipc/libipc.c +5
-0+SAFE_FILE_SCANF (cleanup, path,"%lu" , &max_pids); +tst_resm (TINFO ,"Found limit of processes %lu (from %s)" , max_pids, path); + +return max_pids; +}This is quite generic functionality so we may as well put it into the
include/tst_pid.handlib/tst_pid.castst_get_pids_limit().
Sure, I can move it there.
Quoted 2 lines
And we do already have
tst_get_free_pids_()in there so we can substract that as well if applicable to make it a bit more precise.
I can just merge the code into tst_get_free_pids_(). It's basically the same purpose - how many processes can we have more.
Quoted 4 lines
Also I think that it may make sense to put the part that substract some constant to leave room for the rest of the system to the library as well. There is no point in having this heuristic in each test.
Or pass it as argument to the tst_get_pids_limit()?
Best regards,
Krzysztof
PATCH 1/2 syscalls/msgstress03: fix fork failure on small memory systems
Hi!
Quoted 10 lines
Quoted 2 lines
This is quite generic functionality so we may as well put it into the
include/tst_pid.handlib/tst_pid.castst_get_pids_limit().Sure, I can move it there.
Quoted 2 lines
And we do already have
tst_get_free_pids_()in there so we can substract that as well if applicable to make it a bit more precise.I can just merge the code into
tst_get_free_pids_(). It's basically the same purpose - how many processes can we have more.
Yes actually that would make a lot of sense.
Signature · 2 lines
Cyril Hrubis [email protected]