problems building lchown02.c for uClinux

LTP David Marlin 3 participants 12 messages Aug 17, 2010 – Aug 18, 2010

Share & tools

David Marlin

author

2010-08-17 18:34 UTC #0 lore raw

'lchown02.c' does not build properly due to calls to mmap and get_high_address (which references sbrk) when UCLINUX=1 is defined.

Attached is a patch to skip those sections of the test when building for uClinux.

Signed-off-by d.marlin <[email protected]>

kernel/syscalls/lchown/lchown02.c renamed from kernel/syscalls/lchown/lchown02.c.orig +11 -1

@@ -109,8 +109,12 @@
 int setup3();			/* setup function to test lchown for ENOTDIR */
 int longpath_setup();		/* setup function to test chown for ENAMETOOLONG */
 
-char Longpathname[PATH_MAX + 2];
+#if !defined(UCLINUX)
+char *get_high_address();       /* function from ltp-lib        */
 char High_address_node[64];
+#endif
+
+char Longpathname[PATH_MAX + 2];
 char EXEC_DIR[PATH_MAX];
 char main_test_dir[PATH_MAX + 2];

@@ -123,9 +127,11 @@
 	{
 	SFILE1, "Process is not owner/root", EPERM, setup1}, {
 	SFILE2, "No Search permissions to process", EACCES, setup2}, {
+#if !defined(UCLINUX)
 	High_address_node, "Address beyond address space", EFAULT, no_setup},
 	{
 	(char *)-1, "Negative address", EFAULT, no_setup}, {
+#endif
 	Longpathname, "Pathname too long", ENAMETOOLONG, longpath_setup}, {
 	"", "Pathname is empty", ENOENT, no_setup}, {
 	SFILE3, "Path contains regular file", ENOTDIR, setup3}, {

@@ -184,9 +190,11 @@
 			file_name = Test_cases[ind].pathname;
 			test_desc = Test_cases[ind].desc;
 
+#if !defined(UCLINUX)
 			if (file_name == High_address_node) {
 				file_name = (char *)get_high_address();
 			}
+#endif
 
 			/*
 			 * Call lchown(2) to test different test conditions.

@@ -279,12 +287,14 @@
 		tst_brkm(TBROK | TERRNO, cleanup, "chmod() failed");
 	}
 
+#if !defined(UCLINUX)
 	bad_addr = mmap(0, 1, PROT_NONE,
 			MAP_PRIVATE_EXCEPT_UCLINUX | MAP_ANONYMOUS, 0, 0);
 	if (bad_addr == MAP_FAILED) {
 		tst_brkm(TBROK | TERRNO, cleanup, "mmap failed");
 	}
 	Test_cases[3].pathname = bad_addr;
+#endif
 
 	/* call individual setup functions */
 	for (ind = 0; Test_cases[ind].desc != NULL; ind++) {

------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev

_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Garrett Cooper

2010-08-17 19:55 UTC #1 lore raw

Quoted 6 lines · David Marlin

'lchown02.c' does not build properly due to calls to mmap and get_high_address (which references sbrk) when UCLINUX=1 is defined.
Attached is a patch to skip those sections of the test when building for uClinux.

I understand the mmap(2) part, but couldn't the get_high_address
portion be done in the LTP headers and library files? It would help if there was a breakdown of what functionality is and isn't available in uclinux that's gating builds / testing (I honestly don't know right offhand what those pieces are).
Thanks!
-Garrett

------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

David Marlin

author

2010-08-17 20:01 UTC #2 lore raw

Quoted 9 lines · Garrett Cooper

Quoted 5 lines · David Marlin
'lchown02.c' does not build properly due to calls to mmap and get_high_address (which references sbrk) when UCLINUX=1 is defined.
Attached is a patch to skip those sections of the test when building for uClinux.
I understand the mmap(2) part, but couldn't the get_high_address
portion be done in the LTP headers and library files?

Possibly. I was just copying the way it was handled in some other LTP tests, i.e.,

testcases/kernel/syscalls/link/link04.c
testcases/kernel/syscalls/lstat/lstat02.c

but I'm open to suggestions.

Quoted 4 lines

It would help if
there was a breakdown of what functionality is and isn't available in uclinux that's gating builds / testing (I honestly don't know right offhand what those pieces are).

Me either. I'm just using trial and error at the moment (see what works). Many (most) syscalls tests have code to skip unsupported functions based on UCLINUX=1, but a few seem to be missing that test.

I'm working my way through the list, so suggestions are welcome.

Thank you,

d.marlin
==========

Quoted 2 lines

Thanks!
-Garrett

------------------------------------------------------------------------------
This SF.net email is sponsored by

Mike Frysinger

2010-08-17 20:14 UTC #3 lore raw

Quoted 9 lines · David Marlin

Quoted 4 lines · Garrett Cooper
It would help if
there was a breakdown of what functionality is and isn't available in uclinux that's gating builds / testing (I honestly don't know right offhand what those pieces are).
Me either. I'm just using trial and error at the moment (see what works). Many (most) syscalls tests have code to skip unsupported functions based on UCLINUX=1, but a few seem to be missing that test.

there are some which shouldnt be skipped any more, so i wouldnt use the existence of UCLINUX code as a clear indicator that something is "OK"
-mike

------------------------------------------------------------------------------
This SF.net email is sponsored by

Mike Frysinger

2010-08-17 20:21 UTC #4 lore raw

In reply to Garrett Cooper

Quoted 4 lines · Garrett Cooper

It would help if
there was a breakdown of what functionality is and isn't available in uclinux that's gating builds / testing (I honestly don't know right offhand what those pieces are).

it's not that hard, but maybe it's just because i'm used to it now. some of the checks are outdated though.

i guess i could write up a small ltp guide for people who arent familiar with nommu ...
-mike

------------------------------------------------------------------------------
This SF.net email is sponsored by

Mike Frysinger

2010-08-17 22:18 UTC #5 lore raw

Quoted 2 lines · Mike Frysinger

i guess i could write up a small ltp guide for people who arent familiar with nommu ...

ive written this: doc/nommu-notes.txt

lemme know if you guys have questions/clarifications -mike

------------------------------------------------------------------------------
This SF.net email is sponsored by

Mike Frysinger

2010-08-17 20:39 UTC #6 lore raw

In reply to David Marlin

Quoted 2 lines · David Marlin

'lchown02.c' does not build properly due to calls to mmap and get_high_address (which references sbrk) when UCLINUX=1 is defined.

you'll need to clarify. it builds fine for me, it just may not run correctly due to the assumptions of memory placement.

for runtime issues, how about we change lib/get_high_address.c to return NULL for UCLINUX targets. then in lchown.c, we can check for NULL and skip the test when we get that value.

that leaves your ifdefing of -1 pointer address. any sane nommu port should reject that address in the kernel. if yours isnt, fix your kernel.
-mike

------------------------------------------------------------------------------
This SF.net email is sponsored by

David Marlin

author

2010-08-17 21:42 UTC #7 lore raw

Quoted 6 lines · Mike Frysinger

Quoted 2 lines · David Marlin
'lchown02.c' does not build properly due to calls to mmap and get_high_address (which references sbrk) when UCLINUX=1 is defined.
you'll need to clarify. it builds fine for me, it just may not run correctly due to the assumptions of memory placement.

Thank you for the feedback. I'm new to uClinux, so I'm only going on what I've read, but my understanding is that 'sbrk' is not supported in uClinux. My information may be a bit dated, so a newer (better) reference would be welcome.

Much of the syscalls test code that is "#ifdef'd" out for UCLINUX=1 involves skipping references (directly or indirectly) to sbrk. get_high_address is one of those references, where it directly calls sbrk:

   char *get_high_address(void)
   {
        return (char *)sbrk(0) + (4 * getpagesize());
   }

Most of the testcases skip the call to get_high_address when UCLINUX=1, so I was just following their example.

Quoted 3 lines

for runtime issues, how about we change lib/get_high_address.c to return NULL for UCLINUX targets. then in lchown.c, we can check for NULL and skip the test when we get that value.

I'm flexible, so any approach that works is ok with me. I assume we'll need to locate all the tests that have been skipping get_high_address for uClinux and update them, as appropriate. Will we also need to change the approach for tests that directly reference sbrk, for example,

testcases/kernel/syscalls/stat/stat06.c testcases/kernel/syscalls/setgroups/setgroups04.c

Quoted 3 lines

that leaves your ifdefing of -1 pointer address. any sane nommu port should reject that address in the kernel. if yours isnt, fix your kernel.

Again, I was just following the example of the existing LTP test code, i.e.,

testcases/kernel/syscalls/lstat/lstat02.c

I can adjust to any approach that LTP follows. (I have not actually tried this test on our kernel yet, so it may just work)

Thank you,

d.marlin
==========

Quoted 5 lines

-mike
------------------------------------------------------------------------------
This SF.net email is sponsored by

Quoted 7 lines

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

------------------------------------------------------------------------------
This SF.net email is sponsored by

Mike Frysinger

2010-08-17 22:26 UTC #8 lore raw

Quoted 10 lines · David Marlin

Quoted 6 lines · Mike Frysinger
Quoted 2 lines · David Marlin
'lchown02.c' does not build properly due to calls to mmap and get_high_address (which references sbrk) when UCLINUX=1 is defined.
you'll need to clarify. it builds fine for me, it just may not run correctly due to the assumptions of memory placement.
Thank you for the feedback. I'm new to uClinux, so I'm only going on what I've read, but my understanding is that 'sbrk' is not supported in uClinux.

it depends on how you use it. sbrk(0) should always work since you're only querying the current size. sbrk(1+) might work depending on the current memory layout, but it isnt reliable, so it's recommended people dont use it when possible on nommu. technically speaking, it can even fail under mmu systems, it's just highly unlikely with newly started processes.

the lib/get_high_address.c usage is valid since it does sbrk(0), but overall the function is useless since it is assuming that a few pages after the end of the sbrk() region is unallocated memory. on nommu systems, it could easily be valid (either the same process or a different one or the kernel).

so on nommu/linux, this function should be avoided due to its design flaws, not because any of the unctions it calls do not work.

Quoted 2 lines

My information may be a bit dated, so a newer (better) reference would be welcome.

you will often find nommu stuff undocumented. my experience is that the people who know arent interested in writing documentation (why should they when they already know the answer !?), and there arent that many people who know :P. i dont mind writing documentation though (due to my personal experience of constantly hitting this problem and making my stomach churn), so feel free to ask questions.

Quoted 11 lines

Much of the syscalls test code that is "#ifdef'd" out for UCLINUX=1 involves skipping references (directly or indirectly) to sbrk. get_high_address is one of those references, where it directly calls sbrk:
char *get_high_address(void)
{
return (char *)sbrk(0) + (4 * getpagesize()); }
Most of the testcases skip the call to get_high_address when UCLINUX=1, so I was just following their example.

i like to look forward and start down a better path rather than continue to hedge overgrown crap ;)
-mike

------------------------------------------------------------------------------
This SF.net email is sponsored by

David Marlin

author

2010-08-18 00:13 UTC #9 lore raw

Quoted 17 lines · Mike Frysinger

Quoted 8 lines · David Marlin
Quoted 5 lines · Mike Frysinger
Quoted 2 lines · David Marlin
'lchown02.c' does not build properly due to calls to mmap and get_high_address (which references sbrk) when UCLINUX=1 is defined.
you'll need to clarify. it builds fine for me, it just may not run correctly due to the assumptions of memory placement.
Thank you for the feedback. I'm new to uClinux, so I'm only going on what I've read, but my understanding is that 'sbrk' is not supported in uClinux.
it depends on how you use it. sbrk(0) should always work since you're only querying the current size. sbrk(1+) might work depending on the current memory layout, but it isnt reliable, so it's recommended people dont use it when possible on nommu. technically speaking, it can even fail under mmu systems, it's just highly unlikely with newly started processes.

Quoted 8 lines

the lib/get_high_address.c usage is valid since it does sbrk(0), but overall the function is useless since it is assuming that a few pages after the end of the sbrk() region is unallocated memory. on nommu systems, it could easily be valid (either the same process or a different one or the kernel).
so on nommu/linux, this function should be avoided due to its design flaws, not because any of the unctions it calls do not work.

Thank you for the excellent explanation.

Quoted 5 lines

Quoted 2 lines
Most of the testcases skip the call to get_high_address when UCLINUX=1, so I was just following their example.
i like to look forward and start down a better path rather than continue to hedge overgrown crap ;)

Understood.

When creating patches I typically look to the existing sources for examples. This is because I assume that 1) the current code has already been reviewed and accepted, 2) the people who wrote it know more about what they were trying to accomplish and how they wanted it to work than I do, and 3) consistency in the code and coding style tend to make maintenance easier.

I understand that over time code may become a bit dated and require some redesign/updates, but being new to these bits of code I defer to the experts for that.

There appear to be three primary ways tests are being handled in LTP for uClinux now:
1) "filter" the test directory (FILTER_OUT_DIRS in the Makefile) e.g., testcases/kernel/syscalls/Makefile
2) skip the entire 'main' test, and display "not available in uCLinux" e.g., testcases/kernel/syscalls/mremap/mremap03.c
3) skip only parts of the test
e.g., testcases/kernel/syscalls/stat/stat06.c

There may be other ways that I have not noticed (yet).

I think the question is, do we want to continue using these approaches, or try to standardize on one or two (possibly new) approaches, with some guidelines on when to use each?

I am willing to help with any modifications, but I don't have enough experience with uClinux (or LTP development) to determine the best approach(es) or rules for use. Once guidelines are established I can help update the code (submit patches). As a 'user' of LTP, my goal is just to be able to set UCLINUX=1, build, install, and run the tests on a
system running uClinux. :-)

d.marlin
==========

Quoted 5 lines

-mike
------------------------------------------------------------------------------
This SF.net email is sponsored by

Quoted 7 lines

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

------------------------------------------------------------------------------
This SF.net email is sponsored by

Garrett Cooper

2010-08-18 00:51 UTC #10 lore raw

Quoted 66 lines · David Marlin

Quoted 17 lines · Mike Frysinger
Quoted 8 lines · David Marlin
Quoted 5 lines · Mike Frysinger
Quoted 2 lines · David Marlin
'lchown02.c' does not build properly due to calls to mmap and get_high_address (which references sbrk) when UCLINUX=1 is defined.
you'll need to clarify. it builds fine for me, it just may not run correctly due to the assumptions of memory placement.
Thank you for the feedback. I'm new to uClinux, so I'm only going on what I've read, but my understanding is that 'sbrk' is not supported in uClinux.
it depends on how you use it. sbrk(0) should always work since you're only querying the current size. sbrk(1+) might work depending on the current memory layout, but it isnt reliable, so it's recommended people dont use it when possible on nommu. technically speaking, it can even fail under mmu systems, it's just highly unlikely with newly started processes.
Quoted 8 lines
the lib/get_high_address.c usage is valid since it does sbrk(0), but overall the function is useless since it is assuming that a few pages after the end of the sbrk() region is unallocated memory. on nommu systems, it could easily be valid (either the same process or a different one or the kernel).
so on nommu/linux, this function should be avoided due to its design flaws, not because any of the unctions it calls do not work.
Thank you for the excellent explanation.
:
Quoted 5 lines
Quoted 2 lines
Most of the testcases skip the call to get_high_address when UCLINUX=1, so I was just following their example.
i like to look forward and start down a better path rather than continue to hedge overgrown crap ;)
Understood.
When creating patches I typically look to the existing sources for examples. This is because I assume that 1) the current code has already been reviewed and accepted, 2) the people who wrote it know more about what they were trying to accomplish and how they wanted it to work than I do, and 3) consistency in the code and coding style tend to make maintenance easier.
I understand that over time code may become a bit dated and require some redesign/updates, but being new to these bits of code I defer to the experts for that.
There appear to be three primary ways tests are being handled in LTP for uClinux now:
1) "filter" the test directory (FILTER_OUT_DIRS in the Makefile) e.g., testcases/kernel/syscalls/Makefile
2) skip the entire 'main' test, and display "not available in uCLinux" e.g., testcases/kernel/syscalls/mremap/mremap03.c
3) skip only parts of the test
e.g., testcases/kernel/syscalls/stat/stat06.c
There may be other ways that I have not noticed (yet).
I think the question is, do we want to continue using these approaches, or try to standardize on one or two (possibly new) approaches, with some guidelines on when to use each?

Quoted 6 lines

I am willing to help with any modifications, but I don't have enough experience with uClinux (or LTP development) to determine the best approach(es) or rules for use. Once guidelines are established I can help update the code (submit patches). As a 'user' of LTP, my goal is just to be able to set UCLINUX=1, build, install, and run the tests on a system running uClinux. :-)

FILTER_OUT_DIRS is the most recent addition that I made, but the
only caveat is that the runltp scripts are dumb and can't distinguish between "not built" and "failed" easily (you have to manually go in and inspect the results, which is annoying), so this can add a lot of noise to the output in a test run, depending on what you run. I was going to fix this, but then I switched over to more FreeBSD work than Linux work, and it kind of got shelved. If it's worth picking up and completing, I'll do it, but I need to finish off writing a few more conformance tests and fix a set of build-related items with open_posix_testsuite before I pick that up.
Thanks,
-Garrett

------------------------------------------------------------------------------
This SF.net email is sponsored by

Mike Frysinger

2010-08-18 00:54 UTC #11 lore raw

In reply to David Marlin

Quoted 5 lines · David Marlin

When creating patches I typically look to the existing sources for examples. This is because I assume that 1) the current code has already been reviewed and accepted, 2) the people who wrote it know more about what they were trying to accomplish and how they wanted it to work than I do, and 3) consistency in the code and coding style tend to make maintenance easier.

having been part of (1) a bit, worked with most of the people doing (2), and seen the whole tree with (3), i can safely say that your points, while the ideal we should strive for, when it comes to LTP, i wouldnt hold your breath on any of them.

Quoted 3 lines

I understand that over time code may become a bit dated and require some redesign/updates, but being new to these bits of code I defer to the experts for that.

not really the case here. my understanding is that LTP has seen large code dumps from different large organizations (SGI/IBM/etc...), so the quality (in many respects) can quickly vary.

Quoted 8 lines

There appear to be three primary ways tests are being handled in LTP for uClinux now:
1) "filter" the test directory (FILTER_OUT_DIRS in the Makefile) e.g., testcases/kernel/syscalls/Makefile
2) skip the entire 'main' test, and display "not available in uCLinux" e.g., testcases/kernel/syscalls/mremap/mremap03.c
3) skip only parts of the test
e.g., testcases/kernel/syscalls/stat/stat06.c

(3) is obviously the preference -- some coverage is better than none. between (1) and (2), ive done a little of both, but dont have much preference in either direction.

pros/cons:
(1/con) it can be confusing as to why a test isnt compiled & executed forcing people to read the source code and/or build system for each test to figure out what's going on
(1/pro) saves on build time
(2/con) build time over head, and some people might find the output hard to filter, but we use the same output in other tests to do runtime checking of capabilities
(2/pro) no confusing missing binaries and reasoning for skipped tests are clear & up front

Quoted 3 lines

I think the question is, do we want to continue using these approaches, or try to standardize on one or two (possibly new) approaches, with some guidelines on when to use each?

standardization is good, it's just a matter of coercing someone into doing the actual work

Quoted 6 lines

I am willing to help with any modifications, but I don't have enough experience with uClinux (or LTP development) to determine the best approach(es) or rules for use. Once guidelines are established I can help update the code (submit patches). As a 'user' of LTP, my goal is just to be able to set UCLINUX=1, build, install, and run the tests on a system running uClinux. :-)

doing UCLINUX=1 should be automated a bit now by autoconf tests for fork(). for Blackfin toolchains it certainly works (no fork() symbol).
-mike

------------------------------------------------------------------------------
This SF.net email is sponsored by