Skip to contentSkip to conversation

LTP compilation broken with -fno-common

LTP Petr Vorel 5 participants 17 messages Apr 1, 2020 – Apr 16, 2020 Library code

lore
Share & tools
lore

Petr Vorel

author
#0 lore raw

Hi,

LTP compilation breaks with -fno-common (we don't use as the default). This is the default for GCC 10, is there a way to fix it or do we need to disable it?

Compilation on gcc-10 (or gcc-9 with CFLAGS="-fno-common":

gcc-9 -Werror=implicit-function-declaration -fno-common -g -O2 -fno-strict-aliasing -pipe -Wall -W -Wold-style-definition -W -I../../include -I../../include -I../../include/old/ -L../../lib tst_checkpoint_wait_timeout.c -lltp -o tst_checkpoint_wait_timeout
/usr/lib64/gcc/x86_64-suse-linux/9/../../../../x86_64-suse-linux/bin/ld: ../../lib/libltp.a(tst_test.o):ltp.git/lib/../include/tst_test.h:325: multiple definition of `TCID'; /tmp/cc49yYO6.o:ltp.git/lib/tests/tst_strsig.c:27: first defined here /usr/lib64/gcc/x86_64-suse-linux/9/../../../../x86_64-suse-linux/bin/ld: ../../lib/libltp.a(tst_sys_conf.o):ltp.git/lib/../include/tst_test.h:325: multiple definition of `TCID'; /tmp/cc49yYO6.o:ltp.git/lib/tests/tst_strsig.c:27: first defined here

It's somehow related to the "hack" in include/tst_test.h:

/*
* This is a hack to make the testcases link without defining TCID */
extern const char *TCID;

Kind regards,
Petr

#1 lore raw

Hello,

Quoted 11 lines · Petr Vorel

Hi,

LTP compilation breaks with -fno-common (we don't use as the default). This is the default for GCC 10, is there a way to fix it or do we need to disable it?

Compilation on gcc-10 (or gcc-9 with CFLAGS="-fno-common":

gcc-9 -Werror=implicit-function-declaration -fno-common -g -O2 -fno-strict-aliasing -pipe -Wall -W -Wold-style-definition -W -I../../include -I../../include -I../../include/old/ -L../../lib tst_checkpoint_wait_timeout.c -lltp -o tst_checkpoint_wait_timeout
/usr/lib64/gcc/x86_64-suse-linux/9/../../../../x86_64-suse-linux/bin/ld: ../../lib/libltp.a(tst_test.o):ltp.git/lib/../include/tst_test.h:325: multiple definition of `TCID'; /tmp/cc49yYO6.o:ltp.git/lib/tests/tst_strsig.c:27: first defined here /usr/lib64/gcc/x86_64-suse-linux/9/../../../../x86_64-suse-linux/bin/ld: ../../lib/libltp.a(tst_sys_conf.o):ltp.git/lib/../include/tst_test.h:325: multiple definition of `TCID'; /tmp/cc49yYO6.o:ltp.git/lib/tests/tst_strsig.c:27: first defined here

Quoted 6 lines

It's somehow related to the "hack" in include/tst_test.h:

/*
* This is a hack to make the testcases link without defining TCID */
extern const char *TCID;

Huh? This is defining the variable and with extern which looks correct (at a glance):
https://gcc.gnu.org/gcc-10/porting_to.html

Possibly we need to clean the code up so this variable is not redefined all over the place?

Quoted 3 lines

Kind regards,
Petr

Signature · 2 lines
Thank you,
Richard.
#2 lore raw
In reply to Petr Vorel

Hi!

Quoted 9 lines

LTP compilation breaks with -fno-common (we don't use as the default). This is the default for GCC 10, is there a way to fix it or do we need to disable it?

Compilation on gcc-10 (or gcc-9 with CFLAGS="-fno-common":

gcc-9 -Werror=implicit-function-declaration -fno-common -g -O2 -fno-strict-aliasing -pipe -Wall -W -Wold-style-definition -W -I../../include -I../../include -I../../include/old/ -L../../lib tst_checkpoint_wait_timeout.c -lltp -o tst_checkpoint_wait_timeout
/usr/lib64/gcc/x86_64-suse-linux/9/../../../../x86_64-suse-linux/bin/ld: ../../lib/libltp.a(tst_test.o):ltp.git/lib/../include/tst_test.h:325: multiple definition of `TCID'; /tmp/cc49yYO6.o:ltp.git/lib/tests/tst_strsig.c:27: first defined here /usr/lib64/gcc/x86_64-suse-linux/9/../../../../x86_64-suse-linux/bin/ld: ../../lib/libltp.a(tst_sys_conf.o):ltp.git/lib/../include/tst_test.h:325: multiple definition of `TCID'; /tmp/cc49yYO6.o:ltp.git/lib/tests/tst_strsig.c:27: first defined here

Quoted 6 lines

It's somehow related to the "hack" in include/tst_test.h:

/*
* This is a hack to make the testcases link without defining TCID */
extern const char *TCID;

This is not the original line, the original line is without the extern.

Now this will not be easy to solve since the problem is:

* All old style test define TCID in the test source that also implements main()

* The test library uses that variable in various places

* The variable definition is in the tst_test.h so that all new test also define TCID

We cannot move that definition into tst_test.c beacuse that would mean that old API test would end up with multiple definitions one in the corresponding test and one in the new library library code.

Maybe we can add it with a weak linker attribute. What about this patch:

include/tst_test.h +2 -4
index 259d87d60..352e2cd64 100644
@@ -322,9 +322,7 @@ int main(int argc, char *argv[])
 
 #define TST_TEST_TCONF(message)                                 \
         static struct tst_test test = { .tconf_msg = message  } \
-/*
- * This is a hack to make the testcases link without defining TCID
- */
-const char *TCID;
+
+extern const char *TCID;
 
 #endif /* TST_TEST_H__ */
lib/tst_test.c +5 -0
index 73827d472..4ce8cc301 100644
@@ -32,6 +32,11 @@
 #include "old_device.h"
 #include "old_tmpdir.h"
 
+/*
+ * Hack to get TCID defined in newlib tests
+ */
+const char *TCID __attribute__((weak));
+
 #define LINUX_GIT_URL "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id="
 #define CVE_DB_URL "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-"
 

I guess that we can even drop the TCID extern definition from the tst_test.h header if the weak attribute works fine.

Signature · 2 lines
Cyril Hrubis
[email protected]
#3 lore raw
Quoted 3 lines · Cyril Hrubis

We cannot move that definition into tst_test.c beacuse that would mean that old API test would end up with multiple definitions one in the corresponding test and one in the new library library code.

But we can move it a few lines up in tst_main.h so that it gets defined only together with the default main(), not in every .c file that includes tst_test.h.

Signature · 7 lines
Martin Doucha   [email protected]
QA Engineer for Software Maintenance
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic
#4 lore raw

Hi!

Quoted 7 lines
Quoted 3 lines

We cannot move that definition into tst_test.c beacuse that would mean that old API test would end up with multiple definitions one in the corresponding test and one in the new library library code.

But we can move it a few lines up in tst_main.h so that it gets defined only together with the default main(), not in every .c file that includes tst_test.h.

The problem here is that gcc10 forbids variables in headers, moving it anywhere int the header wouldn't help.

Signature · 2 lines
Cyril Hrubis
[email protected]
#5 lore raw
Quoted 2 lines · Cyril Hrubis

The problem here is that gcc10 forbids variables in headers, moving it anywhere int the header wouldn't help.

I don't think GCC cares about where exactly the variable came from. It's only refusing to link multiple definitions of the same symbol. So if we move the TCID definition into the TST_NO_DEFAULT_MAIN guard block, GCC linker should stop complaining because the symbol name will become unique.

Signature · 7 lines
Martin Doucha   [email protected]
QA Engineer for Software Maintenance
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic
author
#10 lore raw
In reply to Cyril Hrubis

Hi Cyril,

...

Quoted 23 lines

Maybe we can add it with a weak linker attribute. What about this patch:

include/tst_test.h +2 -4
index 259d87d60..352e2cd64 100644
@@ -322,9 +322,7 @@ int main(int argc, char *argv[])
 
 #define TST_TEST_TCONF(message)                                 \
         static struct tst_test test = { .tconf_msg = message  } \
-/*
- * This is a hack to make the testcases link without defining TCID
- */
-const char *TCID;
+
+extern const char *TCID;
 
 #endif /* TST_TEST_H__ */
lib/tst_test.c +0 -0
index 73827d472..4ce8cc301 100644

lib/tst_test.c line 41

Quoted 10 lines
lib/tst_test.c +5 -0
@@ -32,6 +32,11 @@
 #include "old_device.h"
 #include "old_tmpdir.h"
 
+/*
+ * Hack to get TCID defined in newlib tests
+ */
+const char *TCID __attribute__((weak));
+
 #define LINUX_GIT_URL "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id="
 #define CVE_DB_URL "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-"

Yep working locally :).
https://travis-ci.org/github/pevik/ltp/builds/669619899

Quoted 2 lines

I guess that we can even drop the TCID extern definition from the tst_test.h header if the weak attribute works fine.

Also this is working.
Let's wait for travis (also added -fno-common into CFLAGS for all jobs).
https://travis-ci.org/github/pevik/ltp/builds/669620839

Kind regards,
Petr

author
#6 lore raw
In reply to Martin Doucha

Hi Martin,

Quoted 8 lines · Cyril Hrubis
Quoted 2 lines

The problem here is that gcc10 forbids variables in headers, moving it anywhere int the header wouldn't help.

I don't think GCC cares about where exactly the variable came from. It's only refusing to link multiple definitions of the same symbol. So if we move the TCID definition into the TST_NO_DEFAULT_MAIN guard block, GCC linker should stop complaining because the symbol name will become unique.

No, moving the definition above #ifndef TST_NO_DEFAULT_MAIN does not help.

Kind regards,
Petr

#7 lore raw
In reply to Martin Doucha

Hi!

Quoted 7 lines
Quoted 2 lines

The problem here is that gcc10 forbids variables in headers, moving it anywhere int the header wouldn't help.

I don't think GCC cares about where exactly the variable came from. It's only refusing to link multiple definitions of the same symbol. So if we move the TCID definition into the TST_NO_DEFAULT_MAIN guard block, GCC linker should stop complaining because the symbol name will become unique.

See:

https://gcc.gnu.org/gcc-10/porting_to.html

Signature · 2 lines
Cyril Hrubis
[email protected]
#8 lore raw
Quoted 4 lines · Cyril Hrubis

Hi!

Quoted 3 lines
Quoted 2 lines

The problem here is that gcc10 forbids variables in headers, moving it anywhere int the header wouldn't help.

Quoted 9 lines
Quoted 4 lines

I don't think GCC cares about where exactly the variable came from. It's only refusing to link multiple definitions of the same symbol. So if we move the TCID definition into the TST_NO_DEFAULT_MAIN guard block, GCC linker should stop complaining because the symbol name will become unique.

See:

https://gcc.gnu.org/gcc-10/porting_to.html

This patch fixes the "multiple definition of TCID'" error. But never mind because we'll get the "undefined reference to TCID'" error instead from about 50 programs files that use the new library but define their own main().

Reminder that "multiple definition of `TCID'" is a linker error. If defining variables in .h files were really forbidden, it'd be a preprocessor error instead.

include/tst_test.h +6 -4
index 42c02b549..cc163ae51 100644
@@ -308,6 +308,10 @@ char *tst_get_tmpdir(void);
 
 #ifndef TST_NO_DEFAULT_MAIN
 
+/*
+ * This is a hack to make the testcases link without defining TCID
+ */
+const char *TCID;
 static struct tst_test test;
 
 int main(int argc, char *argv[])
@@ -319,9 +323,7 @@ int main(int argc, char *argv[])
 
 #define TST_TEST_TCONF(message)                                 \
         static struct tst_test test = { .tconf_msg = message  } \
-/*
- * This is a hack to make the testcases link without defining TCID
- */
-const char *TCID;
+
+extern const char *TCID;
 
 #endif /* TST_TEST_H__ */
Signature · 7 lines
Martin Doucha   [email protected]
QA Engineer for Software Maintenance
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic
#9 lore raw

Hi!

Quoted 13 lines
Quoted 12 lines
Quoted 7 lines
Quoted 2 lines

The problem here is that gcc10 forbids variables in headers, moving it anywhere int the header wouldn't help.

I don't think GCC cares about where exactly the variable came from. It's only refusing to link multiple definitions of the same symbol. So if we move the TCID definition into the TST_NO_DEFAULT_MAIN guard block, GCC linker should stop complaining because the symbol name will become unique.

See:

https://gcc.gnu.org/gcc-10/porting_to.html

Quoted 8 lines

This patch fixes the "multiple definition of TCID'" error. But never mind because we'll get the "undefined reference to TCID'" error instead from about 50 programs files that use the new library but define their own main().

Reminder that "multiple definition of `TCID'" is a linker error. If defining variables in .h files were really forbidden, it'd be a preprocessor error instead.

Well variables in headers are not exactly forbidden by -fno-common but they are not resolved as a single global variable by the linker anymore, which was the default before gcc10. So effectivelly this disallows having variables in headers if the header is included from more than one compilation unit becuase you end up with a linker error.

The end result is overall the same, no more variables in headers in any non-trivial codebase.

Signature · 2 lines
Cyril Hrubis
[email protected]
author
#11 lore raw
In reply to Cyril Hrubis

Hi all,

Quoted 56 lines
Quoted 4 lines

/*
* This is a hack to make the testcases link without defining TCID */
extern const char *TCID;

This is not the original line, the original line is without the extern.

Now this will not be easy to solve since the problem is:

* All old style test define TCID in the test source that also implements main()

* The test library uses that variable in various places

* The variable definition is in the tst_test.h so that all new test also define TCID

We cannot move that definition into tst_test.c beacuse that would mean that old API test would end up with multiple definitions one in the corresponding test and one in the new library library code.

Maybe we can add it with a weak linker attribute. What about this patch:

include/tst_test.h +2 -4
index 259d87d60..352e2cd64 100644
@@ -322,9 +322,7 @@ int main(int argc, char *argv[])
 
 #define TST_TEST_TCONF(message)                                 \
         static struct tst_test test = { .tconf_msg = message  } \
-/*
- * This is a hack to make the testcases link without defining TCID
- */
-const char *TCID;
+
+extern const char *TCID;
 
 #endif /* TST_TEST_H__ */
lib/tst_test.c +5 -0
index 73827d472..4ce8cc301 100644
@@ -32,6 +32,11 @@
 #include "old_device.h"
 #include "old_tmpdir.h"
 
+/*
+ * Hack to get TCID defined in newlib tests
+ */
+const char *TCID __attribute__((weak));
+
 #define LINUX_GIT_URL "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id="
 #define CVE_DB_URL "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-"
 
Quoted 2 lines

I guess that we can even drop the TCID extern definition from the tst_test.h header if the weak attribute works fine.

I'd be for adding this patch (both version works), Cyril, will you send it to ML / push it?

I'd also be for adding -fno-common to Travis after this fix.

Kind regards,
Petr

#12 lore raw

Hi!

Quoted 5 lines
Quoted 2 lines

I guess that we can even drop the TCID extern definition from the tst_test.h header if the weak attribute works fine.

I'd be for adding this patch (both version works), Cyril, will you send it to ML / push it?

Feel free do that yourself. I would be for the version without extern definition in tst_test.h then.

Quoted 1 line

I'd also be for adding -fno-common to Travis after this fix.

Sounds good.

Signature · 2 lines
Cyril Hrubis
[email protected]
#13 lore raw
In reply to Petr Vorel

Hi Petr,

Petr Vorel <[email protected]> wrote:

...

Quoted 9 lines
Quoted 1 line

I guess that we can even drop the TCID extern definition from the

tst_test.h

Quoted 1 line

header if the weak attribute works fine.

I'd be for adding this patch (both version works), Cyril, will you send it to ML / push it?

I'd also be for adding -fno-common to Travis after this fix.

After building LTP with CFLAGS="-fno-common" locally, I just find more places that need to do the same improvement.

ltp/testcases/kernel/controllers/libcontrollers/libcontrollers.h:56:
multiple definition of `num_line'
ltp/lib/../include/tst_hugepage.h:16: multiple definition of `Hopt'

Sorry for involving the definition in tst_hugetlb.h without 'extern' issue many hours ago. It'd be appreciated if you can fix them together in your coming patch.

Hope this is a timely reminder in case you formatting patch V2 :).

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20200409/10c1c32e/attachment.htm>
author
#14 lore raw

Hi Li,

Quoted 7 lines

After building LTP with CFLAGS="-fno-common" locally, I just find more places that need to do the same improvement.

ltp/testcases/kernel/controllers/libcontrollers/libcontrollers.h:56:
multiple definition of `num_line'
ltp/lib/../include/tst_hugepage.h:16: multiple definition of `Hopt'

Quoted 5 lines

Sorry for involving the definition in tst_hugetlb.h without 'extern' issue many hours ago. It'd be appreciated if you can fix them together in your coming patch.

Hope this is a timely reminder in case you formatting patch V2 :).

Thanks for info, working on the fix :).

Kind regards,
Petr

author
#15 lore raw

[LTP] hugemmap01 warnings with -i [was Re: LTP compilation broken with -fno-common]

In reply to Li Wang

Hi Li,

Quoted 10 lines

After building LTP with CFLAGS="-fno-common" locally, I just find more places that need to do the same improvement.

ltp/testcases/kernel/controllers/libcontrollers/libcontrollers.h:56:
multiple definition of `num_line'
ltp/lib/../include/tst_hugepage.h:16: multiple definition of `Hopt'

Sorry for involving the definition in tst_hugetlb.h without 'extern' issue many hours ago. It'd be appreciated if you can fix them together in your coming patch.

BTW that's a fix 156f91396 ("hugetlb: move nr_opt to tst_hugepage.h")

I haven't checked whether -i worked without warning before, but not it produces it:

./hugemmap01 -s 2 -i 2
tst_hugepage.c:48: INFO: 2 hugepage(s) reserved
tst_test.c:1244: INFO: Timeout per run is 0h 05m 00s
hugemmap01.c:61: PASS: call succeeded
hugemmap01.c:61: PASS: call succeeded
hugemmap01.c:69: WARN: Number of HUGEPAGES_FREE stayed the same. Okay if multiple copies running due to test collision.

Kind regards,
Petr

#16 lore raw

[LTP] hugemmap01 warnings with -i [was Re: LTP compilation broken with -fno-common]

Hi Petr,

Quoted 9 lines · Petr Vorel

Hi Li,

Quoted 6 lines

After building LTP with CFLAGS="-fno-common" locally, I just find more places that need to do the same improvement.

ltp/testcases/kernel/controllers/libcontrollers/libcontrollers.h:56:
multiple definition of `num_line'
ltp/lib/../include/tst_hugepage.h:16: multiple definition of `Hopt'

Quoted 9 lines
Quoted 1 line

Sorry for involving the definition in tst_hugetlb.h without 'extern'

issue

Quoted 2 lines

many hours ago. It'd be appreciated if you can fix them together in your coming patch.

BTW that's a fix 156f91396 ("hugetlb: move nr_opt to tst_hugepage.h")

I haven't checked whether -i worked without warning before, but not it produces it:

This is not introduced by hugetlb recent fixes, the warning always exists there in tag ltp-full-20200120.
Anyway, it's a tiny issue and I will have a look when available :).

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20200416/1abdc025/attachment-0001.htm>