Skip to contentSkip to conversation

Fix printk_once build errors due to __read_mostly

RFC PATCH In review James Hogan 6 participants 15 messages Oct 30, 2013 – Nov 14, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 Fix printk_once build errors due to __read_mostly

Commit 3e39c1ab04ba (printk: mark printk_once test variable __read_mostly) added __read_mostly to the __print_once bool in the printk_once() macro, but __read_mostly is defined in <linux/cache.h> which isn't included from <linux/printk.h>. This results in build errors like this:

arch/metag/mm/l2cache.c: In function 'meta_l2c_setup':
arch/metag/mm/l2cache.c:56: error: '__read_mostly' undeclared

This is fixed by adding an include of <linux/cache.h> from <linux/printk.h> since I don't think printk_once() users should need to include <linux/cache.h> for it to work.

Note that this actually adds a recursive include, since <linux/cache.h> includes <linux/kernel.h>, which includes <linux/printk.h>. The actual dependencies are all in macros so it doesn't actually seem to result in any build failures, but it's clearly less than ideal.

---
This build failure was caught in today's linux-next.

I'm not too keen on this due to the recursive include. Is it better to just workaround the problem by including <linux/cache.h> from the .c file that fails to build, or should the original patch be removed instead?

 include/linux/printk.h | 1 +
 1 file changed, 1 insertion(+)
include/linux/printk.h +1 -0
index abd42dd..b13d00e 100644
@@ -2,6 +2,7 @@
 #define __KERNEL_PRINTK__
 
 #include <stdarg.h>
+#include <linux/cache.h>
 #include <linux/init.h>
 #include <linux/kern_levels.h>
 #include <linux/linkage.h>
Signature · 1 line
1.8.1.2
#1 lore raw
Quoted 9 lines · James Hogan

Commit 3e39c1ab04ba (printk: mark printk_once test variable __read_mostly) added __read_mostly to the __print_once bool in the printk_once() macro, but __read_mostly is defined in <linux/cache.h> which isn't included from <linux/printk.h>. This results in build errors like this:

arch/metag/mm/l2cache.c: In function 'meta_l2c_setup':
arch/metag/mm/l2cache.c:56: error: '__read_mostly' undeclared

Quoted 8 lines

This is fixed by adding an include of <linux/cache.h> from <linux/printk.h> since I don't think printk_once() users should need to include <linux/cache.h> for it to work.

Note that this actually adds a recursive include, since <linux/cache.h> includes <linux/kernel.h>, which includes <linux/printk.h>. The actual dependencies are all in macros so it doesn't actually seem to result in any build failures, but it's clearly less than ideal.

Thanks James.

Oh well. Pity about the recursive #include. It seems most every actual use of __read_mostly gets that cache.h include indirectly.

Perhaps it's better just to remove the __read_mostly patch as the __read_mostly value is pretty low.

Andrew, could you please remove my __read_mostly patch?

I preferred that printk.h not be a file that
could be independently used via #include.

https://lkml.org/lkml/2011/5/25/331
https://lkml.org/lkml/2011/5/25/333

and that only kernel.h should be used.

Maybe that's still a better approach.

#2 lore raw
In reply to James Hogan
Quoted 8 lines · James Hogan

Commit 3e39c1ab04ba (printk: mark printk_once test variable __read_mostly) added __read_mostly to the __print_once bool in the printk_once() macro, but __read_mostly is defined in <linux/cache.h> which isn't included from <linux/printk.h>. This results in build errors like this:

arch/metag/mm/l2cache.c: In function 'meta_l2c_setup':
arch/metag/mm/l2cache.c:56: error: '__read_mostly' undeclared

Fengguang?

Do you have the metag arch cross-compiler in your auto build tests?

Is there a list somewhere of what arches are build tested?

cheers, Joe

#3 lore raw

Hi Joe,

Quoted 7 lines · Joe Perches
Quoted 6 lines · James Hogan

Commit 3e39c1ab04ba (printk: mark printk_once test variable __read_mostly) added __read_mostly to the __print_once bool in the printk_once() macro, but __read_mostly is defined in <linux/cache.h> which isn't included from <linux/printk.h>. This results in build errors like this:

Quoted 6 lines
Quoted 2 lines

arch/metag/mm/l2cache.c: In function 'meta_l2c_setup':
arch/metag/mm/l2cache.c:56: error: '__read_mostly' undeclared

Fengguang?

Do you have the metag arch cross-compiler in your auto build tests?

Nope.

Quoted 1 line

Is there a list somewhere of what arches are build tested?

It's pretty much all the ones supported by

        https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/

Here are my installed ones (the highest version will be used if there are several versions available):

gcc-4.2.4-nolibc/avr32-linux     gcc-4.6.3-nolibc/frv-linux     gcc-4.6.3-nolibc/mips64-linux     gcc-4.6.3-nolibc/sparc-linux
gcc-4.5.1-nolibc/or32-linux      gcc-4.6.3-nolibc/hppa64-linux  gcc-4.6.3-nolibc/mips-linux       gcc-4.6.3-nolibc/x86_64-linux
gcc-4.6.2-nolibc/tilegx-linux    gcc-4.6.3-nolibc/hppa-linux    gcc-4.6.3-nolibc/powerpc64-linux  gcc-4.6.3-nolibc/xtensa-linux
gcc-4.6.3-nolibc/alpha-linux     gcc-4.6.3-nolibc/i386-linux    gcc-4.6.3-nolibc/powerpc-linux    gcc-4.8.0-nolibc/microblaze-linux
gcc-4.6.3-nolibc/am33_2.0-linux  gcc-4.6.3-nolibc/ia64-linux    gcc-4.6.3-nolibc/s390x-linux      gcc-4.8.0-nolibc/powerpc64-linux
gcc-4.6.3-nolibc/cris-linux      gcc-4.6.3-nolibc/m32r-linux    gcc-4.6.3-nolibc/sh4-linux
gcc-4.6.3-nolibc/crisv32-linux   gcc-4.6.3-nolibc/m68k-linux    gcc-4.6.3-nolibc/sparc64-linux

Thanks,
Fengguang

#4 lore raw
In reply to James Hogan
Quoted 9 lines · James Hogan

Commit 3e39c1ab04ba (printk: mark printk_once test variable __read_mostly) added __read_mostly to the __print_once bool in the printk_once() macro, but __read_mostly is defined in <linux/cache.h> which isn't included from <linux/printk.h>. This results in build errors like this:

arch/metag/mm/l2cache.c: In function 'meta_l2c_setup':
arch/metag/mm/l2cache.c:56: error: '__read_mostly' undeclared

Quoted 8 lines

This is fixed by adding an include of <linux/cache.h> from <linux/printk.h> since I don't think printk_once() users should need to include <linux/cache.h> for it to work.

Note that this actually adds a recursive include, since <linux/cache.h> includes <linux/kernel.h>, which includes <linux/printk.h>. The actual dependencies are all in macros so it doesn't actually seem to result in any build failures, but it's clearly less than ideal.

[]

Quoted 4 lines

I'm not too keen on this due to the recursive include. Is it better to just workaround the problem by including <linux/cache.h> from the .c file that fails to build, or should the original patch be removed instead?

Maybe also using this would help avoid some of the circular dependency:

---
 include/linux/cache.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
include/linux/cache.h +2 -2
index 4c57065..17e7e82 100644
@@ -1,11 +1,11 @@
 #ifndef __LINUX_CACHE_H
 #define __LINUX_CACHE_H
 
-#include <linux/kernel.h>
+#include <uapi/linux/kernel.h>
 #include <asm/cache.h>
 
 #ifndef L1_CACHE_ALIGN
-#define L1_CACHE_ALIGN(x) ALIGN(x, L1_CACHE_BYTES)
+#define L1_CACHE_ALIGN(x) __ALIGN_KERNEL(x, L1_CACHE_BYTES)
 #endif
 
 #ifndef SMP_CACHE_BYTES
#5 lore raw
In reply to James Hogan
Quoted 9 lines · On Wed, 30 Oct 2013 10:18:21 +0000 James Hogan <[email protected]> wrote:

Commit 3e39c1ab04ba (printk: mark printk_once test variable __read_mostly) added __read_mostly to the __print_once bool in the printk_once() macro, but __read_mostly is defined in <linux/cache.h> which isn't included from <linux/printk.h>. This results in build errors like this:

arch/metag/mm/l2cache.c: In function 'meta_l2c_setup':
arch/metag/mm/l2cache.c:56: error: '__read_mostly' undeclared

Quoted 8 lines

This is fixed by adding an include of <linux/cache.h> from <linux/printk.h> since I don't think printk_once() users should need to include <linux/cache.h> for it to work.

Note that this actually adds a recursive include, since <linux/cache.h> includes <linux/kernel.h>, which includes <linux/printk.h>. The actual dependencies are all in macros so it doesn't actually seem to result in any build failures, but it's clearly less than ideal.

Yitch.

Quoted 11 lines

---
This build failure was caught in today's linux-next.

I'm not too keen on this due to the recursive include. Is it better to just workaround the problem by including <linux/cache.h> from the .c file that fails to build, or should the original patch be removed instead?

Well, these patches make the kernel worse. We should make it better, not worse.

One approach would be to add a new printk_once.h, which includes printk.h and cache.h.

I can't immediately think of a nice solution so I guess for now I'll drop printk-mark-printk_once-test-variable-__read_mostly.patch.

#6 lore raw
Quoted 1 line · Andrew Morton

On Wed, 30 Oct 2013 10:18:21 +0000 James Hogan <[email protected]> wrote:

[]

Quoted 6 lines
Quoted 4 lines

Note that this actually adds a recursive include, since <linux/cache.h> includes <linux/kernel.h>, which includes <linux/printk.h>. The actual dependencies are all in macros so it doesn't actually seem to result in any build failures, but it's clearly less than ideal.

Yitch.

Yeah.

There are about ~700 files in the tree that use __read_mostly. Only about ~25 of those actually #include <linux/cache.h>

I did a script that adds the #include, but 700 files is a _lot_.

For today's next:

$ git grep -w --name-only __read_mostly | \
  xargs grep -P -l "^\s*#\s*include\s+<linux/cache.h>" | wc -l
28
$ git grep -w --name-only __read_mostly | \
  xargs grep -P -L "^\s*#\s*include\s+<linux/cache.h>" | wc -l
716

Untangling that crud is nasty.

#7 lore raw

PATCH v2 1/1 printk/cache: Mark printk_once test variable __read_mostly

In reply to Andrew Morton

Add #include <linux/cache.h> to define __read_mostly.

Convert cache.h to use uapi/linux/kernel.h instead of linux/kernel.h to avoid recursive #includes.

Convert the ALIGN macro to __KERNEL_ALIGN.

printk_once only sets the bool variable tested
once so mark it __read_mostly.

Neaten the alignment so it matches the rest of the pr_<level>_once #defines too.

---
V2: Add cache.h which was nearly always used
indirectly via #include <some_other_file.h>,
generally module.h
Update cache.h to avoid recursive #include

 include/linux/cache.h  |  4 ++--
 include/linux/printk.h | 19 ++++++++++---------
 2 files changed, 12 insertions(+), 11 deletions(-)
include/linux/cache.h +2 -2
index 4c57065..63a1d97 100644
@@ -1,11 +1,11 @@
 #ifndef __LINUX_CACHE_H
 #define __LINUX_CACHE_H
 
-#include <linux/kernel.h>
+#include <uapi/linux/kernel.h>
 #include <asm/cache.h>
 
 #ifndef L1_CACHE_ALIGN
-#define L1_CACHE_ALIGN(x) ALIGN(x, L1_CACHE_BYTES)
+#define L1_CACHE_ALIGN(x) __KERNEL_ALIGN(x, L1_CACHE_BYTES)
 #endif
 
 #ifndef SMP_CACHE_BYTES
include/linux/printk.h +10 -9
index e6131a78..2449da9 100644
@@ -5,6 +5,7 @@
 #include <linux/init.h>
 #include <linux/kern_levels.h>
 #include <linux/linkage.h>
+#include <linux/cache.h>
 
 extern const char linux_banner[];
 extern const char linux_proc_banner[];
@@ -251,17 +252,17 @@ extern asmlinkage void dump_stack(void) __cold;
  */
 
 #ifdef CONFIG_PRINTK
-#define printk_once(fmt, ...)			\
-({						\
-	static bool __print_once;		\
-						\
-	if (!__print_once) {			\
-		__print_once = true;		\
-		printk(fmt, ##__VA_ARGS__);	\
-	}					\
+#define printk_once(fmt, ...)					\
+({								\
+	static bool __print_once __read_mostly;			\
+								\
+	if (!__print_once) {					\
+		__print_once = true;				\
+		printk(fmt, ##__VA_ARGS__);			\
+	}							\
 })
 #else
-#define printk_once(fmt, ...)			\
+#define printk_once(fmt, ...)					\
 	no_printk(fmt, ##__VA_ARGS__)
 #endif
 
author Reviewed-by on v2 1/1
#8 lore raw

Re: [PATCH V2] printk/cache: Mark printk_once test variable __read_mostly

Quoted 29 lines · Joe Perches

Add #include <linux/cache.h> to define __read_mostly.

Convert cache.h to use uapi/linux/kernel.h instead of linux/kernel.h to avoid recursive #includes.

Convert the ALIGN macro to __KERNEL_ALIGN.

printk_once only sets the bool variable tested
once so mark it __read_mostly.

Neaten the alignment so it matches the rest of the pr_<level>_once #defines too.

---
V2: Add cache.h which was nearly always used
indirectly via #include <some_other_file.h>,
generally module.h
Update cache.h to avoid recursive #include

 include/linux/cache.h  |  4 ++--
 include/linux/printk.h | 19 ++++++++++---------
 2 files changed, 12 insertions(+), 11 deletions(-)
include/linux/cache.h +0 -0
index 4c57065..63a1d97 100644

include/linux/cache.h line 8

Quoted 10 lines
include/linux/cache.h +2 -2
@@ -1,11 +1,11 @@
 #ifndef __LINUX_CACHE_H
 #define __LINUX_CACHE_H
 
-#include <linux/kernel.h>
+#include <uapi/linux/kernel.h>
 #include <asm/cache.h>
 
 #ifndef L1_CACHE_ALIGN
-#define L1_CACHE_ALIGN(x) ALIGN(x, L1_CACHE_BYTES)
+#define L1_CACHE_ALIGN(x) __KERNEL_ALIGN(x, L1_CACHE_BYTES)

I think that should be __ALIGN_KERNEL (same applies to commit message).

Otherwise this patch looks good to me and builds all Meta defconfigs fine.

Thanks
James

#10 lore raw

PATCH v3 1/1 printk/cache: Mark printk_once test variable __read_mostly

In reply to Andrew Morton

Add #include <linux/cache.h> to define __read_mostly.

Convert cache.h to use uapi/linux/kernel.h instead of linux/kernel.h to avoid recursive #includes.

Convert the ALIGN macro to __ALIGN_KERNEL.

printk_once only sets the bool variable tested once so mark it __read_mostly.

Neaten the alignment so it matches the rest of the pr_<level>_once #defines too.

---
V3: Fix cache.h conversion and actually
compiled/built defconfig x86.
V2: Add cache.h which was nearly always used
indirectly via #include <some_other_file.h>, generally module.h
Update cache.h to avoid recursive #include

 include/linux/cache.h  |  4 ++--
 include/linux/printk.h | 19 ++++++++++---------
 2 files changed, 12 insertions(+), 11 deletions(-)
include/linux/cache.h +2 -2
index 4c57065..17e7e82 100644
@@ -1,11 +1,11 @@
 #ifndef __LINUX_CACHE_H
 #define __LINUX_CACHE_H
 
-#include <linux/kernel.h>
+#include <uapi/linux/kernel.h>
 #include <asm/cache.h>
 
 #ifndef L1_CACHE_ALIGN
-#define L1_CACHE_ALIGN(x) ALIGN(x, L1_CACHE_BYTES)
+#define L1_CACHE_ALIGN(x) __ALIGN_KERNEL(x, L1_CACHE_BYTES)
 #endif
 
 #ifndef SMP_CACHE_BYTES
include/linux/printk.h +10 -9
index e6131a78..2449da9 100644
@@ -5,6 +5,7 @@
 #include <linux/init.h>
 #include <linux/kern_levels.h>
 #include <linux/linkage.h>
+#include <linux/cache.h>
 
 extern const char linux_banner[];
 extern const char linux_proc_banner[];
@@ -251,17 +252,17 @@ extern asmlinkage void dump_stack(void) __cold;
  */
 
 #ifdef CONFIG_PRINTK
-#define printk_once(fmt, ...)			\
-({						\
-	static bool __print_once;		\
-						\
-	if (!__print_once) {			\
-		__print_once = true;		\
-		printk(fmt, ##__VA_ARGS__);	\
-	}					\
+#define printk_once(fmt, ...)					\
+({								\
+	static bool __print_once __read_mostly;			\
+								\
+	if (!__print_once) {					\
+		__print_once = true;				\
+		printk(fmt, ##__VA_ARGS__);			\
+	}							\
 })
 #else
-#define printk_once(fmt, ...)			\
+#define printk_once(fmt, ...)					\
 	no_printk(fmt, ##__VA_ARGS__)
 #endif
 
#11 lore raw

Re: [PATCH V3] printk/cache: Mark printk_once test variable __read_mostly

Trying this again with HTML disabled in gmail so mailing lists will see it ...

include/linux/cache.h line 5

Quoted 11 lines · Joe Perches
include/linux/cache.h +1 -1
index 4c57065..17e7e82 100644
@@ -1,11 +1,11 @@
 #ifndef __LINUX_CACHE_H
 #define __LINUX_CACHE_H
 
-#include <linux/kernel.h>
+#include <uapi/linux/kernel.h>
 #include <asm/cache.h>

This bit seems to be responsible for breaking the ia64 build in today's linux-next (tag: next-20131112)

Compiler barfs at:
CC arch/ia64/kernel/patch.o
In file included from arch/ia64/kernel/patch.c:12:
/home/aegl/zx1-smp/arch/ia64/include/asm/processor.h: In function ‘ia64_get_irr’:
/home/aegl/zx1-smp/arch/ia64/include/asm/processor.h:565: error:
implicit declaration of function ‘test_bit’
In file included from include/linux/bitops.h:33,

                 from include/linux/kernel.h:10,
                 from include/asm-generic/bug.h:13,
                 from /home/aegl/zx1-smp/arch/ia64/include/asm/bug.h:12,
                 from include/linux/bug.h:4,
                 from include/linux/thread_info.h:11,
                 from include/asm-generic/preempt.h:4,
                 from arch/ia64/include/generated/asm/preempt.h:1,
                 from include/linux/preempt.h:18,
                 from include/linux/uaccess.h:4,
                 from /home/aegl/zx1-smp/arch/ia64/include/asm/sections.h:10,
                 from arch/ia64/kernel/patch.c:13:

/home/aegl/zx1-smp/arch/ia64/include/asm/bitops.h: At top level:
/home/aegl/zx1-smp/arch/ia64/include/asm/bitops.h:339: error: static declaration of ‘test_bit’ follows non-static declaration /home/aegl/zx1-smp/arch/ia64/include/asm/processor.h:565: error:
previous implicit declaration of ‘test_bit’ was here make[1]: *** [arch/ia64/kernel/patch.o] Error 1
make: *** [arch/ia64/kernel/patch.o] Error 2

Other files choke in the similar way because they no longer implicitly get <linux/kernel.h>
(and all the other files that pulls in). Just backing out that one line of diff fixes it for me.
But presumably is a problem someplace else given the "avoid recursive #includes" bit of the commit message.

-Tony

#13 lore raw

Re: [PATCH V3] printk/cache: Mark printk_once test variable __read_mostly

Quoted 5 lines · Joe Perches

try this:

https://lkml.org/lkml/2013/10/30/604

How is that any different from what is in linux-next? It still has the same change to cache.h that reduces the nested #includes that currently make the build work.

-#include <linux/kernel.h>
+#include <uapi/linux/kernel.h>

-Tony

#14 lore raw

Re: [PATCH V3] printk/cache: Mark printk_once test variable __read_mostly

Quoted 11 lines · Tony Luck
Quoted 5 lines · Joe Perches

try this:

https://lkml.org/lkml/2013/10/30/604

How is that any different from what is in linux-next? It still has the same change to cache.h that reduces the nested #includes that currently make the build work.

Review

Quoted 4 lines
(quoted diff) +1 -2
-#include <linux/kernel.h>
+#include <uapi/linux/kernel.h>
 
-Tony

I believe you were cc'd on this email as
a reply to a kbuild test robot report:

I don't have an ia64 cross compiler here but
my presumption is this fixes the problem.

I believe Andrew has picked it up for his mm
series which is haphazardly merged into -next.

-----------

Quoted 6 lines · kbuild test robot
tree:   git://git.cmpxchg.org/linux-mmotm.git master
head:   7610384ab26340452f06d673c46624927c3901b6
commit: 7231446cc53d71e126acf3d0230239a6b3ad52bf [169/450] printk/cache: Mark printk_once test variable __read_mostly
config: make ARCH=ia64 alldefconfig

All error/warnings:

Hello Fengguang.

Quoted 3 lines

In file included from arch/ia64/kernel/patch.c:12:0:
arch/ia64/include/asm/processor.h: In function 'ia64_get_irr':

Quoted 1 line
Quoted 1 line

arch/ia64/include/asm/processor.h:565:2: error: implicit declaration of function 'test_bit' [-Werror=implicit-function-declaration]

I'm a bit confused how the change I made could cause this error.

Is this an existing warning that's now an error because bitops.h isn't explicitly #included in processor.h?

(possible fix below)

Quoted 43 lines
   In file included from include/linux/bitops.h:22:0,
                    from include/linux/kernel.h:10,
                    from include/asm-generic/bug.h:13,
                    from arch/ia64/include/asm/bug.h:12,
                    from include/linux/bug.h:4,
                    from include/linux/thread_info.h:11,
                    from include/linux/preempt.h:9,
                    from include/linux/uaccess.h:4,
                    from arch/ia64/include/asm/sections.h:10,
                    from arch/ia64/kernel/patch.c:13:
   arch/ia64/include/asm/bitops.h: At top level:
Quoted 1 line
Quoted 1 line

arch/ia64/include/asm/bitops.h:339:1: error: static declaration of 'test_bit' follows non-static declaration

arch/ia64/include/asm/processor.h:565:9: note: previous implicit declaration of 'test_bit' was here
cc1: some warnings being treated as errors

vim +/test_bit +565 arch/ia64/include/asm/processor.h

   549        #define cpu_relax()     ia64_hint(ia64_hint_pause)
   550        
   551        static inline int
   552        ia64_get_irr(unsigned int vector)
   553        {
   554                unsigned int reg = vector / 64;
   555                unsigned int bit = vector % 64;
   556                u64 irr;
   557        
   558                switch (reg) {
   559                case 0: irr = ia64_getreg(_IA64_REG_CR_IRR0); break;
   560                case 1: irr = ia64_getreg(_IA64_REG_CR_IRR1); break;
   561                case 2: irr = ia64_getreg(_IA64_REG_CR_IRR2); break;
   562                case 3: irr = ia64_getreg(_IA64_REG_CR_IRR3); break;
   563                }
   564        
Quoted 1 line

565 return test_bit(bit, &irr);

566 }
567

   568        static inline void
   569        ia64_set_lrr0 (unsigned long val)
   570        {
   571                ia64_setreg(_IA64_REG_CR_LRR0, val);
   572                ia64_srlz_d();
   573        }
Quoted 3 lines

---
0-DAY kernel build testing backend Open Source Technology Center
http://lists.01.org/mailman/listinfo/kbuild Intel Corporation

Maybe this fixes it?

---
 arch/ia64/include/asm/processor.h | 1 +
 1 file changed, 1 insertion(+)
arch/ia64/include/asm/processor.h +1 -0
index 5a84b3a..efd1b92 100644
@@ -71,6 +71,7 @@
 #include <linux/compiler.h>
 #include <linux/threads.h>
 #include <linux/types.h>
+#include <linux/bitops.h>
 
 #include <asm/fpu.h>
 #include <asm/page.h>

--
To unsubscribe from this list: send the line "unsubscribe linux-metag" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html