"Fast Kernel Headers" Tree -v2

I'm pleased to announce -v2 of the "Fast Kernel Headers" tree, which is a comprehensive rework of the Linux kernel's header hierarchy & header dependencies, with the dual goals of:

- speeding up the kernel build (both absolute and incremental build times)

- decoupling subsystem type & API definitions from each other

The fast-headers tree consists of over 25 sub-trees internally, spanning over 2,300 commits, which can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/mingo/tip.git master

# HEAD: 391ce485ced0 headers/deps: Introduce the CONFIG_FAST_HEADERS=y config option

On Sat, Jan 8, 2022 at 5:26 PM Ingo Molnar <[email protected]> wrote:
I've started building randconfig kernels for arm64 and x86, and fixing up things that come up, a few things I have noticed out so far:

Quoted 3 lines · Arnd Bergmann

On Sat, Jan 8, 2022 at 5:26 PM Ingo Molnar <[email protected]> wrote:
I've started building randconfig kernels for arm64 and x86, and fixing up things that come up, a few things I have noticed out so far:

I have run into a couple more specific issues:

* net/smc/smc_ib.c:824:26: error: implicit declaration of function 'cache_line_size' [-Werror=implicit-function-declaration] cache_line_size is generally provided by linux/cache.h, which includes asm/cache.h.
This works on arm64, but not on x86, where asm/cache.h would have to include asm/cpufeature.h, and but it would be good to avoid that because of the implicit linux/percpu.h and linux/bitops.h inclusions. Also, if I add the include, I get this
build failure instead: include/linux/smp_types.h:88:33: error:
requested alignment '20'
is not a positive power of 2

* arm64 has a couple of issues around asm/memory.h, linux/mm_types.h and asm/page.h that can cause loops. I think my latest version has it figured out, but there is probably room for optimization.

* There is no general way to get the get_order() definition, other than including asm/page.h from .c files. On arm64, this shows up in a couple of files after the cleanup. Only xtensa and ia64 define their own version of get_order(), and I think we should just remove those and move the generic version to linux/getorder.h, where any file using it can pick it up. For randconfig builds, I had to add asm/page.h to net/xdp/xsk_queue.c, mm/memtest.c and drivers/target/iscsi/iscsi_target_nego.c, after I removed the indirect include from arch/arm64/include/asm/mmu.h in the previous step.

Quoted 7 lines · Arnd Bergmann

Quoted 3 lines · Arnd Bergmann

On Sat, Jan 8, 2022 at 5:26 PM Ingo Molnar <[email protected]> wrote:
I've started building randconfig kernels for arm64 and x86, and fixing up things that come up, a few things I have noticed out so far:

I have run into a couple more specific issues:

Quoted 11 lines

* net/smc/smc_ib.c:824:26: error: implicit declaration of function 'cache_line_size' [-Werror=implicit-function-declaration] cache_line_size is generally provided by linux/cache.h, which includes asm/cache.h.
This works on arm64, but not on x86, where asm/cache.h would have to include asm/cpufeature.h, and but it would be good to avoid that because of the implicit linux/percpu.h and linux/bitops.h inclusions. Also, if I add the include, I get this
build failure instead: include/linux/smp_types.h:88:33: error:
requested alignment '20'
is not a positive power of 2

Note that this particular one should be fixed in the WIP branch, which is at:

git://git.kernel.org/pub/scm/linux/kernel/git/mingo/tip.git sched/headers

Quoted 3 lines

* arm64 has a couple of issues around asm/memory.h, linux/mm_types.h and asm/page.h that can cause loops. I think my latest version has it figured out, but there is probably room for optimization.

Yeah, this is like the 5th attempt at finding a robust solution. :-/

Quoted 9 lines

* There is no general way to get the get_order() definition, other than including asm/page.h from .c files. On arm64, this shows up in a couple of files after the cleanup. Only xtensa and ia64 define their own version of get_order(), and I think we should just remove those and move the generic version to linux/getorder.h, where any file using it can pick it up. For randconfig builds, I had to add asm/page.h to net/xdp/xsk_queue.c, mm/memtest.c and drivers/target/iscsi/iscsi_target_nego.c, after I removed the indirect include from arch/arm64/include/asm/mmu.h in the previous step.

Would including <linux/mm_page_address.h> be sufficient? That already has an <asm/page.h> inclusion and is vaguely related.

I tried to avoid as many low level headers as possible from the main types headers - and the get_order() functionality also brings in bitops definitions, which I'm still hoping to be able to reduce from its current ~95% utilization in a distro kernel ...

We could add <linux/page_api.h> as well, as a standardized header. We already have page_types.h and et_order() is a page types API.

Quoted 4 lines

I tried to avoid as many low level headers as possible from the main types headers - and the get_order() functionality also brings in bitops definitions, which I'm still hoping to be able to reduce from its current ~95% utilization in a distro kernel ...

Agreed, I think reducing bitops.h and atomic.h usage is fairly important, I think these are even bigger on arm64 than on x86.

Quoted 2 lines

We could add <linux/page_api.h> as well, as a standardized header. We already have page_types.h and et_order() is a page types API.

More generally speaking, do you have a plan for how to document which header to include for getting a particular symbol that is provided by a header we don't want to include directly? I think iwyu has a particular notation for it, but when I looked at using that in 2020 I decided it wouldn't scale to the size of the kernel. I did my own shell script with a long list of regex patterns, but I'm not convinced about that approach either.

Quoted 7 lines · Arnd Bergmann

Quoted 4 lines

I tried to avoid as many low level headers as possible from the main types headers - and the get_order() functionality also brings in bitops definitions, which I'm still hoping to be able to reduce from its current ~95% utilization in a distro kernel ...

Agreed, I think reducing bitops.h and atomic.h usage is fairly important, I think these are even bigger on arm64 than on x86.

So what I'm using for 'header complexity metrics' is rather simple: passing -P -H to the preprocessor: stripping comments & not generating line-markers, and then counting linecount.

Line-markers should *probably* remain, because the real build is generatinginclude/linux/mm_page_address.h them too - but I wanted to gain a crude & easily available metric to
measure 'first-pass parsing complexity'. That's I think where most of the
header bloat is concentrated: later passes don't really get any of the
unused header definitions passed along. (But maybe this is an invalid
assumption, because compiler warnings do get generated by later passes, and
they are generated for mostly-unused header inlines too.)

If we include comments & line-markers then the bloat goes up by another ~2x:

kepler:~/mingo.tip.git> ./st include/linux/sched.h
#include <linux/sched.h> | LOC: 2,186 | headers: 118
kepler:~/mingo.tip.git> ./st include/linux/sched.h
#include <linux/sched.h> | LOC: 4,092 | headers: 0

Quoted 7 lines

If we include comments & line-markers then the bloat goes up by another ~2x:

kepler:~/mingo.tip.git> ./st include/linux/sched.h
#include <linux/sched.h> | LOC: 2,186 | headers: 118
kepler:~/mingo.tip.git> ./st include/linux/sched.h
#include <linux/sched.h> | LOC: 4,092 | headers: 0

The metric I've been focusing on is bytes of the preprocessed header, which is more sensitive to function definitions that get generated from macros, and I multiply this by the number of inclusions (from scanning the .file.o.cmd files). It probably helps to have a couple of metrics and look at all of them occasionally to not miss something important.

In the meantime, I have made some progress on reducing the headers for arm64, on top of your tree from Jan 8, but I have not looked at later changes from your side, and I need to work on this a bit more to ensure this doesn't break other architectures.

For an arm64 allmodconfig build, my additional improvements on top of yours are significant but not as good as I had hoped for, this can still improve I hope:

5.16-rc8-vanilla 32640 seconds user, 3286 seconds sys 5.16-rc8-mingo 22990 seconds user, 2304 seconds sys
5.16-rc8-arnd 19007 seconds user, 1853 seconds sys

As my tree builds any randconfig cleanly, [...]

I keep looking at different configs and find that this has a big impact, some options end up eliminating most of the benefits until I add further changes to clean up certain files. This happened with kasan, kprobes, and lse-atomics for instance. After eliminating all circular includes, I was also able to revisit my old script to visualize the inclusions, see[1] for the current arm64 defconfig output. This version uses my arbitrary metric as font-size, and uses labels for the number of inclusions.

Quoted 13 lines · Ingo Molnar

I'm pleased to announce -v2 of the "Fast Kernel Headers" tree, which is a comprehensive rework of the Linux kernel's header hierarchy & header dependencies, with the dual goals of:

- speeding up the kernel build (both absolute and incremental build times)

- decoupling subsystem type & API definitions from each other

The fast-headers tree consists of over 25 sub-trees internally, spanning over 2,300 commits, which can be found at:

Quoted 3 lines

git://git.kernel.org/pub/scm/linux/kernel/git/mingo/tip.git master

# HEAD: 391ce485ced0 headers/deps: Introduce the CONFIG_FAST_HEADERS=y config option

I've started reading through it at last. I can't say that I'm reviewing every patch, [...]

but at least (almost all) the things I've looked at so far all seem really nice to me, mostly this is the same that I was planning to do as

well, some things I would have done differently but I'm not complaining as you did the work, and some things seem unnecessary but might not be.

I've started building randconfig kernels for arm64 and x86, and fixing up things that come up,
a few things I have noticed out so far:

* 2e98ec93d465 ("headers/prep: Rename constants: SOCK_DESTROY => SOCK_DIAG_SOCK_DESTROY")

This one looks wrong, as you are changing a uapi header, possibly breaking applications at compile time. I think the other one should be renamed instead.

* 04293522a8cb ("headers/deps: ipc/shm: Move the 'struct shmid_ds' definition to ipc/shm.c")
and related patches

Similarly, the IPC structures are uapi headers that I would not
change here for the same reasons.
Even if nothing uses those any more with modern libc
implementations, the structures belong into
uapi, unless we can prove that the old-style sysvipc interface is
completely unused and we
remove the implementation from the kernel as well (I don't think we
want that, but I have not
looked in depth at when it was last used by a libc)

* changing any include/uapi headers to use "#include <uapi/linux/*.h>" is broken because
that makes the headers unusable from userspace, including any of
tools/*/. I think we
can work around this in the headers_install.sh postprocessing step
though, where we already
do unifdef etc.

* For all the header additions to .c files, I assume you are using a set of script, so these could
probably be changed without much trouble. I would suggest applying
them in sequence so
the headers remain sorted alphabetically in the end. It would
probably make sense to
squash those all together to avoid patching certain files many
times over, for the sake
of keeping a slightly saner git history.

* The per-task stuff sounded a bit scary from your descriptions but looking at the actual
implementation I now get it, this looks like a really nice way of doing it.

* I think it would be good to keep the include/linux/syscalls_api.h declarations in the same header as the SYSCALL_DEFINE*() macros, to ensure that the prototypes remain synchronized. Splitting them out will likely also
cause sparse
warnings for missing prototypes (or maybe it should but doesn't at
the moment).

* include/linux/time64_types.h is not a good name, as these are now the default types
after we removed the time32 versions. I'd either rename it to
linux/time_types.h
or split it up between linux/types.h and linux/ktime_types.h

* arm64 needs a couple of minor fixups, see
https://pastebin.com/eSKhz4CL for what
I have so far, feel free to integrate any things that directly make sense.

Quoted 8 lines · Arnd Bergmann

On Sat, Jan 8, 2022 at 5:26 PM Ingo Molnar <[email protected]> wrote:
well, some things I would have done differently but I'm not complaining as you did the work, and some things seem unnecessary but might not be.

I've started building randconfig kernels for arm64 and x86, and fixing up things that come up,
a few things I have noticed out so far:

Quoted 6 lines

* 2e98ec93d465 ("headers/prep: Rename constants: SOCK_DESTROY => SOCK_DIAG_SOCK_DESTROY")

This one looks wrong, as you are changing a uapi header, possibly breaking applications at compile time. I think the other one should be renamed instead.

This is hard: SOCK_DESTROY is one of the main constants for sockets, is well named, fits into an existing in-kernel nomenclature and both me and networking folks would hate to rename it ...

So I'd keep this one and wait for any reported breakage. I don't think we *guarantee* the specific naming of symbols - we guarantee an ABI, and make a best-effort for the rest. The constant is netdiag specific and doesn't seem to be included by any major user-space header in /usr/include.

Quoted 4 lines

* 04293522a8cb ("headers/deps: ipc/shm: Move the 'struct shmid_ds' definition to ipc/shm.c")
and related patches

Quoted 9 lines

Similarly, the IPC structures are uapi headers that I would not
change here for the same reasons.
Even if nothing uses those any more with modern libc
implementations, the structures belong into
uapi, unless we can prove that the old-style sysvipc interface is
completely unused and we
remove the implementation from the kernel as well (I don't think we
want that, but I have not
looked in depth at when it was last used by a libc)

Ok, we can certainly undo this one - but how does it work in practice, as the structure is already defined by libc:

/usr/include/x86_64-linux-gnu/bits/types/struct_shmid_ds.h:struct shmid_ds

/* Data structure describing a shared memory segment. */ struct shmid_ds
{
struct ipc_perm shm_perm; /* operation permission struct */
size_t shm_segsz; /* size of segment in bytes */
#if __TIMESIZE == 32
__time_t shm_atime; /* time of last shmat() */
unsigned long int __shm_atime_high;
__time_t shm_dtime; /* time of last shmdt() */
unsigned long int __shm_dtime_high;
__time_t shm_ctime; /* time of last change by shmctl() */
unsigned long int __shm_ctime_high;
#else

    __time_t shm_atime;                 /* time of last shmat() */
    __time_t shm_dtime;                 /* time of last shmdt() */
    __time_t shm_ctime;                 /* time of last change by shmctl() */
#endif
    __pid_t shm_cpid;                   /* pid of creator */
    __pid_t shm_lpid;                   /* pid of last shmop */
    shmatt_t shm_nattch;                /* number of current attaches */
    __syscall_ulong_t __glibc_reserved5;
    __syscall_ulong_t __glibc_reserved6;

};

Wouldn't this definition conflict with any header use of linux/shm.h?

Quoted 7 lines

* changing any include/uapi headers to use "#include <uapi/linux/*.h>" is broken because
that makes the headers unusable from userspace, including any of
tools/*/. I think we
can work around this in the headers_install.sh postprocessing step
though, where we already
do unifdef etc.

Yeah, so the problem here is on the kernel side, the following innocent looking include in a UAPI header:

   #include <linux/foo.h>

Will turn into a very large header - and unintentionally so - if there happens to be an include/foo.h header.

I.e. normally there's only the UAPI header, which is then included, but in some significant cases that's not so.

IMO it seems so much cleaner to express the intent to only include the UAPI header - so solving this at header-install time would be preferable.

Quoted 9 lines

* For all the header additions to .c files, I assume you are using a set of script, so these could
probably be changed without much trouble. I would suggest applying
them in sequence so
the headers remain sorted alphabetically in the end. It would
probably make sense to
squash those all together to avoid patching certain files many
times over, for the sake
of keeping a slightly saner git history.

Are you suggesting to change the current reverse-alphabetical order of headers:

drivers/auxdisplay/ht16k33.c +9 -0

@@ -8,6 +8,15 @@
  * Copyright (C) 2021 Glider bv
  */
 
+#include <linux/workqueue_api.h>
+#include <linux/wait_api.h>
+#include <linux/sched.h>
+#include <linux/pgtable_api.h>
+#include <linux/of_api.h>
+#include <linux/mm_api.h>
+#include <linux/jiffies.h>
+#include <linux/gfp_api.h>
+#include <linux/device_api_lock.h>
 

... to alphabetical?

Can certainly do that - but this will flip around the commit order: it's much easier to add at the head of the include files section.

Quoted 6 lines

* I think it would be good to keep the include/linux/syscalls_api.h declarations in the same header as the SYSCALL_DEFINE*() macros, to ensure that the prototypes remain synchronized. Splitting them out will likely also
cause sparse
warnings for missing prototypes (or maybe it should but doesn't at
the moment).

Yeah, I suppose we could undo the split:

# -fast-headers-v2:

_______________________
| stripped lines of code
| _____________________________
| | headers included recursively
| | _______________________________
| | | usage in a distro kernel build

 ____________                                                 |             |               |         _________________________________________
| header name                                                 |             |               |        | million lines of comment-stripped C code
|                                                             |             |               |        |
  #include <linux/syscalls_types.h>                           | LOC:  2,397 | headers:  128 |    353 | MLOC:    0.8 |
  #include <linux/syscalls_api.h>                             | LOC: 12,842 | headers:  361 |    167 | MLOC:    2.1 |

The full header used to be a lot bigger:

  # v5.16-rc8

  #include <linux/syscalls.h>                                 | LOC: 40,217 | headers:  604 |    321 | MLOC:   12.9 | ###

Quoted 3 lines

* arm64 needs a couple of minor fixups, see
https://pastebin.com/eSKhz4CL for what
I have so far, feel free to integrate any things that directly make sense.

Thanks! Mind sending the dependency removals in a series, so I can keep attribution?

These bits are usually in separate commits, unless they fix some bug in an existing commit:

 arch/arm64/include/asm/mmu.h                     |    1 -
 arch/arm64/include/asm/pgtable-hwdef.h           |    1 -
 arch/arm64/include/asm/pgtable-prot.h            |    1 -

This bit looks suboptimal - but as you mentioned it fixes tooling build errors:

include/uapi/linux/netdevice.h +1 -0

@@ -26,11 +26,17 @@
 #ifndef _UAPI_LINUX_NETDEVICE_H
 #define _UAPI_LINUX_NETDEVICE_H
 
+#ifdef __KERNEL__
 #include <uapi/linux/if.h>

This KASAN bit:

include/linux/mm_api_kasan.h +1 -1

@@ -18,7 +18,7 @@ static inline u8 page_kasan_tag(const struct page *page)
 {
        u8 tag = 0xff;
 
-       if (kasan_enabled()) {
+       if (IS_ENABLED(CONFIG_KASAN)) {
                tag = (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK;
                tag ^= 0xff;
        }

Is this a fix for some build failure? Upstream it's using kasan_enabled():

This is -v3 of the "Fast Kernel Headers" tree, which is an ongoing rework of the Linux kernel's header hierarchy & header dependencies, with the dual goals of:

- speeding up the kernel build (both absolute and incremental build times)

- decoupling subsystem type & API definitions from each other

The fast-headers tree consists of over 25 sub-trees internally, spanning over 2,300 commits, which can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/mingo/tip.git master

There's various changes in -v3, and it's now ported to the latest kernel (v5.17-rc8).

Diffstat difference:

-v2: 25332 files changed, 178498 insertions(+), 74790 deletions(-) -v3: 25513 files changed, 180947 insertions(+), 74572 deletions(-)

Thanks,

Ingo

Quoted 5 lines · 15.3.2022 12.35 Ingo Molnar ([email protected]) wrote:

This is -v3 of the "Fast Kernel Headers" tree, which is an ongoing rework of the Linux kernel's header hierarchy & header dependencies, with the dual goals of:

Quoted 8 lines

- speeding up the kernel build (both absolute and incremental build times)

- decoupling subsystem type & API definitions from each other

The fast-headers tree consists of over 25 sub-trees internally, spanning over 2,300 commits, which can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/mingo/tip.git master

I have had problems to build master branch (defconfig) with gcc9

    gcc (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0

I did also test v2 and problems where there too. I have no problem with gcc10 or Clang11. Error I get is:

In file included from ./include/linux/rcuwait_api.h:7,

                 from ./include/linux/rcuwait.h:6,
                 from ./include/linux/irq_work.h:7,
                 from ./include/linux/perf_event_types.h:44,
                 from ./include/linux/perf_event_api.h:17,
                 from arch/x86/kernel/kprobes/opt.c:8:

./include/linux/rcuwait_api.h: In function ‘rcuwait_active’:
./include/linux/rcupdate.h:328:9: error: dereferencing pointer to incomplete type ‘struct task_struct’
328 | typeof(*p) *local = (typeof(*p) *__force)READ_ONCE(p); \
| ^
./include/linux/rcupdate.h:439:31: note: in expansion of macro ‘__rcu_access_pointer’
439 | #define rcu_access_pointer(p) __rcu_access_pointer((p),
__UNIQUE_ID(rcu), __rcu)

      |                               ^~~~~~~~~~~~~~~~~~~~

./include/linux/rcuwait_api.h:15:11: note: in expansion of macro ‘rcu_access_pointer’
15 | return !!rcu_access_pointer(w->task);

Argillander

Quoted 7 lines

There's various changes in -v3, and it's now ported to the latest kernel (v5.17-rc8).

Diffstat difference:

-v2: 25332 files changed, 178498 insertions(+), 74790 deletions(-) -v3: 25513 files changed, 180947 insertions(+), 74572 deletions(-)

Hi Kari,

Quoted 41 lines · Kari Argillander

Quoted 5 lines · 15.3.2022 12.35 Ingo Molnar ([email protected]) wrote:

This is -v3 of the "Fast Kernel Headers" tree, which is an ongoing rework of the Linux kernel's header hierarchy & header dependencies, with the dual goals of:

Quoted 8 lines

- speeding up the kernel build (both absolute and incremental build times)

- decoupling subsystem type & API definitions from each other

The fast-headers tree consists of over 25 sub-trees internally, spanning over 2,300 commits, which can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/mingo/tip.git master

I have had problems to build master branch (defconfig) with gcc9

    gcc (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0

I did also test v2 and problems where there too. I have no problem with gcc10 or Clang11. Error I get is:

In file included from ./include/linux/rcuwait_api.h:7,

                 from ./include/linux/rcuwait.h:6,
                 from ./include/linux/irq_work.h:7,
                 from ./include/linux/perf_event_types.h:44,
                 from ./include/linux/perf_event_api.h:17,
                 from arch/x86/kernel/kprobes/opt.c:8:

./include/linux/rcuwait_api.h: In function ‘rcuwait_active’:
./include/linux/rcupdate.h:328:9: error: dereferencing pointer to incomplete type ‘struct task_struct’
328 | typeof(*p) *local = (typeof(*p) *__force)READ_ONCE(p); \
| ^
./include/linux/rcupdate.h:439:31: note: in expansion of macro ‘__rcu_access_pointer’
439 | #define rcu_access_pointer(p) __rcu_access_pointer((p),
__UNIQUE_ID(rcu), __rcu)

      |                               ^~~~~~~~~~~~~~~~~~~~

./include/linux/rcuwait_api.h:15:11: note: in expansion of macro ‘rcu_access_pointer’
15 | return !!rcu_access_pointer(w->task);

Quoted 1 line

Argillander

You could try the patch here:
https://lore.kernel.org/all/[email protected]/

although the build error that it fixes doesn't look exactly the same as yours.

Quoted 7 lines

Quoted 7 lines

There's various changes in -v3, and it's now ported to the latest kernel (v5.17-rc8).

Diffstat difference:

-v2: 25332 files changed, 178498 insertions(+), 74790 deletions(-) -v3: 25513 files changed, 180947 insertions(+), 74572 deletions(-)

--
~Randy

Quoted 45 lines · 22.03.2022 17.37 Randy Dunlap ([email protected]) wrote:

Hi Kari,

Quoted 41 lines · Kari Argillander

Quoted 5 lines · 15.3.2022 12.35 Ingo Molnar ([email protected]) wrote:

This is -v3 of the "Fast Kernel Headers" tree, which is an ongoing rework of the Linux kernel's header hierarchy & header dependencies, with the dual goals of:

Quoted 8 lines

- speeding up the kernel build (both absolute and incremental build times)

- decoupling subsystem type & API definitions from each other

The fast-headers tree consists of over 25 sub-trees internally, spanning over 2,300 commits, which can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/mingo/tip.git master

I have had problems to build master branch (defconfig) with gcc9

    gcc (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0

I did also test v2 and problems where there too. I have no problem with gcc10 or Clang11. Error I get is:

In file included from ./include/linux/rcuwait_api.h:7,

                 from ./include/linux/rcuwait.h:6,
                 from ./include/linux/irq_work.h:7,
                 from ./include/linux/perf_event_types.h:44,
                 from ./include/linux/perf_event_api.h:17,
                 from arch/x86/kernel/kprobes/opt.c:8:

./include/linux/rcuwait_api.h: In function ‘rcuwait_active’:
./include/linux/rcupdate.h:328:9: error: dereferencing pointer to incomplete type ‘struct task_struct’
328 | typeof(*p) *local = (typeof(*p) *__force)READ_ONCE(p); \
| ^
./include/linux/rcupdate.h:439:31: note: in expansion of macro ‘__rcu_access_pointer’
439 | #define rcu_access_pointer(p) __rcu_access_pointer((p),
__UNIQUE_ID(rcu), __rcu)

      |                               ^~~~~~~~~~~~~~~~~~~~

./include/linux/rcuwait_api.h:15:11: note: in expansion of macro ‘rcu_access_pointer’
15 | return !!rcu_access_pointer(w->task);

Quoted 4 lines

Quoted 1 line

Argillander

You could try the patch here:
https://lore.kernel.org/all/[email protected]/

I have to edit it to <linux/cgroup_types.h> as there is no <linux/cgroup-defs.h> with fast headers. I also tried a couple other things but it didn't seem to make a
difference.

Quoted 2 lines

although the build error that it fixes doesn't look exactly the same as yours.

Quite close still. Maybe I should try to bisect this and I will also see how bisectable
this branch is.

Quoted 11 lines

Quoted 7 lines

Quoted 7 lines

There's various changes in -v3, and it's now ported to the latest kernel (v5.17-rc8).

Diffstat difference:

-v2: 25332 files changed, 178498 insertions(+), 74790 deletions(-) -v3: 25513 files changed, 180947 insertions(+), 74572 deletions(-)

--
~Randy

Quoted 57 lines · 22.03.2022 18.22 Kari Argillander ([email protected]) wrote:

Quoted 45 lines · 22.03.2022 17.37 Randy Dunlap ([email protected]) wrote:

Hi Kari,

Quoted 41 lines · Kari Argillander

Quoted 5 lines · 15.3.2022 12.35 Ingo Molnar ([email protected]) wrote:

This is -v3 of the "Fast Kernel Headers" tree, which is an ongoing rework of the Linux kernel's header hierarchy & header dependencies, with the dual goals of:

Quoted 8 lines

- speeding up the kernel build (both absolute and incremental build times)

- decoupling subsystem type & API definitions from each other

The fast-headers tree consists of over 25 sub-trees internally, spanning over 2,300 commits, which can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/mingo/tip.git master

I have had problems to build master branch (defconfig) with gcc9

     gcc (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0

I did also test v2 and problems where there too. I have no problem with gcc10 or Clang11. Error I get is:

In file included from ./include/linux/rcuwait_api.h:7,

                  from ./include/linux/rcuwait.h:6,
                  from ./include/linux/irq_work.h:7,
                  from ./include/linux/perf_event_types.h:44,
                  from ./include/linux/perf_event_api.h:17,
                  from arch/x86/kernel/kprobes/opt.c:8:

./include/linux/rcuwait_api.h: In function ‘rcuwait_active’:
./include/linux/rcupdate.h:328:9: error: dereferencing pointer to incomplete type ‘struct task_struct’
328 | typeof(*p) *local = (typeof(*p) *__force)READ_ONCE(p); \
| ^
./include/linux/rcupdate.h:439:31: note: in expansion of macro ‘__rcu_access_pointer’
439 | #define rcu_access_pointer(p) __rcu_access_pointer((p),
__UNIQUE_ID(rcu), __rcu)

       |                               ^~~~~~~~~~~~~~~~~~~~
./include/linux/rcuwait_api.h:15:11: note: in expansion of macro
‘rcu_access_pointer’
    15 |  return !!rcu_access_pointer(w->task);

Quoted 4 lines

Quoted 1 line

Argillander

You could try the patch here:
https://lore.kernel.org/all/[email protected]/

I have to edit it to <linux/cgroup_types.h> as there is no <linux/cgroup-defs.h> with fast headers. I also tried a couple other things but it didn't seem to make a
difference.

Quoted 6 lines

Quoted 2 lines

although the build error that it fixes doesn't look exactly the same as yours.

Quite close still. Maybe I should try to bisect this and I will also see how bisectable
this branch is.

Ok. I have now bisect first bad to this commit. c4ad6fcb67c4 ("sched/headers: Reorganize, clean up and optimize kernel/sched/fair.c dependencies")
Note that this has been also bisect by others.

With this I get little bit different error:

In file included from ./arch/x86/include/generated/asm/rwonce.h:1,

                  from ./include/linux/compiler.h:255,
                  from ./include/linux/export.h:43,
                  from ./include/linux/linkage.h:7,
                  from ./include/linux/kernel.h:17,
                  from ./include/linux/cpumask.h:10,
                  from ./include/linux/energy_model.h:4,
                  from kernel/sched/fair.c:23:

./include/linux/psi.h: In function ‘cgroup_move_task’:
./include/linux/rcupdate.h:414:36: error: dereferencing pointer to incomplete type ‘struct css_set’
414 | #define RCU_INITIALIZER(v) (typeof(*(v)) __force __rcu *)(v)
| ^~~~

Which is actually the same error that is in Randy's message. Patch of Randy works
on top of this commit. But I cannot get this patch to work with HEAD, but probably
I'm just missing something obvious. Still nice to see that probably a solution is near.

Quoted 11 lines

Quoted 11 lines

Quoted 7 lines

Quoted 7 lines

There's various changes in -v3, and it's now ported to the latest kernel (v5.17-rc8).

Diffstat difference:

-v2: 25332 files changed, 178498 insertions(+), 74790 deletions(-) -v3: 25513 files changed, 180947 insertions(+), 74572 deletions(-)

--
~Randy