genksyms: separating public headers from private header files
Hi Jan, Michal,
I am not sure who maintains genksyms officially, so I am sending this question to the two of you as folks who seemed to have contributed to the tool. :-)
I noticed with genksyms that a symbol is opaquely defined in a public header file (on purpose) and then fully defined in a private header. This is normal practice. Further, symbol checksumming is done on EXPORT_SYMBOLs in a private c file that includes the private header files.
As a result, even though a struct symbol is intentionally opaquely defined in a public header file consumed by a third party module, the symbol checksumming still includes the full definition (because the private c file with the actual export symbol has the full definition). This has made it difficult to modify the private header file struct because it breaks the symbol checksumming.
For example, let's consider
block/blk-core.c:EXPORT_SYMBOL(blk_put_queue);
blk_put_queue will eventually depend on struct blkcq_gq.
Now publicly blkcg_gq is defined opaquely in
include/linux/blkdev.h
and privately in
block/block-cgroup.h
Now when we checksum blk_put_queue both include/linux/blkdev.h and block/block-cgroup.h are included in block/blk-core.c, so blkcg_gq is fully defined for checksumming.
Later if we modify blkcq_gq in block/block-cgroup.h the checksum changes, even though it can debated that block-cgroup.h is a private header file and it should not impact kabi for third party modules.
Have either of you run into this? Or is the argument that private files should not impact the checksum not as strong as I might think? Or is it a technical problem of how to separate the public includes from the private includes in the preprocessed file?
Thanks!
Cheers,
Don
Quoted 38 lines
Quoted 1 line
Quoted 1 line · On 16.07.14 at 17:19, <[email protected]> wrote:
Hi Jan, Michal,
I am not sure who maintains genksyms officially, so I am sending this question to the two of you as folks who seemed to have contributed to the tool. :-)
I noticed with genksyms that a symbol is opaquely defined in a public header file (on purpose) and then fully defined in a private header. This is normal practice. Further, symbol checksumming is done on
EXPORT_SYMBOLsin a private c file that includes the private header files.As a result, even though a struct symbol is intentionally opaquely defined in a public header file consumed by a third party module, the symbol checksumming still includes the full definition (because the private c file with the actual export symbol has the full definition). This has made it difficult to modify the private header file struct because it breaks the symbol checksumming.
For example, let's consider
block/blk-core.c:EXPORT_SYMBOL(blk_put_queue);
blk_put_queuewill eventually depend on structblkcq_gq.Now publicly
blkcg_gqis defined opaquely in
include/linux/blkdev.hand privately in
block/block-cgroup.hNow when we checksum
blk_put_queuebothinclude/linux/blkdev.handblock/block-cgroup.hare included inblock/blk-core.c, soblkcg_gqis fully defined for checksumming.
Quoted 8 lines
Later if we modify
blkcq_gqinblock/block-cgroup.hthe checksum changes, even though it can debated thatblock-cgroup.his a private header file and it should not impact kabi for third party modules.Have either of you run into this? Or is the argument that private files should not impact the checksum not as strong as I might think? Or is it a technical problem of how to separate the public includes from the private includes in the preprocessed file?
Yes, I think we've run into this (if not elsewhere then by seeing [and having to wave] false positive kABI changes). Besides being a technical problem of separating one kind of header from the other, I'm also unsure whether uniformly ignoring definitions in private headers would always be correct. Hence I think a possible solution to this ought to involve manual annotation of structures not to participate in CRC calculations.
Jan
Quoted 47 lines · Jan Beulich
Quoted 38 lines
Quoted 1 line
Quoted 1 line · On 16.07.14 at 17:19, <[email protected]> wrote:
Hi Jan, Michal,
I am not sure who maintains genksyms officially, so I am sending this question to the two of you as folks who seemed to have contributed to the tool. :-)
I noticed with genksyms that a symbol is opaquely defined in a public header file (on purpose) and then fully defined in a private header. This is normal practice. Further, symbol checksumming is done on
EXPORT_SYMBOLsin a private c file that includes the private header files.As a result, even though a struct symbol is intentionally opaquely defined in a public header file consumed by a third party module, the symbol checksumming still includes the full definition (because the private c file with the actual export symbol has the full definition). This has made it difficult to modify the private header file struct because it breaks the symbol checksumming.
For example, let's consider
block/blk-core.c:EXPORT_SYMBOL(blk_put_queue);
blk_put_queuewill eventually depend on structblkcq_gq.Now publicly
blkcg_gqis defined opaquely in
include/linux/blkdev.hand privately in
block/block-cgroup.hNow when we checksum
blk_put_queuebothinclude/linux/blkdev.handblock/block-cgroup.hare included inblock/blk-core.c, soblkcg_gqis fully defined for checksumming.Quoted 8 lines
Later if we modify
blkcq_gqinblock/block-cgroup.hthe checksum changes, even though it can debated thatblock-cgroup.his a private header file and it should not impact kabi for third party modules.Have either of you run into this? Or is the argument that private files should not impact the checksum not as strong as I might think? Or is it a technical problem of how to separate the public includes from the private includes in the preprocessed file?
Quoted 7 lines
Yes, I think we've run into this (if not elsewhere then by seeing [and having to wave] false positive kABI changes). Besides being a technical problem of separating one kind of header from the other, I'm also unsure whether uniformly ignoring definitions in private headers would always be correct. Hence I think a possible solution to this ought to involve manual annotation of structures not to participate in CRC calculations.
Yeah, I wasn't sure how feasible this would be or how to logically prove the correctness of this approach.
I can how tagging each struct could help, just a lot of tagging has to be done and I know our developers may not be proactive in all the right cases.
Thanks for the feedback! I'll see if I can come up with a solution though we can't utilize it for a few years as our RHEL6/7 products kabi checksums are locked down. :-/
Cheers,
Don