Skip to contentSkip to conversation

genksyms: separating public headers from private header files

DISCUSSION Don Zickus 2 participants 3 messages Jul 16, 2014 – Jul 23, 2014

lore
Share & tools
lore

Don Zickus

author
#0 lore raw

Hi Jan, Michal,

I am not sure who maintains genksyms officially, so I am sending this question to the two of you as folks who seemed to have contributed to the tool. :-)

I noticed with genksyms that a symbol is opaquely defined in a public header file (on purpose) and then fully defined in a private header. This is normal practice. Further, symbol checksumming is done on EXPORT_SYMBOLs in a private c file that includes the private header files.

As a result, even though a struct symbol is intentionally opaquely defined in a public header file consumed by a third party module, the symbol checksumming still includes the full definition (because the private c file with the actual export symbol has the full definition). This has made it difficult to modify the private header file struct because it breaks the symbol checksumming.

For example, let's consider

block/blk-core.c:EXPORT_SYMBOL(blk_put_queue);

blk_put_queue will eventually depend on struct blkcq_gq.

Now publicly blkcg_gq is defined opaquely in

include/linux/blkdev.h

and privately in

block/block-cgroup.h

Now when we checksum blk_put_queue both include/linux/blkdev.h and block/block-cgroup.h are included in block/blk-core.c, so blkcg_gq is fully defined for checksumming.

Later if we modify blkcq_gq in block/block-cgroup.h the checksum changes, even though it can debated that block-cgroup.h is a private header file and it should not impact kabi for third party modules.

Have either of you run into this? Or is the argument that private files should not impact the checksum not as strong as I might think? Or is it a technical problem of how to separate the public includes from the private includes in the preprocessed file?

Thanks!

Cheers,
Don

#1 lore raw
Quoted 38 lines
Quoted 1 line
Quoted 1 line · On 16.07.14 at 17:19, <[email protected]> wrote:

Hi Jan, Michal,

I am not sure who maintains genksyms officially, so I am sending this question to the two of you as folks who seemed to have contributed to the tool. :-)

I noticed with genksyms that a symbol is opaquely defined in a public header file (on purpose) and then fully defined in a private header. This is normal practice. Further, symbol checksumming is done on EXPORT_SYMBOLs in a private c file that includes the private header files.

As a result, even though a struct symbol is intentionally opaquely defined in a public header file consumed by a third party module, the symbol checksumming still includes the full definition (because the private c file with the actual export symbol has the full definition). This has made it difficult to modify the private header file struct because it breaks the symbol checksumming.

For example, let's consider

block/blk-core.c:EXPORT_SYMBOL(blk_put_queue);

blk_put_queue will eventually depend on struct blkcq_gq.

Now publicly blkcg_gq is defined opaquely in

include/linux/blkdev.h

and privately in

block/block-cgroup.h

Now when we checksum blk_put_queue both include/linux/blkdev.h and block/block-cgroup.h are included in block/blk-core.c, so blkcg_gq is fully defined for checksumming.

Quoted 8 lines

Later if we modify blkcq_gq in block/block-cgroup.h the checksum changes, even though it can debated that block-cgroup.h is a private header file and it should not impact kabi for third party modules.

Have either of you run into this? Or is the argument that private files should not impact the checksum not as strong as I might think? Or is it a technical problem of how to separate the public includes from the private includes in the preprocessed file?

Yes, I think we've run into this (if not elsewhere then by seeing [and having to wave] false positive kABI changes). Besides being a technical problem of separating one kind of header from the other, I'm also unsure whether uniformly ignoring definitions in private headers would always be correct. Hence I think a possible solution to this ought to involve manual annotation of structures not to participate in CRC calculations.

Jan

author
#2 lore raw
Quoted 47 lines · Jan Beulich
Quoted 38 lines
Quoted 1 line
Quoted 1 line · On 16.07.14 at 17:19, <[email protected]> wrote:

Hi Jan, Michal,

I am not sure who maintains genksyms officially, so I am sending this question to the two of you as folks who seemed to have contributed to the tool. :-)

I noticed with genksyms that a symbol is opaquely defined in a public header file (on purpose) and then fully defined in a private header. This is normal practice. Further, symbol checksumming is done on EXPORT_SYMBOLs in a private c file that includes the private header files.

As a result, even though a struct symbol is intentionally opaquely defined in a public header file consumed by a third party module, the symbol checksumming still includes the full definition (because the private c file with the actual export symbol has the full definition). This has made it difficult to modify the private header file struct because it breaks the symbol checksumming.

For example, let's consider

block/blk-core.c:EXPORT_SYMBOL(blk_put_queue);

blk_put_queue will eventually depend on struct blkcq_gq.

Now publicly blkcg_gq is defined opaquely in

include/linux/blkdev.h

and privately in

block/block-cgroup.h

Now when we checksum blk_put_queue both include/linux/blkdev.h and block/block-cgroup.h are included in block/blk-core.c, so blkcg_gq is fully defined for checksumming.

Quoted 8 lines

Later if we modify blkcq_gq in block/block-cgroup.h the checksum changes, even though it can debated that block-cgroup.h is a private header file and it should not impact kabi for third party modules.

Have either of you run into this? Or is the argument that private files should not impact the checksum not as strong as I might think? Or is it a technical problem of how to separate the public includes from the private includes in the preprocessed file?

Quoted 7 lines

Yes, I think we've run into this (if not elsewhere then by seeing [and having to wave] false positive kABI changes). Besides being a technical problem of separating one kind of header from the other, I'm also unsure whether uniformly ignoring definitions in private headers would always be correct. Hence I think a possible solution to this ought to involve manual annotation of structures not to participate in CRC calculations.

Yeah, I wasn't sure how feasible this would be or how to logically prove the correctness of this approach.

I can how tagging each struct could help, just a lot of tagging has to be done and I know our developers may not be proactive in all the right cases.

Thanks for the feedback! I'll see if I can come up with a solution though we can't utilize it for a few years as our RHEL6/7 products kabi checksums are locked down. :-/

Cheers,
Don