Script to find used sources in the kernel
I've been promising for some time to provide a way to identify which changes on Linux stable branches are relevant to CIP members, and conversely which are irrelevant and don't need to be reviewed.
This is a proof-of-concept of the major part of that, which is to identify which source files are used based on the submitted configurations. Once this is working to everyone's satisfaction, it should be fairly easy to automate the remaining parts, which are (1) merge the source lists for each branch (4.4, 4.19) and (2) check a set of patches or commits against a list.
I implemented:
* A script to select the appropriate value of CROSS_COMPILE for a given configuration (cross-compile-prefix).
* A script to call the assembler or compiler and collect dependency information from the preprocessor into a single file (get_used_sources_wrapper).
* A script that builds a kernel from a given configuration file with get_used_sources_wrapper?wrapped around the asssembler and compilers. This also filters out references to missing files (like firmware) from the configuration file.
I tied these together with a Makefile rule that generates a source list for each of the submitted configurations. I was then able to generate source lists for each of them (although it took quite a few hours).
You can find the scripts and lists on the "benhutchings/get-used- sources" branch of
https://gitlab.com/cip-project/cip-kernel/cip-kernel-config.git
There are a couple of open questions, on which I would like to hear other's opinions:
* Should the source lists be added to the repository or not? If they are added, then they should not be changed by the standard "all" and "clean" targets.
I tend to think that they should be added, because they take a long time to generate and require cross-compilers etc. to be installed.
* If the answer to the above is "yes", should the source lists be generated (a) by members, and sent along with the config files, or
(b) by the kernel maintainers? Option (a) makes a bit more work for members, but option (b) might result in missing build tools due to the config filtering mentioned above.
Please feel free to comment on anything else.
Ben.
Signature · 3 lines
Ben Hutchings, Software Developer ? Codethink Ltd
https://www.codethink.co.uk/ Dale House, 35 Dale Street
Manchester, M1 2HF, United KingdomHi!
Quoted 8 lines
There are a couple of open questions, on which I would like to hear other's opinions:
* Should the source lists be added to the repository or not? If they are added, then they should not be changed by the standard "all" and "clean" targets.
I tend to think that they should be added, because they take a long time to generate and require cross-compilers etc. to be installed.
I believe they should go to the repository, because we may want to manually adjust the lists.
I don't believe "example" configurations we have necessarily have enabled all the options "final products" may need. If some common option (vfat?) is not enabled in our configurations, because it is not hardware specific and not needed for testing, we may still want to review it, because it is common enough that someone will need it...
Best regards,
Pavel
Signature · 9 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: <http://lists.cip-project.org/pipermail/cip-dev/attachments/20190622/b2c00791/attachment.sig>
Quoted 5 lines · Pavel Machek
Hi!
Quoted 3 lines
There are a couple of open questions, on which I would like to hear other's opinions:
Quoted 8 lines
Quoted 5 lines
* Should the source lists be added to the repository or not? If they are added, then they should not be changed by the standard "all" and "clean" targets.
I tend to think that they should be added, because they take a long time to generate and require cross-compilers etc. to be installed.I believe they should go to the repository, because we may want to manually adjust the lists.
I've implemented that option.
Quoted 5 lines
I don't believe "example" configurations we have necessarily have enabled all the options "final products" may need. If some common option (vfat?) is not enabled in our configurations, because it is not hardware specific and not needed for testing, we may still want to review it, because it is common enough that someone will need it...
I hadn't thought of that, but it does seem possible. However, I think that rather than editing a generated list it would be better to have a separate manually maintained list that is merged into the generated list.
Ben.
Signature · 3 lines
Ben Hutchings, Software Developer Codethink Ltd
https://www.codethink.co.uk/ Dale House, 35 Dale Street
Manchester, M1 2HF, United KingdomHi!
(Ressurecting old discussion).
Quoted 12 lines · Ben Hutchings
Quoted 11 lines · Pavel Machek
Hi!
Quoted 8 lines
There are a couple of open questions, on which I would like to hear other's opinions:
* Should the source lists be added to the repository or not? If they are added, then they should not be changed by the standard "all" and "clean" targets.
I tend to think that they should be added, because they take a long time to generate and require cross-compilers etc. to be installed.
Quoted 4 lines
Quoted 2 lines
I believe they should go to the repository, because we may want to manually adjust the lists.
I've implemented that option.
Are the lists stored somewhere? I see the scripts in "cip-kernel-config" but I guess it needs non-trivial setup. I'd eventually like filtered version in lts-commit-list repository.
Quoted 10 lines
Quoted 5 lines
I don't believe "example" configurations we have necessarily have enabled all the options "final products" may need. If some common option (vfat?) is not enabled in our configurations, because it is not hardware specific and not needed for testing, we may still want to review it, because it is common enough that someone will need it...
I hadn't thought of that, but it does seem possible. However, I think that rather than editing a generated list it would be better to have a separate manually maintained list that is merged into the generated list.
I guess first step would be to select few subsystems with large ammounts of -stable patches that we don't need (s390), get the approval and then create automated tools to mark patches as "ignored" when they are not interesting.
Best regards,
Pavel
Signature · 2 lines
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Quoted 17 lines · Pavel Machek
Hi!
(Ressurecting old discussion).
Quoted 12 lines · Ben Hutchings
Quoted 11 lines · Pavel Machek
Hi!
Quoted 8 lines
There are a couple of open questions, on which I would like to hear other's opinions:
* Should the source lists be added to the repository or not? If they are added, then they should not be changed by the standard "all" and "clean" targets.
I tend to think that they should be added, because they take a long time to generate and require cross-compilers etc. to be installed.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
I believe they should go to the repository, because we may want to manually adjust the lists.
I've implemented that option.
Are the lists stored somewhere?
Yes they are the *.sources files.
Quoted 3 lines
I see the scripts in "cip-kernel-config"
but I guess it needs non-trivial setup. I'd eventually like filtered version in lts-commit-list repository.
[...]
They require a kernel repository with the CIP branches, and suitable compilers installed.
Ben.
Pavel
Signature · 3 lines
Ben Hutchings, Software Developer Codethink Ltd
https://www.codethink.co.uk/ Dale House, 35 Dale Street
Manchester, M1 2HF, United Kingdom