xen: rework compat headers generation
v2 cover xen: rework compat headers generation
Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.build-system-xen-include-rework-v2
v2:
- new patch [1/4] to fix issue with command line that can be way too long
- other small changes, and reorder patches
Hi,
This patch series is about 2 improvement. First one is to use $(if_changed, ) in "include/Makefile" to make the generation of the compat headers less verbose and to have the command line part of the decision to rebuild the headers. Second one is to replace one slow script by a much faster one, and save time when generating the headers.
There's some number here:
https://lore.kernel.org/xen-devel/Yp3%2F%2Fc%[email protected]/
On my machine when doing a full build [of the hypervisor], with ccache enabled, it saves about 1.17 seconds (out of ~17s), and without ccache, it saves about 2.0 seconds (out of ~37s).
Thanks.
Anthony PERARD (4):
build,include: rework shell script for headers++.chk
build: remove auto.conf prerequisite from compat/xlat.h target
build: set PERL
build: replace get-fields.sh by a perl script
xen/Makefile | 1 + xen/include/Makefile | 25 +- README | 1 + xen/tools/compat-xlat-header.pl | 539 ++++++++++++++++++++++++++++++++ xen/tools/get-fields.sh | 528 ------------------------------- 5 files changed, 557 insertions(+), 537 deletions(-) create mode 100755 xen/tools/compat-xlat-header.pl delete mode 100644 xen/tools/get-fields.sh
Signature · 1 line
Anthony PERARD
PATCH v2 1/4 build,include: rework shell script for headers++.chk
The command line generated for headers++.chk by make is quite long, and in some environment it is too long. This issue have been seen in Yocto build environment.
Error messages:
make[9]: execvp: /bin/sh: Argument list too long make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
Rework so that we do the foreach loop in shell rather that make to reduce the command line size by a lot. We also need a way to get headers prerequisite for some public headers so we use a switch "case" in shell to be able to do some simple pattern matching. Variables alone in POSIX shell don't allow to work with associative array or variables with "/".
---
Notes:
v2:
- fix typo in commit message
- fix out-of-tree build
v1:
- was sent as a reply to v1 of the series
xen/include/Makefile | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-)
xen/include/Makefile +13 -4
@@ -158,13 +158,22 @@ define cmd_headerscxx_chk touch [email protected]; \ exit 0; \ fi; \ - $(foreach i, $(filter %.h,$^), \ - echo "#include "\"$(i)\" \ + get_prereq() { \ + case $$1 in \ + $(foreach i, $(filter %.h,$^), \ + $(if $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + $(i)$(close) \ + echo "$(foreach j, $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + -include c$(j))";;)) \ + esac; \ + }; \ + for i in $(filter %.h,$^); do \ + echo "#include "\"$$i\" \ | $(CXX) -x c++ -std=gnu++98 -Wall -Werror -D__XEN_TOOLS__ \ -include stdint.h -include $(srcdir)/public/xen.h \ - $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include c$(j)) \ + `get_prereq $$i` \ -S -o /dev/null - \ - || exit $$?; echo $(i) >> [email protected];) \ + || exit $$?; echo $$i >> [email protected]; done; \ mv [email protected] $@ endef
Signature · 1 line
Anthony PERARD
Re: [XEN PATCH v2 1/4] build,include: rework shell script for headers++.chk
Quoted 40 lines · Anthony PERARD
The command line generated for headers++.chk by make is quite long, and in some environment it is too long. This issue have been seen in Yocto build environment.
Error messages:
make[9]: execvp:/bin/sh: Argument list too long make[9]: *** [include/Makefile:181:include/headers++.chk] Error 127Rework so that we do the foreach loop in shell rather that make to reduce the command line size by a lot. We also need a way to get headers prerequisite for some public headers so we use a switch "case" in shell to be able to do some simple pattern matching. Variables alone in POSIX shell don't allow to work with associative array or variables with "/".
Reported-by Bertrand Marquis <[email protected]>Fixes 28e13c7f43 ("build: xen/include: use if_changed")Signed-off-by Anthony PERARD <[email protected]>Reviewed-by Bertrand Marquis <[email protected]>---
Notes:
v2:
- fix typo in commit message
- fix out-of-tree buildv1:
- was sent as a reply to v1 of the seriesxen/include/Makefile | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-)xen/include/Makefile +0
-0@@ -158,13 +158,22 @@ define cmd_headerscxx_chk touch [email protected]; \ exit 0; \ fi; \
xen/include/Makefile define cmd_headerscxx_chk · line 168
Quoted 10 lines
xen/include/Makefile +8
-2- $(foreach i, $(filter %.h,$^), \ - echo "#include "\"$(i)\" \ + get_prereq() { \ + case $$1 in \ + $(foreach i, $(filter %.h,$^), \ + $(if $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + $(i)$(close) \ + echo "$(foreach j, $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + -include c$(j))";;)) \ + esac; \
If I'm reading this right (indentation looks to be a little misleading and hence one needs to count parentheses) the case statement could (in theory) remain without any "body". As per the command language spec I'm looking at this (if it works in the first place) is an extension, and formally there's always at least one label required. Since we aim to be portable in such regards, I'd like to request that there be a final otherwise empty *) line.
xen/include/Makefile define cmd_headerscxx_chk · line 174
Quoted 7 lines
xen/include/Makefile +4
-1+ }; \ + for i in $(filter %.h,$^); do \ + echo "#include "\"$$i\" \ | $(CXX) -x c++ -std=gnu++98 -Wall -Werror -D__XEN_TOOLS__ \ -include stdint.h -include $(srcdir)/public/xen.h \ - $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include c$(j)) \ + `get_prereq $$i` \
While I know we use back-ticked quoting elsewhere, I'd generally recommend to use $() for readability. But maybe others view this exactly the other way around ...
And a question without good context to put at: Isn't headers99.chk in similar need of converting? It looks only slightly less involved than the C++ one.
Jan
Quoted 6 lines
xen/include/Makefile +1
-1-S -o /dev/null - \ - || exit $$?; echo $(i) >> [email protected];) \ + || exit $$?; echo $$i >> [email protected]; done; \ mv [email protected] $@ endef
Re: [XEN PATCH v2 1/4] build,include: rework shell script for headers++.chk
Quoted 20 lines · Jan Beulich
Quoted 8 lines · Anthony PERARD
xen/include/Makefile +0
-0@@ -158,13 +158,22 @@ define cmd_headerscxx_chk touch [email protected]; \ exit 0; \ fi; \xen/include/Makefile define cmd_headerscxx_chk · line 168
Quoted 10 lines
xen/include/Makefile +8
-2- $(foreach i, $(filter %.h,$^), \ - echo "#include "\"$(i)\" \ + get_prereq() { \ + case $$1 in \ + $(foreach i, $(filter %.h,$^), \ + $(if $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + $(i)$(close) \ + echo "$(foreach j, $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + -include c$(j))";;)) \ + esac; \
Quoted 7 lines
If I'm reading this right (indentation looks to be a little misleading and hence one needs to count parentheses) the case statement could (in theory) remain without any "body". As per the command language spec I'm looking at this (if it works in the first place) is an extension, and formally there's always at least one label required. Since we aim to be portable in such regards, I'd like to request that there be a final otherwise empty *) line.
When looking at the shell grammar at [1], an empty body seems to be allowed. But I can add "*)" at the end for peace of mind.
[1] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_10_02
As for misleading indentation, I've got my $EDITOR to show me matching parentheses, so I don't need to count them. But if I rewrite that function as following, would it be easier to follow?
+ get_prereq() { \
+ case $$1 in \
+ $(foreach i, $(filter %.h,$^), \
+ $(if $($(patsubst $(srctree)/%,%,$(i))-prereq), \
+ $(i)$(close) \
+ echo "$(foreach j, $($(patsubst $(srctree)/%,%,$(i))-prereq), \
+ -include c$(j))";;)) \
+ *) ;; \
+ esac; \
+ }; \Quoted 11 lines
xen/include/Makefile define cmd_headerscxx_chk · line 174
Quoted 7 lines
xen/include/Makefile +4
-1+ }; \ + for i in $(filter %.h,$^); do \ + echo "#include "\"$$i\" \ | $(CXX) -x c++ -std=gnu++98 -Wall -Werror -D__XEN_TOOLS__ \ -include stdint.h -include $(srcdir)/public/xen.h \ - $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include c$(j)) \ + `get_prereq $$i` \While I know we use back-ticked quoting elsewhere, I'd generally recommend to use $() for readability. But maybe others view this exactly the other way around ...
Well, in Makefile it's `` vs $$(). At least, we don't have to write $$$(open)$(close) here :-).
I guess $$(get_prereq $$i) isn't too bad here, I'll replace the backquote.
Quoted 3 lines
And a question without good context to put at: Isn't headers99.chk in similar need of converting? It looks only slightly less involved than the C++ one.
It doesn't really need converting at the moment, because there's only two headers to check, so the resulting command line is small. But converting it at the same time is probably a good idea to avoid having two different implementations of the header check.
Thanks,
Signature · 1 line
Anthony PERARD
Re: [XEN PATCH v2 1/4] build,include: rework shell script for headers++.chk
Quoted 29 lines · Anthony PERARD
Quoted 20 lines · Jan Beulich
Quoted 8 lines · Anthony PERARD
xen/include/Makefile +0
-0@@ -158,13 +158,22 @@ define cmd_headerscxx_chk touch [email protected]; \ exit 0; \ fi; \xen/include/Makefile define cmd_headerscxx_chk · line 168
Quoted 10 lines
xen/include/Makefile +8
-2- $(foreach i, $(filter %.h,$^), \ - echo "#include "\"$(i)\" \ + get_prereq() { \ + case $$1 in \ + $(foreach i, $(filter %.h,$^), \ + $(if $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + $(i)$(close) \ + echo "$(foreach j, $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + -include c$(j))";;)) \ + esac; \Quoted 7 lines
If I'm reading this right (indentation looks to be a little misleading and hence one needs to count parentheses) the case statement could (in theory) remain without any "body". As per the command language spec I'm looking at this (if it works in the first place) is an extension, and formally there's always at least one label required. Since we aim to be portable in such regards, I'd like to request that there be a final otherwise empty *) line.
Quoted 4 lines
When looking at the shell grammar at [1], an empty body seems to be allowed. But I can add "*)" at the end for peace of mind.
[1] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_10_02
Hmm, indeed. As opposed to
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_04
Quoted 4 lines
As for misleading indentation, I've got my $EDITOR to show me matching parentheses, so I don't need to count them. But if I rewrite that function as following, would it be easier to follow?
xen/include/Makefile define cmd_headerscxx_chk · line 169
Quoted 10 lines
xen/include/Makefile +10
-0+ get_prereq() { \ + case $$1 in \ + $(foreach i, $(filter %.h,$^), \ + $(if $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + $(i)$(close) \ + echo "$(foreach j, $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + -include c$(j))";;)) \ + *) ;; \ + esac; \ + }; \
Hmm, not really, no (and it may be more obvious looking at the reply context above): My primary concern is the use of hard tabs beyond the leading one (which is uniform across all lines and hence doesn't alter apparent alignment even with the + or > prefixed).
Jan
PATCH v2 1/4 build,include: rework shell script for headers++.chk
In reply to Anthony PERARDThe command line generated for headers++.chk by make is quite long, and in some environment it is too long. This issue have been seen in Yocto build environment.
Error messages:
make[9]: execvp: /bin/sh: Argument list too long make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
Rework so that we do the foreach loop in shell rather that make to reduce the command line size by a lot. We also need a way to get headers prerequisite for some public headers so we use a switch "case" in shell to be able to do some simple pattern matching. Variables alone in POSIX shell don't allow to work with associative array or variables with "/".
Also rework headers99.chk as it has a similar implementation, even if with only two headers to check the command line isn't too long at the moment.
---
Notes:
v3:
- add one more pattern to avoid a possible empty body for "case"
- use $() instead of `` to execute get_prereq()
- also convert headers99_chk
- convert some 'tab' to 'space', have only 1 tab at start of line
v2:
- fix typo in commit message
- fix out-of-tree build
v1:
- was sent as a reply to v1 of the series
xen/include/Makefile | 37 +++++++++++++++++++++++++++++-------- 1 file changed, 29 insertions(+), 8 deletions(-)
xen/include/Makefile +29 -8
@@ -141,13 +141,24 @@ cmd_header_chk = \ quiet_cmd_headers99_chk = CHK $@ define cmd_headers99_chk rm -f [email protected]; \ - $(foreach i, $(filter %.h,$^), \ - echo "#include "\"$(i)\" \ + get_prereq() { \ + case $$1 in \ + $(foreach i, $(filter %.h,$^), \ + $(if $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + $(i)$(close) \ + echo "$(foreach j, $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + -include $(j).h)";;)) \ + *) ;; \ + esac; \ + }; \ + for i in $(filter %.h,$^); do \ + echo "#include "\"$$i\" \ | $(CC) -x c -std=c99 -Wall -Werror \ -include stdint.h \ - $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include $(j).h) \ + $$(get_prereq $$i) \ -S -o /dev/null - \ - || exit $$?; echo $(i) >> [email protected];) \ + || exit $$?; echo $$i >> [email protected]; \ + done; \ mv [email protected] $@ endef
@@ -158,13 +169,23 @@ define cmd_headerscxx_chk touch [email protected]; \ exit 0; \ fi; \ - $(foreach i, $(filter %.h,$^), \ - echo "#include "\"$(i)\" \ + get_prereq() { \ + case $$1 in \ + $(foreach i, $(filter %.h,$^), \ + $(if $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + $(i)$(close) \ + echo "$(foreach j, $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + -include c$(j))";;)) \ + *) ;; \ + esac; \ + }; \ + for i in $(filter %.h,$^); do \ + echo "#include "\"$$i\" \ | $(CXX) -x c++ -std=gnu++98 -Wall -Werror -D__XEN_TOOLS__ \ -include stdint.h -include $(srcdir)/public/xen.h \ - $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include c$(j)) \ + $$(get_prereq $$i) \ -S -o /dev/null - \ - || exit $$?; echo $(i) >> [email protected];) \ + || exit $$?; echo $$i >> [email protected]; done; \ mv [email protected] $@ endef
Signature · 1 line
Anthony PERARD
Re: [XEN PATCH v2.1 1/4] build,include: rework shell script for headers++.chk
Hi Anthony,
Quoted 15 lines · Anthony PERARD
The command line generated for headers++.chk by make is quite long, and in some environment it is too long. This issue have been seen in Yocto build environment.
Error messages:
make[9]: execvp:/bin/sh: Argument list too long make[9]: *** [include/Makefile:181:include/headers++.chk] Error 127Rework so that we do the foreach loop in shell rather that make to reduce the command line size by a lot. We also need a way to get headers prerequisite for some public headers so we use a switch "case" in shell to be able to do some simple pattern matching. Variables alone in POSIX shell don't allow to work with associative array or variables with "/".
Quoted 8 lines
Also rework headers99.chk as it has a similar implementation, even if with only two headers to check the command line isn't too long at the moment.
Reported-by Bertrand Marquis <[email protected]>Fixes 28e13c7f43 ("build: xen/include: use if_changed")Signed-off-by Anthony PERARD <[email protected]>Reviewed-by Bertrand Marquis <[email protected]>
Cheers,
Michal
Jan Beulich
Acked-byRe: [XEN PATCH v2.1 1/4] build,include: rework shell script for headers++.chk
Quoted 22 lines · Michal Orzel
Hi Anthony,
Quoted 19 lines · Anthony PERARD
The command line generated for headers++.chk by make is quite long, and in some environment it is too long. This issue have been seen in Yocto build environment.
Error messages:
make[9]: execvp:/bin/sh: Argument list too long make[9]: *** [include/Makefile:181:include/headers++.chk] Error 127Rework so that we do the foreach loop in shell rather that make to reduce the command line size by a lot. We also need a way to get headers prerequisite for some public headers so we use a switch "case" in shell to be able to do some simple pattern matching. Variables alone in POSIX shell don't allow to work with associative array or variables with "/".
Also rework headers99.chk as it has a similar implementation, even if with only two headers to check the command line isn't too long at the moment.
Quoted 6 lines
Quoted 4 lines
Reported-by Bertrand Marquis <[email protected]>Fixes 28e13c7f43 ("build: xen/include: use if_changed")Signed-off-by Anthony PERARD <[email protected]>Reviewed-by Bertrand Marquis <[email protected]>Reviewed-by Michal Orzel <[email protected]>
Quoted 4 lines
Tested-by Michal Orzel <[email protected]>Cheers,
Michal
Re: [XEN PATCH v2.1 1/4] build,include: rework shell script for headers++.chk
In reply to Anthony PERARDQuoted 15 lines · Anthony PERARD
The command line generated for headers++.chk by make is quite long, and in some environment it is too long. This issue have been seen in Yocto build environment.
Error messages:
make[9]: execvp:/bin/sh: Argument list too long make[9]: *** [include/Makefile:181:include/headers++.chk] Error 127Rework so that we do the foreach loop in shell rather that make to reduce the command line size by a lot. We also need a way to get headers prerequisite for some public headers so we use a switch "case" in shell to be able to do some simple pattern matching. Variables alone in POSIX shell don't allow to work with associative array or variables with "/".
Quoted 8 lines
Also rework headers99.chk as it has a similar implementation, even if with only two headers to check the command line isn't too long at the moment.
Reported-by Bertrand Marquis <[email protected]>Fixes 28e13c7f43 ("build: xen/include: use if_changed")Signed-off-by Anthony PERARD <[email protected]>Reviewed-by Bertrand Marquis <[email protected]>
I have committed this, but strictly speaking imo the R-b should have been dropped because ...
Quoted 8 lines
---
Notes:
v3:
- add one more pattern to avoid a possible empty body for "case"
- use $() instead of `` to executeget_prereq()
- also convertheaders99_chk
- convert some 'tab' to 'space', have only 1 tab at start of line
... at least the added headers99_chk conversion was not a purely mechanical change.
Jan
Quoted 71 lines
v2:
- fix typo in commit message
- fix out-of-tree buildv1:
- was sent as a reply to v1 of the seriesxen/include/Makefile | 37 +++++++++++++++++++++++++++++-------- 1 file changed, 29 insertions(+), 8 deletions(-)xen/include/Makefile +29
-8@@ -141,13 +141,24 @@ cmd_header_chk = \ quiet_cmd_headers99_chk = CHK $@ define cmd_headers99_chk rm -f [email protected]; \ - $(foreach i, $(filter %.h,$^), \ - echo "#include "\"$(i)\" \ + get_prereq() { \ + case $$1 in \ + $(foreach i, $(filter %.h,$^), \ + $(if $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + $(i)$(close) \ + echo "$(foreach j, $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + -include $(j).h)";;)) \ + *) ;; \ + esac; \ + }; \ + for i in $(filter %.h,$^); do \ + echo "#include "\"$$i\" \ | $(CC) -x c -std=c99 -Wall -Werror \ -include stdint.h \ - $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include $(j).h) \ + $$(get_prereq $$i) \ -S -o /dev/null - \ - || exit $$?; echo $(i) >> [email protected];) \ + || exit $$?; echo $$i >> [email protected]; \ + done; \ mv [email protected] $@ endef@@ -158,13 +169,23 @@ define cmd_headerscxx_chk touch [email protected]; \ exit 0; \ fi; \ - $(foreach i, $(filter %.h,$^), \ - echo "#include "\"$(i)\" \ + get_prereq() { \ + case $$1 in \ + $(foreach i, $(filter %.h,$^), \ + $(if $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + $(i)$(close) \ + echo "$(foreach j, $($(patsubst $(srctree)/%,%,$(i))-prereq), \ + -include c$(j))";;)) \ + *) ;; \ + esac; \ + }; \ + for i in $(filter %.h,$^); do \ + echo "#include "\"$$i\" \ | $(CXX) -x c++ -std=gnu++98 -Wall -Werror -D__XEN_TOOLS__ \ -include stdint.h -include $(srcdir)/public/xen.h \ - $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include c$(j)) \ + $$(get_prereq $$i) \ -S -o /dev/null - \ - || exit $$?; echo $(i) >> [email protected];) \ + || exit $$?; echo $$i >> [email protected]; done; \ mv [email protected] $@ endef
PATCH v2 2/4 build: remove auto.conf prerequisite from compat/xlat.h target
In reply to Anthony PERARDNow that the command line generating "xlat.h" is check on rebuild, the header will be regenerated whenever the list of xlat headers changes due to change in ".config". We don't need to force a regeneration for every changes in ".config".
--- xen/include/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
xen/include/Makefile +1 -1
@@ -101,7 +101,7 @@ cmd_xlat_h = \ cat $(filter %.h,$^) >[email protected]; \ mv -f [email protected] $@ -$(obj)/compat/xlat.h: $(addprefix $(obj)/compat/.xlat/,$(xlat-y)) $(obj)/config/auto.conf FORCE +$(obj)/compat/xlat.h: $(addprefix $(obj)/compat/.xlat/,$(xlat-y)) FORCE $(call if_changed,xlat_h) ifeq ($(XEN_TARGET_ARCH),$(XEN_COMPILE_ARCH))
Signature · 1 line
Anthony PERARD
PATCH v2 3/4 build: set PERL
In reply to Anthony PERARDWe are going to use it in a moment.
Also update README about Perl requirement.
---
Notes:
v2:
- update ./README
xen/Makefile | 1 + README | 1 + 2 files changed, 2 insertions(+)
xen/Makefile +1 -0
@@ -22,6 +22,7 @@ PYTHON_INTERPRETER := $(word 1,$(shell which python3 python python2 2>/dev/null) export PYTHON ?= $(PYTHON_INTERPRETER) export CHECKPOLICY ?= checkpolicy +export PERL ?= perl $(if $(filter __%, $(MAKECMDGOALS)), \ $(error targets prefixed with '__' are only for internal use))
README +1 -0
@@ -64,6 +64,7 @@ provided by your OS distributor: * iproute package (/sbin/ip) * GNU bison and GNU flex * ACPI ASL compiler (iasl) + * Perl In addition to the above there are a number of optional build prerequisites. Omitting these will cause the related features to be
Signature · 1 line
Anthony PERARD
Re: [XEN PATCH v2 3/4] build: set PERL
xen/Makefile PYTHON_INTERPRETER := $(word 1,$(shell which python3 python python2 2>/dev/null) · line 28
Quoted 10 lines · Anthony PERARD
xen/Makefile +1
-0@@ -22,6 +22,7 @@ PYTHON_INTERPRETER := $(word 1,$(shell which python3 python python2 2>/dev/null) export PYTHON ?= $(PYTHON_INTERPRETER) export CHECKPOLICY ?= checkpolicy +export PERL ?= perl $(if $(filter __%, $(MAKECMDGOALS)), \ $(error targets prefixed with '__' are only for internal use))
Considering my patch yesterday that moved the exporting of CC etc, I wonder if - at the very least for consistency - this and the neighbouring pre-existing exports shouldn't all be moved below the inclusion of ./Config.mk as well. After all ./config might override any of those. (Yes, the ones here don't prevent such overriding, but only as long as there aren't any further make quirks.)
Since this may want doing in a separate patch, this one is
Jan
Re: [XEN PATCH v2 3/4] build: set PERL
Quoted 9 lines · Jan Beulich
Quoted 8 lines · Anthony PERARD
xen/Makefile +1
-0@@ -22,6 +22,7 @@ PYTHON_INTERPRETER := $(word 1,$(shell which python3 python python2 2>/dev/null) export PYTHON ?= $(PYTHON_INTERPRETER) export CHECKPOLICY ?= checkpolicy +export PERL ?= perl
Quoted 9 lines
xen/Makefile PYTHON_INTERPRETER := $(word 1,$(shell which python3 python python2 2>/dev/null) · line 28
Quoted 2 lines
xen/Makefile +0
-0$(if $(filter __%, $(MAKECMDGOALS)), \ $(error targets prefixed with '__' are only for internal use))Considering my patch yesterday that moved the exporting of CC etc, I wonder if - at the very least for consistency - this and the neighbouring pre-existing exports shouldn't all be moved below the inclusion of ./Config.mk as well. After all ./config might override any of those. (Yes, the ones here don't prevent such overriding, but only as long as there aren't any further make quirks.)
Sounds good, we can move CHECKPOLICY and PERL, but there's a problem with PYTHON. Config.mk define a different value for PYTHON, so the one in xen/Makefile needs to be set before including ./Config.mk.
Thanks,
Signature · 1 line
Anthony PERARD
PATCH v2 4/4 build: replace get-fields.sh by a perl script
In reply to Anthony PERARDThe get-fields.sh which generate all the include/compat/.xlat/*.h headers is quite slow. It takes for example nearly 3 seconds to generate platform.h on a recent machine, or 2.3 seconds for memory.h.
Since it's only text processing, rewriting the mix of shell/sed/python into a single perl script make the generation of those file a lot faster.
I tried to keep a similar look for the code, to keep the code similar between the shell and perl, and to ease review. So some code in perl might look weird or could be written better.
No functional change, the headers generated are identical.
---
Notes:
v2:
- Add .pl extension to the perl script
- remove "-w" from the shebang as it is duplicate of "use warning;"
- Add a note in the commit message that the "headers generated are identical".
xen/include/Makefile | 6 +- xen/tools/compat-xlat-header.pl | 539 ++++++++++++++++++++++++++++++++ xen/tools/get-fields.sh | 528 ------------------------------- 3 files changed, 541 insertions(+), 532 deletions(-) create mode 100755 xen/tools/compat-xlat-header.pl delete mode 100644 xen/tools/get-fields.sh
xen/include/Makefile +2 -4
@@ -60,9 +60,7 @@ cmd_compat_c = \ quiet_cmd_xlat_headers = GEN $@ cmd_xlat_headers = \ - while read what name; do \ - $(SHELL) $(srctree)/tools/get-fields.sh "$$what" compat_$$name $< || exit $$?; \ - done <$(patsubst $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename $<)).lst >[email protected]; \ + $(PERL) $(srctree)/tools/compat-xlat-header.pl $< $(patsubst $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename $<)).lst > [email protected]; \ mv -f [email protected] $@ targets += $(headers-y)
@@ -80,7 +78,7 @@ $(obj)/compat/%.c: $(src)/public/%.h $(srcdir)/xlat.lst $(srctree)/tools/compat- $(call if_changed,compat_c) targets += $(patsubst compat/%, compat/.xlat/%, $(headers-y)) -$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(srctree)/tools/get-fields.sh FORCE +$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(srctree)/tools/compat-xlat-header.pl FORCE $(call if_changed,xlat_headers) quiet_cmd_xlat_lst = GEN $@
xen/tools/compat-xlat-header.pl new file +539 -0
539 lines not on this page · raw message
xen/tools/get-fields.sh deleted +0 -528
528 lines not on this page · raw message
Signature · 1 line
Anthony PERARD
Re: [XEN PATCH v2 4/4] build: replace get-fields.sh by a perl script
Quoted 8 lines · Anthony PERARD
The
get-fields.shwhich generate all theinclude/compat/.xlat/*.hheaders is quite slow. It takes for example nearly 3 seconds to generateplatform.hon a recent machine, or 2.3 seconds formemory.h.Since it's only text processing, rewriting the mix of shell/sed/python into a single perl script make the generation of those file a lot faster.
Quoted 7 lines
I tried to keep a similar look for the code, to keep the code similar between the shell and perl, and to ease review. So some code in perl might look weird or could be written better.
No functional change, the headers generated are identical.
Signed-off-by Anthony PERARD <[email protected]>
Hi,
This patch have been on the mailing list for a little while now, could it be acked then committed along with the previous one now?
Thanks,
Signature · 1 line
Anthony PERARD
Re: [XEN PATCH v2 4/4] build: replace get-fields.sh by a perl script
Quoted 13 lines · Anthony PERARD
Quoted 12 lines · Anthony PERARD
The
get-fields.shwhich generate all theinclude/compat/.xlat/*.hheaders is quite slow. It takes for example nearly 3 seconds to generateplatform.hon a recent machine, or 2.3 seconds formemory.h.Since it's only text processing, rewriting the mix of shell/sed/python into a single perl script make the generation of those file a lot faster.
I tried to keep a similar look for the code, to keep the code similar between the shell and perl, and to ease review. So some code in perl might look weird or could be written better.
Quoted 6 lines
Quoted 3 lines
No functional change, the headers generated are identical.
Signed-off-by Anthony PERARD <[email protected]>This patch have been on the mailing list for a little while now, could it be acked then committed along with the previous one now?
Well, I would certainly be happy to ack it, but only on the basis of a suitable R-b by somebody else. My Perl is close to non-existing, so I can't sensibly review this myself.
Jan