Skip to contentSkip to conversation

xen: Build system improvements

PATCH Superseded Anthony PERARD 4 participants 59 messages Mar 31, 2020 – Apr 20, 2020

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v4 cover xen: Build system improvements

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.build-system-xen-v4

v4:
- some patch already applied.
- Have added patches from "xen/arm: Configure early printk via Kconfig" series.
- Some new patch to add documentation or fix issues, and patch to improve compat header generation.
Other changes are detailed in patches.

v3:
- new patches that do some cleanup or fix issues
- have rework most patches, to have better commit message or change the coding style, or fix issues that I've seen. There were some cases where CFLAGS were missing.
See patch notes for details
- introduce if_changed*. That plenty of new patches on top of what we had in v2. (those changes ignore CONFIG_LTO=y, I'll see about fixing that later)

(There is more to come in order to use fixdep from Linux, but that's not ready)

v2.1:
- some fixes

v2:
Rather than taking Kbuild and making it work with Xen, the v2 takes the opposite approach of slowly transforming our current build system into Kbuild. That have the advantage of keeping all the feature we have and making the patches much easier to review. Kconfig update is done in an other patch series.

Hi,

I have work toward building Xen (the hypervisor) with Linux's build system, Kbuild.

The main reason for that is to be able to have out-of-tree build. It's annoying when a build fail because of the pvshim. Other benefit is a much faster rebuild, and make clean doesn't take ages, and better dependencies to figure out what needs to be rebuild.

So, we are not there yet, but the series already contain quite a few improvement and cleanup. More patches are going to be added to the series.

Cheers,

Anthony PERARD (18):
xen/arm: Rename all early printk macro
xen/arm: Configure early printk via Kconfig
build,arm: Fix deps check of head.o
xen/build: include include/config/auto.conf in main Makefile xen/build: use new $(c_flags) and $(a_flags) instead of $(CFLAGS) xen/build: have the root Makefile generates the CFLAGS
build: Introduce documentation for xen Makefiles xen/build: introduce if_changed and if_changed_rule xen/build: Start using if_changed
xen/build: use if_changed on built_in.o
xen/build: Use if_changed_rules with %.o:%.c targets xen/build: factorise generation of the linker scripts xen/build: Use if_changed for prelink*.o
xen,symbols: rework file symbols selection
xen/build: use if_changed to build guest_%.o build,xsm: Fix multiple call
build,include: rework compat-build-source.py build,include: rework compat-build-header.py

 .gitignore                                    |   1 +
 docs/misc/arm/early-printk.txt                |  71 ++---
 docs/misc/xen-makefiles/makefiles.rst         | 186 +++++++++++
 xen/Kconfig.debug                             |   2 +
 xen/Makefile                                  | 209 +++++++++++--
 xen/Rules.mk                                  | 235 ++++++++------
 xen/arch/arm/Kconfig.debug                    | 289 ++++++++++++++++++
 xen/arch/arm/Makefile                         |  27 +-
 xen/arch/arm/Rules.mk                         |  93 ------
 xen/arch/arm/{Rules.mk => arch.mk}            |  79 +----
 xen/arch/arm/arm32/Makefile                   |   2 +-
 xen/arch/arm/arm32/debug-8250.inc             |   2 +-
 xen/arch/arm/arm32/debug-pl011.inc            |   4 +-
 xen/arch/arm/arm32/debug-scif.inc             |   4 +-
 xen/arch/arm/arm32/debug.S                    |   4 +-
 xen/arch/arm/arm32/head.S                     |  10 +-
 xen/arch/arm/arm64/Makefile                   |   2 +-
 xen/arch/arm/arm64/debug-8250.inc             |   4 +-
 xen/arch/arm/arm64/debug-pl011.inc            |   4 +-
 xen/arch/arm/arm64/debug.S                    |   4 +-
 xen/arch/arm/arm64/head.S                     |  10 +-
 xen/arch/arm/efi/Makefile                     |   2 +-
 .../minios.cfg => xen/arch/x86/Kconfig.debug  |   0
 xen/arch/x86/Makefile                         |  41 +--
 xen/arch/x86/Rules.mk                         |  91 +-----
 xen/arch/x86/{Rules.mk => arch.mk}            |  17 +-
 xen/arch/x86/efi/Makefile                     |   9 +-
 xen/arch/x86/mm/Makefile                      |  14 +-
 xen/arch/x86/mm/hap/Makefile                  |  15 +-
 xen/arch/x86/mm/shadow/Makefile               |  14 +-
 xen/common/libelf/Makefile                    |  14 +-
 xen/common/libfdt/Makefile                    |  13 +-
 xen/include/Makefile                          |  24 +-
 xen/include/asm-arm/early_printk.h            |   2 +-
 xen/scripts/Kbuild.include                    | 112 +++++++
 xen/tools/compat-build-header.py              |  36 ++-
 xen/tools/compat-build-source.py              |   8 +-
 xen/tools/symbols.c                           |  20 +-
 xen/xsm/flask/Makefile                        |  19 +-
 xen/xsm/flask/ss/Makefile                     |   2 +-
 40 files changed, 1145 insertions(+), 550 deletions(-)
 create mode 100644 docs/misc/xen-makefiles/makefiles.rst
 create mode 100644 xen/arch/arm/Kconfig.debug
 copy xen/arch/arm/{Rules.mk => arch.mk} (15%)
 copy stubdom/c/minios.cfg => xen/arch/x86/Kconfig.debug (100%)
 copy xen/arch/x86/{Rules.mk => arch.mk} (87%)
Signature · 1 line
Anthony PERARD
#1 lore raw

PATCH v4 1/18 xen/arm: Rename all early printk macro

We are going to move the generation of the early printk macro into Kconfig. This means all macro will be prefix with CONFIG_. We do that ahead of the change.

We also take the opportunity to better name some variables, which are used by only one driver and wouldn't make sens for other UART driver. Thus,
- EARLY_UART_REG_SHIFT became CONFIG_EARLY_UART_8250_REG_SHIFT
- EARLY_PRINTK_VERSION_* became CONFIG_EARLY_UART_SCIF_VERSION_*

The other variables are change to have the prefix CONFIG_EARLY_UART_ when they change a parameter of the driver. So we have now:
- CONFIG_EARLY_UART_BAUD_RATE
- CONFIG_EARLY_UART_BASE_ADDRESS
- CONFIG_EARLY_UART_INIT

---

Notes:
v4:
- cleanup in head.S, removing extra () and space

Patch v3 and early where in "xen/arm: Configure early printk via Kconfig" series.

v3:
- Revert the renaming of EARLY_PRINTK to CONFIG_EARLY_PRINTK in the makefiles, as this doesn't work well with user provided CONFIG_EARLY_PRINTK.
This is done in the following patch instead.

- rename CONFIG_EARLY_UART_BAUD_RATE to CONFIG_EARLY_UART_PL011_BAUD_RATE

 xen/arch/arm/Rules.mk              | 14 +++++++-------
 xen/arch/arm/arm32/debug-8250.inc  |  2 +-
 xen/arch/arm/arm32/debug-pl011.inc |  4 ++--
 xen/arch/arm/arm32/debug-scif.inc  |  4 ++--
 xen/arch/arm/arm32/debug.S         |  4 ++--
 xen/arch/arm/arm32/head.S          | 10 +++++-----
 xen/arch/arm/arm64/debug-8250.inc  |  4 ++--
 xen/arch/arm/arm64/debug-pl011.inc |  4 ++--
 xen/arch/arm/arm64/debug.S         |  4 ++--
 xen/arch/arm/arm64/head.S          | 10 +++++-----
 xen/include/asm-arm/early_printk.h |  2 +-
 11 files changed, 31 insertions(+), 31 deletions(-)
xen/arch/arm/Rules.mk +7 -7
index 022a3a6f82ba..faa09ea111ec 100644
@@ -66,9 +66,9 @@ endif
 endif
 ifeq ($(EARLY_PRINTK_INC),scif)
 ifneq ($(word 3,$(EARLY_PRINTK_CFG)),)
-CFLAGS-y += -DEARLY_PRINTK_VERSION_$(word 3,$(EARLY_PRINTK_CFG))
+CFLAGS-y += -DCONFIG_EARLY_UART_SCIF_VERSION_$(word 3,$(EARLY_PRINTK_CFG))
 else
-CFLAGS-y += -DEARLY_PRINTK_VERSION_NONE
+CFLAGS-y += -DCONFIG_EARLY_UART_SCIF_VERSION_NONE
 endif
 endif
 
@@ -77,11 +77,11 @@ EARLY_PRINTK := y
 endif
 
 CFLAGS-$(EARLY_PRINTK) += -DCONFIG_EARLY_PRINTK
-CFLAGS-$(EARLY_PRINTK_INIT_UART) += -DEARLY_PRINTK_INIT_UART
-CFLAGS-$(EARLY_PRINTK) += -DEARLY_PRINTK_INC=\"debug-$(EARLY_PRINTK_INC).inc\"
-CFLAGS-$(EARLY_PRINTK) += -DEARLY_PRINTK_BAUD=$(EARLY_PRINTK_BAUD)
-CFLAGS-$(EARLY_PRINTK) += -DEARLY_UART_BASE_ADDRESS=$(EARLY_UART_BASE_ADDRESS)
-CFLAGS-$(EARLY_PRINTK) += -DEARLY_UART_REG_SHIFT=$(EARLY_UART_REG_SHIFT)
+CFLAGS-$(EARLY_PRINTK_INIT_UART) += -DCONFIG_EARLY_UART_INIT
+CFLAGS-$(EARLY_PRINTK) += -DCONFIG_EARLY_PRINTK_INC=\"debug-$(EARLY_PRINTK_INC).inc\"
+CFLAGS-$(EARLY_PRINTK) += -DCONFIG_EARLY_UART_PL011_BAUD_RATE=$(EARLY_PRINTK_BAUD)
+CFLAGS-$(EARLY_PRINTK) += -DCONFIG_EARLY_UART_BASE_ADDRESS=$(EARLY_UART_BASE_ADDRESS)
+CFLAGS-$(EARLY_PRINTK) += -DCONFIG_EARLY_UART_8250_REG_SHIFT=$(EARLY_UART_REG_SHIFT)
 
 else # !CONFIG_DEBUG
 
xen/arch/arm/arm32/debug-8250.inc +1 -1
index 0759a27ee157..c47e8be4aaf3 100644
@@ -23,7 +23,7 @@
  */
 .macro early_uart_ready rb rc
 1:
-        ldr     \rc, [\rb, #(UART_LSR << EARLY_UART_REG_SHIFT)] /* Read LSR */
+        ldr     \rc, [\rb, #(UART_LSR << CONFIG_EARLY_UART_8250_REG_SHIFT)] /* Read LSR */
         tst     \rc, #UART_LSR_THRE     /* Check Xmit holding register flag */
         beq     1b                         /* Wait for the UART to be ready */
 .endm
xen/arch/arm/arm32/debug-pl011.inc +2 -2
index ec462eabab5c..214f68dc95bd 100644
@@ -25,9 +25,9 @@
  * rd: scratch register 2 (unused here)
  */
 .macro early_uart_init rb, rc, rd
-        mov   \rc, #(7372800 / EARLY_PRINTK_BAUD % 16)
+        mov   \rc, #(7372800 / CONFIG_EARLY_UART_PL011_BAUD_RATE % 16)
         str   \rc, [\rb, #FBRD]     /* -> UARTFBRD (Baud divisor fraction) */
-        mov   \rc, #(7372800 / EARLY_PRINTK_BAUD / 16)
+        mov   \rc, #(7372800 / CONFIG_EARLY_UART_PL011_BAUD_RATE / 16)
         str   \rc, [\rb, #IBRD]     /* -> UARTIBRD (Baud divisor integer) */
         mov   \rc, #0x60            /* 8n1 */
         str   \rc, [\rb, #LCR_H]     /* -> UARTLCR_H (Line control) */
xen/arch/arm/arm32/debug-scif.inc +2 -2
index 3f01c909c238..b2b82501e792 100644
@@ -19,10 +19,10 @@
 
 #include <asm/scif-uart.h>
 
-#ifdef EARLY_PRINTK_VERSION_NONE
+#ifdef CONFIG_EARLY_UART_SCIF_VERSION_NONE
 #define STATUS_REG    SCIF_SCFSR
 #define TX_FIFO_REG   SCIF_SCFTDR
-#elif EARLY_PRINTK_VERSION_A
+#elif CONFIG_EARLY_UART_SCIF_VERSION_A
 #define STATUS_REG    SCIFA_SCASSR
 #define TX_FIFO_REG   SCIFA_SCAFTDR
 #endif
xen/arch/arm/arm32/debug.S +2 -2
index 1829b29915e0..e77c76d0debc 100644
@@ -19,8 +19,8 @@
 
 #include <asm/early_printk.h>
 
-#ifdef EARLY_PRINTK_INC
-#include EARLY_PRINTK_INC
+#if defined (CONFIG_EARLY_PRINTK_INC)
+#include CONFIG_EARLY_PRINTK_INC
 #endif
 
 /*
xen/arch/arm/arm32/head.S +5 -5
index e9d356f05c2b..c404fa973e9b 100644
@@ -36,8 +36,8 @@
 #define XEN_FIRST_SLOT      first_table_offset(XEN_VIRT_START)
 #define XEN_SECOND_SLOT     second_table_offset(XEN_VIRT_START)
 
-#if (defined (CONFIG_EARLY_PRINTK)) && (defined (EARLY_PRINTK_INC))
-#include EARLY_PRINTK_INC
+#if defined(CONFIG_EARLY_PRINTK) && defined(CONFIG_EARLY_PRINTK_INC)
+#include CONFIG_EARLY_PRINTK_INC
 #endif
 
 /*
@@ -223,7 +223,7 @@ GLOBAL(init_secondary)
 1:
 
 #ifdef CONFIG_EARLY_PRINTK
-        mov_w r11, EARLY_UART_BASE_ADDRESS   /* r11 := UART base address */
+        mov_w r11, CONFIG_EARLY_UART_BASE_ADDRESS   /* r11 := UART base address */
         PRINT("- CPU ")
         print_reg r7
         PRINT(" booting -\r\n")
@@ -706,8 +706,8 @@ ENTRY(switch_ttbr)
  * Clobbers r0 - r3
  */
 init_uart:
-        mov_w r11, EARLY_UART_BASE_ADDRESS
-#ifdef EARLY_PRINTK_INIT_UART
+        mov_w r11, CONFIG_EARLY_UART_BASE_ADDRESS
+#ifdef CONFIG_EARLY_UART_INIT
         early_uart_init r11, r1, r2
 #endif
         PRINT("- UART enabled -\r\n")
xen/arch/arm/arm64/debug-8250.inc +2 -2
index 53d6828bfafe..30ea13077e22 100644
@@ -25,7 +25,7 @@
  */
 .macro early_uart_ready xb c
 1:
-       ldrb  w\c, [\xb, #UART_LSR << EARLY_UART_REG_SHIFT]
+       ldrb  w\c, [\xb, #UART_LSR << CONFIG_EARLY_UART_8250_REG_SHIFT]
        and w\c, w\c, #UART_LSR_THRE
        cmp w\c, #UART_LSR_THRE
        b.ne 1b
@@ -38,7 +38,7 @@
  */
 .macro early_uart_transmit xb wt
         /* UART_THR  transmit holding */
-        strb   \wt, [\xb, #UART_THR << EARLY_UART_REG_SHIFT]
+        strb   \wt, [\xb, #UART_THR << CONFIG_EARLY_UART_8250_REG_SHIFT]
 .endm
 
 /*
xen/arch/arm/arm64/debug-pl011.inc +2 -2
index 569c3dfbcf47..385deff49b1b 100644
@@ -24,9 +24,9 @@
  * c: scratch register number
  */
 .macro early_uart_init xb, c
-        mov   x\c, #(7372800 / EARLY_PRINTK_BAUD % 16)
+        mov   x\c, #(7372800 / CONFIG_EARLY_UART_PL011_BAUD_RATE % 16)
         strh  w\c, [\xb, #0x28]      /* -> UARTFBRD (Baud divisor fraction) */
-        mov   x\c, #(7372800 / EARLY_PRINTK_BAUD / 16)
+        mov   x\c, #(7372800 / CONFIG_EARLY_UART_PL011_BAUD_RATE / 16)
         strh  w\c, [\xb, #0x24]      /* -> UARTIBRD (Baud divisor integer) */
         mov   x\c, #0x60             /* 8n1 */
         str   w\c, [\xb, #0x2C]      /* -> UARTLCR_H (Line control) */
xen/arch/arm/arm64/debug.S +2 -2
index b7f53ac0519b..71cad9d762b2 100644
@@ -19,8 +19,8 @@
 
 #include <asm/early_printk.h>
 
-#ifdef EARLY_PRINTK_INC
-#include EARLY_PRINTK_INC
+#ifdef CONFIG_EARLY_PRINTK_INC
+#include CONFIG_EARLY_PRINTK_INC
 #endif
 
 /*
xen/arch/arm/arm64/head.S +5 -5
index e5015f93a2d8..5d44667bd89d 100644
@@ -45,8 +45,8 @@
 #define __HEAD_FLAGS            ((__HEAD_FLAG_PAGE_SIZE << 1) | \
                                  (__HEAD_FLAG_PHYS_BASE << 3))
 
-#if (defined (CONFIG_EARLY_PRINTK)) && (defined (EARLY_PRINTK_INC))
-#include EARLY_PRINTK_INC
+#if defined(CONFIG_EARLY_PRINTK) && defined(CONFIG_EARLY_PRINTK_INC)
+#include CONFIG_EARLY_PRINTK_INC
 #endif
 
 /*
@@ -363,7 +363,7 @@ GLOBAL(init_secondary)
 1:
 
 #ifdef CONFIG_EARLY_PRINTK
-        ldr   x23, =EARLY_UART_BASE_ADDRESS /* x23 := UART base address */
+        ldr   x23, =CONFIG_EARLY_UART_BASE_ADDRESS /* x23 := UART base address */
         PRINT("- CPU ")
         print_reg x24
         PRINT(" booting -\r\n")
@@ -843,8 +843,8 @@ ENTRY(switch_ttbr)
  * Clobbers x0 - x1
  */
 init_uart:
-        ldr   x23, =EARLY_UART_BASE_ADDRESS
-#ifdef EARLY_PRINTK_INIT_UART
+        ldr   x23, =CONFIG_EARLY_UART_BASE_ADDRESS
+#ifdef CONFIG_EARLY_UART_INIT
         early_uart_init x23, 0
 #endif
         PRINT("- UART enabled -\r\n")
xen/include/asm-arm/early_printk.h +1 -1
index 078cf701dcb0..d5485decfa9f 100644
@@ -15,7 +15,7 @@
 
 /* need to add the uart address offset in page to the fixmap address */
 #define EARLY_UART_VIRTUAL_ADDRESS \
-    (FIXMAP_ADDR(FIXMAP_CONSOLE) +(EARLY_UART_BASE_ADDRESS & ~PAGE_MASK))
+    (FIXMAP_ADDR(FIXMAP_CONSOLE) + (CONFIG_EARLY_UART_BASE_ADDRESS & ~PAGE_MASK))
 
 #endif /* !CONFIG_EARLY_PRINTK */
 
Signature · 1 line
Anthony PERARD
#2 lore raw

PATCH v4 2/18 xen/arm: Configure early printk via Kconfig

In reply to Anthony PERARD

At the moment, early printk can only be configured on the make command line. It is not very handy because a user has to remove the option everytime it is using another command other than compiling the hypervisor.

Furthermore, early printk is one of the few odds one that are not using Kconfig.

So this is about time to move it to Kconfig.

The new kconfigs options allow a user to eather select a UART driver to use at boot time, and set the parameters, or it is still possible to select a platform which will set the parameters.

If CONFIG_EARLY_PRINTK is present in the environment or on the make command line, make will return an error.

---

Notes:
v4:
- Add range to EARLY_UART_BASE_ADDRESS so that kconfig won't accept address higher than 4G on ARM_32
- have EARLY_PRINTK_THUNDERX depends on ARM_64 because the default base address is above 4G
- Add deprecation warning to the help of the choice of early printk driver/platform.
- in early-printk.txt, add that early printk is available with EXPERT.

Patch v3 and early where in "xen/arm: Configure early printk via Kconfig" series.

v3:
- rename EARLY_PRINK to CONFIG_EARLY_PRINTK in makefile here (which select which object to build).
- rename EARLY_UART_BAUD_RATE to EARLY_UART_PL011_BAUD_RATE
- typos
- drop the list of aliases in early-printk.txt. Kconfig choice menu should be enough.
- reword early-printk.txt.
- rework how EARLY_PRINTK is set to Y
and use that instead of a list of all EARLY_UART_*
- Add a check to ask user to use Kconfig to set early printk.
- rework the possible choice to have all uart driver and platform specific option together.
- have added or reword prompt and help messages of the different options. The platform specific option don't have extended help, the prompt is probably enough.
(The non-platform specific options have the help message that Julien have written in the first version.)
- have made EARLY_UART_INIT dependent on the value of EARLY_UART_PL011_BAUD_RATE so that there is no need to expose _INIT to users.

 docs/misc/arm/early-printk.txt                |  71 ++---
 xen/Kconfig.debug                             |   2 +
 xen/arch/arm/Kconfig.debug                    | 289 ++++++++++++++++++
 xen/arch/arm/Makefile                         |   2 +-
 xen/arch/arm/Rules.mk                         |  74 +----
 xen/arch/arm/arm32/Makefile                   |   2 +-
 xen/arch/arm/arm64/Makefile                   |   2 +-
 .../minios.cfg => xen/arch/x86/Kconfig.debug  |   0
 8 files changed, 319 insertions(+), 123 deletions(-)
 create mode 100644 xen/arch/arm/Kconfig.debug
 copy stubdom/c/minios.cfg => xen/arch/x86/Kconfig.debug (100%)
docs/misc/arm/early-printk.txt +23 -48
index 89e081e51eaf..aa22826075a4 100644
@@ -1,64 +1,39 @@
 How to enable early printk
 
-Early printk can only be enabled if debug=y. You may want to enable it if
-you are debbuging code that executes before the console is initialized.
+Early printk can only be enabled if CONFIG_DEBUG=y or in EXPERT mode. You
+may want to enable it if you are debugging code that executes before the
+console is initialized.
 
 Note that selecting this option will limit Xen to a single UART definition.
 Attempting to boot Xen image on a different platform *will not work*, so this
 option should not be enable for Xens that are intended to be portable.
 
-CONFIG_EARLY_PRINTK=<INC>,<BASE_ADDRESS>,<OTHER_OPTIONS>
+Select one of the "Early printk via * UART" in the choice possible for
+"Early printk" in the "Debugging options" of Kconfig. You will then need to
+set other options, which depends on the driver selected.
 
-<INC> and <BASE_ADDRESS> are mandatory arguments:
+CONFIG_EARLY_UART_BASE_ADDRESS is a mandatory argument, it is the base
+physical address of the UART to use.
 
-  - <INC> is the name of the driver, see xen/arch/arm/arm{32,64}/debug-*.inc
-    (where <INC> corresponds to the wildcarded *).
-  - <BASE_ADDRESS> is the base physical address of the UART to use
+Other options depends on the driver selected:
+  - 8250
+    - CONFIG_EARLY_UART_8250_REG_SHIFT is, optionally, the left-shift to
+      apply to the register offsets within the uart.
+  - pl011
+    - CONFIG_EARLY_UART_PL011_BAUD_RATE is, optionally, a baud rate which
+      should be used to configure the UART at start of day.
 
-<OTHER_OPTIONS> varies depending on <INC>:
+      If CONFIG_EARLY_UART_PL011_BAUD_RATE  is set to 0 then the code will
+      not try to initialize the UART, so that bootloader or firmware
+      settings can be used for maximum compatibility.
+  - scif
+    - CONFIG_EARLY_UART_SCIF_VERSION_* is, optionally, the interface version
+      of the UART. Default to version NONE.
 
-  - 8250,<BASE_ADDRESS>,<REG_SHIFT>
-    - <REG_SHIFT> is, optionally, the left-shift to apply to the
-      register offsets within the uart.
-  - pl011,<BASE_ADDRESS>,<BAUD_RATE>
-    - <BAUD_RATE> is, optionally a baud rate which should be used to
-      configure the UART at start of day.
-
-      If <BAUD_RATE> is not given then the code will not try to
-      initialize the UART, so that bootloader or firmware settings can
-     be used for maximum compatibility.
-  - scif,<BASE_ADDRESS>,<VERSION>
-    - SCIF<VERSION> is, optionally, the interface version of the UART.
-
-      If <VERSION> is not given then the default interface version (SCIF)
-      will be used.
   - For all other uarts there are no additional options.
 
 As a convenience it is also possible to select from a list of
-predefined configurations using CONFIG_EARLY_PRINTK=mach where mach is
-the name of the machine:
-
-  - brcm: printk with 8250 on Broadcom 7445D0 boards with A15 processors.
-  - dra7: printk with 8250 on DRA7 platform
-  - exynos5250: printk with the second UART
-  - fastmodel: printk on ARM Fastmodel software emulators
-  - hikey960: printk with pl011 with Hikey 960
-  - juno: printk with pl011 on Juno platform
-  - lager: printk with SCIF0 on Renesas Lager board (R-Car H2 processor)
-  - midway: printk with the pl011 on Calxeda Midway processors
-  - mvebu: printk with the MVEBU for Marvell Armada 3700 SoCs
-  - omap5432: printk with UART3 on TI OMAP5432 processors
-  - rcar3: printk with SCIF2 on Renesas R-Car Gen3 processors
-  - seattle: printk with pl011 for AMD Seattle processor
-  - sun6i: printk with 8250 on Allwinner A31 processors
-  - sun7i: printk with 8250 on Allwinner A20 processors
-  - thunderx: printk with pl011 for Cavium ThunderX processor
-  - vexpress: printk with pl011 for versatile express
-  - xgene-mcdivitt: printk with 820 on Xgene mcdivitt platform
-  - xgene-storm: printk with 820 on Xgene storm platform
-  - zynqmp: printk with Cadence UART for Xilinx ZynqMP SoCs
-
-These settings are is hardcoded in xen/arch/arm/Rules.mk,
-see there when adding support for new machines.
+predefined configurations available in the list of choice for "Early
+printk" for specific platform.
 
 By default early printk is disabled.
xen/Kconfig.debug +2 -0
index b3511e81a275..ee6ee33b69be 100644
@@ -128,6 +128,8 @@ config XMEM_POOL_POISON
 	  Poison free blocks with 0xAA bytes and verify them when a block is
 	  allocated in order to spot use-after-free issues.
 
+source "arch/$(SRCARCH)/Kconfig.debug"
+
 endif # DEBUG || EXPERT
 
 endmenu
xen/arch/arm/Kconfig.debug new file +289 -0

289 lines not on this page · raw message

xen/arch/arm/Makefile +1 -1
index 1044c2298a05..12f92a4bd3f9 100644
@@ -16,7 +16,7 @@ obj-y += device.o
 obj-y += domain.o
 obj-y += domain_build.init.o
 obj-y += domctl.o
-obj-$(EARLY_PRINTK) += early_printk.o
+obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
 obj-y += gic.o
 obj-y += gic-v2.o
 obj-$(CONFIG_GICV3) += gic-v3.o
xen/arch/arm/Rules.mk +2 -72

78 lines not on this page · raw message

xen/arch/arm/arm32/Makefile +1 -1
index 539bbef298a7..96105d238307 100644
@@ -1,6 +1,6 @@
 obj-y += lib/
 
-obj-$(EARLY_PRINTK) += debug.o
+obj-$(CONFIG_EARLY_PRINTK) += debug.o
 obj-y += domctl.o
 obj-y += domain.o
 obj-y += entry.o
xen/arch/arm/arm64/Makefile +1 -1
index db8565b71a33..40642ff57494 100644
@@ -2,7 +2,7 @@ obj-y += lib/
 
 obj-y += cache.o
 obj-$(CONFIG_HARDEN_BRANCH_PREDICTOR) += bpi.o
-obj-$(EARLY_PRINTK) += debug.o
+obj-$(CONFIG_EARLY_PRINTK) += debug.o
 obj-y += domctl.o
 obj-y += domain.o
 obj-y += entry.o
xen/arch/x86/Kconfig.debug renamed from stubdom/c/minios.cfg +0 -0
similarity index 100%
copy from stubdom/c/minios.cfg
copy to xen/arch/x86/Kconfig.debug
Signature · 1 line
Anthony PERARD
Reviewed-by on v4 2/18
#3 lore raw

Re: [XEN PATCH v4 02/18] xen/arm: Configure early printk via Kconfig

Hi Anthony,

Quoted 14 lines · Anthony PERARD

At the moment, early printk can only be configured on the make command line. It is not very handy because a user has to remove the option everytime it is using another command other than compiling the hypervisor.

Furthermore, early printk is one of the few odds one that are not using Kconfig.

So this is about time to move it to Kconfig.

The new kconfigs options allow a user to eather select a UART driver to use at boot time, and set the parameters, or it is still possible to select a platform which will set the parameters.

Quoted 6 lines

If CONFIG_EARLY_PRINTK is present in the environment or on the make command line, make will return an error.

Cheers,

Quoted 575 lines

---

Notes:
v4:
- Add range to EARLY_UART_BASE_ADDRESS so that kconfig won't accept address higher than 4G on ARM_32
- have EARLY_PRINTK_THUNDERX depends on ARM_64 because the default base address is above 4G
- Add deprecation warning to the help of the choice of early printk driver/platform.
- in early-printk.txt, add that early printk is available with EXPERT.

Patch v3 and early where in "xen/arm: Configure early printk via Kconfig" series.

v3:
- rename EARLY_PRINK to CONFIG_EARLY_PRINTK in makefile here (which select which object to build).
- rename EARLY_UART_BAUD_RATE to EARLY_UART_PL011_BAUD_RATE
- typos
- drop the list of aliases in early-printk.txt. Kconfig choice menu should be enough.
- reword early-printk.txt.
- rework how EARLY_PRINTK is set to Y
and use that instead of a list of all EARLY_UART_*
- Add a check to ask user to use Kconfig to set early printk.
- rework the possible choice to have all uart driver and platform specific option together.
- have added or reword prompt and help messages of the different options. The platform specific option don't have extended help, the prompt is probably enough.
(The non-platform specific options have the help message that Julien have written in the first version.)
- have made EARLY_UART_INIT dependent on the value of EARLY_UART_PL011_BAUD_RATE so that there is no need to expose _INIT to users.

  docs/misc/arm/early-printk.txt                |  71 ++---
  xen/Kconfig.debug                             |   2 +
  xen/arch/arm/Kconfig.debug                    | 289 ++++++++++++++++++
  xen/arch/arm/Makefile                         |   2 +-
  xen/arch/arm/Rules.mk                         |  74 +----
  xen/arch/arm/arm32/Makefile                   |   2 +-
  xen/arch/arm/arm64/Makefile                   |   2 +-
  .../minios.cfg => xen/arch/x86/Kconfig.debug  |   0
  8 files changed, 319 insertions(+), 123 deletions(-)

create mode 100644 xen/arch/arm/Kconfig.debug
copy stubdom/c/minios.cfg => xen/arch/x86/Kconfig.debug (100%)

docs/misc/arm/early-printk.txt +23 -48
index 89e081e51eaf..aa22826075a4 100644
@@ -1,64 +1,39 @@
  How to enable early printk
 
-Early printk can only be enabled if debug=y. You may want to enable it if
-you are debbuging code that executes before the console is initialized.
+Early printk can only be enabled if CONFIG_DEBUG=y or in EXPERT mode. You
+may want to enable it if you are debugging code that executes before the
+console is initialized.
 
  Note that selecting this option will limit Xen to a single UART definition.
  Attempting to boot Xen image on a different platform *will not work*, so this
  option should not be enable for Xens that are intended to be portable.
 
-CONFIG_EARLY_PRINTK=<INC>,<BASE_ADDRESS>,<OTHER_OPTIONS>
+Select one of the "Early printk via * UART" in the choice possible for
+"Early printk" in the "Debugging options" of Kconfig. You will then need to
+set other options, which depends on the driver selected.
 
-<INC> and <BASE_ADDRESS> are mandatory arguments:
+CONFIG_EARLY_UART_BASE_ADDRESS is a mandatory argument, it is the base
+physical address of the UART to use.
 
-  - <INC> is the name of the driver, see xen/arch/arm/arm{32,64}/debug-*.inc
-    (where <INC> corresponds to the wildcarded *).
-  - <BASE_ADDRESS> is the base physical address of the UART to use
+Other options depends on the driver selected:
+  - 8250
+    - CONFIG_EARLY_UART_8250_REG_SHIFT is, optionally, the left-shift to
+      apply to the register offsets within the uart.
+  - pl011
+    - CONFIG_EARLY_UART_PL011_BAUD_RATE is, optionally, a baud rate which
+      should be used to configure the UART at start of day.
 
-<OTHER_OPTIONS> varies depending on <INC>:
+      If CONFIG_EARLY_UART_PL011_BAUD_RATE  is set to 0 then the code will
+      not try to initialize the UART, so that bootloader or firmware
+      settings can be used for maximum compatibility.
+  - scif
+    - CONFIG_EARLY_UART_SCIF_VERSION_* is, optionally, the interface version
+      of the UART. Default to version NONE.
 
-  - 8250,<BASE_ADDRESS>,<REG_SHIFT>
-    - <REG_SHIFT> is, optionally, the left-shift to apply to the
-      register offsets within the uart.
-  - pl011,<BASE_ADDRESS>,<BAUD_RATE>
-    - <BAUD_RATE> is, optionally a baud rate which should be used to
-      configure the UART at start of day.
-
-      If <BAUD_RATE> is not given then the code will not try to
-      initialize the UART, so that bootloader or firmware settings can
-     be used for maximum compatibility.
-  - scif,<BASE_ADDRESS>,<VERSION>
-    - SCIF<VERSION> is, optionally, the interface version of the UART.
-
-      If <VERSION> is not given then the default interface version (SCIF)
-      will be used.
    - For all other uarts there are no additional options.
 
  As a convenience it is also possible to select from a list of
-predefined configurations using CONFIG_EARLY_PRINTK=mach where mach is
-the name of the machine:
-
-  - brcm: printk with 8250 on Broadcom 7445D0 boards with A15 processors.
-  - dra7: printk with 8250 on DRA7 platform
-  - exynos5250: printk with the second UART
-  - fastmodel: printk on ARM Fastmodel software emulators
-  - hikey960: printk with pl011 with Hikey 960
-  - juno: printk with pl011 on Juno platform
-  - lager: printk with SCIF0 on Renesas Lager board (R-Car H2 processor)
-  - midway: printk with the pl011 on Calxeda Midway processors
-  - mvebu: printk with the MVEBU for Marvell Armada 3700 SoCs
-  - omap5432: printk with UART3 on TI OMAP5432 processors
-  - rcar3: printk with SCIF2 on Renesas R-Car Gen3 processors
-  - seattle: printk with pl011 for AMD Seattle processor
-  - sun6i: printk with 8250 on Allwinner A31 processors
-  - sun7i: printk with 8250 on Allwinner A20 processors
-  - thunderx: printk with pl011 for Cavium ThunderX processor
-  - vexpress: printk with pl011 for versatile express
-  - xgene-mcdivitt: printk with 820 on Xgene mcdivitt platform
-  - xgene-storm: printk with 820 on Xgene storm platform
-  - zynqmp: printk with Cadence UART for Xilinx ZynqMP SoCs
-
-These settings are is hardcoded in xen/arch/arm/Rules.mk,
-see there when adding support for new machines.
+predefined configurations available in the list of choice for "Early
+printk" for specific platform.
 
  By default early printk is disabled.
xen/Kconfig.debug +2 -0
index b3511e81a275..ee6ee33b69be 100644
@@ -128,6 +128,8 @@ config XMEM_POOL_POISON
  	  Poison free blocks with 0xAA bytes and verify them when a block is
  	  allocated in order to spot use-after-free issues.
 
+source "arch/$(SRCARCH)/Kconfig.debug"
+
  endif # DEBUG || EXPERT
 
  endmenu
xen/arch/arm/Kconfig.debug new file +289 -0
new file mode 100644
index 000000000000..35ccd132732b
@@ -0,0 +1,289 @@
+choice
+	bool "Early printk"
+	optional
+	help
+		You may want to enable early printk if you are debugging code
+		that executes before the console is initialized.
+
+		Note that selecting this option will limit Xen to a single UART
+		definition. Attempting to boot Xen image on a different
+		platform *will not work*, so this option should not be enable
+		for Xens that are intended to be portable.
+
+		Choose one of the UART drivers for early printk, then you'll
+		have to specify the parameters, like the base address.
+
+		Deprecated: Alternatively, there are platform specific options
+		which will have default values for the various parameters. But
+		such option will soon be removed.
+
+	config EARLY_UART_CHOICE_8250
+		select EARLY_UART_8250
+		bool "Early printk via 8250 UART"
+		help
+			Say Y here if you wish the early printk to direct their
+			output to a 8250 UART. You can use this option to
+			provide the parameters for the 8250 UART rather than
+			selecting one of the platform specific options below if
+			you know the parameters for the port.
+
+			This option is preferred over the platform specific
+			options; the platform specific options are deprecated
+			and will soon be removed.
+	config EARLY_UART_CHOICE_CADENCE
+		select EARLY_UART_CADENCE
+		depends on ARM_64
+		bool "Early printk via Cadence UART"
+		help
+			Say Y here if you wish the early printk to direct their
+			output to a Cadence UART. You can use this option to
+			provide the parameters for the Cadence UART rather than
+			selecting one of the platform specific options below if
+			you know the parameters for the port.
+
+			This option is preferred over the platform specific
+			options; the platform specific options are deprecated
+			and will soon be removed.
+	config EARLY_UART_CHOICE_EXYNOS4210
+		select EARLY_UART_EXYNOS4210
+		depends on ARM_32
+		bool "Early printk via Exynos4210 UART"
+		help
+			Say Y here if you wish the early printk to direct their
+			output to a Exynos 4210 UART. You can use this option to
+			provide the parameters for the Exynos 4210 UART rather than
+			selecting one of the platform specific options below if
+			you know the parameters for the port.
+
+			This option is preferred over the platform specific
+			options; the platform specific options are deprecated
+			and will soon be removed.
+	config EARLY_UART_CHOICE_MESON
+		select EARLY_UART_MESON
+		depends on ARM_64
+		bool "Early printk via MESON UART"
+		help
+			Say Y here if you wish the early printk to direct their
+			output to a MESON UART. You can use this option to
+			provide the parameters for the MESON UART rather than
+			selecting one of the platform specific options below if
+			you know the parameters for the port.
+
+			This option is preferred over the platform specific
+			options; the platform specific options are deprecated
+			and will soon be removed.
+	config EARLY_UART_CHOICE_MVEBU
+		select EARLY_UART_MVEBU
+		depends on ARM_64
+		bool "Early printk via MVEBU UART"
+		help
+			Say Y here if you wish the early printk to direct their
+			output to a MVEBU UART. You can use this option to
+			provide the parameters for the MVEBU UART rather than
+			selecting one of the platform specific options below if
+			you know the parameters for the port.
+
+			This option is preferred over the platform specific
+			options; the platform specific options are deprecated
+			and will soon be removed.
+	config EARLY_UART_CHOICE_PL011
+		select EARLY_UART_PL011
+		bool "Early printk via PL011 UART"
+		help
+			Say Y here if you wish the early printk to direct their
+			output to a PL011 UART. You can use this option to
+			provide the parameters for the PL011 UART rather than
+			selecting one of the platform specific options below if
+			you know the parameters for the port.
+
+			This option is preferred over the platform specific
+			options; the platform specific options are deprecated
+			and will soon be removed.
+	config EARLY_UART_CHOICE_SCIF
+		select EARLY_UART_SCIF
+		bool "Early printk via SCIF UART"
+		help
+			Say Y here if you wish the early printk to direct their
+			output to a SCIF UART. You can use this option to
+			provide the parameters for the SCIF UART rather than
+			selecting one of the platform specific options below if
+			you know the parameters for the port.
+
+			This option is preferred over the platform specific
+			options; the platform specific options are deprecated
+			and will soon be removed.
+
+	config EARLY_PRINTK_BRCM
+		bool "Early printk with 8250 on Broadcom 7445D0 boards with A15 processors"
+		select EARLY_UART_8250
+	config EARLY_PRINTK_DRA7
+		bool "Early printk with 8250 on DRA7 platform"
+		select EARLY_UART_8250
+	config EARLY_PRINTK_EXYNOS5250
+		bool "Early printk with the second UART on Exynos5250"
+		select EARLY_UART_EXYNOS4210
+		depends on ARM_32
+	config EARLY_PRINTK_FASTMODEL
+		bool "Early printk with pl011 on ARM Fastmodel software emulators"
+		select EARLY_UART_PL011
+	config EARLY_PRINTK_HIKEY960
+		bool "Early printk with pl011 with Hikey 960"
+		select EARLY_UART_PL011
+	config EARLY_PRINTK_JUNO
+		bool "Early printk with pl011 on Juno platform"
+		select EARLY_UART_PL011
+	config EARLY_PRINTK_LAGER
+		bool "Early printk with SCIF0 on Renesas Lager board (R-Car H2 processor)"
+		select EARLY_UART_SCIF
+	config EARLY_PRINTK_MIDWAY
+		bool "Early printk with pl011 on Calxeda Midway processors"
+		select EARLY_UART_PL011
+	config EARLY_PRINTK_MVEBU
+		bool "Early printk with MVEBU for Marvell Armada 3700 SoCs"
+		select EARLY_UART_MVEBU
+		depends on ARM_64
+	config EARLY_PRINTK_OMAP5432
+		bool "Early printk with UART3 on TI OMAP5432 processors"
+		select EARLY_UART_8250
+	config EARLY_PRINTK_RCAR3
+		bool "Early printk with SCIF2 on Renesas R-Car Gen3 processors"
+		select EARLY_UART_SCIF
+	config EARLY_PRINTK_SEATTLE
+		bool "Early printk with pl011 for AMD Seattle processor"
+		select EARLY_UART_PL011
+	config EARLY_PRINTK_SUN6I
+		bool "Early printk with 8250 on Allwinner A31 processors"
+		select EARLY_UART_8250
+	config EARLY_PRINTK_SUN7I
+		bool "Early printk with 8250 on Allwinner A20 processors"
+		select EARLY_UART_8250
+	config EARLY_PRINTK_THUNDERX
+		bool "Early printk with pl011 for Cavium ThunderX processor"
+		select EARLY_UART_PL011
+		depends on ARM_64
+	config EARLY_PRINTK_VEXPRESS
+		bool "Early printk with pl011 for versatile express"
+		select EARLY_UART_PL011
+	config EARLY_PRINTK_XGENE_MCDIVITT
+		bool "Early printk with 820 on Xgene mcdivitt platform"
+		select EARLY_UART_8250
+	config EARLY_PRINTK_XGENE_STORM
+		bool "Early printk with 820 on Xgene storm platform"
+		select EARLY_UART_8250
+	config EARLY_PRINTK_ZYNQMP
+		bool "Early printk with Cadence UART for Xilinx ZynqMP SoCs"
+		select EARLY_UART_CADENCE
+		depends on ARM_64
+endchoice
+
+
+config EARLY_UART_8250
+	select EARLY_PRINTK
+	bool
+config EARLY_UART_CADENCE
+	select EARLY_PRINTK
+	bool
+config EARLY_UART_EXYNOS4210
+	select EARLY_PRINTK
+	bool
+config EARLY_UART_MESON
+	select EARLY_PRINTK
+	bool
+config EARLY_UART_MVEBU
+	select EARLY_PRINTK
+	bool
+config EARLY_UART_PL011
+	select EARLY_PRINTK
+	bool
+config EARLY_UART_SCIF
+	select EARLY_PRINTK
+	bool
+
+config EARLY_PRINTK
+	bool
+
+config EARLY_UART_BASE_ADDRESS
+	depends on EARLY_PRINTK
+	hex "Early printk, physical base address of debug UART"
+	range 0x0 0xffffffff if ARM_32
+	default 0xF040AB00 if EARLY_PRINTK_BRCM
+	default 0x4806A000 if EARLY_PRINTK_DRA7
+	default 0x1c090000 if EARLY_PRINTK_FASTMODEL
+	default 0x12c20000 if EARLY_PRINTK_EXYNOS5250
+	default 0xfff32000 if EARLY_PRINTK_HIKEY960
+	default 0x7ff80000 if EARLY_PRINTK_JUNO
+	default 0xe6e60000 if EARLY_PRINTK_LAGER
+	default 0xfff36000 if EARLY_PRINTK_MIDWAY
+	default 0xd0012000 if EARLY_PRINTK_MVEBU
+	default 0x48020000 if EARLY_PRINTK_OMAP5432
+	default 0xe6e88000 if EARLY_PRINTK_RCAR3
+	default 0xe1010000 if EARLY_PRINTK_SEATTLE
+	default 0x01c28000 if EARLY_PRINTK_SUN6I
+	default 0x01c28000 if EARLY_PRINTK_SUN7I
+	default 0x87e024000000 if EARLY_PRINTK_THUNDERX
+	default 0x1c090000 if EARLY_PRINTK_VEXPRESS
+	default 0x1c021000 if EARLY_PRINTK_XGENE_MCDIVITT
+	default 0x1c020000 if EARLY_PRINTK_XGENE_STORM
+	default 0xff000000 if EARLY_PRINTK_ZYNQMP
+
+config EARLY_UART_PL011_BAUD_RATE
+	depends on EARLY_UART_PL011
+	int "Early printk UART baud rate for pl011"
+	help
+		Optionally sets the baud rate which should be used to configure
+		the UART at start of day.
+
+		If EARLY_UART_PL011_BAUD_RATE is set to 0 then the code will
+		not try to initialize the UART, so that bootloader or firmware
+		settings can be used for maximum compatibility.
+
+	default 115200 if EARLY_PRINTK_FASTMODEL
+	default 0
+
+config EARLY_UART_INIT
+	depends on EARLY_UART_PL011 && EARLY_UART_PL011_BAUD_RATE != 0
+	def_bool y
+
+config EARLY_UART_8250_REG_SHIFT
+	depends on EARLY_UART_8250
+	int "Early printk, left-shift to apply to the register offsets within the 8250 UART"
+	help
+		EARLY_UART_8250_REG_SHIFT is, optionally, the left-shift to
+		apply to the register offsets within the UART with early
+		printk.
+
+		Default to 0.
+
+	default 2 if EARLY_PRINTK_BRCM
+	default 2 if EARLY_PRINTK_DRA7
+	default 2 if EARLY_PRINTK_OMAP5432
+	default 2 if EARLY_PRINTK_SUN6I
+	default 2 if EARLY_PRINTK_SUN7I
+	default 2 if EARLY_PRINTK_XGENE_MCDIVITT
+	default 2 if EARLY_PRINTK_XGENE_STORM
+	default 0
+
+choice EARLY_UART_SCIF_VERSION
+	prompt "Early printk UART SCIF interface version"
+	depends on EARLY_UART_SCIF
+	default EARLY_UART_SCIF_VERSION_NONE
+	help
+		Select the interface version of the SCIF UART.
+
+		Select EARLY_UART_SCIF_VERSION_NONE to use the default
+		interface version (SCIF).
+	config EARLY_UART_SCIF_VERSION_NONE
+		bool "default SCIF UART interface"
+	config EARLY_UART_SCIF_VERSION_A
+		bool "SCIF UART interface version A"
+endchoice
+
+config EARLY_PRINTK_INC
+	string
+	default "debug-8250.inc" if EARLY_UART_8250
+	default "debug-cadence.inc" if EARLY_UART_CADENCE
+	default "debug-exynos4210.inc" if EARLY_UART_EXYNOS4210
+	default "debug-meson.inc" if EARLY_UART_MESON
+	default "debug-mvebu.inc" if EARLY_UART_MVEBU
+	default "debug-pl011.inc" if EARLY_UART_PL011
+	default "debug-scif.inc" if EARLY_UART_SCIF
xen/arch/arm/Makefile +1 -1
index 1044c2298a05..12f92a4bd3f9 100644
@@ -16,7 +16,7 @@ obj-y += device.o
  obj-y += domain.o
  obj-y += domain_build.init.o
  obj-y += domctl.o
-obj-$(EARLY_PRINTK) += early_printk.o
+obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
  obj-y += gic.o
  obj-y += gic-v2.o
  obj-$(CONFIG_GICV3) += gic-v3.o
xen/arch/arm/Rules.mk +2 -72
index faa09ea111ec..3ad284aa71a4 100644
@@ -18,76 +18,6 @@ CFLAGS-$(CONFIG_ARM_32) += -mcpu=cortex-a15
  CFLAGS-$(CONFIG_ARM_64) += -mcpu=generic
  CFLAGS-$(CONFIG_ARM_64) += -mgeneral-regs-only # No fp registers etc
 
-EARLY_PRINTK := n
-
-ifeq ($(CONFIG_DEBUG),y)
-
-# See docs/misc/arm/early-printk.txt for syntax
-
-EARLY_PRINTK_brcm           := 8250,0xF040AB00,2
-EARLY_PRINTK_dra7           := 8250,0x4806A000,2
-EARLY_PRINTK_fastmodel      := pl011,0x1c090000,115200
-EARLY_PRINTK_exynos5250     := exynos4210,0x12c20000
-EARLY_PRINTK_hikey960       := pl011,0xfff32000
-EARLY_PRINTK_juno           := pl011,0x7ff80000
-EARLY_PRINTK_lager          := scif,0xe6e60000
-EARLY_PRINTK_midway         := pl011,0xfff36000
-EARLY_PRINTK_mvebu          := mvebu,0xd0012000
-EARLY_PRINTK_omap5432       := 8250,0x48020000,2
-EARLY_PRINTK_rcar3          := scif,0xe6e88000
-EARLY_PRINTK_seattle        := pl011,0xe1010000
-EARLY_PRINTK_sun6i          := 8250,0x01c28000,2
-EARLY_PRINTK_sun7i          := 8250,0x01c28000,2
-EARLY_PRINTK_thunderx       := pl011,0x87e024000000
-EARLY_PRINTK_vexpress       := pl011,0x1c090000
-EARLY_PRINTK_xgene-mcdivitt := 8250,0x1c021000,2
-EARLY_PRINTK_xgene-storm    := 8250,0x1c020000,2
-EARLY_PRINTK_zynqmp         := cadence,0xff000000
-
-ifneq ($(EARLY_PRINTK_$(CONFIG_EARLY_PRINTK)),)
-EARLY_PRINTK_CFG := $(subst $(comma), ,$(EARLY_PRINTK_$(CONFIG_EARLY_PRINTK)))
-else
-EARLY_PRINTK_CFG := $(subst $(comma), ,$(CONFIG_EARLY_PRINTK))
-endif
-
-# Extract configuration from string
-EARLY_PRINTK_INC := $(word 1,$(EARLY_PRINTK_CFG))
-EARLY_UART_BASE_ADDRESS := $(word 2,$(EARLY_PRINTK_CFG))
-
-# UART specific options
-ifeq ($(EARLY_PRINTK_INC),8250)
-EARLY_UART_REG_SHIFT := $(word 3,$(EARLY_PRINTK_CFG))
-endif
-ifeq ($(EARLY_PRINTK_INC),pl011)
-ifneq ($(word 3,$(EARLY_PRINTK_CFG)),)
-EARLY_PRINTK_INIT_UART := y
-EARLY_PRINTK_BAUD := $(word 3,$(EARLY_PRINTK_CFG))
-endif
-endif
-ifeq ($(EARLY_PRINTK_INC),scif)
-ifneq ($(word 3,$(EARLY_PRINTK_CFG)),)
-CFLAGS-y += -DCONFIG_EARLY_UART_SCIF_VERSION_$(word 3,$(EARLY_PRINTK_CFG))
-else
-CFLAGS-y += -DCONFIG_EARLY_UART_SCIF_VERSION_NONE
-endif
-endif
-
-ifneq ($(EARLY_PRINTK_INC),)
-EARLY_PRINTK := y
-endif
-
-CFLAGS-$(EARLY_PRINTK) += -DCONFIG_EARLY_PRINTK
-CFLAGS-$(EARLY_PRINTK_INIT_UART) += -DCONFIG_EARLY_UART_INIT
-CFLAGS-$(EARLY_PRINTK) += -DCONFIG_EARLY_PRINTK_INC=\"debug-$(EARLY_PRINTK_INC).inc\"
-CFLAGS-$(EARLY_PRINTK) += -DCONFIG_EARLY_UART_PL011_BAUD_RATE=$(EARLY_PRINTK_BAUD)
-CFLAGS-$(EARLY_PRINTK) += -DCONFIG_EARLY_UART_BASE_ADDRESS=$(EARLY_UART_BASE_ADDRESS)
-CFLAGS-$(EARLY_PRINTK) += -DCONFIG_EARLY_UART_8250_REG_SHIFT=$(EARLY_UART_REG_SHIFT)
-
-else # !CONFIG_DEBUG
-
-ifneq ($(CONFIG_EARLY_PRINTK),)
-# Early printk is dependant on a debug build.
-$(error CONFIG_EARLY_PRINTK enabled for non-debug build)
-endif
-
+ifneq ($(filter command line environment,$(origin CONFIG_EARLY_PRINTK)),)
+    $(error You must use 'make menuconfig' to enable/disable early printk now)
  endif
xen/arch/arm/arm32/Makefile +1 -1
index 539bbef298a7..96105d238307 100644
@@ -1,6 +1,6 @@
  obj-y += lib/
 
-obj-$(EARLY_PRINTK) += debug.o
+obj-$(CONFIG_EARLY_PRINTK) += debug.o
  obj-y += domctl.o
  obj-y += domain.o
  obj-y += entry.o
xen/arch/arm/arm64/Makefile +1 -1
index db8565b71a33..40642ff57494 100644
@@ -2,7 +2,7 @@ obj-y += lib/
 
  obj-y += cache.o
  obj-$(CONFIG_HARDEN_BRANCH_PREDICTOR) += bpi.o
-obj-$(EARLY_PRINTK) += debug.o
+obj-$(CONFIG_EARLY_PRINTK) += debug.o
  obj-y += domctl.o
  obj-y += domain.o
  obj-y += entry.o
xen/arch/x86/Kconfig.debug renamed from stubdom/c/minios.cfg +0 -0
similarity index 100%
copy from stubdom/c/minios.cfg
copy to xen/arch/x86/Kconfig.debug
Signature · 1 line
Julien Grall
#4 lore raw

PATCH v4 3/18 build,arm: Fix deps check of head.o

In reply to Anthony PERARD

arm*/head.o isn't in obj-y or extra-y, so make don't load the associated .*.d file (or .*.cmd file when if_changed will be used). There is a workaround where .*.d file is added manually into DEPS.

Changing DEPS isn't needed, we can simply add head.o into extra-y and the dependency files will be loaded.

---

Notes:
v4:
- new patch, fix rebuild of head.o with "xen/build: Start using if_changed" applied

 xen/arch/arm/Makefile | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
xen/arch/arm/Makefile +1 -2
index 12f92a4bd3f9..7273f356f190 100644
@@ -62,6 +62,7 @@ obj-$(CONFIG_SBSA_VUART_CONSOLE) += vpl011.o
 obj-y += vsmc.o
 obj-y += vpsci.o
 obj-y += vuart.o
+extra-y += $(TARGET_SUBARCH)/head.o
 
 #obj-bin-y += ....o
 
@@ -72,8 +73,6 @@ endif
 
 ALL_OBJS := $(TARGET_SUBARCH)/head.o $(ALL_OBJS)
 
-DEPS += $(TARGET_SUBARCH)/.head.o.d
-
 ifdef CONFIG_LIVEPATCH
 all_symbols = --all-symbols
 ifdef CONFIG_FAST_SYMBOL_LOOKUP
Signature · 1 line
Anthony PERARD
Acked-by on v4 3/18
#5 lore raw

Re: [XEN PATCH v4 03/18] build,arm: Fix deps check of head.o

Hi Anthony,

Quoted 8 lines · Anthony PERARD

arm*/head.o isn't in obj-y or extra-y, so make don't load the associated .*.d file (or .*.cmd file when if_changed will be used). There is a workaround where .*.d file is added manually into DEPS.

Changing DEPS isn't needed, we can simply add head.o into extra-y and the dependency files will be loaded.

Cheers,

Signature · 1 line
Julien Grall
#6 lore raw

PATCH v4 4/18 xen/build: include include/config/auto.conf in main Makefile

In reply to Anthony PERARD

We are going to generate the CFLAGS early from "xen/Makefile" instead of in "Rules.mk", but we need to include "config/auto.conf", so include it in "Makefile".

Before including "config/auto.conf" we check which make target a user is calling, as some targets don't need "auto.conf". For targets that needs auto.conf, make will generate it (and a default .config if missing).

root-make-done is to avoid doing the calculation again once Rules.mk takes over and is been executed with the root Makefile. When Rules.mk is including xen/Makefile, config-build' and need-config' are undefined so auto.conf will not be included again (it is already included by Rules.mk) and kconfig target are out of reach of Rules.mk.

We are introducing a target %config to catch all targets for kconfig. So we need an extra target %/.config to prevent make from trying to regenerate $(XEN_ROOT)/.config that is included in Config.mk.

The way targets are filtered is inspired by Kbuild, with some code imported from Linux. That's why there is PHONY variable that isn't used yet, for example.

---

Notes:
v4:
- check that root-make-done hasn't been set to an expected value instead of checking if it has been set at all.
- Add a shorthand $(kconfig) to run kconfig targets.

v3:
- filter only for %config instead of both config %config
- keep the multi-target pattern rule trick for include/config/auto.conf instead of using Linux's newer pattern (we dont have tristate.conf so don't need to change it)
- use y/n for root-make-done, config-build, need-config instead of relying on ifdef and ifndef and on assigning an empty value meaning undef
- use space for indentation
- explain why %/.config is suddenly needed.

 xen/Makefile               | 98 +++++++++++++++++++++++++++++---------
 xen/scripts/Kbuild.include |  5 ++
 2 files changed, 80 insertions(+), 23 deletions(-)
xen/Makefile +75 -23
index e5f7b1ae13bc..8375070e0d41 100644
@@ -49,7 +49,73 @@ default: build
 .PHONY: dist
 dist: install
 
-build install:: include/config/auto.conf
+
+ifneq ($(root-make-done),y)
+# section to run before calling Rules.mk, but only once.
+#
+# To make sure we do not include .config for any of the *config targets
+# catch them early, and hand them over to tools/kconfig/Makefile
+
+clean-targets := %clean
+no-dot-config-targets := $(clean-targets) \
+                         uninstall debug cloc \
+                         cscope TAGS tags MAP gtags \
+                         xenversion
+
+config-build    := n
+need-config     := y
+
+ifneq ($(filter $(no-dot-config-targets), $(MAKECMDGOALS)),)
+    ifeq ($(filter-out $(no-dot-config-targets), $(MAKECMDGOALS)),)
+        need-config := n
+    endif
+endif
+
+ifneq ($(filter %config,$(MAKECMDGOALS)),)
+    config-build := y
+endif
+
+export root-make-done := y
+endif # root-make-done
+
+include scripts/Kbuild.include
+
+ifeq ($(config-build),y)
+# ===========================================================================
+# *config targets only - make sure prerequisites are updated, and descend
+# in tools/kconfig to make the *config target
+
+config: FORCE
+	$(MAKE) $(kconfig) $@
+
+# Config.mk tries to include .config file, don't try to remake it
+%/.config: ;
+
+%config: FORCE
+	$(MAKE) $(kconfig) $@
+
+else # !config-build
+
+ifeq ($(need-config),y)
+include include/config/auto.conf
+# Read in dependencies to all Kconfig* files, make sure to run syncconfig if
+# changes are detected.
+include include/config/auto.conf.cmd
+
+# Allow people to just run `make` as before and not force them to configure
+$(KCONFIG_CONFIG):
+	$(MAKE) $(kconfig) defconfig
+
+# The actual configuration files used during the build are stored in
+# include/generated/ and include/config/. Update them if .config is newer than
+# include/config/auto.conf (which mirrors .config).
+#
+# This exploits the 'multi-target pattern rule' trick.
+# The syncconfig should be executed only once to make all the targets.
+include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
+	$(MAKE) $(kconfig) syncconfig
+
+endif # need-config
 
 .PHONY: build install uninstall clean distclean MAP
 build install uninstall debug clean distclean MAP::
@@ -254,9 +320,6 @@ cscope:
 _MAP:
 	$(NM) -n $(TARGET)-syms | grep -v '\(compiled\)\|\(\.o$$\)\|\( [aUw] \)\|\(\.\.ng$$\)\|\(LASH[RL]DI\)' > System.map
 
-.PHONY: FORCE
-FORCE:
-
 %.o %.i %.s: %.c FORCE
 	$(MAKE) -f $(BASEDIR)/Rules.mk -C $(*D) $(@F)
 
@@ -277,25 +340,6 @@ $(foreach base,arch/x86/mm/guest_walk_% \
                arch/x86/mm/shadow/guest_%, \
     $(foreach ext,o i s,$(call build-intermediate,$(base).$(ext))))
 
-kconfig := oldconfig config menuconfig defconfig allyesconfig allnoconfig \
-	nconfig xconfig gconfig savedefconfig listnewconfig olddefconfig \
-	randconfig $(notdir $(wildcard arch/$(SRCARCH)/configs/*_defconfig))
-.PHONY: $(kconfig)
-$(kconfig):
-	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" $@
-
-include/config/%.conf: include/config/auto.conf.cmd $(KCONFIG_CONFIG)
-	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" syncconfig
-
-# Allow people to just run `make` as before and not force them to configure
-$(KCONFIG_CONFIG):
-	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" defconfig
-
-# Break the dependency chain for the first run
-include/config/auto.conf.cmd: ;
-
--include $(BASEDIR)/include/config/auto.conf.cmd
-
 .PHONY: cloc
 cloc:
 	$(eval tmpfile := $(shell mktemp))
@@ -307,3 +351,11 @@ cloc:
 	cloc --list-file=$(tmpfile)
 	rm $(tmpfile)
 
+endif #config-build
+
+PHONY += FORCE
+FORCE:
+
+# Declare the contents of the PHONY variable as phony.  We keep that
+# information in a variable so we can use it in if_changed and friends.
+.PHONY: $(PHONY)
xen/scripts/Kbuild.include +5 -0
index 806c68824ed5..14f68fa42868 100644
@@ -32,3 +32,8 @@ cc-ifversion = $(shell [ $(CONFIG_GCC_VERSION)0 $(1) $(2)000 ] && echo $(3) || e
 # Usage:
 # $(MAKE) $(clean) dir
 clean := -f $(BASEDIR)/scripts/Makefile.clean clean -C
+
+# Shorthand for kconfig
+# Usage:
+# $(MAKE) $(kconfig) target
+kconfig = -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)"
Signature · 1 line
Anthony PERARD
Reviewed-by on v4 4/18
#7 lore raw

Re: [XEN PATCH v4 04/18] xen/build: include include/config/auto.conf in main Makefile

xen/scripts/Kbuild.include cc-ifversion = $(shell [ $(CONFIG_GCC_VERSION)0 $(1) $(2)000 ] && echo $(3) || e · line 39

Quoted 11 lines · Anthony PERARD
xen/scripts/Kbuild.include +5 -0
@@ -32,3 +32,8 @@ cc-ifversion = $(shell [ $(CONFIG_GCC_VERSION)0 $(1) $(2)000 ] && echo $(3) || e
 # Usage:
 # $(MAKE) $(clean) dir
 clean := -f $(BASEDIR)/scripts/Makefile.clean clean -C
+
+# Shorthand for kconfig
+# Usage:
+# $(MAKE) $(kconfig) target
+kconfig = -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)"

Is this going to be needed outside of xen/Makefile? If not, I'd like to ask that it be local to xen/Makefile. With the adjustment or with a reply clarifying to future plans

Jan

#8 lore raw

Re: [XEN PATCH v4 04/18] xen/build: include include/config/auto.conf in main Makefile

Quoted 16 lines · Jan Beulich

xen/scripts/Kbuild.include cc-ifversion = $(shell [ $(CONFIG_GCC_VERSION)0 $(1) $(2)000 ] && echo $(3) || e · line 39

Quoted 11 lines · Anthony PERARD
xen/scripts/Kbuild.include +5 -0
@@ -32,3 +32,8 @@ cc-ifversion = $(shell [ $(CONFIG_GCC_VERSION)0 $(1) $(2)000 ] && echo $(3) || e
 # Usage:
 # $(MAKE) $(clean) dir
 clean := -f $(BASEDIR)/scripts/Makefile.clean clean -C
+
+# Shorthand for kconfig
+# Usage:
+# $(MAKE) $(kconfig) target
+kconfig = -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)"

Is this going to be needed outside of xen/Makefile? If not, I'd like to ask that it be local to xen/Makefile. With the adjustment or with a reply clarifying to future plans

Quoted 1 line

I'll move that to xen/Makefile.
Thanks,

Signature · 1 line
Anthony PERARD
#9 lore raw

PATCH v4 5/18 xen/build: use new $(c_flags) and $(a_flags) instead of $(CFLAGS)

In reply to Anthony PERARD

In a later patch ("xen/build: have the root Makefile generates the CFLAGS), we want to generate the CFLAGS in xen/Makefile, then export it and have Rules.mk use a CFLAGS from the environment variables. That changes the flavor of the CFLAGS and flags intended for one target (like -D__OBJECT_FILE__ and -M%) gets propagated and duplicated. So we start by moving such flags out of $(CFLAGS) and into $(c_flags) which is to be modified by only Rules.mk.

__OBJECT_FILE__ is only used by arch/x86/mm/*.c files, so having it in $(c_flags) is enough, we don't need it in $(a_flags).

For include/Makefile and as-insn we can keep using CFLAGS, but since it doesn't have -M* flags anymore there is no need to filter them out.

The XEN_BUILD_EFI tests in arch/x86/Makefile was filtering out CFLAGS-y, but according to dd40177c1bc8 ("x86-64/EFI: add CFLAGS to check compile"), it was done to filter out -MF. CFLAGS doesn't have those flags anymore, so no filtering is needed.

This is inspired by the way Kbuild generates CFLAGS for each targets.

---

Notes:
v4:
- drop change in as-insn macro, and keep filtering-out -M% %.d

v3:
- include/Makefile: Keep using CFLAGS, but since it doesn't have -M* flags anymore, no need to filter it.
- Write c_flags and a_flags on a single line.
- arch/x86/Makefile: remove the filter-out of dependency flags they are remove from CFLAGS anyway.
(was intended to be done in xen/build: have the root Makefile generates the CFLAGS originally, move the change to this patch).
- also modify as-insn as it is now xen/ only.

 xen/Rules.mk                    | 23 +++++++++++------------
 xen/arch/arm/Makefile           |  4 ++--
 xen/arch/x86/Makefile           |  6 +++---
 xen/arch/x86/mm/Makefile        |  6 +++---
 xen/arch/x86/mm/hap/Makefile    |  6 +++---
 xen/arch/x86/mm/shadow/Makefile |  6 +++---
 xen/include/Makefile            |  2 +-
 7 files changed, 26 insertions(+), 27 deletions(-)
xen/Rules.mk +11 -12
index 9079df7978a7..3408a35dbf53 100644
@@ -57,7 +57,6 @@ CFLAGS += -Werror -Wredundant-decls -Wno-pointer-arith
 $(call cc-option-add,CFLAGS,CC,-Wvla)
 CFLAGS += -pipe -D__XEN__ -include $(BASEDIR)/include/xen/config.h
 CFLAGS-$(CONFIG_DEBUG_INFO) += -g
-CFLAGS += '-D__OBJECT_FILE__="$@"'
 
 ifneq ($(CONFIG_CC_IS_CLANG),y)
 # Clang doesn't understand this command line argument, and doesn't appear to
@@ -70,9 +69,6 @@ AFLAGS += -D__ASSEMBLY__
 
 ALL_OBJS := $(ALL_OBJS-y)
 
-# Get gcc to generate the dependencies for us.
-CFLAGS-y += -MMD -MP -MF $(@D)/.$(@F).d
-
 CFLAGS += $(CFLAGS-y)
 # allow extra CFLAGS externally via EXTRA_CFLAGS_XEN_CORE
 CFLAGS += $(EXTRA_CFLAGS_XEN_CORE)
@@ -146,9 +142,12 @@ endif
 # Always build obj-bin files as binary even if they come from C source. 
 $(obj-bin-y): CFLAGS := $(filter-out -flto,$(CFLAGS))
 
+c_flags = -MMD -MP -MF $(@D)/.$(@F).d $(CFLAGS) '-D__OBJECT_FILE__="$@"'
+a_flags = -MMD -MP -MF $(@D)/.$(@F).d $(AFLAGS)
+
 built_in.o: $(obj-y) $(extra-y)
 ifeq ($(obj-y),)
-	$(CC) $(CFLAGS) -c -x c /dev/null -o $@
+	$(CC) $(c_flags) -c -x c /dev/null -o $@
 else
 ifeq ($(CONFIG_LTO),y)
 	$(LD_LTO) -r -o $@ $(filter-out $(extra-y),$^)
@@ -159,7 +158,7 @@ endif
 
 built_in_bin.o: $(obj-bin-y) $(extra-y)
 ifeq ($(obj-bin-y),)
-	$(CC) $(AFLAGS) -c -x assembler /dev/null -o $@
+	$(CC) $(a_flags) -c -x assembler /dev/null -o $@
 else
 	$(LD) $(LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
 endif
@@ -178,7 +177,7 @@ SRCPATH := $(patsubst $(BASEDIR)/%,%,$(CURDIR))
 
 %.o: %.c Makefile
 ifeq ($(CONFIG_ENFORCE_UNIQUE_SYMBOLS),y)
-	$(CC) $(CFLAGS) -c $< -o $(@D)/.$(@F).tmp -MQ $@
+	$(CC) $(c_flags) -c $< -o $(@D)/.$(@F).tmp -MQ $@
 ifeq ($(CONFIG_CC_IS_CLANG),y)
 	$(OBJCOPY) --redefine-sym $<=$(SRCPATH)/$< $(@D)/.$(@F).tmp $@
 else
@@ -186,11 +185,11 @@ else
 endif
 	rm -f $(@D)/.$(@F).tmp
 else
-	$(CC) $(CFLAGS) -c $< -o $@
+	$(CC) $(c_flags) -c $< -o $@
 endif
 
 %.o: %.S Makefile
-	$(CC) $(AFLAGS) -c $< -o $@
+	$(CC) $(a_flags) -c $< -o $@
 
 $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o Makefile
 	$(OBJDUMP) -h $< | sed -n '/[0-9]/{s,00*,0,g;p;}' | while read idx name sz rest; do \
@@ -205,12 +204,12 @@ $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o Makefile
 	$(OBJCOPY) $(foreach s,$(SPECIAL_DATA_SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
 
 %.i: %.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(CFLAGS)) $< -o $@
+	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) $< -o $@
 
 %.s: %.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(CFLAGS)) -S $< -o $@
+	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -S $< -o $@
 
 %.s: %.S Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(AFLAGS)) $< -o $@
+	$(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
 
 -include $(DEPS_INCLUDE)
xen/arch/arm/Makefile +2 -2
index 7273f356f190..913f6cdeed3f 100644
@@ -120,10 +120,10 @@ $(TARGET)-syms: prelink.o xen.lds
 	rm -f $(@D)/.$(@F).[0-9]*
 
 asm-offsets.s: $(TARGET_SUBARCH)/asm-offsets.c
-	$(CC) $(filter-out -flto,$(CFLAGS)) -S -o $@ $<
+	$(CC) $(filter-out -flto,$(c_flags)) -S -o $@ $<
 
 xen.lds: xen.lds.S
-	$(CC) -P -E -Ui386 $(AFLAGS) -o $@ $<
+	$(CC) -P -E -Ui386 $(a_flags) -o $@ $<
 	sed -e 's/xen\.lds\.o:/xen\.lds:/g' <.xen.lds.d >.xen.lds.d.new
 	mv -f .xen.lds.d.new .xen.lds.d
 
xen/arch/x86/Makefile +3 -3
index e954edbc2e0a..1405525105d9 100644
@@ -168,7 +168,7 @@ EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0
 EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0
 
 # Check if the compiler supports the MS ABI.
-export XEN_BUILD_EFI := $(shell $(CC) $(filter-out $(CFLAGS-y) .%.d,$(CFLAGS)) -c efi/check.c -o efi/check.o 2>/dev/null && echo y)
+export XEN_BUILD_EFI := $(shell $(CC) $(CFLAGS) -c efi/check.c -o efi/check.o 2>/dev/null && echo y)
 # Check if the linker supports PE.
 XEN_BUILD_PE := $(if $(XEN_BUILD_EFI),$(shell $(LD) -mi386pep --subsystem=10 -o efi/check.efi efi/check.o 2>/dev/null && echo y))
 CFLAGS-$(XEN_BUILD_EFI) += -DXEN_BUILD_EFI
@@ -223,7 +223,7 @@ efi/boot.init.o efi/runtime.o efi/compat.o efi/buildid.o efi/relocs-dummy.o: $(B
 efi/boot.init.o efi/runtime.o efi/compat.o efi/buildid.o efi/relocs-dummy.o: ;
 
 asm-offsets.s: $(TARGET_SUBARCH)/asm-offsets.c $(BASEDIR)/include/asm-x86/asm-macros.h
-	$(CC) $(filter-out -Wa$(comma)% -flto,$(CFLAGS)) -S -o $@ $<
+	$(CC) $(filter-out -Wa$(comma)% -flto,$(c_flags)) -S -o $@ $<
 
 asm-macros.i: CFLAGS += -D__ASSEMBLY__ -P
 
@@ -240,7 +240,7 @@ $(BASEDIR)/include/asm-x86/asm-macros.h: asm-macros.i Makefile
 
 efi.lds: AFLAGS += -DEFI
 xen.lds efi.lds: xen.lds.S
-	$(CC) -P -E -Ui386 $(filter-out -Wa$(comma)%,$(AFLAGS)) -o $@ $<
+	$(CC) -P -E -Ui386 $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<
 	sed -e 's/.*\.lds\.o:/$(@F):/g' <.$(@F).d >.$(@F).d.new
 	mv -f .$(@F).d.new .$(@F).d
 
xen/arch/x86/mm/Makefile +3 -3
index d87dc0aa6eeb..a2431fde6bb4 100644
@@ -12,10 +12,10 @@ obj-$(CONFIG_HVM) += p2m-ept.o p2m-pod.o
 obj-y += paging.o
 
 guest_walk_%.o: guest_walk.c Makefile
-	$(CC) $(CFLAGS) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
 
 guest_walk_%.i: guest_walk.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(CFLAGS)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
 
 guest_walk_%.s: guest_walk.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(CFLAGS)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
+	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
xen/arch/x86/mm/hap/Makefile +3 -3
index b14a9aff93d2..22e7ad54bd33 100644
@@ -6,10 +6,10 @@ obj-y += nested_hap.o
 obj-y += nested_ept.o
 
 guest_walk_%level.o: guest_walk.c Makefile
-	$(CC) $(CFLAGS) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
 
 guest_walk_%level.i: guest_walk.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(CFLAGS)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
 
 guest_walk_%level.s: guest_walk.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(CFLAGS)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
+	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
xen/arch/x86/mm/shadow/Makefile +3 -3
index ff03a9937f9b..23d3ff10802c 100644
@@ -7,10 +7,10 @@ obj-y += none.o
 endif
 
 guest_%.o: multi.c Makefile
-	$(CC) $(CFLAGS) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
 
 guest_%.i: multi.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(CFLAGS)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
 
 guest_%.s: multi.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(CFLAGS)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
+	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
xen/include/Makefile +1 -1
index 433bad9055b2..a488a98d8bb7 100644
@@ -64,7 +64,7 @@ compat/%.h: compat/%.i Makefile $(BASEDIR)/tools/compat-build-header.py
 	mv -f [email protected] $@
 
 compat/%.i: compat/%.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)% -M% %.d -include %/include/xen/config.h,$(CFLAGS)) $(cppflags-y) -o $@ $<
+	$(CPP) $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(CFLAGS)) $(cppflags-y) -o $@ $<
 
 compat/%.c: public/%.h xlat.lst Makefile $(BASEDIR)/tools/compat-build-source.py
 	mkdir -p $(@D)
Signature · 1 line
Anthony PERARD
Reviewed-by on v4 5/18
#10 lore raw

Re: [XEN PATCH v4 05/18] xen/build: use new $(c_flags) and $(a_flags) instead of $(CFLAGS)

Quoted 19 lines · Anthony PERARD

In a later patch ("xen/build: have the root Makefile generates the CFLAGS), we want to generate the CFLAGS in xen/Makefile, then export it and have Rules.mk use a CFLAGS from the environment variables. That changes the flavor of the CFLAGS and flags intended for one target (like -D__OBJECT_FILE__ and -M%) gets propagated and duplicated. So we start by moving such flags out of $(CFLAGS) and into $(c_flags) which is to be modified by only Rules.mk.

__OBJECT_FILE__ is only used by arch/x86/mm/*.c files, so having it in $(c_flags) is enough, we don't need it in $(a_flags).

For include/Makefile and as-insn we can keep using CFLAGS, but since it doesn't have -M* flags anymore there is no need to filter them out.

The XEN_BUILD_EFI tests in arch/x86/Makefile was filtering out CFLAGS-y, but according to dd40177c1bc8 ("x86-64/EFI: add CFLAGS to check compile"), it was done to filter out -MF. CFLAGS doesn't have those flags anymore, so no filtering is needed.

Quoted 4 lines

This is inspired by the way Kbuild generates CFLAGS for each targets.

#11 lore raw

PATCH v4 6/18 xen/build: have the root Makefile generates the CFLAGS

In reply to Anthony PERARD

Instead of generating the CFLAGS in Rules.mk everytime we enter a new subdirectory, we are going to generate most of them a single time, and export the result in the environment so that Rules.mk can use it. The only flags left to be generated are the ones that depend on the targets, but the variable $(c_flags) takes care of that.

Arch specific CFLAGS are generated by a new file "arch/*/arch.mk" which is included by the root Makefile.

We export the *FLAGS via the environment variables XEN_*FLAGS because Rules.mk still includes Config.mk and would add duplicated flags to CFLAGS.

When running Rules.mk in the root directory (xen/), the variable root-make-done' is set, so need-config' will remain undef and so the root Makefile will not generate the cflags again.

We can't use CFLAGS in subdirectories to add flags to particular targets, instead start to use CFLAGS-y. Idem for AFLAGS.
So there are two different CFLAGS-y, the one in xen/Makefile (and arch.mk), and the one in subdirs that Rules.mk is going to use. We can't add to XEN_CFLAGS because it is exported, so making change to it might be propagated to subdirectory which isn't intended.

Some style change are introduced in this patch:
when LDFLAGS_DIRECT is included in LDFLAGS
use of CFLAGS-$(CONFIG_INDIRECT_THUNK) instead of ifeq().

There is on FIXME added about LTO build, but since LTO is marked as BROKEN, this commit doesn't attempt to filter -flto flags out of the CFLAGS.

---

Notes:
v4:
- typos
- Adding $(AFLAGS-y) to $(AFLAGS)

v3:
- squash "xen/build: introduce ccflags-y and CFLAGS_$@" here, with those changes:
- rename ccflags-y to simply CFLAGS-y and start using AFLAGS-y in subdirs.
- remove CFLAGS_$@, we don't need it yet.
- fix build of xen.lds and efi.lds which needed -D to be a_flags
- remove arch_ccflags, and modify c_flags directly
with that change, reorder c_flags, so that target specific flags are last.
- remove HAVE_AS_QUOTED_SYM from envvar and check XEN_CFLAGS to find if it's there when adding -D__OBJECT_LABEL__.
- fix missing some flags in AFLAGS
(like -fshort-wchar in xen/arch/x86/efi/Makefile,
and -D__OBJECT_LABEL__ and CFLAGS-stack-boundary)
- keep COV_FLAGS generation in Rules.mk since it doesn't invovle to call CC
- fix clang test for "asm()-s support .include." (in a new patch done ahead)
- include Kconfig.include in xen/Makefile because as-option-add is defined there now.

 xen/Makefile                       | 58 +++++++++++++++++++
 xen/Rules.mk                       | 74 +++++++-----------------
 xen/arch/arm/Makefile              | 10 ++--
 xen/arch/arm/Rules.mk              | 23 --------
 xen/arch/arm/{Rules.mk => arch.mk} |  5 --
 xen/arch/arm/efi/Makefile          |  2 +-
 xen/arch/x86/Makefile              | 24 ++++----
 xen/arch/x86/Rules.mk              | 91 ++----------------------------
 xen/arch/x86/{Rules.mk => arch.mk} | 17 ++----
 xen/arch/x86/efi/Makefile          |  2 +-
 xen/common/libelf/Makefile         |  4 +-
 xen/common/libfdt/Makefile         |  4 +-
 xen/include/Makefile               |  2 +-
 xen/xsm/flask/Makefile             |  2 +-
 xen/xsm/flask/ss/Makefile          |  2 +-
 15 files changed, 115 insertions(+), 205 deletions(-)
 copy xen/arch/arm/{Rules.mk => arch.mk} (85%)
 copy xen/arch/x86/{Rules.mk => arch.mk} (87%)
xen/Makefile +58 -0
index 8375070e0d41..372692841913 100644
@@ -115,6 +115,64 @@ $(KCONFIG_CONFIG):
 include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
 	$(MAKE) $(kconfig) syncconfig
 
+ifeq ($(CONFIG_DEBUG),y)
+CFLAGS += -O1
+else
+CFLAGS += -O2
+endif
+
+ifeq ($(CONFIG_FRAME_POINTER),y)
+CFLAGS += -fno-omit-frame-pointer
+else
+CFLAGS += -fomit-frame-pointer
+endif
+
+CFLAGS += -nostdinc -fno-builtin -fno-common
+CFLAGS += -Werror -Wredundant-decls -Wno-pointer-arith
+$(call cc-option-add,CFLAGS,CC,-Wvla)
+CFLAGS += -pipe -D__XEN__ -include $(BASEDIR)/include/xen/config.h
+CFLAGS-$(CONFIG_DEBUG_INFO) += -g
+
+ifneq ($(CONFIG_CC_IS_CLANG),y)
+# Clang doesn't understand this command line argument, and doesn't appear to
+# have an suitable alternative.  The resulting compiled binary does function,
+# but has an excessively large symbol table.
+CFLAGS += -Wa,--strip-local-absolute
+endif
+
+AFLAGS += -D__ASSEMBLY__
+
+CFLAGS += $(CFLAGS-y)
+# allow extra CFLAGS externally via EXTRA_CFLAGS_XEN_CORE
+CFLAGS += $(EXTRA_CFLAGS_XEN_CORE)
+
+# Most CFLAGS are safe for assembly files:
+#  -std=gnu{89,99} gets confused by #-prefixed end-of-line comments
+#  -flto makes no sense and annoys clang
+AFLAGS += $(filter-out -std=gnu% -flto,$(CFLAGS)) $(AFLAGS-y)
+
+# LDFLAGS are only passed directly to $(LD)
+LDFLAGS += $(LDFLAGS_DIRECT) $(LDFLAGS-y)
+
+ifeq ($(CONFIG_UBSAN),y)
+CFLAGS_UBSAN := -fsanitize=undefined
+else
+CFLAGS_UBSAN :=
+endif
+
+ifeq ($(CONFIG_LTO),y)
+CFLAGS += -flto
+LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
+endif
+
+include $(BASEDIR)/arch/$(TARGET_ARCH)/arch.mk
+
+# define new variables to avoid the ones defines in Config.mk
+export XEN_CFLAGS := $(CFLAGS)
+export XEN_AFLAGS := $(AFLAGS)
+export XEN_LDFLAGS := $(LDFLAGS)
+export CFLAGS_UBSAN
+
 endif # need-config
 
 .PHONY: build install uninstall clean distclean MAP
xen/Rules.mk +19 -55

128 lines not on this page · raw message

xen/arch/arm/Makefile +5 -5
index 913f6cdeed3f..9f1ab2335756 100644
@@ -95,24 +95,24 @@ prelink_lto.o: $(ALL_OBJS)
 
 # Link it with all the binary objects
 prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 else
 prelink.o: $(ALL_OBJS)
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 endif
 
 $(TARGET)-syms: prelink.o xen.lds
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o \
 	    $(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).0.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).1.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
 		| $(BASEDIR)/tools/symbols --xensyms --sysv --sort \
xen/arch/arm/Rules.mk +0 -23
index 3ad284aa71a4..e69de29bb2d1 100644
@@ -1,23 +0,0 @@
-########################################
-# arm-specific definitions
-
-#
-# If you change any of these configuration options then you must
-# 'make clean' before rebuilding.
-#
-
-CFLAGS += -I$(BASEDIR)/include
-
-$(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
-$(call cc-option-add,CFLAGS,CC,-Wnested-externs)
-
-# Prevent floating-point variables from creeping into Xen.
-CFLAGS-$(CONFIG_ARM_32) += -msoft-float
-CFLAGS-$(CONFIG_ARM_32) += -mcpu=cortex-a15
-
-CFLAGS-$(CONFIG_ARM_64) += -mcpu=generic
-CFLAGS-$(CONFIG_ARM_64) += -mgeneral-regs-only # No fp registers etc
-
-ifneq ($(filter command line environment,$(origin CONFIG_EARLY_PRINTK)),)
-    $(error You must use 'make menuconfig' to enable/disable early printk now)
-endif
xen/arch/arm/arch.mk renamed from xen/arch/arm/Rules.mk +0 -5
similarity index 85%
copy from xen/arch/arm/Rules.mk
copy to xen/arch/arm/arch.mk
index 3ad284aa71a4..c8186f58288d 100644
@@ -1,11 +1,6 @@
 ########################################
 # arm-specific definitions
 
-#
-# If you change any of these configuration options then you must
-# 'make clean' before rebuilding.
-#
-
 CFLAGS += -I$(BASEDIR)/include
 
 $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
xen/arch/arm/efi/Makefile +1 -1
index d34c9168914a..e3ff2c3f283c 100644
@@ -1,4 +1,4 @@
-CFLAGS += -fshort-wchar
+CFLAGS-y += -fshort-wchar
 
 obj-y +=  boot.init.o runtime.o
 obj-$(CONFIG_ACPI) +=  efi-dom0.init.o
xen/arch/x86/Makefile +12 -12
index 1405525105d9..a805e9982e85 100644
@@ -121,32 +121,32 @@ prelink-efi_lto.o: $(ALL_OBJS) efi/runtime.o efi/compat.o
 
 # Link it with all the binary objects
 prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 
 prelink-efi.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink-efi_lto.o efi/boot.init.o
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 else
 prelink.o: $(ALL_OBJS)
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 
 prelink-efi.o: $(ALL_OBJS) efi/boot.init.o efi/runtime.o efi/compat.o
-	$(LD) $(LDFLAGS) -r -o $@ $(filter-out %/efi/built_in.o,$^)
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out %/efi/built_in.o,$^)
 endif
 
 $(TARGET)-syms: prelink.o xen.lds
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort \
 		>$(@D)/.$(@F).0.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort $(syms-warn-dup-y) \
 		>$(@D)/.$(@F).1.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
 		| $(BASEDIR)/tools/symbols --xensyms --sysv --sort \
@@ -159,7 +159,7 @@ note.o: $(TARGET)-syms
 		--rename-section=.data=.note.gnu.build-id -S [email protected] $@
 	rm -f [email protected]
 
-EFI_LDFLAGS = $(patsubst -m%,-mi386pep,$(LDFLAGS)) --subsystem=10
+EFI_LDFLAGS = $(patsubst -m%,-mi386pep,$(XEN_LDFLAGS)) --subsystem=10
 EFI_LDFLAGS += --image-base=$(1) --stack=0,0 --heap=0,0 --strip-debug
 EFI_LDFLAGS += --section-alignment=0x200000 --file-alignment=0x20
 EFI_LDFLAGS += --major-image-version=$(XEN_VERSION)
@@ -168,7 +168,7 @@ EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0
 EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0
 
 # Check if the compiler supports the MS ABI.
-export XEN_BUILD_EFI := $(shell $(CC) $(CFLAGS) -c efi/check.c -o efi/check.o 2>/dev/null && echo y)
+export XEN_BUILD_EFI := $(shell $(CC) $(XEN_CFLAGS) -c efi/check.c -o efi/check.o 2>/dev/null && echo y)
 # Check if the linker supports PE.
 XEN_BUILD_PE := $(if $(XEN_BUILD_EFI),$(shell $(LD) -mi386pep --subsystem=10 -o efi/check.efi efi/check.o 2>/dev/null && echo y))
 CFLAGS-$(XEN_BUILD_EFI) += -DXEN_BUILD_EFI
@@ -178,7 +178,7 @@ $(TARGET).efi: ALT_BASE = 0x$(shell $(NM) efi/relocs-dummy.o | sed -n 's, A ALT_
 
 ifneq ($(build_id_linker),)
 ifeq ($(call ld-ver-build-id,$(LD) $(filter -m%,$(EFI_LDFLAGS))),y)
-CFLAGS += -DBUILD_ID_EFI
+CFLAGS-y += -DBUILD_ID_EFI
 EFI_LDFLAGS += $(build_id_linker)
 note_file := efi/buildid.o
 # NB: this must be the last input in the linker call, because inputs following
@@ -225,7 +225,7 @@ efi/boot.init.o efi/runtime.o efi/compat.o efi/buildid.o efi/relocs-dummy.o: ;
 asm-offsets.s: $(TARGET_SUBARCH)/asm-offsets.c $(BASEDIR)/include/asm-x86/asm-macros.h
 	$(CC) $(filter-out -Wa$(comma)% -flto,$(c_flags)) -S -o $@ $<
 
-asm-macros.i: CFLAGS += -D__ASSEMBLY__ -P
+asm-macros.i: CFLAGS-y += -D__ASSEMBLY__ -P
 
 $(BASEDIR)/include/asm-x86/asm-macros.h: asm-macros.i Makefile
 	echo '#if 0' >[email protected]
@@ -238,7 +238,7 @@ $(BASEDIR)/include/asm-x86/asm-macros.h: asm-macros.i Makefile
 	echo '#endif' >>[email protected]
 	$(call move-if-changed,[email protected],$@)
 
-efi.lds: AFLAGS += -DEFI
+efi.lds: AFLAGS-y += -DEFI
 xen.lds efi.lds: xen.lds.S
 	$(CC) -P -E -Ui386 $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<
 	sed -e 's/.*\.lds\.o:/$(@F):/g' <.$(@F).d >.$(@F).d.new
xen/arch/x86/Rules.mk +6 -85

95 lines not on this page · raw message

xen/arch/x86/arch.mk renamed from xen/arch/x86/Rules.mk +6 -11
similarity index 87%
copy from xen/arch/x86/Rules.mk
copy to xen/arch/x86/arch.mk
index 4b7ab784670c..2a51553edb3c 100644
@@ -1,13 +1,12 @@
 ########################################
 # x86-specific definitions
 
-XEN_IMG_OFFSET := 0x200000
+export XEN_IMG_OFFSET := 0x200000
 
 CFLAGS += -I$(BASEDIR)/include
 CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-generic
 CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-default
 CFLAGS += -DXEN_IMG_OFFSET=$(XEN_IMG_OFFSET)
-CFLAGS += '-D__OBJECT_LABEL__=$(subst /,$$,$(subst -,_,$(subst $(BASEDIR)/,,$(CURDIR))/$@))'
 
 # Prevent floating-point variables from creeping into Xen.
 CFLAGS += -msoft-float
@@ -46,9 +45,7 @@ $(call as-option-add,CFLAGS,CC,"rdfsbase %rax",-DHAVE_AS_FSGSBASE)
 $(call as-option-add,CFLAGS,CC,"xsaveopt (%rax)",-DHAVE_AS_XSAVEOPT)
 $(call as-option-add,CFLAGS,CC,"rdseed %eax",-DHAVE_AS_RDSEED)
 $(call as-option-add,CFLAGS,CC,"clwb (%rax)",-DHAVE_AS_CLWB)
-$(call as-option-add,CFLAGS,CC,".equ \"x\"$$(comma)1", \
-                     -U__OBJECT_LABEL__ -DHAVE_AS_QUOTED_SYM \
-                     '-D__OBJECT_LABEL__=$(subst $(BASEDIR)/,,$(CURDIR))/$$@')
+$(call as-option-add,CFLAGS,CC,".equ \"x\"$$(comma)1",-DHAVE_AS_QUOTED_SYM)
 $(call as-option-add,CFLAGS,CC,"invpcid (%rax)$$(comma)%rax",-DHAVE_AS_INVPCID)
 
 # GAS's idea of true is -1.  Clang's idea is 1
@@ -66,15 +63,14 @@ CFLAGS += -mno-red-zone -fpic -fno-asynchronous-unwind-tables
 CFLAGS += -mno-sse $(call cc-option,$(CC),-mskip-rax-setup)
 
 # Compile with thunk-extern, indirect-branch-register if avaiable.
-ifeq ($(CONFIG_INDIRECT_THUNK),y)
-CFLAGS += -mindirect-branch=thunk-extern -mindirect-branch-register
-CFLAGS += -fno-jump-tables
-endif
+CFLAGS-$(CONFIG_INDIRECT_THUNK) += -mindirect-branch=thunk-extern
+CFLAGS-$(CONFIG_INDIRECT_THUNK) += -mindirect-branch-register
+CFLAGS-$(CONFIG_INDIRECT_THUNK) += -fno-jump-tables
 
 # If supported by the compiler, reduce stack alignment to 8 bytes. But allow
 # this to be overridden elsewhere.
 $(call cc-option-add,CFLAGS-stack-boundary,CC,-mpreferred-stack-boundary=3)
-CFLAGS += $(CFLAGS-stack-boundary)
+export CFLAGS-stack-boundary
 
 ifeq ($(CONFIG_UBSAN),y)
 # Don't enable alignment sanitisation.  x86 has efficient unaligned accesses,
@@ -86,4 +82,3 @@ endif
 
 # Set up the assembler include path properly for older toolchains.
 CFLAGS += -Wa,-I$(BASEDIR)/include
-
xen/arch/x86/efi/Makefile +1 -1
index 4bc0a196e9ca..490d791aae2d 100644
@@ -1,4 +1,4 @@
-CFLAGS += -fshort-wchar
+CFLAGS-y += -fshort-wchar
 
 %.o: %.ihex
 	$(OBJCOPY) -I ihex -O binary $< $@
xen/common/libelf/Makefile +2 -2
index 3d9e38f27e65..464c448d9d37 100644
@@ -3,10 +3,10 @@ nocov-y += libelf.o
 
 SECTIONS := text data $(SPECIAL_DATA_SECTIONS)
 
-CFLAGS += -Wno-pointer-sign
+CFLAGS-y += -Wno-pointer-sign
 
 libelf.o: libelf-temp.o Makefile
 	$(OBJCOPY) $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
 
 libelf-temp.o: libelf-tools.o libelf-loader.o libelf-dominfo.o #libelf-relocate.o
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
xen/common/libfdt/Makefile +2 -2
index c075bbf5462a..e2a5e59380a0 100644
@@ -5,10 +5,10 @@ SECTIONS := text data $(SPECIAL_DATA_SECTIONS)
 obj-y += libfdt.o
 nocov-y += libfdt.o
 
-CFLAGS += -I$(BASEDIR)/include/xen/libfdt/
+CFLAGS-y += -I$(BASEDIR)/include/xen/libfdt/
 
 libfdt.o: libfdt-temp.o Makefile
 	$(OBJCOPY) $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
 
 libfdt-temp.o: $(LIBFDT_OBJS)
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
xen/include/Makefile +1 -1
index a488a98d8bb7..2a10725d689b 100644
@@ -64,7 +64,7 @@ compat/%.h: compat/%.i Makefile $(BASEDIR)/tools/compat-build-header.py
 	mv -f [email protected] $@
 
 compat/%.i: compat/%.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(CFLAGS)) $(cppflags-y) -o $@ $<
+	$(CPP) $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(XEN_CFLAGS)) $(cppflags-y) -o $@ $<
 
 compat/%.c: public/%.h xlat.lst Makefile $(BASEDIR)/tools/compat-build-source.py
 	mkdir -p $(@D)
xen/xsm/flask/Makefile +1 -1
index b1fd45421993..011ef5ca91f8 100644
@@ -4,7 +4,7 @@ obj-y += flask_op.o
 
 obj-y += ss/
 
-CFLAGS += -I./include
+CFLAGS-y += -I./include
 
 AWK = awk
 
xen/xsm/flask/ss/Makefile +1 -1
index 046ce8f53326..d32b9e07138e 100644
@@ -8,4 +8,4 @@ obj-y += services.o
 obj-y += conditional.o
 obj-y += mls.o
 
-CFLAGS += -I../include
+CFLAGS-y += -I../include
Signature · 1 line
Anthony PERARD
#12 lore raw

Re: [XEN PATCH v4 06/18] xen/build: have the root Makefile generates the CFLAGS

Quoted 17 lines · Anthony PERARD
xen/Makefile +11 -0
@@ -115,6 +115,64 @@ $(KCONFIG_CONFIG):
 include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
 	$(MAKE) $(kconfig) syncconfig
 
+ifeq ($(CONFIG_DEBUG),y)
+CFLAGS += -O1
+else
+CFLAGS += -O2
+endif
+
+ifeq ($(CONFIG_FRAME_POINTER),y)
+CFLAGS += -fno-omit-frame-pointer
+else
+CFLAGS += -fomit-frame-pointer
+endif

xen/Makefile $(KCONFIG_CONFIG): · line 138

Quoted 10 lines
xen/Makefile +10 -0
+
+CFLAGS += -nostdinc -fno-builtin -fno-common
+CFLAGS += -Werror -Wredundant-decls -Wno-pointer-arith
+$(call cc-option-add,CFLAGS,CC,-Wvla)
+CFLAGS += -pipe -D__XEN__ -include $(BASEDIR)/include/xen/config.h
+CFLAGS-$(CONFIG_DEBUG_INFO) += -g
+
+ifneq ($(CONFIG_CC_IS_CLANG),y)
+# Clang doesn't understand this command line argument, and doesn't appear to
+# have an suitable alternative.  The resulting compiled binary does function,

I realize you only move this, but it would still be nice to adjust this to "... a suitable alternative" on this occasion.

Quoted 22 lines
xen/Makefile +22 -0
+# but has an excessively large symbol table.
+CFLAGS += -Wa,--strip-local-absolute
+endif
+
+AFLAGS += -D__ASSEMBLY__
+
+CFLAGS += $(CFLAGS-y)
+# allow extra CFLAGS externally via EXTRA_CFLAGS_XEN_CORE
+CFLAGS += $(EXTRA_CFLAGS_XEN_CORE)
+
+# Most CFLAGS are safe for assembly files:
+#  -std=gnu{89,99} gets confused by #-prefixed end-of-line comments
+#  -flto makes no sense and annoys clang
+AFLAGS += $(filter-out -std=gnu% -flto,$(CFLAGS)) $(AFLAGS-y)
+
+# LDFLAGS are only passed directly to $(LD)
+LDFLAGS += $(LDFLAGS_DIRECT) $(LDFLAGS-y)
+
+ifeq ($(CONFIG_UBSAN),y)
+CFLAGS_UBSAN := -fsanitize=undefined
+else
+CFLAGS_UBSAN :=

xen/Makefile $(KCONFIG_CONFIG): · line 170

Quoted 10 lines
xen/Makefile +10 -0
+endif
+
+ifeq ($(CONFIG_LTO),y)
+CFLAGS += -flto
+LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
+endif
+
+include $(BASEDIR)/arch/$(TARGET_ARCH)/arch.mk
+
+# define new variables to avoid the ones defines in Config.mk

s/defines/defined/

xen/Makefile obj-bin-y := · line 99

Quoted 7 lines
xen/Makefile +2 -1
@@ -140,10 +95,19 @@ obj-bin-y :=
 endif
 
 # Always build obj-bin files as binary even if they come from C source. 
-$(obj-bin-y): CFLAGS := $(filter-out -flto,$(CFLAGS))
+# FIXME LTO broken, but we would need a different way to filter -flto out
+# $(obj-bin-y): CFLAGS := $(filter-out -flto,$(CFLAGS))

While you mention this in the description, I'm still not overly happy with it getting commented out. What's wrong with making the construct simply act on c_flags?

Jan

#13 lore raw

Re: [XEN PATCH v4 06/18] xen/build: have the root Makefile generates the CFLAGS

Quoted 9 lines · Jan Beulich

xen/Rules.mk obj-bin-y := · line 99

Quoted 4 lines · Anthony PERARD

# Always build obj-bin files as binary even if they come from C source.
-$(obj-bin-y): CFLAGS := $(filter-out -flto,$(CFLAGS))

xen/Rules.mk +2 -0
+# FIXME LTO broken, but we would need a different way to filter -flto out
+# $(obj-bin-y): CFLAGS := $(filter-out -flto,$(CFLAGS))

While you mention this in the description, I'm still not overly happy with it getting commented out. What's wrong with making the construct simply act on c_flags?

It doesn't work.

I tried

    $(obj-bin-y): c_flags := $(filter-out -flto,$(c_flags))

but the $@ expansion was empty.

I guess we could do:

    $(obj-bin-y): XEN_CFLAGS := $(filter-out -flto,$(XEN_CFLAGS))

that's probably enough for now. Even if we can't test it properly.

Signature · 1 line
Anthony PERARD
#14 lore raw

Re: [XEN PATCH v4 06/18] xen/build: have the root Makefile generates the CFLAGS

Quoted 11 lines · Anthony PERARD
Quoted 9 lines · Jan Beulich

xen/Rules.mk obj-bin-y := · line 99

Quoted 4 lines · Anthony PERARD

# Always build obj-bin files as binary even if they come from C source.
-$(obj-bin-y): CFLAGS := $(filter-out -flto,$(CFLAGS))

xen/Rules.mk +2 -0
+# FIXME LTO broken, but we would need a different way to filter -flto out
+# $(obj-bin-y): CFLAGS := $(filter-out -flto,$(CFLAGS))

While you mention this in the description, I'm still not overly happy with it getting commented out. What's wrong with making the construct simply act on c_flags?

Quoted 5 lines

It doesn't work.

I tried

    $(obj-bin-y): c_flags := $(filter-out -flto,$(c_flags))

but the $@ expansion was empty.

Hmm, yes, presumably because of having to use :=. But the old code gives the appearance of having worked despite this fact.

Quoted 3 lines

I guess we could do:

    $(obj-bin-y): XEN_CFLAGS := $(filter-out -flto,$(XEN_CFLAGS))

that's probably enough for now. Even if we can't test it properly.

If -flto gets added to XEN_CFLAGS (not c_flags) - why not?

Jan

#15 lore raw

PATCH v4 7/18 build: Introduce documentation for xen Makefiles

In reply to Anthony PERARD

This start explainning the variables that can be used in the many Makefiles in xen/.

Most of the document copies and modifies text from Linux v5.4 document linux.git/Documentation/kbuild/makefiles.rst. Modification are mostly to avoid mentioning kbuild. Thus I've added the SPDX tag which was only in index.rst in linux.git.

---

Notes:
v4:
- new patch

 docs/misc/xen-makefiles/makefiles.rst | 87 +++++++++++++++++++++++++++
 xen/Rules.mk                          |  4 ++
 2 files changed, 91 insertions(+)
 create mode 100644 docs/misc/xen-makefiles/makefiles.rst
docs/misc/xen-makefiles/makefiles.rst new file +87 -0
new file mode 100644
index 000000000000..a86e3a612d61
@@ -0,0 +1,87 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=============
+Xen Makefiles
+=============
+
+Documentation for the build system of Xen, found in xen.git/xen/.
+
+Makefile files
+==============
+
+Description of the syntax that can be used in most Makefiles named
+'Makefile'. ('xen/Makefile' isn't part of the description.)
+
+'Makefile's are consumed by 'Rules.mk' when building.
+
+Goal definitions
+----------------
+
+	Goal definitions are the main part (heart) of the Makefile.
+	These lines define the files to be built, any special compilation
+	options, and any subdirectories to be entered recursively.
+
+	The most simple makefile contains one line:
+
+	Example::
+
+		obj-y += foo.o
+
+	This tells the build system that there is one object in that
+	directory, named foo.o. foo.o will be built from foo.c or foo.S.
+
+	The following pattern is often used to have object selected
+	depending on the configuration:
+
+	Example::
+
+		obj-$(CONFIG_FOO) += foo.o
+
+	$(CONFIG_FOO) can evaluates to y.
+	If CONFIG_FOO is not y, then the file will not be compiled nor linked.
+
+Descending down in directories
+------------------------------
+
+	A Makefile is only responsible for building objects in its own
+	directory. Files in subdirectories should be taken care of by
+	Makefiles in these subdirs. The build system will automatically
+	invoke make recursively in subdirectories, provided you let it know of
+	them.
+
+	To do so, obj-y is used.
+	acpi lives in a separate directory, and the Makefile present in
+	drivers/ tells the build system to descend down using the following
+	assignment.
+
+	Example::
+
+		#drivers/Makefile
+		obj-$(CONFIG_ACPI) += acpi/
+
+	If CONFIG_ACPI is set to 'y'
+	the corresponding obj- variable will be set, and the build system
+	will descend down in the apci directory.
+	The build system only uses this information to decide that it needs
+	to visit the directory, it is the Makefile in the subdirectory that
+	specifies what is modular and what is built-in.
+
+	It is good practice to use a `CONFIG_` variable when assigning directory
+	names. This allows the build system to totally skip the directory if the
+	corresponding `CONFIG_` option is 'y'.
+
+Compilation flags
+-----------------
+
+    CFLAGS-y and AFLAGS-y
+	These two flags apply only to the makefile in which they
+	are assigned. They are used for all the normal cc, as and ld
+	invocations happening during a recursive build.
+
+	$(CFLAGS-y) is necessary because the top Makefile owns the
+	variable $(XEN_CFLAGS) and uses it for compilation flags for the
+	entire tree. And the variable $(CFLAGS) is modified by Config.mk
+	which evaluated in every subdirs.
+
+	CFLAGS-y specifies options for compiling with $(CC).
+	AFLAGS-y specifies assembler options.
xen/Rules.mk +4 -0
index 0def40a00a09..7f28c3bc6c13 100644
@@ -1,3 +1,7 @@
+#
+# See docs/misc/xen-makefiles/makefiles.rst on variables that can be used in
+# Makefile and are consumed by Rules.mk
+#
 
 -include $(BASEDIR)/include/config/auto.conf
 
Signature · 1 line
Anthony PERARD
Acked-by on v4 7/18
#16 lore raw

Re: [XEN PATCH v4 07/18] build: Introduce documentation for xen Makefiles

Quoted 9 lines · Anthony PERARD

This start explainning the variables that can be used in the many Makefiles in xen/.

Most of the document copies and modifies text from Linux v5.4 document linux.git/Documentation/kbuild/makefiles.rst. Modification are mostly to avoid mentioning kbuild. Thus I've added the SPDX tag which was only in index.rst in linux.git.

with one question:

Quoted 5 lines
docs/misc/xen-makefiles/makefiles.rst new file +5 -0
+Compilation flags
+-----------------
+
+    CFLAGS-y and AFLAGS-y
+	These two flags apply only to the makefile in which they

docs/misc/xen-makefiles/makefiles.rst line 87

Quoted 10 lines
docs/misc/xen-makefiles/makefiles.rst new file +10 -0
+	are assigned. They are used for all the normal cc, as and ld
+	invocations happening during a recursive build.
+
+	$(CFLAGS-y) is necessary because the top Makefile owns the
+	variable $(XEN_CFLAGS) and uses it for compilation flags for the
+	entire tree. And the variable $(CFLAGS) is modified by Config.mk
+	which evaluated in every subdirs.
+
+	CFLAGS-y specifies options for compiling with $(CC).
+	AFLAGS-y specifies assembler options.

Is it perhaps worth mentioning that c_flags and a_flags should not be fiddled with by individual Makefile-s?

Jan

#17 lore raw

Re: [XEN PATCH v4 07/18] build: Introduce documentation for xen Makefiles

Quoted 24 lines · Jan Beulich
Quoted 9 lines · Anthony PERARD

This start explainning the variables that can be used in the many Makefiles in xen/.

Most of the document copies and modifies text from Linux v5.4 document linux.git/Documentation/kbuild/makefiles.rst. Modification are mostly to avoid mentioning kbuild. Thus I've added the SPDX tag which was only in index.rst in linux.git.

with one question:

Quoted 10 lines
docs/misc/xen-makefiles/makefiles.rst new file +10 -0
+Compilation flags
+-----------------
+
+    CFLAGS-y and AFLAGS-y
+	These two flags apply only to the makefile in which they
+	are assigned. They are used for all the normal cc, as and ld
+	invocations happening during a recursive build.
+
+	$(CFLAGS-y) is necessary because the top Makefile owns the
+	variable $(XEN_CFLAGS) and uses it for compilation flags for the
Quoted 8 lines

docs/misc/xen-makefiles/makefiles.rst line 87

Quoted 5 lines
docs/misc/xen-makefiles/makefiles.rst new file +5 -0
+	entire tree. And the variable $(CFLAGS) is modified by Config.mk
+	which evaluated in every subdirs.
+
+	CFLAGS-y specifies options for compiling with $(CC).
+	AFLAGS-y specifies assembler options.

Is it perhaps worth mentioning that c_flags and a_flags should not be fiddled with by individual Makefile-s?

No, I don't think that's needed. Outside of Rules.mk-s nothing modifies c_flags, so there isn't a good reason to modify CFLAGS via c_flags after looking for other examples.
If they do change c_flags anyway, I don't think they would have read that new documentation.

Thanks,

Signature · 1 line
Anthony PERARD
#18 lore raw

PATCH v4 8/18 xen/build: introduce if_changed and if_changed_rule

In reply to Anthony PERARD

The if_changed macro from Linux, in addition to check if any files needs an update, check if the command line has changed since the last invocation. The latter will force a rebuild if any options to the executable have changed.

if_changed_rule checks dependencies like if_changed, but execute rule_$(1) instead of cmd_$(1) when a target needs to be rebuilt. A rule_ macro can call more than one cmd_ macro. One of the cmd_ macro in a rule need to be call using a macro that record the command line, so cmd_and_record is introduced. It is similar to cmd_and_fixup from Linux but without a call to fixdep which we don't have yet. (We will later replace cmd_and_record by cmd_and_fixup.)

Example of a rule_ macro:
define rule_cc_o_c

    $(call cmd_and_record,cc_o_o)
    $(call cmd,objcopy)

endef

This needs one of the call to use cmd_and_record, otherwise no .*.cmd file will be created, and the target will keep been rebuilt.

In order for if_changed to works correctly, we need to load the .%.cmd files that the macro generates, this is done by adding targets in to the $(targets) variable. We use intermediate_targets to add %.init.o dependency %.o to target since there aren't in obj-y.

We also add $(MAKECMDGOALS) to targets so that when running for example make common/memory.i, make will load the associated .%.cmd dependency file.

Beside the if_changed*, we import the machinery used for a "beautify output". The important one is when running make with V=2 which help to debug the makefiles by printing why a target is been rebuilt, via the $(echo-why) macro.

if_changed and if_changed_rule aren't used yet.

Most of this code is copied from Linux v5.4, including the documentation.

---

Notes:
v4:
- Use := in make whenever possible (instead of =)
- insert new string in .gitignore somewhere more plausible.
- import documentation from Linux

 .gitignore                            |   1 +
 docs/misc/xen-makefiles/makefiles.rst |  99 ++++++++++++++++++++++++
 xen/Makefile                          |  53 ++++++++++++-
 xen/Rules.mk                          |  33 +++++++-
 xen/scripts/Kbuild.include            | 107 ++++++++++++++++++++++++++
 5 files changed, 291 insertions(+), 2 deletions(-)
.gitignore +1 -0
index 4ca679ddbc9a..bfa53723b38b 100644
@@ -1,4 +1,5 @@
 .hg
+.*.cmd
 .*.tmp
 *.orig
 *~
docs/misc/xen-makefiles/makefiles.rst +99 -0

102 lines not on this page · raw message

xen/Makefile +52 -1

66 lines not on this page · raw message

xen/Rules.mk +32 -1
index 7f28c3bc6c13..f531fd5e342d 100644
@@ -42,6 +42,7 @@ ALL_OBJS-y               += $(BASEDIR)/arch/$(TARGET_ARCH)/built_in.o
 ALL_OBJS-$(CONFIG_CRYPTO)   += $(BASEDIR)/crypto/built_in.o
 
 # Initialise some variables
+targets :=
 CFLAGS-y :=
 AFLAGS-y :=
 
@@ -69,6 +70,10 @@ $(foreach o,$(filter-out %/,$(obj-y) $(obj-bin-y) $(extra-y)),$(eval $(call gend
 subdir-y := $(subdir-y) $(filter %/, $(obj-y))
 obj-y    := $(patsubst %/, %/built_in.o, $(obj-y))
 
+# $(subdir-obj-y) is the list of objects in $(obj-y) which uses dir/ to
+# tell kbuild to descend
+subdir-obj-y := $(filter %/built_in.o, $(obj-y))
+
 $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS-y += -DINIT_SECTIONS_ONLY
 
 ifeq ($(CONFIG_COVERAGE),y)
@@ -124,6 +129,10 @@ else
 endif
 endif
 
+targets += built_in.o
+targets += $(filter-out $(subdir-obj-y), $(obj-y)) $(extra-y)
+targets += $(MAKECMDGOALS)
+
 built_in_bin.o: $(obj-bin-y) $(extra-y)
 ifeq ($(obj-bin-y),)
 	$(CC) $(a_flags) -c -x assembler /dev/null -o $@
@@ -132,7 +141,7 @@ else
 endif
 
 # Force execution of pattern rules (for which PHONY cannot be directly used).
-.PHONY: FORCE
+PHONY += FORCE
 FORCE:
 
 %/built_in.o: FORCE
@@ -180,4 +189,26 @@ $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o Makefile
 %.s: %.S Makefile
 	$(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
 
+# Add intermediate targets:
+# When building objects with specific suffix patterns, add intermediate
+# targets that the final targets are derived from.
+intermediate_targets = $(foreach sfx, $(2), \
+				$(patsubst %$(strip $(1)),%$(sfx), \
+					$(filter %$(strip $(1)), $(targets))))
+# %.init.o <- %.o
+targets += $(call intermediate_targets, .init.o, .o)
+
 -include $(DEPS_INCLUDE)
+
+# Read all saved command lines and dependencies for the $(targets) we
+# may be building above, using $(if_changed{,_dep}). As an
+# optimization, we don't need to read them if the target does not
+# exist, we will rebuild anyway in that case.
+
+existing-targets := $(wildcard $(sort $(targets)))
+
+-include $(foreach f,$(existing-targets),$(dir $(f)).$(notdir $(f)).cmd)
+
+# Declare the contents of the PHONY variable as phony.  We keep that
+# information in a variable so we can use it in if_changed and friends.
+.PHONY: $(PHONY)
xen/scripts/Kbuild.include +107 -0

121 lines not on this page · raw message

Signature · 1 line
Anthony PERARD
Acked-by on v4 8/18
#19 lore raw

Re: [XEN PATCH v4 08/18] xen/build: introduce if_changed and if_changed_rule

Quoted 36 lines · Anthony PERARD

The if_changed macro from Linux, in addition to check if any files needs an update, check if the command line has changed since the last invocation. The latter will force a rebuild if any options to the executable have changed.

if_changed_rule checks dependencies like if_changed, but execute rule_$(1) instead of cmd_$(1) when a target needs to be rebuilt. A rule_ macro can call more than one cmd_ macro. One of the cmd_ macro in a rule need to be call using a macro that record the command line, so cmd_and_record is introduced. It is similar to cmd_and_fixup from Linux but without a call to fixdep which we don't have yet. (We will later replace cmd_and_record by cmd_and_fixup.)

Example of a rule_ macro:
define rule_cc_o_c

    $(call cmd_and_record,cc_o_o)
    $(call cmd,objcopy)

endef

This needs one of the call to use cmd_and_record, otherwise no .*.cmd file will be created, and the target will keep been rebuilt.

In order for if_changed to works correctly, we need to load the .%.cmd files that the macro generates, this is done by adding targets in to the $(targets) variable. We use intermediate_targets to add %.init.o dependency %.o to target since there aren't in obj-y.

We also add $(MAKECMDGOALS) to targets so that when running for example make common/memory.i, make will load the associated .%.cmd dependency file.

Beside the if_changed*, we import the machinery used for a "beautify output". The important one is when running make with V=2 which help to debug the makefiles by printing why a target is been rebuilt, via the $(echo-why) macro.

Quoted 6 lines

if_changed and if_changed_rule aren't used yet.

Most of this code is copied from Linux v5.4, including the documentation.

with ...

Quoted 37 lines
docs/misc/xen-makefiles/makefiles.rst +31 -0
@@ -85,3 +85,102 @@ Compilation flags
 
 	CFLAGS-y specifies options for compiling with $(CC).
 	AFLAGS-y specifies assembler options.
+
+
+Build system infrastructure
+===========================
+
+This chapter describe some of the macro used when building Xen.
+
+Macros
+------
+
+
+    if_changed
+	if_changed is the infrastructure used for the following commands.
+
+	Usage::
+
+		target: source(s) FORCE
+			$(call if_changed,ld/objcopy/...)
+
+	When the rule is evaluated, it is checked to see if any files
+	need an update, or the command line has changed since the last
+	invocation. The latter will force a rebuild if any options
+	to the executable have changed.
+	Any target that utilises if_changed must be listed in $(targets),
+	otherwise the command line check will fail, and the target will
+	always be built.
+	if_changed may be used in conjunction with custom commands as
+	defined in "Custom commands".
+
+	Note: It is a typical mistake to forget the FORCE prerequisite.
+	Another common pitfall is that whitespace is sometimes

docs/misc/xen-makefiles/makefiles.rst Compilation flags · line 128

Quoted 10 lines
docs/misc/xen-makefiles/makefiles.rst +10 -0
+	significant; for instance, the below will fail (note the extra space
+	after the comma)::
+
+		target: source(s) FORCE
+
+	**WRONG!**	$(call if_changed, ld/objcopy/...)
+
+        Note:
+	      if_changed should not be used more than once per target.
+              It stores the executed command in a corresponding .cmd

... the odd mixing of tabs and spaces here taken care of (I didn't check if there are more) and ...

docs/misc/xen-makefiles/makefiles.rst Compilation flags · line 130

Quoted 2 lines
docs/misc/xen-makefiles/makefiles.rst +2 -0
+
+        file and multiple calls would result in overwrites and

... the apparently stray blank like here dropped.

Jan

#20 lore raw

PATCH v4 9/18 xen/build: Start using if_changed

In reply to Anthony PERARD

This patch start to use if_changed introduced in a previous commit.

Whenever if_changed is called, the target must have FORCE as dependency so that if_changed can check if the command line to be run has changed, so the macro $(real-prereqs) must be used to discover the dependencies without "FORCE".

Whenever a target isn't in obj-y, it should be added to extra-y so the .*.cmd dependency file associated with the target can be loaded. This is done for xsm/flask/ and both common/lib{elf,fdt}/ and arch/x86/Makefile.

For the targets that generate .*.d dependency files, there's going to be two dependency files (.*.d and .*.cmd) until we can merge them together in a later patch via fixdep from Linux.

One cleanup, libelf-relocate.o doesn't exist anymore.

We import cmd_ld and cmd_objcopy from Linux v5.4.

---

Notes:
v4:
- typos
- fix missing FORCE in libfdt/Makefile
- typo flask vs flash in xsm
- in xsm, use *_H_DEPEND in command lines of mkaccess and mkflask instead of $(real-prereq) to avoid including the script as argument of itself.

 xen/Rules.mk               | 68 +++++++++++++++++++++++++++-----------
 xen/arch/arm/Makefile      |  4 +--
 xen/arch/x86/Makefile      |  1 +
 xen/arch/x86/efi/Makefile  |  7 ++--
 xen/common/libelf/Makefile | 12 ++++---
 xen/common/libfdt/Makefile | 11 +++---
 xen/xsm/flask/Makefile     | 17 +++++++---
 7 files changed, 85 insertions(+), 35 deletions(-)
xen/Rules.mk +49 -19
index f531fd5e342d..5e668f5ba0d8 100644
@@ -56,6 +56,18 @@ SPECIAL_DATA_SECTIONS := rodata $(foreach a,1 2 4 8 16, \
 
 include Makefile
 
+# Linking
+# ---------------------------------------------------------------------------
+
+quiet_cmd_ld = LD      $@
+cmd_ld = $(LD) $(XEN_LDFLAGS) -r -o $@ $(real-prereqs)
+
+# Objcopy
+# ---------------------------------------------------------------------------
+
+quiet_cmd_objcopy = OBJCOPY $@
+cmd_objcopy = $(OBJCOPY) $(OBJCOPYFLAGS) $< $@
+
 define gendep
     ifneq ($(1),$(subst /,:,$(1)))
         DEPS += $(dir $(1)).$(notdir $(1)).d
@@ -165,29 +177,47 @@ else
 	$(CC) $(c_flags) -c $< -o $@
 endif
 
-%.o: %.S Makefile
-	$(CC) $(a_flags) -c $< -o $@
+quiet_cmd_cc_o_S = CC      $@
+cmd_cc_o_S = $(CC) $(a_flags) -c $< -o $@
+
+%.o: %.S FORCE
+	$(call if_changed,cc_o_S)
+
+
+quiet_cmd_obj_init_o = INIT_O  $@
+define cmd_obj_init_o
+    $(OBJDUMP) -h $< | sed -n '/[0-9]/{s,00*,0,g;p;}' | while read idx name sz rest; do \
+        case "$$name" in \
+        .*.local) ;; \
+        .text|.text.*|.data|.data.*|.bss) \
+            test $$sz != 0 || continue; \
+            echo "Error: size of $<:$$name is 0x$$sz" >&2; \
+            exit $$(expr $$idx + 1);; \
+        esac; \
+    done; \
+    $(OBJCOPY) $(foreach s,$(SPECIAL_DATA_SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
+endef
+
+$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o FORCE
+	$(call if_changed,obj_init_o)
+
+quiet_cmd_cpp_i_c = CPP     $@
+cmd_cpp_i_c = $(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) $< -o $@
+
+quiet_cmd_cc_s_c = CC      $@
+cmd_cc_s_c = $(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -S $< -o $@
 
-$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o Makefile
-	$(OBJDUMP) -h $< | sed -n '/[0-9]/{s,00*,0,g;p;}' | while read idx name sz rest; do \
-		case "$$name" in \
-		.*.local) ;; \
-		.text|.text.*|.data|.data.*|.bss) \
-			test $$sz != 0 || continue; \
-			echo "Error: size of $<:$$name is 0x$$sz" >&2; \
-			exit $$(expr $$idx + 1);; \
-		esac; \
-	done
-	$(OBJCOPY) $(foreach s,$(SPECIAL_DATA_SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
+quiet_cmd_s_S = CPP     $@
+cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
 
-%.i: %.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) $< -o $@
+%.i: %.c FORCE
+	$(call if_changed,cpp_i_c)
 
-%.s: %.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -S $< -o $@
+%.s: %.c FORCE
+	$(call if_changed,cc_s_c)
 
-%.s: %.S Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
+%.s: %.S FORCE
+	$(call if_changed,cpp_s_S)
 
 # Add intermediate targets:
 # When building objects with specific suffix patterns, add intermediate
xen/arch/arm/Makefile +2 -2
index 9f1ab2335756..b79ad55646bd 100644
@@ -97,8 +97,8 @@ prelink_lto.o: $(ALL_OBJS)
 prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o
 	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 else
-prelink.o: $(ALL_OBJS)
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
+prelink.o: $(ALL_OBJS) FORCE
+	$(call if_changed,ld)
 endif
 
 $(TARGET)-syms: prelink.o xen.lds
xen/arch/x86/Makefile +1 -0
index a805e9982e85..44137d919b66 100644
@@ -71,6 +71,7 @@ obj-$(CONFIG_TBOOT) += tboot.o
 obj-y += hpet.o
 obj-y += vm_event.o
 obj-y += xstate.o
+extra-y += asm-macros.i
 
 x86_emulate.o: x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h
 
xen/arch/x86/efi/Makefile +5 -2
index 490d791aae2d..3e4c395b7535 100644
@@ -1,7 +1,10 @@
 CFLAGS-y += -fshort-wchar
 
-%.o: %.ihex
-	$(OBJCOPY) -I ihex -O binary $< $@
+quiet_cmd_objcopy_o_ihex = OBJCOPY $@
+cmd_objcopy_o_ihex = $(OBJCOPY) -I ihex -O binary $< $@
+
+%.o: %.ihex FORCE
+	$(call if_changed,objcopy_o_ihex)
 
 boot.init.o: buildid.o
 
xen/common/libelf/Makefile +8 -4
index 464c448d9d37..a92326c982e9 100644
@@ -1,12 +1,16 @@
 obj-bin-y := libelf.o
 nocov-y += libelf.o
+libelf-objs := libelf-tools.o libelf-loader.o libelf-dominfo.o
 
 SECTIONS := text data $(SPECIAL_DATA_SECTIONS)
+OBJCOPYFLAGS := $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s))
 
 CFLAGS-y += -Wno-pointer-sign
 
-libelf.o: libelf-temp.o Makefile
-	$(OBJCOPY) $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
+libelf.o: libelf-temp.o FORCE
+	$(call if_changed,objcopy)
 
-libelf-temp.o: libelf-tools.o libelf-loader.o libelf-dominfo.o #libelf-relocate.o
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
+libelf-temp.o: $(libelf-objs) FORCE
+	$(call if_changed,ld)
+
+extra-y += libelf-temp.o $(libelf-objs)
xen/common/libfdt/Makefile +7 -4
index e2a5e59380a0..6bd207cf8ffa 100644
@@ -1,14 +1,17 @@
 include Makefile.libfdt
 
 SECTIONS := text data $(SPECIAL_DATA_SECTIONS)
+OBJCOPYFLAGS := $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s))
 
 obj-y += libfdt.o
 nocov-y += libfdt.o
 
 CFLAGS-y += -I$(BASEDIR)/include/xen/libfdt/
 
-libfdt.o: libfdt-temp.o Makefile
-	$(OBJCOPY) $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
+libfdt.o: libfdt-temp.o FORCE
+	$(call if_changed,objcopy)
 
-libfdt-temp.o: $(LIBFDT_OBJS)
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
+libfdt-temp.o: $(LIBFDT_OBJS) FORCE
+	$(call if_changed,ld)
+
+extra-y += libfdt-temp.o $(LIBFDT_OBJS)
xen/xsm/flask/Makefile +13 -4
index 011ef5ca91f8..7d0831e2b865 100644
@@ -20,12 +20,21 @@ AV_H_FILES = include/av_perm_to_string.h include/av_permissions.h
 ALL_H_FILES = $(FLASK_H_FILES) $(AV_H_FILES)
 
 $(obj-y) ss/built_in.o: $(ALL_H_FILES)
+extra-y += $(ALL_H_FILES)
 
-$(FLASK_H_FILES): $(FLASK_H_DEPEND)
-	$(CONFIG_SHELL) policy/mkflask.sh $(AWK) include $(FLASK_H_DEPEND)
+mkflask := policy/mkflask.sh
+quiet_cmd_mkflask = MKFLASK $@
+cmd_mkflask = $(CONFIG_SHELL) $(mkflask) $(AWK) include $(FLASK_H_DEPEND)
 
-$(AV_H_FILES): $(AV_H_DEPEND)
-	$(CONFIG_SHELL) policy/mkaccess_vector.sh $(AWK) $(AV_H_DEPEND)
+$(FLASK_H_FILES): $(FLASK_H_DEPEND) $(mkflask) FORCE
+	$(call if_changed,mkflask)
+
+mkaccess := policy/mkaccess_vector.sh
+quiet_cmd_mkaccess = MKACCESS VECTOR $@
+cmd_mkaccess = $(CONFIG_SHELL) $(mkaccess) $(AWK) $(AV_H_DEPEND)
+
+$(AV_H_FILES): $(AV_H_DEPEND) $(mkaccess) FORCE
+	$(call if_changed,mkaccess)
 
 obj-bin-$(CONFIG_XSM_FLASK_POLICY) += flask-policy.o
 flask-policy.o: policy.bin
Signature · 1 line
Anthony PERARD
#21 lore raw

PATCH v4 10/18 xen/build: use if_changed on built_in.o

In reply to Anthony PERARD

In the case where $(obj-y) is empty, we also replace $(c_flags) by $(XEN_CFLAGS) to avoid generating an .%.d dependency file. This avoid make trying to include %.h file in the ld command if $(obj-y) isn't empty anymore on a second run.

---

Notes:
v4:
- Have cmd_ld_builtin depends on CONFIG_LTO, which simplify built_in.o rule.

 xen/Rules.mk | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)
xen/Rules.mk +15 -6
index 5e668f5ba0d8..c744175fd6f0 100644
@@ -130,15 +130,24 @@ include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
 c_flags += $(CFLAGS-y)
 a_flags += $(CFLAGS-y) $(AFLAGS-y)
 
-built_in.o: $(obj-y) $(extra-y)
-ifeq ($(obj-y),)
-	$(CC) $(c_flags) -c -x c /dev/null -o $@
-else
+quiet_cmd_ld_builtin = LD      $@
 ifeq ($(CONFIG_LTO),y)
-	$(LD_LTO) -r -o $@ $(filter-out $(extra-y),$^)
+cmd_ld_builtin = \
+    $(LD_LTO) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
 else
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
+cmd_ld_builtin = \
+    $(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
 endif
+
+quiet_cmd_cc_builtin = LD      $@
+cmd_cc_builtin = \
+    $(CC) $(XEN_CFLAGS) -c -x c /dev/null -o $@
+
+built_in.o: $(obj-y) $(extra-y) FORCE
+ifeq ($(obj-y),)
+	$(call if_changed,cc_builtin)
+else
+	$(call if_changed,ld_builtin)
 endif
 
 targets += built_in.o
Signature · 1 line
Anthony PERARD
#22 lore raw

Re: [XEN PATCH v4 10/18] xen/build: use if_changed on built_in.o

Quoted 10 lines · Anthony PERARD
xen/Rules.mk +0 -4
@@ -130,15 +130,24 @@ include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
 c_flags += $(CFLAGS-y)
 a_flags += $(CFLAGS-y) $(AFLAGS-y)
 
-built_in.o: $(obj-y) $(extra-y)
-ifeq ($(obj-y),)
-	$(CC) $(c_flags) -c -x c /dev/null -o $@
-else

xen/Rules.mk include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk · line 140

Quoted 10 lines
xen/Rules.mk +5 -2
+quiet_cmd_ld_builtin = LD      $@
 ifeq ($(CONFIG_LTO),y)
-	$(LD_LTO) -r -o $@ $(filter-out $(extra-y),$^)
+cmd_ld_builtin = \
+    $(LD_LTO) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
 else
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
+cmd_ld_builtin = \
+    $(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
 endif

How about going yet one step further and doing away with the ifeq here altogether:

cmd_ld_builtin-y = \
    $(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
cmd_ld_builtin-$(CONFIG_LTO) = \
    $(LD_LTO) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))

xen/Rules.mk include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk · line 150

Quoted 9 lines
xen/Rules.mk +9 -0
+quiet_cmd_cc_builtin = LD      $@
+cmd_cc_builtin = \
+    $(CC) $(XEN_CFLAGS) -c -x c /dev/null -o $@
+
+built_in.o: $(obj-y) $(extra-y) FORCE
+ifeq ($(obj-y),)
+	$(call if_changed,cc_builtin)
+else
+	$(call if_changed,ld_builtin)
	$(call if_changed,ld_builtin-y)

?

As an aside (not something you introduce) this makes it even more prominent that the use of $(XEN_LDFLAGS) is asymmetric here, for whatever reason (if any). Of course there's other redundancy between the two commands which could be eliminated.

Jan

#23 lore raw

Re: [XEN PATCH v4 10/18] xen/build: use if_changed on built_in.o

Quoted 24 lines · Jan Beulich
Quoted 10 lines · Anthony PERARD
xen/Rules.mk +0 -4
@@ -130,15 +130,24 @@ include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
 c_flags += $(CFLAGS-y)
 a_flags += $(CFLAGS-y) $(AFLAGS-y)
 
-built_in.o: $(obj-y) $(extra-y)
-ifeq ($(obj-y),)
-	$(CC) $(c_flags) -c -x c /dev/null -o $@
-else

xen/Rules.mk include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk · line 140

Quoted 10 lines
xen/Rules.mk +5 -2
+quiet_cmd_ld_builtin = LD      $@
 ifeq ($(CONFIG_LTO),y)
-	$(LD_LTO) -r -o $@ $(filter-out $(extra-y),$^)
+cmd_ld_builtin = \
+    $(LD_LTO) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
 else
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
+cmd_ld_builtin = \
+    $(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
 endif

How about going yet one step further and doing away with the ifeq here altogether:

Quoted 4 lines
cmd_ld_builtin-y = \
    $(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
cmd_ld_builtin-$(CONFIG_LTO) = \
    $(LD_LTO) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))

Please don't.

Logic like this is substantially harder to follow than a plain if/else construct, and clarity is of far higher importance than optimising the line count in the build system.

This trick only works for trivial cases, and interferes with diff's when the logic inevitably becomes less trivial.  LTO support in particular needs a complete overhaul, but there is no way I'm going to touch that with a barge pole until this series is in place.

~Andrew

#24 lore raw

Re: [XEN PATCH v4 10/18] xen/build: use if_changed on built_in.o

Quoted 30 lines · Andrew Cooper
Quoted 24 lines · Jan Beulich
Quoted 10 lines · Anthony PERARD
xen/Rules.mk +0 -4
@@ -130,15 +130,24 @@ include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
 c_flags += $(CFLAGS-y)
 a_flags += $(CFLAGS-y) $(AFLAGS-y)
 
-built_in.o: $(obj-y) $(extra-y)
-ifeq ($(obj-y),)
-	$(CC) $(c_flags) -c -x c /dev/null -o $@
-else

xen/Rules.mk include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk · line 140

Quoted 10 lines
xen/Rules.mk +5 -2
+quiet_cmd_ld_builtin = LD      $@
 ifeq ($(CONFIG_LTO),y)
-	$(LD_LTO) -r -o $@ $(filter-out $(extra-y),$^)
+cmd_ld_builtin = \
+    $(LD_LTO) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
 else
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
+cmd_ld_builtin = \
+    $(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
 endif

How about going yet one step further and doing away with the ifeq here altogether:

Quoted 4 lines
cmd_ld_builtin-y = \
    $(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
cmd_ld_builtin-$(CONFIG_LTO) = \
    $(LD_LTO) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
Quoted 5 lines

Please don't.

Logic like this is substantially harder to follow than a plain if/else construct, and clarity is of far higher importance than optimising the line count in the build system.

I could maybe see the argument if the two definitions were far apart. This suggestion isn't about line count at all, but about clarity. In particular because of the need to use ifeq(,) rather than simple "if" constructs, I view this list model as the better alternative in all cases where it can be made use of.

Quoted 2 lines

This trick only works for trivial cases, and interferes with diff's when the logic inevitably becomes less trivial.

It may, but whether it actually will can't be known until such time as it would get touched. The suggested model may very well also be suitable then.

Anyway, Anthony, I'm not going to insist. This is just another aspect where we would better generally settle on the preferred style to use.

Jan

#25 lore raw

Re: [XEN PATCH v4 10/18] xen/build: use if_changed on built_in.o

Quoted 37 lines · Jan Beulich
Quoted 30 lines · Andrew Cooper
Quoted 24 lines · Jan Beulich
Quoted 10 lines · Anthony PERARD
xen/Rules.mk +0 -4
@@ -130,15 +130,24 @@ include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
 c_flags += $(CFLAGS-y)
 a_flags += $(CFLAGS-y) $(AFLAGS-y)
 
-built_in.o: $(obj-y) $(extra-y)
-ifeq ($(obj-y),)
-	$(CC) $(c_flags) -c -x c /dev/null -o $@
-else

xen/Rules.mk include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk · line 140

Quoted 10 lines
xen/Rules.mk +5 -2
+quiet_cmd_ld_builtin = LD      $@
 ifeq ($(CONFIG_LTO),y)
-	$(LD_LTO) -r -o $@ $(filter-out $(extra-y),$^)
+cmd_ld_builtin = \
+    $(LD_LTO) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
 else
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
+cmd_ld_builtin = \
+    $(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
 endif

How about going yet one step further and doing away with the ifeq here altogether:

Quoted 4 lines
cmd_ld_builtin-y = \
    $(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
cmd_ld_builtin-$(CONFIG_LTO) = \
    $(LD_LTO) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
Quoted 5 lines

Please don't.

Logic like this is substantially harder to follow than a plain if/else construct, and clarity is of far higher importance than optimising the line count in the build system.

Quoted 5 lines

I could maybe see the argument if the two definitions were far apart. This suggestion isn't about line count at all, but about clarity. In particular because of the need to use ifeq(,) rather than simple "if" constructs, I view this list model as the better alternative in all cases where it can be made use of.

We could use "ifdef CONFIG_LTO" to avoid ifeq ;-). But I think you disliked that because CONFIG_LTO could be present in the environment with a value different than "y" and mess up the build system, just to annoy us.

Quoted 9 lines
Quoted 2 lines

This trick only works for trivial cases, and interferes with diff's when the logic inevitably becomes less trivial.

It may, but whether it actually will can't be known until such time as it would get touched. The suggested model may very well also be suitable then.

Anyway, Anthony, I'm not going to insist. This is just another aspect where we would better generally settle on the preferred style to use.

I think if/else is better for alternatives. And we can keep "var-y" for lists with optional items.

Thanks,

Signature · 1 line
Anthony PERARD
#26 lore raw

PATCH v4 11/18 xen/build: Use if_changed_rules with %.o:%.c targets

In reply to Anthony PERARD

Use $(dot-target) to have the target name prefix with a dot.

Now, when the CC command has run, it is recorded in .*.cmd file, then if_changed_rules will compare it on subsequent runs.

---
 xen/Rules.mk | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)
xen/Rules.mk +17 -9
index c744175fd6f0..e126e4972dec 100644
@@ -173,19 +173,27 @@ FORCE:
 
 SRCPATH := $(patsubst $(BASEDIR)/%,%,$(CURDIR))
 
-%.o: %.c Makefile
+quiet_cmd_cc_o_c = CC      $@
 ifeq ($(CONFIG_ENFORCE_UNIQUE_SYMBOLS),y)
-	$(CC) $(c_flags) -c $< -o $(@D)/.$(@F).tmp -MQ $@
-ifeq ($(CONFIG_CC_IS_CLANG),y)
-	$(OBJCOPY) --redefine-sym $<=$(SRCPATH)/$< $(@D)/.$(@F).tmp $@
-else
-	$(OBJCOPY) --redefine-sym $(<F)=$(SRCPATH)/$< $(@D)/.$(@F).tmp $@
-endif
-	rm -f $(@D)/.$(@F).tmp
+    cmd_cc_o_c = $(CC) $(c_flags) -c $< -o $(dot-target).tmp -MQ $@
+    ifeq ($(CONFIG_CC_IS_CLANG),y)
+        cmd_objcopy_fix_sym = $(OBJCOPY) --redefine-sym $<=$(SRCPATH)/$< $(dot-target).tmp $@
+    else
+        cmd_objcopy_fix_sym = $(OBJCOPY) --redefine-sym $(<F)=$(SRCPATH)/$< $(dot-target).tmp $@
+    endif
+    cmd_objcopy_fix_sym += && rm -f $(dot-target).tmp
 else
-	$(CC) $(c_flags) -c $< -o $@
+    cmd_cc_o_c = $(CC) $(c_flags) -c $< -o $@
 endif
 
+define rule_cc_o_c
+    $(call cmd_and_record,cc_o_c)
+    $(call cmd,objcopy_fix_sym)
+endef
+
+%.o: %.c FORCE
+	$(call if_changed_rule,cc_o_c)
+
 quiet_cmd_cc_o_S = CC      $@
 cmd_cc_o_S = $(CC) $(a_flags) -c $< -o $@
 
Signature · 1 line
Anthony PERARD
#27 lore raw

PATCH v4 12/18 xen/build: factorise generation of the linker scripts

In reply to Anthony PERARD

In Arm and X86 makefile, generating the linker script is the same, so we can simply have both call the same macro.

We need to add *.lds files into extra-y so that Rules.mk can find the .*.cmd dependency file and load it.

Change made to the command line:
- Use of $(CPP) instead of $(CC) -E
- Remove -Ui386.
We don't compile Xen for i386 anymore, so that macro is never defined. Also it doesn't make sense on Arm.

---

Notes:
v4:
- fix rebuild by adding FORCE as dependency
- Use $(CPP)
- remove -Ui386
- avoid using "define" for cmd_cc_lds_S, as adding '; \' on each line is still mandatory for if_changed (or cmd) macro to work.

 xen/Rules.mk          | 6 ++++++
 xen/arch/arm/Makefile | 8 ++++----
 xen/arch/x86/Makefile | 8 ++++----
 3 files changed, 14 insertions(+), 8 deletions(-)
xen/Rules.mk +6 -0
index e126e4972dec..616c6ae179d8 100644
@@ -236,6 +236,12 @@ cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
 %.s: %.S FORCE
 	$(call if_changed,cpp_s_S)
 
+# Linker scripts, .lds.S -> .lds
+quiet_cmd_cc_lds_S = LDS     $@
+cmd_cc_lds_S = $(CPP) -P $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<; \
+    sed -e 's/.*\.lds\.o:/$(@F):/g' <$(dot-target).d >$(dot-target).d.new; \
+    mv -f $(dot-target).d.new $(dot-target).d
+
 # Add intermediate targets:
 # When building objects with specific suffix patterns, add intermediate
 # targets that the final targets are derived from.
xen/arch/arm/Makefile +4 -4
index b79ad55646bd..45484d6d11b2 100644
@@ -64,6 +64,8 @@ obj-y += vpsci.o
 obj-y += vuart.o
 extra-y += $(TARGET_SUBARCH)/head.o
 
+extra-y += xen.lds
+
 #obj-bin-y += ....o
 
 ifdef CONFIG_DTB_FILE
@@ -122,10 +124,8 @@ $(TARGET)-syms: prelink.o xen.lds
 asm-offsets.s: $(TARGET_SUBARCH)/asm-offsets.c
 	$(CC) $(filter-out -flto,$(c_flags)) -S -o $@ $<
 
-xen.lds: xen.lds.S
-	$(CC) -P -E -Ui386 $(a_flags) -o $@ $<
-	sed -e 's/xen\.lds\.o:/xen\.lds:/g' <.xen.lds.d >.xen.lds.d.new
-	mv -f .xen.lds.d.new .xen.lds.d
+xen.lds: xen.lds.S FORCE
+	$(call if_changed,cc_lds_S)
 
 dtb.o: $(CONFIG_DTB_FILE)
 
xen/arch/x86/Makefile +4 -4
index 44137d919b66..eb6f7a6aceca 100644
@@ -72,6 +72,7 @@ obj-y += hpet.o
 obj-y += vm_event.o
 obj-y += xstate.o
 extra-y += asm-macros.i
+extra-y += xen.lds
 
 x86_emulate.o: x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h
 
@@ -173,6 +174,7 @@ export XEN_BUILD_EFI := $(shell $(CC) $(XEN_CFLAGS) -c efi/check.c -o efi/check.
 # Check if the linker supports PE.
 XEN_BUILD_PE := $(if $(XEN_BUILD_EFI),$(shell $(LD) -mi386pep --subsystem=10 -o efi/check.efi efi/check.o 2>/dev/null && echo y))
 CFLAGS-$(XEN_BUILD_EFI) += -DXEN_BUILD_EFI
+extra-$(XEN_BUILD_PE) += efi.lds
 
 $(TARGET).efi: VIRT_BASE = 0x$(shell $(NM) efi/relocs-dummy.o | sed -n 's, A VIRT_START$$,,p')
 $(TARGET).efi: ALT_BASE = 0x$(shell $(NM) efi/relocs-dummy.o | sed -n 's, A ALT_START$$,,p')
@@ -240,10 +242,8 @@ $(BASEDIR)/include/asm-x86/asm-macros.h: asm-macros.i Makefile
 	$(call move-if-changed,[email protected],$@)
 
 efi.lds: AFLAGS-y += -DEFI
-xen.lds efi.lds: xen.lds.S
-	$(CC) -P -E -Ui386 $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<
-	sed -e 's/.*\.lds\.o:/$(@F):/g' <.$(@F).d >.$(@F).d.new
-	mv -f .$(@F).d.new .$(@F).d
+xen.lds efi.lds: xen.lds.S FORCE
+	$(call if_changed,cc_lds_S)
 
 boot/mkelf32: boot/mkelf32.c
 	$(HOSTCC) $(HOSTCFLAGS) -o $@ $<
Signature · 1 line
Anthony PERARD
#28 lore raw

Re: [XEN PATCH v4 12/18] xen/build: factorise generation of the linker scripts

Quoted 15 lines · Anthony PERARD

In Arm and X86 makefile, generating the linker script is the same, so we can simply have both call the same macro.

We need to add *.lds files into extra-y so that Rules.mk can find the .*.cmd dependency file and load it.

Change made to the command line:
- Use of $(CPP) instead of $(CC) -E
- Remove -Ui386.
We don't compile Xen for i386 anymore, so that macro is never defined. Also it doesn't make sense on Arm.

---

Quoted 7 lines

Notes:
v4:
- fix rebuild by adding FORCE as dependency
- Use $(CPP)
- remove -Ui386
- avoid using "define" for cmd_cc_lds_S, as adding '; \' on each line is still mandatory for if_changed (or cmd) macro to work.

I still don't believe in there being a need for "; \" there. This actually breaks things, after all:

xen/Rules.mk cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@ · line 243

Quoted 11 lines
xen/Rules.mk +5 -0
@@ -236,6 +236,12 @@ cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
 %.s: %.S FORCE
 	$(call if_changed,cpp_s_S)
 
+# Linker scripts, .lds.S -> .lds
+quiet_cmd_cc_lds_S = LDS     $@
+cmd_cc_lds_S = $(CPP) -P $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<; \
+    sed -e 's/.*\.lds\.o:/$(@F):/g' <$(dot-target).d >$(dot-target).d.new; \
+    mv -f $(dot-target).d.new $(dot-target).d

if $(CPP) or sed fail, previously the whole rule would have failed, which no longer is the case with your use of semicolons. There ought to be a solution to this, ideally one better than adding "set -e" as the first command ("define" would at least deal with the multi-line make issue, but without it being clear to me why the semicolons would be needed I don't think I can suggest anything there at the moment).

Jan

#29 lore raw

Re: [XEN PATCH v4 12/18] xen/build: factorise generation of the linker scripts

Quoted 13 lines · Jan Beulich
Quoted 2 lines · Anthony PERARD

- avoid using "define" for cmd_cc_lds_S, as adding '; \' on each line is still mandatory for if_changed (or cmd) macro to work.

I still don't believe in there being a need for "; \" there. This actually breaks things, after all:

Quoted 6 lines
xen/Rules.mk +0 -0
@@ -236,6 +236,12 @@ cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
 %.s: %.S FORCE
 	$(call if_changed,cpp_s_S)
 
Quoted 13 lines

xen/Rules.mk cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@ · line 243

Quoted 5 lines
xen/Rules.mk +5 -0
+# Linker scripts, .lds.S -> .lds
+quiet_cmd_cc_lds_S = LDS     $@
+cmd_cc_lds_S = $(CPP) -P $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<; \
+    sed -e 's/.*\.lds\.o:/$(@F):/g' <$(dot-target).d >$(dot-target).d.new; \
+    mv -f $(dot-target).d.new $(dot-target).d

if $(CPP) or sed fail, previously the whole rule would have failed, which no longer is the case with your use of semicolons. There ought to be a solution to this, ideally one better than adding "set -e" as the first command ("define" would at least deal with the multi-line make issue, but without it being clear to me why the semicolons would be needed I don't think I can suggest anything there at the moment).

The only macro that will consumes cmd_cc_lds_S (and other cmd_*) is "cmd", it is defined as:

    cmd = @set -e; $(echo-cmd) $(cmd_$(1))

So, "set -e" is already there, and using semicolons in commands is equivalent to using "&&".

With "cmd" alone, multi-line command would work as expected (unless $(echo-cmd) is is trying to print the command line).

It's "if_changed" macro that doesn't work with multi-line commands. It does:

    $(cmd); printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd

With a multiple line command, $(make-cmd) get's expanded to multiple line, so the second argument of "printf" is going to be spread over multiple line in make, and thus multiple shell. We run into this error:
/bin/sh: -c: line 0: unexpected EOF while looking for matching `'' /bin/sh: -c: line 1: syntax error: unexpected end of file

This is why we need to have commands on a single line.

I hope the explanation is clear enough.

Thanks,

Signature · 1 line
Anthony PERARD
#30 lore raw

Re: [XEN PATCH v4 12/18] xen/build: factorise generation of the linker scripts

Quoted 46 lines · Anthony PERARD
Quoted 13 lines · Jan Beulich
Quoted 2 lines · Anthony PERARD

- avoid using "define" for cmd_cc_lds_S, as adding '; \' on each line is still mandatory for if_changed (or cmd) macro to work.

I still don't believe in there being a need for "; \" there. This actually breaks things, after all:

Quoted 6 lines
xen/Rules.mk +0 -0
@@ -236,6 +236,12 @@ cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
 %.s: %.S FORCE
 	$(call if_changed,cpp_s_S)
 
Quoted 13 lines

xen/Rules.mk cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@ · line 243

Quoted 5 lines
xen/Rules.mk +5 -0
+# Linker scripts, .lds.S -> .lds
+quiet_cmd_cc_lds_S = LDS     $@
+cmd_cc_lds_S = $(CPP) -P $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<; \
+    sed -e 's/.*\.lds\.o:/$(@F):/g' <$(dot-target).d >$(dot-target).d.new; \
+    mv -f $(dot-target).d.new $(dot-target).d

if $(CPP) or sed fail, previously the whole rule would have failed, which no longer is the case with your use of semicolons. There ought to be a solution to this, ideally one better than adding "set -e" as the first command ("define" would at least deal with the multi-line make issue, but without it being clear to me why the semicolons would be needed I don't think I can suggest anything there at the moment).

The only macro that will consumes cmd_cc_lds_S (and other cmd_*) is "cmd", it is defined as:

    cmd = @set -e; $(echo-cmd) $(cmd_$(1))

So, "set -e" is already there, and using semicolons in commands is equivalent to using "&&".

With "cmd" alone, multi-line command would work as expected (unless $(echo-cmd) is is trying to print the command line).

It's "if_changed" macro that doesn't work with multi-line commands. It does:

    $(cmd); printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd

With a multiple line command, $(make-cmd) get's expanded to multiple line, so the second argument of "printf" is going to be spread over multiple line in make, and thus multiple shell. We run into this error:
/bin/sh: -c: line 0: unexpected EOF while looking for matching `'' /bin/sh: -c: line 1: syntax error: unexpected end of file

Quoted 3 lines

This is why we need to have commands on a single line.

I hope the explanation is clear enough.

Yes, thanks. One question remains though: Why do we need multiple commands here in the first place, when Linux gets away with one?

Two other remarks: For one the command's name, aiui, ought to be cmd_cpp_lds_S (see Linux). And there ought to be cpp_flags, which would then also be used by e.g. cmd_s_S (instead of both having $(filter-out -Wa$(comma)%,$(a_flags)) open-coded).

Jan

#31 lore raw

Re: [XEN PATCH v4 12/18] xen/build: factorise generation of the linker scripts

Quoted 47 lines · Jan Beulich
Quoted 46 lines · Anthony PERARD
Quoted 13 lines · Jan Beulich
Quoted 2 lines · Anthony PERARD

- avoid using "define" for cmd_cc_lds_S, as adding '; \' on each line is still mandatory for if_changed (or cmd) macro to work.

I still don't believe in there being a need for "; \" there. This actually breaks things, after all:

Quoted 6 lines
xen/Rules.mk +0 -0
@@ -236,6 +236,12 @@ cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
 %.s: %.S FORCE
 	$(call if_changed,cpp_s_S)
 
Quoted 13 lines

xen/Rules.mk cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@ · line 243

Quoted 5 lines
xen/Rules.mk +5 -0
+# Linker scripts, .lds.S -> .lds
+quiet_cmd_cc_lds_S = LDS     $@
+cmd_cc_lds_S = $(CPP) -P $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<; \
+    sed -e 's/.*\.lds\.o:/$(@F):/g' <$(dot-target).d >$(dot-target).d.new; \
+    mv -f $(dot-target).d.new $(dot-target).d

if $(CPP) or sed fail, previously the whole rule would have failed, which no longer is the case with your use of semicolons. There ought to be a solution to this, ideally one better than adding "set -e" as the first command ("define" would at least deal with the multi-line make issue, but without it being clear to me why the semicolons would be needed I don't think I can suggest anything there at the moment).

The only macro that will consumes cmd_cc_lds_S (and other cmd_*) is "cmd", it is defined as:

    cmd = @set -e; $(echo-cmd) $(cmd_$(1))

So, "set -e" is already there, and using semicolons in commands is equivalent to using "&&".

With "cmd" alone, multi-line command would work as expected (unless $(echo-cmd) is is trying to print the command line).

It's "if_changed" macro that doesn't work with multi-line commands. It does:

    $(cmd); printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd

With a multiple line command, $(make-cmd) get's expanded to multiple line, so the second argument of "printf" is going to be spread over multiple line in make, and thus multiple shell. We run into this error:
/bin/sh: -c: line 0: unexpected EOF while looking for matching `'' /bin/sh: -c: line 1: syntax error: unexpected end of file

Quoted 6 lines
Quoted 3 lines

This is why we need to have commands on a single line.

I hope the explanation is clear enough.

Yes, thanks. One question remains though: Why do we need multiple commands here in the first place, when Linux gets away with one?

Actually, Linux also has multiple commands as well. After running CPP, it runs ./fixdep (via if_change_dep) which does at least the same thing as our sed command. We can't use fixdep yet, but I'm working toward it.

Quoted 4 lines

Two other remarks: For one the command's name, aiui, ought to be cmd_cpp_lds_S (see Linux). And there ought to be cpp_flags, which would then also be used by e.g. cmd_s_S (instead of both having $(filter-out -Wa$(comma)%,$(a_flags)) open-coded).

When switching to use CPP instead of CC, I forgot to rename the command, so I'll fix that.

I'll look at introducing cpp_flags.

Thanks,

Signature · 1 line
Anthony PERARD
#32 lore raw

PATCH v4 13/18 xen/build: Use if_changed for prelink*.o

In reply to Anthony PERARD

We change the dependencies of prelink-efi.o so that we can use the same command line. The dependency on efi/built_in.o isn't needed because, we already have:

    efi/*.o: efi/built_in.o

to build efi/*.o files that prelink-efi.o needs.

---

Notes:
v4:
- fix rebuild, prelink.o and prelink-efi.o needs to be in targets

 xen/arch/x86/Makefile | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
xen/arch/x86/Makefile +6 -4
index eb6f7a6aceca..7676fb1c5bc8 100644
@@ -128,11 +128,13 @@ prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o
 prelink-efi.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink-efi_lto.o efi/boot.init.o
 	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 else
-prelink.o: $(ALL_OBJS)
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
+prelink.o: $(ALL_OBJS) FORCE
+	$(call if_changed,ld)
+
+prelink-efi.o: $(filter-out %/efi/built_in.o,$(ALL_OBJS)) efi/boot.init.o efi/runtime.o efi/compat.o FORCE
+	$(call if_changed,ld)
 
-prelink-efi.o: $(ALL_OBJS) efi/boot.init.o efi/runtime.o efi/compat.o
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out %/efi/built_in.o,$^)
+targets += prelink.o prelink-efi.o
 endif
 
 $(TARGET)-syms: prelink.o xen.lds
Signature · 1 line
Anthony PERARD
#33 lore raw

PATCH v4 14/18 xen,symbols: rework file symbols selection

In reply to Anthony PERARD

We want to use the same rune to build mm/*/guest_*.o as the one use to build every other *.o object. The consequence it that file symbols that the program ./symbols prefer changes with CONFIG_ENFORCE_UNIQUE_SYMBOLS=y.

(1) Currently we have those two file symbols:

    guest_walk.c
    guest_walk_2.o
(2) with CONFIG_ENFORCE_UNIQUE_SYMBOLS used on guest_walk.c, we will have:
    arch/x86/mm/guest_walk.c
    guest_walk_2.o

The order in which those symbols are present may be different.

Currently, in case (1) ./symbols chooses the *.o symbol (object file name). But in case (2), may choose the *.c symbol (source file name with path component) if it is first

We want to have ./symbols choose the object file name symbol in both cases. So this patch changes that ./symbols prefer the "object file name" symbol over the "source file name with path component" symbols.

The new intended order of preference is:
- first object file name symbol
- first source file name with path components symbol
- last source file name without any path component symbol

---

Notes:
v4:
- rescope enum symbol_type
- remove setting values to enums, as it's not needed.
- rename the enumeration symbols

commmit rewriting:

We want to use the same rune to build mm/*/guest_*.o as the one use to build every other *.o object. The consequence it that file symbols that the program ./symbols prefere changes with CONFIG_ENFORCE_UNIQUE_SYMBOLS=y.

(1) Currently we have those two file symboles:
guest_walk.c
guest_walk_2.o
(2) with CONFIG_ENFORCE_UNIQUE_SYMBOLS used on guest_walk.c, we will have:
arch/x86/mm/guest_walk.c
guest_walk_2.o

The order in which those symbols are present may be different.

Currently, in case (1) ./symbols chooses the *.o symbol (object file name). But in case (2), may choose the *.c symbol (source file name with path component) if it is first.

This patch changes that ./symbols prefere the "object file name" symbol over the "source file name with path component" symbols.

 xen/tools/symbols.c | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)
xen/tools/symbols.c +16 -4
index 9f9e2c990061..b3a9465b32d3 100644
@@ -84,7 +84,12 @@ static int read_symbol(FILE *in, struct sym_entry *s)
 {
 	char str[500], type[20] = "";
 	char *sym, stype;
-	static enum { symbol, single_source, multi_source } last;
+	static enum symbol_type {
+		symbol,
+		file_source,
+		path_source,
+		obj_file,
+	} last;
 	static char *filename;
 	int rc = -1;
 
@@ -125,13 +130,20 @@ static int read_symbol(FILE *in, struct sym_entry *s)
 		 * prefer the first one if that names an object file or has a
 		 * directory component (to cover multiply compiled files).
 		 */
-		bool multi = strchr(str, '/') || (sym && sym[1] == 'o');
+		enum symbol_type current;
 
-		if (multi || last != multi_source) {
+		if (sym && sym[1] == 'o')
+		    current = obj_file;
+		else if (strchr(str, '/'))
+		    current = path_source;
+		else
+		    current = file_source;
+
+		if (current > last || last == file_source) {
 			free(filename);
 			filename = *str ? strdup(str) : NULL;
+			last = current;
 		}
-		last = multi ? multi_source : single_source;
 		goto skip_tail;
 	}
 
Signature · 1 line
Anthony PERARD
#34 lore raw

Re: [XEN PATCH v4 14/18] xen,symbols: rework file symbols selection

Quoted 11 lines · Anthony PERARD

We want to use the same rune to build mm/*/guest_*.o as the one use to build every other *.o object. The consequence it that file symbols that the program ./symbols prefer changes with CONFIG_ENFORCE_UNIQUE_SYMBOLS=y.

(1) Currently we have those two file symbols:

    guest_walk.c
    guest_walk_2.o
(2) with CONFIG_ENFORCE_UNIQUE_SYMBOLS used on guest_walk.c, we will have:
    arch/x86/mm/guest_walk.c
    guest_walk_2.o
Quoted 8 lines

The order in which those symbols are present may be different.

Currently, in case (1) ./symbols chooses the *.o symbol (object file name). But in case (2), may choose the *.c symbol (source file name with path component) if it is first

We want to have ./symbols choose the object file name symbol in both cases.

I guess the reason for wanting this is somehow connected to the statement at the beginning of the description, but I can't seem to be able to make the connection.

Quoted 7 lines

So this patch changes that ./symbols prefer the "object file name" symbol over the "source file name with path component" symbols.

The new intended order of preference is:
- first object file name symbol
- first source file name with path components symbol
- last source file name without any path component symbol

Isn't this going to lead to ambiguities again when CONFIG_ENFORCE_UNIQUE_SYMBOLS? Several object files (in different dirs) are named the same, after all. Static symbols with the same name in such objects would hence resolve to the same kallsyms name.

Jan

#35 lore raw

Re: [XEN PATCH v4 14/18] xen,symbols: rework file symbols selection

Quoted 18 lines · Jan Beulich
Quoted 17 lines · Anthony PERARD

We want to use the same rune to build mm/*/guest_*.o as the one use to build every other *.o object. The consequence it that file symbols that the program ./symbols prefer changes with CONFIG_ENFORCE_UNIQUE_SYMBOLS=y.

(1) Currently we have those two file symbols:

    guest_walk.c
    guest_walk_2.o
(2) with CONFIG_ENFORCE_UNIQUE_SYMBOLS used on guest_walk.c, we will have:
    arch/x86/mm/guest_walk.c
    guest_walk_2.o

The order in which those symbols are present may be different.

Currently, in case (1) ./symbols chooses the *.o symbol (object file name). But in case (2), may choose the *.c symbol (source file name with path component) if it is first

Quoted 6 lines
Quoted 2 lines

We want to have ./symbols choose the object file name symbol in both cases.

I guess the reason for wanting this is somehow connected to the statement at the beginning of the description, but I can't seem to be able to make the connection.

I'm not sure I can explain it better.

The "object file name" file symbol is used to distinguish between symbols from all mm/*/guest_* objects. The other file symbol present in those object is a "source file name without any path component symbol".

But building those objects with the same rune as any other objects, and having CONFIG_ENFORCE_UNIQUE_SYMBOLS=y, changes the file symbols present in the resulting object. We still have the "object file name" symbol, but now we also have "source file name with path components" symbol. Unfortunately, all mm/*/guest_*.o in one directory are built from the same source file, and thus have the same "source file name" symbol, but have different "object file name" symbol. We still want to be able to distinguish between guest_*.o in one dir, and the only way for that is to use the "object file name" symbol.

Quoted 8 lines
Quoted 7 lines

So this patch changes that ./symbols prefer the "object file name" symbol over the "source file name with path component" symbols.

The new intended order of preference is:
- first object file name symbol
- first source file name with path components symbol
- last source file name without any path component symbol

Quoted 5 lines

Isn't this going to lead to ambiguities again when CONFIG_ENFORCE_UNIQUE_SYMBOLS? Several object files (in different dirs) are named the same, after all. Static symbols with the same name in such objects would hence resolve to the same kallsyms name.

"object file name" symbol are only present in mm/*/guest_*.o objects, they all have different basenames. There is no ambiguity here.

Thanks,

Signature · 1 line
Anthony PERARD
#36 lore raw

Re: [XEN PATCH v4 14/18] xen,symbols: rework file symbols selection

Quoted 32 lines · Anthony PERARD
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Anthony PERARD

We want to use the same rune to build mm/*/guest_*.o as the one use to build every other *.o object. The consequence it that file symbols that the program ./symbols prefer changes with CONFIG_ENFORCE_UNIQUE_SYMBOLS=y.

(1) Currently we have those two file symbols:

    guest_walk.c
    guest_walk_2.o
(2) with CONFIG_ENFORCE_UNIQUE_SYMBOLS used on guest_walk.c, we will have:
    arch/x86/mm/guest_walk.c
    guest_walk_2.o

The order in which those symbols are present may be different.

Currently, in case (1) ./symbols chooses the *.o symbol (object file name). But in case (2), may choose the *.c symbol (source file name with path component) if it is first

Quoted 6 lines
Quoted 2 lines

We want to have ./symbols choose the object file name symbol in both cases.

I guess the reason for wanting this is somehow connected to the statement at the beginning of the description, but I can't seem to be able to make the connection.

I'm not sure I can explain it better.

The "object file name" file symbol is used to distinguish between symbols from all mm/*/guest_* objects. The other file symbol present in those object is a "source file name without any path component symbol".

Quoted 9 lines

But building those objects with the same rune as any other objects, and having CONFIG_ENFORCE_UNIQUE_SYMBOLS=y, changes the file symbols present in the resulting object. We still have the "object file name" symbol, but now we also have "source file name with path components" symbol. Unfortunately, all mm/*/guest_*.o in one directory are built from the same source file, and thus have the same "source file name" symbol, but have different "object file name" symbol. We still want to be able to distinguish between guest_*.o in one dir, and the only way for that is to use the "object file name" symbol.

So where's the difference from how things work right now? The "same rune" aspect doesn't really change - right now we also build with effectively the same logic, just that -DGUEST_PAGING_LEVELS=... gets added. I guess it might help if you showed (for one particular example) how the set of file symbols changes from what we have now (with and without CONFIG_ENFORCE_UNIQUE_SYMBOLS=y) to what there would be with your changes to the symbols utility to what there will be with those changes.

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

So this patch changes that ./symbols prefer the "object file name" symbol over the "source file name with path component" symbols.

The new intended order of preference is:
- first object file name symbol
- first source file name with path components symbol
- last source file name without any path component symbol

Quoted 8 lines
Quoted 5 lines

Isn't this going to lead to ambiguities again when CONFIG_ENFORCE_UNIQUE_SYMBOLS? Several object files (in different dirs) are named the same, after all. Static symbols with the same name in such objects would hence resolve to the same kallsyms name.

"object file name" symbol are only present in mm/*/guest_*.o objects, they all have different basenames. There is no ambiguity here.

At least not right now, I see. Could you make this aspect more explicit by adding something like "(present only in object files produced from multiply compiled sources)" to the first bullet point?

Jan

#37 lore raw

Re: [XEN PATCH v4 14/18] xen,symbols: rework file symbols selection

Quoted 43 lines · Jan Beulich
Quoted 32 lines · Anthony PERARD
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Anthony PERARD

We want to use the same rune to build mm/*/guest_*.o as the one use to build every other *.o object. The consequence it that file symbols that the program ./symbols prefer changes with CONFIG_ENFORCE_UNIQUE_SYMBOLS=y.

(1) Currently we have those two file symbols:

    guest_walk.c
    guest_walk_2.o
(2) with CONFIG_ENFORCE_UNIQUE_SYMBOLS used on guest_walk.c, we will have:
    arch/x86/mm/guest_walk.c
    guest_walk_2.o

The order in which those symbols are present may be different.

Currently, in case (1) ./symbols chooses the *.o symbol (object file name). But in case (2), may choose the *.c symbol (source file name with path component) if it is first

Quoted 6 lines
Quoted 2 lines

We want to have ./symbols choose the object file name symbol in both cases.

I guess the reason for wanting this is somehow connected to the statement at the beginning of the description, but I can't seem to be able to make the connection.

I'm not sure I can explain it better.

The "object file name" file symbol is used to distinguish between symbols from all mm/*/guest_* objects. The other file symbol present in those object is a "source file name without any path component symbol".

Quoted 9 lines

But building those objects with the same rune as any other objects, and having CONFIG_ENFORCE_UNIQUE_SYMBOLS=y, changes the file symbols present in the resulting object. We still have the "object file name" symbol, but now we also have "source file name with path components" symbol. Unfortunately, all mm/*/guest_*.o in one directory are built from the same source file, and thus have the same "source file name" symbol, but have different "object file name" symbol. We still want to be able to distinguish between guest_*.o in one dir, and the only way for that is to use the "object file name" symbol.

Quoted 7 lines

So where's the difference from how things work right now? The "same rune" aspect doesn't really change - right now we also build with effectively the same logic, just that -DGUEST_PAGING_LEVELS=... gets added. I guess it might help if you showed (for one particular example) how the set of file symbols changes from what we have now (with and without CONFIG_ENFORCE_UNIQUE_SYMBOLS=y) to what there would be with your changes to the symbols utility to what there will be with those changes.

The logic to build objects from C files changed in 81ecb38b83b0 ("build:
provide option to disambiguate symbol names"), with objects build with __OBJECT_FILE__ explicitly left alone. So the logic is different now (at least when CONFIG_ENFORCE_UNIQUE_SYMBOLS=y).

I did add the example of building arch/x86/mm/guest_walk_2.o to the commit message, reworded below:

For example, when building arch/x86/mm/guest_walk_2.o from guest_walk.c, this would be the difference of file symbol present in the object when building with CONFIG_ENFORCE_UNIQUE_SYMBOLS=y:

(1) Currently we have those two file symbols:

    guest_walk.c
    guest_walk_2.o
(2) When building with the same rune, we will have:
    arch/x86/mm/guest_walk.c
    guest_walk_2.o

The order in which those symbols are present may be different. Building without CONFIG_ENFORCE_UNIQUE_SYMBOLS will result in (1).

Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 7 lines

So this patch changes that ./symbols prefer the "object file name" symbol over the "source file name with path component" symbols.

The new intended order of preference is:
- first object file name symbol
- first source file name with path components symbol
- last source file name without any path component symbol

Isn't this going to lead to ambiguities again when CONFIG_ENFORCE_UNIQUE_SYMBOLS? Several object files (in different dirs) are named the same, after all. Static symbols with the same name in such objects would hence resolve to the same kallsyms name.

Quoted 6 lines
Quoted 2 lines

"object file name" symbol are only present in mm/*/guest_*.o objects, they all have different basenames. There is no ambiguity here.

At least not right now, I see. Could you make this aspect more explicit by adding something like "(present only in object files produced from multiply compiled sources)" to the first bullet point?

Will do.

Signature · 1 line
Anthony PERARD
#38 lore raw

Re: [XEN PATCH v4 14/18] xen,symbols: rework file symbols selection

Quoted 64 lines · Anthony PERARD
Quoted 43 lines · Jan Beulich
Quoted 32 lines · Anthony PERARD
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Anthony PERARD

We want to use the same rune to build mm/*/guest_*.o as the one use to build every other *.o object. The consequence it that file symbols that the program ./symbols prefer changes with CONFIG_ENFORCE_UNIQUE_SYMBOLS=y.

(1) Currently we have those two file symbols:

    guest_walk.c
    guest_walk_2.o
(2) with CONFIG_ENFORCE_UNIQUE_SYMBOLS used on guest_walk.c, we will have:
    arch/x86/mm/guest_walk.c
    guest_walk_2.o

The order in which those symbols are present may be different.

Currently, in case (1) ./symbols chooses the *.o symbol (object file name). But in case (2), may choose the *.c symbol (source file name with path component) if it is first

Quoted 6 lines
Quoted 2 lines

We want to have ./symbols choose the object file name symbol in both cases.

I guess the reason for wanting this is somehow connected to the statement at the beginning of the description, but I can't seem to be able to make the connection.

I'm not sure I can explain it better.

The "object file name" file symbol is used to distinguish between symbols from all mm/*/guest_* objects. The other file symbol present in those object is a "source file name without any path component symbol".

Quoted 9 lines

But building those objects with the same rune as any other objects, and having CONFIG_ENFORCE_UNIQUE_SYMBOLS=y, changes the file symbols present in the resulting object. We still have the "object file name" symbol, but now we also have "source file name with path components" symbol. Unfortunately, all mm/*/guest_*.o in one directory are built from the same source file, and thus have the same "source file name" symbol, but have different "object file name" symbol. We still want to be able to distinguish between guest_*.o in one dir, and the only way for that is to use the "object file name" symbol.

Quoted 7 lines

So where's the difference from how things work right now? The "same rune" aspect doesn't really change - right now we also build with effectively the same logic, just that -DGUEST_PAGING_LEVELS=... gets added. I guess it might help if you showed (for one particular example) how the set of file symbols changes from what we have now (with and without CONFIG_ENFORCE_UNIQUE_SYMBOLS=y) to what there would be with your changes to the symbols utility to what there will be with those changes.

The logic to build objects from C files changed in 81ecb38b83b0 ("build:
provide option to disambiguate symbol names"), with objects build with __OBJECT_FILE__ explicitly left alone. So the logic is different now (at least when CONFIG_ENFORCE_UNIQUE_SYMBOLS=y).

I did add the example of building arch/x86/mm/guest_walk_2.o to the commit message, reworded below:

For example, when building arch/x86/mm/guest_walk_2.o from guest_walk.c, this would be the difference of file symbol present in the object when building with CONFIG_ENFORCE_UNIQUE_SYMBOLS=y:

Quoted 6 lines

(1) Currently we have those two file symbols:

    guest_walk.c
    guest_walk_2.o
(2) When building with the same rune, we will have:
    arch/x86/mm/guest_walk.c
    guest_walk_2.o

Ah, yes, the changed introductory paragraph makes clear (to me) what presence and what future (1) and (2) are talking about. Yet what I then still don't understand - what is it that makes the path appear when switching to the common rune? Oh - I finally figured it: It's the objcopy step that will apply to all targets uniformly then. Perhaps it's indeed obvious, but it clearly wasn't to me when merely looking at the patch.

With this I'd then wonder whether it wouldn't be a far smaller adjustment to simply skip that --redefine-sym step in case the object file already has a file symbol naming the object file, thus simply retaining the status quo.

Jan

#39 lore raw

Re: [XEN PATCH v4 14/18] xen,symbols: rework file symbols selection

Quoted 80 lines · Jan Beulich
Quoted 64 lines · Anthony PERARD
Quoted 43 lines · Jan Beulich
Quoted 32 lines · Anthony PERARD
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Anthony PERARD

We want to use the same rune to build mm/*/guest_*.o as the one use to build every other *.o object. The consequence it that file symbols that the program ./symbols prefer changes with CONFIG_ENFORCE_UNIQUE_SYMBOLS=y.

(1) Currently we have those two file symbols:

    guest_walk.c
    guest_walk_2.o
(2) with CONFIG_ENFORCE_UNIQUE_SYMBOLS used on guest_walk.c, we will have:
    arch/x86/mm/guest_walk.c
    guest_walk_2.o

The order in which those symbols are present may be different.

Currently, in case (1) ./symbols chooses the *.o symbol (object file name). But in case (2), may choose the *.c symbol (source file name with path component) if it is first

Quoted 6 lines
Quoted 2 lines

We want to have ./symbols choose the object file name symbol in both cases.

I guess the reason for wanting this is somehow connected to the statement at the beginning of the description, but I can't seem to be able to make the connection.

I'm not sure I can explain it better.

The "object file name" file symbol is used to distinguish between symbols from all mm/*/guest_* objects. The other file symbol present in those object is a "source file name without any path component symbol".

Quoted 9 lines

But building those objects with the same rune as any other objects, and having CONFIG_ENFORCE_UNIQUE_SYMBOLS=y, changes the file symbols present in the resulting object. We still have the "object file name" symbol, but now we also have "source file name with path components" symbol. Unfortunately, all mm/*/guest_*.o in one directory are built from the same source file, and thus have the same "source file name" symbol, but have different "object file name" symbol. We still want to be able to distinguish between guest_*.o in one dir, and the only way for that is to use the "object file name" symbol.

Quoted 7 lines

So where's the difference from how things work right now? The "same rune" aspect doesn't really change - right now we also build with effectively the same logic, just that -DGUEST_PAGING_LEVELS=... gets added. I guess it might help if you showed (for one particular example) how the set of file symbols changes from what we have now (with and without CONFIG_ENFORCE_UNIQUE_SYMBOLS=y) to what there would be with your changes to the symbols utility to what there will be with those changes.

The logic to build objects from C files changed in 81ecb38b83b0 ("build:
provide option to disambiguate symbol names"), with objects build with __OBJECT_FILE__ explicitly left alone. So the logic is different now (at least when CONFIG_ENFORCE_UNIQUE_SYMBOLS=y).

I did add the example of building arch/x86/mm/guest_walk_2.o to the commit message, reworded below:

For example, when building arch/x86/mm/guest_walk_2.o from guest_walk.c, this would be the difference of file symbol present in the object when building with CONFIG_ENFORCE_UNIQUE_SYMBOLS=y:

Quoted 6 lines

(1) Currently we have those two file symbols:

    guest_walk.c
    guest_walk_2.o
(2) When building with the same rune, we will have:
    arch/x86/mm/guest_walk.c
    guest_walk_2.o

Ah, yes, the changed introductory paragraph makes clear (to me) what presence and what future (1) and (2) are talking about. Yet what I then still don't understand - what is it that makes the path appear when switching to the common rune? Oh - I finally figured it: It's the objcopy step that will apply to all targets uniformly then. Perhaps it's indeed obvious, but it clearly wasn't to me when merely looking at the patch.

Quoted 4 lines

With this I'd then wonder whether it wouldn't be a far smaller adjustment to simply skip that --redefine-sym step in case the object file already has a file symbol naming the object file, thus simply retaining the status quo.

So, we should call nm' thousands of time, to find out if calling objcopy' is needed, for the 9 objects that doesn't need the extra step?

Or do you mean keeping exception to the rule? And hope that when someone changes the rule, it doesn't forget to check if the exception needs changing as well?

Also, I'm going to have to use this patch later anyway as sometime CC use a full path to the source as file symbol. So this is going to be important when we will run for example
clang -o arch/x86/mm/guest_walk_2.o arch/x86/mm/guest_walk.c. (There isn't a patch for that yet.)

Signature · 1 line
Anthony PERARD
#40 lore raw

Re: [XEN PATCH v4 14/18] xen,symbols: rework file symbols selection

Quoted 81 lines · Anthony PERARD
Quoted 80 lines · Jan Beulich
Quoted 64 lines · Anthony PERARD
Quoted 43 lines · Jan Beulich
Quoted 32 lines · Anthony PERARD
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Anthony PERARD

We want to use the same rune to build mm/*/guest_*.o as the one use to build every other *.o object. The consequence it that file symbols that the program ./symbols prefer changes with CONFIG_ENFORCE_UNIQUE_SYMBOLS=y.

(1) Currently we have those two file symbols:

    guest_walk.c
    guest_walk_2.o
(2) with CONFIG_ENFORCE_UNIQUE_SYMBOLS used on guest_walk.c, we will have:
    arch/x86/mm/guest_walk.c
    guest_walk_2.o

The order in which those symbols are present may be different.

Currently, in case (1) ./symbols chooses the *.o symbol (object file name). But in case (2), may choose the *.c symbol (source file name with path component) if it is first

Quoted 6 lines
Quoted 2 lines

We want to have ./symbols choose the object file name symbol in both cases.

I guess the reason for wanting this is somehow connected to the statement at the beginning of the description, but I can't seem to be able to make the connection.

I'm not sure I can explain it better.

The "object file name" file symbol is used to distinguish between symbols from all mm/*/guest_* objects. The other file symbol present in those object is a "source file name without any path component symbol".

Quoted 9 lines

But building those objects with the same rune as any other objects, and having CONFIG_ENFORCE_UNIQUE_SYMBOLS=y, changes the file symbols present in the resulting object. We still have the "object file name" symbol, but now we also have "source file name with path components" symbol. Unfortunately, all mm/*/guest_*.o in one directory are built from the same source file, and thus have the same "source file name" symbol, but have different "object file name" symbol. We still want to be able to distinguish between guest_*.o in one dir, and the only way for that is to use the "object file name" symbol.

Quoted 7 lines

So where's the difference from how things work right now? The "same rune" aspect doesn't really change - right now we also build with effectively the same logic, just that -DGUEST_PAGING_LEVELS=... gets added. I guess it might help if you showed (for one particular example) how the set of file symbols changes from what we have now (with and without CONFIG_ENFORCE_UNIQUE_SYMBOLS=y) to what there would be with your changes to the symbols utility to what there will be with those changes.

The logic to build objects from C files changed in 81ecb38b83b0 ("build:
provide option to disambiguate symbol names"), with objects build with __OBJECT_FILE__ explicitly left alone. So the logic is different now (at least when CONFIG_ENFORCE_UNIQUE_SYMBOLS=y).

I did add the example of building arch/x86/mm/guest_walk_2.o to the commit message, reworded below:

For example, when building arch/x86/mm/guest_walk_2.o from guest_walk.c, this would be the difference of file symbol present in the object when building with CONFIG_ENFORCE_UNIQUE_SYMBOLS=y:

Quoted 6 lines

(1) Currently we have those two file symbols:

    guest_walk.c
    guest_walk_2.o
(2) When building with the same rune, we will have:
    arch/x86/mm/guest_walk.c
    guest_walk_2.o

Ah, yes, the changed introductory paragraph makes clear (to me) what presence and what future (1) and (2) are talking about. Yet what I then still don't understand - what is it that makes the path appear when switching to the common rune? Oh - I finally figured it: It's the objcopy step that will apply to all targets uniformly then. Perhaps it's indeed obvious, but it clearly wasn't to me when merely looking at the patch.

Quoted 7 lines
Quoted 4 lines

With this I'd then wonder whether it wouldn't be a far smaller adjustment to simply skip that --redefine-sym step in case the object file already has a file symbol naming the object file, thus simply retaining the status quo.

So, we should call nm' thousands of time, to find out if calling objcopy' is needed, for the 9 objects that doesn't need the extra step?

Well that (or rather objdump) was what I was thinking while writing the earlier reply, but you have a point - I can see how treating the bigger change for less build time might be worth it. Yet ...

Quoted 3 lines

Or do you mean keeping exception to the rule? And hope that when someone changes the rule, it doesn't forget to check if the exception needs changing as well?

... "exception" like you put it (requiring special care to keep multiple instances in sync) is not the only way this can be done (and indeed I'd not want something like this). Since you have (in patch 15) e.g.

guest_walk_%.o: guest_walk.c FORCE

	$(call if_changed_rule,cc_o_c)

anyway, the desire to skip the objcopy step could be communicated to the command from here, without needing to clone the command. One way might be a special (phony) dependency, another might be to set some variable along the lines of

guest_walk_%.o: SPECIAL := y

Quoted 5 lines

Also, I'm going to have to use this patch later anyway as sometime CC use a full path to the source as file symbol. So this is going to be important when we will run for example
clang -o arch/x86/mm/guest_walk_2.o arch/x86/mm/guest_walk.c. (There isn't a patch for that yet.)

That's interesting - what will be the goal of that future adjustment?

Jan

#41 lore raw

Re: [XEN PATCH v4 14/18] xen,symbols: rework file symbols selection

Quoted 11 lines · Jan Beulich
Quoted 3 lines · Anthony PERARD

Or do you mean keeping exception to the rule? And hope that when someone changes the rule, it doesn't forget to check if the exception needs changing as well?

... "exception" like you put it (requiring special care to keep multiple instances in sync) is not the only way this can be done (and indeed I'd not want something like this). Since you have (in patch 15) e.g.

guest_walk_%.o: guest_walk.c FORCE

Quoted 8 lines
	$(call if_changed_rule,cc_o_c)

anyway, the desire to skip the objcopy step could be communicated to the command from here, without needing to clone the command. One way might be a special (phony) dependency, another might be to set some variable along the lines of

guest_walk_%.o: SPECIAL := y

I guess something like that could be done. But if possible, I'd like to avoid that.

Quoted 7 lines
Quoted 5 lines

Also, I'm going to have to use this patch later anyway as sometime CC use a full path to the source as file symbol. So this is going to be important when we will run for example
clang -o arch/x86/mm/guest_walk_2.o arch/x86/mm/guest_walk.c. (There isn't a patch for that yet.)

That's interesting - what will be the goal of that future adjustment?

It's a step toward my goal of been able to have out-of-tree build for xen, as stated in my cover letter. In order to do that, I try to adapt Kbuild to build Xen.

Kbuild is building the linux kernel without changing directory, so I'd like to do the same, as it probably makes it easier to do out-of-tree build.

Another tool I'd like to use from Kbuild is ./fixdep, it's a small program that run after running CC and fix the dependency file that CC generates. The main thing it does is to add a dependency on Kconfig options that a source file uses instead of having a dependency on whether any unrelated Kconfig has change at all. But ./fixdep from Linux only works if we build without changing directory. ([1] for more on fixdep)

I guess one advantage of never changing directory is that we can always use relative path in global *FLAGS. There isn't a need to use absolute path, which is an issue when the source tree is moved to a different location. That can easily happen when for example you try to build in a container (mapping the source tree inside it) then try to rebuild from outside. (After using automation/scripts/containerize for example.) And we don't need tricks like the .*.d2 files (which isn't needed in the hypervisor anyway, so far at least).

[1], copied from Linux's scripts/basic/fixdep.c introduction:
If the user re-runs make *config, autoconf.h will be regenerated. make notices that and will rebuild every file which includes autoconf.h, i.e. basically all files. This is extremely annoying if the user just changed CONFIG_HIS_DRIVER from n to m.

So we play the same trick that "mkdep" played before. We replace the dependency on autoconf.h by a dependency on every config option which is mentioned in any of the listed prerequisites.

kconfig populates a tree in include/config/ with an empty file for each config symbol and when the configuration is updated the files representing changed config options are touched which then let make pick up the changes and the files that use the config symbols are rebuilt.

So if the user changes his CONFIG_HIS_DRIVER option, only the objects which depend on "include/config/his/driver.h" will be rebuilt, so most likely only his driver ;-)

Signature · 1 line
Anthony PERARD
#42 lore raw

Re: [XEN PATCH v4 14/18] xen,symbols: rework file symbols selection

Quoted 48 lines · Anthony PERARD
Quoted 11 lines · Jan Beulich
Quoted 3 lines · Anthony PERARD

Or do you mean keeping exception to the rule? And hope that when someone changes the rule, it doesn't forget to check if the exception needs changing as well?

... "exception" like you put it (requiring special care to keep multiple instances in sync) is not the only way this can be done (and indeed I'd not want something like this). Since you have (in patch 15) e.g.

guest_walk_%.o: guest_walk.c FORCE

Quoted 8 lines
	$(call if_changed_rule,cc_o_c)

anyway, the desire to skip the objcopy step could be communicated to the command from here, without needing to clone the command. One way might be a special (phony) dependency, another might be to set some variable along the lines of

guest_walk_%.o: SPECIAL := y

I guess something like that could be done. But if possible, I'd like to avoid that.

Quoted 7 lines
Quoted 5 lines

Also, I'm going to have to use this patch later anyway as sometime CC use a full path to the source as file symbol. So this is going to be important when we will run for example
clang -o arch/x86/mm/guest_walk_2.o arch/x86/mm/guest_walk.c. (There isn't a patch for that yet.)

That's interesting - what will be the goal of that future adjustment?

It's a step toward my goal of been able to have out-of-tree build for xen, as stated in my cover letter. In order to do that, I try to adapt Kbuild to build Xen.

Kbuild is building the linux kernel without changing directory, so I'd like to do the same, as it probably makes it easier to do out-of-tree build.

Another tool I'd like to use from Kbuild is ./fixdep, it's a small program that run after running CC and fix the dependency file that CC generates. The main thing it does is to add a dependency on Kconfig options that a source file uses instead of having a dependency on whether any unrelated Kconfig has change at all. But ./fixdep from Linux only works if we build without changing directory. ([1] for more on fixdep)

Quoted 8 lines

I guess one advantage of never changing directory is that we can always use relative path in global *FLAGS. There isn't a need to use absolute path, which is an issue when the source tree is moved to a different location. That can easily happen when for example you try to build in a container (mapping the source tree inside it) then try to rebuild from outside. (After using automation/scripts/containerize for example.) And we don't need tricks like the .*.d2 files (which isn't needed in the hypervisor anyway, so far at least).

Ah, I see. Out-of-tree builds don't necessarily imply source trees that can also be moved, so you want to actually go a step further.

Jan

#43 lore raw

PATCH v4 15/18 xen/build: use if_changed to build guest_%.o

In reply to Anthony PERARD

Use if_changed for building all guest_%.o objects, and make use of command that already exist.

This patch make a change to the way guest_%.o files are built, and now run the same commands that enforce unique symbols. But with patch "xen,symbols: rework file symbols selection", ./symbols should still select the file symbols directive intended to be used for guest_%.o objects.

---

Notes:

    v4:
    - remove the introduction of Kbuild's CFLAGS_$@
      and simply use make's per-target variable customization.
      Mostly to avoid using $(eval ) which might not work as expected on
      make 3.80.
 xen/arch/x86/mm/Makefile        | 14 ++++++++------
 xen/arch/x86/mm/hap/Makefile    | 15 +++++++++------
 xen/arch/x86/mm/shadow/Makefile | 14 ++++++++------
 3 files changed, 25 insertions(+), 18 deletions(-)
xen/arch/x86/mm/Makefile +8 -6
index a2431fde6bb4..a66a57314489 100644
@@ -11,11 +11,13 @@ obj-y += p2m.o p2m-pt.o
 obj-$(CONFIG_HVM) += p2m-ept.o p2m-pod.o
 obj-y += paging.o
 
-guest_walk_%.o: guest_walk.c Makefile
-	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+guest_walk_%.o guest_walk_%.i guest_walk_%.s: CFLAGS-y += -DGUEST_PAGING_LEVELS=$*
 
-guest_walk_%.i: guest_walk.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+guest_walk_%.o: guest_walk.c FORCE
+	$(call if_changed_rule,cc_o_c)
 
-guest_walk_%.s: guest_walk.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
+guest_walk_%.i: guest_walk.c FORCE
+	$(call if_changed,cpp_i_c)
+
+guest_walk_%.s: guest_walk.c FORCE
+	$(call if_changed,cc_s_c)
xen/arch/x86/mm/hap/Makefile +9 -6
index 22e7ad54bd33..34720b2fbe2e 100644
@@ -5,11 +5,14 @@ obj-y += guest_walk_4level.o
 obj-y += nested_hap.o
 obj-y += nested_ept.o
 
-guest_walk_%level.o: guest_walk.c Makefile
-	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+guest_walk_%level.o guest_walk_%level.i guest_walk_%level.s: \
+    CFLAGS-y += -DGUEST_PAGING_LEVELS=$*
 
-guest_walk_%level.i: guest_walk.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+guest_walk_%level.o: guest_walk.c FORCE
+	$(call if_changed_rule,cc_o_c)
 
-guest_walk_%level.s: guest_walk.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
+guest_walk_%level.i: guest_walk.c FORCE
+	$(call if_changed,cpp_i_c)
+
+guest_walk_%level.s: guest_walk.c FORCE
+	$(call if_changed,cc_s_c)
xen/arch/x86/mm/shadow/Makefile +8 -6
index 23d3ff10802c..e00f9cb1aaba 100644
@@ -6,11 +6,13 @@ else
 obj-y += none.o
 endif
 
-guest_%.o: multi.c Makefile
-	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+guest_%.o guest_%.i guest_%.s: CFLAGS-y += -DGUEST_PAGING_LEVELS=$*
 
-guest_%.i: multi.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+guest_%.o: multi.c FORCE
+	$(call if_changed_rule,cc_o_c)
 
-guest_%.s: multi.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
+guest_%.i: multi.c FORCE
+	$(call if_changed,cpp_i_c)
+
+guest_%.s: multi.c FORCE
+	$(call if_changed,cc_s_c)
Signature · 1 line
Anthony PERARD
#44 lore raw

Re: [XEN PATCH v4 15/18] xen/build: use if_changed to build guest_%.o

Quoted 8 lines · Anthony PERARD

Use if_changed for building all guest_%.o objects, and make use of command that already exist.

This patch make a change to the way guest_%.o files are built, and now run the same commands that enforce unique symbols. But with patch "xen,symbols: rework file symbols selection", ./symbols should still select the file symbols directive intended to be used for guest_%.o objects.

I'm having trouble making the connection between the change to the symbols tool and the adjustments made here:

xen/arch/x86/mm/Makefile obj-y += p2m.o p2m-pt.o · line 15

Quoted 8 lines
xen/arch/x86/mm/Makefile +0 -2
@@ -11,11 +11,13 @@ obj-y += p2m.o p2m-pt.o
 obj-$(CONFIG_HVM) += p2m-ept.o p2m-pod.o
 obj-y += paging.o
 
-guest_walk_%.o: guest_walk.c Makefile
-	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@

The original rules didn't do anything special to arrange for the resulting kallsyms names; these arrangements instead live at the top of the respective source files, in the form of asm()-s.

Jan

#45 lore raw

Re: [XEN PATCH v4 15/18] xen/build: use if_changed to build guest_%.o

Quoted 4 lines · Jan Beulich
Quoted 3 lines · Anthony PERARD

Use if_changed for building all guest_%.o objects, and make use of command that already exist.

Quoted 8 lines
Quoted 5 lines

This patch make a change to the way guest_%.o files are built, and now run the same commands that enforce unique symbols. But with patch "xen,symbols: rework file symbols selection", ./symbols should still select the file symbols directive intended to be used for guest_%.o objects.

I'm having trouble making the connection between the change to the symbols tool and the adjustments made here:

The change to symbol tool is to allow this change.

Quoted 6 lines
Quoted 6 lines
xen/arch/x86/mm/Makefile +0 -0
@@ -11,11 +11,13 @@ obj-y += p2m.o p2m-pt.o
 obj-$(CONFIG_HVM) += p2m-ept.o p2m-pod.o
 obj-y += paging.o
 
Quoted 6 lines

xen/arch/x86/mm/Makefile obj-y += p2m.o p2m-pt.o · line 15

Quoted 2 lines
xen/arch/x86/mm/Makefile +0 -2
-guest_walk_%.o: guest_walk.c Makefile
-	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@

The original rules didn't do anything special to arrange for the resulting kallsyms names; these arrangements instead live at the top of the respective source files, in the form of asm()-s.

They still are. I try to consolidate the number of location which have command that build a target. Those guest_%.o object aren't special enough to deserve to be built in a different way than every other object. They do need a different make rule, but they can use the same command.

Thanks,

Signature · 1 line
Anthony PERARD
#46 lore raw

Re: [XEN PATCH v4 15/18] xen/build: use if_changed to build guest_%.o

Quoted 11 lines · Anthony PERARD
Quoted 10 lines · Jan Beulich
Quoted 8 lines · Anthony PERARD

Use if_changed for building all guest_%.o objects, and make use of command that already exist.

This patch make a change to the way guest_%.o files are built, and now run the same commands that enforce unique symbols. But with patch "xen,symbols: rework file symbols selection", ./symbols should still select the file symbols directive intended to be used for guest_%.o objects.

Quoted 4 lines
Quoted 2 lines

I'm having trouble making the connection between the change to the symbols tool and the adjustments made here:

The change to symbol tool is to allow this change.

I've been understanding the fact, but I still don't understand why the adjustment to that tool is necessary for the change here to be made.

Quoted 13 lines
Quoted 6 lines
Quoted 6 lines
xen/arch/x86/mm/Makefile +0 -0
@@ -11,11 +11,13 @@ obj-y += p2m.o p2m-pt.o
 obj-$(CONFIG_HVM) += p2m-ept.o p2m-pod.o
 obj-y += paging.o
 
Quoted 6 lines

xen/arch/x86/mm/Makefile obj-y += p2m.o p2m-pt.o · line 15

Quoted 2 lines
xen/arch/x86/mm/Makefile +0 -2
-guest_walk_%.o: guest_walk.c Makefile
-	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@

The original rules didn't do anything special to arrange for the resulting kallsyms names; these arrangements instead live at the top of the respective source files, in the form of asm()-s.

Quoted 5 lines

They still are. I try to consolidate the number of location which have command that build a target. Those guest_%.o object aren't special enough to deserve to be built in a different way than every other object. They do need a different make rule, but they can use the same command.

Again, I understand what the goal is, but not what it is that changes (and why) in the produced file symbols, making the utility adjustment necessary. I guess it's obvious to you, but it looks as if I was dense, sorry.

Jan

#47 lore raw

PATCH v4 16/18 build,xsm: Fix multiple call

In reply to Anthony PERARD

Both script mkflask.sh and mkaccess_vector.sh generates multiple files. Exploits the 'multi-target pattern rule' trick to call each scripts only once.

---

Notes:
v4:
- new patch

 xen/xsm/flask/Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
xen/xsm/flask/Makefile +2 -2
index 7d0831e2b865..48577cbe3f04 100644
@@ -26,14 +26,14 @@ mkflask := policy/mkflask.sh
 quiet_cmd_mkflask = MKFLASK $@
 cmd_mkflask = $(CONFIG_SHELL) $(mkflask) $(AWK) include $(FLASK_H_DEPEND)
 
-$(FLASK_H_FILES): $(FLASK_H_DEPEND) $(mkflask) FORCE
+$(patsubst include/%,\%/%,$(FLASK_H_FILES)): $(FLASK_H_DEPEND) $(mkflask) FORCE
 	$(call if_changed,mkflask)
 
 mkaccess := policy/mkaccess_vector.sh
 quiet_cmd_mkaccess = MKACCESS VECTOR $@
 cmd_mkaccess = $(CONFIG_SHELL) $(mkaccess) $(AWK) $(AV_H_DEPEND)
 
-$(AV_H_FILES): $(AV_H_DEPEND) $(mkaccess) FORCE
+$(patsubst include/%,\%/%,$(AV_H_FILES)): $(AV_H_DEPEND) $(mkaccess) FORCE
 	$(call if_changed,mkaccess)
 
 obj-bin-$(CONFIG_XSM_FLASK_POLICY) += flask-policy.o
Signature · 1 line
Anthony PERARD
Reviewed-by on v4 16/18
#48 lore raw

Re: [XEN PATCH v4 16/18] build,xsm: Fix multiple call

Quoted 3 lines · Anthony PERARD

Both script mkflask.sh and mkaccess_vector.sh generates multiple files. Exploits the 'multi-target pattern rule' trick to call each scripts only once.

Isn't this a general fix, which may even want backporting? If so, this would better be at or near the beginning of the series.

xen/xsm/flask/Makefile mkflask := policy/mkflask.sh · line 29

Quoted 8 lines
xen/xsm/flask/Makefile +1 -1
@@ -26,14 +26,14 @@ mkflask := policy/mkflask.sh
 quiet_cmd_mkflask = MKFLASK $@
 cmd_mkflask = $(CONFIG_SHELL) $(mkflask) $(AWK) include $(FLASK_H_DEPEND)
 
-$(FLASK_H_FILES): $(FLASK_H_DEPEND) $(mkflask) FORCE
+$(patsubst include/%,\%/%,$(FLASK_H_FILES)): $(FLASK_H_DEPEND) $(mkflask) FORCE

Since what $(FLASK_H_FILES) contains is well under our control, how about the simpler

$(subst include/,%/,$(FLASK_H_FILES)): ...

? Preferably with this and preferably with it moved ahead

Jan

#49 lore raw

Re: [XEN PATCH v4 16/18] build,xsm: Fix multiple call

Quoted 7 lines · Jan Beulich
Quoted 3 lines · Anthony PERARD

Both script mkflask.sh and mkaccess_vector.sh generates multiple files. Exploits the 'multi-target pattern rule' trick to call each scripts only once.

Isn't this a general fix, which may even want backporting? If so, this would better be at or near the beginning of the series.

It is mostly a performance improvement, avoiding doing the same thing several time. I don't think anything bad happens from concurrent calls, or we would already have bug report I think. But I can try to move the patch up.

Quoted 12 lines

xen/xsm/flask/Makefile mkflask := policy/mkflask.sh · line 29

Quoted 8 lines
xen/xsm/flask/Makefile +1 -1
@@ -26,14 +26,14 @@ mkflask := policy/mkflask.sh
 quiet_cmd_mkflask = MKFLASK $@
 cmd_mkflask = $(CONFIG_SHELL) $(mkflask) $(AWK) include $(FLASK_H_DEPEND)
 
-$(FLASK_H_FILES): $(FLASK_H_DEPEND) $(mkflask) FORCE
+$(patsubst include/%,\%/%,$(FLASK_H_FILES)): $(FLASK_H_DEPEND) $(mkflask) FORCE

Since what $(FLASK_H_FILES) contains is well under our control, how about the simpler

Quoted 4 lines

$(subst include/,%/,$(FLASK_H_FILES)): ...

? Preferably with this and preferably with it moved ahead

I'll do that, thanks,

Signature · 1 line
Anthony PERARD
#50 lore raw

Re: [XEN PATCH v4 16/18] build,xsm: Fix multiple call

Quoted 6 lines · Anthony PERARD
Quoted 5 lines · Jan Beulich
Quoted 3 lines · Anthony PERARD

Both script mkflask.sh and mkaccess_vector.sh generates multiple files. Exploits the 'multi-target pattern rule' trick to call each scripts only once.

Quoted 7 lines
Quoted 2 lines

Isn't this a general fix, which may even want backporting? If so, this would better be at or near the beginning of the series.

It is mostly a performance improvement, avoiding doing the same thing several time. I don't think anything bad happens from concurrent calls, or we would already have bug report I think. But I can try to move the patch up.

Up to three processes in parallel writing to the same file(s) is almost certainly a recipe for eventual / random breakage.

Jan

#51 lore raw

PATCH v4 17/18 build,include: rework compat-build-source.py

In reply to Anthony PERARD

Improvement are:
- give the path to xlat.lst as argument
- include grep -v in compat-build-source.py script, we don't need to write this in several scripted language.
- have 'xlat.lst' path as a variable.

No changes in final compat/%.h headers.

---

Notes:
v4:
- new patch

 xen/include/Makefile             | 11 ++++++-----
 xen/tools/compat-build-source.py |  8 +++++++-
 2 files changed, 13 insertions(+), 6 deletions(-)
xen/include/Makefile +6 -5
index 2a10725d689b..74b26a028902 100644
@@ -50,6 +50,8 @@ public-$(CONFIG_ARM) := $(wildcard public/arch-arm/*.h public/arch-arm/*/*.h)
 .PHONY: all
 all: $(headers-y)
 
+xlat_lst = xlat.lst
+
 compat/%.h: compat/%.i Makefile $(BASEDIR)/tools/compat-build-header.py
 	set -e; id=_$$(echo $@ | tr '[:lower:]-/.' '[:upper:]___'); \
 	echo "#ifndef $$id" >[email protected]; \
@@ -66,10 +68,9 @@ compat/%.h: compat/%.i Makefile $(BASEDIR)/tools/compat-build-header.py
 compat/%.i: compat/%.c Makefile
 	$(CPP) $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(XEN_CFLAGS)) $(cppflags-y) -o $@ $<
 
-compat/%.c: public/%.h xlat.lst Makefile $(BASEDIR)/tools/compat-build-source.py
+compat/%.c: public/%.h $(xlat_lst) Makefile $(BASEDIR)/tools/compat-build-source.py
 	mkdir -p $(@D)
-	grep -v 'DEFINE_XEN_GUEST_HANDLE(long)' $< | \
-	$(PYTHON) $(BASEDIR)/tools/compat-build-source.py >[email protected]
+	$(PYTHON) $(BASEDIR)/tools/compat-build-source.py $(xlat_lst) <$< >[email protected]
 	mv -f [email protected] $@
 
 compat/.xlat/%.h: compat/%.h compat/.xlat/%.lst $(BASEDIR)/tools/get-fields.sh Makefile
@@ -80,12 +81,12 @@ compat/.xlat/%.h: compat/%.h compat/.xlat/%.lst $(BASEDIR)/tools/get-fields.sh M
 	mv -f [email protected] $@
 
 .PRECIOUS: compat/.xlat/%.lst
-compat/.xlat/%.lst: xlat.lst Makefile
+compat/.xlat/%.lst: $(xlat_lst) Makefile
 	mkdir -p $(@D)
 	grep -v '^[[:blank:]]*#' $< | sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,[[:blank:]]+$*\.h[[:blank:]]*$$,,p' >[email protected]
 	$(call move-if-changed,[email protected],$@)
 
-xlat-y := $(shell sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,^[?!][[:blank:]]+[^[:blank:]]+[[:blank:]]+,,p' xlat.lst | uniq)
+xlat-y := $(shell sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,^[?!][[:blank:]]+[^[:blank:]]+[[:blank:]]+,,p' $(xlat_lst) | uniq)
 xlat-y := $(filter $(patsubst compat/%,%,$(headers-y)),$(xlat-y))
 
 compat/xlat.h: $(addprefix compat/.xlat/,$(xlat-y)) Makefile
xen/tools/compat-build-source.py +7 -1
index c664eb85e633..c7fc2c53db61 100755
@@ -12,7 +12,11 @@ pats = [
  [ r"XEN_GUEST_HANDLE(_[0-9A-Fa-f]+)?", r"COMPAT_HANDLE" ],
 ];
 
-xlatf = open('xlat.lst', 'r')
+try:
+    xlatf = open(sys.argv[1], 'r')
+except IndexError:
+    print('missing path to xlat.lst argument')
+    sys.exit(1)
 for line in xlatf.readlines():
     match = re.subn(r"^\s*\?\s+(\w*)\s.*", r"\1", line.rstrip())
     if match[1]:
@@ -24,6 +28,8 @@ for pat in pats:
     pat[0] = re.compile(pat[0])
 
 for line in sys.stdin.readlines():
+    if 'DEFINE_XEN_GUEST_HANDLE(long)' in line:
+        continue
     for pat in pats:
         line = re.sub(pat[0], pat[1], line)
     print(line.rstrip())
Signature · 1 line
Anthony PERARD
#52 lore raw

Re: [XEN PATCH v4 17/18] build, include: rework compat-build-source.py

Quoted 5 lines · Anthony PERARD

Improvement are:
- give the path to xlat.lst as argument
- include grep -v in compat-build-source.py script, we don't need to write this in several scripted language.
- have 'xlat.lst' path as a variable.

The change looks okay, but I'm unsure whether it's really worthwhile. I specifically dislike the last point above, as it makes things less easy to read. I might be willing to ack a patch with this part taken out again; faod I'm not meaning to nak the patch in its current form, but I guess I'm also not going to ack it.

Jan

#53 lore raw

Re: [XEN PATCH v4 17/18] build, include: rework compat-build-source.py

Quoted 7 lines · Jan Beulich
Quoted 5 lines · Anthony PERARD

Improvement are:
- give the path to xlat.lst as argument
- include grep -v in compat-build-source.py script, we don't need to write this in several scripted language.
- have 'xlat.lst' path as a variable.

Quoted 5 lines

The change looks okay, but I'm unsure whether it's really worthwhile. I specifically dislike the last point above, as it makes things less easy to read. I might be willing to ack a patch with this part taken out again; faod I'm not meaning to nak the patch in its current form, but I guess I'm also not going to ack it.

I'll remove the last point from this patch.

Thanks,

Signature · 1 line
Anthony PERARD
#54 lore raw

PATCH v4 18/18 build,include: rework compat-build-header.py

In reply to Anthony PERARD

Replace a mix of shell script and python script by all python script.

Remove the unnecessary "grep -v '^# [0-9]'". It is to hide the linemarkers generated by the preprocessor. But adding -P inhibit there generation, thus the grep isn't needed anymore.

gcc -E -P and clang -E -P have different behavior. While both don't generates linemarkers, gcc also removes all empty lines while clang keep them all. We don't need those empty lines, so we don't generates them in the final compat/%.h headers. (This replace uniq which was only de-duplicating empty line.)

The only changes in the final generated headers it that they don't have empty lines anymore.

---

Notes:
v4:
- new patch

 xen/include/Makefile             | 13 ++----------
 xen/tools/compat-build-header.py | 36 ++++++++++++++++++++++++++++++--
 2 files changed, 36 insertions(+), 13 deletions(-)
xen/include/Makefile +2 -11
index 74b26a028902..7e2d0ff667e8 100644
@@ -53,20 +53,11 @@ all: $(headers-y)
 xlat_lst = xlat.lst
 
 compat/%.h: compat/%.i Makefile $(BASEDIR)/tools/compat-build-header.py
-	set -e; id=_$$(echo $@ | tr '[:lower:]-/.' '[:upper:]___'); \
-	echo "#ifndef $$id" >[email protected]; \
-	echo "#define $$id" >>[email protected]; \
-	echo "#include <xen/compat.h>" >>[email protected]; \
-	$(if $(filter-out compat/arch-%.h,$@),echo "#include <$(patsubst compat/%,public/%,$@)>" >>[email protected];) \
-	$(if $(prefix-y),echo "$(prefix-y)" >>[email protected];) \
-	grep -v '^# [0-9]' $< | \
-	$(PYTHON) $(BASEDIR)/tools/compat-build-header.py | uniq >>[email protected]; \
-	$(if $(suffix-y),echo "$(suffix-y)" >>[email protected];) \
-	echo "#endif /* $$id */" >>[email protected]
+	$(PYTHON) $(BASEDIR)/tools/compat-build-header.py <$< $@ "$(prefix-y)" "$(suffix-y)" >>[email protected]; \
 	mv -f [email protected] $@
 
 compat/%.i: compat/%.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(XEN_CFLAGS)) $(cppflags-y) -o $@ $<
+	$(CPP) -P $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(XEN_CFLAGS)) $(cppflags-y) -o $@ $<
 
 compat/%.c: public/%.h $(xlat_lst) Makefile $(BASEDIR)/tools/compat-build-source.py
 	mkdir -p $(@D)
xen/tools/compat-build-header.py +34 -2
index b85c43f13faf..d89a900ea02c 100755
@@ -2,6 +2,12 @@
 
 import re,sys
 
+try:
+    type(str.maketrans)
+except AttributeError:
+    # For python2
+    import string as str
+
 pats = [
  [ r"__InClUdE__(.*)", r"#include\1\n#pragma pack(4)" ],
  [ r"__IfDeF__ (XEN_HAVE.*)", r"#ifdef \1" ],
@@ -20,7 +26,33 @@ pats = [
  [ r"(^|[^\w])long([^\w]|$$)", r"\1int\2" ]
 ];
 
+output_filename = sys.argv[1]
+
+# tr '[:lower:]-/.' '[:upper:]___'
+header_id = '_' + \
+    output_filename.upper().translate(str.maketrans('-/.','___'))
+
+header = """#ifndef {0}
+#define {0}
+#include <xen/compat.h>""".format(header_id)
+
+print(header)
+
+if not re.match("compat/arch-.*.h$", output_filename):
+    x = output_filename.replace("compat/","public/")
+    print('#include <%s>' % x)
+
+def print_if_nonempty(s):
+    if len(s):
+        print(s)
+
+print_if_nonempty(sys.argv[2])
+
 for line in sys.stdin.readlines():
     for pat in pats:
-        line = re.subn(pat[0], pat[1], line)[0]
-    print(line.rstrip())
+        line = re.sub(pat[0], pat[1], line.rstrip())
+    print_if_nonempty(line)
+
+print_if_nonempty(sys.argv[3])
+
+print("#endif /* %s */" % header_id)
Signature · 1 line
Anthony PERARD
#55 lore raw

Re: [XEN PATCH v4 18/18] build, include: rework compat-build-header.py

Quoted 6 lines · Anthony PERARD

Replace a mix of shell script and python script by all python script.

Remove the unnecessary "grep -v '^# [0-9]'". It is to hide the linemarkers generated by the preprocessor. But adding -P inhibit there generation, thus the grep isn't needed anymore.

Quoted 8 lines

gcc -E -P and clang -E -P have different behavior. While both don't generates linemarkers, gcc also removes all empty lines while clang keep them all. We don't need those empty lines, so we don't generates them in the final compat/%.h headers. (This replace uniq which was only de-duplicating empty line.)

The only changes in the final generated headers it that they don't have empty lines anymore.

Making them harder to read? While typically no-one needs to look at their contents, in case of problems it helps if generated files are half way accessible to a human as well.

Jan

#56 lore raw

Re: [XEN PATCH v4 18/18] build, include: rework compat-build-header.py

Quoted 13 lines · Jan Beulich
Quoted 12 lines · Anthony PERARD

Replace a mix of shell script and python script by all python script.

Remove the unnecessary "grep -v '^# [0-9]'". It is to hide the linemarkers generated by the preprocessor. But adding -P inhibit there generation, thus the grep isn't needed anymore.

gcc -E -P and clang -E -P have different behavior. While both don't generates linemarkers, gcc also removes all empty lines while clang keep them all. We don't need those empty lines, so we don't generates them in the final compat/%.h headers. (This replace uniq which was only de-duplicating empty line.)

Quoted 6 lines
Quoted 2 lines

The only changes in the final generated headers it that they don't have empty lines anymore.

Making them harder to read? While typically no-one needs to look at their contents, in case of problems it helps if generated files are half way accessible to a human as well.

I do think they are still readable. Those empty lines don't add much. There are so many of them that a uniq is needed...

For example, with dm_op.h, we have this:

<<<<<<< before
#pragma pack(4)
typedef uint16_t ioservid_compat_t;
struct compat_dm_op_create_ioreq_server {

    uint8_t handle_bufioreq;
    uint8_t pad[3];

    ioservid_compat_t id;
};

struct compat_dm_op_get_ioreq_server_info {

    ioservid_compat_t id;

    uint16_t flags;

    evtchn_port_compat_t bufioreq_port;

    uint64_t ioreq_gfn;

    uint64_t bufioreq_gfn;
};

struct compat_dm_op_ioreq_server_range {

    ioservid_compat_t id;
    uint16_t pad;

    uint32_t type;

    uint64_t start, end;
};
=======
#pragma pack(4)
typedef uint16_t ioservid_compat_t;
struct compat_dm_op_create_ioreq_server {
    uint8_t handle_bufioreq;
    uint8_t pad[3];
    ioservid_compat_t id;
};
struct compat_dm_op_get_ioreq_server_info {
    ioservid_compat_t id;
    uint16_t flags;
    evtchn_port_compat_t bufioreq_port;
    uint64_t ioreq_gfn;
    uint64_t bufioreq_gfn;
};
struct compat_dm_op_ioreq_server_range {
    ioservid_compat_t id;
    uint16_t pad;
    uint32_t type;
    uint64_t start, end;
};
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

after

Thanks,

Signature · 1 line
Anthony PERARD
#57 lore raw

Re: [XEN PATCH v4 18/18] build, include: rework compat-build-header.py

Quoted 17 lines · Anthony PERARD
Quoted 16 lines · Jan Beulich
Quoted 14 lines · Anthony PERARD

Replace a mix of shell script and python script by all python script.

Remove the unnecessary "grep -v '^# [0-9]'". It is to hide the linemarkers generated by the preprocessor. But adding -P inhibit there generation, thus the grep isn't needed anymore.

gcc -E -P and clang -E -P have different behavior. While both don't generates linemarkers, gcc also removes all empty lines while clang keep them all. We don't need those empty lines, so we don't generates them in the final compat/%.h headers. (This replace uniq which was only de-duplicating empty line.)

The only changes in the final generated headers it that they don't have empty lines anymore.

Quoted 8 lines
Quoted 3 lines

Making them harder to read? While typically no-one needs to look at their contents, in case of problems it helps if generated files are half way accessible to a human as well.

I do think they are still readable. Those empty lines don't add much. There are so many of them that a uniq is needed...

For example, with dm_op.h, we have this:

Let me take a different example, grant_table.h: Not all of the blank lines it has are useful, but I think the file would suffer if all of them got removed.

Jan

#58 lore raw
In reply to Anthony PERARD

Hi Anthony,

Quoted 4 lines · Anthony PERARD

Anthony PERARD (18):
xen/arm: Rename all early printk macro
xen/arm: Configure early printk via Kconfig
build,arm: Fix deps check of head.o

I have merged the first 3 patches.

Cheers,

Signature · 1 line
Julien Grall