Skip to contentSkip to conversation

Toolstack build system improvement, toward non-recursive makefiles

PATCH Superseded Anthony PERARD 5 participants 39 messages Oct 13, 2022 – Oct 17, 2022

v5 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v5 cover Toolstack build system improvement, toward non-recursive makefiles

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.toolstack-build-system-v5

Changes in v5:
- rebased on staging
- added "tools: Rework linking options for ocaml binding libraries"

Changes in v4:
- several new patches
- some changes to other patches listed in their changelogs

Changes in v3:
- rebased
- several new patches, starting with 13/25 "tools/libs/util: cleanup Makefile"
- introducing macros to deal with linking with in-tree xen libraries
- Add -Werror to CFLAGS for all builds in tools/

Changes in v2:
- one new patch
- other changes described in patch notes

Hi everyone,

I've been looking at reworking the build system we have for the "tools/", and transforming it to something that suit it better. There are a lot of dependencies between different sub-directories so it would be nice if GNU make could actually handle them. This is possible with "non-recursive makefiles".

With non-recursive makefiles, make will have to load/include all the makefiles and thus will have complete overview of all the dependencies. This will allow make to build the necessary targets in other directory, and we won't need to build sub-directories one by one.

To help with this transformation, I've chosen to go with a recent project called "subdirmk". It help to deal with the fact that all makefiles will share the same namespace, it is hooked into autoconf, we can easily run make from any subdirectory. Together "autoconf" and "subdirmk" will also help to get closer to be able to do out-of-tree build of the tools, but I'm mainly looking to have non-recursive makefile.

Link to the project:
https://www.chiark.greenend.org.uk/ucgi/~ian/git/subdirmk.git/

But before getting to the main course, I've got quite a few cleanup and some changes to the makefiles. I start the patch series with patches that remove old left over stuff, then start reworking makefiles. They are some common changes like removing the "build" targets in many places as "all" would be the more common way to spell it and "all" is the default target anyway. They are other changes related to the conversion to "subdirmk", I start to use the variable $(TARGETS) in several makefiles, this variable will have a special meaning in subdirmk which will build those target by default.

As for the conversion to non-recursive makefile, with subdirmk, I have this WIP branch, it contains some changes that I'm trying out, some notes, and the conversion, one Makefile per commit. Cleanup are still needed, some makefile not converted yet, but it's otherwise mostly done.

https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.toolstack-build-system-v1-wip-extra

With that branch, you could tried something like:
./configure; cd tools/xl; make
and xl should be built as well as all the xen library needed. Also, things like make clean or rebuild should be faster in the all tools/ directory.

Cheers,

Anthony PERARD (17):
tools/debugger/gdbsx: Fix and cleanup makefiles tools/xentrace: rework Makefile
tools: Introduce $(xenlibs-ldflags, ) macro
tools: Add -Werror by default to all tools/
tools: Remove -Werror everywhere else
tools/hotplug: Generate "hotplugpath.sh" with configure libs/light/gentypes.py: allow to generate headers in subdirectory git-checkout.sh: handle running git-checkout from a different
directory
libs: Avoid exposing -Wl,--version-script to other built library
libs: Fix auto-generation of version-script for unstable libs tools/include: Rework Makefile
libs/light: Rework acpi table build targets
libs/light: Rework generation of include/_libxl_*.h libs/light: Rework targets prerequisites
libs/light: Makefile cleanup
tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go
tools: Rework linking options for ocaml binding libraries

 tools/configure.ac                     |  2 +
 tools/console/client/Makefile          |  1 -
 tools/console/daemon/Makefile          |  1 -
 tools/debugger/gdbsx/Makefile          | 20 +++----
 tools/debugger/gdbsx/gx/Makefile       | 15 +++---
 tools/debugger/gdbsx/xg/Makefile       | 25 +++------
 tools/debugger/kdd/Makefile            |  1 -
 tools/flask/utils/Makefile             |  1 -
 tools/fuzz/cpu-policy/Makefile         |  2 +-
 tools/golang/xenlight/Makefile         |  8 +--
 tools/hotplug/common/Makefile          | 10 +---
 tools/include/Makefile                 | 28 +++++-----
 tools/libs/call/Makefile               |  1 +
 tools/libs/ctrl/Makefile               |  3 --
 tools/libs/devicemodel/Makefile        |  1 +
 tools/libs/evtchn/Makefile             |  1 +
 tools/libs/foreignmemory/Makefile      |  1 +
 tools/libs/gnttab/Makefile             |  1 +
 tools/libs/guest/Makefile              |  3 --
 tools/libs/hypfs/Makefile              |  1 +
 tools/libs/light/Makefile              | 72 +++++++++++++++-----------
 tools/libs/stat/Makefile               |  2 +-
 tools/libs/store/Makefile              |  1 +
 tools/libs/toolcore/Makefile           |  1 +
 tools/libs/toollog/Makefile            |  1 +
 tools/libs/util/Makefile               |  3 --
 tools/libs/vchan/Makefile              |  3 --
 tools/misc/Makefile                    |  1 -
 tools/ocaml/libs/eventchn/Makefile     |  2 +-
 tools/ocaml/libs/xc/Makefile           |  2 +-
 tools/ocaml/libs/xentoollog/Makefile   |  2 +-
 tools/ocaml/libs/xl/Makefile           |  2 +-
 tools/tests/cpu-policy/Makefile        |  2 +-
 tools/tests/depriv/Makefile            |  2 +-
 tools/tests/resource/Makefile          |  1 -
 tools/tests/tsx/Makefile               |  1 -
 tools/tests/xenstore/Makefile          |  1 -
 tools/xcutils/Makefile                 |  2 -
 tools/xenmon/Makefile                  |  1 -
 tools/xenpaging/Makefile               |  1 -
 tools/xenpmd/Makefile                  |  1 -
 tools/xentop/Makefile                  |  2 +-
 tools/xentrace/Makefile                | 29 ++++-------
 tools/xl/Makefile                      |  2 +-
 tools/Rules.mk                         | 20 +++++++
 tools/debugger/gdbsx/Rules.mk          |  2 +-
 tools/firmware/Rules.mk                |  2 -
 tools/libfsimage/common.mk             |  2 +-
 tools/libs/libs.mk                     | 21 +++++---
 tools/libs/light/libxl_x86_acpi.c      |  2 +-
 tools/ocaml/common.make                |  2 +-
 .gitignore                             |  6 ---
 config/Tools.mk.in                     |  1 +
 scripts/git-checkout.sh                |  4 +-
 tools/configure                        | 29 ++++++++++-
 tools/golang/xenlight/gengotypes.py    | 10 +++-
 tools/hotplug/common/hotplugpath.sh.in | 16 ++++++
 tools/libs/light/gentypes.py           |  9 ++--
 tools/xenstore/Makefile.common         |  1 -
 59 files changed, 218 insertions(+), 172 deletions(-)
 create mode 100644 tools/hotplug/common/hotplugpath.sh.in
Signature · 1 line
Anthony PERARD
#1 lore raw

PATCH v5 1/17 tools/debugger/gdbsx: Fix and cleanup makefiles

gdbsx/:
- Make use of subdir facility for the "clean" target.
- No need to remove the *.a, they aren't in this dir.
- Avoid calling "distclean" in subdirs as "distclean" targets do only call "clean", and the "clean" also runs "clean" in subdirs.
- Avoid the need to make "gx_all.a" and "xg_all.a" in the "all" recipe by forcing make to check for update of "xg/xg_all.a" and "gx/gx_all.a" by having "FORCE" as prerequisite. Now, when making "gdbsx", make will recurse even when both *.a already exist.
- List target in $(TARGETS).

gdbsx/*/:
- Fix dependency on *.h.
- Remove some dead code.
- List targets in $(TARGETS).
- Remove "build" target.
- Cleanup "clean" targets.
- remove comments about the choice of "ar" instead of "ld"
- Use "$(AR)" instead of plain "ar".

---

Notes:
missing-ack: GDBSX DEBUGGER

v2:
- also replace plain "ar" by "$(AR)"

 tools/debugger/gdbsx/Makefile    | 20 ++++++++++----------
 tools/debugger/gdbsx/gx/Makefile | 15 +++++++--------
 tools/debugger/gdbsx/xg/Makefile | 25 +++++++------------------
 3 files changed, 24 insertions(+), 36 deletions(-)
tools/debugger/gdbsx/Makefile +10 -10
index 5571450a89..4aaf427c45 100644
@@ -1,20 +1,20 @@
 XEN_ROOT = $(CURDIR)/../../..
 include ./Rules.mk
 
+SUBDIRS-y += gx
+SUBDIRS-y += xg
+
+TARGETS := gdbsx
+
 .PHONY: all
-all:
-	$(MAKE) -C gx
-	$(MAKE) -C xg
-	$(MAKE) gdbsx
+all: $(TARGETS)
 
 .PHONY: clean
-clean:
-	rm -f xg_all.a gx_all.a gdbsx
-	set -e; for d in xg gx; do $(MAKE) -C $$d clean; done
+clean: subdirs-clean
+	rm -f $(TARGETS)
 
 .PHONY: distclean
 distclean: clean
-	set -e; for d in xg gx; do $(MAKE) -C $$d distclean; done
 
 .PHONY: install
 install: all
@@ -28,7 +28,7 @@ uninstall:
 gdbsx: gx/gx_all.a xg/xg_all.a 
 	$(CC) $(LDFLAGS) -o $@ $^
 
-xg/xg_all.a:
+xg/xg_all.a: FORCE
 	$(MAKE) -C xg
-gx/gx_all.a:
+gx/gx_all.a: FORCE
 	$(MAKE) -C gx
tools/debugger/gdbsx/gx/Makefile +7 -8
index 3b8467f799..e9859aea9c 100644
@@ -2,21 +2,20 @@ XEN_ROOT = $(CURDIR)/../../../..
 include ../Rules.mk
 
 GX_OBJS := gx_comm.o gx_main.o gx_utils.o gx_local.o
-GX_HDRS := $(wildcard *.h)
+
+TARGETS := gx_all.a
 
 .PHONY: all
-all: gx_all.a
+all: $(TARGETS)
 
 .PHONY: clean
 clean:
-	rm -rf gx_all.a *.o .*.d
+	rm -f *.o $(TARGETS) $(DEPS_RM)
 
 .PHONY: distclean
 distclean: clean
 
-#%.o: %.c $(GX_HDRS) Makefile
-#	$(CC) -c $(CFLAGS) -o $@ $<
-
-gx_all.a: $(GX_OBJS) Makefile $(GX_HDRS)
-	ar cr $@ $(GX_OBJS)        # problem with ld using -m32 
+gx_all.a: $(GX_OBJS) Makefile
+	$(AR) cr $@ $(GX_OBJS)
 
+-include $(DEPS_INCLUDE)
tools/debugger/gdbsx/xg/Makefile +7 -18
index acdcddf0d5..05325d6d81 100644
@@ -1,35 +1,24 @@
 XEN_ROOT = $(CURDIR)/../../../..
 include ../Rules.mk
 
-XG_HDRS := xg_public.h 
 XG_OBJS := xg_main.o 
 
 CFLAGS += -D__XEN_TOOLS__
 CFLAGS += $(CFLAGS_xeninclude)
 
+TARGETS := xg_all.a
 
 .PHONY: all
-all: build
+all: $(TARGETS)
 
-.PHONY: build
-build: xg_all.a $(XG_HDRS) $(XG_OBJS) Makefile
-# build: mk-symlinks xg_all.a $(XG_HDRS) $(XG_OBJS) Makefile
-# build: mk-symlinks xg_all.a
-
-xg_all.a: $(XG_OBJS) Makefile $(XG_HDRS)
-	ar cr $@ $(XG_OBJS)    # problems using -m32 in ld 
-#	$(LD) -b elf32-i386 $(LDFLAGS) -r -o $@ $^
-#	$(CC) -m32 -c -o $@ $^
-
-# xg_main.o: xg_main.c Makefile $(XG_HDRS)
-#$(CC) -c $(CFLAGS) -o $@ $<
-
-# %.o: %.c $(XG_HDRS) Makefile  -- doesn't work as it won't overwrite Rules.mk
-#%.o: %.c       -- doesn't recompile when .c changed
+xg_all.a: $(XG_OBJS) Makefile
+	$(AR) cr $@ $(XG_OBJS)
 
 .PHONY: clean
 clean:
-	rm -rf xen xg_all.a $(XG_OBJS)  .*.d
+	rm -f $(TARGETS) $(XG_OBJS) $(DEPS_RM)
 
 .PHONY: distclean
 distclean: clean
+
+-include $(DEPS_INCLUDE)
Signature · 1 line
Anthony PERARD
#2 lore raw

PATCH v5 2/17 tools/xentrace: rework Makefile

In reply to Anthony PERARD

Remove "build" targets.

Use "$(TARGETS)" to list binary to be built.

Cleanup "clean" rule.

Also drop conditional install of $(BIN) and $(LIBBIN) as those two variables are now always populated.

---

Notes:
missing-ack: XENTRACE

v4:
- also use := for BIN, SBIN, LIBBIN, SCRIPTS

v2:
- fix typo in title
- drop conditional install of $(BIN) and $(LIBBIN)

 tools/xentrace/Makefile | 27 +++++++++++----------------
 1 file changed, 11 insertions(+), 16 deletions(-)
tools/xentrace/Makefile +11 -16
index 9fb7fc96e7..63f2f6532d 100644
@@ -9,41 +9,36 @@ LDLIBS += $(LDLIBS_libxenevtchn)
 LDLIBS += $(LDLIBS_libxenctrl)
 LDLIBS += $(ARGP_LDFLAGS)
 
-BIN      = xenalyze
-SBIN     = xentrace xentrace_setsize
-LIBBIN   = xenctx
-SCRIPTS  = xentrace_format
+BIN     := xenalyze
+SBIN    := xentrace xentrace_setsize
+LIBBIN  := xenctx
+SCRIPTS := xentrace_format
 
-.PHONY: all
-all: build
+TARGETS := $(BIN) $(SBIN) $(LIBBIN)
 
-.PHONY: build
-build: $(BIN) $(SBIN) $(LIBBIN)
+.PHONY: all
+all: $(TARGETS)
 
 .PHONY: install
-install: build
+install: all
 	$(INSTALL_DIR) $(DESTDIR)$(bindir)
 	$(INSTALL_DIR) $(DESTDIR)$(sbindir)
-	[ -z "$(LIBBIN)" ] || $(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
-ifneq ($(BIN),)
+	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
 	$(INSTALL_PROG) $(BIN) $(DESTDIR)$(bindir)
-endif
 	$(INSTALL_PROG) $(SBIN) $(DESTDIR)$(sbindir)
 	$(INSTALL_PYTHON_PROG) $(SCRIPTS) $(DESTDIR)$(bindir)
-	[ -z "$(LIBBIN)" ] || $(INSTALL_PROG) $(LIBBIN) $(DESTDIR)$(LIBEXEC_BIN)
+	$(INSTALL_PROG) $(LIBBIN) $(DESTDIR)$(LIBEXEC_BIN)
 
 .PHONY: uninstall
 uninstall:
 	rm -f $(addprefix $(DESTDIR)$(LIBEXEC_BIN)/, $(LIBBIN))
 	rm -f $(addprefix $(DESTDIR)$(bindir)/, $(SCRIPTS))
 	rm -f $(addprefix $(DESTDIR)$(sbindir)/, $(SBIN))
-ifneq ($(BIN),)
 	rm -f $(addprefix $(DESTDIR)$(bindir)/, $(BIN))
-endif
 
 .PHONY: clean
 clean:
-	$(RM) *.a *.so *.o *.rpm $(BIN) $(SBIN) $(LIBBIN) $(DEPS_RM)
+	$(RM) *.o $(TARGETS) $(DEPS_RM)
 
 .PHONY: distclean
 distclean: clean
Signature · 1 line
Anthony PERARD
#3 lore raw

PATCH v5 3/17 tools: Introduce $(xenlibs-ldflags, ) macro

In reply to Anthony PERARD

This avoid the need to open-coding the list of flags needed to link with an in-tree Xen library when using -lxen*.

---

Notes:
missing-ack: GOLANG BINDINGS

 tools/golang/xenlight/Makefile | 2 +-
 tools/Rules.mk                 | 8 ++++++++
 2 files changed, 9 insertions(+), 1 deletion(-)
tools/golang/xenlight/Makefile +1 -1
index 64671f246c..00e6d17f2b 100644
@@ -27,7 +27,7 @@ GOXL_GEN_FILES = types.gen.go helpers.gen.go
 # so that it can find the actual library.
 .PHONY: build
 build: xenlight.go $(GOXL_GEN_FILES)
-	CGO_CFLAGS="$(CFLAGS_libxenlight) $(CFLAGS_libxentoollog) $(APPEND_CFLAGS)" CGO_LDFLAGS="$(LDLIBS_libxenlight) $(LDLIBS_libxentoollog) -L$(XEN_libxenlight) -L$(XEN_libxentoollog) $(APPEND_LDFLAGS)" $(GO) build -x
+	CGO_CFLAGS="$(CFLAGS_libxenlight) $(CFLAGS_libxentoollog) $(APPEND_CFLAGS)" CGO_LDFLAGS="$(call xenlibs-ldflags,light toollog) $(APPEND_LDFLAGS)" $(GO) build -x
 
 .PHONY: install
 install: build
tools/Rules.mk +8 -0
index ce77dd2eb1..26958b2948 100644
@@ -105,6 +105,14 @@ define xenlibs-ldlibs
     $(foreach lib,$(1),$(xenlibs-ldlibs-$(lib)))
 endef
 
+# Provide needed flags for linking an in-tree Xen library by an external
+# project (or when it is necessary to link with "-lxen$(1)" instead of using
+# the full path to the library).
+define xenlibs-ldflags
+    $(call xenlibs-rpath,$(1)) \
+    $(foreach lib,$(1),-L$(XEN_ROOT)/tools/libs/$(lib))
+endef
+
 define LIB_defs
  FILENAME_$(1) ?= xen$(1)
  XEN_libxen$(1) = $$(XEN_ROOT)/tools/libs/$(1)
Signature · 1 line
Anthony PERARD
#4 lore raw

PATCH v5 4/17 tools: Add -Werror by default to all tools/

In reply to Anthony PERARD

And provide an option to ./configure to disable it.

A follow-up patch will remove -Werror from every other Makefile in tools/. ("tools: Remove -Werror everywhere else")

---
 tools/configure.ac |  1 +
 tools/Rules.mk     |  4 ++++
 config/Tools.mk.in |  1 +
 tools/configure    | 26 ++++++++++++++++++++++++++
 4 files changed, 32 insertions(+)
tools/configure.ac +1 -0
index 32cbe6bd3c..09059bc569 100644
@@ -81,6 +81,7 @@ m4_include([../m4/header.m4])
 AX_XEN_EXPAND_CONFIG()
 
 # Enable/disable options
+AX_ARG_DEFAULT_ENABLE([werror], [Build tools without -Werror])
 AX_ARG_DEFAULT_DISABLE([rpath], [Build tools with -Wl,-rpath,LIBDIR])
 AX_ARG_DEFAULT_DISABLE([githttp], [Download GIT repositories via HTTP])
 AX_ARG_DEFAULT_ENABLE([monitors], [Disable xenstat and xentop monitoring tools])
tools/Rules.mk +4 -0
index 26958b2948..a165dc4bda 100644
@@ -133,6 +133,10 @@ endif
 
 CFLAGS_libxenlight += $(CFLAGS_libxenctrl)
 
+ifeq ($(CONFIG_WERROR),y)
+CFLAGS += -Werror
+endif
+
 ifeq ($(debug),y)
 # Use -Og if available, -O0 otherwise
 dbg_opt_level := $(call cc-option,$(CC),-Og,-O0)
config/Tools.mk.in +1 -0
index 6c1a0a676f..d0d460f922 100644
@@ -1,5 +1,6 @@
 -include $(XEN_ROOT)/config/Paths.mk
 
+CONFIG_WERROR       := @werror@
 CONFIG_RUMP         := @CONFIG_RUMP@
 ifeq ($(CONFIG_RUMP),y)
 XEN_OS              := NetBSDRump
tools/configure +26 -0
index 41deb7fb96..acd9a04c3b 100755
@@ -716,6 +716,7 @@ ocamltools
 monitors
 githttp
 rpath
+werror
 DEBUG_DIR
 XEN_DUMP_DIR
 XEN_PAGING_DIR
@@ -805,6 +806,7 @@ with_xen_scriptdir
 with_xen_dumpdir
 with_rundir
 with_debugdir
+enable_werror
 enable_rpath
 enable_githttp
 enable_monitors
@@ -1490,6 +1492,7 @@ Optional Features:
   --disable-FEATURE       do not include FEATURE (same as --enable-FEATURE=no)
   --enable-FEATURE[=ARG]  include FEATURE [ARG=yes]
   --disable-largefile     omit support for large files
+  --disable-werror        Build tools without -Werror (default is ENABLED)
   --enable-rpath          Build tools with -Wl,-rpath,LIBDIR (default is
                           DISABLED)
   --enable-githttp        Download GIT repositories via HTTP (default is
@@ -4111,6 +4114,29 @@ DEBUG_DIR=$debugdir_path
 
 # Enable/disable options
 
+# Check whether --enable-werror was given.
+if test "${enable_werror+set}" = set; then :
+  enableval=$enable_werror;
+fi
+
+
+if test "x$enable_werror" = "xno"; then :
+
+    ax_cv_werror="n"
+
+elif test "x$enable_werror" = "xyes"; then :
+
+    ax_cv_werror="y"
+
+elif test -z $ax_cv_werror; then :
+
+    ax_cv_werror="y"
+
+fi
+werror=$ax_cv_werror
+
+
+
 # Check whether --enable-rpath was given.
 if test "${enable_rpath+set}" = set; then :
   enableval=$enable_rpath;
Signature · 1 line
Anthony PERARD
#5 lore raw

PATCH v5 5/17 tools: Remove -Werror everywhere else

In reply to Anthony PERARD

Patch "tools: Add -Werror by default to all tools/" have added "-Werror" to CFLAGS in tools/Rules.mk, remove it from every other makefiles as it is now duplicated.

---

Notes:
missing-ack:
- GDBSX DEBUGGER
- KDD DEBUGGER
- XENTRACE

 tools/console/client/Makefile   | 1 -
 tools/console/daemon/Makefile   | 1 -
 tools/debugger/kdd/Makefile     | 1 -
 tools/flask/utils/Makefile      | 1 -
 tools/fuzz/cpu-policy/Makefile  | 2 +-
 tools/misc/Makefile             | 1 -
 tools/tests/cpu-policy/Makefile | 2 +-
 tools/tests/depriv/Makefile     | 2 +-
 tools/tests/resource/Makefile   | 1 -
 tools/tests/tsx/Makefile        | 1 -
 tools/tests/xenstore/Makefile   | 1 -
 tools/xcutils/Makefile          | 2 --
 tools/xenmon/Makefile           | 1 -
 tools/xenpaging/Makefile        | 1 -
 tools/xenpmd/Makefile           | 1 -
 tools/xentop/Makefile           | 2 +-
 tools/xentrace/Makefile         | 2 --
 tools/xl/Makefile               | 2 +-
 tools/debugger/gdbsx/Rules.mk   | 2 +-
 tools/firmware/Rules.mk         | 2 --
 tools/libfsimage/common.mk      | 2 +-
 tools/libs/libs.mk              | 2 +-
 tools/ocaml/common.make         | 2 +-
 tools/xenstore/Makefile.common  | 1 -
 24 files changed, 9 insertions(+), 27 deletions(-)
tools/console/client/Makefile +0 -1
index e2f2554f92..62d89fdeb9 100644
@@ -1,7 +1,6 @@
 XEN_ROOT=$(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror
 CFLAGS += $(CFLAGS_libxenctrl)
 CFLAGS += $(CFLAGS_libxenstore)
 CFLAGS += -include $(XEN_ROOT)/tools/config.h
tools/console/daemon/Makefile +0 -1
index 99bb33b6a2..9fc3b6711f 100644
@@ -1,7 +1,6 @@
 XEN_ROOT=$(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror
 CFLAGS += $(CFLAGS_libxenctrl)
 CFLAGS += $(CFLAGS_libxenstore)
 CFLAGS += $(CFLAGS_libxenevtchn)
tools/debugger/kdd/Makefile +0 -1
index 26116949d4..a72ad3b1e0 100644
@@ -1,7 +1,6 @@
 XEN_ROOT = $(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS  += -Werror
 CFLAGS  += $(CFLAGS_libxenctrl)
 CFLAGS  += -DXC_WANT_COMPAT_MAP_FOREIGN_API
 LDLIBS  += $(LDLIBS_libxenctrl)
tools/flask/utils/Makefile +0 -1
index 6be134142a..88d7edb6b1 100644
@@ -1,7 +1,6 @@
 XEN_ROOT=$(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror
 CFLAGS += $(CFLAGS_libxenctrl)
 
 TARGETS := flask-loadpolicy flask-setenforce flask-getenforce flask-label-pci flask-get-bool flask-set-bool
tools/fuzz/cpu-policy/Makefile +1 -1
index 41a2230408..6e7743e0aa 100644
@@ -17,7 +17,7 @@ install: all
 
 .PHONY: uninstall
 
-CFLAGS += -Werror $(CFLAGS_xeninclude) -D__XEN_TOOLS__
+CFLAGS += $(CFLAGS_xeninclude) -D__XEN_TOOLS__
 CFLAGS += $(APPEND_CFLAGS) -Og
 
 vpath %.c ../../../xen/lib/x86
tools/misc/Makefile +0 -1
index 0e02401227..1c6e1d6a04 100644
@@ -1,7 +1,6 @@
 XEN_ROOT=$(CURDIR)/../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror
 # Include configure output (config.h)
 CFLAGS += -include $(XEN_ROOT)/tools/config.h
 CFLAGS += $(CFLAGS_libxenevtchn)
tools/tests/cpu-policy/Makefile +1 -1
index 93af9d76fa..c5b81afc71 100644
@@ -36,7 +36,7 @@ install: all
 uninstall:
 	$(RM) -- $(addprefix $(DESTDIR)$(LIBEXEC_BIN)/,$(TARGETS))
 
-CFLAGS += -Werror -D__XEN_TOOLS__
+CFLAGS += -D__XEN_TOOLS__
 CFLAGS += $(CFLAGS_xeninclude)
 CFLAGS += $(APPEND_CFLAGS)
 
tools/tests/depriv/Makefile +1 -1
index 3cba28da25..7d9e3b01bb 100644
@@ -1,7 +1,7 @@
 XEN_ROOT=$(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror -Wno-declaration-after-statement
+CFLAGS += -Wno-declaration-after-statement
 
 CFLAGS += $(CFLAGS_xeninclude)
 CFLAGS += $(CFLAGS_libxenctrl)
tools/tests/resource/Makefile +0 -1
index b3cd70c06d..a5856bf095 100644
@@ -27,7 +27,6 @@ install: all
 uninstall:
 	$(RM) -- $(DESTDIR)$(LIBEXEC_BIN)/$(TARGET)
 
-CFLAGS += -Werror
 CFLAGS += $(CFLAGS_xeninclude)
 CFLAGS += $(CFLAGS_libxenctrl)
 CFLAGS += $(CFLAGS_libxenforeginmemory)
tools/tests/tsx/Makefile +0 -1
index d7d2a5d95e..a4f516b725 100644
@@ -26,7 +26,6 @@ uninstall:
 .PHONY: uninstall
 uninstall:
 
-CFLAGS += -Werror
 CFLAGS += -I$(XEN_ROOT)/tools/libs/ctrl -I$(XEN_ROOT)/tools/libs/guest
 CFLAGS += $(CFLAGS_xeninclude)
 CFLAGS += $(CFLAGS_libxenctrl)
tools/tests/xenstore/Makefile +0 -1
index 239e1dce47..202dda0d3c 100644
@@ -27,7 +27,6 @@ install: all
 uninstall:
 	$(RM) -- $(addprefix $(DESTDIR)$(LIBEXEC_BIN)/,$(TARGETS))
 
-CFLAGS += -Werror
 CFLAGS += $(CFLAGS_libxenstore)
 CFLAGS += $(APPEND_CFLAGS)
 
tools/xcutils/Makefile +0 -2
index e40a2c4bfa..3687f6cd8f 100644
@@ -13,8 +13,6 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 TARGETS := readnotes lsevtchn
 
-CFLAGS += -Werror
-
 CFLAGS_readnotes.o  := $(CFLAGS_libxenevtchn) $(CFLAGS_libxenctrl) $(CFLAGS_libxenguest)
 CFLAGS_lsevtchn.o   := $(CFLAGS_libxenevtchn) $(CFLAGS_libxenctrl)
 
tools/xenmon/Makefile +0 -1
index 3e150b0659..679c4b41a3 100644
@@ -13,7 +13,6 @@
 XEN_ROOT=$(CURDIR)/../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS  += -Werror
 CFLAGS  += $(CFLAGS_libxenevtchn)
 CFLAGS  += $(CFLAGS_libxenctrl)
 LDLIBS  += $(LDLIBS_libxenctrl)
tools/xenpaging/Makefile +0 -1
index e2ed9eaa3f..835cf2b965 100644
@@ -12,7 +12,6 @@ OBJS-y   += xenpaging.o
 OBJS-y   += policy_$(POLICY).o
 OBJS-y   += pagein.o
 
-CFLAGS   += -Werror
 CFLAGS   += -Wno-unused
 
 TARGETS := xenpaging
tools/xenpmd/Makefile +0 -1
index e0d3f06ab2..8da20510b5 100644
@@ -1,7 +1,6 @@
 XEN_ROOT=$(CURDIR)/../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror
 CFLAGS += $(CFLAGS_libxenstore)
 
 LDLIBS += $(LDLIBS_libxenstore)
tools/xentop/Makefile +1 -1
index 7bd96f34d5..70cc2211c5 100644
@@ -13,7 +13,7 @@
 XEN_ROOT=$(CURDIR)/../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -DGCC_PRINTF -Werror $(CFLAGS_libxenstat)
+CFLAGS += -DGCC_PRINTF $(CFLAGS_libxenstat)
 LDLIBS += $(LDLIBS_libxenstat) $(CURSES_LIBS) $(TINFO_LIBS) $(SOCKET_LIBS) -lm
 CFLAGS += -DHOST_$(XEN_OS)
 
tools/xentrace/Makefile +0 -2
index 63f2f6532d..d50d400472 100644
@@ -1,8 +1,6 @@
 XEN_ROOT=$(CURDIR)/../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror
-
 CFLAGS += $(CFLAGS_libxenevtchn)
 CFLAGS += $(CFLAGS_libxenctrl)
 LDLIBS += $(LDLIBS_libxenevtchn)
tools/xl/Makefile +1 -1
index b7f439121a..5f7aa5f46c 100644
@@ -5,7 +5,7 @@
 XEN_ROOT = $(CURDIR)/../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror -Wno-format-zero-length -Wmissing-declarations \
+CFLAGS += -Wno-format-zero-length -Wmissing-declarations \
 	-Wno-declaration-after-statement -Wformat-nonliteral
 CFLAGS += -fPIC
 
tools/debugger/gdbsx/Rules.mk +1 -1
index 920f1c87fb..0610db873b 100644
@@ -1,6 +1,6 @@
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS   += -Werror -Wmissing-prototypes 
+CFLAGS   += -Wmissing-prototypes 
 # (gcc 4.3x and later)   -Wconversion -Wno-sign-conversion
 
 CFLAGS-$(clang) += -Wno-ignored-attributes
tools/firmware/Rules.mk +0 -2
index 278cca01e4..d3482c9ec4 100644
@@ -11,8 +11,6 @@ ifneq ($(debug),y)
 CFLAGS += -DNDEBUG
 endif
 
-CFLAGS += -Werror
-
 $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
 
 $(call cc-option-add,CFLAGS,CC,-fcf-protection=none)
tools/libfsimage/common.mk +1 -1
index 77bc957f27..4fc8c66795 100644
@@ -2,7 +2,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 FSDIR := $(libdir)/xenfsimage
 CFLAGS += -Wno-unknown-pragmas -I$(XEN_ROOT)/tools/libfsimage/common/ -DFSIMAGE_FSDIR=\"$(FSDIR)\"
-CFLAGS += -Werror -D_GNU_SOURCE
+CFLAGS += -D_GNU_SOURCE
 LDFLAGS += -L../common/
 
 PIC_OBJS = $(patsubst %.c,%.opic,$(LIB_SRCS-y))
tools/libs/libs.mk +1 -1
index 2b8e7a6128..e47fb30ed4 100644
@@ -14,7 +14,7 @@ MINOR ?= 0
 
 SHLIB_LDFLAGS += -Wl,--version-script=libxen$(LIBNAME).map
 
-CFLAGS   += -Werror -Wmissing-prototypes
+CFLAGS   += -Wmissing-prototypes
 CFLAGS   += $(CFLAGS_xeninclude)
 CFLAGS   += $(foreach lib, $(USELIBS_$(LIBNAME)), $(CFLAGS_libxen$(lib)))
 
tools/ocaml/common.make +1 -1
index d5478f626f..0c8a597d5b 100644
@@ -9,7 +9,7 @@ OCAMLLEX ?= ocamllex
 OCAMLYACC ?= ocamlyacc
 OCAMLFIND ?= ocamlfind
 
-CFLAGS += -fPIC -Werror -I$(shell ocamlc -where)
+CFLAGS += -fPIC -I$(shell ocamlc -where)
 
 OCAMLOPTFLAG_G := $(shell $(OCAMLOPT) -h 2>&1 | sed -n 's/^  *\(-g\) .*/\1/p')
 OCAMLOPTFLAGS = $(OCAMLOPTFLAG_G) -ccopt "$(LDFLAGS)" -dtypes $(OCAMLINCLUDE) -cc $(CC) -w F -warn-error F
tools/xenstore/Makefile.common +0 -1
index 21b78b0538..ddbac052ac 100644
@@ -9,7 +9,6 @@ XENSTORED_OBJS-$(CONFIG_NetBSD) += xenstored_posix.o
 XENSTORED_OBJS-$(CONFIG_FreeBSD) += xenstored_posix.o
 XENSTORED_OBJS-$(CONFIG_MiniOS) += xenstored_minios.o
 
-CFLAGS += -Werror
 # Include configure output (config.h)
 CFLAGS += -include $(XEN_ROOT)/tools/config.h
 CFLAGS += -I./include
Signature · 1 line
Anthony PERARD
#6 lore raw

Re: [XEN PATCH for-4.17 v5 05/17] tools: Remove -Werror everywhere else

tools/debugger/gdbsx/Rules.mk line 6

Quoted 13 lines · Anthony Perard
tools/debugger/gdbsx/Rules.mk +1 -1
index 920f1c87fb..0610db873b 100644
@@ -1,6 +1,6 @@
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS   += -Werror -Wmissing-prototypes 
+CFLAGS   += -Wmissing-prototypes 
 # (gcc 4.3x and later)   -Wconversion -Wno-sign-conversion
 
 CFLAGS-$(clang) += -Wno-ignored-attributes

This hunk introduces trailing whitespace.  All other hunks are fine.

~Andrew

#7 lore raw

Re: [XEN PATCH for-4.17 v5 05/17] tools: Remove -Werror everywhere else

Quoted 8 lines · Andrew Cooper
Quoted 7 lines · Anthony Perard
tools/debugger/gdbsx/Rules.mk +0 -0
index 920f1c87fb..0610db873b 100644
@@ -1,6 +1,6 @@
 include $(XEN_ROOT)/tools/Rules.mk
 
Quoted 8 lines

tools/debugger/gdbsx/Rules.mk line 6

Quoted 6 lines
tools/debugger/gdbsx/Rules.mk +1 -1
-CFLAGS   += -Werror -Wmissing-prototypes 
+CFLAGS   += -Wmissing-prototypes 
 # (gcc 4.3x and later)   -Wconversion -Wno-sign-conversion
 
 CFLAGS-$(clang) += -Wno-ignored-attributes

This hunk introduces trailing whitespace.

No, but it would be a good opportunity to remove it, I agree.

Cheers,

Signature · 1 line
Anthony PERARD
#8 lore raw

Re: [XEN PATCH for-4.17 v5 05/17] tools: Remove -Werror everywhere else

In reply to Anthony PERARD
Quoted 33 lines · Anthony PERARD

Patch "tools: Add -Werror by default to all tools/" have added "-Werror" to CFLAGS in tools/Rules.mk, remove it from every other makefiles as it is now duplicated.

---

Notes:
missing-ack:
- GDBSX DEBUGGER
- KDD DEBUGGER
- XENTRACE

 tools/console/client/Makefile   | 1 -
 tools/console/daemon/Makefile   | 1 -
 tools/debugger/kdd/Makefile     | 1 -
 tools/flask/utils/Makefile      | 1 -
 tools/fuzz/cpu-policy/Makefile  | 2 +-
 tools/misc/Makefile             | 1 -
 tools/tests/cpu-policy/Makefile | 2 +-
 tools/tests/depriv/Makefile     | 2 +-
 tools/tests/resource/Makefile   | 1 -
 tools/tests/tsx/Makefile        | 1 -
 tools/tests/xenstore/Makefile   | 1 -
 tools/xcutils/Makefile          | 2 --
 tools/xenmon/Makefile           | 1 -
 tools/xenpaging/Makefile        | 1 -
 tools/xenpmd/Makefile           | 1 -
 tools/xentop/Makefile           | 2 +-
 tools/xentrace/Makefile         | 2 --
Quoted 8 lines
 tools/xl/Makefile               | 2 +-
 tools/debugger/gdbsx/Rules.mk   | 2 +-
 tools/firmware/Rules.mk         | 2 --
 tools/libfsimage/common.mk      | 2 +-
 tools/libs/libs.mk              | 2 +-
 tools/ocaml/common.make         | 2 +-
 tools/xenstore/Makefile.common  | 1 -
 24 files changed, 9 insertions(+), 27 deletions(-)

Missing hunk as pointed out by Andrew:

tools/pygrub/setup.py +1 -1
index b8f1dc4590..0e4e3d02d3 100644
@@ -3,7 +3,7 @@ from distutils.ccompiler import new_compiler
 import os
 import sys
 
-extra_compile_args  = [ "-fno-strict-aliasing", "-Werror" ]
+extra_compile_args  = [ "-fno-strict-aliasing" ]
 
 XEN_ROOT = "../.."
 
tools/python/setup.py +1 -1
index 8c95db7769..721a3141d7 100644
@@ -8,7 +8,7 @@ SHLIB_libxenctrl = os.environ['SHLIB_libxenctrl'].split()
 SHLIB_libxenguest = os.environ['SHLIB_libxenguest'].split()
 SHLIB_libxenstore = os.environ['SHLIB_libxenstore'].split()
 
-extra_compile_args  = [ "-fno-strict-aliasing", "-Werror" ]
+extra_compile_args  = [ "-fno-strict-aliasing" ]
 
 PATH_XEN      = XEN_ROOT + "/tools/include"
 PATH_LIBXENTOOLLOG = XEN_ROOT + "/tools/libs/toollog"

We can see the change, or duplicate of "-Werror" in commands run by setup.py by running setup.py build --verbose.

Signature · 1 line
Anthony PERARD
#9 lore raw

PATCH v5 6/17 tools/hotplug: Generate "hotplugpath.sh" with configure

In reply to Anthony PERARD

---

Notes:
v4:
- new patch

 tools/configure.ac                     |  1 +
 tools/hotplug/common/Makefile          | 10 ++--------
 tools/configure                        |  3 ++-
 tools/hotplug/common/hotplugpath.sh.in | 16 ++++++++++++++++
 4 files changed, 21 insertions(+), 9 deletions(-)
 create mode 100644 tools/hotplug/common/hotplugpath.sh.in
tools/configure.ac +1 -0
index 09059bc569..18e481d77e 100644
@@ -7,6 +7,7 @@ AC_INIT([Xen Hypervisor Tools], m4_esyscmd([../version.sh ../xen/Makefile]),
 AC_CONFIG_SRCDIR([libs/light/libxl.c])
 AC_CONFIG_FILES([
 ../config/Tools.mk
+hotplug/common/hotplugpath.sh
 hotplug/FreeBSD/rc.d/xencommons
 hotplug/FreeBSD/rc.d/xendriverdomain
 hotplug/Linux/init.d/sysconfig.xencommons
tools/hotplug/common/Makefile +2 -8
index e8a8dbea6c..62afe1019e 100644
@@ -1,19 +1,14 @@
 XEN_ROOT = $(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-HOTPLUGPATH := hotplugpath.sh
-
 # OS-independent hotplug scripts go in this directory
 
 # Xen scripts to go there.
 XEN_SCRIPTS :=
-XEN_SCRIPT_DATA := $(HOTPLUGPATH)
-
-genpath-target = $(call buildmakevars2file,$(HOTPLUGPATH))
-$(eval $(genpath-target))
+XEN_SCRIPT_DATA := hotplugpath.sh
 
 .PHONY: all
-all: $(HOTPLUGPATH)
+all:
 
 .PHONY: install
 install: install-scripts
@@ -40,7 +35,6 @@ uninstall-scripts:
 
 .PHONY: clean
 clean:
-	rm -f $(HOTPLUGPATH)
 
 .PHONY: distclean
 distclean: clean
tools/configure +2 -1
index acd9a04c3b..6199823f5a 100755
@@ -2456,7 +2456,7 @@ ac_compiler_gnu=$ac_cv_c_compiler_gnu
 
 
 
-ac_config_files="$ac_config_files ../config/Tools.mk hotplug/FreeBSD/rc.d/xencommons hotplug/FreeBSD/rc.d/xendriverdomain hotplug/Linux/init.d/sysconfig.xencommons hotplug/Linux/init.d/sysconfig.xendomains hotplug/Linux/init.d/xen-watchdog hotplug/Linux/init.d/xencommons hotplug/Linux/init.d/xendomains hotplug/Linux/init.d/xendriverdomain hotplug/Linux/launch-xenstore hotplug/Linux/vif-setup hotplug/Linux/xen-hotplug-common.sh hotplug/Linux/xendomains hotplug/NetBSD/rc.d/xencommons hotplug/NetBSD/rc.d/xendriverdomain ocaml/libs/xs/paths.ml ocaml/xenstored/paths.ml ocaml/xenstored/oxenstored.conf"
+ac_config_files="$ac_config_files ../config/Tools.mk hotplug/common/hotplugpath.sh hotplug/FreeBSD/rc.d/xencommons hotplug/FreeBSD/rc.d/xendriverdomain hotplug/Linux/init.d/sysconfig.xencommons hotplug/Linux/init.d/sysconfig.xendomains hotplug/Linux/init.d/xen-watchdog hotplug/Linux/init.d/xencommons hotplug/Linux/init.d/xendomains hotplug/Linux/init.d/xendriverdomain hotplug/Linux/launch-xenstore hotplug/Linux/vif-setup hotplug/Linux/xen-hotplug-common.sh hotplug/Linux/xendomains hotplug/NetBSD/rc.d/xencommons hotplug/NetBSD/rc.d/xendriverdomain ocaml/libs/xs/paths.ml ocaml/xenstored/paths.ml ocaml/xenstored/oxenstored.conf"
 
 ac_config_headers="$ac_config_headers config.h"
 
@@ -10947,6 +10947,7 @@ for ac_config_target in $ac_config_targets
 do
   case $ac_config_target in
     "../config/Tools.mk") CONFIG_FILES="$CONFIG_FILES ../config/Tools.mk" ;;
+    "hotplug/common/hotplugpath.sh") CONFIG_FILES="$CONFIG_FILES hotplug/common/hotplugpath.sh" ;;
     "hotplug/FreeBSD/rc.d/xencommons") CONFIG_FILES="$CONFIG_FILES hotplug/FreeBSD/rc.d/xencommons" ;;
     "hotplug/FreeBSD/rc.d/xendriverdomain") CONFIG_FILES="$CONFIG_FILES hotplug/FreeBSD/rc.d/xendriverdomain" ;;
     "hotplug/Linux/init.d/sysconfig.xencommons") CONFIG_FILES="$CONFIG_FILES hotplug/Linux/init.d/sysconfig.xencommons" ;;
tools/hotplug/common/hotplugpath.sh.in new file +16 -0
new file mode 100644
index 0000000000..1036b884b8
@@ -0,0 +1,16 @@
+sbindir="@sbindir@"
+bindir="@bindir@"
+LIBEXEC="@LIBEXEC@"
+LIBEXEC_BIN="@LIBEXEC_BIN@"
+libdir="@libdir@"
+SHAREDIR="@SHAREDIR@"
+XENFIRMWAREDIR="@XENFIRMWAREDIR@"
+XEN_CONFIG_DIR="@XEN_CONFIG_DIR@"
+XEN_SCRIPT_DIR="@XEN_SCRIPT_DIR@"
+XEN_LOCK_DIR="@XEN_LOCK_DIR@"
+XEN_RUN_DIR="@XEN_RUN_DIR@"
+XEN_PAGING_DIR="@XEN_PAGING_DIR@"
+XEN_DUMP_DIR="@XEN_DUMP_DIR@"
+XEN_LOG_DIR="@XEN_LOG_DIR@"
+XEN_LIB_DIR="@XEN_LIB_DIR@"
+XEN_RUN_STORED="@XEN_RUN_STORED@"
Signature · 1 line
Anthony PERARD
#10 lore raw

Re: [XEN PATCH for-4.17 v5 06/17] tools/hotplug: Generate "hotplugpath.sh" with configure

Quoted 1 line · Anthony Perard

It occurs to me that with this sorted, there's only a single remaining user of buildmakevars2file, and that's for stubdompath.sh.  It would be great if we could convert that final user too.

~Andrew

#11 lore raw

PATCH v5 7/17 libs/light/gentypes.py: allow to generate headers in subdirectory

In reply to Anthony PERARD

This doesn't matter yet but it will when for example the script will be run from tools/ to generate files tools/libs/light/.

---

Notes:
v4:
- new patch

 tools/libs/light/gentypes.py | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
tools/libs/light/gentypes.py +6 -3
index 9a45e45acc..3fe3873242 100644
@@ -584,6 +584,9 @@ def libxl_C_enum_from_string(ty, str, e, indent = "    "):
         s = indent + s
     return s.replace("\n", "\n%s" % indent).rstrip(indent)
 
+def clean_header_define(header_path):
+    return header_path.split('/')[-1].upper().replace('.','_')
+
 
 if __name__ == '__main__':
     if len(sys.argv) != 6:
@@ -598,7 +601,7 @@ if __name__ == '__main__':
 
     f = open(header, "w")
 
-    header_define = header.upper().replace('.','_')
+    header_define = clean_header_define(header)
     f.write("""#ifndef %s
 #define %s
 
@@ -648,7 +651,7 @@ if __name__ == '__main__':
 
     f = open(header_json, "w")
 
-    header_json_define = header_json.upper().replace('.','_')
+    header_json_define = clean_header_define(header_json)
     f.write("""#ifndef %s
 #define %s
 
@@ -672,7 +675,7 @@ if __name__ == '__main__':
 
     f = open(header_private, "w")
 
-    header_private_define = header_private.upper().replace('.','_')
+    header_private_define = clean_header_define(header_private)
     f.write("""#ifndef %s
 #define %s
 
Signature · 1 line
Anthony PERARD
#12 lore raw

PATCH v5 8/17 git-checkout.sh: handle running git-checkout from a different directory

In reply to Anthony PERARD

"$DIR" might not be a full path and it might not have pwd as ".." directory. So use cd - to undo the first cd command.

Also, use basename to make a symbolic link with a relative path.

This doesn't matter yet but it will when for example the commands to clone OVMF is been run from tools/ rather than tools/firmware/.

---

Notes:
v4:
- new patch

 scripts/git-checkout.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
scripts/git-checkout.sh +2 -2
index 20ae31ff23..fd4425ac4e 100755
@@ -19,9 +19,9 @@ if test \! -d $DIR-remote; then
 		cd $DIR-remote.tmp
 		$GIT branch -D dummy >/dev/null 2>&1 ||:
 		$GIT checkout -b dummy $TAG
-		cd ..
+		cd -
 	fi
 	mv $DIR-remote.tmp $DIR-remote
 fi
 rm -f $DIR
-ln -sf $DIR-remote $DIR
+ln -sf $(basename $DIR-remote) $DIR
Signature · 1 line
Anthony PERARD
#13 lore raw

PATCH v5 9/17 libs: Avoid exposing -Wl,--version-script to other built library

In reply to Anthony PERARD

$(SHLIB_LDFLAGS) is used by more targets that the single targets that except it (libxenfoo.so.X.Y). There is also some dynamic libraries in stats/ that uses $(SHLIB_LDFLAGS) (even if those are never built), and there's libxenlight_test.so which doesn't needs a version script.

Also, libxenlight_test.so might failed to build if the version script doesn't exist yet.

For these reasons, avoid changing the generic $(SHLIB_LDFLAGS) flags, and add the flag directly on the command line.

---

Notes:
v4:
- new patch

 tools/libs/libs.mk | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
tools/libs/libs.mk +1 -3
index e47fb30ed4..3eb91fc8f3 100644
@@ -12,8 +12,6 @@ MAJOR := $(shell $(XEN_ROOT)/version.sh $(XEN_ROOT)/xen/Makefile)
 endif
 MINOR ?= 0
 
-SHLIB_LDFLAGS += -Wl,--version-script=libxen$(LIBNAME).map
-
 CFLAGS   += -Wmissing-prototypes
 CFLAGS   += $(CFLAGS_xeninclude)
 CFLAGS   += $(foreach lib, $(USELIBS_$(LIBNAME)), $(CFLAGS_libxen$(lib)))
@@ -85,7 +83,7 @@ lib$(LIB_FILE_NAME).so.$(MAJOR): lib$(LIB_FILE_NAME).so.$(MAJOR).$(MINOR)
 	$(SYMLINK_SHLIB) $< $@
 
 lib$(LIB_FILE_NAME).so.$(MAJOR).$(MINOR): $(PIC_OBJS) libxen$(LIBNAME).map
-	$(CC) $(LDFLAGS) $(PTHREAD_LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,lib$(LIB_FILE_NAME).so.$(MAJOR) $(SHLIB_LDFLAGS) -o $@ $(PIC_OBJS) $(LDLIBS) $(APPEND_LDFLAGS)
+	$(CC) $(LDFLAGS) $(PTHREAD_LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,lib$(LIB_FILE_NAME).so.$(MAJOR) -Wl,--version-script=libxen$(LIBNAME).map $(SHLIB_LDFLAGS) -o $@ $(PIC_OBJS) $(LDLIBS) $(APPEND_LDFLAGS)
 
 # If abi-dumper is available, write out the ABI analysis
 ifneq ($(ABI_DUMPER),)
Signature · 1 line
Anthony PERARD
#14 lore raw

PATCH v5 10/17 libs: Fix auto-generation of version-script for unstable libs

In reply to Anthony PERARD

When there isn't a version-script for a shared library (like for unstable libs), we create one based on the current Xen version. But that version-script became out-of-date as soon as Xen's version changes and make as no way to regenerate the version-script on rebuild.

In order to allow regenerating the script, we'll have a different filename. In order to check if the content is up-to-date, we'll always generated it and compare.

But we also need to know if there's already an existing version script or if we need to generate one, for that we introduce $(version-script) to be used to point to the path of the existing script. (Guessing if a version script exist for a stable library with for example $(wildcard) won't work as a file will exist when building the library without this patch.)

We don't need the version-script unless we are making the shared library so it is removed from the "all" target.

---

Alternatively, we could remove the need for to generate a version-script, but that would mean the exported symbols won't have a version anymore (like before, a few versions of Xen ago). Or, we could use --default-symver, but the symboled would be versionned with the soname (e.g. "libxenctrl.so.4.16.0") instead of "VERS_4.16.0".
---

Notes:
v4:
- new patch

 tools/libs/call/Makefile          |  1 +
 tools/libs/ctrl/Makefile          |  3 ---
 tools/libs/devicemodel/Makefile   |  1 +
 tools/libs/evtchn/Makefile        |  1 +
 tools/libs/foreignmemory/Makefile |  1 +
 tools/libs/gnttab/Makefile        |  1 +
 tools/libs/guest/Makefile         |  3 ---
 tools/libs/hypfs/Makefile         |  1 +
 tools/libs/light/Makefile         |  1 -
 tools/libs/stat/Makefile          |  2 +-
 tools/libs/store/Makefile         |  1 +
 tools/libs/toolcore/Makefile      |  1 +
 tools/libs/toollog/Makefile       |  1 +
 tools/libs/util/Makefile          |  3 ---
 tools/libs/vchan/Makefile         |  3 ---
 tools/libs/libs.mk                | 19 +++++++++++++------
 .gitignore                        |  6 ------
 17 files changed, 23 insertions(+), 26 deletions(-)
tools/libs/call/Makefile +1 -0
index 103f5ad360..56a964b517 100644
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR    = 1
 MINOR    = 3
+version-script := libxencall.map
 
 include Makefile.common
 
tools/libs/ctrl/Makefile +0 -3
index 93442ab389..094e84b8d8 100644
@@ -10,6 +10,3 @@ PKG_CONFIG_NAME := Xencontrol
 NO_HEADERS_CHK := y
 
 include $(XEN_ROOT)/tools/libs/libs.mk
-
-clean::
-	rm -f libxenctrl.map
tools/libs/devicemodel/Makefile +1 -0
index b70dd774e4..20d1d112e7 100644
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR    = 1
 MINOR    = 4
+version-script := libxendevicemodel.map
 
 include Makefile.common
 
tools/libs/evtchn/Makefile +1 -0
index 3dad3840c6..18cdaab89e 100644
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR    = 1
 MINOR    = 2
+version-script := libxenevtchn.map
 
 include Makefile.common
 
tools/libs/foreignmemory/Makefile +1 -0
index b70dd774e4..81398e88b1 100644
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR    = 1
 MINOR    = 4
+version-script := libxenforeignmemory.map
 
 include Makefile.common
 
tools/libs/gnttab/Makefile +1 -0
index 3dad3840c6..4528830bdc 100644
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR    = 1
 MINOR    = 2
+version-script := libxengnttab.map
 
 include Makefile.common
 
tools/libs/guest/Makefile +0 -3
index 19d3ff2fdb..93338a9301 100644
@@ -14,6 +14,3 @@ NO_HEADERS_CHK := y
 include $(XEN_ROOT)/tools/libs/libs.mk
 
 libxenguest.so.$(MAJOR).$(MINOR): LDLIBS += $(ZLIB_LIBS) -lz
-
-clean::
-	rm -f libxenguest.map
tools/libs/hypfs/Makefile +1 -0
index 630e1e6f3e..7fae5c750d 100644
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR    = 1
 MINOR    = 0
+version-script := libxenhypfs.map
 
 LDLIBS += -lz
 
tools/libs/light/Makefile +0 -1
index 13545654c2..1d5716dbba 100644
@@ -252,6 +252,5 @@ clean::
 	$(RM) testidl.c.new testidl.c *.api-ok
 	$(RM) $(TEST_PROGS) libxenlight_test.so libxl_test_*.opic
 	$(RM) -r __pycache__
-	$(RM) libxenlight.map
 	$(RM) $(AUTOSRCS) $(AUTOINCS)
 	$(MAKE) -C $(ACPI_PATH) ACPI_BUILD_DIR=$(CURDIR) clean
tools/libs/stat/Makefile +1 -1
index 7eaf50e91e..ee5c42bf7b 100644
@@ -134,4 +134,4 @@ uninstall:: uninstall-perl-bindings
 endif
 
 clean::
-	$(RM) libxenstat.map $(BINDINGS) $(BINDINGSRC)
+	$(RM) $(BINDINGS) $(BINDINGSRC)
tools/libs/store/Makefile +1 -0
index 2334c953bb..300c271eb0 100644
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR = 4
 MINOR = 0
+version-script := libxenstore.map
 
 ifeq ($(CONFIG_Linux),y)
 LDLIBS += -ldl
tools/libs/toolcore/Makefile +1 -0
index 0d92b68b3b..20671dadd0 100644
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR	= 1
 MINOR	= 0
+version-script := libxentoolcore.map
 
 LIBHEADER := xentoolcore.h
 
tools/libs/toollog/Makefile +1 -0
index 2361b8cbf1..d612227c85 100644
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR	= 1
 MINOR	= 0
+version-script := libxentoollog.map
 
 include Makefile.common
 
tools/libs/util/Makefile +0 -3
index 493d2e00be..e016baf888 100644
@@ -47,6 +47,3 @@ $(OBJS-y) $(PIC_OBJS): $(AUTOINCS)
 %.c %.h:: %.l
 	@rm -f $*.[ch]
 	$(FLEX) --header-file=$*.h --outfile=$*.c $<
-
-clean::
-	$(RM) libxenutil.map
tools/libs/vchan/Makefile +0 -3
index ac2bff66f5..a1ef60ac8e 100644
@@ -11,6 +11,3 @@ OBJS-y += io.o
 NO_HEADERS_CHK := y
 
 include $(XEN_ROOT)/tools/libs/libs.mk
-
-clean::
-	rm -f libxenvchan.map
tools/libs/libs.mk +13 -6
index 3eb91fc8f3..cab8e9704a 100644
@@ -4,6 +4,8 @@
 #   PKG_CONFIG: name of pkg-config file (xen$(LIBNAME).pc if empty)
 #   MAJOR:   major version of lib (Xen version if empty)
 #   MINOR:   minor version of lib (0 if empty)
+#   version-script: Specify the name of a version script to the linker.
+#     (If empty, a temporary one for unstable library is created)
 
 LIBNAME := $(notdir $(CURDIR))
 
@@ -12,6 +14,10 @@ MAJOR := $(shell $(XEN_ROOT)/version.sh $(XEN_ROOT)/xen/Makefile)
 endif
 MINOR ?= 0
 
+ifeq ($(origin version-script), undefined)
+version-script := libxen$(LIBNAME).map.tmp
+endif
+
 CFLAGS   += -Wmissing-prototypes
 CFLAGS   += $(CFLAGS_xeninclude)
 CFLAGS   += $(foreach lib, $(USELIBS_$(LIBNAME)), $(CFLAGS_libxen$(lib)))
@@ -53,7 +59,7 @@ $(PKG_CONFIG_LOCAL): PKG_CONFIG_INCDIR = $(XEN_INCLUDE)
 $(PKG_CONFIG_LOCAL): PKG_CONFIG_LIBDIR = $(CURDIR)
 
 .PHONY: all
-all: $(TARGETS) $(PKG_CONFIG_LOCAL) libxen$(LIBNAME).map $(LIBHEADERS)
+all: $(TARGETS) $(PKG_CONFIG_LOCAL) $(LIBHEADERS)
 
 ifneq ($(NO_HEADERS_CHK),y)
 all: headers.chk
@@ -71,8 +77,9 @@ headers.lst: FORCE
 	@{ set -e; $(foreach h,$(LIBHEADERS),echo $(h);) } > [email protected]
 	@$(call move-if-changed,[email protected],$@)
 
-libxen$(LIBNAME).map:
-	echo 'VERS_$(MAJOR).$(MINOR) { global: *; };' >$@
+libxen$(LIBNAME).map.tmp: FORCE
+	echo 'VERS_$(MAJOR).$(MINOR) { global: *; };' >[email protected]
+	$(call move-if-changed,[email protected],$@)
 
 lib$(LIB_FILE_NAME).a: $(OBJS-y)
 	$(AR) rc $@ $^
@@ -82,8 +89,8 @@ lib$(LIB_FILE_NAME).so: lib$(LIB_FILE_NAME).so.$(MAJOR)
 lib$(LIB_FILE_NAME).so.$(MAJOR): lib$(LIB_FILE_NAME).so.$(MAJOR).$(MINOR)
 	$(SYMLINK_SHLIB) $< $@
 
-lib$(LIB_FILE_NAME).so.$(MAJOR).$(MINOR): $(PIC_OBJS) libxen$(LIBNAME).map
-	$(CC) $(LDFLAGS) $(PTHREAD_LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,lib$(LIB_FILE_NAME).so.$(MAJOR) -Wl,--version-script=libxen$(LIBNAME).map $(SHLIB_LDFLAGS) -o $@ $(PIC_OBJS) $(LDLIBS) $(APPEND_LDFLAGS)
+lib$(LIB_FILE_NAME).so.$(MAJOR).$(MINOR): $(PIC_OBJS) $(version-script)
+	$(CC) $(LDFLAGS) $(PTHREAD_LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,lib$(LIB_FILE_NAME).so.$(MAJOR) -Wl,--version-script=$(version-script) $(SHLIB_LDFLAGS) -o $@ $(PIC_OBJS) $(LDLIBS) $(APPEND_LDFLAGS)
 
 # If abi-dumper is available, write out the ABI analysis
 ifneq ($(ABI_DUMPER),)
@@ -122,7 +129,7 @@ TAGS:
 clean::
 	rm -rf $(TARGETS) *~ $(DEPS_RM) $(OBJS-y) $(PIC_OBJS)
 	rm -f lib$(LIB_FILE_NAME).so.$(MAJOR).$(MINOR) lib$(LIB_FILE_NAME).so.$(MAJOR)
-	rm -f headers.chk headers.lst
+	rm -f headers.chk headers.lst libxen*.map.tmp
 
 .PHONY: distclean
 distclean: clean
.gitignore +0 -6
index 418bdfaebf..7febfc224d 100644
@@ -106,8 +106,6 @@ tools/config.cache
 config/Tools.mk
 config/Stubdom.mk
 config/Docs.mk
-tools/libs/ctrl/libxenctrl.map
-tools/libs/guest/libxenguest.map
 tools/libs/guest/xc_bitops.h
 tools/libs/guest/xc_core.h
 tools/libs/guest/xc_core_arm.h
@@ -117,7 +115,6 @@ tools/libs/light/_*.[ch]
 tools/libs/light/*.pyc
 tools/libs/light/_libxl.api-for-check
 tools/libs/light/*.api-ok
-tools/libs/light/libxenlight.map
 tools/libs/light/libxl-save-helper
 tools/libs/light/dsdt*
 tools/libs/light/mk_dsdt
@@ -127,13 +124,10 @@ tools/libs/light/testidl.c
 tools/libs/light/test_timedereg
 tools/libs/light/test_fdderegrace
 tools/libs/light/tmp.*
-tools/libs/stat/libxenstat.map
 tools/libs/store/list.h
 tools/libs/store/utils.h
 tools/libs/store/xs_lib.c
 tools/libs/util/libxlu_cfg_y.output
-tools/libs/util/libxenutil.map
-tools/libs/vchan/libxenvchan.map
 tools/debugger/gdb/gdb-6.2.1-linux-i386-xen/*
 tools/debugger/gdb/gdb-6.2.1/*
 tools/debugger/gdb/gdb-6.2.1.tar.bz2
Signature · 1 line
Anthony PERARD
#15 lore raw

Re: [XEN PATCH for-4.17 v5 10/17] libs: Fix auto-generation of version-script for unstable libs

Quoted 20 lines · Anthony Perard

When there isn't a version-script for a shared library (like for unstable libs), we create one based on the current Xen version. But that version-script became out-of-date as soon as Xen's version changes and make as no way to regenerate the version-script on rebuild.

In order to allow regenerating the script, we'll have a different filename. In order to check if the content is up-to-date, we'll always generated it and compare.

But we also need to know if there's already an existing version script or if we need to generate one, for that we introduce $(version-script) to be used to point to the path of the existing script. (Guessing if a version script exist for a stable library with for example $(wildcard) won't work as a file will exist when building the library without this patch.)

We don't need the version-script unless we are making the shared library so it is removed from the "all" target.

Quoted 8 lines

---

Alternatively, we could remove the need for to generate a version-script, but that would mean the exported symbols won't have a version anymore (like before, a few versions of Xen ago). Or, we could use --default-symver, but the symboled would be versionned with the soname (e.g. "libxenctrl.so.4.16.0") instead of "VERS_4.16.0".

It turns out that the use of VERS_* everywhere is buggy.  We should have had 'XEN$FOO' in the soname for libraries.  But it's too late for the stable ones.

As far as the unstable libraries go, it really doesn't matter.  The only thing we care about (rightly or wrongly) is that you can't mistake the libraries from different versions of Xen.

--default-symver sounds like it would be ideal, and avoid writing out temporary version scripts.

~Andrew

#16 lore raw

Re: [XEN PATCH for-4.17 v5 10/17] libs: Fix auto-generation of version-script for unstable libs

Quoted 11 lines · Andrew Cooper
Quoted 5 lines · Anthony Perard

Alternatively, we could remove the need for to generate a version-script, but that would mean the exported symbols won't have a version anymore (like before, a few versions of Xen ago). Or, we could use --default-symver, but the symboled would be versionned with the soname (e.g. "libxenctrl.so.4.16.0") instead of "VERS_4.16.0".

It turns out that the use of VERS_* everywhere is buggy.  We should have had 'XEN$FOO' in the soname for libraries.  But it's too late for the stable ones.

Quoted 6 lines

As far as the unstable libraries go, it really doesn't matter.  The only thing we care about (rightly or wrongly) is that you can't mistake the libraries from different versions of Xen.

--default-symver sounds like it would be ideal, and avoid writing out temporary version scripts.

Sounds good to me, I'll rewrite the patch to use --default-symver. Hopefully, that's an old enough option.

Thanks,

Signature · 1 line
Anthony PERARD
#17 lore raw

PATCH v5 11/17 tools/include: Rework Makefile

In reply to Anthony PERARD

Rework "xen-xsm" rules to not have to change directory to run mkflask.sh, and store mkflask.sh path in a var, and use a full path for FLASK_H_DEPEND, and output directory is made relative.

Rename "all-y" target to a more descriptive "xen/lib/x86/all".

Removed the "dist" target which was the only one existing in tools/.

---

Notes:
v4:
- new patch

 tools/include/Makefile | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)
tools/include/Makefile +15 -13
index b488f7ca9f..81c3d09039 100644
@@ -7,17 +7,20 @@ include $(XEN_ROOT)/tools/Rules.mk
 # taken into account, i.e. there should be no rules added here for generating
 # any tools/include/*.h files.
 
-# Relative to $(XEN_ROOT)/xen/xsm/flask
-FLASK_H_DEPEND := policy/initial_sids
+.PHONY: all
+all: xen-foreign xen-dir xen-xsm/.dir
+ifeq ($(CONFIG_X86),y)
+all: xen/lib/x86/all
+endif
 
-.PHONY: all all-y build xen-dir
-all build: all-y xen-foreign xen-dir xen-xsm/.dir
-all-y:
+.PHONY: build
+build: all
 
 .PHONY: xen-foreign
 xen-foreign:
 	$(MAKE) -C xen-foreign
 
+.PHONY: xen-dir
 xen-dir:
 	mkdir -p xen/libelf acpi
 	find xen/ acpi/ -type l -exec rm '{}' +
@@ -36,16 +39,18 @@ ifeq ($(CONFIG_X86),y)
 	ln -s $(XEN_ROOT)/xen/include/xen/lib/x86/Makefile xen/lib/x86/
 endif
 
-all-$(CONFIG_X86): xen-dir
+.PHONY: xen/lib/x86/all
+xen/lib/x86/all: xen-dir
 	$(MAKE) -C xen/lib/x86 all XEN_ROOT=$(XEN_ROOT) PYTHON=$(PYTHON)
 
+MKFLASK := $(XEN_ROOT)/xen/xsm/flask/policy/mkflask.sh
+FLASK_H_DEPEND := $(XEN_ROOT)/xen/xsm/flask/policy/initial_sids
+
 # Not xen/xsm as that clashes with link to
 # $(XEN_ROOT)/xen/include/public/xsm above.
-xen-xsm/.dir: $(XEN_ROOT)/xen/xsm/flask/policy/mkflask.sh \
-	      $(patsubst %,$(XEN_ROOT)/xen/xsm/flask/%,$(FLASK_H_DEPEND))
+xen-xsm/.dir: $(MKFLASK) $(FLASK_H_DEPEND)
 	mkdir -p xen-xsm/flask
-	cd $(XEN_ROOT)/xen/xsm/flask/ && \
-		$(SHELL) policy/mkflask.sh $(AWK) $(CURDIR)/xen-xsm/flask $(FLASK_H_DEPEND)
+	$(SHELL) $(MKFLASK) $(AWK) xen-xsm/flask $(FLASK_H_DEPEND)
 	touch $@
 
 .PHONY: install
@@ -84,8 +89,5 @@ clean:
 	$(MAKE) -C xen-foreign clean
 	rm -f _*.h
 
-.PHONY: dist
-dist: install
-
 .PHONY: distclean
 distclean: clean
Signature · 1 line
Anthony PERARD
#18 lore raw

PATCH v5 12/17 libs/light: Rework acpi table build targets

In reply to Anthony PERARD

Currently, a rebuild of libxl will always rebuild for example "build.o". This is because the target depends on "acpi" which never exist. So instead we will have "build.o" have as prerequisites targets that are actually generated by "acpi", that is $(DSDT_FILES-y).

While "dsdt_*.c" isn't really a dependency for "build.o", a side effect of building that dsdt_*.c is to also generate the "ssdt_*.h" that "build.o" needs, but I don't want to list all the headers needed by "build.o" and duplicate the information available in "libacpi/Makefile" at this time.

Also avoid duplicating the "acpi" target for Arm, and unique one for both architecture. And move the "acpi" target to be with other targets rather than in the middle of the source listing. For the same reason, move the prerequisites listing for both $(DSDT_FILES-y) and "build.o".

---

Notes:
v4:
- new patch

 tools/libs/light/Makefile | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
tools/libs/light/Makefile +11 -7
index 1d5716dbba..4a8b0f7bcf 100644
@@ -32,14 +32,10 @@ ACPI_PATH  = $(XEN_ROOT)/tools/libacpi
 DSDT_FILES-$(CONFIG_X86) = dsdt_pvh.c
 ACPI_OBJS  = $(patsubst %.c,%.o,$(DSDT_FILES-y)) build.o static_tables.o
 ACPI_PIC_OBJS = $(patsubst %.o,%.opic,$(ACPI_OBJS))
-$(DSDT_FILES-y) build.o build.opic: acpi
+
 vpath build.c $(ACPI_PATH)/
 vpath static_tables.c $(ACPI_PATH)/
 
-.PHONY: acpi
-acpi:
-	$(MAKE) -C $(ACPI_PATH) ACPI_BUILD_DIR=$(CURDIR) DSDT_FILES="$(DSDT_FILES-y)"
-
 OBJS-$(CONFIG_X86) += $(ACPI_OBJS)
 
 CFLAGS += -Wno-format-zero-length -Wmissing-declarations \
@@ -58,8 +54,6 @@ ifeq ($(CONFIG_ARM_64),y)
 DSDT_FILES-y = dsdt_anycpu_arm.c
 OBJS-y += libxl_arm_acpi.o
 OBJS-y += $(DSDT_FILES-y:.c=.o)
-dsdt_anycpu_arm.c:
-	$(MAKE) -C $(ACPI_PATH) ACPI_BUILD_DIR=$(CURDIR) DSDT_FILES="$(DSDT_FILES-y)"
 else
 OBJS-$(CONFIG_ARM) += libxl_arm_no_acpi.o
 endif
@@ -191,6 +185,12 @@ all: $(CLIENTS) $(TEST_PROGS) $(AUTOSRCS) $(AUTOINCS)
 
 $(OBJS-y) $(PIC_OBJS) $(SAVE_HELPER_OBJS) $(LIBXL_TEST_OBJS) $(TEST_PROG_OBJS): $(AUTOINCS) libxl.api-ok
 
+$(DSDT_FILES-y): acpi
+
+# Depends on the source files generated by the "acpi" target even if "build.o"
+# don't needs them but do need the headers that are also generated by "acpi".
+build.o build.opic: $(DSDT_FILES-y)
+
 libxl.api-ok: check-libxl-api-rules _libxl.api-for-check
 	$(PERL) $^
 	touch $@
@@ -227,6 +227,10 @@ _libxl_type%.h _libxl_type%_json.h _libxl_type%_private.h _libxl_type%.c: libxl_
 $(XEN_INCLUDE)/_%.h: _%.h
 	$(call move-if-changed,_$*.h,$(XEN_INCLUDE)/_$*.h)
 
+.PHONY: acpi
+acpi:
+	$(MAKE) -C $(ACPI_PATH) ACPI_BUILD_DIR=$(CURDIR) DSDT_FILES="$(DSDT_FILES-y)"
+
 libxenlight_test.so: $(PIC_OBJS) $(LIBXL_TEST_OBJS)
 	$(CC) $(LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,libxenlight.so.$(MAJOR) $(SHLIB_LDFLAGS) -o $@ $^ $(LDLIBS) $(APPEND_LDFLAGS)
 
Signature · 1 line
Anthony PERARD
#19 lore raw

PATCH v5 13/17 libs/light: Rework generation of include/_libxl_*.h

In reply to Anthony PERARD

Instead of moving the public "_libxl_*.h" headers, we make a copy to the destination so that make doesn't try to remake the targets "_libxl_*.h" in libs/light/ again.

A new .PRECIOUS target is added to tell make to not deletes the intermediate targets generated by "gentypes.py".

---

Notes:
v4:
- new patch

 tools/libs/light/Makefile | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/libs/light/Makefile +6 -2
index 4a8b0f7bcf..9329055c98 100644
@@ -215,6 +215,8 @@ libxl_internal_json.h: _libxl_types_internal_json.h
 $(OBJS-y) $(PIC_OBJS) $(LIBXL_TEST_OBJS) $(TEST_PROG_OBJS) $(SAVE_HELPER_OBJS): $(XEN_INCLUDE)/libxl.h
 $(OBJS-y) $(PIC_OBJS) $(LIBXL_TEST_OBJS): libxl_internal.h
 
+# This exploits the 'multi-target pattern rule' trick.
+# gentypes.py should be executed only once to make all the targets.
 _libxl_type%.h _libxl_type%_json.h _libxl_type%_private.h _libxl_type%.c: libxl_type%.idl gentypes.py idl.py
 	$(eval stem = $(notdir $*))
 	$(PYTHON) gentypes.py libxl_type$(stem).idl __libxl_type$(stem).h __libxl_type$(stem)_private.h \
@@ -224,8 +226,10 @@ _libxl_type%.h _libxl_type%_json.h _libxl_type%_private.h _libxl_type%.c: libxl_
 	$(call move-if-changed,__libxl_type$(stem)_json.h,_libxl_type$(stem)_json.h)
 	$(call move-if-changed,__libxl_type$(stem).c,_libxl_type$(stem).c)
 
-$(XEN_INCLUDE)/_%.h: _%.h
-	$(call move-if-changed,_$*.h,$(XEN_INCLUDE)/_$*.h)
+.PRECIOUS: _libxl_type%.h _libxl_type%.c
+
+$(XEN_INCLUDE)/_libxl_%.h: _libxl_%.h
+	cp -f $< $@
 
 .PHONY: acpi
 acpi:
Signature · 1 line
Anthony PERARD
#20 lore raw

PATCH v5 14/17 libs/light: Rework targets prerequisites

In reply to Anthony PERARD

No need for $(AUTOSRCS), GNU make can generate them as needed when trying to build them as needed when trying to build the object. Also, those two AUTOSRCS don't need to be a prerequisite of "all". As for the "clean" target, those two files are already removed via "_*.c".

We don't need $(AUTOINCS) either:
- As for both _libxl_savm_msgs*.h headers, we are adding more selective dependencies so the headers will still be generated as needed.
- "clean" rule already delete the _*.h files, so AUTOINCS aren't needed there.

"libxl_internal_json.h" doesn't seems to have ever existed, so the dependency is removed.

Add few prerequisite for "libxl_internal.h" so all headers that it depends on should be generated. And have $(SAVE_HELPER_OBJS) depends on "libxl_internal.h".

Rework objects prerequisites, to have them dependents on either "libxl.h" or "libxl_internal.h". "libxl.h" is not normally included directly in the source code as "libxl_internal.h" is used instead. But we are adding "libxl.h" as prerequisite of "libxl_internal.h", so generated headers will still be generated as needed.

"testidl.c" doesn't depends on "libxl.h" but "testidl.o" does. Also use automatic variables $< and $@.

---

Notes:
v4:
- new patch

 tools/libs/light/Makefile | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)
tools/libs/light/Makefile +17 -12
index 9329055c98..274e8350bb 100644
@@ -147,9 +147,6 @@ LIBXL_TEST_OBJS += $(foreach t, $(LIBXL_TESTS_INSIDE),libxl_test_$t.opic)
 TEST_PROG_OBJS += $(foreach t, $(LIBXL_TESTS_PROGS),test_$t.o) test_common.o
 TEST_PROGS += $(foreach t, $(LIBXL_TESTS_PROGS),test_$t)
 
-AUTOINCS = _libxl_save_msgs_callout.h _libxl_save_msgs_helper.h
-AUTOSRCS = _libxl_save_msgs_callout.c _libxl_save_msgs_helper.c
-
 CLIENTS = testidl libxl-save-helper
 
 SAVE_HELPER_OBJS = libxl_save_helper.o _libxl_save_msgs_helper.o
@@ -177,13 +174,13 @@ libxl_x86_acpi.o libxl_x86_acpi.opic: CFLAGS += -I$(XEN_ROOT)/tools
 $(SAVE_HELPER_OBJS): CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenevtchn) $(CFLAGS_libxenguest)
 
 testidl.o: CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenlight)
-testidl.c: libxl_types.idl gentest.py $(XEN_INCLUDE)/libxl.h $(AUTOINCS)
-	$(PYTHON) gentest.py libxl_types.idl testidl.c.new
-	mv testidl.c.new testidl.c
+testidl.c: libxl_types.idl gentest.py
+	$(PYTHON) gentest.py $< [email protected]
+	mv -f [email protected] $@
 
-all: $(CLIENTS) $(TEST_PROGS) $(AUTOSRCS) $(AUTOINCS)
+all: $(CLIENTS) $(TEST_PROGS)
 
-$(OBJS-y) $(PIC_OBJS) $(SAVE_HELPER_OBJS) $(LIBXL_TEST_OBJS) $(TEST_PROG_OBJS): $(AUTOINCS) libxl.api-ok
+$(OBJS-y) $(PIC_OBJS) $(SAVE_HELPER_OBJS) $(LIBXL_TEST_OBJS) $(TEST_PROG_OBJS): libxl.api-ok
 
 $(DSDT_FILES-y): acpi
 
@@ -195,7 +192,7 @@ libxl.api-ok: check-libxl-api-rules _libxl.api-for-check
 	$(PERL) $^
 	touch $@
 
-_libxl.api-for-check: $(XEN_INCLUDE)/libxl.h $(AUTOINCS)
+_libxl.api-for-check: $(XEN_INCLUDE)/libxl.h
 	$(CC) $(CPPFLAGS) $(CFLAGS) -c -E $< $(APPEND_CFLAGS) \
 		-DLIBXL_EXTERNAL_CALLERS_ONLY=LIBXL_EXTERNAL_CALLERS_ONLY \
 		>[email protected]
@@ -207,13 +204,22 @@ _libxl_save_msgs_helper.h _libxl_save_msgs_callout.h: \
 	$(PERL) -w $< $@ >[email protected]
 	$(call move-if-changed,[email protected],$@)
 
+#
+# headers dependencies on generated headers
+#
 $(XEN_INCLUDE)/libxl.h: $(XEN_INCLUDE)/_libxl_types.h
 $(XEN_INCLUDE)/libxl_json.h: $(XEN_INCLUDE)/_libxl_types_json.h
+libxl_internal.h: $(XEN_INCLUDE)/libxl.h $(XEN_INCLUDE)/libxl_json.h
 libxl_internal.h: _libxl_types_internal.h _libxl_types_private.h _libxl_types_internal_private.h
-libxl_internal_json.h: _libxl_types_internal_json.h
+libxl_internal.h: _libxl_save_msgs_callout.h
 
-$(OBJS-y) $(PIC_OBJS) $(LIBXL_TEST_OBJS) $(TEST_PROG_OBJS) $(SAVE_HELPER_OBJS): $(XEN_INCLUDE)/libxl.h
+#
+# objects dependencies on headers that depends on generated headers
+#
+$(TEST_PROG_OBJS): $(XEN_INCLUDE)/libxl.h
 $(OBJS-y) $(PIC_OBJS) $(LIBXL_TEST_OBJS): libxl_internal.h
+$(SAVE_HELPER_OBJS): $(XEN_INCLUDE)/libxl.h _libxl_save_msgs_helper.h
+testidl.o: $(XEN_INCLUDE)/libxl.h
 
 # This exploits the 'multi-target pattern rule' trick.
 # gentypes.py should be executed only once to make all the targets.
@@ -260,5 +266,4 @@ clean::
 	$(RM) testidl.c.new testidl.c *.api-ok
 	$(RM) $(TEST_PROGS) libxenlight_test.so libxl_test_*.opic
 	$(RM) -r __pycache__
-	$(RM) $(AUTOSRCS) $(AUTOINCS)
 	$(MAKE) -C $(ACPI_PATH) ACPI_BUILD_DIR=$(CURDIR) clean
Signature · 1 line
Anthony PERARD
#21 lore raw

Re: [XEN PATCH for-4.17 v5 14/17] libs/light: Rework targets prerequisites

Quoted 19 lines · Anthony Perard

No need for $(AUTOSRCS), GNU make can generate them as needed when trying to build them as needed when trying to build the object. Also, those two AUTOSRCS don't need to be a prerequisite of "all". As for the "clean" target, those two files are already removed via "_*.c".

We don't need $(AUTOINCS) either:
- As for both _libxl_savm_msgs*.h headers, we are adding more selective dependencies so the headers will still be generated as needed.
- "clean" rule already delete the _*.h files, so AUTOINCS aren't needed there.

"libxl_internal_json.h" doesn't seems to have ever existed, so the dependency is removed.

Add few prerequisite for "libxl_internal.h" so all headers that it depends on should be generated. And have $(SAVE_HELPER_OBJS) depends on "libxl_internal.h".

Quoted 7 lines

Rework objects prerequisites, to have them dependents on either "libxl.h" or "libxl_internal.h". "libxl.h" is not normally included directly in the source code as "libxl_internal.h" is used instead. But we are adding "libxl.h" as prerequisite of "libxl_internal.h", so generated headers will still be generated as needed.

"testidl.c" doesn't depends on "libxl.h" but "testidl.o" does.

I'm afraid I don't follow here.  How can this be true?

~Andrew

#22 lore raw

Re: [XEN PATCH for-4.17 v5 14/17] libs/light: Rework targets prerequisites

Quoted 26 lines · Andrew Cooper
Quoted 25 lines · Anthony Perard

No need for $(AUTOSRCS), GNU make can generate them as needed when trying to build them as needed when trying to build the object. Also, those two AUTOSRCS don't need to be a prerequisite of "all". As for the "clean" target, those two files are already removed via "_*.c".

We don't need $(AUTOINCS) either:
- As for both _libxl_savm_msgs*.h headers, we are adding more selective dependencies so the headers will still be generated as needed.
- "clean" rule already delete the _*.h files, so AUTOINCS aren't needed there.

"libxl_internal_json.h" doesn't seems to have ever existed, so the dependency is removed.

Add few prerequisite for "libxl_internal.h" so all headers that it depends on should be generated. And have $(SAVE_HELPER_OBJS) depends on "libxl_internal.h".

Rework objects prerequisites, to have them dependents on either "libxl.h" or "libxl_internal.h". "libxl.h" is not normally included directly in the source code as "libxl_internal.h" is used instead. But we are adding "libxl.h" as prerequisite of "libxl_internal.h", so generated headers will still be generated as needed.

Quoted 3 lines
Quoted 1 line

"testidl.c" doesn't depends on "libxl.h" but "testidl.o" does.

I'm afraid I don't follow here.  How can this be true?

From make point-of-view, in order to generate "testidl.c", we only need to execute "gentest.py" which takes "libxl_types.idl" as input. It doesn't even matter if "libxl.h" exist when generating "testidl.c" via make testidl.c.

"libxl.h" is only used later when compiling "testidl.c" into "testidl.o".

I can probably expand the commit message to better explain this.

Thanks,

Signature · 1 line
Anthony PERARD
#23 lore raw

PATCH v5 15/17 libs/light: Makefile cleanup

In reply to Anthony PERARD

Rework "libacpi.h" include in "libxl_x86_acpi.c" as to be more selective about the include path and only add "tools/libacpi/". Also "libxl_dom.c" don't use "libacpi.h" anymore. Use "-iquote" for libacpi headers.

Get rid of the weird "$(eval stem =" in the middle of a recipe and use a make automatic variable "$(*F)" instead.

---

Notes:
v4:
- new patch

 tools/libs/light/Makefile         | 16 +++++++---------
 tools/libs/light/libxl_x86_acpi.c |  2 +-
 2 files changed, 8 insertions(+), 10 deletions(-)
tools/libs/light/Makefile +7 -9
index 274e8350bb..250cc3bd2c 100644
@@ -169,8 +169,7 @@ LDLIBS += $(LDLIBS-y)
 $(OBJS-y) $(PIC_OBJS) $(LIBXL_TEST_OBJS): CFLAGS += $(CFLAGS_LIBXL) -include $(XEN_ROOT)/tools/config.h
 $(ACPI_OBJS) $(ACPI_PIC_OBJS): CFLAGS += -I. -DLIBACPI_STDUTILS=\"$(CURDIR)/libxl_x86_acpi.h\"
 $(TEST_PROG_OBJS) _libxl.api-for-check: CFLAGS += $(CFLAGS_libxentoollog) $(CFLAGS_libxentoolcore)
-libxl_dom.o libxl_dom.opic: CFLAGS += -I$(XEN_ROOT)/tools  # include libacpi/x86.h
-libxl_x86_acpi.o libxl_x86_acpi.opic: CFLAGS += -I$(XEN_ROOT)/tools
+libxl_x86_acpi.o libxl_x86_acpi.opic: CFLAGS += -iquote $(ACPI_PATH)
 $(SAVE_HELPER_OBJS): CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenevtchn) $(CFLAGS_libxenguest)
 
 testidl.o: CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenlight)
@@ -224,13 +223,12 @@ testidl.o: $(XEN_INCLUDE)/libxl.h
 # This exploits the 'multi-target pattern rule' trick.
 # gentypes.py should be executed only once to make all the targets.
 _libxl_type%.h _libxl_type%_json.h _libxl_type%_private.h _libxl_type%.c: libxl_type%.idl gentypes.py idl.py
-	$(eval stem = $(notdir $*))
-	$(PYTHON) gentypes.py libxl_type$(stem).idl __libxl_type$(stem).h __libxl_type$(stem)_private.h \
-		__libxl_type$(stem)_json.h  __libxl_type$(stem).c
-	$(call move-if-changed,__libxl_type$(stem).h,_libxl_type$(stem).h)
-	$(call move-if-changed,__libxl_type$(stem)_private.h,_libxl_type$(stem)_private.h)
-	$(call move-if-changed,__libxl_type$(stem)_json.h,_libxl_type$(stem)_json.h)
-	$(call move-if-changed,__libxl_type$(stem).c,_libxl_type$(stem).c)
+	$(PYTHON) gentypes.py libxl_type$(*F).idl __libxl_type$(*F).h __libxl_type$(*F)_private.h \
+		__libxl_type$(*F)_json.h  __libxl_type$(*F).c
+	$(call move-if-changed,__libxl_type$(*F).h,_libxl_type$(*F).h)
+	$(call move-if-changed,__libxl_type$(*F)_private.h,_libxl_type$(*F)_private.h)
+	$(call move-if-changed,__libxl_type$(*F)_json.h,_libxl_type$(*F)_json.h)
+	$(call move-if-changed,__libxl_type$(*F).c,_libxl_type$(*F).c)
 
 .PRECIOUS: _libxl_type%.h _libxl_type%.c
 
tools/libs/light/libxl_x86_acpi.c +1 -1
index 57a6b63790..22eb160659 100644
@@ -16,7 +16,7 @@
 #include "libxl_arch.h"
 #include <xen/hvm/hvm_info_table.h>
 #include <xen/hvm/e820.h>
-#include "libacpi/libacpi.h"
+#include "libacpi.h"
 
  /* Number of pages holding ACPI tables */
 #define NUM_ACPI_PAGES 16
Signature · 1 line
Anthony PERARD
#24 lore raw

PATCH v5 16/17 tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go

In reply to Anthony PERARD

gengotypes.py creates both "types.gen.go" and "helpers.gen.go", but make can start gengotypes.py twice. Rework the rules so that gengotypes.py is executed only once.

Also, add the ability to provide a path to tell gengotypes.py where to put the files. This doesn't matter yet but it will when for example the script will be run from tools/ to generate the targets.

---

Notes:
v5:
- released-acked to fix occasional CI build issue

v4:
- new patch

 tools/golang/xenlight/Makefile      |  6 ++++--
 tools/golang/xenlight/gengotypes.py | 10 +++++++++-
 2 files changed, 13 insertions(+), 3 deletions(-)
tools/golang/xenlight/Makefile +4 -2
index 00e6d17f2b..c5bb6b94a8 100644
@@ -15,8 +15,10 @@ all: build
 
 GOXL_GEN_FILES = types.gen.go helpers.gen.go
 
-%.gen.go: gengotypes.py $(LIBXL_SRC_DIR)/libxl_types.idl $(LIBXL_SRC_DIR)/idl.py
-	LIBXL_SRC_DIR=$(LIBXL_SRC_DIR) $(PYTHON) gengotypes.py $(LIBXL_SRC_DIR)/libxl_types.idl
+# This exploits the 'multi-target pattern rule' trick.
+# gentypes.py should be executed only once to make all the targets.
+$(subst .gen.,.%.,$(GOXL_GEN_FILES)): gengotypes.py $(LIBXL_SRC_DIR)/libxl_types.idl $(LIBXL_SRC_DIR)/idl.py
+	LIBXL_SRC_DIR=$(LIBXL_SRC_DIR) $(PYTHON) gengotypes.py $(LIBXL_SRC_DIR)/libxl_types.idl $(@D)/types.gen.go $(@D)/helpers.gen.go
 
 # Go will do its own dependency checking, and not actuall go through
 # with the build if none of the input files have changed.
tools/golang/xenlight/gengotypes.py +9 -1
index ac1cf060dd..ff4c2ad216 100644
@@ -723,7 +723,13 @@ def xenlight_golang_fmt_name(name, exported = True):
     return words[0] + ''.join(x.title() for x in words[1:])
 
 if __name__ == '__main__':
+    if len(sys.argv) != 4:
+        print("Usage: gengotypes.py <idl> <types.gen.go> <helpers.gen.go>", file=sys.stderr)
+        sys.exit(1)
+
     idlname = sys.argv[1]
+    path_types = sys.argv[2]
+    path_helpers = sys.argv[3]
 
     (builtins, types) = idl.parse(idlname)
 
@@ -735,9 +741,11 @@ if __name__ == '__main__':
 // source: {}
 
 """.format(os.path.basename(sys.argv[0]),
-           ' '.join([os.path.basename(a) for a in sys.argv[1:]]))
+           os.path.basename(sys.argv[1]))
 
     xenlight_golang_generate_types(types=types,
+                                   path=path_types,
                                    comment=header_comment)
     xenlight_golang_generate_helpers(types=types,
+                                     path=path_helpers,
                                      comment=header_comment)
Signature · 1 line
Anthony PERARD
#25 lore raw

Re: [XEN PATCH for-4.17 v5 16/17] tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go

tools/golang/xenlight/gengotypes.py def xenlight_golang_fmt_name(name, exported = True): · line 727

Quoted 10 lines · Anthony Perard
tools/golang/xenlight/gengotypes.py +2 -0
index ac1cf060dd..ff4c2ad216 100644
@@ -723,7 +723,13 @@ def xenlight_golang_fmt_name(name, exported = True):
     return words[0] + ''.join(x.title() for x in words[1:])
 
 if __name__ == '__main__':
+    if len(sys.argv) != 4:
+        print("Usage: gengotypes.py <idl> <types.gen.go> <helpers.gen.go>", file=sys.stderr)

This breaks with Py2.7.  Needs a

from __future__ import print_function

inserting at the top.

~Andrew

#26 lore raw

Re: [XEN PATCH for-4.17 v5 16/17] tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go

Quoted 18 lines · Andrew Cooper

tools/golang/xenlight/gengotypes.py def xenlight_golang_fmt_name(name, exported = True): · line 727

Quoted 10 lines · Anthony Perard
tools/golang/xenlight/gengotypes.py +0 -0
index ac1cf060dd..ff4c2ad216 100644
@@ -723,7 +723,13 @@ def xenlight_golang_fmt_name(name, exported = True):
    return words[0] + ''.join(x.title() for x in words[1:])
 

if __name__ == '__main__':

tools/golang/xenlight/gengotypes.py +2 -0
+    if len(sys.argv) != 4:
+        print("Usage: gengotypes.py <idl> <types.gen.go> <helpers.gen.go>", file=sys.stderr)

This breaks with Py2.7. Needs a

Quoted 3 lines

from __future__ import print_function

inserting at the top.

Out of curiosity, did you notice this by inspection, or because you specifically tested Python 2.7, or because a system you were using is still actually using Python 2.7?

-George

#27 lore raw

Re: [XEN PATCH for-4.17 v5 16/17] tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go

Quoted 15 lines · George Dunlap
Quoted 15 lines · Andrew Cooper

tools/golang/xenlight/gengotypes.py def xenlight_golang_fmt_name(name, exported = True): · line 727

Quoted 10 lines · Anthony Perard
tools/golang/xenlight/gengotypes.py +0 -0
index ac1cf060dd..ff4c2ad216 100644
@@ -723,7 +723,13 @@ def xenlight_golang_fmt_name(name, exported = True):
    return words[0] + ''.join(x.title() for x in words[1:])
 

if __name__ == '__main__':

tools/golang/xenlight/gengotypes.py +2 -0
+    if len(sys.argv) != 4:
+        print("Usage: gengotypes.py <idl> <types.gen.go> <helpers.gen.go>", file=sys.stderr)

This breaks with Py2.7. Needs a

Quoted 4 lines
Quoted 3 lines

from __future__ import print_function

inserting at the top.

Out of curiosity, did you notice this by inspection, or because you specifically tested Python 2.7, or because a system you were using is still actually using Python 2.7?

Xen's build system can't actually create a build which supports Py2 and Py3, because xen.lowlevel.{xc,xs} only get built once.  It would be nice to fix this, but -ETUITS, so we state a specific version in the specfile and mock ensures there is no trace of the other one.

XenServer is in the process of trying to retire Py2, but it turns out that Xen isn't actually fully Py3 clean yet, so we use Py2 for Xen.

The build breaks because the libxl build writes the .go files even when we don't actually want go bindings in the end.

~Andrew

#28 lore raw

Re: [XEN PATCH for-4.17 v5 16/17] tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go

Quoted 30 lines · Andrew Cooper
Quoted 15 lines · George Dunlap
Quoted 15 lines · Andrew Cooper
Quoted 10 lines · Anthony Perard
tools/golang/xenlight/gengotypes.py +0 -0
index ac1cf060dd..ff4c2ad216 100644
@@ -723,7 +723,13 @@ def xenlight_golang_fmt_name(name, exported = True):
 return words[0] + ''.join(x.title() for x in words[1:])
 

if __name__ == '__main__':
+ if len(sys.argv) != 4:
+ print("Usage: gengotypes.py <idl> <types.gen.go> <helpers.gen.go>", file=sys.stderr)

This breaks with Py2.7. Needs a

Quoted 4 lines
Quoted 3 lines

from __future__ import print_function

inserting at the top.

Out of curiosity, did you notice this by inspection, or because you specifically tested Python 2.7, or because a system you were using is still actually using Python 2.7?

Xen's build system can't actually create a build which supports Py2 and Py3, because xen.lowlevel.{xc,xs} only get built once. It would be nice to fix this, but -ETUITS, so we state a specific version in the specfile and mock ensures there is no trace of the other one.

Quoted 5 lines

XenServer is in the process of trying to retire Py2, but it turns out that Xen isn't actually fully Py3 clean yet, so we use Py2 for Xen.

The build breaks because the libxl build writes the .go files even when we don't actually want go bindings in the end.

I think the generation code is looped in even when golang is disabled so that we can detect IDL changes during development, even on systems which don’t have golang installed. In theory if libxl.idl doesn’t change, it shouldn’t trigger the build? Alternately we could consider skipping the code generation on non-debug builds, since we only really need to detect changes during development.

-George

#29 lore raw

Re: [XEN PATCH for-4.17 v5 16/17] tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go

On 14/10/2022 12:42, George Dunlap wrote:

On 14 Oct 2022, at 12:37, Andrew Cooper <[email protected]<mailto:[email protected]>> wrote:

On 14/10/2022 12:24, George Dunlap wrote:
On 13 Oct 2022, at 16:00, Andrew Cooper <[email protected]<mailto:[email protected]>> wrote:

On 13/10/2022 14:05, Anthony Perard wrote:

tools/golang/xenlight/gengotypes.py +0 -0
index ac1cf060dd..ff4c2ad216 100644
@@ -723,7 +723,13 @@ def xenlight_golang_fmt_name(name, exported = True):

return words[0] + ''.join(x.title() for x in words[1:])

if __name__ == '__main__':
+ if len(sys.argv) != 4:
+ print("Usage: gengotypes.py <idl> <types.gen.go> <helpers.gen.go>", file=sys.stderr) This breaks with Py2.7. Needs a

from __future__ import print_function

inserting at the top.
Out of curiosity, did you notice this by inspection, or because you specifically tested Python 2.7, or because a system you were using is still actually using Python 2.7?

Xen's build system can't actually create a build which supports Py2 and Py3, because xen.lowlevel.{xc,xs} only get built once. It would be nice to fix this, but -ETUITS, so we state a specific version in the specfile and mock ensures there is no trace of the other one.

XenServer is in the process of trying to retire Py2, but it turns out that Xen isn't actually fully Py3 clean yet, so we use Py2 for Xen.

The build breaks because the libxl build writes the .go files even when we don't actually want go bindings in the end.

I think the generation code is looped in even when golang is disabled so that we can detect IDL changes during development, even on systems which don’t have golang installed. In theory if libxl.idl doesn’t change, it shouldn’t trigger the build? Alternately we could consider skipping the code generation on non-debug builds, since we only really need to detect changes during development.

This is a "clean" mock build, from a git archive'd tarball. Any logic like this is going to trigger unconditionally.

It's not a problem IMO - writing out one extra text file is not the slow bit of a Xen build.

~Andrew

Acked-by on v5 16/17
#30 lore raw

Re: [XEN PATCH for-4.17 v5 16/17] tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go

In reply to Anthony PERARD
Quoted 8 lines · Anthony PERARD

gengotypes.py creates both "types.gen.go" and "helpers.gen.go", but make can start gengotypes.py twice. Rework the rules so that gengotypes.py is executed only once.

Quoted 6 lines

Also, add the ability to provide a path to tell gengotypes.py where to put the files. This doesn't matter yet but it will when for example the script will be run from tools/ to generate the targets.

And you can keep the ack when you address the Python 2.7 problem.

-George

#31 lore raw

PATCH v5 17/17 tools: Rework linking options for ocaml binding libraries

In reply to Anthony PERARD

Using a full path to the C libraries when preparing one of the ocaml binding for those libraries make the binding unusable by external project. The full path is somehow embedded and reused by the external project when linking against the binding.

Instead, we will use the proper way to link a library, by using '-l'. For in-tree build, we also need to provide the search directory via '-L'.

(The search path -L are still be embedded, but at least that doesn't prevent the ocaml binding from been used.)

Related-to: xen-project/xen#96

---

Notes:
This patch only partially resolve xen-project/xen#96. At least with this patch, the ocaml binding can be used by external project, but those project will still "see" and try to use the rpath and lib search path as the -L and -W,-rpath-link are still embedded in the ocaml binding.

I'm working on a more involve series of patch which would remove the unnecessary flags. (This would mean that oxenstored and the ocaml test will need those ldflags, instead of the binding.)

v5:
- new patch

 tools/ocaml/libs/eventchn/Makefile   | 2 +-
 tools/ocaml/libs/xc/Makefile         | 2 +-
 tools/ocaml/libs/xentoollog/Makefile | 2 +-
 tools/ocaml/libs/xl/Makefile         | 2 +-
 tools/Rules.mk                       | 8 ++++++++
 5 files changed, 12 insertions(+), 4 deletions(-)
tools/ocaml/libs/eventchn/Makefile +1 -1
index 7362a28d9e..dc560ba49b 100644
@@ -8,7 +8,7 @@ OBJS = xeneventchn
 INTF = $(foreach obj, $(OBJS),$(obj).cmi)
 LIBS = xeneventchn.cma xeneventchn.cmxa
 
-LIBS_xeneventchn = $(LDLIBS_libxenevtchn)
+LIBS_xeneventchn = $(call xenlibs-ldflags-ldlibs,evtchn)
 
 all: $(INTF) $(LIBS) $(PROGRAMS)
 
tools/ocaml/libs/xc/Makefile +1 -1
index 67acc46bee..3b76e9ad7b 100644
@@ -10,7 +10,7 @@ OBJS = xenctrl
 INTF = xenctrl.cmi
 LIBS = xenctrl.cma xenctrl.cmxa
 
-LIBS_xenctrl = $(LDLIBS_libxenctrl) $(LDLIBS_libxenguest)
+LIBS_xenctrl = $(call xenlibs-ldflags-ldlibs,ctrl guest)
 
 xenctrl_OBJS = $(OBJS)
 xenctrl_C_OBJS = xenctrl_stubs
tools/ocaml/libs/xentoollog/Makefile +1 -1
index 9ede2fd124..1645b40faf 100644
@@ -13,7 +13,7 @@ OBJS = xentoollog
 INTF = xentoollog.cmi
 LIBS = xentoollog.cma xentoollog.cmxa
 
-LIBS_xentoollog = $(LDLIBS_libxentoollog)
+LIBS_xentoollog = $(call xenlibs-ldflags-ldlibs,toollog)
 
 xentoollog_OBJS = $(OBJS)
 xentoollog_C_OBJS = xentoollog_stubs
tools/ocaml/libs/xl/Makefile +1 -1
index 7c1c4edced..22d6c93aae 100644
@@ -15,7 +15,7 @@ LIBS = xenlight.cma xenlight.cmxa
 
 OCAMLINCLUDE += -I ../xentoollog
 
-LIBS_xenlight = $(LDLIBS_libxenlight)
+LIBS_xenlight = $(call xenlibs-ldflags-ldlibs,light)
 
 xenlight_OBJS = $(OBJS)
 xenlight_C_OBJS = xenlight_stubs
tools/Rules.mk +8 -0
index a165dc4bda..34d495fff7 100644
@@ -113,6 +113,14 @@ define xenlibs-ldflags
     $(foreach lib,$(1),-L$(XEN_ROOT)/tools/libs/$(lib))
 endef
 
+# Flags for linking against all Xen libraries listed in $(1) but by making use
+# of -L and -l instead of providing a path to the shared library.
+define xenlibs-ldflags-ldlibs
+    $(call xenlibs-ldflags,$(1)) \
+    $(foreach lib,$(1), -l$(FILENAME_$(lib))) \
+    $(foreach lib,$(1),$(xenlibs-ldlibs-$(lib)))
+endef
+
 define LIB_defs
  FILENAME_$(1) ?= xen$(1)
  XEN_libxen$(1) = $$(XEN_ROOT)/tools/libs/$(1)
Signature · 1 line
Anthony PERARD
#32 lore raw
In reply to Anthony PERARD
Quoted 2 lines · Anthony PERARD

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.toolstack-build-system-v5

I've done a full rebuild of this (with the 2 noted fixes) with Xapi, and it does resolve the Ocaml library linking issues.  Thanks.

So T-by.  If an allowance for 4.17 appears, I can see about getting some of the more obviously-trivial areas reviewed and committed.

~Andrew

#33 lore raw

Hi Andrew,

Quoted 8 lines

-----Original Message-----
From: Andrew Cooper <[email protected]>
Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system improvement, toward non-recursive makefiles

Quoted 2 lines · Anthony PERARD

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git

Quoted 7 lines

br.toolstack-build-system-v5

I've done a full rebuild of this (with the 2 noted fixes) with Xapi, and it does resolve the Ocaml library linking issues.  Thanks.

So T-by.  If an allowance for 4.17 appears, I can see about getting some of the more obviously-trivial areas reviewed and committed.

Since this series is quite large and just out of curiosity, do we have a pipeline or CI jobs to test all the modules that affected by the makefile touched by this series? Would be good to have some test results to boost our confidence of committing the series.

Kind regards,
Henry

Quoted 2 lines

~Andrew

#34 lore raw
Quoted 10 lines · Henry Wang
Quoted 10 lines

-----Original Message-----
From: Andrew Cooper <[email protected]>
Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system improvement, toward non-recursive makefiles

Quoted 2 lines · Anthony PERARD

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git

br.toolstack-build-system-v5

Quoted 7 lines
Quoted 5 lines

I've done a full rebuild of this (with the 2 noted fixes) with Xapi, and it does resolve the Ocaml library linking issues.  Thanks.

So T-by.  If an allowance for 4.17 appears, I can see about getting some of the more obviously-trivial areas reviewed and committed.

Thanks.

Quoted 4 lines

Since this series is quite large and just out of curiosity, do we have a pipeline or CI jobs to test all the modules that affected by the makefile touched by this series? Would be good to have some test results to boost our confidence of committing the series.

I did push my branch to Gitlab, and that the pipeline:
https://gitlab.com/xen-project/people/anthonyper/xen/-/pipelines/665987083
(private to xen-project members)
Only failures are I think the same one as on staging, test failure on arm, but no build issues.

I'm not sure that everything is tested, but most of it is.

Cheers,

Signature · 1 line
Anthony PERARD
#35 lore raw

Hi Anthony,

Quoted 19 lines

-----Original Message-----
From: Anthony PERARD <[email protected]>
Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system improvement, toward non-recursive makefiles

Quoted 13 lines · Henry Wang
Quoted 13 lines

-----Original Message-----
From: Andrew Cooper <[email protected]>
Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system improvement, toward non-recursive makefiles

Quoted 2 lines · Anthony PERARD

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git

br.toolstack-build-system-v5

I've done a full rebuild of this (with the 2 noted fixes) with Xapi, and it does resolve the Ocaml library linking issues.� Thanks.

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

So T-by.� If an allowance for 4.17 appears, I can see about getting some of the more obviously-trivial areas reviewed and committed.

Thanks.

No problem, my pleasure.

Quoted 6 lines
Quoted 5 lines

Since this series is quite large and just out of curiosity, do we have a pipeline or CI jobs to test all the modules that affected by the makefile touched by this series? Would be good to have some test results to boost our confidence of committing the series.

Quoted 6 lines

I did push my branch to Gitlab, and that the pipeline:
https://gitlab.com/xen-project/people/anthonyper/xen/-
/pipelines/665987083
(private to xen-project members)
Only failures are I think the same one as on staging, test failure on arm, but no build issues.

Sorry for the arm failure, I will try to respin the patch ASAP and hopefully the CI will be unblocked soon.

Kind regards,
Henry

Quoted 7 lines

I'm not sure that everything is tested, but most of it is.

Cheers,

--
Anthony PERARD

#36 lore raw
In reply to Anthony PERARD
Quoted 23 lines · Anthony PERARD
Quoted 8 lines · Henry Wang
Quoted 8 lines

-----Original Message-----
From: Andrew Cooper <[email protected]>
Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system improvement, toward non-recursive makefiles

Quoted 2 lines · Anthony PERARD

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git

Quoted 8 lines
Quoted 7 lines

br.toolstack-build-system-v5

I've done a full rebuild of this (with the 2 noted fixes) with Xapi, and it does resolve the Ocaml library linking issues.  Thanks.

So T-by.  If an allowance for 4.17 appears, I can see about getting some of the more obviously-trivial areas reviewed and committed.

Thanks.

Quoted 4 lines

Since this series is quite large and just out of curiosity, do we have a pipeline or CI jobs to test all the modules that affected by the makefile touched by this series? Would be good to have some test results to boost our confidence of committing the series.

Quoted 7 lines

I did push my branch to Gitlab, and that the pipeline:
https://gitlab.com/xen-project/people/anthonyper/xen/-/pipelines/665987083
(private to xen-project members)
Only failures are I think the same one as on staging, test failure on arm, but no build issues.

I'm not sure that everything is tested, but most of it is.

The XenServer specfile lists all build artefacts specifically, because we've had too many bugs where things have silently disappeared.

My testing confirms that nothing has appeared or disappeared as a result of these changes, and the functional looks pretty good.  (except for the latest shadow assertion found, which was a regression in XSA-410.  Fix is in review elsewhere.)

~Andrew

#37 lore raw

Hi Andrew,

Quoted 11 lines

-----Original Message-----
From: Andrew Cooper <[email protected]>
Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system improvement, toward non-recursive makefiles
The XenServer specfile lists all build artefacts specifically, because we've had too many bugs where things have silently disappeared.

My testing confirms that nothing has appeared or disappeared as a result of these changes, and the functional looks pretty good.  (except for the latest shadow assertion found, which was a regression in XSA-410.  Fix is in review elsewhere.)

This is good to know. Thanks for the confirmation. No problem from my side then.

Kind regards,
Henry

Quoted 2 lines

~Andrew

Acked-by on the v5 series
#38 lore raw
In reply to Anthony PERARD

On 13 Oct 2022, at 14:04, Anthony PERARD <[email protected]<mailto:[email protected]>> wrote:

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.toolstack-build-system-v5

Changes in v5:
- rebased on staging
- added "tools: Rework linking options for ocaml binding libraries"

Changes in v4:
- several new patches
- some changes to other patches listed in their changelogs

Changes in v3:
- rebased
- several new patches, starting with 13/25 "tools/libs/util: cleanup Makefile"
- introducing macros to deal with linking with in-tree xen libraries
- Add -Werror to CFLAGS for all builds in tools/

Changes in v2:
- one new patch
- other changes described in patch notes