Skip to contentSkip to conversation

debian: Fix xfsprogs FT(C)BFS

PATCH Superseded Bastian Germann 4 participants 13 messages Nov 19, 2021 – Nov 23, 2021

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover debian: Fix xfsprogs FT(C)BFS

Hi,

I am going to include these two patches for the unstable Debian upload of v5.14.0-1.

Thanks,
Bastian

Bastian Germann (2):
debian: Generate .gitcensus instead of .census (Closes: #999743)
debian: Fix FTCBFS: Skip crc32 test (Closes: #999879)

 debian/rules | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
Signature · 1 line
2.30.2
#1 lore raw

PATCH 1/2 debian: Generate .gitcensus instead of .census (Closes: #999743)

Fix the Debian build outside a git tree (e.g., Debian archive builds) by creating an empty .gitcensus instead of .census file on config.

---
 debian/rules | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
debian/rules +4 -4
index 615289b4..6d5b82a8 100755
@@ -43,15 +43,15 @@ built: dibuild config
 	$(MAKE) $(PMAKEFLAGS) default
 	touch built
 
-config: .census
-.census:
+config: .gitcensus
+.gitcensus:
 	@echo "== dpkg-buildpackage: configure" 1>&2
 	$(checkdir)
 	AUTOHEADER=/bin/true dh_autoreconf
 	dh_update_autotools_config
 	$(options) $(MAKE) $(PMAKEFLAGS) include/platform_defs.h
 	cp -f include/install-sh .
-	touch .census
+	touch .gitcensus
 
 dibuild:
 	$(checkdir)
@@ -72,7 +72,7 @@ dibuild:
 clean:
 	@echo "== dpkg-buildpackage: clean" 1>&2
 	$(checkdir)
-	-rm -f built .census mkfs/mkfs.xfs-$(bootpkg)
+	-rm -f built .gitcensus mkfs/mkfs.xfs-$(bootpkg)
 	$(MAKE) distclean
 	-rm -rf $(dirme) $(dirdev) $(dirdi)
 	-rm -f debian/*substvars debian/files* debian/*.debhelper
Signature · 1 line
2.30.2
#4 lore raw

PATCH 2/2 debian: Fix FTCBFS: Skip crc32 test (Closes: #999879)

In reply to Bastian Germann

xfsprogs fails to cross build from source, because it attempts to build its crc32 test with the build architecture compiler and thus fails finding the liburcu, which is only requested for the host architecture. While this test is useful for native builds, it is not that useful for cross builds. Skip it by pre-creating the output file.

---
 debian/rules | 3 +++
 1 file changed, 3 insertions(+)
debian/rules +3 -0
index 6d5b82a8..28cc414d 100755
@@ -40,6 +40,9 @@ build-arch: built
 build-indep: built
 built: dibuild config
 	@echo "== dpkg-buildpackage: build" 1>&2
+ifneq ($(DEB_BUILD_ARCH),$(DEB_HOST_ARCH))
+	touch --date=+3day libfrog/crc32selftest
+endif
 	$(MAKE) $(PMAKEFLAGS) default
 	touch built
 
Signature · 1 line
2.30.2
#5 lore raw

Re: [PATCH 2/2] debian: Fix FTCBFS: Skip crc32 test (Closes: #999879)

Quoted 5 lines · Bastian Germann

xfsprogs fails to cross build from source, because it attempts to build its crc32 test with the build architecture compiler and thus fails finding the liburcu, which is only requested for the host architecture. While this test is useful for native builds, it is not that useful for cross builds. Skip it by pre-creating the output file.

I don't get it. The crcselftest does not use liburcu in any way, nor does it try to link against liburcu, so it should not fail because other parts of xfsprogs use liburcu.

What's the build error that occurs?

Quoted 11 lines
---
 debian/rules | 3 +++
 1 file changed, 3 insertions(+)
debian/rules +0 -0
index 6d5b82a8..28cc414d 100755

debian/rules build-arch: built · line 47

Quoted 8 lines
debian/rules +3 -0
@@ -40,6 +40,9 @@ build-arch: built
 build-indep: built
 built: dibuild config
 	@echo "== dpkg-buildpackage: build" 1>&2
+ifneq ($(DEB_BUILD_ARCH),$(DEB_HOST_ARCH))
+	touch --date=+3day libfrog/crc32selftest
+endif
 	$(MAKE) $(PMAKEFLAGS) default
 	touch built

We need to fix the generic cross-build problem in the xfsprogs code, not slap a distro-specific build band-aid over it.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#6 lore raw

Re: [PATCH 2/2] debian: Fix FTCBFS: Skip crc32 test (Closes: #999879)

Hi Dave,

Quoted 5 lines · Dave Chinner

I don't get it. The crcselftest does not use liburcu in any way, nor does it try to link against liburcu, so it should not fail because other parts of xfsprogs use liburcu.

What's the build error that occurs?

As the build log shows, that's not technically accurate. You can find logs of test builds for various architecture combinations at
http://crossqa.debian.net/src/xfsprogs. This is also available as a link called "cross" in https://tracker.debian.org/xfsprogs.

The relevant part is:

|     [TEST]    CRC32
| In file included from crc32.c:35:
| ../include/platform_defs.h:27:10: fatal error: urcu.h: No such file or directory
|    27 | #include <urcu.h>
|       |          ^~~~~~~~
| compilation terminated.

I failed to figure a good way of dropping either include directive.

Quoted 2 lines

We need to fix the generic cross-build problem in the xfsprogs code, not slap a distro-specific build band-aid over it.

I fully agree with this in principle. However, when I fail to find that upstreamable solution, I try to at least provide a Debian-specific solution to iterate from.

Can you propose a way to drop either #include?

Helmut

#7 lore raw

Re: [PATCH 2/2] debian: Fix FTCBFS: Skip crc32 test (Closes: #999879)

Quoted 24 lines · Helmut Grohne

Hi Dave,

Quoted 5 lines · Dave Chinner

I don't get it. The crcselftest does not use liburcu in any way, nor does it try to link against liburcu, so it should not fail because other parts of xfsprogs use liburcu.

What's the build error that occurs?

As the build log shows, that's not technically accurate. You can find logs of test builds for various architecture combinations at
http://crossqa.debian.net/src/xfsprogs. This is also available as a link called "cross" in https://tracker.debian.org/xfsprogs.

The relevant part is:

|     [TEST]    CRC32
| In file included from crc32.c:35:
| ../include/platform_defs.h:27:10: fatal error: urcu.h: No such file or directory
|    27 | #include <urcu.h>
|       |          ^~~~~~~~
| compilation terminated.

I failed to figure a good way of dropping either include directive.

Quoted 8 lines
Quoted 2 lines

We need to fix the generic cross-build problem in the xfsprogs code, not slap a distro-specific build band-aid over it.

I fully agree with this in principle. However, when I fail to find that upstreamable solution, I try to at least provide a Debian-specific solution to iterate from.

Can you propose a way to drop either #include?

Frankly I don't really see the value in crc32selftest when cross compiling -- sure, the crc32c code works correctly on the build host, but that proves nothing about the (cross-)built binaries that end up in the package.

The selftest code is modular enough that it's included in xfs_io and it seems to run in under 500us even on my ancient raspberry pi 3b+ downclocked to 600mhz. Why don't we just add it to mkfs and repair? Those tools shouldn't be run all that frequently, and now we'll know immediately if crc32c is broken on a user's system.

Then we don't need the build-time selftest at all.

--D

Quoted 2 lines

Helmut

#8 lore raw

Re: [PATCH 2/2] debian: Fix FTCBFS: Skip crc32 test (Closes: #999879)

Quoted 32 lines · Darrick J. Wong
Quoted 31 lines · Helmut Grohne

Hi Dave,

Quoted 5 lines · Dave Chinner

I don't get it. The crcselftest does not use liburcu in any way, nor does it try to link against liburcu, so it should not fail because other parts of xfsprogs use liburcu.

What's the build error that occurs?

As the build log shows, that's not technically accurate. You can find logs of test builds for various architecture combinations at
http://crossqa.debian.net/src/xfsprogs. This is also available as a link called "cross" in https://tracker.debian.org/xfsprogs.

The relevant part is:

|     [TEST]    CRC32
| In file included from crc32.c:35:
| ../include/platform_defs.h:27:10: fatal error: urcu.h: No such file or directory
|    27 | #include <urcu.h>
|       |          ^~~~~~~~
| compilation terminated.

I failed to figure a good way of dropping either include directive.

Quoted 2 lines

We need to fix the generic cross-build problem in the xfsprogs code, not slap a distro-specific build band-aid over it.

I fully agree with this in principle. However, when I fail to find that upstreamable solution, I try to at least provide a Debian-specific solution to iterate from.

Quoted 6 lines
Quoted 1 line

Can you propose a way to drop either #include?

Frankly I don't really see the value in crc32selftest when cross compiling -- sure, the crc32c code works correctly on the build host, but that proves nothing about the (cross-)built binaries that end up in the package.

Yup, that's pretty much what I was getting at.

Quoted 7 lines

The selftest code is modular enough that it's included in xfs_io and it seems to run in under 500us even on my ancient raspberry pi 3b+ downclocked to 600mhz. Why don't we just add it to mkfs and repair? Those tools shouldn't be run all that frequently, and now we'll know immediately if crc32c is broken on a user's system.

Then we don't need the build-time selftest at all.

Sounds reasonable, my only concern is what do users do when xfs-repair fails the self test and they need to fix their filesystem?

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#9 lore raw

Re: [PATCH 2/2] debian: Fix FTCBFS: Skip crc32 test (Closes: #999879)

Quoted 33 lines · Dave Chinner
Quoted 32 lines · Darrick J. Wong
Quoted 31 lines · Helmut Grohne

Hi Dave,

Quoted 5 lines · Dave Chinner

I don't get it. The crcselftest does not use liburcu in any way, nor does it try to link against liburcu, so it should not fail because other parts of xfsprogs use liburcu.

What's the build error that occurs?

As the build log shows, that's not technically accurate. You can find logs of test builds for various architecture combinations at
http://crossqa.debian.net/src/xfsprogs. This is also available as a link called "cross" in https://tracker.debian.org/xfsprogs.

The relevant part is:

|     [TEST]    CRC32
| In file included from crc32.c:35:
| ../include/platform_defs.h:27:10: fatal error: urcu.h: No such file or directory
|    27 | #include <urcu.h>
|       |          ^~~~~~~~
| compilation terminated.

I failed to figure a good way of dropping either include directive.

Quoted 2 lines

We need to fix the generic cross-build problem in the xfsprogs code, not slap a distro-specific build band-aid over it.

I fully agree with this in principle. However, when I fail to find that upstreamable solution, I try to at least provide a Debian-specific solution to iterate from.

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Can you propose a way to drop either #include?

Frankly I don't really see the value in crc32selftest when cross compiling -- sure, the crc32c code works correctly on the build host, but that proves nothing about the (cross-)built binaries that end up in the package.

Yup, that's pretty much what I was getting at.

Though this is pretty simple and should just work, too:

libfrog/crc32.c +2 -1
@@ -29,10 +29,11 @@
  * match the hardware acceleration available on Intel CPUs.
  */
 
+#include <stdio.h>
+#include <sys/types.h>
 #include <inttypes.h>
 #include <asm/types.h>
 #include <sys/time.h>
-#include "platform_defs.h"
 /* For endian conversion routines */
 #include "xfs_arch.h"
 #include "crc32defs.h"

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#11 lore raw

Re: [PATCH 2/2] debian: Fix FTCBFS: Skip crc32 test (Closes: #999879)

In reply to Dave Chinner
Quoted 48 lines · Dave Chinner
Quoted 32 lines · Darrick J. Wong
Quoted 31 lines · Helmut Grohne

Hi Dave,

Quoted 5 lines · Dave Chinner

I don't get it. The crcselftest does not use liburcu in any way, nor does it try to link against liburcu, so it should not fail because other parts of xfsprogs use liburcu.

What's the build error that occurs?

As the build log shows, that's not technically accurate. You can find logs of test builds for various architecture combinations at
http://crossqa.debian.net/src/xfsprogs. This is also available as a link called "cross" in https://tracker.debian.org/xfsprogs.

The relevant part is:

|     [TEST]    CRC32
| In file included from crc32.c:35:
| ../include/platform_defs.h:27:10: fatal error: urcu.h: No such file or directory
|    27 | #include <urcu.h>
|       |          ^~~~~~~~
| compilation terminated.

I failed to figure a good way of dropping either include directive.

Quoted 2 lines

We need to fix the generic cross-build problem in the xfsprogs code, not slap a distro-specific build band-aid over it.

I fully agree with this in principle. However, when I fail to find that upstreamable solution, I try to at least provide a Debian-specific solution to iterate from.

Quoted 6 lines
Quoted 1 line

Can you propose a way to drop either #include?

Frankly I don't really see the value in crc32selftest when cross compiling -- sure, the crc32c code works correctly on the build host, but that proves nothing about the (cross-)built binaries that end up in the package.

Yup, that's pretty much what I was getting at.

Quoted 6 lines

The selftest code is modular enough that it's included in xfs_io and it seems to run in under 500us even on my ancient raspberry pi 3b+ downclocked to 600mhz. Why don't we just add it to mkfs and repair? Those tools shouldn't be run all that frequently, and now we'll know immediately if crc32c is broken on a user's system.

Quoted 5 lines
Quoted 1 line

Then we don't need the build-time selftest at all.

Sounds reasonable, my only concern is what do users do when xfs-repair fails the self test and they need to fix their filesystem?

Find a system with a non-broken CPU, or complain to their support rep or the upstream mailing list. I'd rather they do that than let repair erroneously obliterate all the metadata with incorrect crc validation, or write garbage out that fails to mount immediately after.

--D

Quoted 7 lines

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#10 lore raw

Re: [PATCH 2/2] debian: Fix FTCBFS: Skip crc32 test (Closes: #999879)

In reply to Dave Chinner
Quoted 50 lines · Dave Chinner
Quoted 33 lines · Dave Chinner
Quoted 32 lines · Darrick J. Wong
Quoted 31 lines · Helmut Grohne

Hi Dave,

Quoted 5 lines · Dave Chinner

I don't get it. The crcselftest does not use liburcu in any way, nor does it try to link against liburcu, so it should not fail because other parts of xfsprogs use liburcu.

What's the build error that occurs?

As the build log shows, that's not technically accurate. You can find logs of test builds for various architecture combinations at
http://crossqa.debian.net/src/xfsprogs. This is also available as a link called "cross" in https://tracker.debian.org/xfsprogs.

The relevant part is:

|     [TEST]    CRC32
| In file included from crc32.c:35:
| ../include/platform_defs.h:27:10: fatal error: urcu.h: No such file or directory
|    27 | #include <urcu.h>
|       |          ^~~~~~~~
| compilation terminated.

I failed to figure a good way of dropping either include directive.

Quoted 2 lines

We need to fix the generic cross-build problem in the xfsprogs code, not slap a distro-specific build band-aid over it.

I fully agree with this in principle. However, when I fail to find that upstreamable solution, I try to at least provide a Debian-specific solution to iterate from.

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Can you propose a way to drop either #include?

Frankly I don't really see the value in crc32selftest when cross compiling -- sure, the crc32c code works correctly on the build host, but that proves nothing about the (cross-)built binaries that end up in the package.

Yup, that's pretty much what I was getting at.

Though this is pretty simple and should just work, too:

libfrog/crc32.c +0 -0
@@ -29,10 +29,11 @@
  * match the hardware acceleration available on Intel CPUs.
  */

libfrog/crc32.c line 39

Quoted 10 lines
libfrog/crc32.c +2 -1
 
+#include <stdio.h>
+#include <sys/types.h>
 #include <inttypes.h>
 #include <asm/types.h>
 #include <sys/time.h>
-#include "platform_defs.h"
 /* For endian conversion routines */
 #include "xfs_arch.h"
 #include "crc32defs.h"

That works for my system, though I don't do cross builds frequently so I don't really know if it fixes those outside of my toy environment. I'll send a bigger patch to add selftest to mkfs/repair tomorrow.

--D

Quoted 7 lines

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#12 lore raw

Re: [PATCH 2/2] debian: Fix FTCBFS: Skip crc32 test (Closes: #999879)

In reply to Darrick J. Wong
Quoted 51 lines · Darrick J. Wong
Quoted 50 lines · Dave Chinner
Quoted 32 lines · Darrick J. Wong
Quoted 31 lines · Helmut Grohne

Hi Dave,

Quoted 5 lines · Dave Chinner

I don't get it. The crcselftest does not use liburcu in any way, nor does it try to link against liburcu, so it should not fail because other parts of xfsprogs use liburcu.

What's the build error that occurs?

As the build log shows, that's not technically accurate. You can find logs of test builds for various architecture combinations at
http://crossqa.debian.net/src/xfsprogs. This is also available as a link called "cross" in https://tracker.debian.org/xfsprogs.

The relevant part is:

|     [TEST]    CRC32
| In file included from crc32.c:35:
| ../include/platform_defs.h:27:10: fatal error: urcu.h: No such file or directory
|    27 | #include <urcu.h>
|       |          ^~~~~~~~
| compilation terminated.

I failed to figure a good way of dropping either include directive.

Quoted 2 lines

We need to fix the generic cross-build problem in the xfsprogs code, not slap a distro-specific build band-aid over it.

I fully agree with this in principle. However, when I fail to find that upstreamable solution, I try to at least provide a Debian-specific solution to iterate from.

Quoted 6 lines
Quoted 1 line

Can you propose a way to drop either #include?

Frankly I don't really see the value in crc32selftest when cross compiling -- sure, the crc32c code works correctly on the build host, but that proves nothing about the (cross-)built binaries that end up in the package.

Yup, that's pretty much what I was getting at.

Quoted 7 lines

The selftest code is modular enough that it's included in xfs_io and it seems to run in under 500us even on my ancient raspberry pi 3b+ downclocked to 600mhz. Why don't we just add it to mkfs and repair? Those tools shouldn't be run all that frequently, and now we'll know immediately if crc32c is broken on a user's system.

Then we don't need the build-time selftest at all.

Quoted 8 lines
Quoted 3 lines

Sounds reasonable, my only concern is what do users do when xfs-repair fails the self test and they need to fix their filesystem?

Find a system with a non-broken CPU, or complain to their support rep or the upstream mailing list. I'd rather they do that than let repair erroneously obliterate all the metadata with incorrect crc validation, or write garbage out that fails to mount immediately after.

Ok, sounds like a man page update is in order, then :)

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#2 lore raw

Re: [PATCH 1/2] debian: Generate .gitcensus instead of .census (Closes: #999743)

In reply to Bastian Germann

@Darrick: Thanks for caring about the 2nd patch. Is anyone up to review this patch?

Am 19.11.21 um 18:12 schrieb Bastian Germann:

Quoted 41 lines

Fix the Debian build outside a git tree (e.g., Debian archive builds) by creating an empty .gitcensus instead of .census file on config.

---
  debian/rules | 8 ++++----
  1 file changed, 4 insertions(+), 4 deletions(-)
debian/rules +4 -4
index 615289b4..6d5b82a8 100755
@@ -43,15 +43,15 @@ built: dibuild config
  	$(MAKE) $(PMAKEFLAGS) default
  	touch built
 
-config: .census
-.census:
+config: .gitcensus
+.gitcensus:
  	@echo "== dpkg-buildpackage: configure" 1>&2
  	$(checkdir)
  	AUTOHEADER=/bin/true dh_autoreconf
  	dh_update_autotools_config
  	$(options) $(MAKE) $(PMAKEFLAGS) include/platform_defs.h
  	cp -f include/install-sh .
-	touch .census
+	touch .gitcensus
 
  dibuild:
  	$(checkdir)
@@ -72,7 +72,7 @@ dibuild:
  clean:
  	@echo "== dpkg-buildpackage: clean" 1>&2
  	$(checkdir)
-	-rm -f built .census mkfs/mkfs.xfs-$(bootpkg)
+	-rm -f built .gitcensus mkfs/mkfs.xfs-$(bootpkg)
  	$(MAKE) distclean
  	-rm -rf $(dirme) $(dirdev) $(dirdi)
  	-rm -f debian/*substvars debian/files* debian/*.debhelper
Reviewed-by on v1 1/2
#3 lore raw

Re: [PATCH 1/2] debian: Generate .gitcensus instead of .census (Closes: #999743)

In reply to Bastian Germann
Quoted 4 lines · Bastian Germann

Fix the Debian build outside a git tree (e.g., Debian archive builds) by creating an empty .gitcensus instead of .census file on config.

Heh. I used to have a patch that more or less did this, until I switched to building development packages only from git. Sorry I forgot to come back to this one.

--D

Quoted 40 lines
---
 debian/rules | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
debian/rules +4 -4
index 615289b4..6d5b82a8 100755
@@ -43,15 +43,15 @@ built: dibuild config
 	$(MAKE) $(PMAKEFLAGS) default
 	touch built
 
-config: .census
-.census:
+config: .gitcensus
+.gitcensus:
 	@echo "== dpkg-buildpackage: configure" 1>&2
 	$(checkdir)
 	AUTOHEADER=/bin/true dh_autoreconf
 	dh_update_autotools_config
 	$(options) $(MAKE) $(PMAKEFLAGS) include/platform_defs.h
 	cp -f include/install-sh .
-	touch .census
+	touch .gitcensus
 
 dibuild:
 	$(checkdir)
@@ -72,7 +72,7 @@ dibuild:
 clean:
 	@echo "== dpkg-buildpackage: clean" 1>&2
 	$(checkdir)
-	-rm -f built .census mkfs/mkfs.xfs-$(bootpkg)
+	-rm -f built .gitcensus mkfs/mkfs.xfs-$(bootpkg)
 	$(MAKE) distclean
 	-rm -rf $(dirme) $(dirdev) $(dirdi)
 	-rm -f debian/*substvars debian/files* debian/*.debhelper
Signature · 1 line
2.30.2