Skip to contentSkip to conversation

configs/imx6ullevk_defconfig: remove typo

PATCH New Romain Naour 2 participants 8 messages Jul 27, 2020 Arm i.MX and Layerscape

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/6 configs/imx6ullevk_defconfig: remove typo

Reported by check-dotconfig.py.

Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/657885169

---
 configs/imx6ullevk_defconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
configs/imx6ullevk_defconfig +1 -1
index 4a5558dd6c..81db66268a 100644
@@ -23,7 +23,7 @@ BR2_TARGET_UBOOT=y
 BR2_TARGET_UBOOT_BOARDNAME="mx6ull_14x14_evk"
 BR2_TARGET_UBOOT_CUSTOM_VERSION=y
 BR2_TARGET_UBOOT_CUSTOM_VERSION_VALUE="2020.07"
-BR2_TARGET_UBOOT_FORMAT_DTB_IMX=y"
+BR2_TARGET_UBOOT_FORMAT_DTB_IMX=y
 BR2_TARGET_UBOOT_NEEDS_DTC=y
 
 # required tools to create the SD card image
Signature · 1 line
2.25.4
#1 lore raw

PATCH v3 2/6 configs/rock_pi_4_defconfig: remove BR2_TARGET_UBOOT_NEEDS_PYTHON

BR2_TARGET_UBOOT_NEEDS_PYTHON3 is already selected for this defconfig, BR2_TARGET_UBOOT_NEEDS_PYTHON and BR2_TARGET_UBOOT_NEEDS_PYTHON3 can't be selected at the same time.

Reported by check-dotconfig.py.

Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/657885313

---
 configs/rock_pi_4_defconfig | 1 -
 1 file changed, 1 deletion(-)
configs/rock_pi_4_defconfig +0 -1
index 22ef74be82..401a59bac4 100644
@@ -23,7 +23,6 @@ BR2_TARGET_UBOOT_CUSTOM_REPO_URL="https://github.com/amarula/u-boot-amarula.git"
 BR2_TARGET_UBOOT_CUSTOM_REPO_VERSION="cf2747360fdd1eb0d51dd01a984c7c62c7f714fe"
 BR2_TARGET_UBOOT_BOARD_DEFCONFIG="rock-pi-4-rk3399"
 BR2_TARGET_UBOOT_NEEDS_DTC=y
-BR2_TARGET_UBOOT_NEEDS_PYTHON=y
 BR2_TARGET_UBOOT_NEEDS_PYTHON3=y
 BR2_TARGET_UBOOT_NEEDS_ATF_BL31=y
 BR2_TARGET_UBOOT_NEEDS_ATF_BL31_ELF=y
Signature · 1 line
2.25.4
#2 lore raw

PATCH v3 3/6 configs/rock_pi_n10_defconfig: remove BR2_TARGET_UBOOT_NEEDS_PYTHON

In reply to Romain Naour

BR2_TARGET_UBOOT_NEEDS_PYTHON3 is already selected for this defconfig, BR2_TARGET_UBOOT_NEEDS_PYTHON and BR2_TARGET_UBOOT_NEEDS_PYTHON3 can't be selected at the same time.

Reported by check-dotconfig.py.

Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/657885314

---
 configs/rock_pi_n10_defconfig | 1 -
 1 file changed, 1 deletion(-)
configs/rock_pi_n10_defconfig +0 -1
index bfaa489a20..c622839c89 100644
@@ -23,7 +23,6 @@ BR2_TARGET_UBOOT_CUSTOM_REPO_URL="https://github.com/amarula/u-boot-amarula.git"
 BR2_TARGET_UBOOT_CUSTOM_REPO_VERSION="cf2747360fdd1eb0d51dd01a984c7c62c7f714fe"
 BR2_TARGET_UBOOT_BOARD_DEFCONFIG="rock-pi-n10-rk3399pro"
 BR2_TARGET_UBOOT_NEEDS_DTC=y
-BR2_TARGET_UBOOT_NEEDS_PYTHON=y
 BR2_TARGET_UBOOT_NEEDS_PYTHON3=y
 BR2_TARGET_UBOOT_NEEDS_ATF_BL31=y
 BR2_TARGET_UBOOT_NEEDS_ATF_BL31_ELF=y
Signature · 1 line
2.25.4
#3 lore raw

PATCH v3 4/6 support/scripts: add check-dotconfig.py

In reply to Romain Naour

For the same reason as for 50b747f212be2c9c0f7cf10c674ed488d042715c, we need to check if the generated configuration file (.config) contains all symbols present in the defconfig file.

If not there is an issue with the defconfig.

This script will be used in .gitlab-ci.yml.

Inspired by is_toolchain_usable() function from genrandconfig:
https://git.busybox.net/buildroot/tree/utils/genrandconfig?h=2020.02#n164

Cc Thomas Petazzoni <[email protected]>
Cc Yann E. MORIN <[email protected]>

---
v2: update with the improved patch suggested by Yann.

    http://lists.busybox.net/pipermail/buildroot/2020-April/280679.html
---
 support/scripts/check-dotconfig.py | 44 ++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)
 create mode 100755 support/scripts/check-dotconfig.py
support/scripts/check-dotconfig.py new file +44 -0
new file mode 100755
index 0000000000..81c83a8481
@@ -0,0 +1,44 @@
+#!/usr/bin/env python3
+
+# This scripts check that all lines present in the defconfig are
+# still present in the .config
+
+import sys
+
+
+def main():
+    if not (len(sys.argv) == 3):
+        print("Error: incorrect number of arguments")
+        print("""Usage: check-dotconfig <configfile> <defconfig>""")
+        sys.exit(1)
+
+    configfile = sys.argv[1]
+    defconfig = sys.argv[2]
+
+    # strip() to get rid of trailing \n
+    with open(configfile) as configf:
+        configlines = [l.strip() for l in configf.readlines()]
+
+    defconfiglines = []
+    with open(defconfig) as defconfigf:
+        for line in defconfigf.readlines():
+            # strip() to get rid of trailing \n
+            line = line.strip()
+            if line.startswith("BR2_"):
+                defconfiglines.append(line)
+            elif line.startswith('# BR2_') and line.endswith(' is not set'):
+                defconfiglines.append(line)
+
+    # Check that all the defconfig lines are still present
+    missing = [defconfigline.strip() for defconfigline in defconfiglines
+               if defconfigline not in configlines]
+
+    if len(missing):
+        print("WARN: defconfig {} can't be used:".format(defconfig))
+        for m in missing:
+            print("      Missing: {}".format(m))
+    sys.exit(1 if len(missing) else 0)
+
+
+if __name__ == "__main__":
+    main()
Signature · 1 line
2.25.4
#4 lore raw

PATCH v3 4/6 support/scripts: add check-dotconfig.py

Romain, All,

On 2020-07-27 17:51 +0200, Romain Naour spake thusly:

Quoted 44 lines

For the same reason as for 50b747f212be2c9c0f7cf10c674ed488d042715c, we need to check if the generated configuration file (.config) contains all symbols present in the defconfig file.

If not there is an issue with the defconfig.

This script will be used in .gitlab-ci.yml.

Inspired by is_toolchain_usable() function from genrandconfig:
https://git.busybox.net/buildroot/tree/utils/genrandconfig?h=2020.02#n164

Cc Thomas Petazzoni <[email protected]>
Cc Yann E. MORIN <[email protected]>

---
v2: update with the improved patch suggested by Yann.

    http://lists.busybox.net/pipermail/buildroot/2020-April/280679.html
---
 support/scripts/check-dotconfig.py | 44 ++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)
 create mode 100755 support/scripts/check-dotconfig.py
support/scripts/check-dotconfig.py new file +16 -0
new file mode 100755
index 0000000000..81c83a8481
@@ -0,0 +1,44 @@
+#!/usr/bin/env python3
+
+# This scripts check that all lines present in the defconfig are
+# still present in the .config
+
+import sys
+
+
+def main():
+    if not (len(sys.argv) == 3):
+        print("Error: incorrect number of arguments")
+        print("""Usage: check-dotconfig <configfile> <defconfig>""")
+        sys.exit(1)
+
+    configfile = sys.argv[1]
+    defconfig = sys.argv[2]

support/scripts/check-dotconfig.py line 26

Quoted 10 lines
support/scripts/check-dotconfig.py new file +10 -0
+
+    # strip() to get rid of trailing \n
+    with open(configfile) as configf:
+        configlines = [l.strip() for l in configf.readlines()]
+
+    defconfiglines = []
+    with open(defconfig) as defconfigf:
+        for line in defconfigf.readlines():
+            # strip() to get rid of trailing \n
+            line = line.strip()

I've used a generator to strip the lines when reading:

for line in (line.strip() for line in defconfigf.readlines()):

support/scripts/check-dotconfig.py line 33

Quoted 7 lines
support/scripts/check-dotconfig.py new file +7 -0
+            if line.startswith("BR2_"):
+                defconfiglines.append(line)
+            elif line.startswith('# BR2_') and line.endswith(' is not set'):
+                defconfiglines.append(line)
+
+    # Check that all the defconfig lines are still present
+    missing = [defconfigline.strip() for defconfigline in defconfiglines

No need to strip() here, lines are already stripped. I also shortened the variable from defconfigline to simply line, so that it fits on a single line. (muahaha!)

support/scripts/check-dotconfig.py line 36

Quoted 3 lines
support/scripts/check-dotconfig.py new file +3 -0
+               if defconfigline not in configlines]
+
+    if len(missing):

Testing is a list ie empty is simply done by testing the list;

if missing:

support/scripts/check-dotconfig.py line 40

Quoted 4 lines
support/scripts/check-dotconfig.py new file +4 -0
+        print("WARN: defconfig {} can't be used:".format(defconfig))
+        for m in missing:
+            print("      Missing: {}".format(m))
+    sys.exit(1 if len(missing) else 0)

I moved the sys.exit(1) inside the conditional, it's s much simpler.

Applied to master with those fixes, thanks!

Regards,
Yann E. MORIN.

Quoted 7 lines
support/scripts/check-dotconfig.py new file +4 -0
+
+
+if __name__ == "__main__":
+    main()
Signature · 1 line
2.25.4
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#5 lore raw

PATCH v3 5/6 gitlab-ci: check generated config files

In reply to Romain Naour

Use the script added by the previous patch to check generated config files.

Tested on gitlab:
https://gitlab.com/kubu93/buildroot/pipelines/137597966

Cc Thomas Petazzoni <[email protected]>
---
 support/misc/gitlab-ci.yml.in | 1 +
 1 file changed, 1 insertion(+)
support/misc/gitlab-ci.yml.in +1 -0
index cf9bf5f9cf..01592f7155 100644
@@ -38,6 +38,7 @@ check-package:
     script:
         - echo "Configure Buildroot for ${DEFCONFIG_NAME}"
         - make ${DEFCONFIG_NAME}
+        - ./support/scripts/check-dotconfig.py .config ./configs/${DEFCONFIG_NAME}
         - echo 'Build buildroot'
         - |
             make > >(tee build.log |grep '>>>') 2>&1 || {
Signature · 1 line
2.25.4
#6 lore raw

PATCH v3 6/6 gitlab-ci: check all defconfigs on every push

In reply to Romain Naour

The defconfig check has been introduced by the previous patch before the building each defconfig but those builds are done every week or more.

Checking if a defconfig is valid can be done on every push in the repository since it take few seconds.

This would allow to detect as soon as possible a problem in a defconfig and eventually avoid breaking the build while build testing all defconfig.

Introduce a new job template ".defconfig_check" in gitlab-ci.yml.in and modify the generate-gitlab-ci-yml to create a job for each defconfig to run the test.

Although, we could have used only one job to do all tests, using one job per defconfig allow to identify easily in gitlab which defconfig is falling.

Tested:
https://gitlab.com/kubu93/buildroot/pipelines/138331069
https://gitlab.com/kubu93/buildroot/pipelines/171223758

Cc Yann E. MORIN <[email protected]>
Cc Thomas Petazzoni <[email protected]>
---
 support/misc/gitlab-ci.yml.in          | 13 +++++++++++++
 support/scripts/generate-gitlab-ci-yml |  2 +-
 2 files changed, 14 insertions(+), 1 deletion(-)
support/misc/gitlab-ci.yml.in +13 -0
index 01592f7155..7218ea027e 100644
@@ -34,6 +34,19 @@ check-package:
     script:
         - make check-package
 
+.defconfig_check:
+    script:
+        - echo "Configure Buildroot for ${DEFCONFIG_NAME}"
+        - make ${DEFCONFIG_NAME}
+        - support/scripts/check-dotconfig.py .config configs/${DEFCONFIG_NAME}
+    artifacts:
+        when: on_failure
+        expire_in: 2 weeks
+        paths:
+            - .config
+    before_script:
+        - DEFCONFIG_NAME=$(echo ${CI_JOB_NAME} | sed -e 's,_check$,,g')
+
 .defconfig_base:
     script:
         - echo "Configure Buildroot for ${DEFCONFIG_NAME}"
support/scripts/generate-gitlab-ci-yml +1 -1
index 262a7649b2..5cef1146e2 100755
@@ -10,7 +10,7 @@ cat "${input}"
     cd configs
     LC_ALL=C ls -1 *_defconfig
 ) \
-    | sed 's/$/: { extends: .defconfig }/'
+    | sed -r -e 's/^(.+)$/\1: { extends: .defconfig }\n\1_check: { extends: .defconfig_check }/'
 
 ./support/testing/run-tests -l 2>&1 \
     | sed -r -e '/^test_run \((.*)\).*/!d; s//\1: { extends: .runtime_test }/' \
Signature · 1 line
2.25.4
#7 lore raw

PATCH v3 1/6 configs/imx6ullevk_defconfig: remove typo

In reply to Romain Naour

Romain, All,

On 2020-07-27 17:51 +0200, Romain Naour spake thusly:

Quoted 6 lines

Reported by check-dotconfig.py.

Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/657885169

Whole series applied to master, thanks.

Please see my reply to patch 4 for specific comments on that one.

Regards,
Yann E. MORIN.

Quoted 24 lines
---
 configs/imx6ullevk_defconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
configs/imx6ullevk_defconfig +1 -1
index 4a5558dd6c..81db66268a 100644
@@ -23,7 +23,7 @@ BR2_TARGET_UBOOT=y
 BR2_TARGET_UBOOT_BOARDNAME="mx6ull_14x14_evk"
 BR2_TARGET_UBOOT_CUSTOM_VERSION=y
 BR2_TARGET_UBOOT_CUSTOM_VERSION_VALUE="2020.07"
-BR2_TARGET_UBOOT_FORMAT_DTB_IMX=y"
+BR2_TARGET_UBOOT_FORMAT_DTB_IMX=y
 BR2_TARGET_UBOOT_NEEDS_DTC=y
 
 # required tools to create the SD card image
Signature · 6 lines
2.25.4

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'