Skip to contentSkip to conversation

support/scripts: tool to create fragments

PATCH New Matt Weber 4 participants 10 messages Oct 25, 2016 – Oct 26, 2016

lore
Share & tools
lore

Matt Weber

author
#0 lore raw

PATCH 1/1 support/scripts: tool to create fragments

From: Sam Voss <[email protected]>

Add script to create kconfig fragment; defaults to busybox fragment if only one config is specified (compares to package/busybox/busybox.config).

---
 support/scripts/gen-config-fragment.sh | 59 ++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100755 support/scripts/gen-config-fragment.sh
support/scripts/gen-config-fragment.sh new file +59 -0
new file mode 100755
index 0000000..a8b5345
@@ -0,0 +1,59 @@
+#!/bin/bash
+
+################################################################################
+# DESCRIPTION:
+#     Creates a fragment by comparing two kconfigs. Default usage creates a
+#       busybox fragment, however providing two config files will override this.
+#
+#     Required arguments:
+#       $1 - config #1. This is generally the new one created
+#
+#     Optional arguments:
+#       $2 - config #2: The configuration file to compare against. If none
+#            supplied, will compare against package/busybox/busybox.config
+################################################################################
+usage ()
+{
+use="gen-config-fragment.sh:
+    Creates a fragment by comparing two kconfigs. Default usage creates a
+      busybox fragment, however providing two config files will override this.
+
+    Required arguments:
+      $1 - config #1. This is generally the new one created
+
+    Optional arguments:
+      $2 - config #2: The configuration file to compare against. If none
+           supplied, will compare against package/busybox/busybox.config"
+        printf "$use"
+        exit 1
+}
+
+if [ -z "$1" ]; then
+        usage
+else
+        FirstConfig="$1"
+fi
+
+if [ -z "$2" ]; then
+        SecondConfig="package/busybox/busybox.config"
+else
+        SecondConfig="$2"
+fi
+
+loadedFirstConfig=()
+loadedSecondConfig=()
+while read line; do
+        if [[ ${line:0:1} != '#' ]] || [[ ${line:2:3} == 'BR2' ]]; then
+                loadedFirstConfig+=("${line// /_space_}")
+        fi
+done < $FirstConfig
+
+while read -r line; do
+        if [[ ${line:0:1} != '#' ]] || [[ ${line:2:3} == 'BR2' ]]; then
+                loadedSecondConfig+=("${line// /_space_}")
+        fi
+done < $SecondConfig
+
+
+D=($(comm -23 <(printf '%s\n' "${loadedFirstConfig[@]}" | sort -d) <(printf '%s\n' "${loadedSecondConfig[@]}" | sort -d)))
+printf '%s\n' "${D[@]//_space_/ }" > "$FirstConfig.fragment"
Signature · 1 line
1.9.1
#1 lore raw

PATCH 1/1 support/scripts: tool to create fragments

Hello,

Quoted 7 lines · Matt Weber

From: Sam Voss <[email protected]>

Add script to create kconfig fragment; defaults to busybox fragment if only one config is specified (compares to package/busybox/busybox.config).

Could you show an example on how this script is intended to be used ?

Thanks,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
#2 lore raw

PATCH 1/1 support/scripts: tool to create fragments

Thomas,

Quoted 3 lines

Could you show an example on how this script is intended to be used ?

Assuming you have already done a make linux-savedefconfig and have an older version to compare against (example files given below), it would be used (from the
buildroot root directory) as follows:

./support/scripts/gen-config-fragment.sh output/build/linux-4.8.1/defconfig board/qemu/x86_64/linux-4.8.config

File 1 - output/build/linux-4.8.1/defconfig

CONFIG_SYSVIPC=y
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_SMP=y
# CONFIG_X86_MPPARSE is not set
CONFIG_HYPERVISOR_GUEST=y
CONFIG_PARAVIRT=y
CONFIG_NET=y
CONFIG_PACKET=y
CONFIG_UNIX=y
CONFIG_INET=y
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_VIRTIO_BLK=y
CONFIG_BLK_DEV_SD=y
CONFIG_SCSI_VIRTIO=y
CONFIG_ATA=y
CONFIG_ATA_PIIX=y
CONFIG_NETDEVICES=y
CONFIG_VIRTIO_NET=y
CONFIG_NE2K_PCI=y
CONFIG_8139CP=y
CONFIG_INPUT_EVDEV=y
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_VIRTIO_CONSOLE=y
CONFIG_HW_RANDOM_VIRTIO=m
CONFIG_DRM=y
CONFIG_DRM_QXL=y
CONFIG_DRM_BOCHS=y
CONFIG_DRM_VIRTIO_GPU=y
CONFIG_SOUND=y
CONFIG_SND=y
CONFIG_SND_HDA_INTEL=y
CONFIG_SND_HDA_GENERIC=y
CONFIG_USB=y
CONFIG_USB_XHCI_HCD=y
CONFIG_USB_EHCI_HCD=y
CONFIG_USB_UHCI_HCD=y
CONFIG_USB_STORAGE=y
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_BALLOON=y
CONFIG_VIRTIO_INPUT=y
CONFIG_VIRTIO_MMIO=y
CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y
CONFIG_EXT4_FS=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y

File 2 - Original defconfig - board/qemu/x86_64/linux-4.8.config

CONFIG_SYSVIPC=y
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_SMP=y
CONFIG_HYPERVISOR_GUEST=y
CONFIG_PARAVIRT=y
CONFIG_NET=y
CONFIG_PACKET=y
CONFIG_UNIX=y
CONFIG_INET=y
CONFIG_VIRTIO_BLK=y
CONFIG_BLK_DEV_SD=y
CONFIG_SCSI_VIRTIO=y
CONFIG_ATA=y
CONFIG_ATA_PIIX=y
CONFIG_NETDEVICES=y
CONFIG_VIRTIO_NET=y
CONFIG_NE2K_PCI=y
CONFIG_8139CP=y
CONFIG_INPUT_EVDEV=y
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_VIRTIO_CONSOLE=y
CONFIG_HW_RANDOM_VIRTIO=m
CONFIG_DRM=y
CONFIG_DRM_BOCHS=y
CONFIG_DRM_QXL=y
CONFIG_DRM_VIRTIO_GPU=y
CONFIG_SOUND=y
CONFIG_SND=y
CONFIG_SND_HDA_INTEL=y
CONFIG_SND_HDA_GENERIC=y
CONFIG_USB=y
CONFIG_USB_XHCI_HCD=y
CONFIG_USB_EHCI_HCD=y
CONFIG_USB_UHCI_HCD=y
CONFIG_USB_STORAGE=y
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_BALLOON=y
CONFIG_VIRTIO_INPUT=y
CONFIG_VIRTIO_MMIO=y
CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y
CONFIG_EXT4_FS=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y

Giving output file of - output/build/linux-4.8.1/defconfig.fragment

CONFIG_DEVTMPFS_MOUNT=y
CONFIG_DEVTMPFS=y
# CONFIG_X86_MPPARSE is not set

Should I add an example execution of this in the header of the script?
Something like the following?

#
# Linux Example

# ./support/scripts/gen-config-fragment.sh <output
dir>/build/linux-x.y.z/defconfig
board/custom-board/linux-x.y.z.config
#
# Busybox Example using default base config
# ./support/scripts/gen-config-fragment.sh <output
dir>/build/busybox-x.y.z/.config
#
# Busybox Example using custom base config
# ./support/scripts/gen-config-fragment.sh <output
dir>/build/busybox-x.y.z/.config board/custom-board/busybox.config
#3 lore raw

PATCH 1/1 support/scripts: tool to create fragments

Hello,

Quoted 7 lines · Sam Voss

Assuming you have already done a make linux-savedefconfig and have an older version to compare against (example files given below), it would be used (from the
buildroot root directory) as follows:

./support/scripts/gen-config-fragment.sh output/build/linux-4.8.1/defconfig board/qemu/x86_64/linux-4.8.config

[...]

Quoted 5 lines

Giving output file of - output/build/linux-4.8.1/defconfig.fragment

CONFIG_DEVTMPFS_MOUNT=y
CONFIG_DEVTMPFS=y
# CONFIG_X86_MPPARSE is not set

OK, understood. So in fact, you have reimplemented the diffconfig tool available in the Linux kernel scripts/ directory. For your two configuration files, it shows:

thomas at skate:/tmp$ ~/projets/linux-2.6/scripts/diffconfig config1 config2 -DEVTMPFS y
-DEVTMPFS_MOUNT y
-X86_MPPARSE n

Though I agree that your output format is better, as it can be re-used as a fragment as-is.

However, calling it gen-config-fragment looks a bit wrong to me. What about diffconfig, like in the Linux kernel?

Quoted 5 lines

Should I add an example execution of this in the header of the script?
Something like the following?

#
# Linux Example

Quoted 11 lines
# ./support/scripts/gen-config-fragment.sh <output
dir>/build/linux-x.y.z/defconfig  
board/custom-board/linux-x.y.z.config
#
# Busybox Example using default base config
# ./support/scripts/gen-config-fragment.sh <output
dir>/build/busybox-x.y.z/.config  
#
# Busybox Example using custom base config
# ./support/scripts/gen-config-fragment.sh <output
dir>/build/busybox-x.y.z/.config board/custom-board/busybox.config  

No need to give so many examples, but just one example would be good.

In terms of implementation, should we re-use the diffconfig tool from the kernel, simply tweaked in terms of output format?

Thanks,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
#4 lore raw

PATCH 1/1 support/scripts: tool to create fragments

Thomas,

Quoted 2 lines

OK, understood. So in fact, you have reimplemented the diffconfig tool available in the Linux kernel scripts/ directory.

Interesting, I didn't know this tool existed!

Quoted 2 lines

However, calling it gen-config-fragment looks a bit wrong to me. What about diffconfig, like in the Linux kernel?

I agree with a name change, however might it be confused (in discussion) with the other tool? If that is not a problem I would agree with the new name.

Quoted 1 line

No need to give so many examples, but just one example would be good.

Will update the patch to include one along with the changes Matt mentioned.

Quoted 2 lines

In terms of implementation, should we re-use the diffconfig tool from the kernel, simply tweaked in terms of output format?

I actually don't think this would be a best course of action. As it is written it will work
on busybox or kernel configs, however if we choose to reuse the output of the diffconfig
tool either logic will have to be added to figure out what configs are being generated or
another argument will have to specify it and I'm not sure if it is worth that extra code.

Thanks,

Sam

#5 lore raw

PATCH 1/1 support/scripts: tool to create fragments

In reply to Matt Weber

Sam,

Quoted 63 lines · Matt Weber

From: Sam Voss <[email protected]>

Add script to create kconfig fragment; defaults to busybox fragment if only one config is specified (compares to package/busybox/busybox.config).

---
 support/scripts/gen-config-fragment.sh | 59
++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100755 support/scripts/gen-config-fragment.sh
support/scripts/gen-config-fragment.sh new file +3 -0
new file mode 100755
index 0000000..a8b5345
@@ -0,0 +1,59 @@
+#!/bin/bash
+
+###########################################################

#####################

support/scripts/gen-config-fragment.sh new file +2 -0
+# DESCRIPTION:
+#     Creates a fragment by comparing two kconfigs. Default usage creates

a
+# busybox fragment, however providing two config files will
override this.

support/scripts/gen-config-fragment.sh new file +8 -0
+#
+#     Required arguments:
+#       $1 - config #1. This is generally the new one created
+#
+#     Optional arguments:
+#       $2 - config #2: The configuration file to compare against. If none
+#            supplied, will compare against package/busybox/busybox.config
+###########################################################

#####################

support/scripts/gen-config-fragment.sh new file +5 -0
+usage ()
+{
+use="gen-config-fragment.sh:
+    Creates a fragment by comparing two kconfigs. Default usage creates a
+      busybox fragment, however providing two config files will override

this.

support/scripts/gen-config-fragment.sh new file +18 -0
+
+    Required arguments:
+      $1 - config #1. This is generally the new one created
+
+    Optional arguments:
+      $2 - config #2: The configuration file to compare against. If none
+           supplied, will compare against package/busybox/busybox.config"
+        printf "$use"
+        exit 1
+}
+
+if [ -z "$1" ]; then
+        usage
+else
+        FirstConfig="$1"
+fi
+
+if [ -z "$2" ]; then

support/scripts/gen-config-fragment.sh line 47

Quoted 10 lines
support/scripts/gen-config-fragment.sh new file +10 -0
+        SecondConfig="package/busybox/busybox.config"
+else
+        SecondConfig="$2"
+fi
+
+loadedFirstConfig=()
+loadedSecondConfig=()
+while read line; do
+        if [[ ${line:0:1} != '#' ]] || [[ ${line:2:3} == 'BR2' ]]; then
+                loadedFirstConfig+=("${line// /_space_}")

Oops need to update for kernel config use, you must have just tested against buildroot.

support/scripts/gen-config-fragment.sh line 53

Quoted 7 lines
support/scripts/gen-config-fragment.sh new file +6 -0
+        fi
+done < $FirstConfig
+
+while read -r line; do
+        if [[ ${line:0:1} != '#' ]] || [[ ${line:2:3} == 'BR2' ]]; then
+                loadedSecondConfig+=("${line// /_space_}")

same here

Quoted 11 lines
support/scripts/gen-config-fragment.sh new file +5 -0
+        fi
+done < $SecondConfig
+
+
+D=($(comm -23 <(printf '%s\n' "${loadedFirstConfig[@]}" | sort -d)

<(printf '%s\n' "${loadedSecondConfig[@]}" | sort -d))) +printf '%s\n' "${D[@]//_space_/ }" > "$FirstConfig.fragment" --
1.9.1

Signature · 12 lines
Matthew L Weber / Pr Software Engineer
Airborne Information Systems / Security Systems and Software / Secure
Platforms
MS 131-100, C Ave NE, Cedar Rapids, IA, 52498, USA
www.rockwellcollins.com

Note: Any Export License Required Information and License Restricted Third
Party Intellectual Property (TPIP) content must be encrypted and sent to
matthew.weber at corp.rockwellcollins.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20161025/ef3992ff/attachment.html>
#6 lore raw

PATCH 1/1 support/scripts: tool to create fragments

In reply to Matt Weber

Matt, Sam, All,

On 2016-10-25 15:01 -0500, Matt Weber spake thusly:

Quoted 4 lines

From: Sam Voss <[email protected]>

Add script to create kconfig fragment; defaults to busybox fragment if only one config is specified (compares to package/busybox/busybox.config).

As stated by Thomas, this is re-inventing diffconfig.

The only difference being the output format.

Currently diffconfig can output a diff-like, but it can also output a "merge" style (although I'm not sure what it means, given that a merge on your example is just empty).

I think it would be much better to update diffconfig to allow it to spit out a fragment format. Maybe something like:

    $ diffconfig -f config.base config.modified
    CONFIG_DEVTMPFS_MOUNT=y
    CONFIG_DEVTMPFS=y
    # CONFIG_X86_MPPARSE is not set

Thoughts?

Note: either way, I'd prefer both files to be mandatory arguments, with the original one specified first and the modified one specified last, and that there is no default comparison against busybox.

Otherwise, some review below...

Quoted 31 lines
---
 support/scripts/gen-config-fragment.sh | 59 ++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100755 support/scripts/gen-config-fragment.sh
support/scripts/gen-config-fragment.sh new file +18 -0
new file mode 100755
index 0000000..a8b5345
@@ -0,0 +1,59 @@
+#!/bin/bash
+
+################################################################################
+# DESCRIPTION:
+#     Creates a fragment by comparing two kconfigs. Default usage creates a
+#       busybox fragment, however providing two config files will override this.
+#
+#     Required arguments:
+#       $1 - config #1. This is generally the new one created
+#
+#     Optional arguments:
+#       $2 - config #2: The configuration file to compare against. If none
+#            supplied, will compare against package/busybox/busybox.config
+################################################################################
+usage ()
+{
+use="gen-config-fragment.sh:
+    Creates a fragment by comparing two kconfigs. Default usage creates a

support/scripts/gen-config-fragment.sh line 28

Quoted 10 lines
support/scripts/gen-config-fragment.sh new file +10 -0
+      busybox fragment, however providing two config files will override this.
+
+    Required arguments:
+      $1 - config #1. This is generally the new one created
+
+    Optional arguments:
+      $2 - config #2: The configuration file to compare against. If none
+           supplied, will compare against package/busybox/busybox.config"
+        printf "$use"
+        exit 1

Dont use a variable; use an here-document (leading TABS, repesented by ???? here, are ignored):

    cat <<-_EOF_
    ????gen-config-fragment.sh:
    ????    Creates a fragment by comparing two kconfigs.

    ????Arguments:
    ????    $1 : the unmodified original configuration file
    ????    $2 : the modified configuration file

    ????Blabla...
    _EOF_
Quoted 6 lines
support/scripts/gen-config-fragment.sh new file +6 -0
+}
+
+if [ -z "$1" ]; then
+        usage
+else
+        FirstConfig="$1"

support/scripts/gen-config-fragment.sh line 44

Quoted 10 lines
support/scripts/gen-config-fragment.sh new file +10 -0
+fi
+
+if [ -z "$2" ]; then
+        SecondConfig="package/busybox/busybox.config"
+else
+        SecondConfig="$2"
+fi
+
+loadedFirstConfig=()
+loadedSecondConfig=()

Declare the variables:

declare -a loadedFirstConfig loadedSecondConfig

support/scripts/gen-config-fragment.sh line 46

Quoted 2 lines
support/scripts/gen-config-fragment.sh new file +2 -0
+while read line; do
+        if [[ ${line:0:1} != '#' ]] || [[ ${line:2:3} == 'BR2' ]]; then

Use just plain '[ ... ]' to do tests, since they are POSIX; quote strings:

    if [ "${line:0:3}" = "BR2_" -o "${line:2:4}" = "BR2_" ]; then

support/scripts/gen-config-fragment.sh line 47

Quoted 1 line
support/scripts/gen-config-fragment.sh new file +1 -0
+                loadedFirstConfig+=("${line// /_space_}")

I'm not too fond of the magic placeholder _space_. But I can't find an easy and better solution... Maybe make it even less probable, like:

    _S_P_A_C_E_

support/scripts/gen-config-fragment.sh line 49

Quoted 2 lines
support/scripts/gen-config-fragment.sh new file +2 -0
+        fi
+done < $FirstConfig

Quote strings: "${FirstConfig}"

But there is a better and faster way to fill in the array:

    loadedFirstConfig=( $( sed -r -e 's/ /_S_P_A_C_E_/g' "${FirstConfig}" ) )

But it turns out you don;t need the array in the first place, see below.

support/scripts/gen-config-fragment.sh line 58

Quoted 9 lines
support/scripts/gen-config-fragment.sh new file +9 -0
+
+while read -r line; do
+        if [[ ${line:0:1} != '#' ]] || [[ ${line:2:3} == 'BR2' ]]; then
+                loadedSecondConfig+=("${line// /_space_}")
+        fi
+done < $SecondConfig
+
+
+D=($(comm -23 <(printf '%s\n' "${loadedFirstConfig[@]}" | sort -d) <(printf '%s\n' "${loadedSecondConfig[@]}" | sort -d)))

Line too long; split it.

support/scripts/gen-config-fragment.sh line 59

Quoted 1 line
support/scripts/gen-config-fragment.sh new file +1 -0
+printf '%s\n' "${D[@]//_space_/ }" > "$FirstConfig.fragment"

Just output to stdout; let the user redirect to the file of his choice.

But we can do it much more simply, in a single command. Your script would be just (without even a requirement for bash):

    #!/bin/sh

    usage() {
        cat <<-_EOF_
        Blabla
        _EOF_
    }

    [ ${#} -eq 2 ] || { usage; exit 1; }
    [ -f "${1}" ]  || { usage; exit 1; }
    [ -f "${2}" ]  || { usage; exit 1; }

    comm -23 <( sed -r -e 's/ /_S_P_A_C_E_/g' "${FirstConfig}"  |sort -d ) \
             <( sed -r -e 's/ /_S_P_A_C_E_/g' "${SecondConfig}" |sort -d ) \
    |sed -r -e 's/_S_P_A_C_E_/ /g'

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#7 lore raw

PATCH 1/1 support/scripts: tool to create fragments

Matt, Sam, All,

On 2016-10-26 23:17 +0200, Yann E. MORIN spake thusly:

Quoted 4 lines

On 2016-10-25 15:01 -0500, Matt Weber spake thusly:

Quoted 3 lines

From: Sam Voss <[email protected]>
Add script to create kconfig fragment; defaults to busybox fragment if only one config is specified (compares to package/busybox/busybox.config).

[--SNIP--]

Quoted 18 lines

But we can do it much more simply, in a single command. Your script would be just (without even a requirement for bash):

    #!/bin/sh

    usage() {
        cat <<-_EOF_
        Blabla
        _EOF_
    }

    [ ${#} -eq 2 ] || { usage; exit 1; }
    [ -f "${1}" ]  || { usage; exit 1; }
    [ -f "${2}" ]  || { usage; exit 1; }

    comm -23 <( sed -r -e 's/ /_S_P_A_C_E_/g' "${FirstConfig}"  |sort -d ) \
             <( sed -r -e 's/ /_S_P_A_C_E_/g' "${SecondConfig}" |sort -d ) \
    |sed -r -e 's/_S_P_A_C_E_/ /g'

Of course, you would already noticed that you would have to replace ${FirstConfig} with ${1} and ${SecondConfig} with ${2}. ;-)

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#8 lore raw

PATCH 1/1 support/scripts: tool to create fragments

In reply to Yann E. MORIN

Hello,

Quoted 6 lines · Yann E. MORIN

support/scripts/gen-config-fragment.sh line 44

Quoted 2 lines
support/scripts/gen-config-fragment.sh new file +2 -0
+loadedFirstConfig=()
+loadedSecondConfig=()  

Declare the variables:

declare -a loadedFirstConfig loadedSecondConfig

And don't use camel case naming for variables, please.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
#9 lore raw

PATCH 1/1 support/scripts: tool to create fragments

In reply to Yann E. MORIN

Yann,

Quoted 20 lines · Yann E. MORIN

Matt, Sam, All,

On 2016-10-25 15:01 -0500, Matt Weber spake thusly:

Quoted 4 lines

From: Sam Voss <[email protected]>

Add script to create kconfig fragment; defaults to busybox fragment if only one config is specified (compares to package/busybox/busybox.

config).

As stated by Thomas, this is re-inventing diffconfig.

The only difference being the output format.

Currently diffconfig can output a diff-like, but it can also output a "merge" style (although I'm not sure what it means, given that a merge on your example is just empty).

I think it would be much better to update diffconfig to allow it to spit out a fragment format. Maybe something like:

Quoted 7 lines
    $ diffconfig -f config.base config.modified
    CONFIG_DEVTMPFS_MOUNT=y
    CONFIG_DEVTMPFS=y
    # CONFIG_X86_MPPARSE is not set

Thoughts?

We could propose a wrapper which lives as support/scripts/diffconfig which checks to see if the linux source has been extracted and if it is, calls the diffconfig tool as is. Except for the new case of -f where it does additional formatting.
Most of our use cases are with a complete build we're trying to split out configs. However if you didn't have build the script would be of no use.

Quoted 149 lines

Note: either way, I'd prefer both files to be mandatory arguments, with the original one specified first and the modified one specified last, and that there is no default comparison against busybox.

Otherwise, some review below...

Quoted 4 lines
---
 support/scripts/gen-config-fragment.sh | 59

++++++++++++++++++++++++++++++++++

Review

Quoted 4 lines
 1 file changed, 59 insertions(+)
 create mode 100755 support/scripts/gen-config-fragment.sh
(quoted diff) +0 -0

b/support/scripts/gen-config-fragment.sh

support/scripts/gen-config-fragment.sh line 3

Quoted 8 lines

new file mode 100755
index 0000000..a8b5345

support/scripts/gen-config-fragment.sh new file +3 -0
@@ -0,0 +1,59 @@
+#!/bin/bash
+
+###########################################################

#####################

support/scripts/gen-config-fragment.sh line 5

Quoted 2 lines
support/scripts/gen-config-fragment.sh new file +2 -0
+# DESCRIPTION:
+#     Creates a fragment by comparing two kconfigs. Default usage

creates a

support/scripts/gen-config-fragment.sh line 6

Quoted 1 line
support/scripts/gen-config-fragment.sh new file +1 -0
+#       busybox fragment, however providing two config files will

override this.

support/scripts/gen-config-fragment.sh

Quoted 6 lines
support/scripts/gen-config-fragment.sh new file +6 -0
+#
+#     Required arguments:
+#       $1 - config #1. This is generally the new one created
+#
+#     Optional arguments:
+#       $2 - config #2: The configuration file to compare against. If

none

support/scripts/gen-config-fragment.sh line 13

Quoted 1 line
support/scripts/gen-config-fragment.sh new file +1 -0
+#            supplied, will compare against

package/busybox/busybox.config

support/scripts/gen-config-fragment.sh line 14

Quoted 1 line
support/scripts/gen-config-fragment.sh new file +1 -0
+###########################################################

#####################

support/scripts/gen-config-fragment.sh

Quoted 4 lines
support/scripts/gen-config-fragment.sh new file +4 -0
+usage ()
+{
+use="gen-config-fragment.sh:
+    Creates a fragment by comparing two kconfigs. Default usage creates

a

support/scripts/gen-config-fragment.sh line 19

Quoted 1 line
support/scripts/gen-config-fragment.sh new file +1 -0
+      busybox fragment, however providing two config files will

override this.

support/scripts/gen-config-fragment.sh

Quoted 7 lines
support/scripts/gen-config-fragment.sh new file +7 -0
+
+    Required arguments:
+      $1 - config #1. This is generally the new one created
+
+    Optional arguments:
+      $2 - config #2: The configuration file to compare against. If none
+           supplied, will compare against package/busybox/busybox.

config"

support/scripts/gen-config-fragment.sh line 28

Quoted 2 lines
support/scripts/gen-config-fragment.sh new file +2 -0
+        printf "$use"
+        exit 1

Dont use a variable; use an here-document (leading TABS, repesented by ???? here, are ignored):

    cat <<-_EOF_
    ????gen-config-fragment.sh:
    ????    Creates a fragment by comparing two kconfigs.

    ????Arguments:
    ????    $1 : the unmodified original configuration file
    ????    $2 : the modified configuration file

    ????Blabla...
    _EOF_
Quoted 6 lines
support/scripts/gen-config-fragment.sh new file +6 -0
+}
+
+if [ -z "$1" ]; then
+        usage
+else
+        FirstConfig="$1"

support/scripts/gen-config-fragment.sh line 44

Quoted 10 lines
support/scripts/gen-config-fragment.sh new file +10 -0
+fi
+
+if [ -z "$2" ]; then
+        SecondConfig="package/busybox/busybox.config"
+else
+        SecondConfig="$2"
+fi
+
+loadedFirstConfig=()
+loadedSecondConfig=()

Declare the variables:

declare -a loadedFirstConfig loadedSecondConfig

support/scripts/gen-config-fragment.sh line 46

Quoted 2 lines
support/scripts/gen-config-fragment.sh new file +2 -0
+while read line; do
+        if [[ ${line:0:1} != '#' ]] || [[ ${line:2:3} == 'BR2' ]]; then

Use just plain '[ ... ]' to do tests, since they are POSIX; quote strings:

    if [ "${line:0:3}" = "BR2_" -o "${line:2:4}" = "BR2_" ]; then

support/scripts/gen-config-fragment.sh line 47

Quoted 1 line
support/scripts/gen-config-fragment.sh new file +1 -0
+                loadedFirstConfig+=("${line// /_space_}")

I'm not too fond of the magic placeholder _space_. But I can't find an easy and better solution... Maybe make it even less probable, like:

    _S_P_A_C_E_

support/scripts/gen-config-fragment.sh line 49

Quoted 2 lines
support/scripts/gen-config-fragment.sh new file +2 -0
+        fi
+done < $FirstConfig

Quote strings: "${FirstConfig}"

But there is a better and faster way to fill in the array:

    loadedFirstConfig=( $( sed -r -e 's/ /_S_P_A_C_E_/g' "${FirstConfig}"

) )

But it turns out you don;t need the array in the first place, see below.

support/scripts/gen-config-fragment.sh

Quoted 9 lines
support/scripts/gen-config-fragment.sh new file +9 -0
+
+while read -r line; do
+        if [[ ${line:0:1} != '#' ]] || [[ ${line:2:3} == 'BR2' ]]; then
+                loadedSecondConfig+=("${line// /_space_}")
+        fi
+done < $SecondConfig
+
+
+D=($(comm -23 <(printf '%s\n' "${loadedFirstConfig[@]}" | sort -d)

<(printf '%s\n' "${loadedSecondConfig[@]}" | sort -d)))

Line too long; split it.

support/scripts/gen-config-fragment.sh line 59

Quoted 1 line
support/scripts/gen-config-fragment.sh new file +1 -0
+printf '%s\n' "${D[@]//_space_/ }" > "$FirstConfig.fragment"

Just output to stdout; let the user redirect to the file of his choice.

But we can do it much more simply, in a single command. Your script would be just (without even a requirement for bash):

    #!/bin/sh

    usage() {
        cat <<-_EOF_
        Blabla
        _EOF_
    }

    [ ${#} -eq 2 ] || { usage; exit 1; }
Quoted 9 lines
    [ -f "${1}" ]  || { usage; exit 1; }
    [ -f "${2}" ]  || { usage; exit 1; }

    comm -23 <( sed -r -e 's/ /_S_P_A_C_E_/g' "${FirstConfig}"  |sort -d )
\
             <( sed -r -e 's/ /_S_P_A_C_E_/g' "${SecondConfig}" |sort -d )
\
    |sed -r -e 's/_S_P_A_C_E_/ /g'

Will give that a try. Thanks !

Signature · 12 lines
Matthew L Weber / Pr Software Engineer
Airborne Information Systems / Security Systems and Software / Secure
Platforms
MS 131-100, C Ave NE, Cedar Rapids, IA, 52498, USA
www.rockwellcollins.com

Note: Any Export License Required Information and License Restricted Third
Party Intellectual Property (TPIP) content must be encrypted and sent to
matthew.weber at corp.rockwellcollins.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20161026/3307cc8c/attachment.html>