Skip to contentSkip to conversation

wic: Allow to user defined files as config for bootloaders

PATCH In review [email protected] 2 participants 8 messages Dec 1, 2015 – Dec 7, 2015

lore
Share & tools
lore
#0 lore raw

v2 cover wic: Allow to user defined files as config for bootloaders

From: Mariano Lopez <[email protected]>

These patches add a new option for the bootloaders. This new option allows to use a custom configuration file as the bootloader config with the "configfile" variable in the bootloader line. This is very useful when there is need to create a multiboot image or when there is need to use scripting in the bootloader.

Changes in v2

- Previous version just allowed the have the complete path for the file. Now it will search for the file in the canned-wks folders for all the layers that are part of the project.
- Include an example wks file in canned-wks folder.
- Include an example configuration in canned-wks folder.
- Include the changes in the documentation.
- Include test in self-test.

The following changes since commit 03f15e51998a3ef65a5b68cb7cbf724f4388c289:

sstate: Ensure siginfo and sig files are also touched (2015-11-25 08:09:00 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib mariano/bug8728
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/bug8728

Mariano Lopez (6):
wic: Prepare wicboot to allow custom bootloader config wic/utils/misc.py: Added function to search for files in canned-wks
wic: Allow to use a custom config for bootloaders wic/help.py: Document the new option "configfile" directdisk-bootloader-config.wks: Add example for custom bootloader
config
selftest/wic.py: Add test for custom bootloader config

 meta/lib/oeqa/selftest/wic.py                      |  8 +++
 .../canned-wks/directdisk-bootloader-config.cfg    | 11 ++++
 .../canned-wks/directdisk-bootloader-config.wks    | 10 +++
 scripts/lib/wic/help.py                            |  6 ++
 scripts/lib/wic/kickstart/__init__.py              |  7 +++
 .../lib/wic/kickstart/custom_commands/wicboot.py   |  5 ++
 scripts/lib/wic/plugins/source/bootimg-efi.py      | 67 ++++++++++++++------
 scripts/lib/wic/plugins/source/bootimg-pcbios.py   | 72 +++++++++++++---------
 scripts/lib/wic/utils/misc.py                      | 39 ++++++++++++
 9 files changed, 179 insertions(+), 46 deletions(-)
 create mode 100644 scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg
 create mode 100644 scripts/lib/wic/canned-wks/directdisk-bootloader-config.wks
Signature · 1 line
1.8.4.5
#1 lore raw

PATCH v2 1/6 wic: Prepare wicboot to allow custom bootloader config

From: Mariano Lopez <[email protected]>

Currently wic does the bootloader configuration file on the fly. This change introduce a configfile variable for the bootloader; this is to have a user defined configuration file for the bootloaders (grub, syslinux, and gummiboot). This is particular useful when having a multiboot system or scripts embedded in the configuration file.

[YOCTO #8728]

---
 scripts/lib/wic/kickstart/__init__.py                | 7 +++++++
 scripts/lib/wic/kickstart/custom_commands/wicboot.py | 5 +++++
 2 files changed, 12 insertions(+)
scripts/lib/wic/kickstart/__init__.py +7 -0
index c9b0e51..79b39fb 100644
@@ -97,6 +97,13 @@ def get_timeout(kickstart, default=None):
         return default
     return int(kickstart.handler.bootloader.timeout)
 
+def get_bootloader_file(kickstart, default=None):
+    if not hasattr(kickstart.handler.bootloader, "configfile"):
+        return default
+    if kickstart.handler.bootloader.configfile is None:
+        return default
+    return kickstart.handler.bootloader.configfile
+
 def get_kernel_args(kickstart, default="ro rd.live.image"):
     if not hasattr(kickstart.handler.bootloader, "appendLine"):
         return default
scripts/lib/wic/kickstart/custom_commands/wicboot.py +5 -0
index a3e1852..eb17dab 100644
@@ -35,6 +35,7 @@ class Wic_Bootloader(F8_Bootloader):
         self.menus = ""
         self.ptable = "msdos"
         self.source = ""
+        self.configfile = ""
 
     def _getArgsAsStr(self):
         retval = F8_Bootloader._getArgsAsStr(self)
@@ -45,6 +46,8 @@ class Wic_Bootloader(F8_Bootloader):
             retval += " --ptable=\"%s\"" %(self.ptable,)
         if self.source:
             retval += " --source=%s" % self.source
+        if self.configfile:
+            retval += " --configfile=%s" % self.configfile
 
         return retval
 
@@ -56,5 +59,7 @@ class Wic_Bootloader(F8_Bootloader):
         # use specified source plugin to implement bootloader-specific methods
         parser.add_option("--source", type="string", action="store",
                       dest="source", default=None)
+        parser.add_option("--configfile", type="string", action="store",
+                      dest="configfile", default=None)
         return parser
 
Signature · 1 line
1.8.4.5
#2 lore raw

PATCH v2 2/6 wic/utils/misc.py: Added function to search for files in canned-wks

In reply to [email protected]

From: Mariano Lopez <[email protected]>

This change add two new function to search for files in the canned-wks folder for all the layers included in bblayers.conf. This will be used to search for custom configuration files for the bootloaders.

There are similar functions in the wic engine, but these are focused in wks files only, so it was needed to create new ones.

[YOCTO #8728]

---
 scripts/lib/wic/utils/misc.py | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
scripts/lib/wic/utils/misc.py +39 -0
index 9d75069..d886d75 100644
@@ -17,6 +17,7 @@
 
 import os
 import time
+import wic.engine
 
 def build_name(kscfg, release=None, prefix=None, suffix=None):
     """Construct and return an image name string.
@@ -56,3 +57,41 @@ def build_name(kscfg, release=None, prefix=None, suffix=None):
     ret = prefix + name + suffix
 
     return ret
+
+def find_boot_config(scripts_path, boot_file):
+    """
+    Find a config file with the given name in the canned files dir.
+
+    Return False if not found
+    """
+    if os.path.exists(boot_file):
+        return boot_file
+
+    layers_canned_wks_dir = wic.engine.build_canned_image_list(scripts_path)
+    for canned_wks_dir in layers_canned_wks_dir:
+        for root, dirs, files in os.walk(canned_wks_dir):
+            for fname in files:
+                if fname == boot_file:
+                    fullpath = os.path.join(canned_wks_dir, fname)
+                    return fullpath
+
+    return None
+
+def get_custom_config(boot_file):
+    """
+    Get the custom configuration to be used for the bootloader.
+
+    Return None if the file can't be found.
+    """
+    scripts_path = os.path.abspath(os.path.dirname(__file__))
+    # Get the scripts path of poky
+    for x in range(0, 3):
+        scripts_path = os.path.dirname(scripts_path)
+
+    cfg_file = find_boot_config(scripts_path, boot_file)
+    if cfg_file:
+        with open(cfg_file, "r") as f:
+            config = f.read()
+        return config
+
+    return None
Signature · 1 line
1.8.4.5
#3 lore raw

PATCH v2 3/6 wic: Allow to use a custom config for bootloaders

In reply to [email protected]

From: Mariano Lopez <[email protected]>

This change will allow to use a user defined file as the configuration for the bootloaders (grub, gummiboot, syslinux).

The config file is defined in the wks file with the "configfile" option in the bootloader line.

[YOCTO #8728]

---
 scripts/lib/wic/plugins/source/bootimg-efi.py    | 67 ++++++++++++++++------
 scripts/lib/wic/plugins/source/bootimg-pcbios.py | 72 +++++++++++++++---------
 2 files changed, 93 insertions(+), 46 deletions(-)
scripts/lib/wic/plugins/source/bootimg-efi.py +49 -18
index fa63c6a..5feb79d 100644
@@ -29,6 +29,7 @@ import shutil
 
 from wic import kickstart, msger
 from wic.pluginbase import SourcePlugin
+from wic.utils.misc import get_custom_config
 from wic.utils.oe.misc import exec_cmd, exec_native_cmd, get_bitbake_var, \
                               BOOTDD_EXTRA_SPACE
 
@@ -45,22 +46,37 @@ class BootimgEFIPlugin(SourcePlugin):
         """
         Create loader-specific (grub-efi) config
         """
-        options = creator.ks.handler.bootloader.appendLine
+        configfile = kickstart.get_bootloader_file(creator.ks)
+        custom_cfg = None
+        if configfile:
+            custom_cfg = get_custom_config(configfile)
+            if custom_cfg:
+                # Use a custom configuration for grub
+                grubefi_conf = custom_cfg
+                msger.debug("Using custom configuration file "
+                        "%s for grub.cfg" % configfile)
+            else:
+                msger.error("configfile is specified but failed to "
+                        "get it from %s." % configfile)
 
-        grubefi_conf = ""
-        grubefi_conf += "serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1\n"
-        grubefi_conf += "default=boot\n"
-        timeout = kickstart.get_timeout(creator.ks)
-        if not timeout:
-            timeout = 0
-        grubefi_conf += "timeout=%s\n" % timeout
-        grubefi_conf += "menuentry 'boot'{\n"
+        if not custom_cfg:
+            # Create grub configuration using parameters from wks file
+            options = creator.ks.handler.bootloader.appendLine
+
+            grubefi_conf = ""
+            grubefi_conf += "serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1\n"
+            grubefi_conf += "default=boot\n"
+            timeout = kickstart.get_timeout(creator.ks)
+            if not timeout:
+                timeout = 0
+            grubefi_conf += "timeout=%s\n" % timeout
+            grubefi_conf += "menuentry 'boot'{\n"
 
-        kernel = "/bzImage"
+            kernel = "/bzImage"
 
-        grubefi_conf += "linux %s root=%s rootwait %s\n" \
-            % (kernel, creator.rootdev, options)
-        grubefi_conf += "}\n"
+            grubefi_conf += "linux %s root=%s rootwait %s\n" \
+                % (kernel, creator.rootdev, options)
+            grubefi_conf += "}\n"
 
         msger.debug("Writing grubefi config %s/hdd/boot/EFI/BOOT/grub.cfg" \
                         % cr_workdir)
@@ -95,12 +111,27 @@ class BootimgEFIPlugin(SourcePlugin):
         cfg.write(loader_conf)
         cfg.close()
 
-        kernel = "/bzImage"
+        configfile = kickstart.get_bootloader_file(creator.ks)
+        custom_cfg = None
+        if configfile:
+            custom_cfg = get_custom_config(configfile)
+            if custom_cfg:
+                # Use a custom configuration for gummiboot
+                boot_conf = custom_cfg
+                msger.debug("Using custom configuration file "
+                        "%s for gummiboots's boot.conf" % configfile)
+            else:
+                msger.error("configfile is specified but failed to "
+                        "get it from %s." % configfile)
+
+        if not custom_cfg:
+            # Create gummiboot configuration using parameters from wks file
+            kernel = "/bzImage"
 
-        boot_conf = ""
-        boot_conf += "title boot\n"
-        boot_conf += "linux %s\n" % kernel
-        boot_conf += "options LABEL=Boot root=%s %s\n" % (creator.rootdev, options)
+            boot_conf = ""
+            boot_conf += "title boot\n"
+            boot_conf += "linux %s\n" % kernel
+            boot_conf += "options LABEL=Boot root=%s %s\n" % (creator.rootdev, options)
 
         msger.debug("Writing gummiboot config %s/hdd/boot/loader/entries/boot.conf" \
                         % cr_workdir)
scripts/lib/wic/plugins/source/bootimg-pcbios.py +44 -28

84 lines not on this page · raw message

Signature · 1 line
1.8.4.5
#4 lore raw

PATCH v2 4/6 wic/help.py: Document the new option "configfile"

In reply to [email protected]

From: Mariano Lopez <[email protected]>

This just adds the "configfile" option for the bootloader to wic help.

[YOCTO #8728]

---
 scripts/lib/wic/help.py | 6 ++++++
 1 file changed, 6 insertions(+)
scripts/lib/wic/help.py +6 -0
index 9a778b6..89fb7be 100644
@@ -740,6 +740,12 @@ DESCRIPTION
                   bootloader command-line - for example, the syslinux
                   APPEND or grub kernel command line.
 
+        --configfile: Specifies a user defined configuration file for
+                      the bootloader. This file must be located in the
+                      canned-wks folder or could be the full path to the
+                      file. Using this option will override any other
+                      bootloader option.
+
       Note that bootloader functionality and boot partitions are
       implemented by the various --source plugins that implement
       bootloader functionality; the bootloader command essentially
Signature · 1 line
1.8.4.5
#5 lore raw

PATCH v2 5/6 directdisk-bootloader-config.wks: Add example for custom bootloader config

In reply to [email protected]

From: Mariano Lopez <[email protected]>

Add new wks file as a example for a custom bootloader configuration. This change also includes the configuration that file that will be used.

This example is using syslinux with MBR, the configuration file is almost the same as the one generated by wic. As stated before this is just an example.

[YOCTO #8728]

---
 scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg | 11 +++++++++++
 scripts/lib/wic/canned-wks/directdisk-bootloader-config.wks | 10 ++++++++++
 2 files changed, 21 insertions(+)
 create mode 100644 scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg
 create mode 100644 scripts/lib/wic/canned-wks/directdisk-bootloader-config.wks
scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg new file +11 -0
new file mode 100644
index 0000000..a16bd6a
@@ -0,0 +1,11 @@
+# This is an example configuration file for syslinux.
+PROMPT 0
+TIMEOUT 10
+
+ALLOWOPTIONS 1
+SERIAL 0 115200
+
+DEFAULT boot
+LABEL boot
+KERNEL /vmlinuz
+APPEND label=boot root=/dev/sda2 rootwait rootfstype=ext4 video=vesafb vga=0x318 console=tty0
scripts/lib/wic/canned-wks/directdisk-bootloader-config.wks new file +10 -0
new file mode 100644
index 0000000..4808d04
@@ -0,0 +1,10 @@
+# short-description: Create a 'pcbios' direct disk image with custom bootloader config
+# long-description: Creates a partitioned legacy BIOS disk image that the user
+# can directly dd to boot media. The bootloader configuration source is a user file.
+
+
+part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024
+part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024
+
+bootloader --configfile="directdisk-bootloader-config.cfg"
+
Signature · 1 line
1.8.4.5
#6 lore raw

PATCH v2 6/6 selftest/wic.py: Add test for custom bootloader config

In reply to [email protected]

From: Mariano Lopez <[email protected]>

This change just add anoher test to the wic module. It will try to create a image with a custom bootloader configuration. This test will use the example image directdisk-bootloader-config to test the configfile option for the bootloaders.

[YOCTO #8728]

---
 meta/lib/oeqa/selftest/wic.py | 8 ++++++++
 1 file changed, 8 insertions(+)
meta/lib/oeqa/selftest/wic.py +8 -0
index ea78e22..f4404bf 100644
@@ -256,3 +256,11 @@ class Wic(oeSelfTest):
         self.assertEqual(0, runCmd("wic create %s -e core-image-minimal" \
                                    % image).status)
         self.assertEqual(1, len(glob(self.resultdir + "%s-*direct" % image)))
+
+    #@testcase()
+    def test_directdisk_bootloader_config(self):
+        """Test creation of directdisk-bootloader-config image"""
+        image = "directdisk-bootloader-config"
+        self.assertEqual(0, runCmd("wic create %s -e core-image-minimal" \
+                                   % image).status)
+        self.assertEqual(1, len(glob(self.resultdir + "%s-*direct" % image)))
Signature · 1 line
1.8.4.5
#7 lore raw
In reply to [email protected]

Hi Mariano,

The patchset looks good to me. Thank you.

Quoted 53 lines · On Tue, Dec 01, 2015 at 09:38:58AM +0000, [email protected] wrote:

From: Mariano Lopez <[email protected]>

These patches add a new option for the bootloaders. This new option allows to use a custom configuration file as the bootloader config with the "configfile" variable in the bootloader line. This is very useful when there is need to create a multiboot image or when there is need to use scripting in the bootloader.

Changes in v2

- Previous version just allowed the have the complete path for the file. Now it will search for the file in the canned-wks folders for all the layers that are part of the project.
- Include an example wks file in canned-wks folder.
- Include an example configuration in canned-wks folder.
- Include the changes in the documentation.
- Include test in self-test.

The following changes since commit 03f15e51998a3ef65a5b68cb7cbf724f4388c289:

sstate: Ensure siginfo and sig files are also touched (2015-11-25 08:09:00 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib mariano/bug8728
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/bug8728

Mariano Lopez (6):
wic: Prepare wicboot to allow custom bootloader config wic/utils/misc.py: Added function to search for files in canned-wks
wic: Allow to use a custom config for bootloaders wic/help.py: Document the new option "configfile" directdisk-bootloader-config.wks: Add example for custom bootloader
config
selftest/wic.py: Add test for custom bootloader config

 meta/lib/oeqa/selftest/wic.py                      |  8 +++
 .../canned-wks/directdisk-bootloader-config.cfg    | 11 ++++
 .../canned-wks/directdisk-bootloader-config.wks    | 10 +++
 scripts/lib/wic/help.py                            |  6 ++
 scripts/lib/wic/kickstart/__init__.py              |  7 +++
 .../lib/wic/kickstart/custom_commands/wicboot.py   |  5 ++
 scripts/lib/wic/plugins/source/bootimg-efi.py      | 67 ++++++++++++++------
 scripts/lib/wic/plugins/source/bootimg-pcbios.py   | 72 +++++++++++++---------
 scripts/lib/wic/utils/misc.py                      | 39 ++++++++++++
 9 files changed, 179 insertions(+), 46 deletions(-)
 create mode 100644 scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg
 create mode 100644 scripts/lib/wic/canned-wks/directdisk-bootloader-config.wks
Signature · 1 line
1.8.4.5
Signature · 3 lines
--
Regards,
Ed