Skip to contentSkip to conversation

Introduce Python bindings for libfdt

PATCH Superseded Simon Glass 3 participants 21 messages Feb 5, 2017 – Feb 27, 2017 7 messages not in the archive Kbuild

v4 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

v4 cover Introduce Python bindings for libfdt

At present libfdt consists of only a C implementation. Many scripts are written using Python so it useful to have Python bindings for libfdt. Apparently this has never been attempted before, or if so I cannot find a reference.

This series starts the process of adding this support, with just a bare-bones set of methods.

The v4 series provides binding that can be used like this:

fdt = libfdt.Fdt(open(fname).read())
node = fdt.path_offset('/subnode@1')
print fdt.get_prop(node, 'compatible')
subnode = fdt.first_subnode_quiet(node)
while subnode > 0:
print fdt.get_name(subnode)
subnode = fdt.next_subnode_quiet(subnode)

This version does not include a class for properties. That can be added if it is felt to be useful, but I want to get some basic functionality agreed first. The get_property_by_offset() function would likely benefit from this.

Changes in v4:
- Make the library less pythonic to avoid a shaky illusion
- Drop classes for Node and Prop, along with associated methods
- Include libfdt.h instead of repeating it
- Add support for fdt_getprop()
- Bring in all libfdt functions (but Python support is missing for many)
- Add full comments for Python methods
- Drop tests that are no-longer applicable
- Add a get for getprop()
- Add new patch to adjust libfdt.h to work with swig

Changes in v3:
- Make the library more pythonic
- Add classes for Node and Prop along with methods
- Add an exception class
- Use Python to generate exeptions instead of SWIG
- Add some more tests

Changes in v2:
- Add exceptions when functions return an error
- Correct Python naming to following PEP8
- Use a class to encapsulate the various methods
- Include fdt.h instead of redefining struct fdt_property
- Use bytearray to avoid the SWIG warning 454
- Add comments
- Update tests for new pylibfdt
- Add a few more tests to increase coverage
- Add details on how to obtain full help and code coverage

Simon Glass (5):
Add an initial Python library for libfdt Add tests for pylibfdt
Mention pylibfdt in the documentation Adjust libfdt.h to work with swig
Build pylibfdt as part of the normal build process

 Makefile                   |  16 +-
 README                     |  33 +++
 libfdt/libfdt.h            |   7 +-
 pylibfdt/.gitignore        |   3 +
 pylibfdt/Makefile.pylibfdt |  18 ++
 pylibfdt/libfdt.swig       | 565 +++++++++++++++++++++++++++++++++++++++++++++
 pylibfdt/setup.py          |  34 +++
 tests/pylibfdt_tests.py    | 275 ++++++++++++++++++++++
 tests/run_tests.sh         |  19 +-
 9 files changed, 967 insertions(+), 3 deletions(-)
 create mode 100644 pylibfdt/.gitignore
 create mode 100644 pylibfdt/Makefile.pylibfdt
 create mode 100644 pylibfdt/libfdt.swig
 create mode 100644 pylibfdt/setup.py
 create mode 100644 tests/pylibfdt_tests.py
Signature · 1 line
2.11.0.483.g087da7b7c-goog

A message that is not in the archive (sent off-list or not yet received): 20170205201323.15411-1-sjg-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org

#2 lore raw

PATCH v4 1/5 Add an initial Python library for libfdt

Add Python bindings for a bare-bones set of libfdt functions. These allow navigating the tree and reading node names and properties.

---

Changes in v4:
- Make the library less pythonic to avoid a shaky illusion
- Drop classes for Node and Prop, along with associated methods
- Include libfdt.h instead of repeating it
- Add support for fdt_getprop()
- Bring in all libfdt functions (but Python support is missing for many)
- Add full comments for Python methods

Changes in v3:
- Make the library more pythonic
- Add classes for Node and Prop along with methods
- Add an exception class
- Use Python to generate exeptions instead of SWIG

Changes in v2:
- Add exceptions when functions return an error
- Correct Python naming to following PEP8
- Use a class to encapsulate the various methods
- Include fdt.h instead of redefining struct fdt_property
- Use bytearray to avoid the SWIG warning 454
- Add comments

 pylibfdt/.gitignore        |   3 +
 pylibfdt/Makefile.pylibfdt |  18 ++
 pylibfdt/libfdt.swig       | 565 +++++++++++++++++++++++++++++++++++++++++++++
 pylibfdt/setup.py          |  34 +++
 4 files changed, 620 insertions(+)
 create mode 100644 pylibfdt/.gitignore
 create mode 100644 pylibfdt/Makefile.pylibfdt
 create mode 100644 pylibfdt/libfdt.swig
 create mode 100644 pylibfdt/setup.py
pylibfdt/.gitignore new file +3 -0
new file mode 100644
index 0000000..5e8c5e3
@@ -0,0 +1,3 @@
+libfdt.py
+libfdt.pyc
+libfdt_wrap.c
pylibfdt/Makefile.pylibfdt new file +18 -0
new file mode 100644
index 0000000..fa74dd2
@@ -0,0 +1,18 @@
+# Makefile.pylibfdt
+#
+
+PYLIBFDT_srcs = $(addprefix $(LIBFDT_srcdir)/,$(LIBFDT_SRCS))
+WRAP = $(PYLIBFDT_objdir)/libfdt_wrap.c
+PYMODULE = $(PYLIBFDT_objdir)/_libfdt.so
+
+$(PYMODULE): $(PYLIBFDT_srcs) $(WRAP)
+	@$(VECHO) PYMOD $@
+	python $(PYLIBFDT_objdir)/setup.py "$(CPPFLAGS)" $^
+	mv _libfdt.so $(PYMODULE)
+
+$(WRAP): $(PYLIBFDT_srcdir)/libfdt.swig
+	@$(VECHO) SWIG $@
+	swig -python -o $@ $<
+
+PYLIBFDT_cleanfiles = libfdt_wrap.c libfdt.py libfdt.pyc
+PYLIBFDT_CLEANFILES = $(addprefix $(PYLIBFDT_objdir)/,$(PYLIBFDT_cleanfiles))
pylibfdt/libfdt.swig new file +565 -0

565 lines not on this page · raw message

pylibfdt/setup.py new file +34 -0
new file mode 100644
index 0000000..8f8618e
@@ -0,0 +1,34 @@
+#!/usr/bin/env python
+
+"""
+setup.py file for SWIG libfdt
+"""
+
+from distutils.core import setup, Extension
+import os
+import sys
+
+progname = sys.argv[0]
+cflags = sys.argv[1]
+files = sys.argv[2:]
+
+if cflags:
+    cflags = [flag for flag in cflags.split(' ') if flag]
+else:
+    cflags = None
+
+libfdt_module = Extension(
+    '_libfdt',
+    sources = files,
+    extra_compile_args =  cflags
+)
+
+sys.argv = [progname, '--quiet', 'build_ext', '--inplace']
+
+setup (name = 'libfdt',
+       version = '0.1',
+       author      = "SWIG Docs",
+       description = """Simple swig libfdt from docs""",
+       ext_modules = [libfdt_module],
+       py_modules = ["libfdt"],
+       )
Signature · 1 line
2.11.0.483.g087da7b7c-goog

A message that is not in the archive (sent off-list or not yet received): 20170205201323.15411-2-sjg-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org

#4 lore raw

Re: [PATCH v4 1/5] Add an initial Python library for libfdt

Quoted 174 lines · Simon Glass

Add Python bindings for a bare-bones set of libfdt functions. These allow navigating the tree and reading node names and properties.

---

Changes in v4:
- Make the library less pythonic to avoid a shaky illusion
- Drop classes for Node and Prop, along with associated methods
- Include libfdt.h instead of repeating it
- Add support for fdt_getprop()
- Bring in all libfdt functions (but Python support is missing for many)
- Add full comments for Python methods

Changes in v3:
- Make the library more pythonic
- Add classes for Node and Prop along with methods
- Add an exception class
- Use Python to generate exeptions instead of SWIG

Changes in v2:
- Add exceptions when functions return an error
- Correct Python naming to following PEP8
- Use a class to encapsulate the various methods
- Include fdt.h instead of redefining struct fdt_property
- Use bytearray to avoid the SWIG warning 454
- Add comments

 pylibfdt/.gitignore        |   3 +
 pylibfdt/Makefile.pylibfdt |  18 ++
 pylibfdt/libfdt.swig       | 565 +++++++++++++++++++++++++++++++++++++++++++++
 pylibfdt/setup.py          |  34 +++
 4 files changed, 620 insertions(+)
 create mode 100644 pylibfdt/.gitignore
 create mode 100644 pylibfdt/Makefile.pylibfdt
 create mode 100644 pylibfdt/libfdt.swig
 create mode 100644 pylibfdt/setup.py
pylibfdt/.gitignore new file +3 -0
new file mode 100644
index 0000000..5e8c5e3
@@ -0,0 +1,3 @@
+libfdt.py
+libfdt.pyc
+libfdt_wrap.c
pylibfdt/Makefile.pylibfdt new file +18 -0
new file mode 100644
index 0000000..fa74dd2
@@ -0,0 +1,18 @@
+# Makefile.pylibfdt
+#
+
+PYLIBFDT_srcs = $(addprefix $(LIBFDT_srcdir)/,$(LIBFDT_SRCS))
+WRAP = $(PYLIBFDT_objdir)/libfdt_wrap.c
+PYMODULE = $(PYLIBFDT_objdir)/_libfdt.so
+
+$(PYMODULE): $(PYLIBFDT_srcs) $(WRAP)
+	@$(VECHO) PYMOD $@
+	python $(PYLIBFDT_objdir)/setup.py "$(CPPFLAGS)" $^
+	mv _libfdt.so $(PYMODULE)
+
+$(WRAP): $(PYLIBFDT_srcdir)/libfdt.swig
+	@$(VECHO) SWIG $@
+	swig -python -o $@ $<
+
+PYLIBFDT_cleanfiles = libfdt_wrap.c libfdt.py libfdt.pyc
+PYLIBFDT_CLEANFILES = $(addprefix $(PYLIBFDT_objdir)/,$(PYLIBFDT_cleanfiles))
pylibfdt/libfdt.swig new file +97 -0
new file mode 100644
index 0000000..ac478d5
@@ -0,0 +1,565 @@
+/*
+ * pylibfdt - Flat Device Tree manipulation in Python
+ * Copyright (C) 2017 Google, Inc.
+ * Written by Simon Glass <[email protected]>
+ *
+ * libfdt is dual licensed: you can use it either under the terms of
+ * the GPL, or the BSD license, at your option.
+ *
+ *  a) This library is free software; you can redistribute it and/or
+ *     modify it under the terms of the GNU General Public License as
+ *     published by the Free Software Foundation; either version 2 of the
+ *     License, or (at your option) any later version.
+ *
+ *     This library is distributed in the hope that it will be useful,
+ *     but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *     GNU General Public License for more details.
+ *
+ *     You should have received a copy of the GNU General Public
+ *     License along with this library; if not, write to the Free
+ *     Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston,
+ *     MA 02110-1301 USA
+ *
+ * Alternatively,
+ *
+ *  b) Redistribution and use in source and binary forms, with or
+ *     without modification, are permitted provided that the following
+ *     conditions are met:
+ *
+ *     1. Redistributions of source code must retain the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer.
+ *     2. Redistributions in binary form must reproduce the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer in the documentation and/or other materials
+ *        provided with the distribution.
+ *
+ *     THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ *     CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ *     INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+ *     MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ *     DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ *     CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *     SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ *     NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ *     LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ *     HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ *     CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
+ *     OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
+ *     EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+%module libfdt
+
+%{
+#define SWIG_FILE_WITH_INIT
+#include "libfdt.h"
+%}
+
+%pythoncode %{
+
+import struct
+
+# Error codes, corresponding to FDT_ERR_... in libfdt.h
+(NOTFOUND,
+        EXISTS,
+        NOSPACE,
+        BADOFFSET,
+        BADPATH,
+        BADPHANDLE,
+        BADSTATE,
+        TRUNCATED,
+        BADMAGIC,
+        BADVERSION,
+        BADSTRUCTURE,
+        BADLAYOUT,
+        INTERNAL,
+        BADNCELLS,
+        BADVALUE,
+        BADOVERLAY) = range(1, 17)
+
+class FdtException(Exception):
+    """An exception caused by an error such as one of the codes above"""
+    def __init__(self, err):
+        self.err = err
+
+    def __str__(self):
+        return 'pylibfdt error %d: %s' % (self.err, fdt_strerror(self.err))
+
+def fdt32_to_cpu(val):
+    """Convert a device-tree cell value into a native integer"""
+    return struct.unpack("=I", struct.pack(">I", val))[0]
+
+def data(prop):
+    """Extract the data from a property
+
+    This is an internal function only.

pylibfdt/libfdt.swig line 107

Quoted 10 lines
pylibfdt/libfdt.swig new file +10 -0
+
+    Args:
+        prop: Property structure, as returned by get_property_by_offset()
+
+    Returns:
+        The property data as a bytearray
+    """
+    buf = bytearray(fdt32_to_cpu(prop.len))
+    pylibfdt_copy_data(buf, prop)
+    return buf

AFAICT this data() function (and the C functions it uses) aren't actually used any more. I think you can remove them.

Quoted 19 lines
pylibfdt/libfdt.swig new file +19 -0
+
+def strerror(fdt_err):
+    """Get the string for an error number
+
+    Args:
+        fdt_err: Error number (-ve)
+
+    Returns:
+        String containing the associated error
+    """
+    return fdt_strerror(fdt_err)
+
+def check_err(val, quiet=False):
+    """Raise an error if the return value is -ve
+
+    This is used to check for errors returned by libfdt C functions.
+
+    Args:
+        val: Return value from a libfdt function

pylibfdt/libfdt.swig line 136

Quoted 10 lines
pylibfdt/libfdt.swig new file +10 -0
+        quiet: True to ignore the NOTFOUND error, False to raise on all errors
+
+    Returns:
+        val if val >= 0
+
+    Raises
+        FdtException if val < 0
+    """
+    if val < 0:
+        if not quiet or val != -NOTFOUND:

I don't really like hardcoding the specialness of NOTFOUND here - depending on the call, there could be different error codes which are relatively harmless.

Instead of a quiet boolean, you could pass in a set (list, sequence, whatever) of error codes to be considered quiet.

Quoted 26 lines
pylibfdt/libfdt.swig new file +26 -0
+            raise FdtException(val)
+    return val
+
+def check_err_null(val, quiet=False):
+    """Raise an error if the return value is NULL
+
+    This is used to check for a NULL return value from certain libfdt C
+    functions
+
+    Args:
+        val: Return value from a libfdt function
+        quiet: True to ignore the NOTFOUND error, False to raise on all errors
+
+    Returns:
+        val if val is a list, None if not
+
+    Raises
+        FdtException if quiet is False and val indicates an error was
+           reported. If quiet if True then an FdtException is raised only if
+           the error is something other than -NOTFOUND.
+    """
+    # Normally a tuple is returned which contains the data and its length.
+    # If we get just an integer error code, it means the function failed.
+    if not isinstance(val, list):
+        if not quiet or val != -NOTFOUND:
+            raise FdtException(val)

pylibfdt/libfdt.swig line 172

Quoted 10 lines
pylibfdt/libfdt.swig new file +10 -0
+        return None,
+    return val
+
+class Fdt:
+    """Device tree class, supporting all operations
+
+    The Fdt object is created is created from a device tree binary file,
+    e.g. with something like:
+
+       fdt = Fdt(open("filename.dtb").read())

I noticed when playing with this that doing Fdt("filename.dtb") will appear to succeed, but then (of course) fail with BADMAGIC as soon as you do anything. Might be an idea to at least do a magic number check in the constructor.

Quoted 47 lines
pylibfdt/libfdt.swig new file +47 -0
+
+    Operations can then be performed using the methods in this class. Each
+    method xxx(args...) corresponds to a libfdt function fdt_xxx(fdt, args...).
+
+    Almost all methods raise an FdtException if an error occurs. The
+    following does not:
+
+        string() - since it has no error checking
+
+    To avoid this behaviour a 'quiet' version is provided for some functions.
+    This behaves as for the normal version except that it will not raise
+    an exception in the case of an FDT_ERR_NOTFOUND error: it will simply
+    return the -NOTFOUND error code or None.
+    """
+    def __init__(self, data):
+        self._fdt = bytearray(data)
+
+    def string(self, offset):
+        """Get a string given its offset
+
+        This is an internal function.
+
+        Args:
+            offset: FDT offset in big-endian format
+
+        Returns:
+            string value at that offset
+        """
+        return fdt_string(self._fdt, fdt32_to_cpu(offset))
+
+    def path_offset(self, path):
+        """Get the offset for a given path
+
+        Args:
+            path: Path to the required node, e.g. '/node@3/subnode@1'
+
+        Returns:
+            Node offset
+
+        Raises
+            FdtException if the path is not valid
+        """
+        return check_err(fdt_path_offset(self._fdt, path))
+
+    def path_offset_quiet(self, path):
+        """Get the offset for a given path
+

pylibfdt/libfdt.swig line 229

Quoted 10 lines
pylibfdt/libfdt.swig new file +10 -0
+        Args:
+            path: Path to the required node, e.g. '/node@3/subnode@1'
+
+        Returns:
+            Node offset, or -NOTFOUND if the path is not value
+
+        Raises
+            FdtException if any error occurs other than NOTFOUND
+        """
+        return check_err(fdt_path_offset(self._fdt, path), True)

Ugh. I don't much like the idea of making quiet versions of a bunch of entry points. I think we need to pick one:
a) Just return error codes (not very Pythonic, but you can't have everything)
b) Always raise exceptions, and the user has to try: except: to quiet then (creating different error classes for the different codes could make this easier).

Quoted 69 lines
pylibfdt/libfdt.swig new file +69 -0
+    def first_property_offset(self, nodeoffset):
+        """Get the offset of the first property in a node offset
+
+        Args:
+            nodeoffset: Offset to the node to check
+
+        Returns:
+            Offset of the first property
+
+        Raises
+            FdtException if the associated node has no properties, or some
+                other error occurred
+        """
+        return check_err(fdt_first_property_offset(self._fdt, nodeoffset))
+
+    def first_property_offset_quiet(self, nodeoffset):
+        """Get the offset of the first property in a node offset
+
+        Args:
+            nodeoffset: Offset to the node to check
+
+        Returns:
+            Offset of the first property, or -NOTFOUND if the node has no
+                properties
+
+        Raises
+            FdtException if any other error occurs
+        """
+        return check_err(fdt_first_property_offset(self._fdt, nodeoffset), True)
+
+    def next_property_offset(self, prop_offset):
+        """Get the next property in a node
+
+        Args:
+            prop_offset: Offset of the previous property
+
+        Returns:
+            Offset of the next property
+
+        Raises:
+            FdtException if the associated node has no more properties, or
+                some other error occurred
+        """
+        return check_err(fdt_next_property_offset(self._fdt, prop_offset))
+
+    def next_property_offset_quiet(self, prop_offset):
+        """Get the next property in a node
+
+        Args:
+            prop_offset: Offset of the previous property
+
+        Returns:
+            Offset ot the next property, or -NOTFOUND if there are no more
+            properties
+
+        Raises:
+            FdtException if any other error occurs
+        """
+        return check_err(fdt_next_property_offset(self._fdt, prop_offset), True)
+
+    def get_name(self, nodeoffset):
+        """Get the name of a node
+
+        Args:
+            nodeoffset: Offset of node to check
+
+        Returns:
+            Node name
+

pylibfdt/libfdt.swig line 309

Quoted 10 lines
pylibfdt/libfdt.swig new file +10 -0
+        Raises:
+            FdtException on error (e.g. nodeoffset is invalid)
+        """
+        return check_err_null(fdt_get_name(self._fdt, nodeoffset))[0]
+
+    def get_property_by_offset(self, prop_offset):
+        """Obtains a property that can be examined
+
+        TODO([email protected]): Consider returning a property object instead,
+        with the data extracted from the Fdt.

AFAICT you're already copying the property value out of the fdt as a bytestring / bytearray. I don't see what a property object would encode apart from that bytestring.

Quoted 138 lines
pylibfdt/libfdt.swig new file +138 -0
+        Args:
+            prop_offset: Offset of property (e.g. from first_property_offset())
+
+        Returns:
+            Property object with members:
+                tag: Big-endian device tree tag value
+                len: Big-endian property length
+                nameoff: Big-endian string offset for use with string()
+
+            Use data() on the return value to obtain the property value.
+
+        Raises:
+            FdtException on error (e.g. invalid prop_offset or device
+            tree format)
+        """
+        return check_err_null(fdt_get_property_by_offset(self._fdt,
+                                                         prop_offset))[0]
+
+    def first_subnode(self, nodeoffset):
+        """Find the first subnode of a parent node
+
+        Args:
+            nodeoffset: Node offset of parent node
+
+        Returns:
+            The offset of the first subnode, if any
+
+        Raises:
+            FdtException if no subnode found or other error occurs
+        """
+        return check_err(fdt_first_subnode(self._fdt, nodeoffset))
+
+    def first_subnode_quiet(self, nodeoffset):
+        """Find the first subnode of a node
+
+        Args:
+            nodeoffset: Node offset of parent node
+
+        Returns:
+            The offset of the first subnode, or -NOTFOUND if none
+
+        Raises:
+            FdtException on error (e.g. invalid nodeoffset)
+        """
+        return check_err(fdt_first_subnode(self._fdt, nodeoffset), True)
+
+    def next_subnode(self, nodeoffset):
+        """Find the next subnode
+
+        Args:
+            nodeoffset: Node offset of previous subnode
+
+        Returns:
+            The offset of the next subnode, if any
+
+        Raises:
+            FdtException if no more subnode found or other error occurs
+        """
+        return check_err(fdt_next_subnode(self._fdt, nodeoffset))
+
+    def next_subnode_quiet(self, nodeoffset):
+        """Find the next subnode
+
+        Args:
+            nodeoffset: Node offset of previous subnode
+
+        Returns:
+            The offset of the next subnode, or -NOTFOUND if none
+
+        Raises:
+            FdtException on error (e.g. invalid nodeoffset)
+        """
+        return check_err(fdt_next_subnode(self._fdt, nodeoffset), True)
+
+    def totalsize(self):
+        """Return the total size of the device tree
+
+        Returns:
+            Total tree size in bytes
+
+        Raises:
+            FdtException if any error occurs
+        """
+        return check_err(fdt_totalsize(self._fdt))
+
+    def off_dt_struct(self):
+        """Return the start of the device tree struct area
+
+        Returns:
+            Start offset of struct area
+
+        Raises:
+            FdtException if any error occurs
+        """
+        return check_err(fdt_off_dt_struct(self._fdt))
+
+    def pack(self):
+        """Pack the device tree to remove unused space
+
+        This adjusts the tree in place.
+
+        Raises:
+            FdtException if any error occurs
+        """
+        return check_err(fdt_pack(self._fdt))
+
+    def delprop(self, nodeoffset, prop_name):
+        """Delete a property from a node
+
+        Args:
+            nodeoffset: Node offset containing property to delete
+            prop_name: Name of property to delete
+
+        Raises:
+            FdtError if the property does not exist, or another error occurs
+        """
+        return check_err(fdt_delprop(self._fdt, nodeoffset, prop_name))
+
+    def getprop(self, nodeoffset, prop_name):
+        """Get a property from a node
+
+        Args:
+            nodeoffset: Node offset containing property to get
+            prop_name: Name of property to get
+
+        Returns:
+            Value of property as a string
+
+        Raises:
+            FdtError if any error occurs (e.g. the property is not found)
+        """
+        return check_err_null(fdt_getprop(self._fdt, nodeoffset, prop_name))[0]
+
+    def getprop_quiet(self, nodeoffset, prop_name):
+        """Get a property from a node
+
+        Args:
+            nodeoffset: Node offset containing property to get

pylibfdt/libfdt.swig line 458

Quoted 10 lines
pylibfdt/libfdt.swig new file +10 -0
+            prop_name: Name of property to get
+
+        Returns:
+            Value of property as a string, or None if not found
+
+        Raises:
+            FdtError if an error occurs (e.g. nodeoffset is invalid)
+        """
+        return check_err_null(fdt_getprop(self._fdt, nodeoffset, prop_name),
+                              True)[0]

There are a bunch of missing things here. e.g. fdt_subnode_offset(), off_dt_strings(), fdt_parent_offset(), ...

Quoted 147 lines
pylibfdt/libfdt.swig new file +107 -0
+%}
+
+%rename(fdt_property) fdt_property_func;
+
+typedef int fdt32_t;
+
+%include "libfdt/fdt.h"
+
+%include "typemaps.i"
+
+/*
+ * Unfortunately the defintiion of pybuffer_mutable_binary() in my Python
+ * version appears to be broken:
+ * pylibfdt/libfdt_wrap.c: In function ‘_wrap_pylibfdt_copy_data’:
+ * pylibfdt/libfdt_wrap.c:3603:22: error: ‘size’ undeclared (first use in this
+ * function)
+ *   arg2 = (size_t) (size/sizeof(char));
+ *
+ * This version works correctly.
+ */
+%define %mypybuffer_mutable_binary(TYPEMAP, SIZE)
+%typemap(in) (TYPEMAP, SIZE)(int res, Py_ssize_t size = 0, void *buf = 0)
+{
+	res = PyObject_AsWriteBuffer($input, &buf, &size);
+	if (res < 0) {
+		PyErr_Clear();
+		%argument_fail(res, "(TYPEMAP, SIZE)", $symname, $argnum);
+	}
+	$1 = ($1_ltype)buf;
+	$2 = ($2_ltype)(size1 / sizeof($*1_type));
+}
+%enddef
+
+/* This is used to copy property data into a bytearray */
+%mypybuffer_mutable_binary(char *str, size_t size);
+void pylibfdt_copy_data(char *str, size_t size,
+			const struct fdt_property *prop);
+
+/* Most functions don't change the device tree, so use a const void * */
+%typemap(in) (const void *) {
+	if (!PyByteArray_Check($input)) {
+		SWIG_exception_fail(SWIG_TypeError, "in method '" "$symname"
+			"', argument " "$argnum"" of type '" "$type""'");
+	}
+	$1 = (void *)PyByteArray_AsString($input);
+}
+
+/* Some functions do change the device tree, so use void * */
+%typemap(in) (void *) {
+	if (!PyByteArray_Check($input)) {
+		SWIG_exception_fail(SWIG_TypeError, "in method '" "$symname"
+			"', argument " "$argnum"" of type '" "$type""'");
+	}
+	$1 = PyByteArray_AsString($input);
+}
+
+%inline %{
+
+/**
+ * pylibfdt_copy_data() - Copy data from a property to the given buffer
+ *
+ * This is used by the data() function to place the contents of a property
+ * into a bytearray.
+ *
+ * @buf: Destination pointer (typically the start of the bytearray)
+ * @size: Number of bytes to copy (size of bytearray)
+ * @prop: Property to copy
+ */
+void pylibfdt_copy_data(char *buf, size_t size, const struct fdt_property *prop)
+{
+	memcpy(buf, prop + 1, size);
+}
+
+%}
+
+%apply int *OUTPUT { int *lenp };
+
+/* Handle a few special cases which conflict with the typemap below */
+const void *fdt_offset_ptr(const void *fdt, int offset, unsigned int checklen);
+const void *fdt_getprop_namelen(const void *fdt, int nodeoffset,
+				const char *name, int namelen, int *lenp);
+
+/* typemap used for fdt_getprop() */
+%typemap(out) (const void *) {
+	if (!$1)
+		$result = Py_None;
+	else
+		/* TODO([email protected]): Can we avoid the 'arg4'? */
+		$result = Py_BuildValue("s#", $1, *arg4);
+}
+
+/* We have both struct fdt_property and a function fdt_property() */
+%warnfilter(302) fdt_property;
+
+%include <../libfdt/libfdt.h>
+
+/* These are macros in the header so have to be redefined here */
+int fdt_magic(const void *fdt);
+int fdt_totalsize(const void *fdt);
+int fdt_off_dt_struct(const void *fdt);
+int fdt_off_dt_strings(const void *fdt);
+int fdt_off_mem_rsvmap(const void *fdt);
+int fdt_version(const void *fdt);
+int fdt_last_comp_version(const void *fdt);
+int fdt_boot_cpuid_phys(const void *fdt);
+int fdt_size_dt_strings(const void *fdt);
+int fdt_size_dt_struct(const void *fdt);
pylibfdt/setup.py new file +34 -0
new file mode 100644
index 0000000..8f8618e
@@ -0,0 +1,34 @@
+#!/usr/bin/env python
+
+"""
+setup.py file for SWIG libfdt
+"""
+
+from distutils.core import setup, Extension
+import os
+import sys
+
+progname = sys.argv[0]
+cflags = sys.argv[1]
+files = sys.argv[2:]
+
+if cflags:
+    cflags = [flag for flag in cflags.split(' ') if flag]
+else:
+    cflags = None
+
+libfdt_module = Extension(
+    '_libfdt',
+    sources = files,
+    extra_compile_args =  cflags
+)
+
+sys.argv = [progname, '--quiet', 'build_ext', '--inplace']
+
+setup (name = 'libfdt',
+       version = '0.1',
+       author      = "SWIG Docs",
+       description = """Simple swig libfdt from docs""",
+       ext_modules = [libfdt_module],
+       py_modules = ["libfdt"],
+       )
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#5 lore raw

Re: [PATCH v4 1/5] Add an initial Python library for libfdt

Hi David,

Quoted 39 lines · David Gibson
Quoted 38 lines · Simon Glass

Add Python bindings for a bare-bones set of libfdt functions. These allow navigating the tree and reading node names and properties.

---

Changes in v4:
- Make the library less pythonic to avoid a shaky illusion
- Drop classes for Node and Prop, along with associated methods
- Include libfdt.h instead of repeating it
- Add support for fdt_getprop()
- Bring in all libfdt functions (but Python support is missing for many)
- Add full comments for Python methods

Changes in v3:
- Make the library more pythonic
- Add classes for Node and Prop along with methods
- Add an exception class
- Use Python to generate exeptions instead of SWIG

Changes in v2:
- Add exceptions when functions return an error
- Correct Python naming to following PEP8
- Use a class to encapsulate the various methods
- Include fdt.h instead of redefining struct fdt_property
- Use bytearray to avoid the SWIG warning 454
- Add comments

 pylibfdt/.gitignore        |   3 +
 pylibfdt/Makefile.pylibfdt |  18 ++
 pylibfdt/libfdt.swig       | 565 +++++++++++++++++++++++++++++++++++++++++++++
 pylibfdt/setup.py          |  34 +++
 4 files changed, 620 insertions(+)
 create mode 100644 pylibfdt/.gitignore
 create mode 100644 pylibfdt/Makefile.pylibfdt
 create mode 100644 pylibfdt/libfdt.swig
 create mode 100644 pylibfdt/setup.py

[..]

Quoted 9 lines
Quoted 9 lines
pylibfdt/libfdt.swig new file +9 -0
+def data(prop):
+    """Extract the data from a property
+
+    This is an internal function only.
+
+    Args:
+        prop: Property structure, as returned by get_property_by_offset()
+
+    Returns:
Quoted 8 lines

pylibfdt/libfdt.swig line 107

Quoted 5 lines
pylibfdt/libfdt.swig new file +5 -0
+        The property data as a bytearray
+    """
+    buf = bytearray(fdt32_to_cpu(prop.len))
+    pylibfdt_copy_data(buf, prop)
+    return buf

AFAICT this data() function (and the C functions it uses) aren't actually used any more. I think you can remove them.

It is used in tests, and needs to be used to process the return value from fdt_get_property_by_offset(). You ask later what we need from that function other than the property contents. We need the property name.

So we could create a Prop class (again) with the name and data in it, and have get_property_by_offset return it. Of course this means that scanning the tree requires allocating space for the property contents all over again, when you use it or not, but this is Python.

Quoted 35 lines
Quoted 20 lines
pylibfdt/libfdt.swig new file +19 -0
+
+def strerror(fdt_err):
+    """Get the string for an error number
+
+    Args:
+        fdt_err: Error number (-ve)
+
+    Returns:
+        String containing the associated error
+    """
+    return fdt_strerror(fdt_err)
+
+def check_err(val, quiet=False):
+    """Raise an error if the return value is -ve
+
+    This is used to check for errors returned by libfdt C functions.
+
+    Args:
+        val: Return value from a libfdt function

pylibfdt/libfdt.swig line 136

Quoted 10 lines
pylibfdt/libfdt.swig new file +10 -0
+        quiet: True to ignore the NOTFOUND error, False to raise on all errors
+
+    Returns:
+        val if val >= 0
+
+    Raises
+        FdtException if val < 0
+    """
+    if val < 0:
+        if not quiet or val != -NOTFOUND:

I don't really like hardcoding the specialness of NOTFOUND here - depending on the call, there could be different error codes which are relatively harmless.

Quoted 2 lines

Instead of a quiet boolean, you could pass in a set (list, sequence, whatever) of error codes to be considered quiet.

OK I can do that, except that later you suggest that the whole idea is wrong. See below.

Quoted 34 lines
Quoted 34 lines
pylibfdt/libfdt.swig new file +33 -0
+            raise FdtException(val)
+    return val
+
+def check_err_null(val, quiet=False):
+    """Raise an error if the return value is NULL
+
+    This is used to check for a NULL return value from certain libfdt C
+    functions
+
+    Args:
+        val: Return value from a libfdt function
+        quiet: True to ignore the NOTFOUND error, False to raise on all errors
+
+    Returns:
+        val if val is a list, None if not
+
+    Raises
+        FdtException if quiet is False and val indicates an error was
+           reported. If quiet if True then an FdtException is raised only if
+           the error is something other than -NOTFOUND.
+    """
+    # Normally a tuple is returned which contains the data and its length.
+    # If we get just an integer error code, it means the function failed.
+    if not isinstance(val, list):
+        if not quiet or val != -NOTFOUND:
+            raise FdtException(val)
+        return None,
+    return val
+
+class Fdt:
+    """Device tree class, supporting all operations
+
+    The Fdt object is created is created from a device tree binary file,
Quoted 8 lines

pylibfdt/libfdt.swig line 172

Quoted 3 lines
pylibfdt/libfdt.swig new file +3 -0
+    e.g. with something like:
+
+       fdt = Fdt(open("filename.dtb").read())

I noticed when playing with this that doing Fdt("filename.dtb") will appear to succeed, but then (of course) fail with BADMAGIC as soon as you do anything. Might be an idea to at least do a magic number check in the constructor.

OK I'll add that.

Quoted 59 lines
Quoted 48 lines
pylibfdt/libfdt.swig new file +47 -0
+
+    Operations can then be performed using the methods in this class. Each
+    method xxx(args...) corresponds to a libfdt function fdt_xxx(fdt, args...).
+
+    Almost all methods raise an FdtException if an error occurs. The
+    following does not:
+
+        string() - since it has no error checking
+
+    To avoid this behaviour a 'quiet' version is provided for some functions.
+    This behaves as for the normal version except that it will not raise
+    an exception in the case of an FDT_ERR_NOTFOUND error: it will simply
+    return the -NOTFOUND error code or None.
+    """
+    def __init__(self, data):
+        self._fdt = bytearray(data)
+
+    def string(self, offset):
+        """Get a string given its offset
+
+        This is an internal function.
+
+        Args:
+            offset: FDT offset in big-endian format
+
+        Returns:
+            string value at that offset
+        """
+        return fdt_string(self._fdt, fdt32_to_cpu(offset))
+
+    def path_offset(self, path):
+        """Get the offset for a given path
+
+        Args:
+            path: Path to the required node, e.g. '/node@3/subnode@1'
+
+        Returns:
+            Node offset
+
+        Raises
+            FdtException if the path is not valid
+        """
+        return check_err(fdt_path_offset(self._fdt, path))
+
+    def path_offset_quiet(self, path):
+        """Get the offset for a given path
+

pylibfdt/libfdt.swig line 229

Quoted 10 lines
pylibfdt/libfdt.swig new file +10 -0
+        Args:
+            path: Path to the required node, e.g. '/node@3/subnode@1'
+
+        Returns:
+            Node offset, or -NOTFOUND if the path is not value
+
+        Raises
+            FdtException if any error occurs other than NOTFOUND
+        """
+        return check_err(fdt_path_offset(self._fdt, path), True)
Quoted 7 lines

Ugh. I don't much like the idea of making quiet versions of a bunch of entry points. I think we need to pick one:
a) Just return error codes (not very Pythonic, but you can't have everything)
b) Always raise exceptions, and the user has to try: except: to quiet then (creating different error classes for the different codes could make this easier).

I don't like it either, but on the other hand, you did ask for exceptions.

I'm happy to go back to a) if that suits you.

Re b) this is really ugly IMO. Exceptions shouldn't be used in normal program flow - e.g. they can appear in logs and cause confusion. Also the syntax is not great.

The reason for the 'quiet' function is that it handles 99% of the cases where you get an error return from a libfdt function. Also it means that you don't have to check return values most of the time, but still catch 'bad' things like out of space, bad offset, etc. without blundering onwards.

As I say I don't like having 'quiet' versions of the functions. There is one other option I can think of, which is to add an additional argument to each function like this:

def path_offset(self, path, quiet=False):

Then you can add 'quiet=True' to avoid the exception. Then you use the same function name, and most of the time you get exceptions when there are problems, but it's easy to avoid when you know there is a chance an error will be returned.

If we go that way, then you are probably going to ask what is so special about -NOTFOUND.

So we could make quiet a list (or perhaps Set) of exceptions, like this:

def path_offset(self, path, quiet=None) # or perhaps quiet=[]

and have callers do:

fdt.path_offset("/", quiet=[NOTFOUND])

and I suppose we could define a constant for [NOTFOUND] also.

That might be better all round?

Quoted 76 lines
Quoted 76 lines
pylibfdt/libfdt.swig new file +75 -0
+    def first_property_offset(self, nodeoffset):
+        """Get the offset of the first property in a node offset
+
+        Args:
+            nodeoffset: Offset to the node to check
+
+        Returns:
+            Offset of the first property
+
+        Raises
+            FdtException if the associated node has no properties, or some
+                other error occurred
+        """
+        return check_err(fdt_first_property_offset(self._fdt, nodeoffset))
+
+    def first_property_offset_quiet(self, nodeoffset):
+        """Get the offset of the first property in a node offset
+
+        Args:
+            nodeoffset: Offset to the node to check
+
+        Returns:
+            Offset of the first property, or -NOTFOUND if the node has no
+                properties
+
+        Raises
+            FdtException if any other error occurs
+        """
+        return check_err(fdt_first_property_offset(self._fdt, nodeoffset), True)
+
+    def next_property_offset(self, prop_offset):
+        """Get the next property in a node
+
+        Args:
+            prop_offset: Offset of the previous property
+
+        Returns:
+            Offset of the next property
+
+        Raises:
+            FdtException if the associated node has no more properties, or
+                some other error occurred
+        """
+        return check_err(fdt_next_property_offset(self._fdt, prop_offset))
+
+    def next_property_offset_quiet(self, prop_offset):
+        """Get the next property in a node
+
+        Args:
+            prop_offset: Offset of the previous property
+
+        Returns:
+            Offset ot the next property, or -NOTFOUND if there are no more
+            properties
+
+        Raises:
+            FdtException if any other error occurs
+        """
+        return check_err(fdt_next_property_offset(self._fdt, prop_offset), True)
+
+    def get_name(self, nodeoffset):
+        """Get the name of a node
+
+        Args:
+            nodeoffset: Offset of node to check
+
+        Returns:
+            Node name
+
+        Raises:
+            FdtException on error (e.g. nodeoffset is invalid)
+        """
+        return check_err_null(fdt_get_name(self._fdt, nodeoffset))[0]
+
+    def get_property_by_offset(self, prop_offset):
Quoted 8 lines

pylibfdt/libfdt.swig line 309

Quoted 4 lines
pylibfdt/libfdt.swig new file +4 -0
+        """Obtains a property that can be examined
+
+        TODO([email protected]): Consider returning a property object instead,
+        with the data extracted from the Fdt.

AFAICT you're already copying the property value out of the fdt as a bytestring / bytearray. I don't see what a property object would encode apart from that bytestring.

The property name.

[...]

Quoted 11 lines
Quoted 11 lines
pylibfdt/libfdt.swig new file +11 -0
+    def getprop_quiet(self, nodeoffset, prop_name):
+        """Get a property from a node
+
+        Args:
+            nodeoffset: Node offset containing property to get
+            prop_name: Name of property to get
+
+        Returns:
+            Value of property as a string, or None if not found
+
+        Raises:
Quoted 8 lines

pylibfdt/libfdt.swig line 458

Quoted 4 lines
pylibfdt/libfdt.swig new file +4 -0
+            FdtError if an error occurs (e.g. nodeoffset is invalid)
+        """
+        return check_err_null(fdt_getprop(self._fdt, nodeoffset, prop_name),
+                              True)[0]

There are a bunch of missing things here. e.g. fdt_subnode_offset(), off_dt_strings(), fdt_parent_offset(), ...

Yes, I've only made a start. I don't want to implement the entire thing right away! I want to get something basic agreed and accepted. Then we can build on it. Refactoring everything as we iterate towards a good binding takes time, and the functions I have chosen are fairly representative of common uses.

[..]

Regards,
Simon

A message that is not in the archive (sent off-list or not yet received): CAPnjgZ31rZqoYTg4m=3yAFuT2UuL9i4qc4+w-G34Aq75sdfrfA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#7 lore raw

Re: [PATCH v4 1/5] Add an initial Python library for libfdt

Quoted 62 lines · Simon Glass

Hi David,

Quoted 39 lines · David Gibson
Quoted 38 lines · Simon Glass

Add Python bindings for a bare-bones set of libfdt functions. These allow navigating the tree and reading node names and properties.

---

Changes in v4:
- Make the library less pythonic to avoid a shaky illusion
- Drop classes for Node and Prop, along with associated methods
- Include libfdt.h instead of repeating it
- Add support for fdt_getprop()
- Bring in all libfdt functions (but Python support is missing for many)
- Add full comments for Python methods

Changes in v3:
- Make the library more pythonic
- Add classes for Node and Prop along with methods
- Add an exception class
- Use Python to generate exeptions instead of SWIG

Changes in v2:
- Add exceptions when functions return an error
- Correct Python naming to following PEP8
- Use a class to encapsulate the various methods
- Include fdt.h instead of redefining struct fdt_property
- Use bytearray to avoid the SWIG warning 454
- Add comments

 pylibfdt/.gitignore        |   3 +
 pylibfdt/Makefile.pylibfdt |  18 ++
 pylibfdt/libfdt.swig       | 565 +++++++++++++++++++++++++++++++++++++++++++++
 pylibfdt/setup.py          |  34 +++
 4 files changed, 620 insertions(+)
 create mode 100644 pylibfdt/.gitignore
 create mode 100644 pylibfdt/Makefile.pylibfdt
 create mode 100644 pylibfdt/libfdt.swig
 create mode 100644 pylibfdt/setup.py

[..]

Quoted 9 lines
Quoted 9 lines
pylibfdt/libfdt.swig new file +9 -0
+def data(prop):
+    """Extract the data from a property
+
+    This is an internal function only.
+
+    Args:
+        prop: Property structure, as returned by get_property_by_offset()
+
+    Returns:
Quoted 8 lines

pylibfdt/libfdt.swig line 107

Quoted 5 lines
pylibfdt/libfdt.swig new file +5 -0
+        The property data as a bytearray
+    """
+    buf = bytearray(fdt32_to_cpu(prop.len))
+    pylibfdt_copy_data(buf, prop)
+    return buf

AFAICT this data() function (and the C functions it uses) aren't actually used any more. I think you can remove them.

Quoted 4 lines

It is used in tests, and needs to be used to process the return value from fdt_get_property_by_offset(). You ask later what we need from that function other than the property contents. We need the property name.

Ok.. so why can't you just return a tuple of two bytestrings?

Quoted 4 lines

So we could create a Prop class (again) with the name and data in it, and have get_property_by_offset return it. Of course this means that scanning the tree requires allocating space for the property contents all over again, when you use it or not, but this is Python.

Yeah, I think trying to avoid lots of allocations when working in Python is not really going to fly.

Quoted 154 lines
Quoted 36 lines
Quoted 21 lines
pylibfdt/libfdt.swig new file +19 -0
+
+def strerror(fdt_err):
+    """Get the string for an error number
+
+    Args:
+        fdt_err: Error number (-ve)
+
+    Returns:
+        String containing the associated error
+    """
+    return fdt_strerror(fdt_err)
+
+def check_err(val, quiet=False):
+    """Raise an error if the return value is -ve
+
+    This is used to check for errors returned by libfdt C functions.
+
+    Args:
+        val: Return value from a libfdt function

pylibfdt/libfdt.swig line 136

Quoted 10 lines
pylibfdt/libfdt.swig new file +10 -0
+        quiet: True to ignore the NOTFOUND error, False to raise on all errors
+
+    Returns:
+        val if val >= 0
+
+    Raises
+        FdtException if val < 0
+    """
+    if val < 0:
+        if not quiet or val != -NOTFOUND:

I don't really like hardcoding the specialness of NOTFOUND here - depending on the call, there could be different error codes which are relatively harmless.

Quoted 2 lines

Instead of a quiet boolean, you could pass in a set (list, sequence, whatever) of error codes to be considered quiet.

OK I can do that, except that later you suggest that the whole idea is wrong. See below.

Quoted 34 lines
Quoted 34 lines
pylibfdt/libfdt.swig new file +33 -0
+            raise FdtException(val)
+    return val
+
+def check_err_null(val, quiet=False):
+    """Raise an error if the return value is NULL
+
+    This is used to check for a NULL return value from certain libfdt C
+    functions
+
+    Args:
+        val: Return value from a libfdt function
+        quiet: True to ignore the NOTFOUND error, False to raise on all errors
+
+    Returns:
+        val if val is a list, None if not
+
+    Raises
+        FdtException if quiet is False and val indicates an error was
+           reported. If quiet if True then an FdtException is raised only if
+           the error is something other than -NOTFOUND.
+    """
+    # Normally a tuple is returned which contains the data and its length.
+    # If we get just an integer error code, it means the function failed.
+    if not isinstance(val, list):
+        if not quiet or val != -NOTFOUND:
+            raise FdtException(val)
+        return None,
+    return val
+
+class Fdt:
+    """Device tree class, supporting all operations
+
+    The Fdt object is created is created from a device tree binary file,
Quoted 8 lines

pylibfdt/libfdt.swig line 172

Quoted 3 lines
pylibfdt/libfdt.swig new file +3 -0
+    e.g. with something like:
+
+       fdt = Fdt(open("filename.dtb").read())

I noticed when playing with this that doing Fdt("filename.dtb") will appear to succeed, but then (of course) fail with BADMAGIC as soon as you do anything. Might be an idea to at least do a magic number check in the constructor.

OK I'll add that.

Quoted 59 lines
Quoted 48 lines
pylibfdt/libfdt.swig new file +47 -0
+
+    Operations can then be performed using the methods in this class. Each
+    method xxx(args...) corresponds to a libfdt function fdt_xxx(fdt, args...).
+
+    Almost all methods raise an FdtException if an error occurs. The
+    following does not:
+
+        string() - since it has no error checking
+
+    To avoid this behaviour a 'quiet' version is provided for some functions.
+    This behaves as for the normal version except that it will not raise
+    an exception in the case of an FDT_ERR_NOTFOUND error: it will simply
+    return the -NOTFOUND error code or None.
+    """
+    def __init__(self, data):
+        self._fdt = bytearray(data)
+
+    def string(self, offset):
+        """Get a string given its offset
+
+        This is an internal function.
+
+        Args:
+            offset: FDT offset in big-endian format
+
+        Returns:
+            string value at that offset
+        """
+        return fdt_string(self._fdt, fdt32_to_cpu(offset))
+
+    def path_offset(self, path):
+        """Get the offset for a given path
+
+        Args:
+            path: Path to the required node, e.g. '/node@3/subnode@1'
+
+        Returns:
+            Node offset
+
+        Raises
+            FdtException if the path is not valid
+        """
+        return check_err(fdt_path_offset(self._fdt, path))
+
+    def path_offset_quiet(self, path):
+        """Get the offset for a given path
+

pylibfdt/libfdt.swig line 229

Quoted 10 lines
pylibfdt/libfdt.swig new file +10 -0
+        Args:
+            path: Path to the required node, e.g. '/node@3/subnode@1'
+
+        Returns:
+            Node offset, or -NOTFOUND if the path is not value
+
+        Raises
+            FdtException if any error occurs other than NOTFOUND
+        """
+        return check_err(fdt_path_offset(self._fdt, path), True)
Quoted 7 lines

Ugh. I don't much like the idea of making quiet versions of a bunch of entry points. I think we need to pick one:
a) Just return error codes (not very Pythonic, but you can't have everything)
b) Always raise exceptions, and the user has to try: except: to quiet then (creating different error classes for the different codes could make this easier).

Quoted 7 lines

I don't like it either, but on the other hand, you did ask for exceptions.

I'm happy to go back to a) if that suits you.

Re b) this is really ugly IMO. Exceptions shouldn't be used in normal program flow - e.g. they can appear in logs and cause confusion. Also the syntax is not great.

Yeah, I guess I agree. Although Python does essentially use this approach internally for iterators, it's mostly hidden by the syntactic sugar.

Quoted 22 lines

The reason for the 'quiet' function is that it handles 99% of the cases where you get an error return from a libfdt function. Also it means that you don't have to check return values most of the time, but still catch 'bad' things like out of space, bad offset, etc. without blundering onwards.

As I say I don't like having 'quiet' versions of the functions. There is one other option I can think of, which is to add an additional argument to each function like this:

def path_offset(self, path, quiet=False):

Then you can add 'quiet=True' to avoid the exception. Then you use the same function name, and most of the time you get exceptions when there are problems, but it's easy to avoid when you know there is a chance an error will be returned.

If we go that way, then you are probably going to ask what is so special about -NOTFOUND.

So we could make quiet a list (or perhaps Set) of exceptions, like this:

Quoted 7 lines

def path_offset(self, path, quiet=None) # or perhaps quiet=[]

and have callers do:

fdt.path_offset("/", quiet=[NOTFOUND])

and I suppose we could define a constant for [NOTFOUND] also.

Yes, this looks better than the other alternatives I think - it's basically an extension of what I suggested for check_err(). I think you can exploit duck typing here to let the user use a list or set as they please (basically just allow anything supporting the 'in' operator).

Quoted 1 line

That might be better all round?

Yes, I think so. The remaining question, of course, is whether to a) have the quiet set default to nothing, or b) default to the "usually quiet" errors (NOTFOUND for most functions). (b) will be less verbose for common use cases, but (a) reduces invisible magic which is generally a good goal.

Quoted 85 lines
Quoted 76 lines
Quoted 76 lines
pylibfdt/libfdt.swig new file +75 -0
+    def first_property_offset(self, nodeoffset):
+        """Get the offset of the first property in a node offset
+
+        Args:
+            nodeoffset: Offset to the node to check
+
+        Returns:
+            Offset of the first property
+
+        Raises
+            FdtException if the associated node has no properties, or some
+                other error occurred
+        """
+        return check_err(fdt_first_property_offset(self._fdt, nodeoffset))
+
+    def first_property_offset_quiet(self, nodeoffset):
+        """Get the offset of the first property in a node offset
+
+        Args:
+            nodeoffset: Offset to the node to check
+
+        Returns:
+            Offset of the first property, or -NOTFOUND if the node has no
+                properties
+
+        Raises
+            FdtException if any other error occurs
+        """
+        return check_err(fdt_first_property_offset(self._fdt, nodeoffset), True)
+
+    def next_property_offset(self, prop_offset):
+        """Get the next property in a node
+
+        Args:
+            prop_offset: Offset of the previous property
+
+        Returns:
+            Offset of the next property
+
+        Raises:
+            FdtException if the associated node has no more properties, or
+                some other error occurred
+        """
+        return check_err(fdt_next_property_offset(self._fdt, prop_offset))
+
+    def next_property_offset_quiet(self, prop_offset):
+        """Get the next property in a node
+
+        Args:
+            prop_offset: Offset of the previous property
+
+        Returns:
+            Offset ot the next property, or -NOTFOUND if there are no more
+            properties
+
+        Raises:
+            FdtException if any other error occurs
+        """
+        return check_err(fdt_next_property_offset(self._fdt, prop_offset), True)
+
+    def get_name(self, nodeoffset):
+        """Get the name of a node
+
+        Args:
+            nodeoffset: Offset of node to check
+
+        Returns:
+            Node name
+
+        Raises:
+            FdtException on error (e.g. nodeoffset is invalid)
+        """
+        return check_err_null(fdt_get_name(self._fdt, nodeoffset))[0]
+
+    def get_property_by_offset(self, prop_offset):
Quoted 8 lines

pylibfdt/libfdt.swig line 309

Quoted 4 lines
pylibfdt/libfdt.swig new file +4 -0
+        """Obtains a property that can be examined
+
+        TODO([email protected]): Consider returning a property object instead,
+        with the data extracted from the Fdt.

AFAICT you're already copying the property value out of the fdt as a bytestring / bytearray. I don't see what a property object would encode apart from that bytestring.

Quoted 1 line

The property name.

Hm, true. Alright, how about this: reintroduce a Property object, and also a version of the get_property() vs. getprop() variants which are in the C library. get_property() and get_property_by_offset() will return Property objects which, to be clear, are *copies* of a property name & value, not a handle to a property within the tree. getprop() will return just the value bytestring().

Property should probably be derived from collections.namedtuple, since it is basically just a wrapper on a (name, value) tuple.

Quoted 23 lines

[...]

Quoted 11 lines
Quoted 11 lines
pylibfdt/libfdt.swig new file +11 -0
+    def getprop_quiet(self, nodeoffset, prop_name):
+        """Get a property from a node
+
+        Args:
+            nodeoffset: Node offset containing property to get
+            prop_name: Name of property to get
+
+        Returns:
+            Value of property as a string, or None if not found
+
+        Raises:
Quoted 8 lines

pylibfdt/libfdt.swig line 458

Quoted 4 lines
pylibfdt/libfdt.swig new file +4 -0
+            FdtError if an error occurs (e.g. nodeoffset is invalid)
+        """
+        return check_err_null(fdt_getprop(self._fdt, nodeoffset, prop_name),
+                              True)[0]

There are a bunch of missing things here. e.g. fdt_subnode_offset(), off_dt_strings(), fdt_parent_offset(), ...

Quoted 5 lines

Yes, I've only made a start. I don't want to implement the entire thing right away! I want to get something basic agreed and accepted. Then we can build on it. Refactoring everything as we iterate towards a good binding takes time, and the functions I have chosen are fairly representative of common uses.

Ok, fair enough.

Quoted 5 lines

[..]

Regards,
Simon

Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#8 lore raw

Re: [PATCH v4 1/5] Add an initial Python library for libfdt

Hi David, hi Simon,

Am Montag, 13. Februar 2017, 16:20:22 CET schrieb David Gibson:

Quoted 4 lines · Simon Glass
Quoted 3 lines

Hi David,

On 9 February 2017 at 21:37, David Gibson <[email protected]>

Quoted 70 lines · wrote:
Quoted 63 lines
Quoted 41 lines · Simon Glass
Quoted 40 lines

Add Python bindings for a bare-bones set of libfdt functions. These allow
navigating the tree and reading node names and properties.

---

Changes in v4:
- Make the library less pythonic to avoid a shaky illusion
- Drop classes for Node and Prop, along with associated methods
- Include libfdt.h instead of repeating it
- Add support for fdt_getprop()
- Bring in all libfdt functions (but Python support is missing for many)
- Add full comments for Python methods

Changes in v3:
- Make the library more pythonic
- Add classes for Node and Prop along with methods
- Add an exception class
- Use Python to generate exeptions instead of SWIG

Changes in v2:
- Add exceptions when functions return an error
- Correct Python naming to following PEP8
- Use a class to encapsulate the various methods
- Include fdt.h instead of redefining struct fdt_property
- Use bytearray to avoid the SWIG warning 454
- Add comments

 pylibfdt/.gitignore        |   3 +
 pylibfdt/Makefile.pylibfdt |  18 ++
 pylibfdt/libfdt.swig       | 565
 +++++++++++++++++++++++++++++++++++++++++++++ pylibfdt/setup.py      
    |  34 +++
 4 files changed, 620 insertions(+)
 create mode 100644 pylibfdt/.gitignore
 create mode 100644 pylibfdt/Makefile.pylibfdt
 create mode 100644 pylibfdt/libfdt.swig
 create mode 100644 pylibfdt/setup.py

[..]

Quoted 10 lines
Quoted 10 lines
pylibfdt/libfdt.swig new file +7 -0
+def data(prop):
+    """Extract the data from a property
+
+    This is an internal function only.
+
+    Args:
+        prop: Property structure, as returned by

get_property_by_offset()

(quoted diff) +2 -0
+
+    Returns:
Quoted 8 lines

pylibfdt/libfdt.swig line 107

Quoted 5 lines
pylibfdt/libfdt.swig new file +5 -0
+        The property data as a bytearray
+    """
+    buf = bytearray(fdt32_to_cpu(prop.len))
+    pylibfdt_copy_data(buf, prop)
+    return buf

AFAICT this data() function (and the C functions it uses) aren't actually used any more. I think you can remove them.

Quoted 4 lines

It is used in tests, and needs to be used to process the return value from fdt_get_property_by_offset(). You ask later what we need from that function other than the property contents. We need the property name.

Ok.. so why can't you just return a tuple of two bytestrings?

Quoted 93 lines
Quoted 4 lines

So we could create a Prop class (again) with the name and data in it, and have get_property_by_offset return it. Of course this means that scanning the tree requires allocating space for the property contents all over again, when you use it or not, but this is Python.

Yeah, I think trying to avoid lots of allocations when working in Python is not really going to fly.

Quoted 85 lines
Quoted 29 lines
Quoted 28 lines
pylibfdt/libfdt.swig new file +20 -0
+
+def strerror(fdt_err):
+    """Get the string for an error number
+
+    Args:
+        fdt_err: Error number (-ve)
+
+    Returns:
+        String containing the associated error
+    """
+    return fdt_strerror(fdt_err)
+
+def check_err(val, quiet=False):
+    """Raise an error if the return value is -ve
+
+    This is used to check for errors returned by libfdt C functions.
+
+    Args:
+        val: Return value from a libfdt function
+        quiet: True to ignore the NOTFOUND error, False to raise on

all errors +

pylibfdt/libfdt.swig new file +7 -0
+    Returns:
+        val if val >= 0
+
+    Raises
+        FdtException if val < 0
+    """
+    if val < 0:
Quoted 7 lines
Quoted 1 line

+ if not quiet or val != -NOTFOUND:

I don't really like hardcoding the specialness of NOTFOUND here - depending on the call, there could be different error codes which are relatively harmless.

Instead of a quiet boolean, you could pass in a set (list, sequence, whatever) of error codes to be considered quiet.

OK I can do that, except that later you suggest that the whole idea is wrong. See below.

Quoted 45 lines
Quoted 37 lines
pylibfdt/libfdt.swig new file +7 -0
+            raise FdtException(val)
+    return val
+
+def check_err_null(val, quiet=False):
+    """Raise an error if the return value is NULL
+
+    This is used to check for a NULL return value from certain libfdt

C

pylibfdt/libfdt.swig new file +5 -0
+    functions
+
+    Args:
+        val: Return value from a libfdt function
+        quiet: True to ignore the NOTFOUND error, False to raise on

all errors +

pylibfdt/libfdt.swig new file +6 -0
+    Returns:
+        val if val is a list, None if not
+
+    Raises
+        FdtException if quiet is False and val indicates an error was
+           reported. If quiet if True then an FdtException is raised

only if + the error is something other than -NOTFOUND.

(quoted diff) +2 -0
+    """
+    # Normally a tuple is returned which contains the data and its

length.
+ # If we get just an integer error code, it means the function
failed.

pylibfdt/libfdt.swig new file +10 -0
+    if not isinstance(val, list):
+        if not quiet or val != -NOTFOUND:
+            raise FdtException(val)
+        return None,
+    return val
+
+class Fdt:
+    """Device tree class, supporting all operations
+
+    The Fdt object is created is created from a device tree binary

file,

pylibfdt/libfdt.swig line 172

Quoted 3 lines
pylibfdt/libfdt.swig new file +3 -0
+    e.g. with something like:
+
+       fdt = Fdt(open("filename.dtb").read())

I noticed when playing with this that doing Fdt("filename.dtb") will appear to succeed, but then (of course) fail with BADMAGIC as soon as you do anything. Might be an idea to at least do a magic number check in the constructor.

in general this will fail once we start thinking of creating files, e.g. due to rearchitecting a DT into multiple ones with includes or creating new DTs. However, with taking r/w modes into account the check can be skipped when the file is opened for writing or deferred to a later read call as it is just now...

Quoted 121 lines
Quoted 74 lines

OK I'll add that.

Quoted 64 lines
Quoted 48 lines
(quoted diff) +2 -0
+
+    Operations can then be performed using the methods in this class.

Each
+ method xxx(args...) corresponds to a libfdt function fdt_xxx(fdt,
args...). +

pylibfdt/libfdt.swig new file +6 -0
+    Almost all methods raise an FdtException if an error occurs. The
+    following does not:
+
+        string() - since it has no error checking
+
+    To avoid this behaviour a 'quiet' version is provided for some

functions. + This behaves as for the normal version except that it
will not raise + an exception in the case of an FDT_ERR_NOTFOUND
error: it will simply + return the -NOTFOUND error code or None.

pylibfdt/libfdt.swig new file +34 -0
+    """
+    def __init__(self, data):
+        self._fdt = bytearray(data)
+
+    def string(self, offset):
+        """Get a string given its offset
+
+        This is an internal function.
+
+        Args:
+            offset: FDT offset in big-endian format
+
+        Returns:
+            string value at that offset
+        """
+        return fdt_string(self._fdt, fdt32_to_cpu(offset))
+
+    def path_offset(self, path):
+        """Get the offset for a given path
+
+        Args:
+            path: Path to the required node, e.g. '/node@3/subnode@1'
+
+        Returns:
+            Node offset
+
+        Raises
+            FdtException if the path is not valid
+        """
+        return check_err(fdt_path_offset(self._fdt, path))
+
+    def path_offset_quiet(self, path):
+        """Get the offset for a given path
+

pylibfdt/libfdt.swig line 229

Quoted 10 lines
pylibfdt/libfdt.swig new file +10 -0
+        Args:
+            path: Path to the required node, e.g. '/node@3/subnode@1'
+
+        Returns:
+            Node offset, or -NOTFOUND if the path is not value
+
+        Raises
+            FdtException if any error occurs other than NOTFOUND
+        """
+        return check_err(fdt_path_offset(self._fdt, path), True)

Ugh. I don't much like the idea of making quiet versions of a bunch

of entry points. I think we need to pick one:
a) Just return error codes (not very Pythonic, but you can't have

Quoted 6 lines

everything)

b) Always raise exceptions, and the user has to try: except: to

quiet then (creating different error classes for the different codes could make this easier).

Quoted 7 lines

I don't like it either, but on the other hand, you did ask for exceptions.

I'm happy to go back to a) if that suits you.

Re b) this is really ugly IMO. Exceptions shouldn't be used in normal program flow - e.g. they can appear in logs and cause confusion. Also the syntax is not great.

Yeah, I guess I agree. Although Python does essentially use this approach internally for iterators, it's mostly hidden by the syntactic sugar.

Quoted 21 lines

The reason for the 'quiet' function is that it handles 99% of the cases where you get an error return from a libfdt function. Also it means that you don't have to check return values most of the time, but still catch 'bad' things like out of space, bad offset, etc. without blundering onwards.

As I say I don't like having 'quiet' versions of the functions. There is one other option I can think of, which is to add an additional

argument to each function like this:
def path_offset(self, path, quiet=False):
Then you can add 'quiet=True' to avoid the exception. Then you use the same function name, and most of the time you get exceptions when there are problems, but it's easy to avoid when you know there is a chance an error will be returned.

If we go that way, then you are probably going to ask what is so special about -NOTFOUND.

So we could make quiet a list (or perhaps Set) of exceptions, like this:

Quoted 7 lines

def path_offset(self, path, quiet=None) # or perhaps quiet=[]

and have callers do:

fdt.path_offset("/", quiet=[NOTFOUND])

and I suppose we could define a constant for [NOTFOUND] also.

Yes, this looks better than the other alternatives I think - it's basically an extension of what I suggested for check_err(). I think you can exploit duck typing here to let the user use a list or set as they please (basically just allow anything supporting the 'in' operator).

Quoted 7 lines
Quoted 1 line

That might be better all round?

Yes, I think so. The remaining question, of course, is whether to a) have the quiet set default to nothing, or b) default to the "usually quiet" errors (NOTFOUND for most functions). (b) will be less verbose for common use cases, but (a) reduces invisible magic which is generally a good goal.

I like the idea of hiding the not so important errors. I would also throw in a little bit different approach to this by adding a property to the Fdt object used to control this. It can be configured by the user during instantiation or later on. This just keeps the interfaces of the single function(s) cleaner. However I am not sure how this is usually handled, so that we could just follow already agreed best practices.

Quoted 123 lines
Quoted 90 lines
Quoted 81 lines
Quoted 81 lines
pylibfdt/libfdt.swig new file +11 -0
+    def first_property_offset(self, nodeoffset):
+        """Get the offset of the first property in a node offset
+
+        Args:
+            nodeoffset: Offset to the node to check
+
+        Returns:
+            Offset of the first property
+
+        Raises
+            FdtException if the associated node has no properties, or

some

pylibfdt/libfdt.swig new file +3 -0
+                other error occurred
+        """
+        return check_err(fdt_first_property_offset(self._fdt,

nodeoffset))

pylibfdt/libfdt.swig new file +9 -0
+
+    def first_property_offset_quiet(self, nodeoffset):
+        """Get the offset of the first property in a node offset
+
+        Args:
+            nodeoffset: Offset to the node to check
+
+        Returns:
+            Offset of the first property, or -NOTFOUND if the node has

no

pylibfdt/libfdt.swig new file +6 -0
+                properties
+
+        Raises
+            FdtException if any other error occurs
+        """
+        return check_err(fdt_first_property_offset(self._fdt,

nodeoffset), True) +

pylibfdt/libfdt.swig new file +11 -0
+    def next_property_offset(self, prop_offset):
+        """Get the next property in a node
+
+        Args:
+            prop_offset: Offset of the previous property
+
+        Returns:
+            Offset of the next property
+
+        Raises:
+            FdtException if the associated node has no more

properties, or

pylibfdt/libfdt.swig new file +3 -0
+                some other error occurred
+        """
+        return check_err(fdt_next_property_offset(self._fdt,

prop_offset))

pylibfdt/libfdt.swig new file +9 -0
+
+    def next_property_offset_quiet(self, prop_offset):
+        """Get the next property in a node
+
+        Args:
+            prop_offset: Offset of the previous property
+
+        Returns:
+            Offset ot the next property, or -NOTFOUND if there are no

more

pylibfdt/libfdt.swig new file +6 -0
+            properties
+
+        Raises:
+            FdtException if any other error occurs
+        """
+        return check_err(fdt_next_property_offset(self._fdt,

prop_offset), True) +

pylibfdt/libfdt.swig new file +15 -0
+    def get_name(self, nodeoffset):
+        """Get the name of a node
+
+        Args:
+            nodeoffset: Offset of node to check
+
+        Returns:
+            Node name
+
+        Raises:
+            FdtException on error (e.g. nodeoffset is invalid)
+        """
+        return check_err_null(fdt_get_name(self._fdt, nodeoffset))[0]
+
+    def get_property_by_offset(self, prop_offset):
Quoted 8 lines
Quoted 4 lines
pylibfdt/libfdt.swig new file +3 -0
+        """Obtains a property that can be examined
+
+        TODO([email protected]): Consider returning a property object

instead, + with the data extracted from the Fdt.

AFAICT you're already copying the property value out of the fdt as a bytestring / bytearray. I don't see what a property object would encode apart from that bytestring.

Quoted 1 line

The property name.

Hm, true. Alright, how about this: reintroduce a Property object, and also a version of the get_property() vs. getprop() variants which are in the C library. get_property() and get_property_by_offset() will return Property objects which, to be clear, are *copies* of a property name & value, not a handle to a property within the tree. getprop() will return just the value bytestring().

Property should probably be derived from collections.namedtuple, since it is basically just a wrapper on a (name, value) tuple.

Quoted 21 lines

[...]

Quoted 18 lines
Quoted 14 lines
pylibfdt/libfdt.swig new file +14 -0
+    def getprop_quiet(self, nodeoffset, prop_name):
+        """Get a property from a node
+
+        Args:
+            nodeoffset: Node offset containing property to get
+            prop_name: Name of property to get
+
+        Returns:
+            Value of property as a string, or None if not found
+
+        Raises:
+            FdtError if an error occurs (e.g. nodeoffset is invalid)
+        """
+        return check_err_null(fdt_getprop(self._fdt, nodeoffset,
Quoted 1 line

prop_name), + True)[0]

There are a bunch of missing things here. e.g. fdt_subnode_offset(), off_dt_strings(), fdt_parent_offset(), ...

Quoted 13 lines
Quoted 5 lines

Yes, I've only made a start. I don't want to implement the entire thing right away! I want to get something basic agreed and accepted. Then we can build on it. Refactoring everything as we iterate towards a good binding takes time, and the functions I have chosen are fairly representative of common uses.

Ok, fair enough.

Quoted 5 lines

[..]

Regards,
Simon

Regards,
Ulrich

#9 lore raw

Re: [PATCH v4 1/5] Add an initial Python library for libfdt

Quoted 171 lines · Ulrich Langenbach

Hi David, hi Simon,

Am Montag, 13. Februar 2017, 16:20:22 CET schrieb David Gibson:

Quoted 4 lines · Simon Glass
Quoted 3 lines

Hi David,

On 9 February 2017 at 21:37, David Gibson <[email protected]>

Quoted 70 lines · wrote:
Quoted 63 lines
Quoted 41 lines · Simon Glass
Quoted 40 lines

Add Python bindings for a bare-bones set of libfdt functions. These allow
navigating the tree and reading node names and properties.

---

Changes in v4:
- Make the library less pythonic to avoid a shaky illusion
- Drop classes for Node and Prop, along with associated methods
- Include libfdt.h instead of repeating it
- Add support for fdt_getprop()
- Bring in all libfdt functions (but Python support is missing for many)
- Add full comments for Python methods

Changes in v3:
- Make the library more pythonic
- Add classes for Node and Prop along with methods
- Add an exception class
- Use Python to generate exeptions instead of SWIG

Changes in v2:
- Add exceptions when functions return an error
- Correct Python naming to following PEP8
- Use a class to encapsulate the various methods
- Include fdt.h instead of redefining struct fdt_property
- Use bytearray to avoid the SWIG warning 454
- Add comments

 pylibfdt/.gitignore        |   3 +
 pylibfdt/Makefile.pylibfdt |  18 ++
 pylibfdt/libfdt.swig       | 565
 +++++++++++++++++++++++++++++++++++++++++++++ pylibfdt/setup.py      
    |  34 +++
 4 files changed, 620 insertions(+)
 create mode 100644 pylibfdt/.gitignore
 create mode 100644 pylibfdt/Makefile.pylibfdt
 create mode 100644 pylibfdt/libfdt.swig
 create mode 100644 pylibfdt/setup.py

[..]

Quoted 10 lines
Quoted 10 lines
(quoted diff) +7 -0
+def data(prop):
+    """Extract the data from a property
+
+    This is an internal function only.
+
+    Args:
+        prop: Property structure, as returned by

get_property_by_offset()

(quoted diff) +2 -0
+
+    Returns:
Quoted 8 lines

Review

Quoted 5 lines
(quoted diff) +5 -0
+        The property data as a bytearray
+    """
+    buf = bytearray(fdt32_to_cpu(prop.len))
+    pylibfdt_copy_data(buf, prop)
+    return buf

AFAICT this data() function (and the C functions it uses) aren't actually used any more. I think you can remove them.

Quoted 4 lines

It is used in tests, and needs to be used to process the return value from fdt_get_property_by_offset(). You ask later what we need from that function other than the property contents. We need the property name.

Ok.. so why can't you just return a tuple of two bytestrings?

Quoted 93 lines
Quoted 4 lines

So we could create a Prop class (again) with the name and data in it, and have get_property_by_offset return it. Of course this means that scanning the tree requires allocating space for the property contents all over again, when you use it or not, but this is Python.

Yeah, I think trying to avoid lots of allocations when working in Python is not really going to fly.

Quoted 85 lines
Quoted 29 lines
Quoted 28 lines
(quoted diff) +20 -0
+
+def strerror(fdt_err):
+    """Get the string for an error number
+
+    Args:
+        fdt_err: Error number (-ve)
+
+    Returns:
+        String containing the associated error
+    """
+    return fdt_strerror(fdt_err)
+
+def check_err(val, quiet=False):
+    """Raise an error if the return value is -ve
+
+    This is used to check for errors returned by libfdt C functions.
+
+    Args:
+        val: Return value from a libfdt function
+        quiet: True to ignore the NOTFOUND error, False to raise on

all errors +

(quoted diff) +7 -0
+    Returns:
+        val if val >= 0
+
+    Raises
+        FdtException if val < 0
+    """
+    if val < 0:
Quoted 7 lines
Quoted 1 line

+ if not quiet or val != -NOTFOUND:

I don't really like hardcoding the specialness of NOTFOUND here - depending on the call, there could be different error codes which are relatively harmless.

Instead of a quiet boolean, you could pass in a set (list, sequence, whatever) of error codes to be considered quiet.

OK I can do that, except that later you suggest that the whole idea is wrong. See below.

Quoted 45 lines
Quoted 37 lines
(quoted diff) +7 -0
+            raise FdtException(val)
+    return val
+
+def check_err_null(val, quiet=False):
+    """Raise an error if the return value is NULL
+
+    This is used to check for a NULL return value from certain libfdt

C

(quoted diff) +5 -0
+    functions
+
+    Args:
+        val: Return value from a libfdt function
+        quiet: True to ignore the NOTFOUND error, False to raise on

all errors +

(quoted diff) +6 -0
+    Returns:
+        val if val is a list, None if not
+
+    Raises
+        FdtException if quiet is False and val indicates an error was
+           reported. If quiet if True then an FdtException is raised

only if + the error is something other than -NOTFOUND.

(quoted diff) +2 -0
+    """
+    # Normally a tuple is returned which contains the data and its

length.
+ # If we get just an integer error code, it means the function
failed.

(quoted diff) +10 -0
+    if not isinstance(val, list):
+        if not quiet or val != -NOTFOUND:
+            raise FdtException(val)
+        return None,
+    return val
+
+class Fdt:
+    """Device tree class, supporting all operations
+
+    The Fdt object is created is created from a device tree binary

file,

Review

Quoted 3 lines
(quoted diff) +3 -0
+    e.g. with something like:
+
+       fdt = Fdt(open("filename.dtb").read())

I noticed when playing with this that doing Fdt("filename.dtb") will appear to succeed, but then (of course) fail with BADMAGIC as soon as you do anything. Might be an idea to at least do a magic number check in the constructor.

Quoted 5 lines

in general this will fail once we start thinking of creating files, e.g. due to rearchitecting a DT into multiple ones with includes or creating new DTs. However, with taking r/w modes into account the check can be skipped when the file is opened for writing or deferred to a later read call as it is just now...

Hrm, not if we're smart about it. I'd envisage several ways of invoking the Fdt constructor. One like this would read in a file and do a basic verification. We'd also need variants for creating a new device tree. We'd need one variant for creating a tree using the sequential write functions, which would call fdt_create() in the constructor. Then we'd need another for creating a fresh tree using the random access read-write functions, which would call fdt_create_empty_tree() in the constructor.

Quoted 128 lines
Quoted 121 lines
Quoted 74 lines

OK I'll add that.

Quoted 64 lines
Quoted 48 lines
(quoted diff) +2 -0
+
+    Operations can then be performed using the methods in this class.

Each
+ method xxx(args...) corresponds to a libfdt function fdt_xxx(fdt,
args...). +

(quoted diff) +6 -0
+    Almost all methods raise an FdtException if an error occurs. The
+    following does not:
+
+        string() - since it has no error checking
+
+    To avoid this behaviour a 'quiet' version is provided for some

functions. + This behaves as for the normal version except that it
will not raise + an exception in the case of an FDT_ERR_NOTFOUND
error: it will simply + return the -NOTFOUND error code or None.

(quoted diff) +34 -0
+    """
+    def __init__(self, data):
+        self._fdt = bytearray(data)
+
+    def string(self, offset):
+        """Get a string given its offset
+
+        This is an internal function.
+
+        Args:
+            offset: FDT offset in big-endian format
+
+        Returns:
+            string value at that offset
+        """
+        return fdt_string(self._fdt, fdt32_to_cpu(offset))
+
+    def path_offset(self, path):
+        """Get the offset for a given path
+
+        Args:
+            path: Path to the required node, e.g. '/node@3/subnode@1'
+
+        Returns:
+            Node offset
+
+        Raises
+            FdtException if the path is not valid
+        """
+        return check_err(fdt_path_offset(self._fdt, path))
+
+    def path_offset_quiet(self, path):
+        """Get the offset for a given path
+

Review

Quoted 10 lines
(quoted diff) +10 -0
+        Args:
+            path: Path to the required node, e.g. '/node@3/subnode@1'
+
+        Returns:
+            Node offset, or -NOTFOUND if the path is not value
+
+        Raises
+            FdtException if any error occurs other than NOTFOUND
+        """
+        return check_err(fdt_path_offset(self._fdt, path), True)

Ugh. I don't much like the idea of making quiet versions of a bunch

of entry points. I think we need to pick one:
a) Just return error codes (not very Pythonic, but you can't have

Quoted 6 lines

everything)

b) Always raise exceptions, and the user has to try: except: to

quiet then (creating different error classes for the different codes could make this easier).

Quoted 7 lines

I don't like it either, but on the other hand, you did ask for exceptions.

I'm happy to go back to a) if that suits you.

Re b) this is really ugly IMO. Exceptions shouldn't be used in normal program flow - e.g. they can appear in logs and cause confusion. Also the syntax is not great.

Yeah, I guess I agree. Although Python does essentially use this approach internally for iterators, it's mostly hidden by the syntactic sugar.

Quoted 21 lines

The reason for the 'quiet' function is that it handles 99% of the cases where you get an error return from a libfdt function. Also it means that you don't have to check return values most of the time, but still catch 'bad' things like out of space, bad offset, etc. without blundering onwards.

As I say I don't like having 'quiet' versions of the functions. There is one other option I can think of, which is to add an additional

argument to each function like this:
def path_offset(self, path, quiet=False):
Then you can add 'quiet=True' to avoid the exception. Then you use the same function name, and most of the time you get exceptions when there are problems, but it's easy to avoid when you know there is a chance an error will be returned.

If we go that way, then you are probably going to ask what is so special about -NOTFOUND.

So we could make quiet a list (or perhaps Set) of exceptions, like this:

Quoted 7 lines

def path_offset(self, path, quiet=None) # or perhaps quiet=[]

and have callers do:

fdt.path_offset("/", quiet=[NOTFOUND])

and I suppose we could define a constant for [NOTFOUND] also.

Yes, this looks better than the other alternatives I think - it's basically an extension of what I suggested for check_err(). I think you can exploit duck typing here to let the user use a list or set as they please (basically just allow anything supporting the 'in' operator).

Quoted 7 lines
Quoted 1 line

That might be better all round?

Yes, I think so. The remaining question, of course, is whether to a) have the quiet set default to nothing, or b) default to the "usually quiet" errors (NOTFOUND for most functions). (b) will be less verbose for common use cases, but (a) reduces invisible magic which is generally a good goal.

Quoted 6 lines

I like the idea of hiding the not so important errors. I would also throw in a little bit different approach to this by adding a property to the Fdt object used to control this. It can be configured by the user during instantiation or later on. This just keeps the interfaces of the single function(s) cleaner. However I am not sure how this is usually handled, so that we could just follow already agreed best practices.

I dislike the idea of putting this into the overall object, because the obvious choices for "quiet" errors varies with the function you're using. NOTFOUND is by far the most common, but EXISTS could also be a non serious error in a number of circumstances. When we start adding write functions, NOSPACE becomes extremely common.

Quoted 140 lines
Quoted 124 lines
Quoted 91 lines
Quoted 82 lines
Quoted 82 lines
(quoted diff) +11 -0
+    def first_property_offset(self, nodeoffset):
+        """Get the offset of the first property in a node offset
+
+        Args:
+            nodeoffset: Offset to the node to check
+
+        Returns:
+            Offset of the first property
+
+        Raises
+            FdtException if the associated node has no properties, or

some

(quoted diff) +3 -0
+                other error occurred
+        """
+        return check_err(fdt_first_property_offset(self._fdt,

nodeoffset))

(quoted diff) +9 -0
+
+    def first_property_offset_quiet(self, nodeoffset):
+        """Get the offset of the first property in a node offset
+
+        Args:
+            nodeoffset: Offset to the node to check
+
+        Returns:
+            Offset of the first property, or -NOTFOUND if the node has

no

(quoted diff) +6 -0
+                properties
+
+        Raises
+            FdtException if any other error occurs
+        """
+        return check_err(fdt_first_property_offset(self._fdt,

nodeoffset), True) +

(quoted diff) +11 -0
+    def next_property_offset(self, prop_offset):
+        """Get the next property in a node
+
+        Args:
+            prop_offset: Offset of the previous property
+
+        Returns:
+            Offset of the next property
+
+        Raises:
+            FdtException if the associated node has no more

properties, or

(quoted diff) +3 -0
+                some other error occurred
+        """
+        return check_err(fdt_next_property_offset(self._fdt,

prop_offset))

(quoted diff) +9 -0
+
+    def next_property_offset_quiet(self, prop_offset):
+        """Get the next property in a node
+
+        Args:
+            prop_offset: Offset of the previous property
+
+        Returns:
+            Offset ot the next property, or -NOTFOUND if there are no

more

(quoted diff) +6 -0
+            properties
+
+        Raises:
+            FdtException if any other error occurs
+        """
+        return check_err(fdt_next_property_offset(self._fdt,

prop_offset), True) +

(quoted diff) +15 -0
+    def get_name(self, nodeoffset):
+        """Get the name of a node
+
+        Args:
+            nodeoffset: Offset of node to check
+
+        Returns:
+            Node name
+
+        Raises:
+            FdtException on error (e.g. nodeoffset is invalid)
+        """
+        return check_err_null(fdt_get_name(self._fdt, nodeoffset))[0]
+
+    def get_property_by_offset(self, prop_offset):
Quoted 8 lines
Quoted 4 lines
(quoted diff) +3 -0
+        """Obtains a property that can be examined
+
+        TODO([email protected]): Consider returning a property object

instead, + with the data extracted from the Fdt.

AFAICT you're already copying the property value out of the fdt as a bytestring / bytearray. I don't see what a property object would encode apart from that bytestring.

Quoted 1 line

The property name.

Hm, true. Alright, how about this: reintroduce a Property object, and also a version of the get_property() vs. getprop() variants which are in the C library. get_property() and get_property_by_offset() will return Property objects which, to be clear, are *copies* of a property name & value, not a handle to a property within the tree. getprop() will return just the value bytestring().

Property should probably be derived from collections.namedtuple, since it is basically just a wrapper on a (name, value) tuple.

Quoted 21 lines

[...]

Quoted 18 lines
Quoted 14 lines
(quoted diff) +14 -0
+    def getprop_quiet(self, nodeoffset, prop_name):
+        """Get a property from a node
+
+        Args:
+            nodeoffset: Node offset containing property to get
+            prop_name: Name of property to get
+
+        Returns:
+            Value of property as a string, or None if not found
+
+        Raises:
+            FdtError if an error occurs (e.g. nodeoffset is invalid)
+        """
+        return check_err_null(fdt_getprop(self._fdt, nodeoffset,
Quoted 1 line

prop_name), + True)[0]

There are a bunch of missing things here. e.g. fdt_subnode_offset(), off_dt_strings(), fdt_parent_offset(), ...

Quoted 13 lines
Quoted 5 lines

Yes, I've only made a start. I don't want to implement the entire thing right away! I want to get something basic agreed and accepted. Then we can build on it. Refactoring everything as we iterate towards a good binding takes time, and the functions I have chosen are fairly representative of common uses.

Ok, fair enough.

Quoted 5 lines

[..]

Regards,
Simon

Regards,
Ulrich

Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

A message that is not in the archive (sent off-list or not yet received): [email protected]

#11 lore raw

Re: [PATCH v4 1/5] Add an initial Python library for libfdt

Hi,

Am Donnerstag, 16. Februar 2017, 13:11:23 CET schrieb David Gibson:

Quoted 192 lines · Ulrich Langenbach
Quoted 185 lines

Hi David, hi Simon,

Am Montag, 13. Februar 2017, 16:20:22 CET schrieb David Gibson:

Quoted 5 lines · Simon Glass
Quoted 4 lines

Hi David,

On 9 February 2017 at 21:37, David Gibson <[email protected]>

Quoted 72 lines · wrote:
Quoted 65 lines
Quoted 43 lines · Simon Glass
Quoted 42 lines

Add Python bindings for a bare-bones set of libfdt functions. These allow
navigating the tree and reading node names and properties.

---

Changes in v4:
- Make the library less pythonic to avoid a shaky illusion
- Drop classes for Node and Prop, along with associated methods
- Include libfdt.h instead of repeating it
- Add support for fdt_getprop()
- Bring in all libfdt functions (but Python support is missing for many)
- Add full comments for Python methods

Changes in v3:
- Make the library more pythonic
- Add classes for Node and Prop along with methods
- Add an exception class
- Use Python to generate exeptions instead of SWIG

Changes in v2:
- Add exceptions when functions return an error
- Correct Python naming to following PEP8
- Use a class to encapsulate the various methods
- Include fdt.h instead of redefining struct fdt_property
- Use bytearray to avoid the SWIG warning 454
- Add comments

 pylibfdt/.gitignore        |   3 +
 pylibfdt/Makefile.pylibfdt |  18 ++
 pylibfdt/libfdt.swig       | 565

+++++++++++++++++++++++++++++++++++++++++++++ pylibfdt/setup.py

    |  34 +++
 4 files changed, 620 insertions(+)
 create mode 100644 pylibfdt/.gitignore
 create mode 100644 pylibfdt/Makefile.pylibfdt
 create mode 100644 pylibfdt/libfdt.swig
 create mode 100644 pylibfdt/setup.py

[..]

Quoted 10 lines
Quoted 10 lines
(quoted diff) +7 -0
+def data(prop):
+    """Extract the data from a property
+
+    This is an internal function only.
+
+    Args:
+        prop: Property structure, as returned by

get_property_by_offset()

(quoted diff) +2 -0
+
+    Returns:
Quoted 8 lines

Review

Quoted 5 lines
(quoted diff) +5 -0
+        The property data as a bytearray
+    """
+    buf = bytearray(fdt32_to_cpu(prop.len))
+    pylibfdt_copy_data(buf, prop)
+    return buf

AFAICT this data() function (and the C functions it uses) aren't actually used any more. I think you can remove them.

Quoted 4 lines

It is used in tests, and needs to be used to process the return value from fdt_get_property_by_offset(). You ask later what we need from that function other than the property contents. We need the property name.

Ok.. so why can't you just return a tuple of two bytestrings?

Quoted 102 lines
Quoted 4 lines

So we could create a Prop class (again) with the name and data in it, and have get_property_by_offset return it. Of course this means that scanning the tree requires allocating space for the property contents all over again, when you use it or not, but this is Python.

Yeah, I think trying to avoid lots of allocations when working in Python is not really going to fly.

Quoted 94 lines
Quoted 31 lines
Quoted 30 lines
(quoted diff) +16 -0
+
+def strerror(fdt_err):
+    """Get the string for an error number
+
+    Args:
+        fdt_err: Error number (-ve)
+
+    Returns:
+        String containing the associated error
+    """
+    return fdt_strerror(fdt_err)
+
+def check_err(val, quiet=False):
+    """Raise an error if the return value is -ve
+
+    This is used to check for errors returned by libfdt C

functions.

(quoted diff) +4 -0
+
+    Args:
+        val: Return value from a libfdt function
+        quiet: True to ignore the NOTFOUND error, False to raise

on
all errors +

(quoted diff) +7 -0
+    Returns:
+        val if val >= 0
+
+    Raises
+        FdtException if val < 0
+    """
+    if val < 0:
Quoted 8 lines
Quoted 1 line

+ if not quiet or val != -NOTFOUND:

I don't really like hardcoding the specialness of NOTFOUND here - depending on the call, there could be different error codes which are
relatively harmless.

Instead of a quiet boolean, you could pass in a set (list, sequence, whatever) of error codes to be considered quiet.

OK I can do that, except that later you suggest that the whole idea is wrong. See below.

Quoted 51 lines
Quoted 41 lines
(quoted diff) +7 -0
+            raise FdtException(val)
+    return val
+
+def check_err_null(val, quiet=False):
+    """Raise an error if the return value is NULL
+
+    This is used to check for a NULL return value from certain

libfdt
C

(quoted diff) +5 -0
+    functions
+
+    Args:
+        val: Return value from a libfdt function
+        quiet: True to ignore the NOTFOUND error, False to raise

on
all errors +

(quoted diff) +5 -0
+    Returns:
+        val if val is a list, None if not
+
+    Raises
+        FdtException if quiet is False and val indicates an error

was
+ reported. If quiet if True then an FdtException is
raised
only if + the error is something other than -NOTFOUND.

(quoted diff) +2 -0
+    """
+    # Normally a tuple is returned which contains the data and its

length.
+ # If we get just an integer error code, it means the function
failed.

(quoted diff) +10 -0
+    if not isinstance(val, list):
+        if not quiet or val != -NOTFOUND:
+            raise FdtException(val)
+        return None,
+    return val
+
+class Fdt:
+    """Device tree class, supporting all operations
+
+    The Fdt object is created is created from a device tree binary

file,

Review

Quoted 3 lines
(quoted diff) +3 -0
+    e.g. with something like:
+
+       fdt = Fdt(open("filename.dtb").read())

I noticed when playing with this that doing Fdt("filename.dtb") will appear to succeed, but then (of course) fail with BADMAGIC as soon as
you do anything. Might be an idea to at least do a magic number check
in the constructor.

Quoted 5 lines

in general this will fail once we start thinking of creating files, e.g. due to rearchitecting a DT into multiple ones with includes or creating new DTs. However, with taking r/w modes into account the check can be skipped when the file is opened for writing or deferred to a later read call as it is just now...

Quoted 8 lines

Hrm, not if we're smart about it. I'd envisage several ways of invoking the Fdt constructor. One like this would read in a file and do a basic verification. We'd also need variants for creating a new device tree. We'd need one variant for creating a tree using the sequential write functions, which would call fdt_create() in the constructor. Then we'd need another for creating a fresh tree using the random access read-write functions, which would call fdt_create_empty_tree() in the constructor.

Ok, that sounds good to me.

Quoted 145 lines
Quoted 138 lines
Quoted 130 lines
Quoted 81 lines

OK I'll add that.

Quoted 71 lines
Quoted 56 lines
(quoted diff) +2 -0
+
+    Operations can then be performed using the methods in this

class.
Each
+ method xxx(args...) corresponds to a libfdt function
fdt_xxx(fdt,
args...). +
+ Almost all methods raise an FdtException if an error occurs.
The

(quoted diff) +5 -0
+    following does not:
+
+        string() - since it has no error checking
+
+    To avoid this behaviour a 'quiet' version is provided for some

functions. + This behaves as for the normal version except that
it
will not raise + an exception in the case of an FDT_ERR_NOTFOUND
error: it will simply + return the -NOTFOUND error code or None.

(quoted diff) +22 -0
+    """
+    def __init__(self, data):
+        self._fdt = bytearray(data)
+
+    def string(self, offset):
+        """Get a string given its offset
+
+        This is an internal function.
+
+        Args:
+            offset: FDT offset in big-endian format
+
+        Returns:
+            string value at that offset
+        """
+        return fdt_string(self._fdt, fdt32_to_cpu(offset))
+
+    def path_offset(self, path):
+        """Get the offset for a given path
+
+        Args:
+            path: Path to the required node, e.g.

'/node@3/subnode@1'

(quoted diff) +14 -0
+
+        Returns:
+            Node offset
+
+        Raises
+            FdtException if the path is not valid
+        """
+        return check_err(fdt_path_offset(self._fdt, path))
+
+    def path_offset_quiet(self, path):
+        """Get the offset for a given path
+
+        Args:
+            path: Path to the required node, e.g.

'/node@3/subnode@1'

Review

Quoted 8 lines
(quoted diff) +8 -0
+
+        Returns:
+            Node offset, or -NOTFOUND if the path is not value
+
+        Raises
+            FdtException if any error occurs other than NOTFOUND
+        """
+        return check_err(fdt_path_offset(self._fdt, path), True)

Ugh. I don't much like the idea of making quiet versions of a bunch

of entry points. I think we need to pick one:
a) Just return error codes (not very Pythonic, but you can't have

Quoted 7 lines

everything)

b) Always raise exceptions, and the user has to try: except: to

quiet then (creating different error classes for the different
codes could make this easier).

Quoted 8 lines

I don't like it either, but on the other hand, you did ask for exceptions.

I'm happy to go back to a) if that suits you.

Re b) this is really ugly IMO. Exceptions shouldn't be used in normal program flow - e.g. they can appear in logs and cause confusion. Also the syntax is not great.

Yeah, I guess I agree. Although Python does essentially use this approach internally for iterators, it's mostly hidden by the syntactic sugar.

Quoted 22 lines

The reason for the 'quiet' function is that it handles 99% of the cases where you get an error return from a libfdt function. Also it means that you don't have to check return values most of the time, but still catch 'bad' things like out of space, bad offset, etc. without blundering onwards.

As I say I don't like having 'quiet' versions of the functions. There is one other option I can think of, which is to add an additional

argument to each function like this:
def path_offset(self, path, quiet=False):
Then you can add 'quiet=True' to avoid the exception. Then you use the same function name, and most of the time you get exceptions when there are problems, but it's easy to avoid when you know there is a chance an error will be returned.

If we go that way, then you are probably going to ask what is so special about -NOTFOUND.

So we could make quiet a list (or perhaps Set) of exceptions, like this:

Quoted 7 lines

def path_offset(self, path, quiet=None) # or perhaps quiet=[]

and have callers do:

fdt.path_offset("/", quiet=[NOTFOUND])

and I suppose we could define a constant for [NOTFOUND] also.

Yes, this looks better than the other alternatives I think - it's basically an extension of what I suggested for check_err(). I think you can exploit duck typing here to let the user use a list or set as they please (basically just allow anything supporting the 'in' operator).

Quoted 7 lines
Quoted 1 line

That might be better all round?

Yes, I think so. The remaining question, of course, is whether to a) have the quiet set default to nothing, or b) default to the "usually quiet" errors (NOTFOUND for most functions). (b) will be less verbose for common use cases, but (a) reduces invisible magic which is generally a good goal.

Quoted 6 lines

I like the idea of hiding the not so important errors. I would also throw in a little bit different approach to this by adding a property to the Fdt object used to control this. It can be configured by the user during instantiation or later on. This just keeps the interfaces of the single function(s) cleaner. However I am not sure how this is usually handled, so that we could just follow already agreed best practices.

Quoted 5 lines

I dislike the idea of putting this into the overall object, because the obvious choices for "quiet" errors varies with the function you're using. NOTFOUND is by far the most common, but EXISTS could also be a non serious error in a number of circumstances. When we start adding write functions, NOSPACE becomes extremely common.

I understand that this definitely is a design trade-off for flexibility. I was just thinking that a lot of people will probably use that lib in a single project for a single use case and then they could adjust the libs behavior for that instance globally. We could also enable both approaches, by setting the default of the functions to None and use a global setting in that case. This would enable users to set both, global default behavior and local specific behavior.
However that could make usage even more complicated than easier.

Quoted 138 lines
Quoted 138 lines
Quoted 138 lines
Quoted 95 lines
Quoted 86 lines
Quoted 86 lines
(quoted diff) +11 -0
+    def first_property_offset(self, nodeoffset):
+        """Get the offset of the first property in a node offset
+
+        Args:
+            nodeoffset: Offset to the node to check
+
+        Returns:
+            Offset of the first property
+
+        Raises
+            FdtException if the associated node has no properties,

or
some

(quoted diff) +3 -0
+                other error occurred
+        """
+        return check_err(fdt_first_property_offset(self._fdt,

nodeoffset))

(quoted diff) +9 -0
+
+    def first_property_offset_quiet(self, nodeoffset):
+        """Get the offset of the first property in a node offset
+
+        Args:
+            nodeoffset: Offset to the node to check
+
+        Returns:
+            Offset of the first property, or -NOTFOUND if the node

has
no

(quoted diff) +6 -0
+                properties
+
+        Raises
+            FdtException if any other error occurs
+        """
+        return check_err(fdt_first_property_offset(self._fdt,

nodeoffset), True) +

(quoted diff) +11 -0
+    def next_property_offset(self, prop_offset):
+        """Get the next property in a node
+
+        Args:
+            prop_offset: Offset of the previous property
+
+        Returns:
+            Offset of the next property
+
+        Raises:
+            FdtException if the associated node has no more

properties, or

(quoted diff) +3 -0
+                some other error occurred
+        """
+        return check_err(fdt_next_property_offset(self._fdt,

prop_offset))

(quoted diff) +9 -0
+
+    def next_property_offset_quiet(self, prop_offset):
+        """Get the next property in a node
+
+        Args:
+            prop_offset: Offset of the previous property
+
+        Returns:
+            Offset ot the next property, or -NOTFOUND if there are

no
more

(quoted diff) +6 -0
+            properties
+
+        Raises:
+            FdtException if any other error occurs
+        """
+        return check_err(fdt_next_property_offset(self._fdt,

prop_offset), True) +

(quoted diff) +13 -0
+    def get_name(self, nodeoffset):
+        """Get the name of a node
+
+        Args:
+            nodeoffset: Offset of node to check
+
+        Returns:
+            Node name
+
+        Raises:
+            FdtException on error (e.g. nodeoffset is invalid)
+        """
+        return check_err_null(fdt_get_name(self._fdt,

nodeoffset))[0]

(quoted diff) +3 -0
+
+    def get_property_by_offset(self, prop_offset):
+        """Obtains a property that can be examined
Quoted 8 lines
Quoted 4 lines
(quoted diff) +2 -0
+
+        TODO([email protected]): Consider returning a property

object
instead, + with the data extracted from the Fdt.

AFAICT you're already copying the property value out of the fdt as a bytestring / bytearray. I don't see what a property object would encode apart from that bytestring.

Quoted 1 line

The property name.

Hm, true. Alright, how about this: reintroduce a Property object, and also a version of the get_property() vs. getprop() variants which are in the C library. get_property() and get_property_by_offset() will return Property objects which, to be clear, are *copies* of a property name & value, not a handle to a property within the tree. getprop() will return just the value bytestring().

Property should probably be derived from collections.namedtuple, since it is basically just a wrapper on a (name, value) tuple.

Quoted 23 lines

[...]

Quoted 12 lines
Quoted 12 lines
(quoted diff) +12 -0
+    def getprop_quiet(self, nodeoffset, prop_name):
+        """Get a property from a node
+
+        Args:
+            nodeoffset: Node offset containing property to get
+            prop_name: Name of property to get
+
+        Returns:
+            Value of property as a string, or None if not found
+
+        Raises:
+            FdtError if an error occurs (e.g. nodeoffset is
Quoted 8 lines
Quoted 4 lines

invalid)

(quoted diff) +2 -0
+        """
+        return check_err_null(fdt_getprop(self._fdt, nodeoffset,

prop_name), + True)[0]

There are a bunch of missing things here. e.g.
fdt_subnode_offset(),
off_dt_strings(), fdt_parent_offset(), ...

Quoted 5 lines

Yes, I've only made a start. I don't want to implement the entire thing right away! I want to get something basic agreed and accepted. Then we can build on it. Refactoring everything as we iterate towards a good binding takes time, and the functions I have chosen are fairly representative of common uses.

Ok, fair enough.

Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines

[..]

Regards,
Simon

Regards,
Ulrich

Regards,
Ulrich

#12 lore raw

Re: [PATCH v4 1/5] Add an initial Python library for libfdt

In reply to a message not in the archive
Quoted 58 lines · Simon Glass

Add Python bindings for a bare-bones set of libfdt functions. These allow navigating the tree and reading node names and properties.

---

Changes in v4:
- Make the library less pythonic to avoid a shaky illusion
- Drop classes for Node and Prop, along with associated methods
- Include libfdt.h instead of repeating it
- Add support for fdt_getprop()
- Bring in all libfdt functions (but Python support is missing for many)
- Add full comments for Python methods

Changes in v3:
- Make the library more pythonic
- Add classes for Node and Prop along with methods
- Add an exception class
- Use Python to generate exeptions instead of SWIG

Changes in v2:
- Add exceptions when functions return an error
- Correct Python naming to following PEP8
- Use a class to encapsulate the various methods
- Include fdt.h instead of redefining struct fdt_property
- Use bytearray to avoid the SWIG warning 454
- Add comments

 pylibfdt/.gitignore        |   3 +
 pylibfdt/Makefile.pylibfdt |  18 ++
 pylibfdt/libfdt.swig       | 565 +++++++++++++++++++++++++++++++++++++++++++++
 pylibfdt/setup.py          |  34 +++
 4 files changed, 620 insertions(+)
 create mode 100644 pylibfdt/.gitignore
 create mode 100644 pylibfdt/Makefile.pylibfdt
 create mode 100644 pylibfdt/libfdt.swig
 create mode 100644 pylibfdt/setup.py
pylibfdt/.gitignore new file +3 -0
new file mode 100644
index 0000000..5e8c5e3
@@ -0,0 +1,3 @@
+libfdt.py
+libfdt.pyc
+libfdt_wrap.c
pylibfdt/Makefile.pylibfdt new file +5 -0
new file mode 100644
index 0000000..fa74dd2
@@ -0,0 +1,18 @@
+# Makefile.pylibfdt
+#
+
+PYLIBFDT_srcs = $(addprefix $(LIBFDT_srcdir)/,$(LIBFDT_SRCS))
+WRAP = $(PYLIBFDT_objdir)/libfdt_wrap.c

pylibfdt/Makefile.pylibfdt line 15

Quoted 10 lines
pylibfdt/Makefile.pylibfdt new file +10 -0
+PYMODULE = $(PYLIBFDT_objdir)/_libfdt.so
+
+$(PYMODULE): $(PYLIBFDT_srcs) $(WRAP)
+	@$(VECHO) PYMOD $@
+	python $(PYLIBFDT_objdir)/setup.py "$(CPPFLAGS)" $^
+	mv _libfdt.so $(PYMODULE)
+
+$(WRAP): $(PYLIBFDT_srcdir)/libfdt.swig
+	@$(VECHO) SWIG $@
+	swig -python -o $@ $<

Forgot to mention this before: please wrap this in a $(SWIG) make variable.

Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#13 lore raw

PATCH v4 2/5 Add tests for pylibfdt

In reply to a message not in the archive

Add a set of tests to cover the functionality in pylibfdt.

---

Changes in v4:
- Drop tests that are no-longer applicable
- Add a get for getprop()

Changes in v3:
- Add some more tests

Changes in v2:
- Update tests for new pylibfdt
- Add a few more tests to increase coverage

 tests/pylibfdt_tests.py | 275 ++++++++++++++++++++++++++++++++++++++++++++++++
 tests/run_tests.sh      |  19 +++-
 2 files changed, 293 insertions(+), 1 deletion(-)
 create mode 100644 tests/pylibfdt_tests.py
tests/pylibfdt_tests.py new file +275 -0
new file mode 100644
index 0000000..66eb45b
@@ -0,0 +1,275 @@
+# pylibfdt - Tests for Flat Device Tree manipulation in Python
+# Copyright (C) 2017 Google, Inc.
+# Written by Simon Glass <[email protected]>
+#
+# libfdt is dual licensed: you can use it either under the terms of
+# the GPL, or the BSD license, at your option.
+#
+#  a) This library is free software; you can redistribute it and/or
+#     modify it under the terms of the GNU General Public License as
+#     published by the Free Software Foundation; either version 2 of the
+#     License, or (at your option) any later version.
+#
+#     This library is distributed in the hope that it will be useful,
+#     but WITHOUT ANY WARRANTY; without even the implied warranty of
+#     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#     GNU General Public License for more details.
+#
+#     You should have received a copy of the GNU General Public
+#     License along with this library; if not, write to the Free
+#     Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston,
+#     MA 02110-1301 USA
+#
+# Alternatively,
+#
+#  b) Redistribution and use in source and binary forms, with or
+#     without modification, are permitted provided that the following
+#     conditions are met:
+#
+#     1. Redistributions of source code must retain the above
+#        copyright notice, this list of conditions and the following
+#        disclaimer.
+#     2. Redistributions in binary form must reproduce the above
+#        copyright notice, this list of conditions and the following
+#        disclaimer in the documentation and/or other materials
+#        provided with the distribution.
+#
+#     THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+#     CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+#     INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+#     MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+#     DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+#     CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#     SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+#     NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+#     LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+#     HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+#     CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
+#     OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
+#     EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+#
+
+import sys
+import unittest
+
+sys.path.append('../pylibfdt')
+import libfdt
+from libfdt import FdtException
+
+# Offsets of properties in the root node
+ROOT_PROPS = (8, 32, 48, 68, 92, 108)
+
+def get_err(err_code):
+    """Convert an error code into an error message
+
+    Args:
+        err_code: Error code value (FDT_ERR_...)
+
+    Returns:
+        String error code
+    """
+    return 'pylibfdt error %d: %s' % (-err_code, libfdt.fdt_strerror(-err_code))
+
+def _ReadFdt(fname):
+    """Read a device tree file into an Fdt object, ready for use
+
+    Args:
+        fname: Filename to read from
+
+    Returns:
+        Fdt bytearray suitable for passing to libfdt functions
+    """
+    return libfdt.Fdt(open(fname).read())
+
+class PyLibfdtTests(unittest.TestCase):
+    """Test class for pylibfdt
+
+    Properties:
+        fdt: Device tree file used for testing
+    """
+
+    def setUp(self):
+        """Read in the device tree we use for testing"""
+        self.fdt = _ReadFdt('test_tree1.dtb')
+
+    def GetPropList(self, node_path):
+        """Read a list of properties from a node
+
+        Args:
+            node_path: Full path to node, e.g. '/subnode@1/subsubnode'
+
+        Returns:
+            List of property names for that node, e.g. ['compatible', 'reg']
+        """
+        prop_list = []
+        node = self.fdt.path_offset(node_path)
+        poffset = self.fdt.first_property_offset_quiet(node)
+        while poffset > 0:
+            pdata = self.fdt.get_property_by_offset(poffset)
+            prop_list.append(self.fdt.string(pdata.nameoff))
+            poffset = self.fdt.next_property_offset_quiet(poffset)
+        return prop_list
+
+    def testImport(self):
+        """Check that we can import the library correctly"""
+        self.assertEquals(type(libfdt), type(sys))
+
+    def testPathOffset(self):
+        """Check that we can find the offset of a node"""
+        self.assertEquals(self.fdt.path_offset('/'), 0)
+        self.assertEquals(self.fdt.path_offset('/subnode@1'), 124)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.NOTFOUND)):
+            self.fdt.path_offset('/wibble')
+        self.assertEquals(self.fdt.path_offset_quiet('/wibble'),
+                          -libfdt.NOTFOUND)
+
+    def testPropertyOffset(self):
+        """Walk through all the properties in the root node"""
+        self.assertEquals(self.fdt.first_property_offset(0), ROOT_PROPS[0])
+        for pos in range(len(ROOT_PROPS) - 1):
+            self.assertEquals(self.fdt.next_property_offset(ROOT_PROPS[pos]),
+                              ROOT_PROPS[pos + 1])
+        self.assertEquals(self.fdt.next_property_offset_quiet(ROOT_PROPS[-1]),
+                          -libfdt.NOTFOUND)
+
+    def testPropertyOffsetExceptions(self):
+        """Check that exceptions are raised as expected"""
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.NOTFOUND)):
+            self.fdt.next_property_offset(108)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.next_property_offset(107)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.first_property_offset_quiet(107)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.next_property_offset_quiet(107)
+
+        node = self.fdt.path_offset('/subnode@1/ss1')
+        self.assertEquals(self.fdt.first_property_offset_quiet(node),
+                          -libfdt.NOTFOUND)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.NOTFOUND)):
+            self.fdt.first_property_offset(node)
+
+    def testGetName(self):
+        """Check that we can get the name of a node"""
+        self.assertEquals(self.fdt.get_name(0), '')
+        node = self.fdt.path_offset('/subnode@1/subsubnode')
+        self.assertEquals(self.fdt.get_name(node), 'subsubnode')
+
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.get_name(-2)
+
+    def testGetString(self):
+        """Test that we can get a string from the string table"""
+        node = self.fdt.path_offset('/subnode@2')
+        poffset = self.fdt.first_property_offset(node)
+        pdata = self.fdt.get_property_by_offset(poffset)
+        self.assertEquals(self.fdt.string(pdata.nameoff), 'reg')
+
+    def testGetPropertyByOffset(self):
+        """Check that we can read the name and contents of a property"""
+        root = self.fdt.path_offset('/')
+        poffset = self.fdt.first_property_offset(root)
+        pdata = self.fdt.get_property_by_offset(poffset)
+        self.assertEquals(libfdt.fdt32_to_cpu(pdata.tag), 3)
+        self.assertEquals(libfdt.fdt32_to_cpu(pdata.len), 11)
+        self.assertEquals(self.fdt.string(pdata.nameoff), 'compatible')
+        self.assertEquals(libfdt.data(pdata), 'test_tree1\0')
+
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.get_property_by_offset(-2)
+
+    def testGetPropertyByOffset(self):
+        """Check that we can read the details of a property"""
+        root = self.fdt.path_offset('/')
+        poffset = self.fdt.first_property_offset(root)
+        prop = self.fdt.get_property_by_offset(poffset)
+        self.assertEquals(self.fdt.string(prop.nameoff), 'compatible')
+        self.assertEquals(libfdt.data(prop), 'test_tree1\0')
+
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.get_property_by_offset(-2)
+
+    def testGetProp(self):
+        """Check that we can read the contents of a property by name"""
+        root = self.fdt.path_offset('/')
+        data = self.fdt.getprop(root, "compatible")
+        self.assertEquals(data, 'test_tree1\0')
+        self.assertEquals(None, self.fdt.getprop_quiet(root, 'missing'))
+
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.NOTFOUND)):
+            self.fdt.getprop(root, 'missing')
+
+        node = self.fdt.path_offset('/subnode@1/subsubnode')
+        data = self.fdt.getprop(node, "compatible")
+        self.assertEquals(data, 'subsubnode1\0subsubnode\0')
+
+    def testStrError(self):
+        """Check that we can get an error string"""
+        self.assertEquals(libfdt.strerror(-libfdt.NOTFOUND),
+                          'FDT_ERR_NOTFOUND')
+
+    def testFirstNextSubnodeOffset(self):
+        """Check that we can walk through subnodes"""
+        node_list = []
+        node = self.fdt.first_subnode_quiet(0)
+        while node >= 0:
+            node_list.append(self.fdt.get_name(node))
+            node = self.fdt.next_subnode_quiet(node)
+        self.assertEquals(node_list, ['subnode@1', 'subnode@2'])
+
+    def testFirstNextSubnodeOffsetExceptions(self):
+        """Check except handling for first/next subnode functions"""
+        node = self.fdt.path_offset_quiet('/subnode@1/subsubnode')
+        self.assertEquals(self.fdt.first_subnode_quiet(node), -libfdt.NOTFOUND)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.NOTFOUND)):
+            self.fdt.first_subnode(node)
+
+        node = self.fdt.path_offset_quiet('/subnode@1/ss1')
+        self.assertEquals(self.fdt.next_subnode_quiet(node), -libfdt.NOTFOUND)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.NOTFOUND)):
+            self.fdt.next_subnode(node)
+
+    def testDeleteProperty(self):
+        """Test that we can delete a property"""
+        node_name = '/subnode@1'
+        self.assertEquals(self.GetPropList(node_name),
+                          ['compatible', 'reg', 'prop-int'])
+        node = self.fdt.path_offset('/%s' % node_name)
+        self.assertEquals(self.fdt.delprop(node, 'reg'), 0)
+        self.assertEquals(self.GetPropList(node_name),
+                          ['compatible', 'prop-int'])
+
+    def testHeader(self):
+        """Test that we can access the header values"""
+        self.assertEquals(self.fdt.totalsize(), 693)
+        self.assertEquals(self.fdt.off_dt_struct(), 88)
+
+    def testPack(self):
+        """Test that we can pack the tree after deleting something"""
+        self.assertEquals(self.fdt.totalsize(), 693)
+        node = self.fdt.path_offset_quiet('/subnode@2')
+        self.assertEquals(self.fdt.delprop(node, 'prop-int'), 0)
+        self.assertEquals(self.fdt.totalsize(), 693)
+        self.assertEquals(self.fdt.pack(), 0)
+        self.assertEquals(self.fdt.totalsize(), 677)
+
+    def testBadPropertyOffset(self):
+        """Test that bad property offsets are detected"""
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.get_property_by_offset(13)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.first_property_offset(3)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.next_property_offset(3)
+
+    def testBadPathOffset(self):
+        """Test that bad path names are detected"""
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADPATH)):
+            self.fdt.path_offset('not-present')
+
+    def testEndian(self):
+        """Check that we can convert from FDT (big endian) to native endian"""
+        self.assertEquals(libfdt.fdt32_to_cpu(0x10000000), 0x10)
+
+if __name__ == "__main__":
+    unittest.main()
tests/run_tests.sh +18 -1
index ed489db..144feb6 100755
@@ -769,6 +769,20 @@ fdtdump_tests () {
     run_fdtdump_test fdtdump.dts
 }
 
+pylibfdt_tests () {
+    TMP=/tmp/tests.stderr.$$
+    python pylibfdt_tests.py 2> ${TMP}
+    result=$(head -1 ${TMP} | awk \
+        '{ for (i = 1; i <= length($0); i++) { \
+            result = substr($0,i,1); fail = fail + (result == "F"); \
+            ok = ok + (result == ".")}; } END { print fail,  ok, fail + ok}')
+
+    # Extract the test results and add them to our totals
+    tot_fail=$((tot_fail + $(echo $result | cut -d" " -f 1)))
+    tot_pass=$((tot_pass + $(echo $result | cut -d" " -f 2)))
+    tot_tests=$((tot_tests + $(echo $result | cut -d" " -f 3)))
+}
+
 while getopts "vt:me" ARG ; do
     case $ARG in
 	"v")
@@ -787,7 +801,7 @@ while getopts "vt:me" ARG ; do
 done
 
 if [ -z "$TESTSETS" ]; then
-    TESTSETS="libfdt utilfdt dtc dtbs_equal fdtget fdtput fdtdump"
+    TESTSETS="libfdt utilfdt dtc dtbs_equal fdtget fdtput fdtdump pylibfdt"
 fi
 
 # Make sure we don't have stale blobs lying around
@@ -816,6 +830,9 @@ for set in $TESTSETS; do
 	"fdtdump")
 	    fdtdump_tests
 	    ;;
+	"pylibfdt")
+	    pylibfdt_tests
+	    ;;
     esac
 done
 
Signature · 1 line
2.11.0.483.g087da7b7c-goog

A message that is not in the archive (sent off-list or not yet received): 20170205201323.15411-3-sjg-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org

#15 lore raw

Re: [PATCH v4 2/5] Add tests for pylibfdt

Quoted 3 lines · Simon Glass

Add a set of tests to cover the functionality in pylibfdt.

I'd suggest as a base writing Python-equivalent tests for everything in tree1_tests().

Quoted 343 lines

---

Changes in v4:
- Drop tests that are no-longer applicable
- Add a get for getprop()

Changes in v3:
- Add some more tests

Changes in v2:
- Update tests for new pylibfdt
- Add a few more tests to increase coverage

 tests/pylibfdt_tests.py | 275 ++++++++++++++++++++++++++++++++++++++++++++++++
 tests/run_tests.sh      |  19 +++-
 2 files changed, 293 insertions(+), 1 deletion(-)
 create mode 100644 tests/pylibfdt_tests.py
tests/pylibfdt_tests.py new file +275 -0
new file mode 100644
index 0000000..66eb45b
@@ -0,0 +1,275 @@
+# pylibfdt - Tests for Flat Device Tree manipulation in Python
+# Copyright (C) 2017 Google, Inc.
+# Written by Simon Glass <[email protected]>
+#
+# libfdt is dual licensed: you can use it either under the terms of
+# the GPL, or the BSD license, at your option.
+#
+#  a) This library is free software; you can redistribute it and/or
+#     modify it under the terms of the GNU General Public License as
+#     published by the Free Software Foundation; either version 2 of the
+#     License, or (at your option) any later version.
+#
+#     This library is distributed in the hope that it will be useful,
+#     but WITHOUT ANY WARRANTY; without even the implied warranty of
+#     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#     GNU General Public License for more details.
+#
+#     You should have received a copy of the GNU General Public
+#     License along with this library; if not, write to the Free
+#     Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston,
+#     MA 02110-1301 USA
+#
+# Alternatively,
+#
+#  b) Redistribution and use in source and binary forms, with or
+#     without modification, are permitted provided that the following
+#     conditions are met:
+#
+#     1. Redistributions of source code must retain the above
+#        copyright notice, this list of conditions and the following
+#        disclaimer.
+#     2. Redistributions in binary form must reproduce the above
+#        copyright notice, this list of conditions and the following
+#        disclaimer in the documentation and/or other materials
+#        provided with the distribution.
+#
+#     THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+#     CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+#     INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+#     MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+#     DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+#     CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#     SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+#     NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+#     LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+#     HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+#     CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
+#     OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
+#     EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+#
+
+import sys
+import unittest
+
+sys.path.append('../pylibfdt')
+import libfdt
+from libfdt import FdtException
+
+# Offsets of properties in the root node
+ROOT_PROPS = (8, 32, 48, 68, 92, 108)
+
+def get_err(err_code):
+    """Convert an error code into an error message
+
+    Args:
+        err_code: Error code value (FDT_ERR_...)
+
+    Returns:
+        String error code
+    """
+    return 'pylibfdt error %d: %s' % (-err_code, libfdt.fdt_strerror(-err_code))
+
+def _ReadFdt(fname):
+    """Read a device tree file into an Fdt object, ready for use
+
+    Args:
+        fname: Filename to read from
+
+    Returns:
+        Fdt bytearray suitable for passing to libfdt functions
+    """
+    return libfdt.Fdt(open(fname).read())
+
+class PyLibfdtTests(unittest.TestCase):
+    """Test class for pylibfdt
+
+    Properties:
+        fdt: Device tree file used for testing
+    """
+
+    def setUp(self):
+        """Read in the device tree we use for testing"""
+        self.fdt = _ReadFdt('test_tree1.dtb')
+
+    def GetPropList(self, node_path):
+        """Read a list of properties from a node
+
+        Args:
+            node_path: Full path to node, e.g. '/subnode@1/subsubnode'
+
+        Returns:
+            List of property names for that node, e.g. ['compatible', 'reg']
+        """
+        prop_list = []
+        node = self.fdt.path_offset(node_path)
+        poffset = self.fdt.first_property_offset_quiet(node)
+        while poffset > 0:
+            pdata = self.fdt.get_property_by_offset(poffset)
+            prop_list.append(self.fdt.string(pdata.nameoff))
+            poffset = self.fdt.next_property_offset_quiet(poffset)
+        return prop_list
+
+    def testImport(self):
+        """Check that we can import the library correctly"""
+        self.assertEquals(type(libfdt), type(sys))
+
+    def testPathOffset(self):
+        """Check that we can find the offset of a node"""
+        self.assertEquals(self.fdt.path_offset('/'), 0)
+        self.assertEquals(self.fdt.path_offset('/subnode@1'), 124)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.NOTFOUND)):
+            self.fdt.path_offset('/wibble')
+        self.assertEquals(self.fdt.path_offset_quiet('/wibble'),
+                          -libfdt.NOTFOUND)
+
+    def testPropertyOffset(self):
+        """Walk through all the properties in the root node"""
+        self.assertEquals(self.fdt.first_property_offset(0), ROOT_PROPS[0])
+        for pos in range(len(ROOT_PROPS) - 1):
+            self.assertEquals(self.fdt.next_property_offset(ROOT_PROPS[pos]),
+                              ROOT_PROPS[pos + 1])
+        self.assertEquals(self.fdt.next_property_offset_quiet(ROOT_PROPS[-1]),
+                          -libfdt.NOTFOUND)
+
+    def testPropertyOffsetExceptions(self):
+        """Check that exceptions are raised as expected"""
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.NOTFOUND)):
+            self.fdt.next_property_offset(108)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.next_property_offset(107)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.first_property_offset_quiet(107)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.next_property_offset_quiet(107)
+
+        node = self.fdt.path_offset('/subnode@1/ss1')
+        self.assertEquals(self.fdt.first_property_offset_quiet(node),
+                          -libfdt.NOTFOUND)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.NOTFOUND)):
+            self.fdt.first_property_offset(node)
+
+    def testGetName(self):
+        """Check that we can get the name of a node"""
+        self.assertEquals(self.fdt.get_name(0), '')
+        node = self.fdt.path_offset('/subnode@1/subsubnode')
+        self.assertEquals(self.fdt.get_name(node), 'subsubnode')
+
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.get_name(-2)
+
+    def testGetString(self):
+        """Test that we can get a string from the string table"""
+        node = self.fdt.path_offset('/subnode@2')
+        poffset = self.fdt.first_property_offset(node)
+        pdata = self.fdt.get_property_by_offset(poffset)
+        self.assertEquals(self.fdt.string(pdata.nameoff), 'reg')
+
+    def testGetPropertyByOffset(self):
+        """Check that we can read the name and contents of a property"""
+        root = self.fdt.path_offset('/')
+        poffset = self.fdt.first_property_offset(root)
+        pdata = self.fdt.get_property_by_offset(poffset)
+        self.assertEquals(libfdt.fdt32_to_cpu(pdata.tag), 3)
+        self.assertEquals(libfdt.fdt32_to_cpu(pdata.len), 11)
+        self.assertEquals(self.fdt.string(pdata.nameoff), 'compatible')
+        self.assertEquals(libfdt.data(pdata), 'test_tree1\0')
+
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.get_property_by_offset(-2)
+
+    def testGetPropertyByOffset(self):
+        """Check that we can read the details of a property"""
+        root = self.fdt.path_offset('/')
+        poffset = self.fdt.first_property_offset(root)
+        prop = self.fdt.get_property_by_offset(poffset)
+        self.assertEquals(self.fdt.string(prop.nameoff), 'compatible')
+        self.assertEquals(libfdt.data(prop), 'test_tree1\0')
+
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.get_property_by_offset(-2)
+
+    def testGetProp(self):
+        """Check that we can read the contents of a property by name"""
+        root = self.fdt.path_offset('/')
+        data = self.fdt.getprop(root, "compatible")
+        self.assertEquals(data, 'test_tree1\0')
+        self.assertEquals(None, self.fdt.getprop_quiet(root, 'missing'))
+
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.NOTFOUND)):
+            self.fdt.getprop(root, 'missing')
+
+        node = self.fdt.path_offset('/subnode@1/subsubnode')
+        data = self.fdt.getprop(node, "compatible")
+        self.assertEquals(data, 'subsubnode1\0subsubnode\0')
+
+    def testStrError(self):
+        """Check that we can get an error string"""
+        self.assertEquals(libfdt.strerror(-libfdt.NOTFOUND),
+                          'FDT_ERR_NOTFOUND')
+
+    def testFirstNextSubnodeOffset(self):
+        """Check that we can walk through subnodes"""
+        node_list = []
+        node = self.fdt.first_subnode_quiet(0)
+        while node >= 0:
+            node_list.append(self.fdt.get_name(node))
+            node = self.fdt.next_subnode_quiet(node)
+        self.assertEquals(node_list, ['subnode@1', 'subnode@2'])
+
+    def testFirstNextSubnodeOffsetExceptions(self):
+        """Check except handling for first/next subnode functions"""
+        node = self.fdt.path_offset_quiet('/subnode@1/subsubnode')
+        self.assertEquals(self.fdt.first_subnode_quiet(node), -libfdt.NOTFOUND)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.NOTFOUND)):
+            self.fdt.first_subnode(node)
+
+        node = self.fdt.path_offset_quiet('/subnode@1/ss1')
+        self.assertEquals(self.fdt.next_subnode_quiet(node), -libfdt.NOTFOUND)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.NOTFOUND)):
+            self.fdt.next_subnode(node)
+
+    def testDeleteProperty(self):
+        """Test that we can delete a property"""
+        node_name = '/subnode@1'
+        self.assertEquals(self.GetPropList(node_name),
+                          ['compatible', 'reg', 'prop-int'])
+        node = self.fdt.path_offset('/%s' % node_name)
+        self.assertEquals(self.fdt.delprop(node, 'reg'), 0)
+        self.assertEquals(self.GetPropList(node_name),
+                          ['compatible', 'prop-int'])
+
+    def testHeader(self):
+        """Test that we can access the header values"""
+        self.assertEquals(self.fdt.totalsize(), 693)
+        self.assertEquals(self.fdt.off_dt_struct(), 88)
+
+    def testPack(self):
+        """Test that we can pack the tree after deleting something"""
+        self.assertEquals(self.fdt.totalsize(), 693)
+        node = self.fdt.path_offset_quiet('/subnode@2')
+        self.assertEquals(self.fdt.delprop(node, 'prop-int'), 0)
+        self.assertEquals(self.fdt.totalsize(), 693)
+        self.assertEquals(self.fdt.pack(), 0)
+        self.assertEquals(self.fdt.totalsize(), 677)
+
+    def testBadPropertyOffset(self):
+        """Test that bad property offsets are detected"""
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.get_property_by_offset(13)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.first_property_offset(3)
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADOFFSET)):
+            self.fdt.next_property_offset(3)
+
+    def testBadPathOffset(self):
+        """Test that bad path names are detected"""
+        with self.assertRaisesRegexp(FdtException, get_err(libfdt.BADPATH)):
+            self.fdt.path_offset('not-present')
+
+    def testEndian(self):
+        """Check that we can convert from FDT (big endian) to native endian"""
+        self.assertEquals(libfdt.fdt32_to_cpu(0x10000000), 0x10)
+
+if __name__ == "__main__":
+    unittest.main()
tests/run_tests.sh +18 -1
index ed489db..144feb6 100755
@@ -769,6 +769,20 @@ fdtdump_tests () {
     run_fdtdump_test fdtdump.dts
 }
 
+pylibfdt_tests () {
+    TMP=/tmp/tests.stderr.$$
+    python pylibfdt_tests.py 2> ${TMP}
+    result=$(head -1 ${TMP} | awk \
+        '{ for (i = 1; i <= length($0); i++) { \
+            result = substr($0,i,1); fail = fail + (result == "F"); \
+            ok = ok + (result == ".")}; } END { print fail,  ok, fail + ok}')
+
+    # Extract the test results and add them to our totals
+    tot_fail=$((tot_fail + $(echo $result | cut -d" " -f 1)))
+    tot_pass=$((tot_pass + $(echo $result | cut -d" " -f 2)))
+    tot_tests=$((tot_tests + $(echo $result | cut -d" " -f 3)))
+}
+
 while getopts "vt:me" ARG ; do
     case $ARG in
 	"v")
@@ -787,7 +801,7 @@ while getopts "vt:me" ARG ; do
 done
 
 if [ -z "$TESTSETS" ]; then
-    TESTSETS="libfdt utilfdt dtc dtbs_equal fdtget fdtput fdtdump"
+    TESTSETS="libfdt utilfdt dtc dtbs_equal fdtget fdtput fdtdump pylibfdt"
 fi
 
 # Make sure we don't have stale blobs lying around
@@ -816,6 +830,9 @@ for set in $TESTSETS; do
 	"fdtdump")
 	    fdtdump_tests
 	    ;;
+	"pylibfdt")
+	    pylibfdt_tests
+	    ;;
     esac
 done
 
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#16 lore raw

Re: [PATCH v4 2/5] Add tests for pylibfdt

Hi David,

Quoted 8 lines · David Gibson
Quoted 3 lines · Simon Glass

Add a set of tests to cover the functionality in pylibfdt.

I'd suggest as a base writing Python-equivalent tests for everything in tree1_tests().

OK I'll go through and do that (for the functions I've chosen to implement) once we have the swig interface set.

[..]

Regards,
Simon

#17 lore raw

PATCH v4 3/5 Mention pylibfdt in the documentation

In reply to a message not in the archive

Add a note about pylibfdt in the README.

---

Changes in v4: None
Changes in v3: None
Changes in v2:
- Add details on how to obtain full help and code coverage

 README | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
README +33 -0
index f92008f..7191d1b 100644
@@ -7,6 +7,39 @@ DTC and LIBFDT are maintained by:
 David Gibson <[email protected]>
 Jon Loeliger <jdl-CYoMK+44s/[email protected]>
 
+
+Python library
+--------------
+
+A Python library is also available. To build this you will need to install
+swig and Python development files. On Debian distributions:
+
+   sudo apt-get install swig python-dev
+
+The library provides an Fdt class which you can use like this:
+
+    fdt = _ReadFdt('test_tree1.dtb')
+    node = fdt.path_offset('/test-node')
+    prop = fdt.first_property_offset(node)
+    print 'Property name: %s' % fdt.string(prop.nameoff)
+    print 'Property data: %s' % fdt.data(prop.nameoff)
+
+You will find tests in tests/pylibfdt_tests.py showing how to use each
+method. Help is available using the Python help command, e.g.:
+
+    $ cd pylibfdt
+    $ python -c "import libfdt; help(libfdt)"
+
+If you add new features, please check code coverage:
+
+    $ sudo apt-get install python-pip python-pytest
+    $ sudo pip install coverage
+    $ cd tests
+    $ coverage run pylibfdt_tests.py
+    $ coverage html
+    # Open 'htmlcov/index.html' in your browser
+
+
 Mailing list
 ------------
 The following list is for discussion about dtc and libfdt implementation
Signature · 1 line
2.11.0.483.g087da7b7c-goog

A message that is not in the archive (sent off-list or not yet received): 20170205201323.15411-4-sjg-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org

#19 lore raw

Re: [PATCH v4 3/5] Mention pylibfdt in the documentation

Quoted 23 lines · Simon Glass

Add a note about pylibfdt in the README.

---

Changes in v4: None
Changes in v3: None
Changes in v2:
- Add details on how to obtain full help and code coverage

 README | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
README +2 -0
index f92008f..7191d1b 100644
@@ -7,6 +7,39 @@ DTC and LIBFDT are maintained by:
 David Gibson <[email protected]>
 Jon Loeliger <jdl-CYoMK+44s/[email protected]>
 
+
+Python library

README DTC and LIBFDT are maintained by: · line 21

Quoted 10 lines
README +10 -0
+--------------
+
+A Python library is also available. To build this you will need to install
+swig and Python development files. On Debian distributions:
+
+   sudo apt-get install swig python-dev
+
+The library provides an Fdt class which you can use like this:
+
+    fdt = _ReadFdt('test_tree1.dtb')

This seems to be using the test wrapper _ReadFdt(0 rather than the fdt class proper.

README DTC and LIBFDT are maintained by: · line 25

Quoted 4 lines
README +4 -0
+    node = fdt.path_offset('/test-node')
+    prop = fdt.first_property_offset(node)
+    print 'Property name: %s' % fdt.string(prop.nameoff)
+    print 'Property data: %s' % fdt.data(prop.nameoff)

I think this is not quite up to date with the current version.

In addition, I think the example would be more informative if you showed an interactive session, demonstrating how the offsets and property values are encoded as Python ints and strings.

Quoted 19 lines
README +16 -0
+You will find tests in tests/pylibfdt_tests.py showing how to use each
+method. Help is available using the Python help command, e.g.:
+
+    $ cd pylibfdt
+    $ python -c "import libfdt; help(libfdt)"
+
+If you add new features, please check code coverage:
+
+    $ sudo apt-get install python-pip python-pytest
+    $ sudo pip install coverage
+    $ cd tests
+    $ coverage run pylibfdt_tests.py
+    $ coverage html
+    # Open 'htmlcov/index.html' in your browser
+
+
 Mailing list
 ------------
 The following list is for discussion about dtc and libfdt implementation
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#20 lore raw

Re: [PATCH v4 3/5] Mention pylibfdt in the documentation

Hi David,

Quoted 43 lines · David Gibson
Quoted 23 lines · Simon Glass

Add a note about pylibfdt in the README.

---

Changes in v4: None
Changes in v3: None
Changes in v2:
- Add details on how to obtain full help and code coverage

 README | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
README +2 -0
index f92008f..7191d1b 100644
@@ -7,6 +7,39 @@ DTC and LIBFDT are maintained by:
 David Gibson <[email protected]>
 Jon Loeliger <jdl-CYoMK+44s/[email protected]>
 
+
+Python library

README DTC and LIBFDT are maintained by: · line 21

Quoted 10 lines
README +10 -0
+--------------
+
+A Python library is also available. To build this you will need to install
+swig and Python development files. On Debian distributions:
+
+   sudo apt-get install swig python-dev
+
+The library provides an Fdt class which you can use like this:
+
+    fdt = _ReadFdt('test_tree1.dtb')

This seems to be using the test wrapper _ReadFdt(0 rather than the fdt class proper.

README DTC and LIBFDT are maintained by: · line 25

Quoted 4 lines
README +4 -0
+    node = fdt.path_offset('/test-node')
+    prop = fdt.first_property_offset(node)
+    print 'Property name: %s' % fdt.string(prop.nameoff)
+    print 'Property data: %s' % fdt.data(prop.nameoff)
Quoted 5 lines

I think this is not quite up to date with the current version.

In addition, I think the example would be more informative if you showed an interactive session, demonstrating how the offsets and property values are encoded as Python ints and strings.

OK, sounds good. I will revisit it once the interface is nailed down. I can also expand this with a few more examples.

Regards,
Simon

#21 lore raw

PATCH v4 4/5 Adjust libfdt.h to work with swig

In reply to a message not in the archive

There are a few places where libfdt.h cannot be used as is with swig:

- macros like fdt_totalsize() have to be defined as C declarations
- fdt_offset_ptr() and fdt_getprop_namelen() need special treatment due to a TODO in the wrapper for fdt_getprop()

The second one can hopefully be resolved by someone with more knowledge of SWIG than me.

Add #ifdefs to work around this problem.

---

Changes in v4:
- Add new patch to adjust libfdt.h to work with swig

Changes in v3: None
Changes in v2: None

 libfdt/libfdt.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
libfdt/libfdt.h +6 -1
index c69e918..2e78754 100644
@@ -143,7 +143,9 @@
 /* Low-level functions (you probably don't need these)                */
 /**********************************************************************/
 
+#ifndef SWIG /* Use a special rule in libfdt.swig instead */
 const void *fdt_offset_ptr(const void *fdt, int offset, unsigned int checklen);
+#endif
 static inline void *fdt_offset_ptr_w(void *fdt, int offset, int checklen)
 {
 	return (void *)(uintptr_t)fdt_offset_ptr(fdt, offset, checklen);
@@ -210,7 +212,7 @@ int fdt_next_subnode(const void *fdt, int offset);
 /**********************************************************************/
 /* General functions                                                  */
 /**********************************************************************/
-
+#ifndef SWIG /* Repeated in libfdt.swig (we cannot use macros) */
 #define fdt_get_header(fdt, field) \
 	(fdt32_to_cpu(((const struct fdt_header *)(fdt))->field))
 #define fdt_magic(fdt)			(fdt_get_header(fdt, magic))
@@ -223,6 +225,7 @@ int fdt_next_subnode(const void *fdt, int offset);
 #define fdt_boot_cpuid_phys(fdt)	(fdt_get_header(fdt, boot_cpuid_phys))
 #define fdt_size_dt_strings(fdt)	(fdt_get_header(fdt, size_dt_strings))
 #define fdt_size_dt_struct(fdt)		(fdt_get_header(fdt, size_dt_struct))
+#endif /* SWIG */
 
 #define __fdt_set_hdr(name) \
 	static inline void fdt_set_##name(void *fdt, uint32_t val) \
@@ -638,8 +641,10 @@ const void *fdt_getprop_by_offset(const void *fdt, int offset,
  * Identical to fdt_getprop(), but only examine the first namelen
  * characters of name for matching the property name.
  */
+#ifndef SWIG /* Use a special rule in libfdt.swig instead */
 const void *fdt_getprop_namelen(const void *fdt, int nodeoffset,
 				const char *name, int namelen, int *lenp);
+#endif
 static inline void *fdt_getprop_namelen_w(void *fdt, int nodeoffset,
 					  const char *name, int namelen,
 					  int *lenp)
Signature · 1 line
2.11.0.483.g087da7b7c-goog

A message that is not in the archive (sent off-list or not yet received): 20170205201323.15411-5-sjg-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org

#23 lore raw

Re: [PATCH v4 4/5] Adjust libfdt.h to work with swig

Quoted 5 lines · Simon Glass

There are a few places where libfdt.h cannot be used as is with swig:

- macros like fdt_totalsize() have to be defined as C declarations
- fdt_offset_ptr() and fdt_getprop_namelen() need special treatment due to a TODO in the wrapper for fdt_getprop()

fdt_offset_ptr() is a very low-level function which should be rarely needed outside libfdt internals, even in C. I think it makes sense to exclude it from the Python interface.

fdt_getprop_namelen() is primarily to make certain things easier in C where copying and slicing up strings can be quite painful. Since that's trivial in Python, I don't think there's any need to expose the namelen() functions in Python.

Quoted 33 lines

The second one can hopefully be resolved by someone with more knowledge of SWIG than me.

Add #ifdefs to work around this problem.

---

Changes in v4:
- Add new patch to adjust libfdt.h to work with swig

Changes in v3: None
Changes in v2: None

 libfdt/libfdt.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
libfdt/libfdt.h +2 -0
index c69e918..2e78754 100644
@@ -143,7 +143,9 @@
 /* Low-level functions (you probably don't need these)                */
 /**********************************************************************/
 
+#ifndef SWIG /* Use a special rule in libfdt.swig instead */
 const void *fdt_offset_ptr(const void *fdt, int offset, unsigned int checklen);
+#endif
 static inline void *fdt_offset_ptr_w(void *fdt, int offset, int checklen)
 {
 	return (void *)(uintptr_t)fdt_offset_ptr(fdt, offset, checklen);

libfdt/libfdt.h int fdt_next_subnode(const void *fdt, int offset); · line 215

Quoted 5 lines
libfdt/libfdt.h +1 -1
@@ -210,7 +212,7 @@ int fdt_next_subnode(const void *fdt, int offset);
 /**********************************************************************/
 /* General functions                                                  */
 /**********************************************************************/
-
+#ifndef SWIG /* Repeated in libfdt.swig (we cannot use macros) */

You need to duplicate them in libfdt.swig, certainly, but why do you need to explicitly cut them out here? Isn't swig smart enough to know it can't do anything with these?

Quoted 22 lines
libfdt/libfdt.h +3 -0
 #define fdt_get_header(fdt, field) \
 	(fdt32_to_cpu(((const struct fdt_header *)(fdt))->field))
 #define fdt_magic(fdt)			(fdt_get_header(fdt, magic))
@@ -223,6 +225,7 @@ int fdt_next_subnode(const void *fdt, int offset);
 #define fdt_boot_cpuid_phys(fdt)	(fdt_get_header(fdt, boot_cpuid_phys))
 #define fdt_size_dt_strings(fdt)	(fdt_get_header(fdt, size_dt_strings))
 #define fdt_size_dt_struct(fdt)		(fdt_get_header(fdt, size_dt_struct))
+#endif /* SWIG */
 
 #define __fdt_set_hdr(name) \
 	static inline void fdt_set_##name(void *fdt, uint32_t val) \
@@ -638,8 +641,10 @@ const void *fdt_getprop_by_offset(const void *fdt, int offset,
  * Identical to fdt_getprop(), but only examine the first namelen
  * characters of name for matching the property name.
  */
+#ifndef SWIG /* Use a special rule in libfdt.swig instead */
 const void *fdt_getprop_namelen(const void *fdt, int nodeoffset,
 				const char *name, int namelen, int *lenp);
+#endif
 static inline void *fdt_getprop_namelen_w(void *fdt, int nodeoffset,
 					  const char *name, int namelen,
 					  int *lenp)
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#24 lore raw

Re: [PATCH v4 4/5] Adjust libfdt.h to work with swig

Hi David,

Quoted 10 lines · David Gibson
Quoted 5 lines · Simon Glass

There are a few places where libfdt.h cannot be used as is with swig:

- macros like fdt_totalsize() have to be defined as C declarations
- fdt_offset_ptr() and fdt_getprop_namelen() need special treatment due to a TODO in the wrapper for fdt_getprop()

fdt_offset_ptr() is a very low-level function which should be rarely needed outside libfdt internals, even in C. I think it makes sense to exclude it from the Python interface.

Sounds good.

Quoted 5 lines

fdt_getprop_namelen() is primarily to make certain things easier in C where copying and slicing up strings can be quite painful. Since that's trivial in Python, I don't think there's any need to expose the namelen() functions in Python.

Yes that's a good point, will drop it.

Quoted 27 lines
Quoted 27 lines

The second one can hopefully be resolved by someone with more knowledge of SWIG than me.

Add #ifdefs to work around this problem.

---

Changes in v4:
- Add new patch to adjust libfdt.h to work with swig

Changes in v3: None
Changes in v2: None

 libfdt/libfdt.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
libfdt/libfdt.h +0 -0
index c69e918..2e78754 100644
@@ -143,7 +143,9 @@
 /* Low-level functions (you probably don't need these)                */
 /**********************************************************************/
 
Quoted 16 lines

libfdt/libfdt.h int fdt_next_subnode(const void *fdt, int offset); · line 215

Quoted 12 lines
libfdt/libfdt.h +3 -1
+#ifndef SWIG /* Use a special rule in libfdt.swig instead */
 const void *fdt_offset_ptr(const void *fdt, int offset, unsigned int checklen);
+#endif
 static inline void *fdt_offset_ptr_w(void *fdt, int offset, int checklen)
 {
      return (void *)(uintptr_t)fdt_offset_ptr(fdt, offset, checklen);
@@ -210,7 +212,7 @@ int fdt_next_subnode(const void *fdt, int offset);
 /**********************************************************************/
 /* General functions                                                  */
 /**********************************************************************/
-
+#ifndef SWIG /* Repeated in libfdt.swig (we cannot use macros) */

You need to duplicate them in libfdt.swig, certainly, but why do you need to explicitly cut them out here? Isn't swig smart enough to know it can't do anything with these?

Because the #define stuffs up my declarations. But it's easy to put the declarations first to avoid that problem, so I'll do that.

[..]

Regards,
Simon

#25 lore raw

PATCH v4 5/5 Build pylibfdt as part of the normal build process

In reply to a message not in the archive

Possible this needs to be made optional. For now just hook it up.

---

Changes in v4: None
Changes in v3: None
Changes in v2: None

 Makefile | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)
Makefile +15 -1
index ce05eba..c516d9f 100644
@@ -115,7 +115,7 @@ BIN += fdtput
 
 SCRIPTS = dtdiff
 
-all: $(BIN) libfdt
+all: $(BIN) libfdt pylibfdt
 
 
 ifneq ($(DEPTARGETS),)
@@ -202,6 +202,19 @@ dist:
 	cat ../dtc-$(dtc_version).tar | \
 		gzip -9 > ../dtc-$(dtc_version).tar.gz
 
+
+#
+# Rules for pylibfdt
+#
+PYLIBFDT_srcdir = pylibfdt
+PYLIBFDT_objdir = pylibfdt
+
+include $(PYLIBFDT_srcdir)/Makefile.pylibfdt
+
+.PHONY: pylibfdt
+pylibfdt: $(PYLIBFDT_objdir)/_libfdt.so
+
+
 #
 # Release signing and uploading
 # This is for maintainer convenience, don't try this at home.
@@ -246,6 +259,7 @@ STD_CLEANFILES = *~ *.o *.$(SHAREDLIB_EXT) *.d *.a *.i *.s core a.out vgcore.* \
 clean: libfdt_clean tests_clean
 	@$(VECHO) CLEAN
 	rm -f $(STD_CLEANFILES)
+	rm -f $(PYLIBFDT_CLEANFILES)
 	rm -f $(VERSION_FILE)
 	rm -f $(BIN)
 	rm -f dtc-*.tar dtc-*.tar.sign dtc-*.tar.asc
Signature · 1 line
2.11.0.483.g087da7b7c-goog
#26 lore raw
In reply to a message not in the archive
Quoted 18 lines · Simon Glass

At present libfdt consists of only a C implementation. Many scripts are written using Python so it useful to have Python bindings for libfdt. Apparently this has never been attempted before, or if so I cannot find a reference.

This series starts the process of adding this support, with just a bare-bones set of methods.

The v4 series provides binding that can be used like this:

fdt = libfdt.Fdt(open(fname).read())
node = fdt.path_offset('/subnode@1')
print fdt.get_prop(node, 'compatible')
subnode = fdt.first_subnode_quiet(node)
while subnode > 0:
print fdt.get_name(subnode)
subnode = fdt.next_subnode_quiet(subnode)

Quoted 4 lines

This version does not include a class for properties. That can be added if it is felt to be useful, but I want to get some basic functionality agreed first. The get_property_by_offset() function would likely benefit from this.

Sorry I've taken so long to look at this, I've finally given it a review.

Fwiw, I ran into someone at linux.conf.au who also seemed to be rather interested in a Python libfdt wrapper.

Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#27 lore raw

Hi David,

Quoted 19 lines · David Gibson
Quoted 18 lines · Simon Glass

At present libfdt consists of only a C implementation. Many scripts are written using Python so it useful to have Python bindings for libfdt. Apparently this has never been attempted before, or if so I cannot find a reference.

This series starts the process of adding this support, with just a bare-bones set of methods.

The v4 series provides binding that can be used like this:

fdt = libfdt.Fdt(open(fname).read())
node = fdt.path_offset('/subnode@1')
print fdt.get_prop(node, 'compatible')
subnode = fdt.first_subnode_quiet(node)
while subnode > 0:
print fdt.get_name(subnode)
subnode = fdt.next_subnode_quiet(subnode)

Quoted 7 lines
Quoted 4 lines

This version does not include a class for properties. That can be added if it is felt to be useful, but I want to get some basic functionality agreed first. The get_property_by_offset() function would likely benefit from this.

Sorry I've taken so long to look at this, I've finally given it a review.

That's OK. Thanks for looking. I'll try to spin this quickly.

Quoted 3 lines

Fwiw, I ran into someone at linux.conf.au who also seemed to be rather interested in a Python libfdt wrapper.

OK good - there was someone on the U-Boot list also.

Regards,
Simo