Skip to contentSkip to conversation

xen: rework compat headers generation

PATCH Superseded Anthony PERARD 4 participants 12 messages Jan 16, 2023 – Jan 19, 2023

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover xen: rework compat headers generation

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.build-system-xen-include-rework-v3

v3:
- Rewrite script into python instead of perl.
(last patch of the series)

v2:
- new patch [1/4] to fix issue with command line that can be way too long
- other small changes, and reorder patches

Hi,

This patch series is about 2 improvement. First one is to use $(if_changed, ) in "include/Makefile" to make the generation of the compat headers less verbose and to have the command line part of the decision to rebuild the headers. Second one is to replace one slow script by a much faster one, and save time when generating the headers.

Thanks.

Anthony PERARD (1):
build: replace get-fields.sh by a python script

 xen/include/Makefile            |   6 +-
 xen/tools/compat-xlat-header.py | 468 ++++++++++++++++++++++++++++
 xen/tools/get-fields.sh         | 528 --------------------------------
 3 files changed, 470 insertions(+), 532 deletions(-)
 create mode 100644 xen/tools/compat-xlat-header.py
 delete mode 100644 xen/tools/get-fields.sh
Signature · 1 line
Anthony PERARD
#1 lore raw

PATCH v3 1/1 build: replace get-fields.sh by a python script

The get-fields.sh which generate all the include/compat/.xlat/*.h headers is quite slow. It takes for example nearly 3 seconds to generate platform.h on a recent machine, or 2.3 seconds for memory.h.

Rewriting the mix of shell/sed/python into a single python script make the generation of those file a lot faster.

No functional change, the headers generated are identical.

---

Notes:
To test the header generation, I've submit a branch to gitlab ci, where all the headers where generated, and for each one both the shell script and the python script where run and the result of both compared.

v3:
convert to python script instead of perl
- this should allow more developper do be able to review/edit it.
- it avoid adding a dependency on perl for the hypervisor build.

It can be twice as slow than the perl version, but overall, when doing a build with make, there isn't much difference between the perl and python script.
We might be able to speed the python script up by precompiling the many regex, but it's probably not worth it. (python already have a cache of compiled regex, but I think it's small, maybe 10 or so)

v2:
- Add .pl extension to the perl script
- remove "-w" from the shebang as it is duplicate of "use warning;"
- Add a note in the commit message that the "headers generated are identical".

 xen/include/Makefile            |   6 +-
 xen/tools/compat-xlat-header.py | 468 ++++++++++++++++++++++++++++
 xen/tools/get-fields.sh         | 528 --------------------------------
 3 files changed, 470 insertions(+), 532 deletions(-)
 create mode 100644 xen/tools/compat-xlat-header.py
 delete mode 100644 xen/tools/get-fields.sh
xen/include/Makefile +2 -4
index 65be310eca..b950423efe 100644
@@ -60,9 +60,7 @@ cmd_compat_c = \
 
 quiet_cmd_xlat_headers = GEN     $@
 cmd_xlat_headers = \
-    while read what name; do \
-        $(SHELL) $(srctree)/tools/get-fields.sh "$$what" compat_$$name $< || exit $$?; \
-    done <$(patsubst $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename $<)).lst >[email protected]; \
+    $(PYTHON) $(srctree)/tools/compat-xlat-header.py $< $(patsubst $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename $<)).lst > [email protected]; \
     mv -f [email protected] $@
 
 targets += $(headers-y)
@@ -80,7 +78,7 @@ $(obj)/compat/%.c: $(src)/public/%.h $(srcdir)/xlat.lst $(srctree)/tools/compat-
 	$(call if_changed,compat_c)
 
 targets += $(patsubst compat/%, compat/.xlat/%, $(headers-y))
-$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(srctree)/tools/get-fields.sh FORCE
+$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(srctree)/tools/compat-xlat-header.py FORCE
 	$(call if_changed,xlat_headers)
 
 quiet_cmd_xlat_lst = GEN     $@
xen/tools/compat-xlat-header.py new file +468 -0

468 lines not on this page · raw message

xen/tools/get-fields.sh deleted +0 -528

528 lines not on this page · raw message

Signature · 1 line
Anthony PERARD
#2 lore raw

Re: [XEN PATCH v3 1/1] build: replace get-fields.sh by a python script

Quoted 76 lines · Anthony PERARD

The get-fields.sh which generate all the include/compat/.xlat/*.h headers is quite slow. It takes for example nearly 3 seconds to generate platform.h on a recent machine, or 2.3 seconds for memory.h.

Rewriting the mix of shell/sed/python into a single python script make the generation of those file a lot faster.

No functional change, the headers generated are identical.

---

Notes:
To test the header generation, I've submit a branch to gitlab ci, where all the headers where generated, and for each one both the shell script and the python script where run and the result of both compared.

v3:
convert to python script instead of perl
- this should allow more developper do be able to review/edit it.
- it avoid adding a dependency on perl for the hypervisor build.

It can be twice as slow than the perl version, but overall, when doing a build with make, there isn't much difference between the perl and python script.
We might be able to speed the python script up by precompiling the many regex, but it's probably not worth it. (python already have a cache of compiled regex, but I think it's small, maybe 10 or so)

v2:
- Add .pl extension to the perl script
- remove "-w" from the shebang as it is duplicate of "use warning;"
- Add a note in the commit message that the "headers generated are identical".

xen/include/Makefile            |   6 +-
xen/tools/compat-xlat-header.py | 468 ++++++++++++++++++++++++++++
xen/tools/get-fields.sh         | 528 --------------------------------
3 files changed, 470 insertions(+), 532 deletions(-)

create mode 100644 xen/tools/compat-xlat-header.py delete mode 100644 xen/tools/get-fields.sh

xen/include/Makefile +0 -0
index 65be310eca..b950423efe 100644
@@ -60,9 +60,7 @@ cmd_compat_c = \
 

quiet_cmd_xlat_headers = GEN $@
cmd_xlat_headers = \

xen/include/Makefile +1 -3
-    while read what name; do \
-        $(SHELL) $(srctree)/tools/get-fields.sh "$$what" compat_$$name $< || exit $$?; \
-    done <$(patsubst $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename $<)).lst >[email protected]; \
+    $(PYTHON) $(srctree)/tools/compat-xlat-header.py $< $(patsubst $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename $<)).lst > [email protected]; \
    mv -f [email protected] $@

targets += $(headers-y)

xen/include/Makefile +1 -1
@@ -80,7 +78,7 @@ $(obj)/compat/%.c: $(src)/public/%.h $(srcdir)/xlat.lst $(srctree)/tools/compat-
 $(call if_changed,compat_c)
 
 targets += $(patsubst compat/%, compat/.xlat/%, $(headers-y))
-$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(srctree)/tools/get-fields.sh FORCE
+$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(srctree)/tools/compat-xlat-header.py FORCE
 $(call if_changed,xlat_headers)
 

quiet_cmd_xlat_lst = GEN $@

xen/tools/compat-xlat-header.py new file +0 -0
new file mode 100644
index 0000000000..c1b361ac56

xen/tools/compat-xlat-header.py line 1

Quoted 1 line
xen/tools/compat-xlat-header.py new file +1 -0
@@ -0,0 +1,468 @@
+#!/usr/bin/env python

Would it make sense to start with python3 since it is a new script?

#3 lore raw

Re: [XEN PATCH v3 1/1] build: replace get-fields.sh by a python script

Quoted 10 lines · Luca Fancellu

xen/tools/compat-xlat-header.py line 1

Quoted 8 lines · Anthony PERARD
xen/tools/compat-xlat-header.py new file +1 -0
new file mode 100644
index 0000000000..c1b361ac56
@@ -0,0 +1,468 @@
+#!/usr/bin/env python

Would it make sense to start with python3 since it is a new script?

That shebang isn't even used as the script doesn't even have the execution bit set. So why do you say that the script isn't python3? Not really asking, just been pedantic :-)

Even if it's a new script, it isn't a new project. We can't depend on brand new functionality from our dependencies. We need to be able to build the hypervisor with old build toolchain / distribution.

Anyway, I did start by writing a python3 script in all its glory (or at least some of the new part of the language that I know about), but I had to rework it to be able to use it on older distribution. Our centos7 container in our GitLab CI seems to use python2.7.

So I had to stop using str.removeprefix() and I introduce some function doing the same thing instead (so that works with older than python 3.9). Then I had to stop using f-strings and use %-formatting instead. Then use "m.groups()[0]" instead of "m[1]" where "m" is a match result from re.match() and other.
And use the classing "from __future__ ..." preamble.

Cheers,

Signature · 1 line
Anthony PERARD
#4 lore raw

Re: [XEN PATCH v3 1/1] build: replace get-fields.sh by a python script

Quoted 16 lines · Anthony PERARD
Quoted 10 lines · Luca Fancellu

xen/tools/compat-xlat-header.py line 1

Quoted 8 lines · Anthony PERARD
xen/tools/compat-xlat-header.py new file +1 -0
new file mode 100644
index 0000000000..c1b361ac56
@@ -0,0 +1,468 @@
+#!/usr/bin/env python

Would it make sense to start with python3 since it is a new script?

Quoted 3 lines

That shebang isn't even used as the script doesn't even have the execution bit set. So why do you say that the script isn't python3? Not really asking, just been pedantic :-)

Yes I didn’t pay attention to that

Quoted 9 lines

Even if it's a new script, it isn't a new project. We can't depend on brand new functionality from our dependencies. We need to be able to build the hypervisor with old build toolchain / distribution.

Anyway, I did start by writing a python3 script in all its glory (or at least some of the new part of the language that I know about), but I had to rework it to be able to use it on older distribution. Our centos7 container in our GitLab CI seems to use python2.7.

That makes sense, thanks for the explanation.

Quoted 12 lines

So I had to stop using str.removeprefix() and I introduce some function doing the same thing instead (so that works with older than python 3.9). Then I had to stop using f-strings and use %-formatting instead. Then use "m.groups()[0]" instead of "m[1]" where "m" is a match result from re.match() and other.
And use the classing "from __future__ ..." preamble.

Cheers,

Signature · 1 line
Anthony PERARD
#5 lore raw

Re: [XEN PATCH v3 1/1] build: replace get-fields.sh by a python script

In reply to Anthony PERARD
Quoted 10 lines · Anthony PERARD

The get-fields.sh which generate all the include/compat/.xlat/*.h headers is quite slow. It takes for example nearly 3 seconds to generate platform.h on a recent machine, or 2.3 seconds for memory.h.

Rewriting the mix of shell/sed/python into a single python script make the generation of those file a lot faster.

No functional change, the headers generated are identical.

I'll happily trust the testing you've done, and that the outputs are equivalent.  However, I have some general python suggestions.

Quoted 22 lines
xen/tools/compat-xlat-header.py new file +16 -0
new file mode 100644
index 0000000000..c1b361ac56
@@ -0,0 +1,468 @@
+#!/usr/bin/env python
+
+from __future__ import print_function
+import re
+import sys
+
+typedefs = []
+
+def removeprefix(s, prefix):
+    if s.startswith(prefix):
+        return s[len(prefix):]
+    return s
+
+def removesuffix(s, suffix):
+    if s.endswith(suffix):
+        return s[:-len(suffix)]

xen/tools/compat-xlat-header.py line 26

Quoted 10 lines
xen/tools/compat-xlat-header.py new file +10 -0
+    return s
+
+def get_fields(looking_for, header_tokens):
+    level = 1
+    aggr = 0
+    fields = []
+    name = ''
+
+    for token in header_tokens:
+        if token in ['struct', 'union']:

('struct', 'union')

A tuple here rather than a list is marginally more efficient.

xen/tools/compat-xlat-header.py line 38

Quoted 12 lines
xen/tools/compat-xlat-header.py new file +12 -0
+            if level == 1:
+                aggr = 1
+                fields = []
+                name = ''
+        elif token == '{':
+            level += 1
+        elif token == '}':
+            level -= 1
+            if level == 1 and name == looking_for:
+                fields.append(token)
+                return fields
+        elif re.match(r'^[a-zA-Z_]', token):

Here and many other places, you've got re.{search,match} with repeated regexes.   This can end up being quite expensive, and we already had one build system slowdown caused by it.

Up near typedefs at the top, you want something like:

token_re = re.compile('[a-zA-Z_]')

to prepare the regex once at the start of day, and and use 'elif token_re.match(token)' here.

Another thing to note, the difference between re.search and re.match is that match has an implicit '^' anchor.  You need to be aware of this when compiling one regex to be used with both.

All of this said, where is 0-9 in the token regex?  Have we just got extremely lucky with having no embedded digits in identifiers thus far?

xen/tools/compat-xlat-header.py line 50

Quoted 12 lines
xen/tools/compat-xlat-header.py new file +12 -0
+            if not (aggr == 0 or name != ''):
+                name = token
+
+        if aggr != 0:
+            fields.append(token)
+
+    return []
+
+def get_typedefs(tokens):
+    level = 1
+    state = 0
+    typedefs = []

This shadows the global typedefs list, but the result of calling this function is simply assigned to it.  Looking at the code, the global list is used in several places

It would be better to "global typedefs" here, and make this function void, unless you want to modify handle_field() and build_body() to take it in as a regular parameter.

I'm pretty sure typedefs actually wants to be a dict rather than a list (will have better "id in typedefs" performance lower down), but that wants matching with code changes elsewhere, and probably wants doing separately.

Quoted 116 lines
xen/tools/compat-xlat-header.py new file +116 -0
+    for token in tokens:
+        if token == 'typedef':
+            if level == 1:
+                state = 1
+        elif re.match(r'^COMPAT_HANDLE\((.*)\)$', token):
+            if not (level != 1 or state != 1):
+                state = 2
+        elif token in ['[', '{']:
+            level += 1
+        elif token in [']', '}']:
+            level -= 1
+        elif token == ';':
+            if level == 1:
+                state = 0
+        elif re.match(r'^[a-zA-Z_]', token):
+            if not (level != 1 or state != 2):
+                typedefs.append(token)
+    return typedefs
+
+def build_enums(name, tokens):
+    level = 1
+    kind = ''
+    named = ''
+    fields = []
+    members = []
+    id = ''
+
+    for token in tokens:
+        if token in ['struct', 'union']:
+            if not level != 2:
+                fields = ['']
+            kind = "%s;%s" % (token, kind)
+        elif token == '{':
+            level += 1
+        elif token == '}':
+            level -= 1
+            if level == 1:
+                subkind = kind
+                (subkind, _, _) = subkind.partition(';')
+                if subkind == 'union':
+                    print("\nenum XLAT_%s {" % (name,))
+                    for m in members:
+                        print("    XLAT_%s_%s," % (name, m))
+                    print("};")
+                return
+            elif level == 2:
+                named = '?'
+        elif re.match(r'^[a-zA-Z_]', token):
+            id = token
+            k = kind
+            (_, _, k) = k.partition(';')
+            if named != '' and k != '':
+                if len(fields) > 0 and fields[0] == '':
+                    fields.pop(0)
+                build_enums("%s_%s" % (name, token), fields)
+                named = '!'
+        elif token == ',':
+            if level == 2:
+                members.append(id)
+        elif token == ';':
+            if level == 2:
+                members.append(id)
+            if named != '':
+                (_, _, kind) = kind.partition(';')
+            named = ''
+        if len(fields) != 0:
+            fields.append(token)
+
+def handle_field(prefix, name, id, type, fields):
+    if len(fields) == 0:
+        print(" \\")
+        if type == '':
+            print("%s(_d_)->%s = (_s_)->%s;" % (prefix, id, id), end='')
+        else:
+            k = id.replace('.', '_')
+            print("%sXLAT_%s_HNDL_%s(_d_, _s_);" % (prefix, name, k), end='')
+    elif not re.search(r'[{}]', ' '.join(fields)):
+        tag = ' '.join(fields)
+        tag = re.sub(r'\s*(struct|union)\s+(compat_)?(\w+)\s.*', '\\3', tag)
+        print(" \\")
+        print("%sXLAT_%s(&(_d_)->%s, &(_s_)->%s);" % (prefix, tag, id, id), end='')
+    else:
+        func_id = id
+        func_tokens = fields
+        kind = ''
+        array = ""
+        level = 1
+        arrlvl = 1
+        array_type = ''
+        id = ''
+        type = ''
+        fields = []
+        for token in func_tokens:
+            if token in ['struct', 'union']:
+                if level == 2:
+                    fields = ['']
+                if level == 1:
+                    kind = token
+                    if kind == 'union':
+                        tmp = func_id.replace('.', '_')
+                        print(" \\")
+                        print("%sswitch (%s) {" % (prefix, tmp), end='')
+            elif token == '{':
+                level += 1
+                id = ''
+            elif token == '}':
+                level -= 1
+                id = ''
+                if level == 1 and kind == 'union':
+                    print(" \\")
+                    print("%s}" % (prefix,), end='')
+            elif token == '[':
+                if level != 2 or arrlvl != 1:
+                    pass
+                elif array == '':
+                    array = ' '

xen/tools/compat-xlat-header.py line 176

Quoted 10 lines
xen/tools/compat-xlat-header.py new file +10 -0
+                else:
+                    array = "%s;" % (array,)
+                arrlvl += 1
+            elif token == ']':
+                arrlvl -= 1
+            elif re.match(r'^COMPAT_HANDLE\((.*)\)$', token):
+                if level == 2 and id == '':
+                    m = re.match(r'^COMPAT_HANDLE\((.*)\)$', token)
+                    type = m.groups()[0]
+                    type = removeprefix(type, 'compat_')

So this pattern is what lead to the introduction of the := operator in Python 3.8, but we obviously can't use it yet.

At least pre-compiling the regexes will reduce the hit from the double evaluation here.

~Andrew

P.S. I probably don't want to know why we have to special case evtchn port, argo port and domain handle.  I think it says more about the this bodge of a parser than anything else...

#6 lore raw

Re: [XEN PATCH v3 1/1] build: replace get-fields.sh by a python script

Quoted 7 lines · Andrew Cooper
Quoted 1 line · Anthony PERARD

+ elif re.match(r'^[a-zA-Z_]', token):

[...]

All of this said, where is 0-9 in the token regex?  Have we just got extremely lucky with having no embedded digits in identifiers thus far?

That's checking for just the first character, which can't be a digit?

Quoted 3 lines

P.S. I probably don't want to know why we have to special case evtchn port, argo port and domain handle.  I think it says more about the this bodge of a parser than anything else...

Iirc something broke without it, but it's been too long and spending a reasonable amount of time trying to re-construct I couldn't come up with anything. I didn't want to go as far as put time into actually trying out what (if anything) breaks with those removed. What I'm puzzled about is that argo and evtchn port types are handled in different places.

For the domain handle iirc the exception was attributed to it being the only typedef of an array which is embedded in other structures.

Jan

#7 lore raw

Re: [XEN PATCH v3 1/1] build: replace get-fields.sh by a python script

Quoted 8 lines · Jan Beulich
Quoted 6 lines · Andrew Cooper
Quoted 1 line · Anthony PERARD

+ elif re.match(r'^[a-zA-Z_]', token):

[...]

All of this said, where is 0-9 in the token regex?  Have we just got extremely lucky with having no embedded digits in identifiers thus far?

That's checking for just the first character, which can't be a digit?

So it is...

But nothing good can possibly come of having a token here which matches on the first char but mismatches later.

Quoted 11 lines
Quoted 4 lines

P.S. I probably don't want to know why we have to special case evtchn port, argo port and domain handle.  I think it says more about the this bodge of a parser than anything else...

Iirc something broke without it, but it's been too long and spending a reasonable amount of time trying to re-construct I couldn't come up with anything. I didn't want to go as far as put time into actually trying out what (if anything) breaks with those removed. What I'm puzzled about is that argo and evtchn port types are handled in different places.

Quoted 2 lines

For the domain handle iirc the exception was attributed to it being the only typedef of an array which is embedded in other structures.

I refer back to "bodge of a parser".

~Andrew

#8 lore raw

Re: [XEN PATCH v3 1/1] build: replace get-fields.sh by a python script

In reply to Andrew Cooper
Quoted 10 lines · Andrew Cooper

xen/tools/compat-xlat-header.py line 50

Quoted 4 lines · Anthony PERARD
xen/tools/compat-xlat-header.py new file +4 -0
+def get_typedefs(tokens):
+    level = 1
+    state = 0
+    typedefs = []

I'm pretty sure typedefs actually wants to be a dict rather than a list (will have better "id in typedefs" performance lower down), but that wants matching with code changes elsewhere, and probably wants doing separately.

I'm not sure that going to make a difference to have "id in ()" instead of "id in []". I just found out that typedefs is always empty...

I don't know what get_typedefs() is supposed to do, or at least if it works as expected, because it always returns "" or an empty list. (even the shell version)

So, it would actually be a bit faster to not call get_typedefs(), but I don't know if that's safe.

Quoted 18 lines
Quoted 18 lines
xen/tools/compat-xlat-header.py new file +18 -0
+    for token in tokens:
+        if token == 'typedef':
+            if level == 1:
+                state = 1
+        elif re.match(r'^COMPAT_HANDLE\((.*)\)$', token):
+            if not (level != 1 or state != 1):
+                state = 2
+        elif token in ['[', '{']:
+            level += 1
+        elif token in [']', '}']:
+            level -= 1
+        elif token == ';':
+            if level == 1:
+                state = 0
+        elif re.match(r'^[a-zA-Z_]', token):
+            if not (level != 1 or state != 2):
+                typedefs.append(token)
+    return typedefs
Signature · 1 line
Anthony PERARD
#9 lore raw

Re: [XEN PATCH v3 1/1] build: replace get-fields.sh by a python script

Quoted 6 lines · Anthony PERARD
Quoted 5 lines · Andrew Cooper
Quoted 4 lines · Anthony PERARD
xen/tools/compat-xlat-header.py new file +4 -0
+def get_typedefs(tokens):
+    level = 1
+    state = 0
+    typedefs = []
Quoted 6 lines
Quoted 4 lines

I'm pretty sure typedefs actually wants to be a dict rather than a list (will have better "id in typedefs" performance lower down), but that wants matching with code changes elsewhere, and probably wants doing separately.

I'm not sure that going to make a difference to have "id in ()" instead of "id in []".

It will in the middle of a tight loop.  Less pointer chasing in memory. But it is very marginal.

Quoted 8 lines

I just found out that typedefs is always empty...

I don't know what get_typedefs() is supposed to do, or at least if it works as expected, because it always returns "" or an empty list. (even the shell version)

So, it would actually be a bit faster to not call get_typedefs(), but I don't know if that's safe.

If it is dead logic even at the shell level, drop it.  Perhaps a prereq patch, because removing the complexity first will make this patch simpler to follow.

The conversion is atypical python, and perf will improve (which is the point of this patch anyway).

~Andrew

#10 lore raw

Re: [XEN PATCH v3 1/1] build: replace get-fields.sh by a python script

In reply to Anthony PERARD
Quoted 15 lines · Anthony PERARD
Quoted 10 lines · Andrew Cooper

xen/tools/compat-xlat-header.py line 50

Quoted 4 lines · Anthony PERARD
xen/tools/compat-xlat-header.py new file +4 -0
+def get_typedefs(tokens):
+    level = 1
+    state = 0
+    typedefs = []

I'm pretty sure typedefs actually wants to be a dict rather than a list (will have better "id in typedefs" performance lower down), but that wants matching with code changes elsewhere, and probably wants doing separately.

I'm not sure that going to make a difference to have "id in ()" instead of "id in []". I just found out that typedefs is always empty...

Quoted 6 lines

I don't know what get_typedefs() is supposed to do, or at least if it works as expected, because it always returns "" or an empty list. (even the shell version)

So, it would actually be a bit faster to not call get_typedefs(), but I don't know if that's safe.

There's exactly one instance that this would take care of:

typedef XEN_GUEST_HANDLE(char) tmem_cli_va_t;

But tmem.h isn't being processed anymore, and hence right now the list would always be empty. Are we going to be able to guarantee that going forward?

Jan

#11 lore raw

Re: [XEN PATCH v3 1/1] build: replace get-fields.sh by a python script

Quoted 20 lines · Jan Beulich
Quoted 13 lines · Anthony PERARD
Quoted 9 lines · Andrew Cooper

xen/tools/compat-xlat-header.py line 50

Quoted 4 lines · Anthony PERARD
xen/tools/compat-xlat-header.py new file +4 -0
+def get_typedefs(tokens):
+    level = 1
+    state = 0
+    typedefs = []

I'm pretty sure typedefs actually wants to be a dict rather than a list (will have better "id in typedefs" performance lower down), but that wants matching with code changes elsewhere, and probably wants doing separately.

I'm not sure that going to make a difference to have "id in ()" instead of "id in []". I just found out that typedefs is always empty...

Quoted 6 lines

I don't know what get_typedefs() is supposed to do, or at least if it works as expected, because it always returns "" or an empty list. (even the shell version)

So, it would actually be a bit faster to not call get_typedefs(), but I don't know if that's safe.

Quoted 7 lines

There's exactly one instance that this would take care of:

typedef XEN_GUEST_HANDLE(char) tmem_cli_va_t;

But tmem.h isn't being processed anymore, and hence right now the list would always be empty. Are we going to be able to guarantee that going forward?

IMO that's a code pattern we wouldn't want to repeat moving forward.

There's already too much magic in a guest handle, without hiding it behind a typedef too.

~Andrew