Skip to content

RFC bitbake: optimize file parsing speed

Dongxiao Xu poky 1 version New

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest2 patches 0 repliesNew

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/2bitbake: Make setVar a special case of setVarFlagbitbake/lib/bb/data_smart.py+28 -320 replies
2/2bitbake: Change the way to implement ??= operatorbitbake/lib/bb/data_smart.py bitbake/lib/bb/parse/ast.py+3 -70 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 0/2][RFC] bitbake: optimize file parsing speed

Hi Richard,

This RFC/pull request aims to speed up the bb file parsing speed. Please help to review it, thanks!

We observed that the ??= operator costs a lot of cycles when parsing bb files, this patchset implement the ??= in getVar() function. If the normal getVar returns None, it will check the defaultval flag and return its value.

Besides, 0001 patch also fixes a bug that if foo_OVERA[bar] = "x" and foo is never set as a variable directly, the override is never expanded.

I slightly modified 0001 patch compared with the previous discussion version. If simply copy the setVar logic into setVarFlag, I found that variable "do_install_append_virtclass-native" will both exist in self._special_values and self._seen_overrides, this is because when setting "content" for "do_install_append_virtclass-native", the variable will be recorded in self._special_values, and when setting "func" flag for "do_install_append_virtclass-native", the variable will be recorded in self._seen_overrides. This version of patch could fix the issue.

Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: dxu4/parsefile
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dxu4/parsefile

Thanks,
Dongxiao Xu <[email protected]>
---

Dongxiao Xu (2):
bitbake: Make setVar a special case of setVarFlag
bitbake: Change the way to implement ??= operator

 bitbake/lib/bb/data_smart.py |   63 ++++++++++++++++++++---------------------
 bitbake/lib/bb/parse/ast.py  |    7 ----
 2 files changed, 31 insertions(+), 39 deletions(-)