Skip to contentSkip to conversation

oeqa/selftest/cases/devtool.py: avoid .pyc race

PATCH Superseded Tim Orling 2 participants 2 messages Sep 29, 2020 – Sep 30, 2020

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Tim Orling

author
#0 lore raw

PATCH 1/1 oeqa/selftest/cases/devtool.py: avoid .pyc race

From: Tim Orling <[email protected]>

In certain conditions, most likely under heavy load on the AutoBuiler, the prebuilt .pyc files are attempting to be executed before they have been completely copied. Avoid this by not copying the .pyc files (nor the __pycache__ directory). The impact of python3-native recreating the .pyc files should hopefully be negligible.

YOCTO#13421
YOCTO#13803

---
 meta/lib/oeqa/selftest/cases/devtool.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
meta/lib/oeqa/selftest/cases/devtool.py +2 -1
index b383ed9c50b..a3d2e9ea7cc 100644
@@ -56,7 +56,8 @@ def setUpModule():
                     if pth.startswith(canonical_layerpath):
                         if relpth.endswith('/'):
                             destdir = os.path.join(corecopydir, relpth)
-                            shutil.copytree(pth, destdir)
+                            # avoid race condition by not copying .pyc files YPBZ#13421,13803
+                            shutil.copytree(pth, destdir, ignore=ignore_patterns('*.pyc', '__pycache__'))
                         else:
                             destdir = os.path.join(corecopydir, os.path.dirname(relpth))
                             bb.utils.mkdirhier(destdir)
Signature · 1 line
2.25.0
#1 lore raw
Quoted 10 lines · Tim Orling

From: Tim Orling <[email protected]>

In certain conditions, most likely under heavy load on the AutoBuiler, the prebuilt .pyc files are attempting to be executed before they have been completely copied. Avoid this by not copying the .pyc files (nor the __pycache__ directory). The impact of python3-native recreating the .pyc files should hopefully be negligible.

pyc makes difference only when loading, execution is still same. So it will depend
if we are invoking the script repeatedly or is it invoked once and executed for longer time.
benefits might be less in later case.

Quoted 28 lines

YOCTO#13421
YOCTO#13803

---
 meta/lib/oeqa/selftest/cases/devtool.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
meta/lib/oeqa/selftest/cases/devtool.py +2 -2
index b383ed9c50b..a3d2e9ea7cc 100644
@@ -56,7 +56,8 @@ def setUpModule():
                     if pth.startswith(canonical_layerpath):
                         if relpth.endswith('/'):
                             destdir = os.path.join(corecopydir, relpth)
-                            shutil.copytree(pth, destdir)
+                            # avoid race condition by not copying .pyc files YPBZ#13421,13803
+                            shutil.copytree(pth, destdir, ignore=ignore_patterns('*.pyc', '__pycache__'))
                         else:
                             destdir = os.path.join(corecopydir, os.path.dirname(relpth))
                             bb.utils.mkdirhier(destdir)
--

2.25.0