Return-Path: Received: from relay.appriver.com (relay.appriver.com [207.97.230.34]) by mail.openembedded.org (Postfix) with ESMTP id ADCA46010D for ; Thu, 19 Nov 2015 11:48:37 +0000 (UTC) Received: from [86.30.112.98] (HELO elite.brightsign) by relay.appriver.com (CommuniGate Pro SMTP 6.1.2) with ESMTP id 642874853; Thu, 19 Nov 2015 06:48:37 -0500 Received: from chuckie.brightsign ([172.30.1.25] helo=chuckie) by elite.brightsign with esmtp (Exim 4.84) (envelope-from ) id 1ZzNha-000EDV-KD; Thu, 19 Nov 2015 11:48:34 +0000 Received: from mac by chuckie with local (Exim 4.84) (envelope-from ) id 1ZzNha-0006I6-Jh; Thu, 19 Nov 2015 11:48:34 +0000 From: Mike Crowe To: openembedded-core@lists.openembedded.org Date: Thu, 19 Nov 2015 11:48:26 +0000 Message-Id: <1447933706-24145-1-git-send-email-mac@mcrowe.com> X-Mailer: git-send-email 2.1.4 Cc: Mike Crowe Subject: [PATCH v2] image.py: avoid mkdir race when building multiple images X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2015 11:48:37 -0000 If multiple images are being built simultaneously against the same sysroot then the call to os.makedirs in Image._write_wic_env can fail with: File: '.../meta/lib/oe/image.py', lineno: 341, function: _write_wic_env 0337: """ 0338: stdir = self.d.getVar('STAGING_DIR_TARGET', True) 0339: outdir = os.path.join(stdir, 'imgdata') 0340: if not os.path.exists(outdir): *** 0341: os.makedirs(outdir) 0342: basename = self.d.getVar('IMAGE_BASENAME', True) 0343: with open(os.path.join(outdir, basename) + '.env', 'w') as envf: 0344: for var in self.d.getVar('WICVARS', True).split(): 0345: value = self.d.getVar(var, True) File: '/usr/lib/python2.7/os.py', lineno: 157, function: makedirs 0153: if e.errno != errno.EEXIST: 0154: raise 0155: if tail == curdir: # xxx/newdir/. exists if xxx/newdir exists 0156: return *** 0157: mkdir(name, mode) 0158: 0159:def removedirs(name): 0160: """removedirs(path) 0161: Exception: OSError: [Errno 17] File exists: '.../tmp-glibc/sysroots/cheetah/imgdata' Using bb.utils.mkdirhier() protects against this. There's also little point in checking to see if the directory already exists - we might as well just try and create it regardless. Once the directory has been created, there's no race on the actual file since the filename contains IMAGE_BASENAME. Signed-off-by: Mike Crowe --- meta/lib/oe/image.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py index b9eb3de..17b096d 100644 --- a/meta/lib/oe/image.py +++ b/meta/lib/oe/image.py @@ -337,8 +337,7 @@ class Image(ImageDepGraph): """ stdir = self.d.getVar('STAGING_DIR_TARGET', True) outdir = os.path.join(stdir, 'imgdata') - if not os.path.exists(outdir): - os.makedirs(outdir) + bb.utils.mkdirhier(outdir) basename = self.d.getVar('IMAGE_BASENAME', True) with open(os.path.join(outdir, basename) + '.env', 'w') as envf: for var in self.d.getVar('WICVARS', True).split(): -- 2.1.4