Skip to contentSkip to conversation

check if lockfile is writable (bug 606 patch V2)

PATCH Applied Yu Ke 3 participants 6 messages Dec 29, 2010 – Dec 30, 2010

lore
Share & tools
lore

Yu Ke

author
#0 lore raw

cover check if lockfile is writable (bug 606 patch V2)

I've sent a patch for bug 606 before, which add sanity check for $DL_DIR.

After digging more, the root cause is hidden in bb.utils.lockfile(), so it is more general to add check there. hence comes out the V2 patch for bug606. please discard my V1 patch.

Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: kyu3/bug606-v2
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=kyu3/bug606-v2

Thanks,
Yu Ke <[email protected]>
---

Yu Ke (1):
bb.utils: check if lock file is writable, to fix bug 606

 bitbake/lib/bb/utils.py |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)
author
#1 lore raw

PATCH 1/1 bb.utils: check if lock file is writable, to fix bug 606

Bug 606 report that if $DL_DIR is read-only, do_fetch will simply hang without any error message.

The root cause is that: bb.fetch.go()->bb.utils.lockfile() will try to lock file ${DL_DIR}/xxxxx.lock. Since ${DL_DIR} is read-only, it will cause IOError exception. Although lockfile() can catch the exception, currently code simply ignore all the exception and continue the loop. it make sense if the exception is caused by locking contention, but in the read-only $DL_DIR case, it cause endless waiting unfortunately.

So this patch add read-only check for lockfile to avoid the silent hang.

Fix [BUGID #606]

---
 bitbake/lib/bb/utils.py |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)
bitbake/lib/bb/utils.py +4 -0
index f468faf..e02969c 100644
@@ -401,6 +401,10 @@ def lockfile(name):
         bb.msg.error(bb.msg.domain.Util, "Error, lockfile path does not exist!: %s" % path)
         sys.exit(1)
 
+    if not os.access(path, os.W_OK):
+        bb.msg.error(bb.msg.domain.Util, "Error, lockfile path is not writable!: %s" % path)
+        sys.exit(1)
+
     while True:
         # If we leave the lockfiles lying around there is no problem
         # but we should clean up after ourselves. This gives potential
Signature · 1 line
1.7.0.4
#2 lore raw
In reply to Yu Ke
Quoted 6 lines · Yu Ke

I've sent a patch for bug 606 before, which add sanity check for $DL_DIR.

After digging more, the root cause is hidden in bb.utils.lockfile(), so it is more general to add check there. hence comes out the V2 patch for bug606. please discard my V1 patch.

Ke,

I still wonder about this. If the DL_DIR is read-only, since it is a shared environment (as example), when a fetch occurs, it may be for different reasons.

A couple of examples are:
1) an upstream fetch was missed (world build, but different hardware)
2) a developer changed the version information

In cases like this we really do want to do the fetch, so how can we handle that issue correctly, and not just fail and force the user to re-download all the packages (which may not be possible).

One option I can think of is to add a "WRITABLE_DL_DIR" (losy name but you get the idea) or something like that which if the DL_DIR is read-only could be written to for the lock file and ultimately the fetch of the missing upstream package.

Thoughts?

Sau!

Quoted 20 lines

Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: kyu3/bug606-v2
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=kyu3/bug606-v2

Thanks,
Yu Ke<[email protected]>
---

Yu Ke (1):
bb.utils: check if lock file is writable, to fix bug 606

  bitbake/lib/bb/utils.py |    4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)

_______________________________________________
poky mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/poky

#3 lore raw
Quoted 8 lines · Saul Wold

I still wonder about this. If the DL_DIR is read-only, since it is a shared environment (as example), when a fetch occurs, it may be for different reasons.

A couple of examples are:
1) an upstream fetch was missed (world build, but different hardware)
2) a developer changed the version information

Quoted 8 lines

In cases like this we really do want to do the fetch, so how can we handle that issue correctly, and not just fail and force the user to re-download all the packages (which may not be possible).

One option I can think of is to add a "WRITABLE_DL_DIR" (losy name but you get the idea) or something like that which if the DL_DIR is read-only could be written to for the lock file and ultimately the fetch of the missing upstream package.

DL_DIR *must* be writeable. We don't support the use case where it is not. We handle readonly directory sources as part of the mirror handling. It might be not a lot is written to DL_DIR, that is just fine.

I therefore agree with Ke's patch which errors if the directory is writeable.

Cheers,

Richard

author
#4 lore raw
Quoted 13 lines · Richard Purdie
Quoted 12 lines · Saul Wold

I still wonder about this. If the DL_DIR is read-only, since it is a shared environment (as example), when a fetch occurs, it may be for different reasons.

A couple of examples are:
1) an upstream fetch was missed (world build, but different hardware)
2) a developer changed the version information

In cases like this we really do want to do the fetch, so how can we handle that issue correctly, and not just fail and force the user to re-download all the packages (which may not be possible).

Quoted 8 lines
Quoted 4 lines

One option I can think of is to add a "WRITABLE_DL_DIR" (losy name but you get the idea) or something like that which if the DL_DIR is read-only could be written to for the lock file and ultimately the fetch of the missing upstream package.

DL_DIR *must* be writeable. We don't support the use case where it is not. We handle readonly directory sources as part of the mirror handling. It might be not a lot is written to DL_DIR, that is just fine.

Yes, so this patch does not intended to support read-only DL_DIR, it just provides more verbal information to use instead of silently hang, so that user can know what is wrong and fix it accordingly.

Regards
Ke

Quoted 14 lines

I therefore agree with Ke's patch which errors if the directory is writeable.

Cheers,

Richard

_______________________________________________
poky mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/poky

#5 lore raw
In reply to Yu Ke
Quoted 6 lines · Yu Ke

I've sent a patch for bug 606 before, which add sanity check for $DL_DIR.

After digging more, the root cause is hidden in bb.utils.lockfile(), so it is more general to add check there. hence comes out the V2 patch for bug606. please discard my V1 patch.

Quoted 6 lines

Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: kyu3/bug606-v2
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=kyu3/bug606-v2

Thanks,
Yu Ke <[email protected]>

Merged into master, thanks.

Richard