hashserv: client: Fix AF_UNIX path length limits
PATCH 1/1 hashserv: client: Fix AF_UNIX path length limits
Restores a fix for unix domain socket path length limits when using the synchronous hash equivalence client that was accidentally removed when the async client was added.
Unfortunately, it's much more difficult to fix the same problem when using the async client directly due to the interaction of chdir() and async code, but this will at least restore the old behavior in the synchronous case.
--- bitbake/lib/hashserv/client.py | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)
bitbake/lib/hashserv/client.py +12 -3
@@ -40,7 +40,7 @@ class AsyncClient(object): self._connect_sock = connect_sock - async def _connect(self): + async def connect(self): if self.reader is None or self.writer is None: (self.reader, self.writer) = await self._connect_sock()
@@ -62,7 +62,7 @@ class AsyncClient(object): count = 0 while True: try: - await self._connect() + await self.connect() return await proc() except ( OSError,
@@ -190,7 +190,6 @@ class Client(object): for call in ( "connect_tcp", - "connect_unix", "close", "get_unihash", "report_unihash",
@@ -209,6 +208,16 @@ class Client(object): return wrapper + def connect_unix(self, path): + # AF_UNIX has path length issues so chdir here to workaround + cwd = os.getcwd() + try: + os.chdir(os.path.dirname(path)) + self.loop.run_until_complete(self.client.connect_unix(path)) + self.loop.run_until_complete(self.client.connect()) + finally: + os.chdir(cwd) + @property def max_chunk(self): return self.client.max_chunk
Signature · 1 line
2.29.2
hi Joshua,
Quoted 11 lines · Joshua Watt
Restores a fix for unix domain socket path length limits when using the synchronous hash equivalence client that was accidentally removed when the async client was added.
Unfortunately, it's much more difficult to fix the same problem when using the async client directly due to the interaction of
chdir()and async code, but this will at least restore the old behavior in the synchronous case.Signed-off-by Joshua Watt <[email protected]>
the meta-qcom CI started failing.. and I have the feeling this is because of this change. I am now seeing many errors, and this:
00:02:31.320 OSError: AF_UNIX path too long
Here is the link to the full Jenkins log:
https://ci.linaro.org/job/lt-qcom-openembedded-meta-qcom-master-premerge/136/MACHINE=dragonboard-410c,TCLIBC=glibc,label=docker-buster-amd64/console
This is a test build we do for each Github PR. This build is using this commit from Poky/master:
982688103619 (build-appliance-image: Update to master head revision)
We tested another PR 2 days ago, and it went fine:
https://ci.linaro.org/job/lt-qcom-openembedded-meta-qcom-master-premerge/134/MACHINE=dragonboard-410c,TCLIBC=glibc,label=docker-buster-amd64/consoleFull
This build was based on this commit:
982688103619 (build-appliance-image: Update to master head revision)
Looking at the error log, and this commit, it looks related to me.. though I am not sure I understand the details ;)
Quoted 57 lines
--- bitbake/lib/hashserv/client.py | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)bitbake/lib/hashserv/client.py +12
-4@@ -40,7 +40,7 @@ class AsyncClient(object): self._connect_sock = connect_sock - async def _connect(self): + async def connect(self): if self.reader is None or self.writer is None: (self.reader, self.writer) = await self._connect_sock()@@ -62,7 +62,7 @@ class AsyncClient(object): count = 0 while True: try: - await self._connect() + await self.connect() return await proc() except ( OSError,@@ -190,7 +190,6 @@ class Client(object): for call in ( "connect_tcp", - "connect_unix", "close", "get_unihash", "report_unihash",@@ -209,6 +208,16 @@ class Client(object): return wrapper + def connect_unix(self, path): + # AF_UNIX has path length issues so chdir here to workaround + cwd = os.getcwd() + try: + os.chdir(os.path.dirname(path)) + self.loop.run_until_complete(self.client.connect_unix(path)) + self.loop.run_until_complete(self.client.connect()) + finally: + os.chdir(cwd) + @property def max_chunk(self): return self.client.max_chunk --2.29.2
Quoted 28 lines · Nicolas Dechesne
hi Joshua,
Quoted 11 lines · Joshua Watt
Restores a fix for unix domain socket path length limits when using the synchronous hash equivalence client that was accidentally removed when the async client was added.
Unfortunately, it's much more difficult to fix the same problem when using the async client directly due to the interaction of
chdir()and async code, but this will at least restore the old behavior in the synchronous case.Signed-off-by Joshua Watt <[email protected]>the meta-qcom CI started failing.. and I have the feeling this is because of this change. I am now seeing many errors, and this:
00:02:31.320 OSError:AF_UNIXpath too longHere is the link to the full Jenkins log:
https://ci.linaro.org/job/lt-qcom-openembedded-meta-qcom-master-premerge/136/MACHINE=dragonboard-410c,TCLIBC=glibc,label=docker-buster-amd64/consoleThis is a test build we do for each Github PR. This build is using this commit from Poky/master:
982688103619 (build-appliance-image: Update to master head revision)
Quoted 7 lines
We tested another PR 2 days ago, and it went fine:
https://ci.linaro.org/job/lt-qcom-openembedded-meta-qcom-master-premerge/134/MACHINE=dragonboard-410c,TCLIBC=glibc,label=docker-buster-amd64/consoleFull
This build was based on this commit:
982688103619 (build-appliance-image: Update to master head revision)Looking at the error log, and this commit, it looks related to me.. though I am not sure I understand the details ;)
Argh, this is because I was silly and the fix is outright wrong. Sorry about that, I'll fix it quick (and maybe add a test this time!)
Quoted 58 lines
Quoted 58 lines
--- bitbake/lib/hashserv/client.py | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)bitbake/lib/hashserv/client.py +12
-4@@ -40,7 +40,7 @@ class AsyncClient(object): self._connect_sock = connect_sock - async def _connect(self): + async def connect(self): if self.reader is None or self.writer is None: (self.reader, self.writer) = await self._connect_sock()@@ -62,7 +62,7 @@ class AsyncClient(object): count = 0 while True: try: - await self._connect() + await self.connect() return await proc() except ( OSError,@@ -190,7 +190,6 @@ class Client(object): for call in ( "connect_tcp", - "connect_unix", "close", "get_unihash", "report_unihash",@@ -209,6 +208,16 @@ class Client(object): return wrapper + def connect_unix(self, path): + # AF_UNIX has path length issues so chdir here to workaround + cwd = os.getcwd() + try: + os.chdir(os.path.dirname(path)) + self.loop.run_until_complete(self.client.connect_unix(path)) + self.loop.run_until_complete(self.client.connect()) + finally: + os.chdir(cwd) + @property def max_chunk(self): return self.client.max_chunk --2.29.2
Quoted 36 lines · Joshua Watt
Quoted 33 lines · Nicolas Dechesne
hi Joshua,
Quoted 11 lines · Joshua Watt
Restores a fix for unix domain socket path length limits when using the synchronous hash equivalence client that was accidentally removed when the async client was added.
Unfortunately, it's much more difficult to fix the same problem when using the async client directly due to the interaction of
chdir()and async code, but this will at least restore the old behavior in the synchronous case.Signed-off-by Joshua Watt <[email protected]>the meta-qcom CI started failing.. and I have the feeling this is because of this change. I am now seeing many errors, and this:
00:02:31.320 OSError:AF_UNIXpath too longHere is the link to the full Jenkins log:
https://ci.linaro.org/job/lt-qcom-openembedded-meta-qcom-master-premerge/136/MACHINE=dragonboard-410c,TCLIBC=glibc,label=docker-buster-amd64/consoleThis is a test build we do for each Github PR. This build is using this commit from Poky/master:
982688103619 (build-appliance-image: Update to master head revision)We tested another PR 2 days ago, and it went fine:
https://ci.linaro.org/job/lt-qcom-openembedded-meta-qcom-master-premerge/134/MACHINE=dragonboard-410c,TCLIBC=glibc,label=docker-buster-amd64/consoleFull
This build was based on this commit:
982688103619 (build-appliance-image: Update to master head revision)
Quoted 5 lines
Quoted 2 lines
Looking at the error log, and this commit, it looks related to me.. though I am not sure I understand the details ;)
Argh, this is because I was silly and the fix is outright wrong. Sorry about that, I'll fix it quick (and maybe add a test this time!)
ah, cool! well, the self claim about being silly ;) but it's good we found it so quickly! I was starting to worry about something wrong in our infrastructure!
if you need me to run a test, let me know, I can run our CI with your patch applied..
Quoted 59 lines
Quoted 59 lines
Quoted 59 lines
--- bitbake/lib/hashserv/client.py | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)bitbake/lib/hashserv/client.py +12
-4@@ -40,7 +40,7 @@ class AsyncClient(object): self._connect_sock = connect_sock - async def _connect(self): + async def connect(self): if self.reader is None or self.writer is None: (self.reader, self.writer) = await self._connect_sock()@@ -62,7 +62,7 @@ class AsyncClient(object): count = 0 while True: try: - await self._connect() + await self.connect() return await proc() except ( OSError,@@ -190,7 +190,6 @@ class Client(object): for call in ( "connect_tcp", - "connect_unix", "close", "get_unihash", "report_unihash",@@ -209,6 +208,16 @@ class Client(object): return wrapper + def connect_unix(self, path): + # AF_UNIX has path length issues so chdir here to workaround + cwd = os.getcwd() + try: + os.chdir(os.path.dirname(path)) + self.loop.run_until_complete(self.client.connect_unix(path)) + self.loop.run_until_complete(self.client.connect()) + finally: + os.chdir(cwd) + @property def max_chunk(self): return self.client.max_chunk --2.29.2