Skip to content

Add support for readonly upstream hash equivalence server

Joshua Watt bitbake-devel 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest2 patches 3 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/2bitbake: hashserve: Add async clientbitbake/lib/hashserv/__init__.py bitbake/lib/hashserv/client.py+143 -1083 replies
2/2bitbake: hashserve: Add support for readonly upstreambitbake/lib/hashserv/__init__.py bitbake/lib/hashserv/client.py bitbake/lib/hashserv/server.py +1 more+268 -860 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [bitbake-devel][PATCH 0/2] Add support for readonly upstream hash equivalence server

Adds support for a hash equivalence server to specify an "upstream" server where it will look for hash if they are not found in the local database. This upstream is "read-only" from the downstream servers perspective (e.g. it will only read hashes from the upstream server, not report equivalencies back to it), but this is not currently enforced by the upstream server.

The intended use case is to have a local hash equivalent server that uses an upstream server, such as the one used by some CI jobs generating a sstate cache.

There is some additional optimization that could be done; currently the downstream server will ask the upstream server for a hash on any local miss, which incurs a lot of latency because it must make a roundtrip query. Ideally, there could be multiple requests interleaved to reduce the overall latency, but bitbake currently only requests one hash at a time so this isn't possible.

Joshua Watt (2):
bitbake: hashserve: Add async client
bitbake: hashserve: Add support for readonly upstream

 bitbake/lib/hashserv/__init__.py |  52 ++++---
 bitbake/lib/hashserv/client.py   | 243 +++++++++++++++++--------------
 bitbake/lib/hashserv/server.py   | 149 +++++++++++++++----
 bitbake/lib/hashserv/tests.py    | 147 +++++++++++++++----
 4 files changed, 404 insertions(+), 187 deletions(-)
Signature · 1 line
2.29.2