Skip to content

hashserv: Pipeline Upstream Queries

Joshua Watt bitbake-devel 3 versions – New

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest11 patches 0 repliesNew
  2. v210 patches 4 repliesSuperseded
  3. v16 patches 1 replySuperseded

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/11asyncrpc: Add Task Grouplib/bb/asyncrpc/__init__.py lib/bb/asyncrpc/taskgroup.py+53 -00 replies
2/11asyncrpc: serv: Use Task Grouplib/bb/asyncrpc/serv.py+1 -10 replies
3/11asyncrpc: serv: Cancel all clients on server stoplib/bb/asyncrpc/serv.py+9 -20 replies
4/11hashserv: tests: Improve test logginglib/hashserv/tests.py+3 -10 replies
5/11hashserv: client: Add asynchronous streaming APIbin/bitbake-hashclient lib/hashserv/client.py lib/hashserv/tests.py+275 -800 replies
6/11hashserv: server: Add queued streaming APIlib/hashserv/server.py+75 -210 replies
7/11hashserv: server: Use streaming and queue API for upstream unihash querieslib/hashserv/server.py+38 -140 replies
8/11hashserv: server: Use streaming and queue API for upstream exist querieslib/hashserv/server.py lib/hashserv/tests.py+31 -50 replies
9/11hashserv: tests: Add more upstream testslib/hashserv/tests.py+141 -110 replies
10/11hashserv: tests: Add test for upstream pipelininglib/hashserv/tests.py+96 -00 replies
11/11hashserv: server: Fix upstream get-unihash miss truncating streamlib/hashserv/server.py lib/hashserv/tests.py+55 -20 replies

Reviews trailers per patch

None of the 11 patches has a Reviewed-by or Acked-by.

PatchMichal Sieron
1/11
2/11
3/11
4/11
5/11
6/11
7/11
8/11
9/11
10/11Co-authored-by
11/11

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v3 thread

Cover letter [bitbake-devel][PATCH v3 00/11] hashserv: Pipeline Upstream Queries

Adds a true asynchronous streaming API to the hash equivalence clients, then uses that streaming API to pipeline requests to an upstream server. This means that when a server wants to defer a request to an upstream, it no longer has to wait for a complete round-trip API call. Instead, it can use the faster streaming API to send the requests, and continue processing other requests while a response inbound.

V2: Fixes several bugs related to when the connection between the client and upstream server was disconnected, and added tests for them.

V3: Fixes an unexpected disconnect when there was a unihash miss from the upstream server while streaming

Joshua Watt (10):
asyncrpc: Add Task Group
asyncrpc: serv: Use Task Group
asyncrpc: serv: Cancel all clients on server stop
hashserv: tests: Improve test logging
hashserv: client: Add asynchronous streaming API
hashserv: server: Add queued streaming API
hashserv: server: Use streaming and queue API for upstream unihash
queries
hashserv: server: Use streaming and queue API for upstream exist
queries
hashserv: tests: Add more upstream tests
hashserv: tests: Add test for upstream pipelining

Michal Sieron (1):
hashserv: server: Fix upstream get-unihash miss truncating stream

 bin/bitbake-hashclient       |   2 +-
 lib/bb/asyncrpc/__init__.py  |   1 +
 lib/bb/asyncrpc/serv.py      |  13 +-
 lib/bb/asyncrpc/taskgroup.py |  52 ++++++
 lib/hashserv/client.py       | 349 +++++++++++++++++++++++++++--------
 lib/hashserv/server.py       | 197 ++++++++++++++++----
 lib/hashserv/tests.py        | 298 ++++++++++++++++++++++++++++--
 7 files changed, 776 insertions(+), 136 deletions(-)
 create mode 100644 lib/bb/asyncrpc/taskgroup.py
Signature · 1 line
2.55.0