bpftool: Consume ring buffer maps with event_pipe
Subsystems (4)
v3 cover bpftool: Consume ring buffer maps with event_pipe
Extend map event_pipe to BPF ring buffer maps using libbpf's ring buffer manager. Print complete records in plain or JSON format, reject CPU/index selectors for ring buffers, and document the shared consumer position.
Changes in v3:
- Remove redundant wording from the manual while retaining the warning about consuming records and sharing the consumer position.
- Explain callback stopping and signal handling in the commit message.
- Rebase onto current bpf-next and use Assisted-by: LLM.
The implementation and v2 tests, including pretty output, pinned paths, empty streams and the perf-event-array regression case, are unchanged.
v2: https://lore.kernel.org/r/[email protected]
Request: https://github.com/libbpf/bpftool/issues/54
The previous version's CI PR expired after repeated "Patch is empty" reports. The revised series applies cleanly to the current tree and is sent as a new thread using git format-patch and git send-email.
Validation:
- Built bpftool and its manual pages, and built the focused selftest runner with LLVM 20.
- All 17 bpftool_ringbuf subtests passed with no skips or failures in an x86-64 KVM guest running Linux 7.3.0-rc2 from current bpf/master.
- The four bpftool series were tested together: all 69 subtests passed. The cpumask suite (36 subtests) and test_maps also passed with no skips.
- Bash syntax and the existing bpftool synchronization checks passed.
This was a focused run; unrelated selftests requiring unavailable kernel features were excluded from the build with PERMISSIVE=1.
Tianyi Chen (2):
bpftool: Read ring buffer maps with event_pipe selftests/bpf: Cover bpftool ring buffer event_pipe
.../bpf/bpftool/Documentation/bpftool-map.rst | 14 +- tools/bpf/bpftool/bash-completion/bpftool | 4 +- tools/bpf/bpftool/map_perf_ring.c | 87 +++- .../bpf/prog_tests/bpftool_ringbuf.c | 425 ++++++++++++++++++ .../selftests/bpf/progs/bpftool_ringbuf.c | 47 ++ 5 files changed, 553 insertions(+), 24 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c create mode 100644 tools/testing/selftests/bpf/progs/bpftool_ringbuf.c
base-commit: af0b84a9215d951d16f26b7ee34353b970cf5d4e
Signature · 1 line
2.55.0
PATCH v3 1/2 bpftool: Read ring buffer maps with event_pipe
From: Tianyi Chen <[email protected]>
map event_pipe only accepts perf event arrays, leaving no built-in way to inspect records produced through the BPF ring buffer API. Extend it to consume queued and live BPF_MAP_TYPE_RINGBUF records in plain or JSON output, retaining the existing perf event array path.
Ring buffers have a shared consumer position and no implicit CPU or timestamp. Document that this command consumes records rather than observing them passively, and reject perf-only CPU/index selectors.
A producer can keep ring_buffer__poll() busy after a stop signal, so return -EINTR from the record callback when stopping. Keep stdio out of the shared signal handler and preserve callback output errors.
--- .../bpf/bpftool/Documentation/bpftool-map.rst | 14 ++- tools/bpf/bpftool/bash-completion/bpftool | 4 +- tools/bpf/bpftool/map_perf_ring.c | 87 ++++++++++++++----- 3 files changed, 81 insertions(+), 24 deletions(-)
tools/bpf/bpftool/Documentation/bpftool-map.rst +13 -1
@@ -120,7 +120,8 @@ bpftool map pin *MAP* *FILE* character ('.'), which is reserved for future extensions of *bpffs*. bpftool map event_pipe *MAP* [cpu *N* index *M*] - Read events from a **BPF_MAP_TYPE_PERF_EVENT_ARRAY** map. + Read events from a **BPF_MAP_TYPE_PERF_EVENT_ARRAY** or + **BPF_MAP_TYPE_RINGBUF** map. Install perf rings into a perf event array map and dump output of any **bpf_perf_event_output**\ () call in the kernel. By default read the
@@ -134,6 +135,17 @@ bpftool map event_pipe *MAP* [cpu *N* index *M*] existing ring. Any other application will stop receiving events if it installed its rings earlier. + For a ring buffer map, consume records submitted by BPF programs, including + records already queued before the command starts. **cpu** and **index** + are not supported. Each record is printed in full, including embedded zero + bytes. Plain output reports the record size followed by hexadecimal + bytes; JSON output contains **size** and **data** fields, with **data** an + array of byte values. Ring buffer records have no implicit CPU or timestamp. + + Consuming a ring buffer advances its shared consumer position, so this + command must not run alongside another consumer of the same map. + **BPF_MAP_TYPE_USER_RINGBUF** maps are not supported. + bpftool map peek *MAP* Peek next value in the queue or stack.
tools/bpf/bpftool/bash-completion/bpftool +2 -2
@@ -878,11 +878,11 @@ _bpftool() return 0 ;; id) - _bpftool_get_map_ids_for_type perf_event_array + _bpftool_get_map_ids_for_type '"type": "\(perf_event_array\|ringbuf\)"' return 0 ;; name) - _bpftool_get_map_names_for_type perf_event_array + _bpftool_get_map_names_for_type '"type": "\(perf_event_array\|ringbuf\)"' return 0 ;; cpu)
tools/bpf/bpftool/map_perf_ring.c +66 -21
144 lines not on this page · raw message
Signature · 1 line
2.55.0
Quentin Monnet
maintainerRe: [PATCH bpf-next v3 1/2] bpftool: Read ring buffer maps with event_pipe
2026-09-11 10:51 UTC+0800 ~ Tianyi Chen <[email protected]>
Quoted 11 lines
From: Tianyi Chen <[email protected]>
map
event_pipeonly accepts perf event arrays, leaving no built-in way to inspect records produced through the BPF ring buffer API. Extend it to consume queued and liveBPF_MAP_TYPE_RINGBUFrecords in plain or JSON output, retaining the existing perf event array path.Ring buffers have a shared consumer position and no implicit CPU or timestamp. Document that this command consumes records rather than observing them passively, and reject perf-only CPU/index selectors.
Quoted 8 lines
A producer can keep
ring_buffer__poll()busy after a stop signal, so return-EINTRfrom the record callback when stopping. Keep stdio out of the shared signal handler and preserve callback output errors.Assisted-by LLMSigned-off-by Tianyi Chen <[email protected]>
Hi, thanks for picking these issues, and apologies for the delay. I've seen your other series as well, but let's take them one after the other, if you don't mind.
Note: When you reply, including to AI bots, please keep the context from the discussion and avoid replying with only your answer in the email, it makes it hard to follow the conversation.
The two patches from this series look good overall, I only have minor comments, please see inline below.
Quoted 43 lines
--- .../bpf/bpftool/Documentation/bpftool-map.rst | 14 ++- tools/bpf/bpftool/bash-completion/bpftool | 4 +- tools/bpf/bpftool/map_perf_ring.c | 87 ++++++++++++++----- 3 files changed, 81 insertions(+), 24 deletions(-)tools/bpf/bpftool/Documentation/bpftool-map.rst +13
-1@@ -120,7 +120,8 @@ bpftool map pin *MAP* *FILE* character ('.'), which is reserved for future extensions of *bpffs*. bpftool map event_pipe *MAP* [cpu *N* index *M*] - Read events from a **BPF_MAP_TYPE_PERF_EVENT_ARRAY** map. + Read events from a **BPF_MAP_TYPE_PERF_EVENT_ARRAY** or + **BPF_MAP_TYPE_RINGBUF** map. Install perf rings into a perf event array map and dump output of any **bpf_perf_event_output**\ () call in the kernel. By default read the@@ -134,6 +135,17 @@ bpftool map event_pipe *MAP* [cpu *N* index *M*] existing ring. Any other application will stop receiving events if it installed its rings earlier. + For a ring buffer map, consume records submitted by BPF programs, including + records already queued before the command starts. **cpu** and **index** + are not supported. Each record is printed in full, including embedded zero + bytes. Plain output reports the record size followed by hexadecimal + bytes; JSON output contains **size** and **data** fields, with **data** an + array of byte values. Ring buffer records have no implicit CPU or timestamp. + + Consuming a ring buffer advances its shared consumer position, so this + command must not run alongside another consumer of the same map. + **BPF_MAP_TYPE_USER_RINGBUF** maps are not supported. + bpftool map peek *MAP* Peek next value in the queue or stack.tools/bpf/bpftool/bash-completion/bpftool +0
-0
tools/bpf/bpftool/bash-completion/bpftool _bpftool() · line 885
Quoted 10 lines
tools/bpf/bpftool/bash-completion/bpftool +2
-2@@ -878,11 +878,11 @@ _bpftool() return 0 ;; id) - _bpftool_get_map_ids_for_type perf_event_array + _bpftool_get_map_ids_for_type '"type": "\(perf_event_array\|ringbuf\)"' return 0 ;; name) - _bpftool_get_map_names_for_type perf_event_array + _bpftool_get_map_names_for_type '"type": "\(perf_event_array\|ringbuf\)"'
I'm not a fan of this change, it doesn't look clean. Let's adjust the functions to take multiple arguments, instead; see my proposal at the end of this message.
Quoted 69 lines
tools/bpf/bpftool/bash-completion/bpftool +0
-0return 0 ;; cpu)tools/bpf/bpftool/map_perf_ring.c +25
-3@@ -27,7 +27,7 @@#define MMAP_PAGE_CNT 16 -static volatile bool stop; +static volatile sig_atomic_t stop;struct perf_event_sample {struct perf_event_header header;@@ -44,7 +44,6 @@ struct perf_event_lost {static void int_exit (int signo) { -fprintf (stderr,"Stopping...\n" ); stop = true; }@@ -107,6 +106,27 @@ print_bpf_output(void *private_data, int cpu, struct perf_event_header *event)return LIBBPF_PERF_EVENT_CONT ; } +static int print_ringbuf_output (void *ctx,void *data,size_t size) +{ +if (json_output) { +jsonw_start_object (json_wtr); +jsonw_uint_field (json_wtr,"size" , size); +jsonw_name (json_wtr,"data" ); +print_data_json (data, size); +jsonw_end_object (json_wtr); + }else { +printf ("== size: %zu =====\n" , size); +fprint_hex (stdout, data, size," " ); +printf ("\n" ); + } + +if (fflush (stdout)) +return errno ? -errno : -EIO ; + +/* A producer can keep poll() busy even after a signal arrives. */ +return stop ? -EINTR :0 ; +} +int do_event_pipe (int argc,char **argv) {struct perf_event_attr perf_attr = {@@ -123,18 +143,26 @@ int do_event_pipe(int argc, char **argv) .cpu = -1, .idx = -1, }; -struct perf_buffer *pb; +struct perf_buffer *pb =NULL ; +struct ring_buffer *rb =NULL ;__u32 map_info_len;int err, map_fd; + stop = false; map_info_len =sizeof (map_info); map_fd =map_parse_fd_and_info (&argc, &argv, &map_info, &map_info_len,0 );if (map_fd <0 )return -1 ;
tools/bpf/bpftool/map_perf_ring.c int do_event_pipe(int argc, char **argv) · line 165
Quoted 10 lines
tools/bpf/bpftool/map_perf_ring.c +8
-2-if (map_info.type !=BPF_MAP_TYPE_PERF_EVENT_ARRAY ) { -p_err ("map is not a perf event array" ); +if (map_info.type !=BPF_MAP_TYPE_PERF_EVENT_ARRAY && + map_info.type !=BPF_MAP_TYPE_RINGBUF ) { +p_err ("map is not a perf event array or ring buffer" ); + goto err_close_map; + } + +if (map_info.type ==BPF_MAP_TYPE_RINGBUF && argc) { +p_err ("ring buffer maps do not support cpu or index arguments" );
Maybe restrict this message for when we find explicitly "cpu" or "index" as argument, and leave the regular parsing error for other argument names? If the user adds a trailing "foobar" argument, it's strange to get an error message saying "cpu" or "index" are not supported.
Quoted 41 lines
tools/bpf/bpftool/map_perf_ring.c +18
-9goto err_close_map; }@@ -184,15 +212,24 @@ int do_event_pipe(int argc, char **argv) ctx.idx =0 ; } - opts.cpu_cnt = ctx.all_cpus ?0 :1 ; - opts.cpus = &ctx.cpu; - opts.map_keys = &ctx.idx; - pb =perf_buffer__new_raw (map_fd,MMAP_PAGE_CNT , &perf_attr, - print_bpf_output, &ctx, &opts); -if (!pb) { -p_err ("failed to create perf buffer: %s (%d)" , -strerror (errno), errno); - goto err_close_map; +if (map_info.type ==BPF_MAP_TYPE_RINGBUF ) { + rb =ring_buffer__new (map_fd, print_ringbuf_output,NULL ,NULL ); +if (!rb) { +p_err ("failed to create ring buffer: %s (%d)" , +strerror (errno), errno); + goto err_close_map; + } + }else { + opts.cpu_cnt = ctx.all_cpus ?0 :1 ; + opts.cpus = &ctx.cpu; + opts.map_keys = &ctx.idx; + pb =perf_buffer__new_raw (map_fd,MMAP_PAGE_CNT , &perf_attr, + print_bpf_output, &ctx, &opts); +if (!pb) { +p_err ("failed to create perf buffer: %s (%d)" , +strerror (errno), errno); + goto err_close_map; + } }signal (SIGINT , int_exit);@@ -202,25 +239,33 @@ int do_event_pipe(int argc, char **argv)if (json_output)jsonw_start_array (json_wtr);
tools/bpf/bpftool/map_perf_ring.c int do_event_pipe(int argc, char **argv) · line 247
Quoted 10 lines
tools/bpf/bpftool/map_perf_ring.c +4
-4+ err =0 ;while (!stop) { - err =perf_buffer__poll (pb,200 ); + err = rb ?ring_buffer__poll (rb,200 ) :perf_buffer__poll (pb,200 );if (err <0 && err != -EINTR ) { -p_err ("perf buffer polling failed: %s (%d)" , -strerror (errno), errno); - goto err_close_pb; +fprintf (stderr,"Error: %s buffer polling failed: %s (%d)\n" , + rb ?"ring" :"perf" ,strerror (-err), -err);
Please use p_err() instead of fprintf().
tools/bpf/bpftool/map_perf_ring.c int do_event_pipe(int argc, char **argv) · line 258
Quoted 11 lines
tools/bpf/bpftool/map_perf_ring.c +6
-0+break ; } + err =0 ; } +if (stop) +fprintf (stderr,"Stopping...\n" );if (json_output)jsonw_end_array (json_wtr); +if (fflush (stdout)) { +fprintf (stderr,"Error: failed to write events: %s\n" ,strerror (errno));
p_err("...") rather than fprintf(stderr, "Error: ... \n").
tools/bpf/bpftool/map_perf_ring.c int do_event_pipe(int argc, char **argv) · line 264
Quoted 6 lines
tools/bpf/bpftool/map_perf_ring.c +4
-0+ err =-1 ; + } +ring_buffer__free (rb);perf_buffer__free (pb); +/* Both buffer managers borrow map_fd. */
Nit: Let's drop this comment.
tools/bpf/bpftool/map_perf_ring.c int do_event_pipe(int argc, char **argv) · line 271
Quoted 10 lines
tools/bpf/bpftool/map_perf_ring.c +1
-3close (map_fd); -return 0 ; +return err <0 ?-1 :0 ; -err_close_pb: -perf_buffer__free (pb); err_close_map:close (map_fd);return -1 ;
bash completion proposal below:
------
tools/bpf/bpftool/bash-completion/bpftool +30 -24
@@ -50,12 +50,20 @@ _bpftool_get_map_ids() command sed -n 's/.*"id": \(.*\),$/\1/p' )" -- "$cur" ) ) } -# Takes map type and adds matching map ids to the list of suggestions. -_bpftool_get_map_ids_for_type() +# Takes one or more map types and prints an extended regular expression +# matching the "type" field of any of them in bpftool's JSON output. +_bpftool_build_type_pattern() { - local type="$1" + local IFS='|' + printf '"type": "(%s)"\n' "$*" +} + +# Takes map types and adds matching map ids to the list of suggestions. +_bpftool_get_map_ids_for_types() +{ + local types=$(_bpftool_build_type_pattern "$@") COMPREPLY+=( $( compgen -W "$( bpftool -jp map 2>&1 | \ - command grep -C2 "$type" | \ + command grep -C2 -E "$types" | \ command sed -n 's/.*"id": \(.*\),$/\1/p' )" -- "$cur" ) ) }
@@ -65,12 +73,12 @@ _bpftool_get_map_names() command sed -n 's/.*"name": \(.*\),$/\1/p' )" -- "$cur" ) ) } -# Takes map type and adds matching map names to the list of suggestions. -_bpftool_get_map_names_for_type() +# Takes map types and adds matching map names to the list of suggestions. +_bpftool_get_map_names_for_types() { - local type="$1" + local types=$(_bpftool_build_type_pattern "$@") COMPREPLY+=( $( compgen -W "$( bpftool -jp map 2>&1 | \ - command grep -C2 "$type" | \ + command grep -C2 -E "$types" | \ command sed -n 's/.*"name": \(.*\),$/\1/p' )" -- "$cur" ) ) }
@@ -177,10 +185,10 @@ _bpftool_map_update_get_id() if [[ $value -eq 0 ]]; then case "$command" in push) - _bpftool_get_map_ids_for_type stack + _bpftool_get_map_ids_for_types stack ;; enqueue) - _bpftool_get_map_ids_for_type queue + _bpftool_get_map_ids_for_types queue ;; *) _bpftool_get_map_ids
@@ -223,10 +231,10 @@ _bpftool_map_update_get_name() if [[ $value -eq 0 ]]; then case "$command" in push) - _bpftool_get_map_names_for_type stack + _bpftool_get_map_names_for_types stack ;; enqueue) - _bpftool_get_map_names_for_type queue + _bpftool_get_map_names_for_types queue ;; *) _bpftool_get_map_names
@@ -617,10 +625,10 @@ _bpftool() COMPREPLY=( $( compgen -W "$STRUCT_OPS_TYPE" -- "$cur" ) ) ;; id) - _bpftool_get_map_ids_for_type struct_ops + _bpftool_get_map_ids_for_types struct_ops ;; name) - _bpftool_get_map_names_for_type struct_ops + _bpftool_get_map_names_for_types struct_ops ;; esac return 0
@@ -679,14 +687,13 @@ _bpftool() id) case "$command" in peek) - _bpftool_get_map_ids_for_type stack - _bpftool_get_map_ids_for_type queue + _bpftool_get_map_ids_for_types stack queue ;; pop) - _bpftool_get_map_ids_for_type stack + _bpftool_get_map_ids_for_types stack ;; dequeue) - _bpftool_get_map_ids_for_type queue + _bpftool_get_map_ids_for_types queue ;; *) _bpftool_get_map_ids
@@ -697,14 +704,13 @@ _bpftool() name) case "$command" in peek) - _bpftool_get_map_names_for_type stack - _bpftool_get_map_names_for_type queue + _bpftool_get_map_names_for_types stack queue ;; pop) - _bpftool_get_map_names_for_type stack + _bpftool_get_map_names_for_types stack ;; dequeue) - _bpftool_get_map_names_for_type queue + _bpftool_get_map_names_for_types queue ;; *) _bpftool_get_map_names
@@ -878,11 +884,11 @@ _bpftool() return 0 ;; id) - _bpftool_get_map_ids_for_type '"type": "\(perf_event_array\|ringbuf\)"' + _bpftool_get_map_ids_for_types perf_event_array ringbuf return 0 ;; name) - _bpftool_get_map_names_for_type '"type": "\(perf_event_array\|ringbuf\)"' + _bpftool_get_map_names_for_types perf_event_array ringbuf return 0 ;; cpu)
Re: [PATCH bpf-next v3 1/2] bpftool: Read ring buffer maps with event_pipe
Hi Quentin,
Quoted 1 line
Let's adjust the functions to take multiple arguments, instead
Thanks, v4 follows your completion proposal and also addresses the argument diagnostic, p_err() and redundant-comment feedback.
Quoted 2 lines
As bpf-ci's review says, please look into reusing the existing helpers if possible.
The test now reuses detect_bpftool_path() and checks the exec-error write. All 18 subtests pass, including the new unknown-argument case. I also checked completion matching and plain/JSON polling-error output.
v4 is a new thread:
https://lore.kernel.org/r/[email protected]
Thanks,
Tianyi
PATCH v3 2/2 selftests/bpf: Cover bpftool ring buffer event_pipe
In reply to Tianyi ChenFrom: Tianyi Chen <[email protected]>
Produce known ring buffer records and check complete plain, JSON and pretty JSON output. Exercise ID and pinned map selection, SIGINT and SIGTERM shutdown, empty streams and invalid map types or selectors.
Also produce a perf event sample and check its existing header and raw payload output. Use a payload whose size plus the raw sample length field is aligned to eight bytes so the expected bytes exclude implicit perf padding.
--- .../bpf/prog_tests/bpftool_ringbuf.c | 425 ++++++++++++++++++ .../selftests/bpf/progs/bpftool_ringbuf.c | 47 ++ 2 files changed, 472 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c create mode 100644 tools/testing/selftests/bpf/progs/bpftool_ringbuf.c
tools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c new file +425 -0
@@ -0,0 +1,425 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <ctype.h> +#include <fcntl.h> +#include <poll.h> +#include <signal.h> +#include <sys/mman.h> +#include <sys/wait.h> +#include <test_progs.h> +#include "bpftool_ringbuf.skel.h" + +#define WAIT_STEPS 500 +#define WAIT_US 10000 + +struct consumer { +pid_t pid; +int fd; +}; + +static const char *bpftool_path (void ) +{ +const char *path =getenv ("BPFTOOL" ); + +if (path) +return access (path,X_OK ) ?NULL : path; +if (!access ("./tools/sbin/bpftool" ,X_OK )) +return "./tools/sbin/bpftool" ; +if (!access ("../tools/sbin/bpftool" ,X_OK )) +return "../tools/sbin/bpftool" ; +return NULL ; +} + +static void consumer_cleanup (struct consumer *child) +{ +if (child->pid >0 ) { +kill (child->pid,SIGKILL ); +while (waitpid (child->pid,NULL ,0 ) <0 && errno ==EINTR ) + ; + child->pid =-1 ; + } +if (child->fd >=0 ) { +close (child->fd); + child->fd =-1 ; + } +} + +static bool consumer_start (struct consumer *child,int map_fd,const char *format, +const char *pin_path,const char *option, +bool pair,bool capture_errors) +{ +struct bpf_map_info info = {}; +__u32 len =sizeof (info); +const char *path =bpftool_path (); +char *argv[12 ], id[16 ]; +int out[2 ], ready[2 ], n =0 , err, err_fd; +struct pollfd pfd; + +if (!ASSERT_OK_PTR (path,"bpftool path (set BPFTOOL to override)" ) || + !ASSERT_OK (bpf_map_get_info_by_fd (map_fd, &info, &len),"map info" )) +return false; +snprintf (id,sizeof (id),"%u" , info.id); + argv[n++] = (char *)path; +if (format) + argv[n++] = (char *)format; + argv[n++] ="map" ; + argv[n++] ="event_pipe" ; + argv[n++] = pin_path ?"pinned" :"id" ; + argv[n++] = pin_path ? (char *)pin_path : id; +if (option) { + argv[n++] = (char *)option; + argv[n++] ="0" ; +if (pair) { + argv[n++] ="index" ; + argv[n++] ="0" ; + } + } + argv[n] =NULL ; +if (!ASSERT_OK (pipe2 (out,O_CLOEXEC ),"output pipe" )) +return false; +if (!ASSERT_OK (pipe2 (ready,O_CLOEXEC ),"exec pipe" )) { +close (out[0 ]); +close (out[1 ]); +return false; + } + child->pid =fork (); +if (!child->pid) { +close (out[0 ]); +close (ready[0 ]); + err_fd = capture_errors ? out[1 ] :open ("/dev/null" ,O_WRONLY ); +if (dup2 (out[1 ],STDOUT_FILENO ) <0 || +dup2 (err_fd,STDERR_FILENO ) <0 ) + goto exec_fail; +if (!capture_errors) +close (err_fd); +close (out[1 ]); +execv (path, argv); +exec_fail: + err = errno; +write (ready[1 ], &err,sizeof (err)); +_exit (127 ); + } +close (out[1 ]); +close (ready[1 ]); + child->fd = out[0 ]; + pfd = (struct pollfd ) { .fd = ready[0 ], .events =POLLIN }; +/* EOF on the close-on-exec pipe distinguishes exec from inherited handlers. */ + err = child->pid >0 ?poll (&pfd,1 ,WAIT_STEPS *WAIT_US /1000 ) :-1 ; +if (!ASSERT_GT (child->pid,0 ,"fork" ) || + !ASSERT_GT (err,0 ,"exec timeout" ) || + !ASSERT_EQ (read (ready[0 ], &err,sizeof (err)),0 ,"exec" )) { +close (ready[0 ]); +consumer_cleanup (child); +return false; + } +close (ready[0 ]); +return true; +} + +static bool consumer_finish (struct consumer *child,int signo,bool success, +char *output,size_t size) +{ +int status =0 , i; +pid_t ret =0 ; +ssize_t n; +size_t used =0 ; + +if (!ASSERT_OK (fcntl (child->fd,F_SETFL ,O_NONBLOCK ),"nonblocking output" )) +return false; +if (signo && !ASSERT_OK (kill (child->pid, signo),"signal consumer" )) +return false; +for (i =0 ; i <WAIT_STEPS ; i++) { +while (used < size -1 && + (n =read (child->fd, output + used, size -1 - used)) >0 ) + used += n; + ret =waitpid (child->pid, &status,WNOHANG ); +if (ret == child->pid) +break ; +if (ret <0 && errno !=EINTR ) +break ; +usleep (WAIT_US ); + } +if (!ASSERT_EQ (ret, child->pid,"bounded consumer exit" )) +return false; + child->pid =-1 ; +while (used < size -1 && (n =read (child->fd, output + used, size -1 - used)) >0 ) + used += n; + output[used] ='\0' ; +return ASSERT_TRUE (WIFEXITED (status),"normal exit" ) && +ASSERT_EQ (WEXITSTATUS (status) ==0 , success,"exit status" ); +} + +static bool consumer_ready (struct consumer *child) +{ +unsigned long long caught; +char path[64 ], line[256 ]; +int i; +FILE *f; + +snprintf (path,sizeof (path),"/proc/%d/status" , child->pid); +for (i =0 ; i <WAIT_STEPS ; i++) { + f =fopen (path,"r" ); +if (!f) +break ; +while (fgets (line,sizeof (line), f)) { +if (sscanf (line,"SigCgt: %llx" , &caught) ==1 && + (caught & (1ULL << (SIGINT -1 ))) && + (caught & (1ULL << (SIGTERM -1 )))) { +fclose (f); +return true; + } + } +fclose (f); +usleep (WAIT_US ); + } +return ASSERT_TRUE (false,"consumer signal handlers ready" ); +} + +static bool emit_record (struct bpftool_ringbuf *skel,int record) +{ +char packet[64 ] = {}; + +LIBBPF_OPTS (bpf_test_run_opts, opts, + .data_in = packet, + .data_size_in = sizeof(packet), + ); + + skel->bss->record = record; +return ASSERT_OK (bpf_prog_test_run_opts (bpf_program__fd (skel->progs.produce), + &opts),"produce record" ) && +ASSERT_OK (skel->bss->output_err,"ringbuf output" ); +} + +static bool consumed (unsigned long *position,unsigned long expected) +{ +int i; + +for (i =0 ; i <WAIT_STEPS ; i++) { +if (__atomic_load_n (position, __ATOMIC_ACQUIRE) == expected) +return true; +usleep (WAIT_US ); + } +return ASSERT_EQ (*position, expected,"consumer position" ); +} + +static void check_json (char *output,const char *expected) +{ +char *src = output, *dst = output; +bool quoted = false, escaped = false; + +/* Ignore formatting whitespace while checking the entire JSON document. */ +while (*src) { +if (quoted || !isspace ((unsigned char )*src)) + *dst++ = *src; +if (!escaped && *src =='"' ) + quoted = !quoted; + escaped = !escaped && quoted && *src =='\\' ; + src++; + } + *dst ='\0' ; +ASSERT_STREQ (output, expected,"JSON records" ); +} + +static void test_consumer (const char *format,bool idle,int signo,bool pinned) +{ +struct consumer child = { .pid =-1 , .fd =-1 }; +struct bpftool_ringbuf *skel; +unsigned long *position =MAP_FAILED ; +int page_size =getpagesize (), fd; +char pin_dir[] ="/sys/fs/bpf/bpftool_ringbuf_XXXXXX" ; +char pin_path[sizeof (pin_dir) +sizeof ("/map" )]; +bool dir_created = false, map_pinned = false; +char output[4096 ]; +struct pollfd pfd; + + skel =bpftool_ringbuf__open (); +if (!ASSERT_OK_PTR (skel,"open" )) +return ; +bpf_map__set_max_entries (skel->maps.ringbuf, page_size); +if (!ASSERT_OK (bpftool_ringbuf__load (skel),"load" )) + goto out; + fd =bpf_map__fd (skel->maps.ringbuf); +if (pinned) { +if (!ASSERT_OK_PTR (mkdtemp (pin_dir),"create pin directory" )) + goto out; + dir_created = true; +snprintf (pin_path,sizeof (pin_path),"%s/map" , pin_dir); +if (!ASSERT_OK (bpf_obj_pin (fd, pin_path),"pin ringbuf" )) + goto out; + map_pinned = true; + } + position =mmap (NULL , page_size,PROT_READ ,MAP_SHARED , fd,0 ); +if (!ASSERT_NEQ (position,MAP_FAILED ,"consumer mmap" )) + goto out; +if (!idle && (!emit_record (skel,0 ) || !emit_record (skel,1 ))) + goto out; +if (!consumer_start (&child, fd, format, pinned ? pin_path :NULL , +NULL , false, false) || + !consumer_ready (&child)) + goto out; +if (!idle) { +/* Both prefilled records occupy 16 bytes including their headers. */ +if (!consumed (position,32 )) + goto out; + pfd = (struct pollfd ) { .fd = child.fd, .events =POLLIN }; +if (!ASSERT_GT (poll (&pfd,1 ,WAIT_STEPS *WAIT_US /1000 ),0 , +"records flushed before exit" ) || + !ASSERT_TRUE (pfd.revents &POLLIN ,"record output readable" ) || + !emit_record (skel,2 ) || !consumed (position,64 )) + goto out; + } +if (!consumer_finish (&child, signo, true, output,sizeof (output))) + goto out; +if (format && !strcmp (format,"-p" )) +ASSERT_STREQ (output, idle ?"[]\n" : +"[{\n" +" \"size\": 2,\n" +" \"data\": [0,255\n" +" ]\n" +" },{\n" +" \"size\": 5,\n" +" \"data\": [1,2,3,4,5\n" +" ]\n" +" },{\n" +" \"size\": 17,\n" +" \"data\": [16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32\n" +" ]\n" +" }\n" +"]\n" ,"pretty JSON records" ); +else if (format) +check_json (output, idle ?"[]" : +"[{\"size\":2,\"data\":[0,255]}," +"{\"size\":5,\"data\":[1,2,3,4,5]}," +"{\"size\":17,\"data\":[16,17,18,19,20,21,22,23," +"24,25,26,27,28,29,30,31,32]}]" ); +else +ASSERT_STREQ (output, idle ?"" : +"== size: 2 =====\n00 ff\n" +"== size: 5 =====\n01 02 03 04 05\n" +"== size: 17 =====\n10 11 12 13 14 15 16 17 " +"18 19 1a 1b 1c 1d 1e 1f\n20\n" ,"plain records" ); +out: +consumer_cleanup (&child); +if (position !=MAP_FAILED ) +munmap (position, page_size); +if (map_pinned) +ASSERT_OK (unlink (pin_path),"unpin ringbuf" ); +if (dir_created) +ASSERT_OK (rmdir (pin_dir),"remove pin directory" ); +bpftool_ringbuf__destroy (skel); +} + +static void test_perf_consumer (void ) +{ +struct consumer child = { .pid =-1 , .fd =-1 }; +unsigned long long seconds, nanoseconds; +struct bpftool_ringbuf *skel; +char output[16384 ], expected[16384 ]; +int nr_cpus, cpu, index, offset =0 , i, used =0 , fields; +struct pollfd pfd; + + nr_cpus =libbpf_num_possible_cpus (); +if (!ASSERT_GT (nr_cpus,0 ,"possible cpus" )) +return ; + skel =bpftool_ringbuf__open (); +if (!ASSERT_OK_PTR (skel,"open" )) +return ; +bpf_map__set_max_entries (skel->maps.ringbuf,getpagesize ()); +bpf_map__set_max_entries (skel->maps.perfbuf, nr_cpus); +if (!ASSERT_OK (bpftool_ringbuf__load (skel),"load" ) || + !consumer_start (&child,bpf_map__fd (skel->maps.perfbuf),NULL , +NULL ,NULL , false, false) || !consumer_ready (&child) || + !emit_record (skel,3 )) + goto out; +/* One large record flushes the existing buffered perf output callback. */ + pfd = (struct pollfd ) { .fd = child.fd, .events =POLLIN }; +if (!ASSERT_GT (poll (&pfd,1 ,WAIT_STEPS *WAIT_US /1000 ),0 , +"perf record output" ) || + !ASSERT_TRUE (pfd.revents &POLLIN ,"perf output readable" ) || + !consumer_finish (&child,SIGINT , true, output,sizeof (output))) + goto out; + fields =sscanf (output,"== @%llu.%llu CPU: %d index: %d =====\n%n" , + &seconds, &nanoseconds, &cpu, &index, &offset); +if (!ASSERT_EQ (fields,4 ,"perf header" ) || + !ASSERT_GT (offset,0 ,"perf payload offset" )) + goto out; +ASSERT_GT (seconds *1000000000ULL + nanoseconds,0 ,"perf timestamp" ); +ASSERT_LT (nanoseconds,1000000000ULL ,"perf timestamp nanoseconds" ); +ASSERT_GE (cpu,0 ,"perf cpu" ); +ASSERT_LT (cpu, nr_cpus,"perf cpu range" ); +ASSERT_EQ (index, cpu,"perf index" ); +for (i =0 ; i <sizeof (skel->rodata->perf_payload); i++) { +const char *separator = !i ?"" : !(i %16 ) ?"\n" : + !(i %8 ) ?" " :" " ; + + used +=snprintf (expected + used,sizeof (expected) - used, +"%s%02x" , separator, i ==1 ?0xff :0 ); + } +snprintf (expected + used,sizeof (expected) - used,"\n" ); +ASSERT_STREQ (output + offset, expected,"perf payload" ); +out: +consumer_cleanup (&child); +bpftool_ringbuf__destroy (skel); +} + +static void test_reject (enum bpf_map_type type,const char *option,bool pair,bool json) +{ +struct consumer child = { .pid =-1 , .fd =-1 }; +bool ring = type ==BPF_MAP_TYPE_RINGBUF || type ==BPF_MAP_TYPE_USER_RINGBUF ; +const char *expected = option ? +"{\"error\":\"ring buffer maps do not support cpu or index arguments\"}" : +"{\"error\":\"map is not a perf event array or ring buffer\"}" ; +char output[4096 ]; +int fd; + + fd =bpf_map_create (type,NULL , ring ?0 :4 , ring ?0 :4 , + ring ?getpagesize () :1 ,NULL ); +if (!ASSERT_GE (fd,0 ,"create map" )) +return ; +if (consumer_start (&child, fd, json ?"-j" :NULL ,NULL , option, pair, true) && +consumer_finish (&child,0 , false, output,sizeof (output))) { +if (json) +check_json (output, expected); +else +ASSERT_GT (strlen (output),0 ,"error diagnostic" ); + } +consumer_cleanup (&child); +close (fd); +} + +void test_bpftool_ringbuf (void ) +{ +if (test__start_subtest ("perf_event_array" )) +test_perf_consumer (); +if (test__start_subtest ("plain" )) +test_consumer (NULL , false,SIGINT , false); +if (test__start_subtest ("json" )) +test_consumer ("-j" , false,SIGTERM , false); +if (test__start_subtest ("pretty_json" )) +test_consumer ("-p" , false,SIGINT , false); +if (test__start_subtest ("pinned_plain" )) +test_consumer (NULL , false,SIGTERM , true); +if (test__start_subtest ("pinned_json" )) +test_consumer ("-j" , false,SIGINT , true); +if (test__start_subtest ("pinned_pretty_json" )) +test_consumer ("-p" , false,SIGTERM , true); +if (test__start_subtest ("idle_sigint" )) +test_consumer (NULL , true,SIGINT , false); +if (test__start_subtest ("idle_sigterm_json" )) +test_consumer ("-j" , true,SIGTERM , false); +if (test__start_subtest ("idle_sigint_pretty_json" )) +test_consumer ("-p" , true,SIGINT , false); +if (test__start_subtest ("reject_array" )) +test_reject (BPF_MAP_TYPE_ARRAY ,NULL , false, false); +if (test__start_subtest ("reject_user_ringbuf" )) +test_reject (BPF_MAP_TYPE_USER_RINGBUF ,NULL , false, false); +if (test__start_subtest ("reject_cpu" )) +test_reject (BPF_MAP_TYPE_RINGBUF ,"cpu" , false, false); +if (test__start_subtest ("reject_index" )) +test_reject (BPF_MAP_TYPE_RINGBUF ,"index" , false, false); +if (test__start_subtest ("reject_cpu_index" )) +test_reject (BPF_MAP_TYPE_RINGBUF ,"cpu" , true, false); +if (test__start_subtest ("reject_user_ringbuf_json" )) +test_reject (BPF_MAP_TYPE_USER_RINGBUF ,NULL , false, true); +if (test__start_subtest ("reject_cpu_index_json" )) +test_reject (BPF_MAP_TYPE_RINGBUF ,"cpu" , true, true); +}
tools/testing/selftests/bpf/progs/bpftool_ringbuf.c new file +47 -0
47 lines not on this page · raw message
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v3 2/2] selftests/bpf: Cover bpftool ring buffer event_pipe
Does bpftool_path() need to duplicate the path discovery logic that already exists in bpftool_helpers.c?
Quoted 26 lines
tools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c new file +20
-0@@ -0,0 +1,425 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <ctype.h> +#include <fcntl.h> +#include <poll.h> +#include <signal.h> +#include <sys/mman.h> +#include <sys/wait.h> +#include <test_progs.h> +#include "bpftool_ringbuf.skel.h" + +#define WAIT_STEPS 500 +#define WAIT_US 10000 + +struct consumer { +pid_t pid; +int fd; +}; + +static const char *bpftool_path (void ) +{
tools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c line 30
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c new file +10
-0+const char *path =getenv ("BPFTOOL" ); + +if (path) +return access (path,X_OK ) ?NULL : path; +if (!access ("./tools/sbin/bpftool" ,X_OK )) +return "./tools/sbin/bpftool" ; +if (!access ("../tools/sbin/bpftool" ,X_OK )) +return "../tools/sbin/bpftool" ; +return NULL ; +}
Does bpftool_path() need to duplicate the path discovery logic that already exists in bpftool_helpers.c?
tools/testing/selftests/bpf/bpftool_helpers.c:detect_bpftool_path() performs the identical three-step search:
bpftool_helpers.c:detect_bpftool_path() {
...
#define BPFTOOL_DEFAULT_PATH "tools/sbin/bpftool"
env_path = getenv("BPFTOOL");
if (env_path && access(env_path, X_OK) == 0) ...
snprintf(tmp, sizeof(tmp), "./%s", BPFTOOL_DEFAULT_PATH);
if (access(tmp, X_OK) == 0) ...
snprintf(tmp, sizeof(tmp), "../s", BPFTOOL_DEFAULT_PATH);
if (access(tmp, X_OK) == 0) ...
}The existing implementation is already linked into test_progs via TRUNNER_EXTRA_SOURCES, and it includes diagnostics that this copy drops:
"bpftool '%s' doesn't exist or is not executable" and "Failed to detect bpftool path, use BPFTOOL env var to override", plus comments explaining why the '../' fallback exists (for running non-default test_progs flavors like no_alu32 or cpuv4).
The public API (run_bpftool_command/get_bpftool_command_output) uses popen(), which doesn't work here since this test needs fork/exec with a controllable child pid for kill/waitpid and a readable fd for poll/read. Could the existing detect_bpftool_path() be exported in bpftool_helpers.h to provide just the path lookup without the popen machinery?
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c new file +9
-0+ +static void consumer_cleanup (struct consumer *child) +{ +if (child->pid >0 ) { +kill (child->pid,SIGKILL ); +while (waitpid (child->pid,NULL ,0 ) <0 && errno ==EINTR ) + ; + child->pid =-1 ; + }
tools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c line 49
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c new file +10
-0+if (child->fd >=0 ) { +close (child->fd); + child->fd =-1 ; + } +} + +static bool consumer_start (struct consumer *child,int map_fd,const char *format, +const char *pin_path,const char *option, +bool pair,bool capture_errors) +{
This isn't a bug, but would a small options struct make the call sites more readable than the trailing positional booleans?
Most call sites pass NULL and false for the last few parameters:
consumer_start(&child, fd, format, pinned ? pin_path : NULL,
NULL, false, false)
consumer_start(&child, bpf_map__fd(skel->maps.perfbuf), NULL,
NULL, NULL, false, false)
At these call sites nothing indicates which NULL is option and which false is pair versus capture_errors. Only the reject path ever passes non-NULL option or true pair/capture_errors, so those flags could be isolated (perhaps a wrapper for the reject case) so the common call sites don't have to carry them.
[ ... ]
tools/testing/selftests/bpf/progs/bpftool_ringbuf.c
Quoted 5 lines
tools/testing/selftests/bpf/progs/bpftool_ringbuf.c new file +0
-0
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34557186664
Quentin Monnet
maintainerRe: [PATCH bpf-next v3 2/2] selftests/bpf: Cover bpftool ring buffer event_pipe
In reply to Tianyi Chen2026-09-11 10:51 UTC+0800 ~ Tianyi Chen <[email protected]>
Quoted 46 lines
From: Tianyi Chen <[email protected]>
Produce known ring buffer records and check complete plain, JSON and pretty JSON output. Exercise ID and pinned map selection, SIGINT and SIGTERM shutdown, empty streams and invalid map types or selectors.
Also produce a perf event sample and check its existing header and raw payload output. Use a payload whose size plus the raw sample length field is aligned to eight bytes so the expected bytes exclude implicit perf padding.
Assisted-by LLMSigned-off-by Tianyi Chen <[email protected]>--- .../bpf/prog_tests/bpftool_ringbuf.c | 425 ++++++++++++++++++ .../selftests/bpf/progs/bpftool_ringbuf.c | 47 ++ 2 files changed, 472 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c create mode 100644 tools/testing/selftests/bpf/progs/bpftool_ringbuf.ctools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c new file +20
-0@@ -0,0 +1,425 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <ctype.h> +#include <fcntl.h> +#include <poll.h> +#include <signal.h> +#include <sys/mman.h> +#include <sys/wait.h> +#include <test_progs.h> +#include "bpftool_ringbuf.skel.h" + +#define WAIT_STEPS 500 +#define WAIT_US 10000 + +struct consumer { +pid_t pid; +int fd; +}; + +static const char *bpftool_path (void ) +{
tools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c line 30
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c new file +10
-0+const char *path =getenv ("BPFTOOL" ); + +if (path) +return access (path,X_OK ) ?NULL : path; +if (!access ("./tools/sbin/bpftool" ,X_OK )) +return "./tools/sbin/bpftool" ; +if (!access ("../tools/sbin/bpftool" ,X_OK )) +return "../tools/sbin/bpftool" ; +return NULL ; +}
As bpf-ci's review says, please look into reusing the existing helpers if possible.
Quoted 58 lines
tools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c new file +58
-0+ +static void consumer_cleanup (struct consumer *child) +{ +if (child->pid >0 ) { +kill (child->pid,SIGKILL ); +while (waitpid (child->pid,NULL ,0 ) <0 && errno ==EINTR ) + ; + child->pid =-1 ; + } +if (child->fd >=0 ) { +close (child->fd); + child->fd =-1 ; + } +} + +static bool consumer_start (struct consumer *child,int map_fd,const char *format, +const char *pin_path,const char *option, +bool pair,bool capture_errors) +{ +struct bpf_map_info info = {}; +__u32 len =sizeof (info); +const char *path =bpftool_path (); +char *argv[12 ], id[16 ]; +int out[2 ], ready[2 ], n =0 , err, err_fd; +struct pollfd pfd; + +if (!ASSERT_OK_PTR (path,"bpftool path (set BPFTOOL to override)" ) || + !ASSERT_OK (bpf_map_get_info_by_fd (map_fd, &info, &len),"map info" )) +return false; +snprintf (id,sizeof (id),"%u" , info.id); + argv[n++] = (char *)path; +if (format) + argv[n++] = (char *)format; + argv[n++] ="map" ; + argv[n++] ="event_pipe" ; + argv[n++] = pin_path ?"pinned" :"id" ; + argv[n++] = pin_path ? (char *)pin_path : id; +if (option) { + argv[n++] = (char *)option; + argv[n++] ="0" ; +if (pair) { + argv[n++] ="index" ; + argv[n++] ="0" ; + } + } + argv[n] =NULL ; +if (!ASSERT_OK (pipe2 (out,O_CLOEXEC ),"output pipe" )) +return false; +if (!ASSERT_OK (pipe2 (ready,O_CLOEXEC ),"exec pipe" )) { +close (out[0 ]); +close (out[1 ]); +return false; + } + child->pid =fork (); +if (!child->pid) { +close (out[0 ]); +close (ready[0 ]); + err_fd = capture_errors ? out[1 ] :open ("/dev/null" ,O_WRONLY );
tools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c line 98
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpftool_ringbuf.c new file +10
-0+if (dup2 (out[1 ],STDOUT_FILENO ) <0 || +dup2 (err_fd,STDERR_FILENO ) <0 ) + goto exec_fail; +if (!capture_errors) +close (err_fd); +close (out[1 ]); +execv (path, argv); +exec_fail: + err = errno; +write (ready[1 ], &err,sizeof (err));
I think that ignoring the return value from write() can produce a compiler warning (-Wunused-result).
[...]
Looks OK otherwise, thank you
A message that is not in the archive (sent off-list or not yet received): e7211adda619bf43e17dbaf59ffec76ca2442726a5e22485cd783ed9a5e9c655@mail.kernel.org
Hi CI team,
Could you rerun the failing x86_64 GCC 15 configuration for series 1162498 and, if it fails again, compare the unpatched baseline?
The only failing subtest was rhash/test_rhash_iter: key_sum was 2553 instead of 2080, and elem_count was 78 instead of 64. All 17 bpftool_ringbuf subtests passed in that same job:
https://github.com/kernel-patches/bpf/actions/runs/34557186977/job/103134336101
The x86_64 GCC 15 ASAN job passed both RHASH and ring-buffer tests:
https://github.com/kernel-patches/bpf/actions/runs/34557186977/job/103134793523
This series changes bpftool, documentation and its selftests; it does not change the RHASH implementation or tests. For a local baseline check, I built test_progs from unpatched bpf-next and ran ./test_progs -t rhash/test_rhash_iter 200 times. The selected subtest passed all 200 runs without skips. This used an x86-64 KVM guest with a bpf/master Linux 7.3-rc2 kernel built with GCC 13 and LLVM 20 BPF objects, so it does not reproduce the CI toolchain or configuration. The RHASH implementation and two relevant test files match the CI pre-merge baseline.
Source inspection suggests deferred table resizing could lead the iterator to revisit entries, but I have not established that as this failure's cause. A rerun and baseline comparison would help distinguish an intermittent RHASH issue from a regression in this series.
Thanks,
Tianyi