Skip to contentSkip to conversation

libbpf: Make uprobe attachment APK aware

PATCH Superseded Daniel Müller 3 participants 16 messages Feb 17, 2023 – Mar 1, 2023

Subsystems (2)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

cover libbpf: Make uprobe attachment APK aware

On Android, APKs (android packages; zip packages with somewhat prescriptive contents) are first class citizens in the system: the shared objects contained in them don't exist in unpacked form on the file system. Rather, they are mmaped directly from within the archive and the archive is also what the kernel is aware of.

For users that complicates the process of attaching a uprobe to a function contained in a shared object in one such APK: they'd have to find the byte offset of said function from the beginning of the archive. That is cumbersome to do manually and can be fragile, because various changes could invalidate said offset.

That is why for uprobes inside ELF files (not inside an APK), commit d112c9ce249b ("libbpf: Support function name-based attach uprobes") added support for attaching to symbols by name. On Android, that mechanism currently does not work, because this logic is not APK aware.

This patch set introduces first class support for attaching uprobes to functions inside ELF objects contained in APKs via function names. We add support for recognizing the following syntax for a binary path:
<archive>!/<binary-in-archive>

(e.g., /system/app/test-app.apk!/lib/arm64-v8a/libc++.so)

This syntax is common in the Android eco system and used by tools such as simpleperf. It is also what is being proposed for bcc [0].

If the user provides such a binary path, we find <binary-in-archive> (lib/arm64-v8a/libc++.so in the example) inside of <archive> (/system/app/test-app.apk). We perform the regular ELF offset search inside the binary and add that to the offset within the archive itself, to retrieve the offset at which to attach the uprobe.

[0] https://github.com/iovisor/bcc/pull/4440

Daniel Müller (3):
libbpf: Implement basic zip archive parsing support
libbpf: Introduce elf_find_func_offset_from_elf_file() function
libbpf: Add support for attaching uprobes to shared objects in APKs

 tools/lib/bpf/Build    |   2 +-
 tools/lib/bpf/libbpf.c | 137 ++++++++++++---
 tools/lib/bpf/zip.c    | 371 +++++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/zip.h    |  47 ++++++
 4 files changed, 533 insertions(+), 24 deletions(-)
 create mode 100644 tools/lib/bpf/zip.c
 create mode 100644 tools/lib/bpf/zip.h
Signature · 1 line
2.30.2
#1 lore raw

PATCH 1/3 libbpf: Implement basic zip archive parsing support

This change implements support for reading zip archives, including opening an archive, finding an entry based on its path and name in it, and closing it.
The code was copied from https://github.com/iovisor/bcc/pull/4440, which implements similar functionality for bcc. The author confirmed that he is fine with this usage and the corresponding relicensing. I adjusted it to adhere to libbpf coding standards.

---
 tools/lib/bpf/Build |   2 +-
 tools/lib/bpf/zip.c | 378 ++++++++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/zip.h |  47 ++++++
 3 files changed, 426 insertions(+), 1 deletion(-)
 create mode 100644 tools/lib/bpf/zip.c
 create mode 100644 tools/lib/bpf/zip.h
tools/lib/bpf/Build +1 -1
index 5a3dfb..b8b0a63 100644
@@ -1,4 +1,4 @@
 libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \
 	    netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \
 	    btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \
-	    usdt.o
+	    usdt.o zip.o
tools/lib/bpf/zip.c new file +378 -0

378 lines not on this page · raw message

tools/lib/bpf/zip.h new file +47 -0

47 lines not on this page · raw message

Signature · 1 line
2.30.2
Acked-by Michał Gregorczyk on v1 1/3
#2 lore raw

Re: [PATCH bpf-next 1/3] libbpf: Implement basic zip archive parsing support

Quoted 8 lines · Daniel Müller

This change implements support for reading zip archives, including opening an archive, finding an entry based on its path and name in it, and closing it.
The code was copied from https://github.com/iovisor/bcc/pull/4440, which implements similar functionality for bcc. The author confirmed that he is fine with this usage and the corresponding relicensing. I adjusted it to adhere to libbpf coding standards.

Let's record this with

tag added next to your Signed-off-by. I've added Michal to cc for visibility.

Quoted 67 lines
---
 tools/lib/bpf/Build |   2 +-
 tools/lib/bpf/zip.c | 378 ++++++++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/zip.h |  47 ++++++
 3 files changed, 426 insertions(+), 1 deletion(-)
 create mode 100644 tools/lib/bpf/zip.c
 create mode 100644 tools/lib/bpf/zip.h
tools/lib/bpf/Build +1 -1
index 5a3dfb..b8b0a63 100644
@@ -1,4 +1,4 @@
 libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \
            netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \
            btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \
-           usdt.o
+           usdt.o zip.o
tools/lib/bpf/zip.c new file +41 -0
new file mode 100644
index 0000000..59ec79
@@ -0,0 +1,378 @@
+// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+/*
+ * Routines for dealing with .zip archives.
+ *
+ * Copyright (c) Meta Platforms, Inc. and affiliates.
+ */
+
+#include <fcntl.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <unistd.h>
+
+#include "zip.h"
+
+/* Specification of ZIP file format can be found here:
+ * https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT
+ * For a high level overview of the structure of a ZIP file see
+ * sections 4.3.1 - 4.3.6.
+ *
+ * Data structures appearing in ZIP files do not contain any
+ * padding and they might be misaligned. To allow us to safely
+ * operate on pointers to such structures and their members, without
+ * worrying of platform specific alignment issues, we define
+ * unaligned_uint16_t and unaligned_uint32_t types with no alignment
+ * requirements.
+ */
+typedef struct {
+       uint8_t raw[2];
+} unaligned_uint16_t;
+
+static uint16_t unaligned_uint16_read(unaligned_uint16_t value)
+{
+       uint16_t return_value;
+
+       memcpy(&return_value, value.raw, sizeof(return_value));
+       return return_value;
+}
+
+typedef struct {

tools/lib/bpf/zip.c line 51

Quoted 10 lines
tools/lib/bpf/zip.c new file +10 -0
+       uint8_t raw[4];
+} unaligned_uint32_t;
+
+static uint32_t unaligned_uint32_read(unaligned_uint32_t value)
+{
+       uint32_t return_value;
+
+       memcpy(&return_value, value.raw, sizeof(return_value));
+       return return_value;
+}

it feels like an overkill to use this unaligned_xxx_t approach. Let's use __attribute__((packed)) on corresponding structs and just uint16_t/uint32_t field types.

Quoted 19 lines
tools/lib/bpf/zip.c new file +19 -0
+
+#define END_OF_CD_RECORD_MAGIC 0x06054b50
+
+/* See section 4.3.16 of the spec. */
+struct end_of_central_directory_record {
+       /* Magic value equal to END_OF_CD_RECORD_MAGIC */
+       unaligned_uint32_t magic;
+
+       /* Number of the file containing this structure or 0xFFFF if ZIP64 archive.
+        * Zip archive might span multiple files (disks).
+        */
+       unaligned_uint16_t this_disk;
+
+       /* Number of the file containing the beginning of the central directory or
+        * 0xFFFF if ZIP64 archive.
+        */
+       unaligned_uint16_t cd_disk;
+
+       /* Number of central directory records on this disk or 0xFFFF if ZIP64

tools/lib/bpf/zip.c line 80

Quoted 10 lines
tools/lib/bpf/zip.c new file +10 -0
+        * archive.
+        */
+       unaligned_uint16_t cd_records;
+
+       /* Number of central directory records on all disks or 0xFFFF if ZIP64
+        * archive.
+        */
+       unaligned_uint16_t cd_records_total;
+
+       /* Size of the central directory recrod or 0xFFFFFFFF if ZIP64 archive. */

typo: record

tools/lib/bpf/zip.c line 88

Quoted 8 lines
tools/lib/bpf/zip.c new file +8 -0
+       unaligned_uint32_t cd_size;
+
+       /* Offset of the central directory from the beginning of the archive or
+        * 0xFFFFFFFF if ZIP64 archive.
+        */
+       unaligned_uint32_t cd_offset;
+
+       /* Length of comment data following end of central driectory record. */

typo: directory

tools/lib/bpf/zip.c line 100

Quoted 12 lines
tools/lib/bpf/zip.c new file +12 -0
+       unaligned_uint16_t comment_length;
+
+       /* Up to 64k of arbitrary bytes. */
+       /* uint8_t comment[comment_length] */
+};
+
+#define CD_FILE_HEADER_MAGIC 0x02014b50
+#define FLAG_ENCRYPTED (1 << 0)
+#define FLAG_HAS_DATA_DESCRIPTOR (1 << 3)
+
+/* See section 4.3.12 of the spec. */
+struct central_directory_file_header {

naming nit here and below: we use CD in constants, but spell out "central_directory", which feels a bit mouthful. Let's use "cd" consistently throughout (even if it's a bit confusing with "change directory", but I think in the context of parsing ZIP contents should be recognizable enough)

Quoted 41 lines
tools/lib/bpf/zip.c new file +41 -0
+       /* Magic value equal to CD_FILE_HEADER_MAGIC. */
+       unaligned_uint32_t magic;
+       unaligned_uint16_t version;
+       /* Minimum zip version needed to extract the file. */
+       unaligned_uint16_t min_version;
+       unaligned_uint16_t flags;
+       unaligned_uint16_t compression;
+       unaligned_uint16_t last_modified_time;
+       unaligned_uint16_t last_modified_date;
+       unaligned_uint32_t crc;
+       unaligned_uint32_t compressed_size;
+       unaligned_uint32_t uncompressed_size;
+       unaligned_uint16_t file_name_length;
+       unaligned_uint16_t extra_field_length;
+       unaligned_uint16_t file_comment_length;
+       /* Number of the disk where the file starts or 0xFFFF if ZIP64 archive. */
+       unaligned_uint16_t disk;
+       unaligned_uint16_t internal_attributes;
+       unaligned_uint32_t external_attributes;
+       /* Offset from the start of the disk containing the local file header to the
+        * start of the local file header.
+        */
+       unaligned_uint32_t offset;
+};
+
+#define LOCAL_FILE_HEADER_MAGIC 0x04034b50
+
+/* See section 4.3.7 of the spec. */
+struct local_file_header {
+       /* Magic value equal to LOCAL_FILE_HEADER_MAGIC. */
+       unaligned_uint32_t magic;
+       /* Minimum zip version needed to extract the file. */
+       unaligned_uint16_t min_version;
+       unaligned_uint16_t flags;
+       unaligned_uint16_t compression;
+       unaligned_uint16_t last_modified_time;
+       unaligned_uint16_t last_modified_date;
+       unaligned_uint32_t crc;
+       unaligned_uint32_t compressed_size;
+       unaligned_uint32_t uncompressed_size;
+       unaligned_uint16_t file_name_length;

tools/lib/bpf/zip.c line 151

Quoted 10 lines
tools/lib/bpf/zip.c new file +10 -0
+       unaligned_uint16_t extra_field_length;
+};
+
+struct zip_archive {
+       void *data;
+       uint32_t size;
+       uint32_t cd_offset;
+       uint32_t cd_records;
+};
+
$ rg -w 'uint\d\d_t' | wc -l
21
$ rg -w '__u\d\d' | wc -l
873

seems like we overwhelmingly prefer __u32/__u16 in libbpf code base, let's use those instead?

tools/lib/bpf/zip.c line 160

Quoted 9 lines
tools/lib/bpf/zip.c new file +9 -0
+static void *check_access(struct zip_archive *archive, uint32_t offset, uint32_t size)
+{
+       if (offset + size > archive->size || offset > offset + size) {
+               return NULL;
+       }
+       return archive->data + offset;
+}
+
+/* Returns 0 on success, -1 on error and -2 if the eocd indicates

let's use -EINVAL and -ENOTSUP instead of -1 and -2

Quoted 56 lines
tools/lib/bpf/zip.c new file +56 -0
+ * the archive uses features which are not supported.
+ */
+static int try_parse_end_of_central_directory(struct zip_archive *archive, uint32_t offset)
+{
+       struct end_of_central_directory_record *eocd =
+               check_access(archive, offset, sizeof(struct end_of_central_directory_record));
+       uint16_t comment_length, cd_records;
+       uint32_t cd_offset, cd_size;
+
+       if (!eocd || unaligned_uint32_read(eocd->magic) != END_OF_CD_RECORD_MAGIC) {
+               return -1;
+       }
+
+       comment_length = unaligned_uint16_read(eocd->comment_length);
+       if (offset + sizeof(struct end_of_central_directory_record) + comment_length !=
+           archive->size) {
+               return -1;
+       }
+
+       cd_records = unaligned_uint16_read(eocd->cd_records);
+       if (unaligned_uint16_read(eocd->this_disk) != 0 ||
+           unaligned_uint16_read(eocd->cd_disk) != 0 ||
+           unaligned_uint16_read(eocd->cd_records_total) != cd_records) {
+               /* This is a valid eocd, but we only support single-file non-ZIP64 archives. */
+               return -2;
+       }
+
+       cd_offset = unaligned_uint32_read(eocd->cd_offset);
+       cd_size = unaligned_uint32_read(eocd->cd_size);
+       if (!check_access(archive, cd_offset, cd_size)) {
+               return -1;
+       }
+
+       archive->cd_offset = cd_offset;
+       archive->cd_records = cd_records;
+       return 0;
+}
+
+static int find_central_directory(struct zip_archive *archive)
+{
+       uint32_t offset;
+       int64_t limit;
+       int rc = -1;
+
+       if (archive->size <= sizeof(struct end_of_central_directory_record)) {
+               return -1;
+       }
+
+       /* Because the end of central directory ends with a variable length array of
+        * up to 0xFFFF bytes we can't know exactly where it starts and need to
+        * search for it at the end of the file, scanning the (limit, offset] range.
+        */
+       offset = archive->size - sizeof(struct end_of_central_directory_record);
+       limit = (int64_t)offset - (1 << 16);
+
+       for (; offset >= 0 && offset > limit && rc == -1; offset--) {

tools/lib/bpf/zip.c line 226

Quoted 10 lines
tools/lib/bpf/zip.c new file +10 -0
+               rc = try_parse_end_of_central_directory(archive, offset);
+       }
+
+       return rc;
+}
+
+struct zip_archive *zip_archive_open(const char *path)
+{
+       struct zip_archive *archive;
+       int fd = open(path, O_RDONLY);

let's not do complicated operations during variable initialization, let's just init fd right before checking it for <0

tools/lib/bpf/zip.c line 232

Quoted 6 lines
tools/lib/bpf/zip.c new file +6 -0
+       off_t size;
+       void *data;
+
+       if (fd < 0) {
+               return NULL;
+       }

no {} for single-line if statement

tools/lib/bpf/zip.c line 245

Quoted 13 lines
tools/lib/bpf/zip.c new file +13 -0
+
+       size = lseek(fd, 0, SEEK_END);
+       if (size == (off_t)-1 || size > UINT32_MAX) {
+               close(fd);
+               return NULL;
+       }
+
+       data = mmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0);
+       close(fd);
+
+       if (data == MAP_FAILED) {
+               return NULL;
+       }

ditto about {}

tools/lib/bpf/zip.c line 258

Quoted 13 lines
tools/lib/bpf/zip.c new file +13 -0
+
+       archive = malloc(sizeof(struct zip_archive));
+       if (!archive) {
+               munmap(data, size);
+               return NULL;
+       };
+
+       archive->data = data;
+       archive->size = size;
+       if (find_central_directory(archive)) {
+               munmap(data, size);
+               free(archive);
+               archive = NULL;

return NULL; ?

tools/lib/bpf/zip.c line 271

Quoted 13 lines
tools/lib/bpf/zip.c new file +13 -0
+       }
+
+       return archive;
+}
+
+void zip_archive_close(struct zip_archive *archive)
+{
+       munmap(archive->data, archive->size);
+       free(archive);
+}
+
+static struct local_file_header *local_file_header_at_offset(struct zip_archive *archive,
+                                                            uint32_t offset)

is there a non-local file in ZIP archive? Seems a bit too verbose, just "file_header_at_offset"? and seeing "get_entry_at_offset" below, should it be "get_file_header_at_offset" then?

tools/lib/bpf/zip.c line 274

Quoted 3 lines
tools/lib/bpf/zip.c new file +3 -0
+{
+       struct local_file_header *lfh =
+               check_access(archive, offset, sizeof(struct local_file_header));

empty line between variables and code

tools/lib/bpf/zip.c line 277

Quoted 3 lines
tools/lib/bpf/zip.c new file +3 -0
+       if (!lfh || unaligned_uint32_read(lfh->magic) != LOCAL_FILE_HEADER_MAGIC) {
+               return NULL;
+       }

here and below, same {} issue

tools/lib/bpf/zip.c line 283

Quoted 6 lines
tools/lib/bpf/zip.c new file +6 -0
+       return lfh;
+}
+
+static int get_entry_at_offset(struct zip_archive *archive, uint32_t offset, struct zip_entry *out)
+{
+       struct local_file_header *lfh = local_file_header_at_offset(archive, offset);

same as above, let's not do extensive operations during variable initialization, please split

tools/lib/bpf/zip.c line 289

Quoted 6 lines
tools/lib/bpf/zip.c new file +6 -0
+       uint16_t flags, name_length, extra_field_length;
+       uint32_t compressed_size;
+       const char *name;
+       void *data;
+
+       offset += sizeof(struct local_file_header);

logically this probably should happen after we checked lfh for NULL?

tools/lib/bpf/zip.c line 291

Quoted 2 lines
tools/lib/bpf/zip.c new file +2 -0
+       if (!lfh) {
+               return -1;

here and everywhere, let's use kernel error code constants, e.g., -EINVAL

Quoted 11 lines
tools/lib/bpf/zip.c new file +11 -0
+       };
+
+       flags = unaligned_uint16_read(lfh->flags);
+       if ((flags & FLAG_ENCRYPTED) || (flags & FLAG_HAS_DATA_DESCRIPTOR)) {
+               return -1;
+       }
+
+       name_length = unaligned_uint16_read(lfh->file_name_length);
+       name = check_access(archive, offset, name_length);
+       offset += name_length;
+       if (!name) {

tools/lib/bpf/zip.c line 312

Quoted 10 lines
tools/lib/bpf/zip.c new file +10 -0
+               return -1;
+       }
+
+       extra_field_length = unaligned_uint16_read(lfh->extra_field_length);
+       if (!check_access(archive, offset, extra_field_length)) {
+               return -1;
+       }
+       offset += extra_field_length;
+
+       compressed_size = unaligned_uint32_read(lfh->compressed_size);

why not fill out out->data_length instead of all these local variables?

Quoted 7 lines
tools/lib/bpf/zip.c new file +7 -0
+       data = check_access(archive, offset, compressed_size);
+       if (!data) {
+               return -1;
+       }
+
+       out->compression = unaligned_uint16_read(lfh->compression);
+       out->name_length = name_length;

tools/lib/bpf/zip.c line 329

Quoted 10 lines
tools/lib/bpf/zip.c new file +10 -0
+       out->name = name;
+       out->data = data;
+       out->data_length = compressed_size;
+       out->data_offset = offset;
+
+       return 0;
+}
+
+static struct central_directory_file_header *cd_file_header_at_offset(struct zip_archive *archive,
+                                                                     uint32_t offset)

this function is called just once below, why a helper?

tools/lib/bpf/zip.c line 332

Quoted 3 lines
tools/lib/bpf/zip.c new file +3 -0
+{
+       struct central_directory_file_header *cdfh =
+               check_access(archive, offset, sizeof(struct central_directory_file_header));

empty line

tools/lib/bpf/zip.c line 343

Quoted 11 lines
tools/lib/bpf/zip.c new file +11 -0
+       if (!cdfh || unaligned_uint32_read(cdfh->magic) != CD_FILE_HEADER_MAGIC) {
+               return NULL;
+       }
+       return cdfh;
+}
+
+int zip_archive_find_entry(struct zip_archive *archive, const char *file_name,
+                          struct zip_entry *out)
+{
+       size_t file_name_length = strlen(file_name);
+

no need for empty line, let's keep variable declarations in one block

tools/lib/bpf/zip.c line 352

Quoted 9 lines
tools/lib/bpf/zip.c new file +9 -0
+       uint32_t i, offset = archive->cd_offset;
+
+       for (i = 0; i < archive->cd_records; ++i) {
+               struct central_directory_file_header *cdfh =
+                       cd_file_header_at_offset(archive, offset);
+               uint16_t cdfh_name_length, cdfh_flags;
+               const char *cdfh_name;
+
+               offset += sizeof(struct central_directory_file_header);

same, logically offset should be updated after we made sure we have cd_file_header

Quoted 32 lines
tools/lib/bpf/zip.c new file +26 -0
+               if (!cdfh) {
+                       return -1;
+               }
+
+               cdfh_name_length = unaligned_uint16_read(cdfh->file_name_length);
+               cdfh_name = check_access(archive, offset, cdfh_name_length);
+               if (!cdfh_name) {
+                       return -1;
+               }
+
+               cdfh_flags = unaligned_uint16_read(cdfh->flags);
+               if ((cdfh_flags & FLAG_ENCRYPTED) == 0 &&
+                   (cdfh_flags & FLAG_HAS_DATA_DESCRIPTOR) == 0 &&
+                   file_name_length == cdfh_name_length &&
+                   memcmp(file_name, archive->data + offset, file_name_length) == 0) {
+                       return get_entry_at_offset(archive, unaligned_uint32_read(cdfh->offset),
+                                                  out);
+               }
+
+               offset += cdfh_name_length;
+               offset += unaligned_uint16_read(cdfh->extra_field_length);
+               offset += unaligned_uint16_read(cdfh->file_comment_length);
+       }
+
+       return -1;
+}
tools/lib/bpf/zip.h new file +0 -0
new file mode 100644
index 0000000..a9083f

tools/lib/bpf/zip.h line 8

Quoted 8 lines
tools/lib/bpf/zip.h new file +8 -0
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
+
+#ifndef __LIBBPF_ZIP_H
+#define __LIBBPF_ZIP_H
+
+#include <stdint.h>
+
+/* Represents an opened zip archive.

s/opened/open/

Quoted 42 lines
tools/lib/bpf/zip.h new file +39 -1
+ * Only basic ZIP files are supported, in particular the following are not
+ * supported:
+ * - encryption
+ * - streaming
+ * - multi-part ZIP files
+ * - ZIP64
+ */
+struct zip_archive;
+
+/* Carries information on name, compression method, and data corresponding to a
+ * file in a zip archive.
+ */
+struct zip_entry {
+       /* Compression method as defined in pkzip spec. 0 means data is uncompressed. */
+       uint16_t compression;
+
+       /* Non-null terminated name of the file. */
+       const char *name;
+       /* Length of the file name. */
+       uint16_t name_length;
+
+       /* Pointer to the file data. */
+       const void *data;
+       /* Length of the file data. */
+       uint32_t data_length;
+       /* Offset of the file data within the archive. */
+       uint32_t data_offset;
+};
+
+/* Open a zip archive. Returns NULL in case of an error. */
+struct zip_archive *zip_archive_open(const char *path);
+
+/* Close a zip archive and release resources. */
+void zip_archive_close(struct zip_archive *archive);
+
+/* Look up an entry corresponding to a file in given zip archive. */
+int zip_archive_find_entry(struct zip_archive *archive, const char *name, struct zip_entry *out);
+
+#endif
--

2.30.2

#3 lore raw

Re: [PATCH bpf-next 1/3] libbpf: Implement basic zip archive parsing support

Quoted 153 lines · Andrii Nakryiko
Quoted 8 lines · Daniel Müller

This change implements support for reading zip archives, including opening an archive, finding an entry based on its path and name in it, and closing it.
The code was copied from https://github.com/iovisor/bcc/pull/4440, which implements similar functionality for bcc. The author confirmed that he is fine with this usage and the corresponding relicensing. I adjusted it to adhere to libbpf coding standards.

Let's record this with

tag added next to your Signed-off-by. I've added Michal to cc for visibility.

Quoted 67 lines
---
 tools/lib/bpf/Build |   2 +-
 tools/lib/bpf/zip.c | 378 ++++++++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/zip.h |  47 ++++++
 3 files changed, 426 insertions(+), 1 deletion(-)
 create mode 100644 tools/lib/bpf/zip.c
 create mode 100644 tools/lib/bpf/zip.h
tools/lib/bpf/Build +1 -1
index 5a3dfb..b8b0a63 100644
@@ -1,4 +1,4 @@
 libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \
            netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \
            btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \
-           usdt.o
+           usdt.o zip.o
tools/lib/bpf/zip.c new file +41 -0
new file mode 100644
index 0000000..59ec79
@@ -0,0 +1,378 @@
+// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+/*
+ * Routines for dealing with .zip archives.
+ *
+ * Copyright (c) Meta Platforms, Inc. and affiliates.
+ */
+
+#include <fcntl.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <unistd.h>
+
+#include "zip.h"
+
+/* Specification of ZIP file format can be found here:
+ * https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT
+ * For a high level overview of the structure of a ZIP file see
+ * sections 4.3.1 - 4.3.6.
+ *
+ * Data structures appearing in ZIP files do not contain any
+ * padding and they might be misaligned. To allow us to safely
+ * operate on pointers to such structures and their members, without
+ * worrying of platform specific alignment issues, we define
+ * unaligned_uint16_t and unaligned_uint32_t types with no alignment
+ * requirements.
+ */
+typedef struct {
+       uint8_t raw[2];
+} unaligned_uint16_t;
+
+static uint16_t unaligned_uint16_read(unaligned_uint16_t value)
+{
+       uint16_t return_value;
+
+       memcpy(&return_value, value.raw, sizeof(return_value));
+       return return_value;
+}
+
+typedef struct {

tools/lib/bpf/zip.c line 51

Quoted 10 lines
tools/lib/bpf/zip.c new file +10 -0
+       uint8_t raw[4];
+} unaligned_uint32_t;
+
+static uint32_t unaligned_uint32_read(unaligned_uint32_t value)
+{
+       uint32_t return_value;
+
+       memcpy(&return_value, value.raw, sizeof(return_value));
+       return return_value;
+}

it feels like an overkill to use this unaligned_xxx_t approach. Let's use __attribute__((packed)) on corresponding structs and just uint16_t/uint32_t field types.

Quoted 19 lines
tools/lib/bpf/zip.c new file +19 -0
+
+#define END_OF_CD_RECORD_MAGIC 0x06054b50
+
+/* See section 4.3.16 of the spec. */
+struct end_of_central_directory_record {
+       /* Magic value equal to END_OF_CD_RECORD_MAGIC */
+       unaligned_uint32_t magic;
+
+       /* Number of the file containing this structure or 0xFFFF if ZIP64 archive.
+        * Zip archive might span multiple files (disks).
+        */
+       unaligned_uint16_t this_disk;
+
+       /* Number of the file containing the beginning of the central directory or
+        * 0xFFFF if ZIP64 archive.
+        */
+       unaligned_uint16_t cd_disk;
+
+       /* Number of central directory records on this disk or 0xFFFF if ZIP64

tools/lib/bpf/zip.c line 80

Quoted 10 lines
tools/lib/bpf/zip.c new file +10 -0
+        * archive.
+        */
+       unaligned_uint16_t cd_records;
+
+       /* Number of central directory records on all disks or 0xFFFF if ZIP64
+        * archive.
+        */
+       unaligned_uint16_t cd_records_total;
+
+       /* Size of the central directory recrod or 0xFFFFFFFF if ZIP64 archive. */

typo: record

tools/lib/bpf/zip.c line 88

Quoted 8 lines
tools/lib/bpf/zip.c new file +8 -0
+       unaligned_uint32_t cd_size;
+
+       /* Offset of the central directory from the beginning of the archive or
+        * 0xFFFFFFFF if ZIP64 archive.
+        */
+       unaligned_uint32_t cd_offset;
+
+       /* Length of comment data following end of central driectory record. */

typo: directory

Quoted 11 lines
tools/lib/bpf/zip.c new file +11 -0
+       unaligned_uint16_t comment_length;
+
+       /* Up to 64k of arbitrary bytes. */
+       /* uint8_t comment[comment_length] */
+};
+
+#define CD_FILE_HEADER_MAGIC 0x02014b50
+#define FLAG_ENCRYPTED (1 << 0)
+#define FLAG_HAS_DATA_DESCRIPTOR (1 << 3)
+
+/* See section 4.3.12 of the spec. */
Quoted 8 lines

tools/lib/bpf/zip.c line 100

Quoted 1 line
tools/lib/bpf/zip.c new file +1 -0
+struct central_directory_file_header {

naming nit here and below: we use CD in constants, but spell out "central_directory", which feels a bit mouthful. Let's use "cd" consistently throughout (even if it's a bit confusing with "change directory", but I think in the context of parsing ZIP contents should be recognizable enough)

Sure.

Quoted 57 lines
Quoted 41 lines
tools/lib/bpf/zip.c new file +41 -0
+       /* Magic value equal to CD_FILE_HEADER_MAGIC. */
+       unaligned_uint32_t magic;
+       unaligned_uint16_t version;
+       /* Minimum zip version needed to extract the file. */
+       unaligned_uint16_t min_version;
+       unaligned_uint16_t flags;
+       unaligned_uint16_t compression;
+       unaligned_uint16_t last_modified_time;
+       unaligned_uint16_t last_modified_date;
+       unaligned_uint32_t crc;
+       unaligned_uint32_t compressed_size;
+       unaligned_uint32_t uncompressed_size;
+       unaligned_uint16_t file_name_length;
+       unaligned_uint16_t extra_field_length;
+       unaligned_uint16_t file_comment_length;
+       /* Number of the disk where the file starts or 0xFFFF if ZIP64 archive. */
+       unaligned_uint16_t disk;
+       unaligned_uint16_t internal_attributes;
+       unaligned_uint32_t external_attributes;
+       /* Offset from the start of the disk containing the local file header to the
+        * start of the local file header.
+        */
+       unaligned_uint32_t offset;
+};
+
+#define LOCAL_FILE_HEADER_MAGIC 0x04034b50
+
+/* See section 4.3.7 of the spec. */
+struct local_file_header {
+       /* Magic value equal to LOCAL_FILE_HEADER_MAGIC. */
+       unaligned_uint32_t magic;
+       /* Minimum zip version needed to extract the file. */
+       unaligned_uint16_t min_version;
+       unaligned_uint16_t flags;
+       unaligned_uint16_t compression;
+       unaligned_uint16_t last_modified_time;
+       unaligned_uint16_t last_modified_date;
+       unaligned_uint32_t crc;
+       unaligned_uint32_t compressed_size;
+       unaligned_uint32_t uncompressed_size;
+       unaligned_uint16_t file_name_length;

tools/lib/bpf/zip.c line 151

Quoted 10 lines
tools/lib/bpf/zip.c new file +10 -0
+       unaligned_uint16_t extra_field_length;
+};
+
+struct zip_archive {
+       void *data;
+       uint32_t size;
+       uint32_t cd_offset;
+       uint32_t cd_records;
+};
+
$ rg -w 'uint\d\d_t' | wc -l
21
$ rg -w '__u\d\d' | wc -l
873
Quoted 2 lines

seems like we overwhelmingly prefer __u32/__u16 in libbpf code base, let's use those instead?

Sure.

Quoted 11 lines

tools/lib/bpf/zip.c line 160

Quoted 9 lines
tools/lib/bpf/zip.c new file +9 -0
+static void *check_access(struct zip_archive *archive, uint32_t offset, uint32_t size)
+{
+       if (offset + size > archive->size || offset > offset + size) {
+               return NULL;
+       }
+       return archive->data + offset;
+}
+
+/* Returns 0 on success, -1 on error and -2 if the eocd indicates

let's use -EINVAL and -ENOTSUP instead of -1 and -2

Sure.

Quoted 120 lines
Quoted 56 lines
tools/lib/bpf/zip.c new file +56 -0
+ * the archive uses features which are not supported.
+ */
+static int try_parse_end_of_central_directory(struct zip_archive *archive, uint32_t offset)
+{
+       struct end_of_central_directory_record *eocd =
+               check_access(archive, offset, sizeof(struct end_of_central_directory_record));
+       uint16_t comment_length, cd_records;
+       uint32_t cd_offset, cd_size;
+
+       if (!eocd || unaligned_uint32_read(eocd->magic) != END_OF_CD_RECORD_MAGIC) {
+               return -1;
+       }
+
+       comment_length = unaligned_uint16_read(eocd->comment_length);
+       if (offset + sizeof(struct end_of_central_directory_record) + comment_length !=
+           archive->size) {
+               return -1;
+       }
+
+       cd_records = unaligned_uint16_read(eocd->cd_records);
+       if (unaligned_uint16_read(eocd->this_disk) != 0 ||
+           unaligned_uint16_read(eocd->cd_disk) != 0 ||
+           unaligned_uint16_read(eocd->cd_records_total) != cd_records) {
+               /* This is a valid eocd, but we only support single-file non-ZIP64 archives. */
+               return -2;
+       }
+
+       cd_offset = unaligned_uint32_read(eocd->cd_offset);
+       cd_size = unaligned_uint32_read(eocd->cd_size);
+       if (!check_access(archive, cd_offset, cd_size)) {
+               return -1;
+       }
+
+       archive->cd_offset = cd_offset;
+       archive->cd_records = cd_records;
+       return 0;
+}
+
+static int find_central_directory(struct zip_archive *archive)
+{
+       uint32_t offset;
+       int64_t limit;
+       int rc = -1;
+
+       if (archive->size <= sizeof(struct end_of_central_directory_record)) {
+               return -1;
+       }
+
+       /* Because the end of central directory ends with a variable length array of
+        * up to 0xFFFF bytes we can't know exactly where it starts and need to
+        * search for it at the end of the file, scanning the (limit, offset] range.
+        */
+       offset = archive->size - sizeof(struct end_of_central_directory_record);
+       limit = (int64_t)offset - (1 << 16);
+
+       for (; offset >= 0 && offset > limit && rc == -1; offset--) {

tools/lib/bpf/zip.c line 226

Quoted 10 lines
tools/lib/bpf/zip.c new file +10 -0
+               rc = try_parse_end_of_central_directory(archive, offset);
+       }
+
+       return rc;
+}
+
+struct zip_archive *zip_archive_open(const char *path)
+{
+       struct zip_archive *archive;
+       int fd = open(path, O_RDONLY);

let's not do complicated operations during variable initialization, let's just init fd right before checking it for <0

tools/lib/bpf/zip.c line 232

Quoted 6 lines
tools/lib/bpf/zip.c new file +6 -0
+       off_t size;
+       void *data;
+
+       if (fd < 0) {
+               return NULL;
+       }

no {} for single-line if statement

tools/lib/bpf/zip.c line 245

Quoted 13 lines
tools/lib/bpf/zip.c new file +13 -0
+
+       size = lseek(fd, 0, SEEK_END);
+       if (size == (off_t)-1 || size > UINT32_MAX) {
+               close(fd);
+               return NULL;
+       }
+
+       data = mmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0);
+       close(fd);
+
+       if (data == MAP_FAILED) {
+               return NULL;
+       }

ditto about {}

tools/lib/bpf/zip.c line 258

Quoted 13 lines
tools/lib/bpf/zip.c new file +13 -0
+
+       archive = malloc(sizeof(struct zip_archive));
+       if (!archive) {
+               munmap(data, size);
+               return NULL;
+       };
+
+       archive->data = data;
+       archive->size = size;
+       if (find_central_directory(archive)) {
+               munmap(data, size);
+               free(archive);
+               archive = NULL;

return NULL; ?

Quoted 9 lines
tools/lib/bpf/zip.c new file +9 -0
+       }
+
+       return archive;
+}
+
+void zip_archive_close(struct zip_archive *archive)
+{
+       munmap(archive->data, archive->size);
+       free(archive);
Quoted 8 lines

tools/lib/bpf/zip.c line 271

Quoted 4 lines
tools/lib/bpf/zip.c new file +4 -0
+}
+
+static struct local_file_header *local_file_header_at_offset(struct zip_archive *archive,
+                                                            uint32_t offset)

is there a non-local file in ZIP archive? Seems a bit too verbose, just "file_header_at_offset"? and seeing "get_entry_at_offset" below, should it be "get_file_header_at_offset" then?

It's called that by the standard. See section 4.3.7 Local file header in
https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT

Quoted 22 lines

tools/lib/bpf/zip.c line 274

Quoted 3 lines
tools/lib/bpf/zip.c new file +3 -0
+{
+       struct local_file_header *lfh =
+               check_access(archive, offset, sizeof(struct local_file_header));

empty line between variables and code

tools/lib/bpf/zip.c line 277

Quoted 3 lines
tools/lib/bpf/zip.c new file +3 -0
+       if (!lfh || unaligned_uint32_read(lfh->magic) != LOCAL_FILE_HEADER_MAGIC) {
+               return NULL;
+       }

here and below, same {} issue

tools/lib/bpf/zip.c line 283

Quoted 6 lines
tools/lib/bpf/zip.c new file +6 -0
+       return lfh;
+}
+
+static int get_entry_at_offset(struct zip_archive *archive, uint32_t offset, struct zip_entry *out)
+{
+       struct local_file_header *lfh = local_file_header_at_offset(archive, offset);

same as above, let's not do extensive operations during variable initialization, please split

Quoted 8 lines

tools/lib/bpf/zip.c line 289

Quoted 6 lines
tools/lib/bpf/zip.c new file +6 -0
+       uint16_t flags, name_length, extra_field_length;
+       uint32_t compressed_size;
+       const char *name;
+       void *data;
+
+       offset += sizeof(struct local_file_header);

logically this probably should happen after we checked lfh for NULL?

Sure.

Quoted 4 lines

tools/lib/bpf/zip.c line 291

Quoted 2 lines
tools/lib/bpf/zip.c new file +2 -0
+       if (!lfh) {
+               return -1;

here and everywhere, let's use kernel error code constants, e.g., -EINVAL

Sure.

Quoted 15 lines
Quoted 15 lines
tools/lib/bpf/zip.c new file +15 -0
+       };
+
+       flags = unaligned_uint16_read(lfh->flags);
+       if ((flags & FLAG_ENCRYPTED) || (flags & FLAG_HAS_DATA_DESCRIPTOR)) {
+               return -1;
+       }
+
+       name_length = unaligned_uint16_read(lfh->file_name_length);
+       name = check_access(archive, offset, name_length);
+       offset += name_length;
+       if (!name) {
+               return -1;
+       }
+
+       extra_field_length = unaligned_uint16_read(lfh->extra_field_length);
Quoted 8 lines

tools/lib/bpf/zip.c line 312

Quoted 6 lines
tools/lib/bpf/zip.c new file +6 -0
+       if (!check_access(archive, offset, extra_field_length)) {
+               return -1;
+       }
+       offset += extra_field_length;
+
+       compressed_size = unaligned_uint32_read(lfh->compressed_size);

why not fill out out->data_length instead of all these local variables?

Because the function contract is that 'out' is left untouched on error.

Quoted 11 lines
Quoted 11 lines
tools/lib/bpf/zip.c new file +11 -0
+       data = check_access(archive, offset, compressed_size);
+       if (!data) {
+               return -1;
+       }
+
+       out->compression = unaligned_uint16_read(lfh->compression);
+       out->name_length = name_length;
+       out->name = name;
+       out->data = data;
+       out->data_length = compressed_size;
+       out->data_offset = offset;
Quoted 8 lines

tools/lib/bpf/zip.c line 329

Quoted 6 lines
tools/lib/bpf/zip.c new file +6 -0
+
+       return 0;
+}
+
+static struct central_directory_file_header *cd_file_header_at_offset(struct zip_archive *archive,
+                                                                     uint32_t offset)

this function is called just once below, why a helper?

Will inline it.

Quoted 119 lines

tools/lib/bpf/zip.c line 332

Quoted 3 lines
tools/lib/bpf/zip.c new file +3 -0
+{
+       struct central_directory_file_header *cdfh =
+               check_access(archive, offset, sizeof(struct central_directory_file_header));

empty line

tools/lib/bpf/zip.c line 343

Quoted 11 lines
tools/lib/bpf/zip.c new file +11 -0
+       if (!cdfh || unaligned_uint32_read(cdfh->magic) != CD_FILE_HEADER_MAGIC) {
+               return NULL;
+       }
+       return cdfh;
+}
+
+int zip_archive_find_entry(struct zip_archive *archive, const char *file_name,
+                          struct zip_entry *out)
+{
+       size_t file_name_length = strlen(file_name);
+

no need for empty line, let's keep variable declarations in one block

tools/lib/bpf/zip.c line 352

Quoted 9 lines
tools/lib/bpf/zip.c new file +9 -0
+       uint32_t i, offset = archive->cd_offset;
+
+       for (i = 0; i < archive->cd_records; ++i) {
+               struct central_directory_file_header *cdfh =
+                       cd_file_header_at_offset(archive, offset);
+               uint16_t cdfh_name_length, cdfh_flags;
+               const char *cdfh_name;
+
+               offset += sizeof(struct central_directory_file_header);

same, logically offset should be updated after we made sure we have cd_file_header

Quoted 32 lines
tools/lib/bpf/zip.c new file +26 -0
+               if (!cdfh) {
+                       return -1;
+               }
+
+               cdfh_name_length = unaligned_uint16_read(cdfh->file_name_length);
+               cdfh_name = check_access(archive, offset, cdfh_name_length);
+               if (!cdfh_name) {
+                       return -1;
+               }
+
+               cdfh_flags = unaligned_uint16_read(cdfh->flags);
+               if ((cdfh_flags & FLAG_ENCRYPTED) == 0 &&
+                   (cdfh_flags & FLAG_HAS_DATA_DESCRIPTOR) == 0 &&
+                   file_name_length == cdfh_name_length &&
+                   memcmp(file_name, archive->data + offset, file_name_length) == 0) {
+                       return get_entry_at_offset(archive, unaligned_uint32_read(cdfh->offset),
+                                                  out);
+               }
+
+               offset += cdfh_name_length;
+               offset += unaligned_uint16_read(cdfh->extra_field_length);
+               offset += unaligned_uint16_read(cdfh->file_comment_length);
+       }
+
+       return -1;
+}
tools/lib/bpf/zip.h new file +0 -0
new file mode 100644
index 0000000..a9083f

tools/lib/bpf/zip.h line 8

Quoted 8 lines
tools/lib/bpf/zip.h new file +8 -0
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
+
+#ifndef __LIBBPF_ZIP_H
+#define __LIBBPF_ZIP_H
+
+#include <stdint.h>
+
+/* Represents an opened zip archive.

s/opened/open/

Quoted 42 lines
tools/lib/bpf/zip.h new file +39 -1
+ * Only basic ZIP files are supported, in particular the following are not
+ * supported:
+ * - encryption
+ * - streaming
+ * - multi-part ZIP files
+ * - ZIP64
+ */
+struct zip_archive;
+
+/* Carries information on name, compression method, and data corresponding to a
+ * file in a zip archive.
+ */
+struct zip_entry {
+       /* Compression method as defined in pkzip spec. 0 means data is uncompressed. */
+       uint16_t compression;
+
+       /* Non-null terminated name of the file. */
+       const char *name;
+       /* Length of the file name. */
+       uint16_t name_length;
+
+       /* Pointer to the file data. */
+       const void *data;
+       /* Length of the file data. */
+       uint32_t data_length;
+       /* Offset of the file data within the archive. */
+       uint32_t data_offset;
+};
+
+/* Open a zip archive. Returns NULL in case of an error. */
+struct zip_archive *zip_archive_open(const char *path);
+
+/* Close a zip archive and release resources. */
+void zip_archive_close(struct zip_archive *archive);
+
+/* Look up an entry corresponding to a file in given zip archive. */
+int zip_archive_find_entry(struct zip_archive *archive, const char *name, struct zip_entry *out);
+
+#endif
--

2.30.2

#4 lore raw

PATCH 2/3 libbpf: Introduce elf_find_func_offset_from_elf_file() function

In reply to Daniel Müller

This change splits the elf_find_func_offset() function in two:
elf_find_func_offset(), which now accepts an already opened Elf object instead of a path to a file that is to be opened, as well as elf_find_func_offset_from_elf_file(), which opens a binary based on a path and then invokes elf_find_func_offset() on the Elf object. Having this split in responsibilities will allow us to call elf_find_func_offset() from other code paths on Elf objects that did not necessarily come from a file on disk.

---
 tools/lib/bpf/libbpf.c | 55 +++++++++++++++++++++++++++---------------
 1 file changed, 35 insertions(+), 20 deletions(-)
tools/lib/bpf/libbpf.c +35 -20
index 05c4db3..a474f49 100644
@@ -10531,32 +10531,19 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
 	return NULL;
 }
 
-/* Find offset of function name in object specified by path.  "name" matches
- * symbol name or name@@LIB for library functions.
+/* Find offset of function name in the provided ELF object.  "binary_path" is
+ * the path to the ELF binary represented by "elf", and only used for error
+ * reporting matters.  "name" matches symbol name or name@@LIB for library
+ * functions.
  */
-static long elf_find_func_offset(const char *binary_path, const char *name)
+static long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
 {
-	int fd, i, sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
+	int i, sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
 	bool is_shared_lib, is_name_qualified;
-	char errmsg[STRERR_BUFSIZE];
 	long ret = -ENOENT;
 	size_t name_len;
 	GElf_Ehdr ehdr;
-	Elf *elf;
 
-	fd = open(binary_path, O_RDONLY | O_CLOEXEC);
-	if (fd < 0) {
-		ret = -errno;
-		pr_warn("failed to open %s: %s\n", binary_path,
-			libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
-		return ret;
-	}
-	elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
-	if (!elf) {
-		pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
-		close(fd);
-		return -LIBBPF_ERRNO__FORMAT;
-	}
 	if (!gelf_getehdr(elf, &ehdr)) {
 		pr_warn("elf: failed to get ehdr from %s: %s\n", binary_path, elf_errmsg(-1));
 		ret = -LIBBPF_ERRNO__FORMAT;
@@ -10682,6 +10669,34 @@ static long elf_find_func_offset(const char *binary_path, const char *name)
 		}
 	}
 out:
+	return ret;
+}
+
+/* Find offset of function name in ELF object specified by path.  "name" matches
+ * symbol name or name@@LIB for library functions.
+ */
+static long elf_find_func_offset_from_elf_file(const char *binary_path, const char *name)
+{
+	char errmsg[STRERR_BUFSIZE];
+	long ret = -ENOENT;
+	Elf *elf;
+	int fd;
+
+	fd = open(binary_path, O_RDONLY | O_CLOEXEC);
+	if (fd < 0) {
+		ret = -errno;
+		pr_warn("failed to open %s: %s\n", binary_path,
+			libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
+		return ret;
+	}
+	elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
+	if (!elf) {
+		pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
+		close(fd);
+		return -LIBBPF_ERRNO__FORMAT;
+	}
+
+	ret = elf_find_func_offset(elf, binary_path, name);
 	elf_end(elf);
 	close(fd);
 	return ret;
@@ -10805,7 +10820,7 @@ bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
 	if (func_name) {
 		long sym_off;
 
-		sym_off = elf_find_func_offset(binary_path, func_name);
+		sym_off = elf_find_func_offset_from_elf_file(binary_path, func_name);
 		if (sym_off < 0)
 			return libbpf_err_ptr(sym_off);
 		func_offset += sym_off;
Signature · 1 line
2.30.2
#5 lore raw

Re: [PATCH bpf-next 2/3] libbpf: Introduce elf_find_func_offset_from_elf_file() function

Quoted 10 lines · Daniel Müller

This change splits the elf_find_func_offset() function in two:
elf_find_func_offset(), which now accepts an already opened Elf object instead of a path to a file that is to be opened, as well as elf_find_func_offset_from_elf_file(), which opens a binary based on a path and then invokes elf_find_func_offset() on the Elf object. Having this split in responsibilities will allow us to call elf_find_func_offset() from other code paths on Elf objects that did not necessarily come from a file on disk.

Quoted 5 lines
---
 tools/lib/bpf/libbpf.c | 55 +++++++++++++++++++++++++++---------------
 1 file changed, 35 insertions(+), 20 deletions(-)

Looks good, just few pedantic nits

Quoted 44 lines
tools/lib/bpf/libbpf.c +6 -19
index 05c4db3..a474f49 100644
@@ -10531,32 +10531,19 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
        return NULL;
 }
 
-/* Find offset of function name in object specified by path.  "name" matches
- * symbol name or name@@LIB for library functions.
+/* Find offset of function name in the provided ELF object.  "binary_path" is
+ * the path to the ELF binary represented by "elf", and only used for error
+ * reporting matters.  "name" matches symbol name or name@@LIB for library
+ * functions.
  */
-static long elf_find_func_offset(const char *binary_path, const char *name)
+static long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
 {
-       int fd, i, sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
+       int i, sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
        bool is_shared_lib, is_name_qualified;
-       char errmsg[STRERR_BUFSIZE];
        long ret = -ENOENT;
        size_t name_len;
        GElf_Ehdr ehdr;
-       Elf *elf;
 
-       fd = open(binary_path, O_RDONLY | O_CLOEXEC);
-       if (fd < 0) {
-               ret = -errno;
-               pr_warn("failed to open %s: %s\n", binary_path,
-                       libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
-               return ret;
-       }
-       elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
-       if (!elf) {
-               pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
-               close(fd);
-               return -LIBBPF_ERRNO__FORMAT;
-       }
        if (!gelf_getehdr(elf, &ehdr)) {
                pr_warn("elf: failed to get ehdr from %s: %s\n", binary_path, elf_errmsg(-1));
                ret = -LIBBPF_ERRNO__FORMAT;

tools/lib/bpf/libbpf.c static long elf_find_func_offset(const char *binary_path, const char *name) · line 10675

Quoted 7 lines
tools/lib/bpf/libbpf.c +4 -0
@@ -10682,6 +10669,34 @@ static long elf_find_func_offset(const char *binary_path, const char *name)
                }
        }
 out:
+       return ret;
+}
+
+/* Find offset of function name in ELF object specified by path.  "name" matches

nit: seems like it's original spelling, but let's remove these double spaces (same above in elf_find_func_offset comment)

tools/lib/bpf/libbpf.c static long elf_find_func_offset(const char *binary_path, const char *name) · line 10678

Quoted 3 lines
tools/lib/bpf/libbpf.c +3 -0
+ * symbol name or name@@LIB for library functions.
+ */
+static long elf_find_func_offset_from_elf_file(const char *binary_path, const char *name)

"from_file" would be enough, reads a bit tautological otherwise

tools/lib/bpf/libbpf.c static long elf_find_func_offset(const char *binary_path, const char *name) · line 10685

Quoted 7 lines
tools/lib/bpf/libbpf.c +7 -0
+{
+       char errmsg[STRERR_BUFSIZE];
+       long ret = -ENOENT;
+       Elf *elf;
+       int fd;
+
+       fd = open(binary_path, O_RDONLY | O_CLOEXEC);

btw, this reminded me that in patch #1 we probably want to pass O_CLOEXEC as well?

Quoted 29 lines
tools/lib/bpf/libbpf.c +15 -2
+       if (fd < 0) {
+               ret = -errno;
+               pr_warn("failed to open %s: %s\n", binary_path,
+                       libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
+               return ret;
+       }
+       elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
+       if (!elf) {
+               pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
+               close(fd);
+               return -LIBBPF_ERRNO__FORMAT;
+       }
+
+       ret = elf_find_func_offset(elf, binary_path, name);
        elf_end(elf);
        close(fd);
        return ret;
@@ -10805,7 +10820,7 @@ bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
        if (func_name) {
                long sym_off;
 
-               sym_off = elf_find_func_offset(binary_path, func_name);
+               sym_off = elf_find_func_offset_from_elf_file(binary_path, func_name);
                if (sym_off < 0)
                        return libbpf_err_ptr(sym_off);
                func_offset += sym_off;
--

2.30.2

#6 lore raw

Re: [PATCH bpf-next 2/3] libbpf: Introduce elf_find_func_offset_from_elf_file() function

Quoted 66 lines · Andrii Nakryiko
Quoted 10 lines · Daniel Müller

This change splits the elf_find_func_offset() function in two:
elf_find_func_offset(), which now accepts an already opened Elf object instead of a path to a file that is to be opened, as well as elf_find_func_offset_from_elf_file(), which opens a binary based on a path and then invokes elf_find_func_offset() on the Elf object. Having this split in responsibilities will allow us to call elf_find_func_offset() from other code paths on Elf objects that did not necessarily come from a file on disk.

Quoted 5 lines
---
 tools/lib/bpf/libbpf.c | 55 +++++++++++++++++++++++++++---------------
 1 file changed, 35 insertions(+), 20 deletions(-)

Looks good, just few pedantic nits

Quoted 46 lines
tools/lib/bpf/libbpf.c +6 -19
index 05c4db3..a474f49 100644
@@ -10531,32 +10531,19 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
        return NULL;
 }
 
-/* Find offset of function name in object specified by path.  "name" matches
- * symbol name or name@@LIB for library functions.
+/* Find offset of function name in the provided ELF object.  "binary_path" is
+ * the path to the ELF binary represented by "elf", and only used for error
+ * reporting matters.  "name" matches symbol name or name@@LIB for library
+ * functions.
  */
-static long elf_find_func_offset(const char *binary_path, const char *name)
+static long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
 {
-       int fd, i, sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
+       int i, sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
        bool is_shared_lib, is_name_qualified;
-       char errmsg[STRERR_BUFSIZE];
        long ret = -ENOENT;
        size_t name_len;
        GElf_Ehdr ehdr;
-       Elf *elf;
 
-       fd = open(binary_path, O_RDONLY | O_CLOEXEC);
-       if (fd < 0) {
-               ret = -errno;
-               pr_warn("failed to open %s: %s\n", binary_path,
-                       libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
-               return ret;
-       }
-       elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
-       if (!elf) {
-               pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
-               close(fd);
-               return -LIBBPF_ERRNO__FORMAT;
-       }
        if (!gelf_getehdr(elf, &ehdr)) {
                pr_warn("elf: failed to get ehdr from %s: %s\n", binary_path, elf_errmsg(-1));
                ret = -LIBBPF_ERRNO__FORMAT;
@@ -10682,6 +10669,34 @@ static long elf_find_func_offset(const char *binary_path, const char *name)
                }
        }
Quoted 8 lines

tools/lib/bpf/libbpf.c static long elf_find_func_offset(const char *binary_path, const char *name) · line 10675

Quoted 5 lines
tools/lib/bpf/libbpf.c +4 -0
 out:
+       return ret;
+}
+
+/* Find offset of function name in ELF object specified by path.  "name" matches

nit: seems like it's original spelling, but let's remove these double spaces (same above in elf_find_func_offset comment)

Done.

Quoted 5 lines

tools/lib/bpf/libbpf.c static long elf_find_func_offset(const char *binary_path, const char *name) · line 10678

Quoted 3 lines
tools/lib/bpf/libbpf.c +3 -0
+ * symbol name or name@@LIB for library functions.
+ */
+static long elf_find_func_offset_from_elf_file(const char *binary_path, const char *name)

"from_file" would be enough, reads a bit tautological otherwise

Sure.

Quoted 10 lines

tools/lib/bpf/libbpf.c static long elf_find_func_offset(const char *binary_path, const char *name) · line 10685

Quoted 7 lines
tools/lib/bpf/libbpf.c +7 -0
+{
+       char errmsg[STRERR_BUFSIZE];
+       long ret = -ENOENT;
+       Elf *elf;
+       int fd;
+
+       fd = open(binary_path, O_RDONLY | O_CLOEXEC);

btw, this reminded me that in patch #1 we probably want to pass O_CLOEXEC as well?

Sounds good.

Quoted 29 lines
Quoted 29 lines
tools/lib/bpf/libbpf.c +15 -2
+       if (fd < 0) {
+               ret = -errno;
+               pr_warn("failed to open %s: %s\n", binary_path,
+                       libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
+               return ret;
+       }
+       elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
+       if (!elf) {
+               pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
+               close(fd);
+               return -LIBBPF_ERRNO__FORMAT;
+       }
+
+       ret = elf_find_func_offset(elf, binary_path, name);
        elf_end(elf);
        close(fd);
        return ret;
@@ -10805,7 +10820,7 @@ bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
        if (func_name) {
                long sym_off;
 
-               sym_off = elf_find_func_offset(binary_path, func_name);
+               sym_off = elf_find_func_offset_from_elf_file(binary_path, func_name);
                if (sym_off < 0)
                        return libbpf_err_ptr(sym_off);
                func_offset += sym_off;
--

2.30.2

#7 lore raw

PATCH 3/3 libbpf: Add support for attaching uprobes to shared objects in APKs

In reply to Daniel Müller

This change adds support for attaching uprobes to shared objects located in APKs, which is relevant for Android systems where various libraries may reside in APKs. To make that happen, we extend the syntax for the "binary path" argument to attach to with that supported by various Android tools:
<archive>!/<binary-in-archive>

For example:
/system/app/test-app/test-app.apk!/lib/arm64-v8a/libc++_shared.so

APKs need to be specified via full path, i.e., we do not attempt to resolve mere file names by searching system directories.

We cannot currently test this functionality end-to-end in an automated fashion, because it relies on an Android system being present, but there is no support for that in CI. I have tested the functionality manually, by creating a libbpf program containing a uretprobe, attaching it to a function inside a shared object inside an APK, and verifying the sanity of the returned values.

---
 tools/lib/bpf/libbpf.c | 84 ++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 80 insertions(+), 4 deletions(-)
tools/lib/bpf/libbpf.c +80 -4
index a474f49..79ab85f 100644
@@ -53,6 +53,7 @@
 #include "libbpf_internal.h"
 #include "hashmap.h"
 #include "bpf_gen_internal.h"
+#include "zip.h"
 
 #ifndef BPF_FS_MAGIC
 #define BPF_FS_MAGIC		0xcafe4a11
@@ -10702,6 +10703,60 @@ static long elf_find_func_offset_from_elf_file(const char *binary_path, const ch
 	return ret;
 }
 
+/* Find offset of function name in archive specified by path. Currently
+ * supported are .zip files that do not compress their contents (as used on
+ * Android in the form of APKs, for example).  "file_name" is the name of the
+ * ELF file inside the archive.  "func_name" matches symbol name or name@@LIB
+ * for library functions.
+ */
+static long elf_find_func_offset_from_archive(const char *archive_path, const char *file_name,
+					      const char *func_name)
+{
+	struct zip_archive *archive;
+	struct zip_entry entry;
+	long ret = -ENOENT;
+	Elf *elf;
+
+	archive = zip_archive_open(archive_path);
+	if (!archive) {
+		pr_warn("failed to open %s\n", archive_path);
+		return -LIBBPF_ERRNO__FORMAT;
+	}
+
+	if (zip_archive_find_entry(archive, file_name, &entry)) {
+		pr_warn("zip: could not find archive member %s in %s\n", file_name, archive_path);
+		ret = -LIBBPF_ERRNO__FORMAT;
+		goto out;
+	}
+
+	if (entry.compression) {
+		pr_warn("zip: entry %s of %s is compressed and cannot be handled\n", file_name,
+			archive_path);
+		ret = -LIBBPF_ERRNO__FORMAT;
+		goto out;
+	}
+
+	elf = elf_memory((void *)entry.data, entry.data_length);
+	if (!elf) {
+		pr_warn("elf: could not read elf file %s from %s: %s\n", file_name, archive_path,
+			elf_errmsg(-1));
+		ret = -LIBBPF_ERRNO__FORMAT;
+		goto out;
+	}
+
+	ret = elf_find_func_offset(elf, file_name, func_name);
+	if (ret > 0) {
+		ret += entry.data_offset;
+		pr_debug("elf: symbol address match for '%s' in '%s': 0x%lx\n", func_name,
+			 archive_path, ret);
+	}
+	elf_end(elf);
+
+out:
+	zip_archive_close(archive);
+	return ret;
+}
+
 static const char *arch_specific_lib_paths(void)
 {
 	/*
@@ -10789,6 +10844,9 @@ bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
 {
 	DECLARE_LIBBPF_OPTS(bpf_perf_event_opts, pe_opts);
 	char errmsg[STRERR_BUFSIZE], *legacy_probe = NULL;
+	const char *archive_path = NULL;
+	const char *archive_sep = NULL;
+	char full_archive_path[PATH_MAX];
 	char full_binary_path[PATH_MAX];
 	struct bpf_link *link;
 	size_t ref_ctr_off;
@@ -10806,9 +10864,21 @@ bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
 	if (!binary_path)
 		return libbpf_err_ptr(-EINVAL);
 
-	if (!strchr(binary_path, '/')) {
-		err = resolve_full_path(binary_path, full_binary_path,
-					sizeof(full_binary_path));
+	/* Check if "binary_path" refers to an archive. */
+	archive_sep = strstr(binary_path, "!/");
+	if (archive_sep) {
+		if (archive_sep - binary_path >= sizeof(full_archive_path)) {
+			return libbpf_err_ptr(-EINVAL);
+		}
+
+		strncpy(full_archive_path, binary_path, archive_sep - binary_path);
+		full_archive_path[archive_sep - binary_path] = 0;
+		archive_path = full_archive_path;
+
+		strcpy(full_binary_path, archive_sep + 2);
+		binary_path = full_binary_path;
+	} else if (!strchr(binary_path, '/')) {
+		err = resolve_full_path(binary_path, full_binary_path, sizeof(full_binary_path));
 		if (err) {
 			pr_warn("prog '%s': failed to resolve full path for '%s': %d\n",
 				prog->name, binary_path, err);
@@ -10820,7 +10890,13 @@ bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
 	if (func_name) {
 		long sym_off;
 
-		sym_off = elf_find_func_offset_from_elf_file(binary_path, func_name);
+		if (archive_path) {
+			sym_off = elf_find_func_offset_from_archive(archive_path, binary_path,
+								    func_name);
+			binary_path = archive_path;
+		} else {
+			sym_off = elf_find_func_offset_from_elf_file(binary_path, func_name);
+		}
 		if (sym_off < 0)
 			return libbpf_err_ptr(sym_off);
 		func_offset += sym_off;
Signature · 1 line
2.30.2
#8 lore raw

Re: [PATCH bpf-next 3/3] libbpf: Add support for attaching uprobes to shared objects in APKs

Quoted 3 lines · Daniel Müller

This change adds support for attaching uprobes to shared objects located in APKs, which is relevant for Android systems where various libraries

Is there a good link with description of APK that we can record somewhere in the comments for future us?

Also, does .apk contains only shared libraries, or it could be also just a binary?

Quoted 10 lines

may reside in APKs. To make that happen, we extend the syntax for the "binary path" argument to attach to with that supported by various Android tools:
<archive>!/<binary-in-archive>

For example:
/system/app/test-app/test-app.apk!/lib/arm64-v8a/libc++_shared.so

APKs need to be specified via full path, i.e., we do not attempt to resolve mere file names by searching system directories.

mere?

Quoted 26 lines

We cannot currently test this functionality end-to-end in an automated fashion, because it relies on an Android system being present, but there is no support for that in CI. I have tested the functionality manually, by creating a libbpf program containing a uretprobe, attaching it to a function inside a shared object inside an APK, and verifying the sanity of the returned values.

---
 tools/lib/bpf/libbpf.c | 84 ++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 80 insertions(+), 4 deletions(-)
tools/lib/bpf/libbpf.c +1 -0
index a474f49..79ab85f 100644
@@ -53,6 +53,7 @@
 #include "libbpf_internal.h"
 #include "hashmap.h"
 #include "bpf_gen_internal.h"
+#include "zip.h"
 
 #ifndef BPF_FS_MAGIC
 #define BPF_FS_MAGIC           0xcafe4a11

tools/lib/bpf/libbpf.c static long elf_find_func_offset_from_elf_file(const char *binary_path, const ch · line 10710

Quoted 8 lines
tools/lib/bpf/libbpf.c +5 -0
@@ -10702,6 +10703,60 @@ static long elf_find_func_offset_from_elf_file(const char *binary_path, const ch
        return ret;
 }
 
+/* Find offset of function name in archive specified by path. Currently
+ * supported are .zip files that do not compress their contents (as used on
+ * Android in the form of APKs, for example).  "file_name" is the name of the
+ * ELF file inside the archive.  "func_name" matches symbol name or name@@LIB
+ * for library functions.

These double spaces after dot were not intended, let's not add more.

tools/lib/bpf/libbpf.c static long elf_find_func_offset_from_elf_file(const char *binary_path, const ch · line 10722

Quoted 12 lines
tools/lib/bpf/libbpf.c +12 -0
+ */
+static long elf_find_func_offset_from_archive(const char *archive_path, const char *file_name,
+                                             const char *func_name)
+{
+       struct zip_archive *archive;
+       struct zip_entry entry;
+       long ret = -ENOENT;
+       Elf *elf;
+
+       archive = zip_archive_open(archive_path);
+       if (!archive) {
+               pr_warn("failed to open %s\n", archive_path);

add "zip: " prefix?

Quoted 9 lines
tools/lib/bpf/libbpf.c +9 -0
+               return -LIBBPF_ERRNO__FORMAT;
+       }
+
+       if (zip_archive_find_entry(archive, file_name, &entry)) {
+               pr_warn("zip: could not find archive member %s in %s\n", file_name, archive_path);
+               ret = -LIBBPF_ERRNO__FORMAT;
+               goto out;
+       }
+

tools/lib/bpf/libbpf.c static long elf_find_func_offset_from_elf_file(const char *binary_path, const ch · line 10741

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+       if (entry.compression) {
+               pr_warn("zip: entry %s of %s is compressed and cannot be handled\n", file_name,
+                       archive_path);
+               ret = -LIBBPF_ERRNO__FORMAT;
+               goto out;
+       }
+
+       elf = elf_memory((void *)entry.data, entry.data_length);
+       if (!elf) {
+               pr_warn("elf: could not read elf file %s from %s: %s\n", file_name, archive_path,

I kind of like preserving the "archive/path!/file/path" consistently through error messages when referring to file within APK, WDYT?

tools/lib/bpf/libbpf.c static long elf_find_func_offset_from_elf_file(const char *binary_path, const ch · line 10751

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+                       elf_errmsg(-1));
+               ret = -LIBBPF_ERRNO__FORMAT;
+               goto out;
+       }
+
+       ret = elf_find_func_offset(elf, file_name, func_name);
+       if (ret > 0) {
+               ret += entry.data_offset;
+               pr_debug("elf: symbol address match for '%s' in '%s': 0x%lx\n", func_name,
+                        archive_path, ret);

so for debugging I feel like we'll want to know both entry.data_offset and original ELF offset, let's report all three offset (including the final calculated one)?

Quoted 12 lines
tools/lib/bpf/libbpf.c +8 -0
+       }
+       elf_end(elf);
+
+out:
+       zip_archive_close(archive);
+       return ret;
+}
+
 static const char *arch_specific_lib_paths(void)
 {
        /*

tools/lib/bpf/libbpf.c bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, · line 10848

Quoted 5 lines
tools/lib/bpf/libbpf.c +2 -0
@@ -10789,6 +10844,9 @@ bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
 {
        DECLARE_LIBBPF_OPTS(bpf_perf_event_opts, pe_opts);
        char errmsg[STRERR_BUFSIZE], *legacy_probe = NULL;
+       const char *archive_path = NULL;
+       const char *archive_sep = NULL;

nit: combine on a single line?

Quoted 5 lines
tools/lib/bpf/libbpf.c +1 -0
+       char full_archive_path[PATH_MAX];
        char full_binary_path[PATH_MAX];
        struct bpf_link *link;
        size_t ref_ctr_off;

tools/lib/bpf/libbpf.c bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, · line 10870

Quoted 10 lines
tools/lib/bpf/libbpf.c +4 -3
@@ -10806,9 +10864,21 @@ bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
        if (!binary_path)
                return libbpf_err_ptr(-EINVAL);
 
-       if (!strchr(binary_path, '/')) {
-               err = resolve_full_path(binary_path, full_binary_path,
-                                       sizeof(full_binary_path));
+       /* Check if "binary_path" refers to an archive. */
+       archive_sep = strstr(binary_path, "!/");
+       if (archive_sep) {
+               if (archive_sep - binary_path >= sizeof(full_archive_path)) {

very unlikely to happen, I wouldn't bother checking, especially that strncpy will just truncate and make us fail anyways

tools/lib/bpf/libbpf.c bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, · line 10874

Quoted 4 lines
tools/lib/bpf/libbpf.c +4 -0
+                       return libbpf_err_ptr(-EINVAL);
+               }
+
+               strncpy(full_archive_path, binary_path, archive_sep - binary_path);

let's use saner libbpf_strlcpy() instead of strncpy, we stopped using strncpy relatively recently

tools/lib/bpf/libbpf.c bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, · line 10875

Quoted 1 line
tools/lib/bpf/libbpf.c +1 -0
+               full_archive_path[archive_sep - binary_path] = 0;

strlcpy makes sure the resulting string is zero-terminated.

But note that full_archive_path[0] is not guaranteed to be zero, so strncpy/strlcpy might preserve some garbage in front. Let's make sure full_archive_path[0] = '\0'; before manipulating that buffer

tools/lib/bpf/libbpf.c bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, · line 10879

Quoted 4 lines
tools/lib/bpf/libbpf.c +4 -0
+               archive_path = full_archive_path;
+
+               strcpy(full_binary_path, archive_sep + 2);
+               binary_path = full_binary_path;

no need to copy, just binary_path = archive_sep + 2;? And thus we can reuse full_binary_path buffer for archive path (we can rename it to be more generic "full_path" name or something)

Quoted 23 lines
tools/lib/bpf/libbpf.c +9 -2
+       } else if (!strchr(binary_path, '/')) {
+               err = resolve_full_path(binary_path, full_binary_path, sizeof(full_binary_path));
                if (err) {
                        pr_warn("prog '%s': failed to resolve full path for '%s': %d\n",
                                prog->name, binary_path, err);
@@ -10820,7 +10890,13 @@ bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
        if (func_name) {
                long sym_off;
 
-               sym_off = elf_find_func_offset_from_elf_file(binary_path, func_name);
+               if (archive_path) {
+                       sym_off = elf_find_func_offset_from_archive(archive_path, binary_path,
+                                                                   func_name);
+                       binary_path = archive_path;
+               } else {
+                       sym_off = elf_find_func_offset_from_elf_file(binary_path, func_name);
+               }
                if (sym_off < 0)
                        return libbpf_err_ptr(sym_off);
                func_offset += sym_off;
--

2.30.2

#9 lore raw

Re: [PATCH bpf-next 3/3] libbpf: Add support for attaching uprobes to shared objects in APKs

Quoted 7 lines · Andrii Nakryiko
Quoted 3 lines · Daniel Müller

This change adds support for attaching uprobes to shared objects located in APKs, which is relevant for Android systems where various libraries

Is there a good link with description of APK that we can record somewhere in the comments for future us?

Perhaps
https://en.wikipedia.org/w/index.php?title=Apk_(file_format)&oldid=1139099120#Package_contents.

Will add it.

Quoted 2 lines

Also, does .apk contains only shared libraries, or it could be also just a binary?

It probably could also be for a binary, judging from applications being available for download in the form of APKs.

Quoted 5 lines
Quoted 5 lines

may reside in APKs. To make that happen, we extend the syntax for the "binary path" argument to attach to with that supported by various Android tools:
<archive>!/<binary-in-archive>

Quoted 7 lines
Quoted 5 lines

For example:
/system/app/test-app/test-app.apk!/lib/arm64-v8a/libc++_shared.so

APKs need to be specified via full path, i.e., we do not attempt to resolve mere file names by searching system directories.

mere?

Yes?

Quoted 29 lines
Quoted 29 lines

We cannot currently test this functionality end-to-end in an automated fashion, because it relies on an Android system being present, but there is no support for that in CI. I have tested the functionality manually, by creating a libbpf program containing a uretprobe, attaching it to a function inside a shared object inside an APK, and verifying the sanity of the returned values.

---
 tools/lib/bpf/libbpf.c | 84 ++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 80 insertions(+), 4 deletions(-)
tools/lib/bpf/libbpf.c +1 -0
index a474f49..79ab85f 100644
@@ -53,6 +53,7 @@
 #include "libbpf_internal.h"
 #include "hashmap.h"
 #include "bpf_gen_internal.h"
+#include "zip.h"
 
 #ifndef BPF_FS_MAGIC
 #define BPF_FS_MAGIC           0xcafe4a11
@@ -10702,6 +10703,60 @@ static long elf_find_func_offset_from_elf_file(const char *binary_path, const ch
        return ret;
 }
 
Quoted 7 lines

tools/lib/bpf/libbpf.c static long elf_find_func_offset_from_elf_file(const char *binary_path, const ch · line 10710

Quoted 5 lines
tools/lib/bpf/libbpf.c +5 -0
+/* Find offset of function name in archive specified by path. Currently
+ * supported are .zip files that do not compress their contents (as used on
+ * Android in the form of APKs, for example).  "file_name" is the name of the
+ * ELF file inside the archive.  "func_name" matches symbol name or name@@LIB
+ * for library functions.

These double spaces after dot were not intended, let's not add more.

Sure.

Quoted 6 lines
Quoted 6 lines
tools/lib/bpf/libbpf.c +6 -0
+ */
+static long elf_find_func_offset_from_archive(const char *archive_path, const char *file_name,
+                                             const char *func_name)
+{
+       struct zip_archive *archive;
+       struct zip_entry entry;
Quoted 8 lines

tools/lib/bpf/libbpf.c static long elf_find_func_offset_from_elf_file(const char *binary_path, const ch · line 10722

Quoted 6 lines
tools/lib/bpf/libbpf.c +6 -0
+       long ret = -ENOENT;
+       Elf *elf;
+
+       archive = zip_archive_open(archive_path);
+       if (!archive) {
+               pr_warn("failed to open %s\n", archive_path);

add "zip: " prefix?

Added.

Quoted 14 lines
Quoted 14 lines
tools/lib/bpf/libbpf.c +14 -0
+               return -LIBBPF_ERRNO__FORMAT;
+       }
+
+       if (zip_archive_find_entry(archive, file_name, &entry)) {
+               pr_warn("zip: could not find archive member %s in %s\n", file_name, archive_path);
+               ret = -LIBBPF_ERRNO__FORMAT;
+               goto out;
+       }
+
+       if (entry.compression) {
+               pr_warn("zip: entry %s of %s is compressed and cannot be handled\n", file_name,
+                       archive_path);
+               ret = -LIBBPF_ERRNO__FORMAT;
+               goto out;
Quoted 8 lines

tools/lib/bpf/libbpf.c static long elf_find_func_offset_from_elf_file(const char *binary_path, const ch · line 10741

Quoted 5 lines
tools/lib/bpf/libbpf.c +5 -0
+       }
+
+       elf = elf_memory((void *)entry.data, entry.data_length);
+       if (!elf) {
+               pr_warn("elf: could not read elf file %s from %s: %s\n", file_name, archive_path,

I kind of like preserving the "archive/path!/file/path" consistently through error messages when referring to file within APK, WDYT?

It seems valuable to me to make it clear that we "parsed" the string correctly and split it into the expected parts.

Quoted 11 lines

tools/lib/bpf/libbpf.c static long elf_find_func_offset_from_elf_file(const char *binary_path, const ch · line 10751

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+                       elf_errmsg(-1));
+               ret = -LIBBPF_ERRNO__FORMAT;
+               goto out;
+       }
+
+       ret = elf_find_func_offset(elf, file_name, func_name);
+       if (ret > 0) {
+               ret += entry.data_offset;
+               pr_debug("elf: symbol address match for '%s' in '%s': 0x%lx\n", func_name,
+                        archive_path, ret);
Quoted 3 lines

so for debugging I feel like we'll want to know both entry.data_offset and original ELF offset, let's report all three offset (including the final calculated one)?

I added one more pr_debug() printing the entry offset. The ELF offset is reported by elf_find_func_offset() and the final offset here.

Quoted 28 lines
Quoted 12 lines
tools/lib/bpf/libbpf.c +8 -0
+       }
+       elf_end(elf);
+
+out:
+       zip_archive_close(archive);
+       return ret;
+}
+
 static const char *arch_specific_lib_paths(void)
 {
        /*

tools/lib/bpf/libbpf.c bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, · line 10848

Quoted 5 lines
tools/lib/bpf/libbpf.c +2 -0
@@ -10789,6 +10844,9 @@ bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
 {
        DECLARE_LIBBPF_OPTS(bpf_perf_event_opts, pe_opts);
        char errmsg[STRERR_BUFSIZE], *legacy_probe = NULL;
+       const char *archive_path = NULL;
+       const char *archive_sep = NULL;

nit: combine on a single line?

Quoted 8 lines
tools/lib/bpf/libbpf.c +1 -0
+       char full_archive_path[PATH_MAX];
        char full_binary_path[PATH_MAX];
        struct bpf_link *link;
        size_t ref_ctr_off;
@@ -10806,9 +10864,21 @@ bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
        if (!binary_path)
                return libbpf_err_ptr(-EINVAL);
 
Quoted 10 lines

tools/lib/bpf/libbpf.c bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, · line 10870

Quoted 7 lines
tools/lib/bpf/libbpf.c +4 -3
-       if (!strchr(binary_path, '/')) {
-               err = resolve_full_path(binary_path, full_binary_path,
-                                       sizeof(full_binary_path));
+       /* Check if "binary_path" refers to an archive. */
+       archive_sep = strstr(binary_path, "!/");
+       if (archive_sep) {
+               if (archive_sep - binary_path >= sizeof(full_archive_path)) {

very unlikely to happen, I wouldn't bother checking, especially that strncpy will just truncate and make us fail anyways

How will it "make us fail"? It will silently truncate the path, no?

Quoted 7 lines

tools/lib/bpf/libbpf.c bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, · line 10874

Quoted 4 lines
tools/lib/bpf/libbpf.c +4 -0
+                       return libbpf_err_ptr(-EINVAL);
+               }
+
+               strncpy(full_archive_path, binary_path, archive_sep - binary_path);

let's use saner libbpf_strlcpy() instead of strncpy, we stopped using strncpy relatively recently

Okay.

Quoted 7 lines

tools/lib/bpf/libbpf.c bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, · line 10875

Quoted 1 line
tools/lib/bpf/libbpf.c +1 -0
+               full_archive_path[archive_sep - binary_path] = 0;

strlcpy makes sure the resulting string is zero-terminated.

But note that full_archive_path[0] is not guaranteed to be zero, so strncpy/strlcpy might preserve some garbage in front. Let's make sure full_archive_path[0] = '\0'; before manipulating that buffer

Sure.

Quoted 8 lines

tools/lib/bpf/libbpf.c bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, · line 10879

Quoted 4 lines
tools/lib/bpf/libbpf.c +4 -0
+               archive_path = full_archive_path;
+
+               strcpy(full_binary_path, archive_sep + 2);
+               binary_path = full_binary_path;

no need to copy, just binary_path = archive_sep + 2;? And thus we can reuse full_binary_path buffer for archive path (we can rename it to be more generic "full_path" name or something)

Okay.

Quoted 23 lines
Quoted 23 lines
tools/lib/bpf/libbpf.c +9 -2
+       } else if (!strchr(binary_path, '/')) {
+               err = resolve_full_path(binary_path, full_binary_path, sizeof(full_binary_path));
                if (err) {
                        pr_warn("prog '%s': failed to resolve full path for '%s': %d\n",
                                prog->name, binary_path, err);
@@ -10820,7 +10890,13 @@ bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
        if (func_name) {
                long sym_off;
 
-               sym_off = elf_find_func_offset_from_elf_file(binary_path, func_name);
+               if (archive_path) {
+                       sym_off = elf_find_func_offset_from_archive(archive_path, binary_path,
+                                                                   func_name);
+                       binary_path = archive_path;
+               } else {
+                       sym_off = elf_find_func_offset_from_elf_file(binary_path, func_name);
+               }
                if (sym_off < 0)
                        return libbpf_err_ptr(sym_off);
                func_offset += sym_off;
--

2.30.2

#10 lore raw

Re: [PATCH bpf-next 3/3] libbpf: Add support for attaching uprobes to shared objects in APKs

Quoted 29 lines · Daniel Müller
Quoted 7 lines · Andrii Nakryiko
Quoted 3 lines · Daniel Müller

This change adds support for attaching uprobes to shared objects located in APKs, which is relevant for Android systems where various libraries

Is there a good link with description of APK that we can record somewhere in the comments for future us?

Perhaps
https://en.wikipedia.org/w/index.php?title=Apk_(file_format)&oldid=1139099120#Package_contents.

Will add it.

Quoted 2 lines

Also, does .apk contains only shared libraries, or it could be also just a binary?

It probably could also be for a binary, judging from applications being available for download in the form of APKs.

Quoted 8 lines
Quoted 8 lines

may reside in APKs. To make that happen, we extend the syntax for the "binary path" argument to attach to with that supported by various Android tools:
<archive>!/<binary-in-archive>

For example:
/system/app/test-app/test-app.apk!/lib/arm64-v8a/libc++_shared.so

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

APKs need to be specified via full path, i.e., we do not attempt to resolve mere file names by searching system directories.

mere?

Yes?

I'm just confused what "resolve mere file names" means in this context. Like, which file names are not "mere"?

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

We cannot currently test this functionality end-to-end in an automated fashion, because it relies on an Android system being present, but there is no support for that in CI. I have tested the functionality manually, by creating a libbpf program containing a uretprobe, attaching it to a function inside a shared object inside an APK, and verifying the sanity of the returned values.

---
 tools/lib/bpf/libbpf.c | 84 ++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 80 insertions(+), 4 deletions(-)

[...]

Quoted 23 lines
Quoted 14 lines
Quoted 14 lines
tools/lib/bpf/libbpf.c +14 -0
+               return -LIBBPF_ERRNO__FORMAT;
+       }
+
+       if (zip_archive_find_entry(archive, file_name, &entry)) {
+               pr_warn("zip: could not find archive member %s in %s\n", file_name, archive_path);
+               ret = -LIBBPF_ERRNO__FORMAT;
+               goto out;
+       }
+
+       if (entry.compression) {
+               pr_warn("zip: entry %s of %s is compressed and cannot be handled\n", file_name,
+                       archive_path);
+               ret = -LIBBPF_ERRNO__FORMAT;
+               goto out;
Quoted 8 lines

tools/lib/bpf/libbpf.c static long elf_find_func_offset_from_elf_file(const char *binary_path, const ch · line 10741

Quoted 5 lines
tools/lib/bpf/libbpf.c +5 -0
+       }
+
+       elf = elf_memory((void *)entry.data, entry.data_length);
+       if (!elf) {
+               pr_warn("elf: could not read elf file %s from %s: %s\n", file_name, archive_path,

I kind of like preserving the "archive/path!/file/path" consistently through error messages when referring to file within APK, WDYT?

Quoted 2 lines

It seems valuable to me to make it clear that we "parsed" the string correctly and split it into the expected parts.

it's debatable, if the user doesn't trust libbpf to handle "archive/path!/file/path" spec correctly, then it's too bad. My point here is to keep it consistent with the way that user is specifying it in SEC("") definition

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
tools/lib/bpf/libbpf.c +10 -0
+                       elf_errmsg(-1));
+               ret = -LIBBPF_ERRNO__FORMAT;
+               goto out;
+       }
+
+       ret = elf_find_func_offset(elf, file_name, func_name);
+       if (ret > 0) {
+               ret += entry.data_offset;
+               pr_debug("elf: symbol address match for '%s' in '%s': 0x%lx\n", func_name,
+                        archive_path, ret);
Quoted 6 lines
Quoted 3 lines

so for debugging I feel like we'll want to know both entry.data_offset and original ELF offset, let's report all three offset (including the final calculated one)?

I added one more pr_debug() printing the entry offset. The ELF offset is reported by elf_find_func_offset() and the final offset here.

sure, but here we can have all of that conveniently in a single (debug) log message, so why not?

Quoted 40 lines
Quoted 29 lines
Quoted 13 lines
tools/lib/bpf/libbpf.c +8 -0
+       }
+       elf_end(elf);
+
+out:
+       zip_archive_close(archive);
+       return ret;
+}
+
 static const char *arch_specific_lib_paths(void)
 {
        /*

tools/lib/bpf/libbpf.c bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, · line 10848

Quoted 5 lines
tools/lib/bpf/libbpf.c +2 -0
@@ -10789,6 +10844,9 @@ bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
 {
        DECLARE_LIBBPF_OPTS(bpf_perf_event_opts, pe_opts);
        char errmsg[STRERR_BUFSIZE], *legacy_probe = NULL;
+       const char *archive_path = NULL;
+       const char *archive_sep = NULL;

nit: combine on a single line?

Quoted 8 lines
tools/lib/bpf/libbpf.c +1 -0
+       char full_archive_path[PATH_MAX];
        char full_binary_path[PATH_MAX];
        struct bpf_link *link;
        size_t ref_ctr_off;
@@ -10806,9 +10864,21 @@ bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
        if (!binary_path)
                return libbpf_err_ptr(-EINVAL);
 
Quoted 10 lines

tools/lib/bpf/libbpf.c bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, · line 10870

Quoted 7 lines
tools/lib/bpf/libbpf.c +4 -3
-       if (!strchr(binary_path, '/')) {
-               err = resolve_full_path(binary_path, full_binary_path,
-                                       sizeof(full_binary_path));
+       /* Check if "binary_path" refers to an archive. */
+       archive_sep = strstr(binary_path, "!/");
+       if (archive_sep) {
+               if (archive_sep - binary_path >= sizeof(full_archive_path)) {

very unlikely to happen, I wouldn't bother checking, especially that strncpy will just truncate and make us fail anyways

Quoted 1 line

How will it "make us fail"? It will silently truncate the path, no?

right, it will be invalid path. But we don't expect this, because we allocated PATH_MAX, so it's only if user goes crazy and makes up some huge invalid path, which never was going to succeed anyways. So I'd drop this check altogether.

Quoted 11 lines
Quoted 8 lines

tools/lib/bpf/libbpf.c bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, · line 10874

Quoted 5 lines
tools/lib/bpf/libbpf.c +4 -0
+                       return libbpf_err_ptr(-EINVAL);
+               }
+
+               strncpy(full_archive_path, binary_path, archive_sep - binary_path);

let's use saner libbpf_strlcpy() instead of strncpy, we stopped using strncpy relatively recently

Okay.

[...]

#11 lore raw

Re: [PATCH bpf-next 3/3] libbpf: Add support for attaching uprobes to shared objects in APKs

Quoted 33 lines · Andrii Nakryiko
Quoted 32 lines · Daniel Müller
Quoted 7 lines · Andrii Nakryiko
Quoted 3 lines · Daniel Müller

This change adds support for attaching uprobes to shared objects located in APKs, which is relevant for Android systems where various libraries

Is there a good link with description of APK that we can record somewhere in the comments for future us?

Perhaps
https://en.wikipedia.org/w/index.php?title=Apk_(file_format)&oldid=1139099120#Package_contents.

Will add it.

Quoted 2 lines

Also, does .apk contains only shared libraries, or it could be also just a binary?

It probably could also be for a binary, judging from applications being available for download in the form of APKs.

Quoted 11 lines
Quoted 10 lines

may reside in APKs. To make that happen, we extend the syntax for the "binary path" argument to attach to with that supported by various Android tools:
<archive>!/<binary-in-archive>

For example:
/system/app/test-app/test-app.apk!/lib/arm64-v8a/libc++_shared.so

APKs need to be specified via full path, i.e., we do not attempt to resolve mere file names by searching system directories.

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

mere?

Yes?

I'm just confused what "resolve mere file names" means in this context. Like, which file names are not "mere"?

It's meant to convey the fact that a "mere file name" is not everything we could be dealing with. It could also be a full path.

[...]

Thanks,
Daniel

#12 lore raw

Re: [PATCH bpf-next 3/3] libbpf: Add support for attaching uprobes to shared objects in APKs

Quoted 37 lines · Daniel Müller
Quoted 35 lines · Andrii Nakryiko
Quoted 34 lines · Daniel Müller
Quoted 7 lines · Andrii Nakryiko
Quoted 3 lines · Daniel Müller

This change adds support for attaching uprobes to shared objects located in APKs, which is relevant for Android systems where various libraries

Is there a good link with description of APK that we can record somewhere in the comments for future us?

Perhaps
https://en.wikipedia.org/w/index.php?title=Apk_(file_format)&oldid=1139099120#Package_contents.

Will add it.

Quoted 2 lines

Also, does .apk contains only shared libraries, or it could be also just a binary?

It probably could also be for a binary, judging from applications being available for download in the form of APKs.

Quoted 12 lines
Quoted 10 lines

may reside in APKs. To make that happen, we extend the syntax for the "binary path" argument to attach to with that supported by various Android tools:
<archive>!/<binary-in-archive>

For example:
/system/app/test-app/test-app.apk!/lib/arm64-v8a/libc++_shared.so

APKs need to be specified via full path, i.e., we do not attempt to resolve mere file names by searching system directories.

mere?

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Yes?

I'm just confused what "resolve mere file names" means in this context. Like, which file names are not "mere"?

It's meant to convey the fact that a "mere file name" is not everything we could be dealing with. It could also be a full path.

Ah, ok, I see. So you are just saying that we do not attempt path resolution like we do for .so and binaries. This wasn't clear to me.

Quoted 5 lines

[...]

Thanks,
Daniel

#13 lore raw
In reply to Daniel Müller
Quoted 24 lines · Daniel Müller

On Android, APKs (android packages; zip packages with somewhat prescriptive contents) are first class citizens in the system: the shared objects contained in them don't exist in unpacked form on the file system. Rather, they are mmaped directly from within the archive and the archive is also what the kernel is aware of.

For users that complicates the process of attaching a uprobe to a function contained in a shared object in one such APK: they'd have to find the byte offset of said function from the beginning of the archive. That is cumbersome to do manually and can be fragile, because various changes could invalidate said offset.

That is why for uprobes inside ELF files (not inside an APK), commit d112c9ce249b ("libbpf: Support function name-based attach uprobes") added support for attaching to symbols by name. On Android, that mechanism currently does not work, because this logic is not APK aware.

This patch set introduces first class support for attaching uprobes to functions inside ELF objects contained in APKs via function names. We add support for recognizing the following syntax for a binary path:
<archive>!/<binary-in-archive>

(e.g., /system/app/test-app.apk!/lib/arm64-v8a/libc++.so)

Quoted 9 lines

This syntax is common in the Android eco system and used by tools such as simpleperf. It is also what is being proposed for bcc [0].

If the user provides such a binary path, we find <binary-in-archive> (lib/arm64-v8a/libc++.so in the example) inside of <archive> (/system/app/test-app.apk). We perform the regular ELF offset search inside the binary and add that to the offset within the archive itself, to retrieve the offset at which to attach the uprobe.

I have to look in a bit more depth here, but my first thought is if we need the APK specifics in libbpf itself? Would having additional uprobe opts that specify elf memory and some sort of "don't attach, just figure out offset" flag work? Then you could perhaps do the work in patch 3 outside of libbpf, calling attach once to get the offset within the elf (using the changes in patch 2 to support ELF memory), then a second time to do the attach using the offset previously computed.

Then you could implement the APK handling in a custom SEC() handler which runs based on seeing an APK path or apk_uprobe/ prefix. Is that approach feasible? I'm guessing there's something I'm missing, but it would be good to understand what that is. Thanks!

Alan

Quoted 15 lines

[0] https://github.com/iovisor/bcc/pull/4440

Daniel Müller (3):
libbpf: Implement basic zip archive parsing support
libbpf: Introduce elf_find_func_offset_from_elf_file() function
libbpf: Add support for attaching uprobes to shared objects in APKs

 tools/lib/bpf/Build    |   2 +-
 tools/lib/bpf/libbpf.c | 137 ++++++++++++---
 tools/lib/bpf/zip.c    | 371 +++++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/zip.h    |  47 ++++++
 4 files changed, 533 insertions(+), 24 deletions(-)
 create mode 100644 tools/lib/bpf/zip.c
 create mode 100644 tools/lib/bpf/zip.h
#14 lore raw
Quoted 44 lines · Alan Maguire
Quoted 24 lines · Daniel Müller

On Android, APKs (android packages; zip packages with somewhat prescriptive contents) are first class citizens in the system: the shared objects contained in them don't exist in unpacked form on the file system. Rather, they are mmaped directly from within the archive and the archive is also what the kernel is aware of.

For users that complicates the process of attaching a uprobe to a function contained in a shared object in one such APK: they'd have to find the byte offset of said function from the beginning of the archive. That is cumbersome to do manually and can be fragile, because various changes could invalidate said offset.

That is why for uprobes inside ELF files (not inside an APK), commit d112c9ce249b ("libbpf: Support function name-based attach uprobes") added support for attaching to symbols by name. On Android, that mechanism currently does not work, because this logic is not APK aware.

This patch set introduces first class support for attaching uprobes to functions inside ELF objects contained in APKs via function names. We add support for recognizing the following syntax for a binary path:
<archive>!/<binary-in-archive>

(e.g., /system/app/test-app.apk!/lib/arm64-v8a/libc++.so)

Quoted 9 lines

This syntax is common in the Android eco system and used by tools such as simpleperf. It is also what is being proposed for bcc [0].

If the user provides such a binary path, we find <binary-in-archive> (lib/arm64-v8a/libc++.so in the example) inside of <archive> (/system/app/test-app.apk). We perform the regular ELF offset search inside the binary and add that to the offset within the archive itself, to retrieve the offset at which to attach the uprobe.

I have to look in a bit more depth here, but my first thought is if we need the APK specifics in libbpf itself? Would having additional uprobe opts that specify elf memory and some sort of "don't attach, just figure out offset" flag work? Then you could perhaps do the work in patch 3 outside of libbpf, calling attach once to get the offset within the elf (using the changes in patch 2 to support ELF memory), then a second time to do the attach using the offset previously computed.

Quoted 4 lines

Then you could implement the APK handling in a custom SEC() handler which runs based on seeing an APK path or apk_uprobe/ prefix. Is that approach feasible? I'm guessing there's something I'm missing, but it would be good to understand what that is. Thanks!

Thanks for taking a look! From what I understand what you laid out could work as well (though the devil may be in the detail here; I am not particularly familiar with custom SEC handlers and so unless it's being prototyped I can't say for certain).
That being said, I am not sure I see how it is superior: it strikes me as more complicated just from a control flow and orchestration point of view. It also does not seem more user friendly to work with. As mentioned in the description, the proposed syntax addition is common in the eco system. I would think that supporting it benefits users, which in turn helps with adoption of libbpf usage on Android systems.

Thanks,
Daniel

[...]

#15 lore raw
Quoted 51 lines · Daniel Müller
Quoted 44 lines · Alan Maguire
Quoted 24 lines · Daniel Müller

On Android, APKs (android packages; zip packages with somewhat prescriptive contents) are first class citizens in the system: the shared objects contained in them don't exist in unpacked form on the file system. Rather, they are mmaped directly from within the archive and the archive is also what the kernel is aware of.

For users that complicates the process of attaching a uprobe to a function contained in a shared object in one such APK: they'd have to find the byte offset of said function from the beginning of the archive. That is cumbersome to do manually and can be fragile, because various changes could invalidate said offset.

That is why for uprobes inside ELF files (not inside an APK), commit d112c9ce249b ("libbpf: Support function name-based attach uprobes") added support for attaching to symbols by name. On Android, that mechanism currently does not work, because this logic is not APK aware.

This patch set introduces first class support for attaching uprobes to functions inside ELF objects contained in APKs via function names. We add support for recognizing the following syntax for a binary path:
<archive>!/<binary-in-archive>

(e.g., /system/app/test-app.apk!/lib/arm64-v8a/libc++.so)

Quoted 9 lines

This syntax is common in the Android eco system and used by tools such as simpleperf. It is also what is being proposed for bcc [0].

If the user provides such a binary path, we find <binary-in-archive> (lib/arm64-v8a/libc++.so in the example) inside of <archive> (/system/app/test-app.apk). We perform the regular ELF offset search inside the binary and add that to the offset within the archive itself, to retrieve the offset at which to attach the uprobe.

I have to look in a bit more depth here, but my first thought is if we need the APK specifics in libbpf itself? Would having additional uprobe opts that specify elf memory and some sort of "don't attach, just figure out offset" flag work? Then you could perhaps do the work in patch 3 outside of libbpf, calling attach once to get the offset within the elf (using the changes in patch 2 to support ELF memory), then a second time to do the attach using the offset previously computed.

Quoted 4 lines

Then you could implement the APK handling in a custom SEC() handler which runs based on seeing an APK path or apk_uprobe/ prefix. Is that approach feasible? I'm guessing there's something I'm missing, but it would be good to understand what that is. Thanks!

Quoted 11 lines

Thanks for taking a look! From what I understand what you laid out could work as well (though the devil may be in the detail here; I am not particularly familiar with custom SEC handlers and so unless it's being prototyped I can't say for certain).
That being said, I am not sure I see how it is superior: it strikes me as more complicated just from a control flow and orchestration point of view. It also does not seem more user friendly to work with. As mentioned in the description, the proposed syntax addition is common in the eco system. I would think that supporting it benefits users, which in turn helps with adoption of libbpf usage on Android systems.

+1. Yes, a lot of stuff could be implemented with a custom SEC() handler, but here the point is to have good out-of-the-box declarative support for very typical APK-based attachments.

Quoted 4 lines

Thanks,
Daniel

[...]