Skip to contentSkip to conversation

Add a dt_provider_discover() function

PATCH Superseded [email protected] 2 participants 9 messages Oct 16, 2024 – Oct 28, 2024

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 2/19 Add a dt_provider_discover() function

From: Eugene Loh <[email protected]>

It is possible for new providers and probes to appear after dtrace_go(). Add a function that can be called for such discovery, on a per-provider basis. Each provider that supports such discovery needs to provide:

*) a function discover() that performs discovery on behalf of the provider

*) a function add_probe() that adds a particular (discovered) probe

To support looping over providers, move dt_providers[] from dt_open.c to dt_provider.c and extern it in dt_provider.h.

---
 libdtrace/dt_bpf.c      |  6 +++++
 libdtrace/dt_open.c     | 22 +---------------
 libdtrace/dt_provider.c | 57 +++++++++++++++++++++++++++++++++++++++++
 libdtrace/dt_provider.h |  6 +++++
 libdtrace/dt_work.c     |  3 +++
 5 files changed, 73 insertions(+), 21 deletions(-)
libdtrace/dt_bpf.c +6 -0
index ad11d178e..7a36b6187 100644
@@ -1275,6 +1275,12 @@ dt_bpf_load_progs(dtrace_hdl_t *dtp, uint_t cflags)
 		int		fd;
 		int		rc = -1;
 
+		if (prp->prov->impl->prog_type == BPF_PROG_TYPE_UNSPEC) {
+			if (prp->prov->impl->add_probe)
+				prp->prov->impl->add_probe(dtp, prp);
+			continue;
+		}
+
 		dp = prp->difo;
 		if (dp == NULL)
 			continue;
libdtrace/dt_open.c +1 -21
index 848141ddc..1f586fc4f 100644
@@ -61,26 +61,6 @@ const dt_version_t _dtrace_versions[] = {
 	0
 };
 
-/*
- * List of provider modules that register providers and probes.  A single
- * provider module may create multiple providers.
- */
-static const dt_provimpl_t *dt_providers[] = {
-	&dt_dtrace,		/* list dt_dtrace first */
-	&dt_cpc,
-	&dt_fbt_fprobe,
-	&dt_io,
-	&dt_ip,
-	&dt_lockstat,
-	&dt_proc,
-	&dt_profile,
-	&dt_rawtp,
-	&dt_sched,
-	&dt_sdt,
-	&dt_syscall,
-	&dt_uprobe,
-};
-
 /*
  * Table of global identifiers.  This is used to populate the global identifier
  * hash when a new dtrace client open occurs.  For more info see dt_ident.h.
@@ -1210,7 +1190,7 @@ dtrace_init(dtrace_hdl_t *dtp)
 	 * known providers.
 	 */
 	dt_probe_init(dtp);
-	for (i = 0; i < ARRAY_SIZE(dt_providers); i++) {
+	for (i = 0; dt_providers[i]; i++) {
 		int	n;
 
 		n = dt_providers[i]->populate(dtp);
libdtrace/dt_provider.c +57 -0

70 lines not on this page · raw message

libdtrace/dt_provider.h +6 -0
index 17b1844cb..384d4fd33 100644
@@ -62,6 +62,9 @@ typedef struct dt_provimpl {
 	int (*probe_info)(dtrace_hdl_t *dtp,	/* get probe info */
 			  const struct dt_probe *prp,
 			  int *argcp, dt_argdesc_t **argvp);
+	int (*discover)(dtrace_hdl_t *dtp);	/* discover new probes */
+	int (*add_probe)(dtrace_hdl_t *dtp,	/* add a new probe */
+			 struct dt_probe *prp);
 	void (*detach)(dtrace_hdl_t *dtp,	/* probe cleanup */
 		       const struct dt_probe *prb);
 	void (*probe_destroy)(dtrace_hdl_t *dtp, /* free probe data */
@@ -86,6 +89,8 @@ extern dt_provimpl_t dt_sdt;
 extern dt_provimpl_t dt_syscall;
 extern dt_provimpl_t dt_uprobe;
 
+extern const dt_provimpl_t *dt_providers[];
+
 typedef struct dt_provider {
 	dt_list_t pv_list;		/* list forward/back pointers */
 	struct dt_hentry he;		/* htab links */
@@ -110,6 +115,7 @@ extern dt_provider_t *dt_provider_create(dtrace_hdl_t *, const char *,
 					 const dt_provimpl_t *,
 					 const dtrace_pattr_t *, void *);
 extern int dt_provider_xref(dtrace_hdl_t *, dt_provider_t *, id_t);
+extern int dt_provider_discover(dtrace_hdl_t *dtp);
 
 #ifdef	__cplusplus
 }
libdtrace/dt_work.c +3 -0
index 11335345a..5318cdb7e 100644
@@ -362,6 +362,9 @@ dtrace_work(dtrace_hdl_t *dtp, FILE *fp, dtrace_consume_probe_f *pfunc,
 	dtrace_workstatus_t	rval;
 	int			gen;
 
+	if (dt_provider_discover(dtp) < 0)
+		return DTRACE_WORKSTATUS_ERROR;
+
 	switch (dtrace_status(dtp)) {
 	case DTRACE_STATUS_EXITED:
 	case DTRACE_STATUS_STOPPED:
Signature · 1 line
2.43.5
#1 lore raw

PATCH v6 7/19 Create the BPF usdt_names and usdt_prids maps

From: Eugene Loh <[email protected]>

As USDT processes come and go, the set of overlying probes for an underlying probe will change. Hence, we will move to a scheme in which an underlying probe program will walk all possible clauses that any of its possible overlying probes might call, for each overlying probe only executing the clauses that apply, using a bitmask.

In this patch, we create and update the BPF "usdt_prids" map. This is a hash map, where:

*) the key (size: dtp->dt_usdt_pridsmap_ksz) comprises
- the PID of the firing process
- the PRID of the underlying probe

*) the value (size: dtp->dt_usdt_pridsmap_vsz) comprises
- the PRID over the overlying USDT probe
- a bit mask indicating which clauses should be called

As USDT processes start up, we also add new overlying USDT probes, whose name elements must be retrievable by prid for the built-in variables probeprov, probemod, probefunc, and probename. While those strings are currently in the BPF "strtab" map, that monolithic table cannot safely be updated by the consumer while the kernel might be reading the map. Therefore, we introduce a new variable, dtp->dt_nprobes, that records the number of probes at the time of dtrace_go(). For a prid < dtp->nprobes, get_bvar() uses the existing scheme to retrieve probe name elements. For newer prids, necessarily USDT probes, use the new BPF "usdt_names" map instead.

The number of new USDT probes that can be accommodated at any one time is set by the new "nusdtprobes" option.

The size of the bit mask limits the number of clauses, currently 64, an underlying probe might call. This is relatively easy to extend; nevertheless, that work is left for a future patch.

We also want to be able to add new underlying probes, even after dtrace_go() has finished. Therefore, make dt_bpf_load_prog() callable outside of dt_bpf.c and call dt_construct(), dt_link(), dt_bpf_load_prog(), and attach() to add any new underlying probes.

---
 bpf/get_bvar.c                   |  72 ++++---
 include/dtrace/options_defines.h |   4 +-
 libdtrace/dt_bpf.c               |  38 +++-
 libdtrace/dt_bpf.h               |   4 +
 libdtrace/dt_bpf_maps.h          |   9 +
 libdtrace/dt_cc.c                |   3 +
 libdtrace/dt_dlibs.c             |   3 +
 libdtrace/dt_impl.h              |   5 +-
 libdtrace/dt_open.c              |   8 +
 libdtrace/dt_options.c           |   1 +
 libdtrace/dt_prov_uprobe.c       | 331 ++++++++++++++++++++++++++++++-
 libdtrace/dt_provider.c          |   1 +
 libdtrace/dt_provider.h          |   1 +
 13 files changed, 450 insertions(+), 30 deletions(-)
bpf/get_bvar.c +47 -25
index 37f29a591..ceb700bf1 100644
@@ -22,6 +22,7 @@
 extern struct bpf_map_def cpuinfo;
 extern struct bpf_map_def probes;
 extern struct bpf_map_def state;
+extern struct bpf_map_def usdt_names;
 
 extern uint64_t PC;
 extern uint64_t STBSZ;
@@ -29,6 +30,7 @@ extern uint64_t STKSIZ;
 extern uint64_t BOOTTM;
 extern uint64_t STACK_OFF;
 extern uint64_t STACK_SKIP;
+extern uint64_t NPROBES;
 
 #define error(dctx, fault, illval) \
 	({ \
@@ -122,32 +124,52 @@ noinline uint64_t dt_get_bvar(const dt_dctx_t *dctx, uint32_t id, uint32_t idx)
 	case DIF_VAR_PROBEMOD:
 	case DIF_VAR_PROBEFUNC:
 	case DIF_VAR_PROBENAME: {
-		uint32_t	key;
-		dt_bpf_probe_t	*pinfo;
-		uint64_t	off;
-
-		key = mst->prid;
-		pinfo = bpf_map_lookup_elem(&probes, &key);
-		if (pinfo == NULL)
-			return (uint64_t)dctx->strtab;
-
-		switch (id) {
-		case DIF_VAR_PROBEPROV:
-			off = pinfo->prv;
-			break;
-		case DIF_VAR_PROBEMOD:
-			off = pinfo->mod;
-			break;
-		case DIF_VAR_PROBEFUNC:
-			off = pinfo->fun;
-			break;
-		case DIF_VAR_PROBENAME:
-			off = pinfo->prb;
+		uint32_t	key = mst->prid;
+
+		if (key < ((uint64_t)&NPROBES)) {
+			dt_bpf_probe_t	*pinfo;
+			uint64_t	off;
+
+			pinfo = bpf_map_lookup_elem(&probes, &key);
+			if (pinfo == NULL)
+				return (uint64_t)dctx->strtab;
+
+			switch (id) {
+			case DIF_VAR_PROBEPROV:
+				off = pinfo->prv;
+				break;
+			case DIF_VAR_PROBEMOD:
+				off = pinfo->mod;
+				break;
+			case DIF_VAR_PROBEFUNC:
+				off = pinfo->fun;
+				break;
+			case DIF_VAR_PROBENAME:
+				off = pinfo->prb;
+			}
+			if (off > (uint64_t)&STBSZ)
+				return (uint64_t)dctx->strtab;
+
+			return (uint64_t)(dctx->strtab + off);
+		} else {
+			char *s;
+
+			s = bpf_map_lookup_elem(&usdt_names, &key);
+			if (s == NULL)
+				return (uint64_t)dctx->strtab;
+
+			switch (id) {
+			case DIF_VAR_PROBENAME:
+				s += DTRACE_FUNCNAMELEN;
+			case DIF_VAR_PROBEFUNC:
+				s += DTRACE_MODNAMELEN;
+			case DIF_VAR_PROBEMOD:
+				s += DTRACE_PROVNAMELEN;
+			case DIF_VAR_PROBEPROV:
+			}
+
+			return (uint64_t)s;
 		}
-		if (off > (uint64_t)&STBSZ)
-			return (uint64_t)dctx->strtab;
-
-		return (uint64_t)(dctx->strtab + off);
 	}
 	case DIF_VAR_PID: {
 		uint64_t	val = bpf_get_current_pid_tgid();
include/dtrace/options_defines.h +3 -1
index 80246be8c..7a49b89f2 100644
@@ -63,7 +63,9 @@
 #define	DTRACEOPT_SCRATCHSIZE	33	/* max scratch size permitted */
 #define	DTRACEOPT_LOCKMEM	34	/* max locked memory */
 #define	DTRACEOPT_PRINTSIZE	35	/* max # bytes printed by print() action */
-#define	DTRACEOPT_MAX		36	/* number of options */
+#define	DTRACEOPT_NUSDTPROBES	36	/* max number of (added) USDT probes */
+
+#define	DTRACEOPT_MAX		37	/* number of options */
 
 #define	DTRACEOPT_UNSET		(dtrace_optval_t)-2	/* unset option */
 
libdtrace/dt_bpf.c +37 -1
index 7a36b6187..1024867c9 100644
@@ -940,6 +940,41 @@ gmap_create_probes(dtrace_hdl_t *dtp)
 	return 0;
 }
 
+/*
+ * Create the 'usdt_names' and 'usdt_prids' BPF maps.
+ *
+ * 'usdt_names':  a global hash map indexed by PRID and whose value has probe
+ *                name elements at fixed offsets within the value.  This map
+ *                is used for get_bvar() to look up probe name elements for
+ *                any prid that was created after dtrace_go().
+ *
+ * 'usdt_prids':  a global hash map indexed by (pid, underlying probe ID).
+ *                The value is a probe ID for the overlying USDT probe and
+ *                a bit mask indicating which clauses to execute for this pid.
+ *
+ *                For a given (pid, PRID) key, there can be at most one
+ *                overlying USDT probe.
+ */
+static int
+gmap_create_usdt(dtrace_hdl_t *dtp)
+{
+	size_t nusdtprobes = dtp->dt_options[DTRACEOPT_NUSDTPROBES];
+
+	dtp->dt_usdt_namesmap_fd = create_gmap(dtp, "usdt_names", BPF_MAP_TYPE_HASH,
+	    sizeof(dtrace_id_t), DTRACE_FULLNAMELEN, nusdtprobes);
+	if (dtp->dt_usdt_namesmap_fd == -1)
+		return -1;
+
+	dtp->dt_usdt_pridsmap_fd = create_gmap(dtp, "usdt_prids", BPF_MAP_TYPE_HASH,
+	    sizeof(usdt_prids_map_key_t), sizeof(usdt_prids_map_val_t), nusdtprobes);
+	if (dtp->dt_usdt_pridsmap_fd == -1)
+		return -1;
+
+	dtp->dt_nprobes = dtp->dt_probe_id;
+
+	return 0;
+}
+
 /*
  * Create the 'gvars' BPF map.
  *
@@ -1045,6 +1080,7 @@ dt_bpf_gmap_create(dtrace_hdl_t *dtp)
 	CREATE_MAP(scratchmem)
 	CREATE_MAP(strtab)
 	CREATE_MAP(probes)
+	CREATE_MAP(usdt)
 	CREATE_MAP(gvars)
 	CREATE_MAP(lvars)
 	CREATE_MAP(dvars)
@@ -1093,7 +1129,7 @@ dt_bpf_reloc_prog(dtrace_hdl_t *dtp, const dtrace_difo_t *dp)
  *
  * Note that DTrace generates BPF programs that are licensed under the GPL.
  */
-static int
+int
 dt_bpf_load_prog(dtrace_hdl_t *dtp, const dt_probe_t *prp,
 		 const dtrace_difo_t *dp, uint_t cflags)
 {
libdtrace/dt_bpf.h +4 -0
index 5716d2320..6518de663 100644
@@ -14,6 +14,7 @@
 #include <dtrace/difo.h>
 #include <dt_btf.h>
 #include <dt_impl.h>
+#include <dt_probe.h>
 
 struct dtrace_hdl;
 
@@ -54,6 +55,7 @@ extern "C" {
 #define DT_CONST_ZERO_OFF		20
 #define DT_CONST_STACK_OFF		21
 #define DT_CONST_STACK_SKIP		22
+#define DT_CONST_NPROBES		23
 
 #define DT_BPF_LOG_SIZE_DEFAULT	(UINT32_MAX >> 8)
 #define DT_BPF_LOG_SIZE_SMALL	4096
@@ -88,6 +90,8 @@ extern int dt_bpf_prog_load(struct dtrace_hdl *, const struct dt_probe *prp,
 			    size_t sz);
 extern int dt_bpf_raw_tracepoint_open(const void *tp, int fd);
 extern int dt_bpf_make_progs(struct dtrace_hdl *, uint_t);
+extern int dt_bpf_load_prog(dtrace_hdl_t *dtp, const dt_probe_t *prp,
+			    const dtrace_difo_t *dp, uint_t cflags);
 extern int dt_bpf_load_progs(struct dtrace_hdl *, uint_t);
 extern void dt_bpf_init(struct dtrace_hdl *dtp);
 
libdtrace/dt_bpf_maps.h +9 -0
index 0dd36b16c..3a42ee5ad 100644
@@ -42,6 +42,15 @@ struct dt_bpf_cpuinfo {
 	uint64_t	lockstat_stime;	/* lockstat: spin time */
 };
 
+typedef struct usdt_prids_map_key {
+	int		pid;		/* should be pid_t, unistd.h? */
+	uint32_t	uprid;		/* should be dtrace_id_t, sys/dtrace_types.h */
+} usdt_prids_map_key_t;
+typedef struct usdt_prids_map_val {
+	uint32_t	prid;		/* should be dtrace_id_t, sys/dtrace_types.h */
+	long long	mask;
+} usdt_prids_map_val_t;
+
 #ifdef  __cplusplus
 }
 #endif
libdtrace/dt_cc.c +3 -0
index 4202771a9..fa16f8e5f 100644
@@ -1064,6 +1064,9 @@ dt_link_construct(dtrace_hdl_t *dtp, const dt_probe_t *prp, dtrace_difo_t *dp,
 				nrp->dofr_data = sizeof(uint64_t)
 				    * dtp->dt_options[DTRACEOPT_MAXFRAMES];
 				continue;
+			case DT_CONST_NPROBES:
+				nrp->dofr_data = dtp->dt_nprobes;
+				continue;
 			case DT_CONST_BOOTTM:
 				if (boottime == 0 && get_boottime())
 					return -1;
libdtrace/dt_dlibs.c +3 -0
index ba4d4abef..07d22afdc 100644
@@ -71,6 +71,8 @@ static const dt_ident_t		dt_bpf_symbols[] = {
 	DT_BPF_SYMBOL(state, DT_IDENT_PTR),
 	DT_BPF_SYMBOL(strtab, DT_IDENT_PTR),
 	DT_BPF_SYMBOL(tuples, DT_IDENT_PTR),
+	DT_BPF_SYMBOL(usdt_names, DT_IDENT_PTR),
+	DT_BPF_SYMBOL(usdt_prids, DT_IDENT_PTR),
 
 	/* BPF internal identifiers */
 	DT_BPF_SYMBOL_ID(PRID, DT_IDENT_SCALAR, DT_CONST_PRID),
@@ -95,6 +97,7 @@ static const dt_ident_t		dt_bpf_symbols[] = {
 	DT_BPF_SYMBOL_ID(ZERO_OFF, DT_IDENT_SCALAR, DT_CONST_ZERO_OFF),
 	DT_BPF_SYMBOL_ID(STACK_OFF, DT_IDENT_SCALAR, DT_CONST_STACK_OFF),
 	DT_BPF_SYMBOL_ID(STACK_SKIP, DT_IDENT_SCALAR, DT_CONST_STACK_SKIP),
+	DT_BPF_SYMBOL_ID(NPROBES, DT_IDENT_SCALAR, DT_CONST_NPROBES),
 
 	/* End-of-list marker */
 	{ NULL, }
libdtrace/dt_impl.h +4 -1
index 340dc1960..36afa4c68 100644
@@ -319,7 +319,8 @@ struct dtrace_hdl {
 	 */
 	struct dt_probe **dt_probes; /* array of probes */
 	size_t dt_probes_sz;	/* size of array of probes */
-	uint32_t dt_probe_id;	/* next available probe id */
+	dtrace_id_t dt_probe_id; /* next available probe id */
+	dtrace_id_t dt_nprobes;	/* number of probes, at dtrace_go() */
 
 	struct dt_probe *dt_error; /* ERROR probe */
 
@@ -389,6 +390,8 @@ struct dtrace_hdl {
 	int dt_aggmap_fd;	/* file descriptor for the 'aggs' BPF map */
 	int dt_genmap_fd;	/* file descriptor for the 'agggen' BPF map */
 	int dt_cpumap_fd;	/* file descriptor for the 'cpuinfo' BPF map */
+	int dt_usdt_pridsmap_fd; /* file descriptor for the 'usdt_prids' BPF map */
+	int dt_usdt_namesmap_fd; /* file descriptor for the 'usdt_names' BPF map */
 	dtrace_handle_err_f *dt_errhdlr; /* error handler, if any */
 	void *dt_errarg;	/* error handler argument */
 	dtrace_handle_drop_f *dt_drophdlr; /* drop handler, if any */
libdtrace/dt_open.c +8 -0
index 1f586fc4f..e1972aa82 100644
@@ -797,6 +797,14 @@ dt_vopen(int version, int flags, int *errp,
 	dtp->dt_options[DTRACEOPT_SWITCHRATE] = 0;
 	dtp->dt_options[DTRACEOPT_AGGRATE] = 0;
 
+	/*
+	 * Set the default maximum number of (added) USDT probes.
+	 */
+	dtp->dt_options[DTRACEOPT_NUSDTPROBES] = 256;
+
+	/*
+	 * Pre-processor.
+	 */
 	dtp->dt_cpp_argv[0] = (char *)strbasename(dtp->dt_cpp_path);
 
 	snprintf(isadef, sizeof(isadef), "-D__SUNW_D_%u",
libdtrace/dt_options.c +1 -0
index ec53358b3..377b396b0 100644
@@ -1137,6 +1137,7 @@ static const dt_option_t _dtrace_rtoptions[] = {
 	{ "jstackstrsize", dt_opt_size, DTRACEOPT_JSTACKSTRSIZE },
 	{ "lockmem", dt_opt_lockmem, DTRACEOPT_LOCKMEM },
 	{ "maxframes", dt_opt_runtime, DTRACEOPT_MAXFRAMES },
+	{ "nusdtprobes", dt_opt_runtime, DTRACEOPT_NUSDTPROBES },
 	{ "nspec", dt_opt_runtime, DTRACEOPT_NSPEC },
 	{ "pcapsize", dt_opt_pcapsize, DTRACEOPT_PCAPSIZE },
 	{ "scratchsize", dt_opt_scratchsize, DTRACEOPT_SCRATCHSIZE },
libdtrace/dt_prov_uprobe.c +329 -2

368 lines not on this page · raw message

libdtrace/dt_provider.c +1 -0
index 5f9766183..dd17627ba 100644
@@ -41,6 +41,7 @@ const dt_provimpl_t *dt_providers[] = {
 	&dt_sdt,
 	&dt_syscall,
 	&dt_uprobe,
+	&dt_usdt,
 	NULL
 };
 
libdtrace/dt_provider.h +1 -0
index 384d4fd33..8f143dcea 100644
@@ -88,6 +88,7 @@ extern dt_provimpl_t dt_sched;
 extern dt_provimpl_t dt_sdt;
 extern dt_provimpl_t dt_syscall;
 extern dt_provimpl_t dt_uprobe;
+extern dt_provimpl_t dt_usdt;
 
 extern const dt_provimpl_t *dt_providers[];
 
Signature · 1 line
2.43.5
Reviewed-by on v6 7/19
#2 lore raw

Re: [PATCH v6 07/19] Create the BPF usdt_names and usdt_prids maps

Quoted 38 lines · On Wed, Oct 16, 2024 at 12:01:37PM -0400, [email protected] wrote:

From: Eugene Loh <[email protected]>

As USDT processes come and go, the set of overlying probes for an underlying probe will change. Hence, we will move to a scheme in which an underlying probe program will walk all possible clauses that any of its possible overlying probes might call, for each overlying probe only executing the clauses that apply, using a bitmask.

In this patch, we create and update the BPF "usdt_prids" map. This is a hash map, where:

*) the key (size: dtp->dt_usdt_pridsmap_ksz) comprises
- the PID of the firing process
- the PRID of the underlying probe

*) the value (size: dtp->dt_usdt_pridsmap_vsz) comprises
- the PRID over the overlying USDT probe
- a bit mask indicating which clauses should be called

As USDT processes start up, we also add new overlying USDT probes, whose name elements must be retrievable by prid for the built-in variables probeprov, probemod, probefunc, and probename. While those strings are currently in the BPF "strtab" map, that monolithic table cannot safely be updated by the consumer while the kernel might be reading the map. Therefore, we introduce a new variable, dtp->dt_nprobes, that records the number of probes at the time of dtrace_go(). For a prid < dtp->nprobes, get_bvar() uses the existing scheme to retrieve probe name elements. For newer prids, necessarily USDT probes, use the new BPF "usdt_names" map instead.

The number of new USDT probes that can be accommodated at any one time is set by the new "nusdtprobes" option.

The size of the bit mask limits the number of clauses, currently 64, an underlying probe might call. This is relatively easy to extend; nevertheless, that work is left for a future patch.

Quoted 6 lines

We also want to be able to add new underlying probes, even after dtrace_go() has finished. Therefore, make dt_bpf_load_prog() callable outside of dt_bpf.c and call dt_construct(), dt_link(), dt_bpf_load_prog(), and attach() to add any new underlying probes.

... small comment below...

Quoted 510 lines
---
 bpf/get_bvar.c                   |  72 ++++---
 include/dtrace/options_defines.h |   4 +-
 libdtrace/dt_bpf.c               |  38 +++-
 libdtrace/dt_bpf.h               |   4 +
 libdtrace/dt_bpf_maps.h          |   9 +
 libdtrace/dt_cc.c                |   3 +
 libdtrace/dt_dlibs.c             |   3 +
 libdtrace/dt_impl.h              |   5 +-
 libdtrace/dt_open.c              |   8 +
 libdtrace/dt_options.c           |   1 +
 libdtrace/dt_prov_uprobe.c       | 331 ++++++++++++++++++++++++++++++-
 libdtrace/dt_provider.c          |   1 +
 libdtrace/dt_provider.h          |   1 +
 13 files changed, 450 insertions(+), 30 deletions(-)
bpf/get_bvar.c +47 -25
index 37f29a591..ceb700bf1 100644
@@ -22,6 +22,7 @@
 extern struct bpf_map_def cpuinfo;
 extern struct bpf_map_def probes;
 extern struct bpf_map_def state;
+extern struct bpf_map_def usdt_names;
 
 extern uint64_t PC;
 extern uint64_t STBSZ;
@@ -29,6 +30,7 @@ extern uint64_t STKSIZ;
 extern uint64_t BOOTTM;
 extern uint64_t STACK_OFF;
 extern uint64_t STACK_SKIP;
+extern uint64_t NPROBES;
 
 #define error(dctx, fault, illval) \
 	({ \
@@ -122,32 +124,52 @@ noinline uint64_t dt_get_bvar(const dt_dctx_t *dctx, uint32_t id, uint32_t idx)
 	case DIF_VAR_PROBEMOD:
 	case DIF_VAR_PROBEFUNC:
 	case DIF_VAR_PROBENAME: {
-		uint32_t	key;
-		dt_bpf_probe_t	*pinfo;
-		uint64_t	off;
-
-		key = mst->prid;
-		pinfo = bpf_map_lookup_elem(&probes, &key);
-		if (pinfo == NULL)
-			return (uint64_t)dctx->strtab;
-
-		switch (id) {
-		case DIF_VAR_PROBEPROV:
-			off = pinfo->prv;
-			break;
-		case DIF_VAR_PROBEMOD:
-			off = pinfo->mod;
-			break;
-		case DIF_VAR_PROBEFUNC:
-			off = pinfo->fun;
-			break;
-		case DIF_VAR_PROBENAME:
-			off = pinfo->prb;
+		uint32_t	key = mst->prid;
+
+		if (key < ((uint64_t)&NPROBES)) {
+			dt_bpf_probe_t	*pinfo;
+			uint64_t	off;
+
+			pinfo = bpf_map_lookup_elem(&probes, &key);
+			if (pinfo == NULL)
+				return (uint64_t)dctx->strtab;
+
+			switch (id) {
+			case DIF_VAR_PROBEPROV:
+				off = pinfo->prv;
+				break;
+			case DIF_VAR_PROBEMOD:
+				off = pinfo->mod;
+				break;
+			case DIF_VAR_PROBEFUNC:
+				off = pinfo->fun;
+				break;
+			case DIF_VAR_PROBENAME:
+				off = pinfo->prb;
+			}
+			if (off > (uint64_t)&STBSZ)
+				return (uint64_t)dctx->strtab;
+
+			return (uint64_t)(dctx->strtab + off);
+		} else {
+			char *s;
+
+			s = bpf_map_lookup_elem(&usdt_names, &key);
+			if (s == NULL)
+				return (uint64_t)dctx->strtab;
+
+			switch (id) {
+			case DIF_VAR_PROBENAME:
+				s += DTRACE_FUNCNAMELEN;
+			case DIF_VAR_PROBEFUNC:
+				s += DTRACE_MODNAMELEN;
+			case DIF_VAR_PROBEMOD:
+				s += DTRACE_PROVNAMELEN;
+			case DIF_VAR_PROBEPROV:
+			}
+
+			return (uint64_t)s;
 		}
-		if (off > (uint64_t)&STBSZ)
-			return (uint64_t)dctx->strtab;
-
-		return (uint64_t)(dctx->strtab + off);
 	}
 	case DIF_VAR_PID: {
 		uint64_t	val = bpf_get_current_pid_tgid();
include/dtrace/options_defines.h +3 -1
index 80246be8c..7a49b89f2 100644
@@ -63,7 +63,9 @@
 #define	DTRACEOPT_SCRATCHSIZE	33	/* max scratch size permitted */
 #define	DTRACEOPT_LOCKMEM	34	/* max locked memory */
 #define	DTRACEOPT_PRINTSIZE	35	/* max # bytes printed by print() action */
-#define	DTRACEOPT_MAX		36	/* number of options */
+#define	DTRACEOPT_NUSDTPROBES	36	/* max number of (added) USDT probes */
+
+#define	DTRACEOPT_MAX		37	/* number of options */
 
 #define	DTRACEOPT_UNSET		(dtrace_optval_t)-2	/* unset option */
 
libdtrace/dt_bpf.c +37 -1
index 7a36b6187..1024867c9 100644
@@ -940,6 +940,41 @@ gmap_create_probes(dtrace_hdl_t *dtp)
 	return 0;
 }
 
+/*
+ * Create the 'usdt_names' and 'usdt_prids' BPF maps.
+ *
+ * 'usdt_names':  a global hash map indexed by PRID and whose value has probe
+ *                name elements at fixed offsets within the value.  This map
+ *                is used for get_bvar() to look up probe name elements for
+ *                any prid that was created after dtrace_go().
+ *
+ * 'usdt_prids':  a global hash map indexed by (pid, underlying probe ID).
+ *                The value is a probe ID for the overlying USDT probe and
+ *                a bit mask indicating which clauses to execute for this pid.
+ *
+ *                For a given (pid, PRID) key, there can be at most one
+ *                overlying USDT probe.
+ */
+static int
+gmap_create_usdt(dtrace_hdl_t *dtp)
+{
+	size_t nusdtprobes = dtp->dt_options[DTRACEOPT_NUSDTPROBES];
+
+	dtp->dt_usdt_namesmap_fd = create_gmap(dtp, "usdt_names", BPF_MAP_TYPE_HASH,
+	    sizeof(dtrace_id_t), DTRACE_FULLNAMELEN, nusdtprobes);
+	if (dtp->dt_usdt_namesmap_fd == -1)
+		return -1;
+
+	dtp->dt_usdt_pridsmap_fd = create_gmap(dtp, "usdt_prids", BPF_MAP_TYPE_HASH,
+	    sizeof(usdt_prids_map_key_t), sizeof(usdt_prids_map_val_t), nusdtprobes);
+	if (dtp->dt_usdt_pridsmap_fd == -1)
+		return -1;
+
+	dtp->dt_nprobes = dtp->dt_probe_id;
+
+	return 0;
+}
+
 /*
  * Create the 'gvars' BPF map.
  *
@@ -1045,6 +1080,7 @@ dt_bpf_gmap_create(dtrace_hdl_t *dtp)
 	CREATE_MAP(scratchmem)
 	CREATE_MAP(strtab)
 	CREATE_MAP(probes)
+	CREATE_MAP(usdt)
 	CREATE_MAP(gvars)
 	CREATE_MAP(lvars)
 	CREATE_MAP(dvars)
@@ -1093,7 +1129,7 @@ dt_bpf_reloc_prog(dtrace_hdl_t *dtp, const dtrace_difo_t *dp)
  *
  * Note that DTrace generates BPF programs that are licensed under the GPL.
  */
-static int
+int
 dt_bpf_load_prog(dtrace_hdl_t *dtp, const dt_probe_t *prp,
 		 const dtrace_difo_t *dp, uint_t cflags)
 {
libdtrace/dt_bpf.h +4 -0
index 5716d2320..6518de663 100644
@@ -14,6 +14,7 @@
 #include <dtrace/difo.h>
 #include <dt_btf.h>
 #include <dt_impl.h>
+#include <dt_probe.h>
 
 struct dtrace_hdl;
 
@@ -54,6 +55,7 @@ extern "C" {
 #define DT_CONST_ZERO_OFF		20
 #define DT_CONST_STACK_OFF		21
 #define DT_CONST_STACK_SKIP		22
+#define DT_CONST_NPROBES		23
 
 #define DT_BPF_LOG_SIZE_DEFAULT	(UINT32_MAX >> 8)
 #define DT_BPF_LOG_SIZE_SMALL	4096
@@ -88,6 +90,8 @@ extern int dt_bpf_prog_load(struct dtrace_hdl *, const struct dt_probe *prp,
 			    size_t sz);
 extern int dt_bpf_raw_tracepoint_open(const void *tp, int fd);
 extern int dt_bpf_make_progs(struct dtrace_hdl *, uint_t);
+extern int dt_bpf_load_prog(dtrace_hdl_t *dtp, const dt_probe_t *prp,
+			    const dtrace_difo_t *dp, uint_t cflags);
 extern int dt_bpf_load_progs(struct dtrace_hdl *, uint_t);
 extern void dt_bpf_init(struct dtrace_hdl *dtp);
 
libdtrace/dt_bpf_maps.h +9 -0
index 0dd36b16c..3a42ee5ad 100644
@@ -42,6 +42,15 @@ struct dt_bpf_cpuinfo {
 	uint64_t	lockstat_stime;	/* lockstat: spin time */
 };
 
+typedef struct usdt_prids_map_key {
+	int		pid;		/* should be pid_t, unistd.h? */
+	uint32_t	uprid;		/* should be dtrace_id_t, sys/dtrace_types.h */
+} usdt_prids_map_key_t;
+typedef struct usdt_prids_map_val {
+	uint32_t	prid;		/* should be dtrace_id_t, sys/dtrace_types.h */
+	long long	mask;
+} usdt_prids_map_val_t;
+
 #ifdef  __cplusplus
 }
 #endif
libdtrace/dt_cc.c +3 -0
index 4202771a9..fa16f8e5f 100644
@@ -1064,6 +1064,9 @@ dt_link_construct(dtrace_hdl_t *dtp, const dt_probe_t *prp, dtrace_difo_t *dp,
 				nrp->dofr_data = sizeof(uint64_t)
 				    * dtp->dt_options[DTRACEOPT_MAXFRAMES];
 				continue;
+			case DT_CONST_NPROBES:
+				nrp->dofr_data = dtp->dt_nprobes;
+				continue;
 			case DT_CONST_BOOTTM:
 				if (boottime == 0 && get_boottime())
 					return -1;
libdtrace/dt_dlibs.c +3 -0
index ba4d4abef..07d22afdc 100644
@@ -71,6 +71,8 @@ static const dt_ident_t		dt_bpf_symbols[] = {
 	DT_BPF_SYMBOL(state, DT_IDENT_PTR),
 	DT_BPF_SYMBOL(strtab, DT_IDENT_PTR),
 	DT_BPF_SYMBOL(tuples, DT_IDENT_PTR),
+	DT_BPF_SYMBOL(usdt_names, DT_IDENT_PTR),
+	DT_BPF_SYMBOL(usdt_prids, DT_IDENT_PTR),
 
 	/* BPF internal identifiers */
 	DT_BPF_SYMBOL_ID(PRID, DT_IDENT_SCALAR, DT_CONST_PRID),
@@ -95,6 +97,7 @@ static const dt_ident_t		dt_bpf_symbols[] = {
 	DT_BPF_SYMBOL_ID(ZERO_OFF, DT_IDENT_SCALAR, DT_CONST_ZERO_OFF),
 	DT_BPF_SYMBOL_ID(STACK_OFF, DT_IDENT_SCALAR, DT_CONST_STACK_OFF),
 	DT_BPF_SYMBOL_ID(STACK_SKIP, DT_IDENT_SCALAR, DT_CONST_STACK_SKIP),
+	DT_BPF_SYMBOL_ID(NPROBES, DT_IDENT_SCALAR, DT_CONST_NPROBES),
 
 	/* End-of-list marker */
 	{ NULL, }
libdtrace/dt_impl.h +4 -1
index 340dc1960..36afa4c68 100644
@@ -319,7 +319,8 @@ struct dtrace_hdl {
 	 */
 	struct dt_probe **dt_probes; /* array of probes */
 	size_t dt_probes_sz;	/* size of array of probes */
-	uint32_t dt_probe_id;	/* next available probe id */
+	dtrace_id_t dt_probe_id; /* next available probe id */
+	dtrace_id_t dt_nprobes;	/* number of probes, at dtrace_go() */
 
 	struct dt_probe *dt_error; /* ERROR probe */
 
@@ -389,6 +390,8 @@ struct dtrace_hdl {
 	int dt_aggmap_fd;	/* file descriptor for the 'aggs' BPF map */
 	int dt_genmap_fd;	/* file descriptor for the 'agggen' BPF map */
 	int dt_cpumap_fd;	/* file descriptor for the 'cpuinfo' BPF map */
+	int dt_usdt_pridsmap_fd; /* file descriptor for the 'usdt_prids' BPF map */
+	int dt_usdt_namesmap_fd; /* file descriptor for the 'usdt_names' BPF map */
 	dtrace_handle_err_f *dt_errhdlr; /* error handler, if any */
 	void *dt_errarg;	/* error handler argument */
 	dtrace_handle_drop_f *dt_drophdlr; /* drop handler, if any */
libdtrace/dt_open.c +8 -0
index 1f586fc4f..e1972aa82 100644
@@ -797,6 +797,14 @@ dt_vopen(int version, int flags, int *errp,
 	dtp->dt_options[DTRACEOPT_SWITCHRATE] = 0;
 	dtp->dt_options[DTRACEOPT_AGGRATE] = 0;
 
+	/*
+	 * Set the default maximum number of (added) USDT probes.
+	 */
+	dtp->dt_options[DTRACEOPT_NUSDTPROBES] = 256;
+
+	/*
+	 * Pre-processor.
+	 */
 	dtp->dt_cpp_argv[0] = (char *)strbasename(dtp->dt_cpp_path);
 
 	snprintf(isadef, sizeof(isadef), "-D__SUNW_D_%u",
libdtrace/dt_options.c +1 -0
index ec53358b3..377b396b0 100644
@@ -1137,6 +1137,7 @@ static const dt_option_t _dtrace_rtoptions[] = {
 	{ "jstackstrsize", dt_opt_size, DTRACEOPT_JSTACKSTRSIZE },
 	{ "lockmem", dt_opt_lockmem, DTRACEOPT_LOCKMEM },
 	{ "maxframes", dt_opt_runtime, DTRACEOPT_MAXFRAMES },
+	{ "nusdtprobes", dt_opt_runtime, DTRACEOPT_NUSDTPROBES },
 	{ "nspec", dt_opt_runtime, DTRACEOPT_NSPEC },
 	{ "pcapsize", dt_opt_pcapsize, DTRACEOPT_PCAPSIZE },
 	{ "scratchsize", dt_opt_scratchsize, DTRACEOPT_SCRATCHSIZE },
libdtrace/dt_prov_uprobe.c +149 -2
index bb172ace2..ceb331719 100644
@@ -39,6 +39,7 @@
 #include "dt_probe.h"
 #include "dt_pid.h"
 #include "dt_string.h"
+#include "port.h"
 
 /* Provider name for the underlying probes. */
 static const char	prvname[] = "uprobe";
@@ -63,6 +64,11 @@ typedef struct list_probe {
 	dt_probe_t	*probe;
 } list_probe_t;
 
+typedef struct list_key {
+	dt_list_t		list;
+	usdt_prids_map_key_t	key;
+} list_key_t;
+
 static const dtrace_pattr_t	pattr = {
 { DTRACE_STABILITY_EVOLVING, DTRACE_STABILITY_EVOLVING, DTRACE_CLASS_ISA },
 { DTRACE_STABILITY_PRIVATE, DTRACE_STABILITY_PRIVATE, DTRACE_CLASS_UNKNOWN },
@@ -82,8 +88,15 @@ static int populate(dtrace_hdl_t *dtp)
 	    dt_provider_create(dtp, dt_uprobe_is_enabled.name,
 			       &dt_uprobe_is_enabled, &pattr, NULL) == NULL ||
 	    dt_provider_create(dtp, dt_pid.name, &dt_pid, &pattr,
-			       NULL) == NULL ||
-	    dt_provider_create(dtp, dt_usdt.name, &dt_usdt, &pattr,
+			       NULL) == NULL)
+		return -1;			/* errno already set */
+
+	return 0;
+}
+
+static int populate_usdt(dtrace_hdl_t *dtp)
+{
+	if (dt_provider_create(dtp, dt_usdt.name, &dt_usdt, &pattr,
 			       NULL) == NULL)
 		return -1;			/* errno already set */
 
@@ -123,6 +136,316 @@ static void probe_destroy(dtrace_hdl_t *dtp, void *datap)
 	free_probe_list(dtp, datap);
 }
 
+/*
+ * Disable an overlying USDT probe.
+ */
+static void probe_disable(dtrace_hdl_t *dtp, dt_probe_t *prp)
+{
+	list_probe_t	*pup;
+
+	/* Remove from enablings. */
+	dt_list_delete(&dtp->dt_enablings, prp);
+
+	/* Make it evident from the probe that it is not in enablings. */
+	((dt_list_t *)prp)->dl_prev = NULL;
+	((dt_list_t *)prp)->dl_next = NULL;
+
+	/* Free up its list of underlying probes. */
+	while ((pup = dt_list_next(prp->prv_data)) != NULL) {
+		dt_list_delete(prp->prv_data, pup);
+		dt_free(dtp, pup);
+	}
+	dt_free(dtp, prp->prv_data);
+	prp->prv_data = NULL;
+}
+
+/*
+ * Clean up stale pids from among the USDT probes.
+ */
+static int
+clean_usdt_probes(dtrace_hdl_t *dtp)
+{
+	int			fdprids = dtp->dt_usdt_pridsmap_fd;
+	int			fdnames = dtp->dt_usdt_namesmap_fd;
+	usdt_prids_map_key_t	key, nxt;
+	usdt_prids_map_val_t	val;
+	list_key_t		keys_to_delete, *elem, *elem_next;
+	dt_probe_t		*prp, *prp_next;
+
+	/* Initialize list of usdt_prids keys to delete. */
+	memset(&keys_to_delete, 0, sizeof(keys_to_delete));
+
+	/* Initialize usdt_prids key to a pid/uprid that cannot be found. */
+	key.pid = 0;
+	key.uprid = 0;
+
+	/* Loop over usdt_prids entries. */
+	while (dt_bpf_map_next_key(fdprids, &key, &nxt) == 0) {
+		memcpy(&key, &nxt, sizeof(usdt_prids_map_key_t));
+
+		if (dt_bpf_map_lookup(fdprids, &key, &val) == -1)
+			return dt_set_errno(dtp, EDT_BPF);
+
+		/* Check if the process is still running. */
+		if (!Pexists(key.pid)) {
+			/*
+			 * Delete the usdt_names entry.
+			 *
+			 * Note that a PRID might correspond to multiple
+			 * sites.  So, as we loop over usdt_prids entries,
+			 * we might delete the same usdt_names entry
+			 * multiple times.  That's okay.
+			 */
+			dt_bpf_map_delete(fdnames, &val.prid);
+
+			/*
+			 * Delete the usdt_prids entry.
+			 *
+			 * Note that we do not want to disrupt the iterator.
+			 * So we just add the key to a list and will walk
+			 * the list later for actual deletion.
+			 */
+			elem = calloc(1, sizeof(list_key_t));
+			elem->key.pid = key.pid;
+			elem->key.uprid = key.uprid;
+			dt_list_append((dt_list_t *)&keys_to_delete, elem);
+
+			continue;
+		}
+
+		/*
+		 * FIXME.  There might be another case, where the process
+		 * is still running, but some of its USDT probes are gone?
+		 * So maybe we have to check for the existence of one of
+		 *     dtrace_probedesc_t *pdp = dtp->dt_probes[val.prid]->desc;
+		 *     char *prv = ...pdp->prv minus the numerial part;
+		 *
+		 *     /run/dtrace/probes/$pid/$pdp->prv/$pdp->mod/$pdp->fun/$pdp->prb
+		 *     /run/dtrace/stash/dof-pid/$pid/0/parsed/$prv:$pdp->mod:$pdp->fun:$pdp->prb
+		 *     /run/dtrace/stash/dof-pid/$pid/.../parsed/$prv:$pdp->mod:$pdp->fun:$pdp->prb
+		 */
+	}
+
+	/*
+	 * Delete the usdt_prids keys in our list.
+	 */
+	for (elem = dt_list_next(&keys_to_delete); elem != NULL; elem = elem_next) {
+		elem_next = dt_list_next(elem);
+
+		dt_bpf_map_delete(fdprids, &elem->key);
+		free(elem);
+	}
+
+	/* Clean up enablings. */
+	for (prp = dt_list_next(&dtp->dt_enablings); prp != NULL; prp = prp_next) {
+		pid_t		pid;
+
+		prp_next = dt_list_next(prp);
+
+		/* Make sure it is an overlying USDT probe. */
+		if (prp->prov->impl != &dt_usdt)
+			continue;
+
+		/* FIXME passing in NULL pcb and dpr wreaks havoc on error reporting? */
+		/*
+		 * Nick writes:
+		 * This is a general problem with running compiler-adjacent things outside
+		 * compile time. I think we should adjust dt_pid_error() so that it works
+		 * with NULL pcb and dpr at once, probably by using the code path for
+		 * pcb != NULL and augmenting it so that it passes in NULL for the region and
+		 * filename args and 0 for the lineno if pcb is NULL. (dt_set_errmsg can
+		 * already handle this case.)
+		 */
+		pid = dt_pid_get_pid(prp->desc, dtp, NULL, NULL);
+
+		if (Pexists(pid))
+			continue;
+
+		probe_disable(dtp, prp);
+	}
+
+	return 0;
+}
+
+static int add_probe_uprobe(dtrace_hdl_t *dtp, dt_probe_t *prp)
+{
+	dtrace_difo_t   *dp;

libdtrace/dt_prov_uprobe.c static void probe_destroy(dtrace_hdl_t *dtp, void *datap) · line 282

Quoted 10 lines
libdtrace/dt_prov_uprobe.c +10 -0
+	int		cflags, fd, rc = -1;
+	dtrace_optval_t	dest_ok = DTRACEOPT_UNSET;
+
+	/*
+	 * Strictly speaking, we want the value passed in to
+	 * dtrace_go().  In practice, its flags pertain to
+	 * compilation and disassembly, which at this stage
+	 * no longer interest us.
+	 */
+	cflags = 0;

This is something we may want to look at later (maybe add a FIXME) because your assertion that cflags no longer interests us isn't entirely true because the construction of the trampoline might be something we do want to get debug output about (e.g. disassembly).

Quoted 215 lines
libdtrace/dt_prov_uprobe.c +170 -0
+
+	/* Check if the probe is already set up. */
+	if (prp->difo)
+		return 0;
+
+	/* Make program. */
+	dp = dt_construct(dtp, prp, cflags, NULL);
+	if (dp == NULL)
+		return 0;        // FIXME in dt_bpf_make_progs() this is a fatal error; should we do the same here?
+	prp->difo = dp;
+
+	/* Load program. */
+	if (dt_link(dtp, prp, dp, NULL) == -1)
+		return 0;        // FIXME in dt_bpf_load_progs() this is a fatal error; should we do the same here?
+
+	dtrace_getopt(dtp, "destructive", &dest_ok);
+	if (dp->dtdo_flags & DIFOFLG_DESTRUCTIVE &&
+	    dest_ok == DTRACEOPT_UNSET)
+		return dt_set_errno(dtp, EDT_DESTRUCTIVE);
+
+	fd = dt_bpf_load_prog(dtp, prp, dp, cflags);
+	if (fd == -1)
+		return 0;        // FIXME in dt_bpf_load_progs() this is a fatal error; should we do the same here?
+
+	if (prp->prov->impl->attach)
+		rc = prp->prov->impl->attach(dtp, prp, fd);
+
+	if (rc == -ENOTSUPP) {
+		char    *s;
+
+		close(fd);
+		if (asprintf(&s, "Failed to enable %s:%s:%s:%s",
+			      prp->desc->prv, prp->desc->mod,
+			      prp->desc->fun, prp->desc->prb) == -1)
+			return dt_set_errno(dtp, EDT_ENABLING_ERR);
+		dt_handle_rawerr(dtp, s);
+		free(s);
+	} else if (rc < 0) {
+		close(fd);
+		return dt_set_errno(dtp, EDT_ENABLING_ERR);
+	}
+
+	return 0;
+}
+
+static int add_probe_usdt(dtrace_hdl_t *dtp, dt_probe_t *prp)
+{
+	char				probnam[DTRACE_FULLNAMELEN], *p;
+	const dtrace_probedesc_t	*pdp = prp->desc;
+	int				fd = dtp->dt_usdt_namesmap_fd;
+	pid_t				pid;
+	list_probe_t			*pup;
+
+	/* Add probe name elements to usdt_names map. */
+	p = probnam;
+	memset(p, 0, sizeof(probnam));
+	snprintf(p, DTRACE_PROVNAMELEN, "%s", pdp->prv);
+	p += DTRACE_PROVNAMELEN;
+	snprintf(p, DTRACE_MODNAMELEN, "%s", pdp->mod);
+	p += DTRACE_MODNAMELEN;
+	snprintf(p, DTRACE_FUNCNAMELEN, "%s", pdp->fun);
+	p += DTRACE_FUNCNAMELEN;
+	snprintf(p, DTRACE_NAMELEN, "%s", pdp->prb);
+	if (dt_bpf_map_update(fd, &pdp->id, probnam) == -1)
+		assert(0);   // FIXME do something here
+
+	/* FIXME passing in NULL pcb and dpr wreaks havoc on error reporting? */
+	/*
+	 * Nick writes:
+	 * This is a general problem with running compiler-adjacent things outside
+	 * compile time. I think we should adjust dt_pid_error() so that it works
+	 * with NULL pcb and dpr at once, probably by using the code path for
+	 * pcb != NULL and augmenting it so that it passes in NULL for the region and
+	 * filename args and 0 for the lineno if pcb is NULL. (dt_set_errmsg can
+	 * already handle this case.)
+	 */
+	pid = dt_pid_get_pid(prp->desc, dtp, NULL, NULL);
+
+	/* Even though we just enabled this, check it's still live. */
+	if (!Pexists(pid)) {
+		probe_disable(dtp, prp);
+		dt_bpf_map_delete(fd, &pdp->id);
+
+		return 0;
+	}
+
+	/* Add prid and bit mask to usdt_prids map. */
+	for (pup = prp->prv_data; pup != NULL; pup = dt_list_next(pup)) {
+		dt_probe_t		*uprp = pup->probe;
+		long long		mask = 0, bit = 1;
+		usdt_prids_map_key_t	key;
+		usdt_prids_map_val_t	val;
+		dt_uprobe_t		*upp = uprp->prv_data;
+
+		/*
+		 * For is-enabled probes, the bit mask does not matter.
+		 * It is possible that we have this underlying probe due to
+		 * an overlying pid-offset probe and that we will not know
+		 * until later, when some new pid is created, that we also
+		 * have an overlying USDT is-enabled probe, but missing this
+		 * optimization opportunity is okay.
+		 */
+		if (uprp->prov->impl == &dt_uprobe && !(upp->flags & PP_IS_ENABLED)) {
+			int n;
+
+			for (n = 0; n < dtp->dt_stmt_nextid; n++) {
+				dtrace_stmtdesc_t *stp;
+
+				stp = dtp->dt_stmts[n];
+				if (stp == NULL)
+					continue;
+
+				if (dt_gmatch(prp->desc->prv, stp->dtsd_ecbdesc->dted_probe.prv) &&
+				    dt_gmatch(prp->desc->mod, stp->dtsd_ecbdesc->dted_probe.mod) &&
+				    dt_gmatch(prp->desc->fun, stp->dtsd_ecbdesc->dted_probe.fun) &&
+				    dt_gmatch(prp->desc->prb, stp->dtsd_ecbdesc->dted_probe.prb))
+					mask |= bit;
+
+				bit <<= 1;
+			}
+		}
+
+		key.pid = pid;
+		key.uprid = uprp->desc->id;
+
+		val.prid = prp->desc->id;
+		val.mask = mask;
+
+		// FIXME Check return value, but how should errors be handled?
+		dt_bpf_map_update(dtp->dt_usdt_pridsmap_fd, &key, &val);
+	}
+
+	return 0;
+}
+
+/*
+ * Discover new probes.
+ */
+static int discover(dtrace_hdl_t *dtp)
+{
+	int		i;
+	dt_pcb_t	pcb;
+
+	/* Clean up stale pids from among the USDT probes. */
+	clean_usdt_probes(dtp);
+
+	/* Discover new probes, placing them in dt_probes[]. */
+	/*
+	 * pcb is only used inside of dt_pid_error() to get:
+	 *     pcb->pcb_region
+	 *     pcb->pcb_filetag
+	 *     pcb->pcb_fileptr
+	 * While pcb cannot be NULL, these other things apparently can be.
+	 */
+	memset(&pcb, 0, sizeof(dt_pcb_t));
+	for (i = 0; i < dtp->dt_stmt_nextid; i++) {
+		dtrace_stmtdesc_t *stp;
+
+		stp = dtp->dt_stmts[i];
+		if (stp == NULL)
+			continue;
+		dt_pid_create_usdt_probes(&stp->dtsd_ecbdesc->dted_probe, dtp, &pcb);
+	}
+
+	return 0;
+}
 
 /*
  * Look up or create an underlying (real) probe, corresponding directly to a
@@ -782,6 +1105,7 @@ dt_provimpl_t	dt_uprobe = {
 	.probe_info	= &probe_info,
 	.detach		= &detach,
 	.probe_destroy	= &probe_destroy_underlying,
+	.add_probe	= &add_probe_uprobe,
 };
 
 /*
@@ -816,7 +1140,10 @@ dt_provimpl_t	dt_pid = {
 dt_provimpl_t	dt_usdt = {
 	.name		= "usdt",
 	.prog_type	= BPF_PROG_TYPE_UNSPEC,
+	.populate	= &populate_usdt,
 	.provide_probe	= &provide_usdt_probe,
 	.enable		= &enable_usdt,
 	.probe_destroy	= &probe_destroy,
+	.discover	= &discover,
+	.add_probe	= &add_probe_usdt,
 };
libdtrace/dt_provider.c +1 -0
index 5f9766183..dd17627ba 100644
@@ -41,6 +41,7 @@ const dt_provimpl_t *dt_providers[] = {
 	&dt_sdt,
 	&dt_syscall,
 	&dt_uprobe,
+	&dt_usdt,
 	NULL
 };
 
libdtrace/dt_provider.h +1 -0
index 384d4fd33..8f143dcea 100644
@@ -88,6 +88,7 @@ extern dt_provimpl_t dt_sched;
 extern dt_provimpl_t dt_sdt;
 extern dt_provimpl_t dt_syscall;
 extern dt_provimpl_t dt_uprobe;
+extern dt_provimpl_t dt_usdt;
 
 extern const dt_provimpl_t *dt_providers[];
 
Signature · 1 line
2.43.5
#3 lore raw

PATCH v4 9/19 Use usdt_prids map to call clauses conditionally for USDT probes

In reply to [email protected]

From: Eugene Loh <[email protected]>

This version supports only up to 64 clauses for an underlying probe, but it can be extended to more clauses.

---
 libdtrace/dt_prov_uprobe.c            | 166 ++++++++++++++++++-------
 test/unittest/usdt/tst.nusdtprobes.r  |   5 +
 test/unittest/usdt/tst.nusdtprobes.sh | 172 ++++++++++++++++++++++++++
 3 files changed, 300 insertions(+), 43 deletions(-)
 create mode 100644 test/unittest/usdt/tst.nusdtprobes.r
 create mode 100755 test/unittest/usdt/tst.nusdtprobes.sh
libdtrace/dt_prov_uprobe.c +123 -43
index b010d8687..268708179 100644
@@ -739,11 +739,16 @@ static void enable_usdt(dtrace_hdl_t *dtp, dt_probe_t *prp)
  */
 static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 {
+	dtrace_hdl_t		*dtp = pcb->pcb_hdl;
 	dt_irlist_t		*dlp = &pcb->pcb_ir;
 	const dt_probe_t	*uprp = pcb->pcb_probe;
 	const dt_uprobe_t	*upp = uprp->prv_data;
 	const list_probe_t	*pop;
 	uint_t			lbl_exit = pcb->pcb_exitlbl;
+	dt_ident_t		*usdt_prids = dt_dlib_get_map(dtp, "usdt_prids");
+	int			n;
+
+	assert(usdt_prids != NULL);
 
 	dt_cg_tramp_prologue(pcb);                             // call this only once... is PRID set/relocated correctly?
 
@@ -755,14 +760,17 @@ static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 	dt_cg_tramp_copy_regs(pcb);                            // call this only once for all clauses?
 
 	/*
-	 * Loop over overlying probes, calling clauses for those that match:
+	 * pid probes.
+	 *
+	 * Loop over overlying pid probes, calling clauses for those that match:
 	 *
-	 *	for overlying probes (that match except possibly for pid)
+	 *	for overlying pid probes (that match except possibly for pid)
 	 *		if (pid matches) {
 	 *			dctx->mst->prid = PRID1;
 	 *			< any number of clause calls >
 	 *		}
 	 */
+
 	for (pop = dt_list_next(&upp->probes); pop != NULL;
 	     pop = dt_list_next(pop)) {
 		const dt_probe_t	*prp = pop->probe;
@@ -770,6 +778,9 @@ static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 		pid_t			pid;
 		dt_ident_t		*idp;
 
+		if (prp->prov->impl != &dt_pid)
+			continue;
+
 		pid = dt_pid_get_pid(prp->desc, pcb->pcb_hdl, pcb, NULL);
 		assert(pid != -1);
 
@@ -801,6 +812,93 @@ static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 			   BPF_NOP());
 	}
 
+	/*
+	 * USDT
+	 */
+
+	/* In some cases, we know there are no USDT probes. */  // FIXME: add more checks
+	if (upp->flags & PP_IS_RETURN)
+		goto out;
+
+	dt_cg_tramp_copy_args_from_regs(pcb, 0);
+
+	/*
+	 * Retrieve the PID of the process that caused the probe to fire.
+	 */
+	emit(dlp,  BPF_CALL_HELPER(BPF_FUNC_get_current_pid_tgid));
+	emit(dlp,  BPF_ALU64_IMM(BPF_RSH, BPF_REG_0, 32));
+
+	/*
+	 * Look up in the BPF 'usdt_prids' map.  Space for the look-up key
+	 * will be used on the BPF stack:
+	 *
+	 *     offset                                       value
+	 *
+	 *     -sizeof(usdt_prids_map_key_t)                pid (in %r0)
+	 *
+	 *     -sizeof(usdt_prids_map_key_t) + sizeof(pid_t)
+	 *     ==
+	 *     -sizeof(dtrace_id_t)                         underlying-probe prid
+	 */
+	emit(dlp,  BPF_STORE(BPF_W, BPF_REG_9, (int)(-sizeof(usdt_prids_map_key_t)), BPF_REG_0));
+	emit(dlp,  BPF_STORE_IMM(BPF_W, BPF_REG_9, (int)(-sizeof(dtrace_id_t)), uprp->desc->id));
+	dt_cg_xsetx(dlp, usdt_prids, DT_LBL_NONE, BPF_REG_1, usdt_prids->di_id);
+	emit(dlp,  BPF_MOV_REG(BPF_REG_2, BPF_REG_9));
+	emit(dlp,  BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, (int)(-sizeof(usdt_prids_map_key_t))));
+	emit(dlp,  BPF_CALL_HELPER(BPF_FUNC_map_lookup_elem));
+	emit(dlp,  BPF_BRANCH_IMM(BPF_JEQ, BPF_REG_0, 0, lbl_exit));
+
+	/* Read the PRID from the table lookup and store to mst->prid. */
+	emit(dlp,  BPF_LOAD(BPF_W, BPF_REG_1, BPF_REG_0, 0));
+	emit(dlp,  BPF_STORE(BPF_W, BPF_REG_7, DMST_PRID, BPF_REG_1));
+
+	/* Read the bit mask from the table lookup in %r6. */    // FIXME someday, extend this past 64 bits
+	emit(dlp,  BPF_LOAD(BPF_DW, BPF_REG_6, BPF_REG_0, offsetof(usdt_prids_map_val_t, mask)));
+
+	/*
+	 * Hold the bit mask in %r6 between clause calls.
+	 */
+	for (n = 0; n < dtp->dt_stmt_nextid; n++) {
+		dtrace_stmtdesc_t *stp;
+		dt_ident_t	*idp;
+		uint_t		lbl_next = dt_irlist_label(dlp);
+
+		stp = dtp->dt_stmts[n];
+		if (stp == NULL)
+			continue;
+
+		idp = stp->dtsd_clause;
+
+		/* If the lowest %r6 bit is 0, skip over this clause. */
+		emit(dlp,  BPF_MOV_REG(BPF_REG_1, BPF_REG_6));
+		emit(dlp,  BPF_ALU64_IMM(BPF_AND, BPF_REG_1, 1));
+		emit(dlp,  BPF_BRANCH_IMM(BPF_JEQ, BPF_REG_1, 0, lbl_next));
+
+		/*
+		 *      if (*dctx.act != act)   // ldw %r0, [%r9 + DCTX_ACT]
+		 *	      goto exit;      // ldw %r0, [%r0 + 0]
+		 *			      // jne %r0, act, lbl_exit
+		 */
+		emit(dlp,  BPF_LOAD(BPF_DW, BPF_REG_0, BPF_REG_9, DCTX_ACT));
+		emit(dlp,  BPF_LOAD(BPF_W, BPF_REG_0, BPF_REG_0, 0));
+		emit(dlp,  BPF_BRANCH_IMM(BPF_JNE, BPF_REG_0, DT_ACTIVITY_ACTIVE, lbl_exit));
+
+		/* dctx.mst->scratch_top = 8 */
+		emit(dlp,  BPF_STORE_IMM(BPF_W, BPF_REG_7, DMST_SCRATCH_TOP, 8));
+
+		/* Call clause. */
+		emit(dlp,  BPF_MOV_REG(BPF_REG_1, BPF_REG_9));
+		emite(dlp, BPF_CALL_FUNC(idp->di_id), idp);
+
+		/* Finished this clause. */
+		emitl(dlp, lbl_next,
+			   BPF_NOP());
+
+		/* Right-shift %r6. */
+		emit(dlp,  BPF_ALU64_IMM(BPF_RSH, BPF_REG_6, 1));
+	}
+
+out:
 	dt_cg_tramp_return(pcb);
 
 	return 0;
@@ -846,10 +944,9 @@ static int trampoline_is_enabled(dt_pcb_t *pcb, uint_t exitlbl)
 {
 	dt_irlist_t		*dlp = &pcb->pcb_ir;
 	const dt_probe_t	*uprp = pcb->pcb_probe;
-	const dt_uprobe_t	*upp = uprp->prv_data;
-	const list_probe_t	*pop;
-	uint_t			lbl_assign = dt_irlist_label(dlp);
-	uint_t			lbl_exit = pcb->pcb_exitlbl;
+	dt_ident_t		*usdt_prids = dt_dlib_get_map(pcb->pcb_hdl, "usdt_prids");
+
+	assert(usdt_prids != NULL);
 
 	dt_cg_tramp_prologue(pcb);
 
@@ -858,7 +955,6 @@ static int trampoline_is_enabled(dt_pcb_t *pcb, uint_t exitlbl)
 	 *				//     (%r7 = dctx->mst)
 	 *				//     (%r8 = dctx->ctx)
 	 */
-
 	dt_cg_tramp_copy_regs(pcb);
 
 	/*
@@ -876,46 +972,30 @@ static int trampoline_is_enabled(dt_pcb_t *pcb, uint_t exitlbl)
 	emit(dlp,  BPF_ALU64_IMM(BPF_RSH, BPF_REG_0, 32));
 
 	/*
-	 * Generate a composite conditional clause, as above, except that rather
-	 * than emitting call_clauses, we emit copyouts instead, using
-	 * copyout_val() above:
+	 * Look up in the BPF 'usdt_prids' map.  Space for the look-up key
+	 * will be used on the BPF stack:
 	 *
-	 *	if (pid == PID1) {
-	 *		goto assign;
-	 *	} else if (pid == PID2) {
-	 *		goto assign;
-	 *	} else if (pid == ...) {
-	 *		goto assign;
-	 *	}
-	 *	goto exit;
-	 *	assign:
-	 *	    *arg0 = 1;
-	 *	goto exit;
+	 *     offset                                       value
 	 *
-	 * It is valid and safe to use %r0 to hold the pid value because there
-	 * are no assignments to %r0 possible in between the conditional
-	 * statements.
+	 *     -sizeof(usdt_prids_map_key_t)                pid (in %r0)
+	 *
+	 *     -sizeof(usdt_prids_map_key_t) + sizeof(pid_t)
+	 *     ==
+	 *     -sizeof(dtrace_id_t)                         underlying-probe prid
 	 */
-	for (pop = dt_list_next(&upp->probes); pop != NULL;
-	     pop = dt_list_next(pop)) {
-		const dt_probe_t	*prp = pop->probe;
-		pid_t			pid;
-		dt_ident_t		*idp;
+	emit(dlp,  BPF_STORE(BPF_W, BPF_REG_9, (int)(-sizeof(usdt_prids_map_key_t)), BPF_REG_0));
+	emit(dlp,  BPF_STORE_IMM(BPF_W, BPF_REG_9, (int)(-sizeof(dtrace_id_t)), uprp->desc->id));
+	dt_cg_xsetx(dlp, usdt_prids, DT_LBL_NONE, BPF_REG_1, usdt_prids->di_id);
+	emit(dlp,  BPF_MOV_REG(BPF_REG_2, BPF_REG_9));
+	emit(dlp,  BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, (int)(-sizeof(usdt_prids_map_key_t))));
+	emit(dlp,  BPF_CALL_HELPER(BPF_FUNC_map_lookup_elem));
+	emit(dlp,  BPF_BRANCH_IMM(BPF_JEQ, BPF_REG_0, 0, pcb->pcb_exitlbl));
 
-		pid = dt_pid_get_pid(prp->desc, pcb->pcb_hdl, pcb, NULL);
-		assert(pid != -1);
-
-		idp = dt_dlib_add_probe_var(pcb->pcb_hdl, prp);
-		assert(idp != NULL);
-
-		/*
-		 * Check whether this pid-provider probe serves the current
-		 * process, and copy out a 1 into arg 0 if so.
-		 */
-		emit(dlp,  BPF_BRANCH_IMM(BPF_JEQ, BPF_REG_0, pid, lbl_assign));
-	}
-	emit(dlp,  BPF_JUMP(lbl_exit));
-	copyout_val(pcb, lbl_assign, 1, 0);
+	/*
+	 * If we succeeded, then we use copyout_val() above to assign:
+	 *	    *arg0 = 1;
+	 */
+	copyout_val(pcb, DT_LBL_NONE, 1, 0);
 
 	dt_cg_tramp_return(pcb);
 
test/unittest/usdt/tst.nusdtprobes.r new file +5 -0
new file mode 100644
index 000000000..d894af92e
@@ -0,0 +1,5 @@
+try ""
+try "-xnusdtprobes=40"
+try "-xnusdtprobes=39"
+Files check.txt.sorted and dtrace.out.sorted differ
+success
test/unittest/usdt/tst.nusdtprobes.sh new file +172 -0

172 lines not on this page · raw message

Signature · 1 line
2.43.5
Reviewed-by on v4 9/19
#4 lore raw

Re: [PATCH v4 09/19] Use usdt_prids map to call clauses conditionally for USDT probes

Quoted 6 lines · On Wed, Oct 16, 2024 at 12:01:38PM -0400, [email protected] wrote:

From: Eugene Loh <[email protected]>

This version supports only up to 64 clauses for an underlying probe, but it can be extended to more clauses.

... with a very tiny pedantic comment and some comment about future work.

Quoted 36 lines
---
 libdtrace/dt_prov_uprobe.c            | 166 ++++++++++++++++++-------
 test/unittest/usdt/tst.nusdtprobes.r  |   5 +
 test/unittest/usdt/tst.nusdtprobes.sh | 172 ++++++++++++++++++++++++++
 3 files changed, 300 insertions(+), 43 deletions(-)
 create mode 100644 test/unittest/usdt/tst.nusdtprobes.r
 create mode 100755 test/unittest/usdt/tst.nusdtprobes.sh
libdtrace/dt_prov_uprobe.c +7 -1
index b010d8687..268708179 100644
@@ -739,11 +739,16 @@ static void enable_usdt(dtrace_hdl_t *dtp, dt_probe_t *prp)
  */
 static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 {
+	dtrace_hdl_t		*dtp = pcb->pcb_hdl;
 	dt_irlist_t		*dlp = &pcb->pcb_ir;
 	const dt_probe_t	*uprp = pcb->pcb_probe;
 	const dt_uprobe_t	*upp = uprp->prv_data;
 	const list_probe_t	*pop;
 	uint_t			lbl_exit = pcb->pcb_exitlbl;
+	dt_ident_t		*usdt_prids = dt_dlib_get_map(dtp, "usdt_prids");
+	int			n;
+
+	assert(usdt_prids != NULL);
 
 	dt_cg_tramp_prologue(pcb);                             // call this only once... is PRID set/relocated correctly?
 
@@ -755,14 +760,17 @@ static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 	dt_cg_tramp_copy_regs(pcb);                            // call this only once for all clauses?
 
 	/*
-	 * Loop over overlying probes, calling clauses for those that match:
+	 * pid probes.
+	 *

libdtrace/dt_prov_uprobe.c static int trampoline(dt_pcb_t *pcb, uint_t exitlbl) · line 773

Quoted 10 lines
libdtrace/dt_prov_uprobe.c +3 -1
+	 * Loop over overlying pid probes, calling clauses for those that match:
 	 *
-	 *	for overlying probes (that match except possibly for pid)
+	 *	for overlying pid probes (that match except possibly for pid)
 	 *		if (pid matches) {
 	 *			dctx->mst->prid = PRID1;
 	 *			< any number of clause calls >
 	 *		}
 	 */
+

Pedantic - but no empty line is needed here.

libdtrace/dt_prov_uprobe.c static int trampoline(dt_pcb_t *pcb, uint_t exitlbl) · line 782

Quoted 9 lines
libdtrace/dt_prov_uprobe.c +2 -0
 	for (pop = dt_list_next(&upp->probes); pop != NULL;
 	     pop = dt_list_next(pop)) {
 		const dt_probe_t	*prp = pop->probe;
@@ -770,6 +778,9 @@ static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 		pid_t			pid;
 		dt_ident_t		*idp;
 
+		if (prp->prov->impl != &dt_pid)
+			continue;

Stuff for future: put pid probes and USDT probes in separate lists so you don't need to have a conditional like this.

Quoted 379 lines
libdtrace/dt_prov_uprobe.c +111 -41
+
 		pid = dt_pid_get_pid(prp->desc, pcb->pcb_hdl, pcb, NULL);
 		assert(pid != -1);
 
@@ -801,6 +812,93 @@ static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 			   BPF_NOP());
 	}
 
+	/*
+	 * USDT
+	 */
+
+	/* In some cases, we know there are no USDT probes. */  // FIXME: add more checks
+	if (upp->flags & PP_IS_RETURN)
+		goto out;
+
+	dt_cg_tramp_copy_args_from_regs(pcb, 0);
+
+	/*
+	 * Retrieve the PID of the process that caused the probe to fire.
+	 */
+	emit(dlp,  BPF_CALL_HELPER(BPF_FUNC_get_current_pid_tgid));
+	emit(dlp,  BPF_ALU64_IMM(BPF_RSH, BPF_REG_0, 32));
+
+	/*
+	 * Look up in the BPF 'usdt_prids' map.  Space for the look-up key
+	 * will be used on the BPF stack:
+	 *
+	 *     offset                                       value
+	 *
+	 *     -sizeof(usdt_prids_map_key_t)                pid (in %r0)
+	 *
+	 *     -sizeof(usdt_prids_map_key_t) + sizeof(pid_t)
+	 *     ==
+	 *     -sizeof(dtrace_id_t)                         underlying-probe prid
+	 */
+	emit(dlp,  BPF_STORE(BPF_W, BPF_REG_9, (int)(-sizeof(usdt_prids_map_key_t)), BPF_REG_0));
+	emit(dlp,  BPF_STORE_IMM(BPF_W, BPF_REG_9, (int)(-sizeof(dtrace_id_t)), uprp->desc->id));
+	dt_cg_xsetx(dlp, usdt_prids, DT_LBL_NONE, BPF_REG_1, usdt_prids->di_id);
+	emit(dlp,  BPF_MOV_REG(BPF_REG_2, BPF_REG_9));
+	emit(dlp,  BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, (int)(-sizeof(usdt_prids_map_key_t))));
+	emit(dlp,  BPF_CALL_HELPER(BPF_FUNC_map_lookup_elem));
+	emit(dlp,  BPF_BRANCH_IMM(BPF_JEQ, BPF_REG_0, 0, lbl_exit));
+
+	/* Read the PRID from the table lookup and store to mst->prid. */
+	emit(dlp,  BPF_LOAD(BPF_W, BPF_REG_1, BPF_REG_0, 0));
+	emit(dlp,  BPF_STORE(BPF_W, BPF_REG_7, DMST_PRID, BPF_REG_1));
+
+	/* Read the bit mask from the table lookup in %r6. */    // FIXME someday, extend this past 64 bits
+	emit(dlp,  BPF_LOAD(BPF_DW, BPF_REG_6, BPF_REG_0, offsetof(usdt_prids_map_val_t, mask)));
+
+	/*
+	 * Hold the bit mask in %r6 between clause calls.
+	 */
+	for (n = 0; n < dtp->dt_stmt_nextid; n++) {
+		dtrace_stmtdesc_t *stp;
+		dt_ident_t	*idp;
+		uint_t		lbl_next = dt_irlist_label(dlp);
+
+		stp = dtp->dt_stmts[n];
+		if (stp == NULL)
+			continue;
+
+		idp = stp->dtsd_clause;
+
+		/* If the lowest %r6 bit is 0, skip over this clause. */
+		emit(dlp,  BPF_MOV_REG(BPF_REG_1, BPF_REG_6));
+		emit(dlp,  BPF_ALU64_IMM(BPF_AND, BPF_REG_1, 1));
+		emit(dlp,  BPF_BRANCH_IMM(BPF_JEQ, BPF_REG_1, 0, lbl_next));
+
+		/*
+		 *      if (*dctx.act != act)   // ldw %r0, [%r9 + DCTX_ACT]
+		 *	      goto exit;      // ldw %r0, [%r0 + 0]
+		 *			      // jne %r0, act, lbl_exit
+		 */
+		emit(dlp,  BPF_LOAD(BPF_DW, BPF_REG_0, BPF_REG_9, DCTX_ACT));
+		emit(dlp,  BPF_LOAD(BPF_W, BPF_REG_0, BPF_REG_0, 0));
+		emit(dlp,  BPF_BRANCH_IMM(BPF_JNE, BPF_REG_0, DT_ACTIVITY_ACTIVE, lbl_exit));
+
+		/* dctx.mst->scratch_top = 8 */
+		emit(dlp,  BPF_STORE_IMM(BPF_W, BPF_REG_7, DMST_SCRATCH_TOP, 8));
+
+		/* Call clause. */
+		emit(dlp,  BPF_MOV_REG(BPF_REG_1, BPF_REG_9));
+		emite(dlp, BPF_CALL_FUNC(idp->di_id), idp);
+
+		/* Finished this clause. */
+		emitl(dlp, lbl_next,
+			   BPF_NOP());
+
+		/* Right-shift %r6. */
+		emit(dlp,  BPF_ALU64_IMM(BPF_RSH, BPF_REG_6, 1));
+	}
+
+out:
 	dt_cg_tramp_return(pcb);
 
 	return 0;
@@ -846,10 +944,9 @@ static int trampoline_is_enabled(dt_pcb_t *pcb, uint_t exitlbl)
 {
 	dt_irlist_t		*dlp = &pcb->pcb_ir;
 	const dt_probe_t	*uprp = pcb->pcb_probe;
-	const dt_uprobe_t	*upp = uprp->prv_data;
-	const list_probe_t	*pop;
-	uint_t			lbl_assign = dt_irlist_label(dlp);
-	uint_t			lbl_exit = pcb->pcb_exitlbl;
+	dt_ident_t		*usdt_prids = dt_dlib_get_map(pcb->pcb_hdl, "usdt_prids");
+
+	assert(usdt_prids != NULL);
 
 	dt_cg_tramp_prologue(pcb);
 
@@ -858,7 +955,6 @@ static int trampoline_is_enabled(dt_pcb_t *pcb, uint_t exitlbl)
 	 *				//     (%r7 = dctx->mst)
 	 *				//     (%r8 = dctx->ctx)
 	 */
-
 	dt_cg_tramp_copy_regs(pcb);
 
 	/*
@@ -876,46 +972,30 @@ static int trampoline_is_enabled(dt_pcb_t *pcb, uint_t exitlbl)
 	emit(dlp,  BPF_ALU64_IMM(BPF_RSH, BPF_REG_0, 32));
 
 	/*
-	 * Generate a composite conditional clause, as above, except that rather
-	 * than emitting call_clauses, we emit copyouts instead, using
-	 * copyout_val() above:
+	 * Look up in the BPF 'usdt_prids' map.  Space for the look-up key
+	 * will be used on the BPF stack:
 	 *
-	 *	if (pid == PID1) {
-	 *		goto assign;
-	 *	} else if (pid == PID2) {
-	 *		goto assign;
-	 *	} else if (pid == ...) {
-	 *		goto assign;
-	 *	}
-	 *	goto exit;
-	 *	assign:
-	 *	    *arg0 = 1;
-	 *	goto exit;
+	 *     offset                                       value
 	 *
-	 * It is valid and safe to use %r0 to hold the pid value because there
-	 * are no assignments to %r0 possible in between the conditional
-	 * statements.
+	 *     -sizeof(usdt_prids_map_key_t)                pid (in %r0)
+	 *
+	 *     -sizeof(usdt_prids_map_key_t) + sizeof(pid_t)
+	 *     ==
+	 *     -sizeof(dtrace_id_t)                         underlying-probe prid
 	 */
-	for (pop = dt_list_next(&upp->probes); pop != NULL;
-	     pop = dt_list_next(pop)) {
-		const dt_probe_t	*prp = pop->probe;
-		pid_t			pid;
-		dt_ident_t		*idp;
+	emit(dlp,  BPF_STORE(BPF_W, BPF_REG_9, (int)(-sizeof(usdt_prids_map_key_t)), BPF_REG_0));
+	emit(dlp,  BPF_STORE_IMM(BPF_W, BPF_REG_9, (int)(-sizeof(dtrace_id_t)), uprp->desc->id));
+	dt_cg_xsetx(dlp, usdt_prids, DT_LBL_NONE, BPF_REG_1, usdt_prids->di_id);
+	emit(dlp,  BPF_MOV_REG(BPF_REG_2, BPF_REG_9));
+	emit(dlp,  BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, (int)(-sizeof(usdt_prids_map_key_t))));
+	emit(dlp,  BPF_CALL_HELPER(BPF_FUNC_map_lookup_elem));
+	emit(dlp,  BPF_BRANCH_IMM(BPF_JEQ, BPF_REG_0, 0, pcb->pcb_exitlbl));
 
-		pid = dt_pid_get_pid(prp->desc, pcb->pcb_hdl, pcb, NULL);
-		assert(pid != -1);
-
-		idp = dt_dlib_add_probe_var(pcb->pcb_hdl, prp);
-		assert(idp != NULL);
-
-		/*
-		 * Check whether this pid-provider probe serves the current
-		 * process, and copy out a 1 into arg 0 if so.
-		 */
-		emit(dlp,  BPF_BRANCH_IMM(BPF_JEQ, BPF_REG_0, pid, lbl_assign));
-	}
-	emit(dlp,  BPF_JUMP(lbl_exit));
-	copyout_val(pcb, lbl_assign, 1, 0);
+	/*
+	 * If we succeeded, then we use copyout_val() above to assign:
+	 *	    *arg0 = 1;
+	 */
+	copyout_val(pcb, DT_LBL_NONE, 1, 0);
 
 	dt_cg_tramp_return(pcb);
 
test/unittest/usdt/tst.nusdtprobes.r new file +5 -0
new file mode 100644
index 000000000..d894af92e
@@ -0,0 +1,5 @@
+try ""
+try "-xnusdtprobes=40"
+try "-xnusdtprobes=39"
+Files check.txt.sorted and dtrace.out.sorted differ
+success
test/unittest/usdt/tst.nusdtprobes.sh new file +172 -0
new file mode 100755
index 000000000..50f18a6ca
@@ -0,0 +1,172 @@
+#!/bin/bash
+#
+# Oracle Linux DTrace.
+# Copyright (c) 2024, Oracle and/or its affiliates. All rights reserved.
+# Licensed under the Universal Permissive License v 1.0 as shown at
+# http://oss.oracle.com/licenses/upl.
+#
+# This test verifies the nusdtprobes option.
+# @@timeout: 100
+
+dtrace=$1
+
+# Set up test directory.
+
+DIRNAME=$tmpdir/nusdtprobes.$$.$RANDOM
+mkdir -p $DIRNAME
+cd $DIRNAME
+
+# Make the trigger.
+
+cat << EOF > prov.d
+provider testprov {
+	probe foo0();
+	probe foo1();
+	probe foo2();
+	probe foo3();
+	probe foo4();
+	probe foo5();
+	probe foo6();
+	probe foo7();
+	probe foo8();
+	probe foo9();
+};
+EOF
+
+cat << EOF > main.c
+#include <unistd.h>
+#include "prov.h"
+
+int
+main(int argc, char **argv)
+{
+	while (1) {
+		usleep(1000);
+
+		TESTPROV_FOO0();
+		TESTPROV_FOO1();
+		TESTPROV_FOO2();
+		TESTPROV_FOO3();
+		TESTPROV_FOO4();
+		TESTPROV_FOO5();
+		TESTPROV_FOO6();
+		TESTPROV_FOO7();
+		TESTPROV_FOO8();
+		TESTPROV_FOO9();
+	}
+
+	return 0;
+}
+EOF
+
+# Build the trigger.
+
+$dtrace -h -s prov.d
+if [ $? -ne 0 ]; then
+	echo "failed to generate header file" >&2
+	exit 1
+fi
+cc $test_cppflags -c main.c
+if [ $? -ne 0 ]; then
+	echo "failed to compile test" >&2
+	exit 1
+fi
+$dtrace -G -64 -s prov.d main.o
+if [ $? -ne 0 ]; then
+	echo "failed to create DOF" >&2
+	exit 1
+fi
+cc $test_cppflags -o main main.o prov.o
+if [ $? -ne 0 ]; then
+	echo "failed to link final executable" >&2
+	exit 1
+fi
+
+# Test nusdtprobes settings.
+#
+# We will start teams of processes, each with 4 members, each in turn
+# with 10 USDT probes.  So, regardless of how many teams are run in
+# succession, at any one time DTrace needs room for at least 40 USDT
+# probes.  The default and -xnusdtprobes=40 settings should work, but
+# -xnusdtprobes=39 should not.
+nteams=2
+nmmbrs=4
+
+for nusdt in "" "-xnusdtprobes=40" "-xnusdtprobes=39"; do
+
+	echo try '"'$nusdt'"'
+
+	# Start DTrace.
+
+	rm -f dtrace.out
+	$dtrace $dt_flags $nusdt -Zq -o dtrace.out -n '
+	testprov*:::
+	{
+		@[probeprov, probemod, probefunc, probename] = count();
+	}' &
+	dtpid=$!
+	sleep 2
+	if [[ ! -d /proc/$dtpid ]]; then
+		echo ERROR dtrace died
+		cat dtrace.out
+		exit 1
+	fi
+
+	# Start teams of processes, only one team at a time.
+
+	rm -f check.txt
+	for (( iteam = 0; iteam < $nteams; iteam++ )); do
+		# Start the team, writing out expected output.
+		sleep 2
+		for (( immbr = 0; immbr < $nmmbrs; immbr++ )); do
+			./main &
+			pids[$immbr]=$!
+			for j in `seq 0 9`; do
+				echo testprov${pids[$immbr]} main main foo$j >> check.txt
+			done
+		done
+
+		# Kill the team.
+		sleep 3
+		for (( immbr = 0; immbr < $nmmbrs; immbr++ )); do
+			kill ${pids[$immbr]}
+		done
+	done
+
+	# Kill DTrace.
+
+	kill $dtpid
+	wait
+
+	# Strip the count() value out since we do not know its exact value.
+
+	awk 'NF == 5 { print $1, $2, $3, $4 }' dtrace.out | sort > dtrace.out.sorted
+
+	# Check.
+
+	sort check.txt > check.txt.sorted
+	if [ x$nusdt == x"-xnusdtprobes=39" ]; then
+		# Results should not agree with check.txt.
+		if diff -q check.txt.sorted dtrace.out.sorted; then
+			echo ERROR unexpected agreement
+			cat dtrace.out
+			exit 0
+		fi
+	else
+		# Results should agree with check.txt.
+		if ! diff -q check.txt.sorted dtrace.out.sorted; then
+			echo ERROR output disagrees
+			echo === expected ===
+			cat check.txt.sorted
+			echo === got ===
+			cat dtrace.out.sorted
+			echo === diff ===
+			diff check.txt.sorted dtrace.out.sorted
+			exit 1
+		fi
+	fi
+done
+
+echo success
+
+exit 0
Signature · 1 line
2.43.5
#5 lore raw

PATCH v3 14/19 Ignore clauses in USDT trampoline if we know they are impossible

In reply to [email protected]

From: Eugene Loh <[email protected]>

An underlying probe might have to call all sorts of clauses if new USDT processes start up. Currently, the underlying probe trampoline simply loops over all clauses, deciding whether to call a clause based on a run-time bit mask.

While this approach works, it can mean, e.g., that clauses are being loaded unnecessarily into BPF programs, bloating code. It also means that the trampoline is looping over unnecessarily many clauses.

Meanwhile, it is possible to know in certain cases that a clause could never be called for a particular underlying probe. The heuristics can be tricky, but that challenge can be faced incrementally.

Introduce an ignore_clause() function that, for an underlying probe, determines whether some clause (denoted by index) can safely be ignored.

The same ignore_clause() function should be called both during code generation of the trampoline as well as during the construction of the bit mask. Further, for a given clause n and underlying probe upp, the function should always give the same output. The set of ignored clauses should remain fixed over the lifetime of an underlying probe.

For now, conservatively, ignore_clause() ignores nothing. Later patches will introduce heuristics for ignoring clauses.

---
 libdtrace/dt_prov_uprobe.c | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)
libdtrace/dt_prov_uprobe.c +19 -1
index 0289c9223..a3c04478c 100644
@@ -263,6 +263,17 @@ clean_usdt_probes(dtrace_hdl_t *dtp)
 	return 0;
 }
 
+/*
+ * Judge whether clause "n" could ever be called as a USDT probe
+ * for this underlying probe.
+ */
+static int
+ignore_clause(dtrace_hdl_t *dtp, int n, const dt_probe_t *uprp)
+{
+	/* To be safe, ignore nothing. */
+	return 0;
+}
+
 static int add_probe_uprobe(dtrace_hdl_t *dtp, dt_probe_t *prp)
 {
 	dtrace_difo_t   *dp;
@@ -388,6 +399,9 @@ static int add_probe_usdt(dtrace_hdl_t *dtp, dt_probe_t *prp)
 				if (stp == NULL)
 					continue;
 
+				if (ignore_clause(dtp, n, uprp))
+					continue;
+
 				if (dt_gmatch(prp->desc->prv, stp->dtsd_ecbdesc->dted_probe.prv) &&
 				    dt_gmatch(prp->desc->mod, stp->dtsd_ecbdesc->dted_probe.mod) &&
 				    dt_gmatch(prp->desc->fun, stp->dtsd_ecbdesc->dted_probe.fun) &&
@@ -862,13 +876,17 @@ static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 	for (n = 0; n < dtp->dt_stmt_nextid; n++) {
 		dtrace_stmtdesc_t *stp;
 		dt_ident_t	*idp;
-		uint_t		lbl_next = dt_irlist_label(dlp);
+		uint_t		lbl_next;
 
 		stp = dtp->dt_stmts[n];
 		if (stp == NULL)
 			continue;
 
+		if (ignore_clause(dtp, n, uprp))
+			continue;
+
 		idp = stp->dtsd_clause;
+		lbl_next = dt_irlist_label(dlp);
 
 		/* If the lowest %r6 bit is 0, skip over this clause. */
 		emit(dlp,  BPF_MOV_REG(BPF_REG_1, BPF_REG_6));
Signature · 1 line
2.43.5
Reviewed-by on v3 14/19
#6 lore raw

Re: [PATCH v3 14/19] Ignore clauses in USDT trampoline if we know they are impossible

Quoted 25 lines · On Wed, Oct 16, 2024 at 12:01:39PM -0400, [email protected] wrote:

From: Eugene Loh <[email protected]>

An underlying probe might have to call all sorts of clauses if new USDT processes start up. Currently, the underlying probe trampoline simply loops over all clauses, deciding whether to call a clause based on a run-time bit mask.

While this approach works, it can mean, e.g., that clauses are being loaded unnecessarily into BPF programs, bloating code. It also means that the trampoline is looping over unnecessarily many clauses.

Meanwhile, it is possible to know in certain cases that a clause could never be called for a particular underlying probe. The heuristics can be tricky, but that challenge can be faced incrementally.

Introduce an ignore_clause() function that, for an underlying probe, determines whether some clause (denoted by index) can safely be ignored.

The same ignore_clause() function should be called both during code generation of the trampoline as well as during the construction of the bit mask. Further, for a given clause n and underlying probe upp, the function should always give the same output. The set of ignored clauses should remain fixed over the lifetime of an underlying probe.

Quoted 4 lines

For now, conservatively, ignore_clause() ignores nothing. Later patches will introduce heuristics for ignoring clauses.

Quoted 58 lines
---
 libdtrace/dt_prov_uprobe.c | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)
libdtrace/dt_prov_uprobe.c +19 -1
index 0289c9223..a3c04478c 100644
@@ -263,6 +263,17 @@ clean_usdt_probes(dtrace_hdl_t *dtp)
 	return 0;
 }
 
+/*
+ * Judge whether clause "n" could ever be called as a USDT probe
+ * for this underlying probe.
+ */
+static int
+ignore_clause(dtrace_hdl_t *dtp, int n, const dt_probe_t *uprp)
+{
+	/* To be safe, ignore nothing. */
+	return 0;
+}
+
 static int add_probe_uprobe(dtrace_hdl_t *dtp, dt_probe_t *prp)
 {
 	dtrace_difo_t   *dp;
@@ -388,6 +399,9 @@ static int add_probe_usdt(dtrace_hdl_t *dtp, dt_probe_t *prp)
 				if (stp == NULL)
 					continue;
 
+				if (ignore_clause(dtp, n, uprp))
+					continue;
+
 				if (dt_gmatch(prp->desc->prv, stp->dtsd_ecbdesc->dted_probe.prv) &&
 				    dt_gmatch(prp->desc->mod, stp->dtsd_ecbdesc->dted_probe.mod) &&
 				    dt_gmatch(prp->desc->fun, stp->dtsd_ecbdesc->dted_probe.fun) &&
@@ -862,13 +876,17 @@ static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 	for (n = 0; n < dtp->dt_stmt_nextid; n++) {
 		dtrace_stmtdesc_t *stp;
 		dt_ident_t	*idp;
-		uint_t		lbl_next = dt_irlist_label(dlp);
+		uint_t		lbl_next;
 
 		stp = dtp->dt_stmts[n];
 		if (stp == NULL)
 			continue;
 
+		if (ignore_clause(dtp, n, uprp))
+			continue;
+
 		idp = stp->dtsd_clause;
+		lbl_next = dt_irlist_label(dlp);
 
 		/* If the lowest %r6 bit is 0, skip over this clause. */
 		emit(dlp,  BPF_MOV_REG(BPF_REG_1, BPF_REG_6));
Signature · 1 line
2.43.5
#7 lore raw
In reply to [email protected]
Quoted 27 lines · On Wed, Oct 16, 2024 at 12:01:36PM -0400, [email protected] wrote:

From: Eugene Loh <[email protected]>

It is possible for new providers and probes to appear after dtrace_go(). Add a function that can be called for such discovery, on a per-provider basis. Each provider that supports such discovery needs to provide:

*) a function discover() that performs discovery on behalf of the provider

*) a function add_probe() that adds a particular (discovered) probe

To support looping over providers, move dt_providers[] from dt_open.c to dt_provider.c and extern it in dt_provider.h.

---
 libdtrace/dt_bpf.c      |  6 +++++
 libdtrace/dt_open.c     | 22 +---------------
 libdtrace/dt_provider.c | 57 +++++++++++++++++++++++++++++++++++++++++
 libdtrace/dt_provider.h |  6 +++++
 libdtrace/dt_work.c     |  3 +++
 5 files changed, 73 insertions(+), 21 deletions(-)
libdtrace/dt_bpf.c +0 -0
index ad11d178e..7a36b6187 100644

libdtrace/dt_bpf.c dt_bpf_load_progs(dtrace_hdl_t *dtp, uint_t cflags) · line 1281

Quoted 7 lines
libdtrace/dt_bpf.c +4 -0
@@ -1275,6 +1275,12 @@ dt_bpf_load_progs(dtrace_hdl_t *dtp, uint_t cflags)
 		int		fd;
 		int		rc = -1;
 
+		if (prp->prov->impl->prog_type == BPF_PROG_TYPE_UNSPEC) {
+			if (prp->prov->impl->add_probe)
+				prp->prov->impl->add_probe(dtp, prp);
+			continue;

This could benefit from a comment perhaps? It took me a bit to figure out why we would want this here, since this is the case where the probes are already known to exist (they were discovered during program compilation). So, needing a call to add_probe seems out-of-place here.

I do understand (after digging through the code a bit) that it is needed for USDT because you add probe name data to the usdt_names BPF map and probe data to the usdt_prids BPF map from the add_probe() callback. But thatis far from obvious... But try to keep the comment non-USDT specific if you can. Perhaps just mention this is needed to handle probe types that are discoverable at a later time, because they may need additional processing.

I think that testing for prog_type == UNSPEC is not appropriate here. That seems to be a case of leaking USDT-specific implementation detail into this generic function. One way to deal with this might be to add a test in add_probe_uprobe() on dtp->dt_active. If dt_active == 0, then add_probe_uprobe() should immediately return (because the probe already exists in the system). If dt_active == 1, then it is a runtime discovered probe.

Quoted 169 lines
libdtrace/dt_bpf.c +2 -0
+		}
+
 		dp = prp->difo;
 		if (dp == NULL)
 			continue;
libdtrace/dt_open.c +1 -21
index 848141ddc..1f586fc4f 100644
@@ -61,26 +61,6 @@ const dt_version_t _dtrace_versions[] = {
 	0
 };
 
-/*
- * List of provider modules that register providers and probes.  A single
- * provider module may create multiple providers.
- */
-static const dt_provimpl_t *dt_providers[] = {
-	&dt_dtrace,		/* list dt_dtrace first */
-	&dt_cpc,
-	&dt_fbt_fprobe,
-	&dt_io,
-	&dt_ip,
-	&dt_lockstat,
-	&dt_proc,
-	&dt_profile,
-	&dt_rawtp,
-	&dt_sched,
-	&dt_sdt,
-	&dt_syscall,
-	&dt_uprobe,
-};
-
 /*
  * Table of global identifiers.  This is used to populate the global identifier
  * hash when a new dtrace client open occurs.  For more info see dt_ident.h.
@@ -1210,7 +1190,7 @@ dtrace_init(dtrace_hdl_t *dtp)
 	 * known providers.
 	 */
 	dt_probe_init(dtp);
-	for (i = 0; i < ARRAY_SIZE(dt_providers); i++) {
+	for (i = 0; dt_providers[i]; i++) {
 		int	n;
 
 		n = dt_providers[i]->populate(dtp);
libdtrace/dt_provider.c +57 -0
index 8cfef0ba2..5f9766183 100644
@@ -18,10 +18,32 @@
 #include <port.h>
 
 #include <dt_provider.h>
+#include <dt_probe.h>
 #include <dt_module.h>
 #include <dt_string.h>
 #include <dt_list.h>
 
+/*
+ * List of provider modules that register providers and probes.  A single
+ * provider module may create multiple providers.
+ */
+const dt_provimpl_t *dt_providers[] = {
+	&dt_dtrace,		/* list dt_dtrace first */
+	&dt_cpc,
+	&dt_fbt_fprobe,
+	&dt_io,
+	&dt_ip,
+	&dt_lockstat,
+	&dt_proc,
+	&dt_profile,
+	&dt_rawtp,
+	&dt_sched,
+	&dt_sdt,
+	&dt_syscall,
+	&dt_uprobe,
+	NULL
+};
+
 static uint32_t
 dt_provider_hval(const dt_provider_t *pvp)
 {
@@ -149,3 +171,38 @@ dt_provider_xref(dtrace_hdl_t *dtp, dt_provider_t *pvp, id_t id)
 	BT_SET(pvp->pv_xrefs, id);
 	return 0;
 }
+
+int
+dt_provider_discover(dtrace_hdl_t *dtp)
+{
+	int i, prid = dtp->dt_probe_id;
+
+	/* Discover new probes. */
+	for (i = 0; dt_providers[i]; i++) {
+		int n;
+
+		if (dt_providers[i]->discover == NULL)
+			continue;
+
+		n = dt_providers[i]->discover(dtp);
+		if (n < 0)
+			return -1;        /* errno is already set */
+	}
+
+	/* Add them. */
+	for ( ; prid < dtp->dt_probe_id; prid++) {
+		dt_probe_t	*prp = dtp->dt_probes[prid];
+		int		rc;
+
+		dt_probe_enable(dtp, prp);
+
+		if (prp->prov->impl->add_probe == NULL)
+			continue;
+
+		rc = prp->prov->impl->add_probe(dtp, prp);
+		if (rc < 0)
+			return rc;
+	}
+
+	return 0;
+}
libdtrace/dt_provider.h +6 -0
index 17b1844cb..384d4fd33 100644
@@ -62,6 +62,9 @@ typedef struct dt_provimpl {
 	int (*probe_info)(dtrace_hdl_t *dtp,	/* get probe info */
 			  const struct dt_probe *prp,
 			  int *argcp, dt_argdesc_t **argvp);
+	int (*discover)(dtrace_hdl_t *dtp);	/* discover new probes */
+	int (*add_probe)(dtrace_hdl_t *dtp,	/* add a new probe */
+			 struct dt_probe *prp);
 	void (*detach)(dtrace_hdl_t *dtp,	/* probe cleanup */
 		       const struct dt_probe *prb);
 	void (*probe_destroy)(dtrace_hdl_t *dtp, /* free probe data */
@@ -86,6 +89,8 @@ extern dt_provimpl_t dt_sdt;
 extern dt_provimpl_t dt_syscall;
 extern dt_provimpl_t dt_uprobe;
 
+extern const dt_provimpl_t *dt_providers[];
+
 typedef struct dt_provider {
 	dt_list_t pv_list;		/* list forward/back pointers */
 	struct dt_hentry he;		/* htab links */
@@ -110,6 +115,7 @@ extern dt_provider_t *dt_provider_create(dtrace_hdl_t *, const char *,
 					 const dt_provimpl_t *,
 					 const dtrace_pattr_t *, void *);
 extern int dt_provider_xref(dtrace_hdl_t *, dt_provider_t *, id_t);
+extern int dt_provider_discover(dtrace_hdl_t *dtp);
 
 #ifdef	__cplusplus
 }
libdtrace/dt_work.c +3 -0
index 11335345a..5318cdb7e 100644
@@ -362,6 +362,9 @@ dtrace_work(dtrace_hdl_t *dtp, FILE *fp, dtrace_consume_probe_f *pfunc,
 	dtrace_workstatus_t	rval;
 	int			gen;
 
+	if (dt_provider_discover(dtp) < 0)
+		return DTRACE_WORKSTATUS_ERROR;
+
 	switch (dtrace_status(dtp)) {
 	case DTRACE_STATUS_EXITED:
 	case DTRACE_STATUS_STOPPED:
Signature · 1 line
2.43.5
author
#8 lore raw
Quoted 15 lines · Kris Van Hees

libdtrace/dt_bpf.c dt_bpf_load_progs(dtrace_hdl_t *dtp, uint_t cflags) · line 1281

Quoted 9 lines · On Wed, Oct 16, 2024 at 12:01:36PM -0400, [email protected] wrote:
libdtrace/dt_bpf.c +4 -0
@@ -1275,6 +1275,12 @@ dt_bpf_load_progs(dtrace_hdl_t *dtp, uint_t cflags)
  		int		fd;
  		int		rc = -1;
 
+		if (prp->prov->impl->prog_type == BPF_PROG_TYPE_UNSPEC) {
+			if (prp->prov->impl->add_probe)
+				prp->prov->impl->add_probe(dtp, prp);
+			continue;

This could benefit from a comment perhaps? It took me a bit to figure out why we would want this here, since this is the case where the probes are already known to exist (they were discovered during program compilation). So, needing a call to add_probe seems out-of-place here.

Quoted 6 lines

I do understand (after digging through the code a bit) that it is needed for USDT because you add probe name data to the usdt_names BPF map and probe data to the usdt_prids BPF map from the add_probe() callback. But thatis far from obvious... But try to keep the comment non-USDT specific if you can. Perhaps just mention this is needed to handle probe types that are discoverable at a later time, because they may need additional processing.

How about:
                /* Handle probes discovered after compilation. */

Quoted 3 lines

I think that testing for prog_type == UNSPEC is not appropriate here. That seems to be a case of leaking USDT-specific implementation detail into this generic function.

It's simply a construct that's lifted from dt_bpf_make_progs().

Quoted 4 lines

One way to deal with this might be to add a test in add_probe_uprobe() on dtp->dt_active. If dt_active == 0, then add_probe_uprobe() should immediately return (because the probe already exists in the system). If dt_active == 1, then it is a runtime discovered probe.

I don't understand how that's clearer and even having trouble convincing myself this is right.  Also, the dt_active stuff would belong in the succeeding patch, already reviewed.  Anyhow, I tried the change and it tests fine.  So, I'll send out a next version of this patch, along with a new version of the succeeding patch (which had already been reviewed).

Quoted 5 lines
Quoted 5 lines
libdtrace/dt_bpf.c +2 -0
+		}
+
  		dp = prp->difo;
  		if (dp == NULL)
  			continue;