Skip to contentSkip to conversation

service: don't walk out of bounds when checking services

PATCH Superseded Aaron Conole 4 participants 11 messages Nov 26, 2019 – Feb 14, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 service: don't walk out of bounds when checking services

The service_valid call is used without properly bounds checking the input parameter. Almost all instances of the service_valid call are inside a for() loop that prevents excessive walks, but some of the public APIs don't bounds check and will pass invalid arguments.

Prevent this by using SERVICE_GET_OR_ERR_RET where it makes sense, and adding a bounds check to one service_valid() use.

Fixes 8d39d3e237c2 ("service: fix race in service on app lcore function")
Fixes e9139a32f6e8 ("service: add function to run on app lcore")
Fixes e30dd31847d2 ("service: add mechanism for quiescing")
---
 lib/librte_eal/common/rte_service.c | 23 +++++++++++------------
 1 file changed, 11 insertions(+), 12 deletions(-)
lib/librte_eal/common/rte_service.c +11 -12
index 79235c03f8..73de7bbade 100644
@@ -345,11 +345,12 @@ rte_service_runner_do_callback(struct rte_service_spec_impl *s,
 
 
 static inline int32_t
-service_run(uint32_t i, struct core_state *cs, uint64_t service_mask)
+service_run(uint32_t i, struct core_state *cs, uint64_t service_mask,
+	    struct rte_service_spec_impl *s)
 {
-	if (!service_valid(i))
-		return -EINVAL;
-	struct rte_service_spec_impl *s = &rte_services[i];
+	if (!s)
+		SERVICE_VALID_GET_OR_ERR_RET(i, s, -EINVAL);
+
 	if (s->comp_runstate != RUNSTATE_RUNNING ||
 			s->app_runstate != RUNSTATE_RUNNING ||
 			!(service_mask & (UINT64_C(1) << i))) {
@@ -383,7 +384,7 @@ rte_service_may_be_active(uint32_t id)
 	int32_t lcore_count = rte_service_lcore_list(ids, RTE_MAX_LCORE);
 	int i;
 
-	if (!service_valid(id))
+	if (id >= RTE_SERVICE_NUM_MAX || !service_valid(id))
 		return -EINVAL;
 
 	for (i = 0; i < lcore_count; i++) {
@@ -397,12 +398,10 @@ rte_service_may_be_active(uint32_t id)
 int32_t
 rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe)
 {
-	/* run service on calling core, using all-ones as the service mask */
-	if (!service_valid(id))
-		return -EINVAL;
-
 	struct core_state *cs = &lcore_states[rte_lcore_id()];
-	struct rte_service_spec_impl *s = &rte_services[id];
+	struct rte_service_spec_impl *s;
+
+	SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL);
 
 	/* Atomically add this core to the mapped cores first, then examine if
 	 * we can run the service. This avoids a race condition between
@@ -418,7 +417,7 @@ rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe)
 		return -EBUSY;
 	}
 
-	int ret = service_run(id, cs, UINT64_MAX);
+	int ret = service_run(id, cs, UINT64_MAX, s);
 
 	if (serialize_mt_unsafe)
 		rte_atomic32_dec(&s->num_mapped_cores);
@@ -439,7 +438,7 @@ rte_service_runner_func(void *arg)
 
 		for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) {
 			/* return value ignored as no change to code flow */
-			service_run(i, cs, service_mask);
+			service_run(i, cs, service_mask, NULL);
 		}
 
 		cs->loops++;
Signature · 1 line
2.21.0
Acked-by on v1 1/1
#1 lore raw
Quoted 16 lines

-----Original Message-----
From: Aaron Conole <[email protected]>
Sent: Tuesday, November 26, 2019 8:56 AM
To: [email protected]
Cc: Van Haaren, Harry <[email protected]>; Richardson, Bruce <[email protected]>; Pavan Nikhilesh <[email protected]>; Eads, Gage <[email protected]>; Thomas Monjalon <[email protected]>; David Marchand <[email protected]>
Subject: [PATCH] service: don't walk out of bounds when checking services

The service_valid call is used without properly bounds checking the input parameter. Almost all instances of the service_valid call are inside a for() loop that prevents excessive walks, but some of the public APIs don't bounds check and will pass invalid arguments.

Quoted 7 lines

Prevent this by using SERVICE_GET_OR_ERR_RET where it makes sense, and adding a bounds check to one service_valid() use.

Fixes 8d39d3e237c2 ("service: fix race in service on app lcore function")
Fixes e9139a32f6e8 ("service: add function to run on app lcore")
Fixes e30dd31847d2 ("service: add mechanism for quiescing")

Thanks,
Gage

#2 lore raw
In reply to Aaron Conole
Quoted 25 lines · Aaron Conole

The service_valid call is used without properly bounds checking the input parameter. Almost all instances of the service_valid call are inside a for() loop that prevents excessive walks, but some of the public APIs don't bounds check and will pass invalid arguments.

Prevent this by using SERVICE_GET_OR_ERR_RET where it makes sense, and adding a bounds check to one service_valid() use.

Fixes 8d39d3e237c2 ("service: fix race in service on app lcore function")
Fixes e9139a32f6e8 ("service: add function to run on app lcore")
Fixes e30dd31847d2 ("service: add mechanism for quiescing")
---
 lib/librte_eal/common/rte_service.c | 23 +++++++++++------------
 1 file changed, 11 insertions(+), 12 deletions(-)
lib/librte_eal/common/rte_service.c +0 -0
index 79235c03f8..73de7bbade 100644
@@ -345,11 +345,12 @@ rte_service_runner_do_callback(struct rte_service_spec_impl *s,
 
 
 static inline int32_t

lib/librte_eal/common/rte_service.c rte_service_runner_do_callback(struct rte_service_spec_impl *s, · line 353

Quoted 10 lines
lib/librte_eal/common/rte_service.c +5 -4
-service_run(uint32_t i, struct core_state *cs, uint64_t service_mask)
+service_run(uint32_t i, struct core_state *cs, uint64_t service_mask,
+           struct rte_service_spec_impl *s)
 {
-       if (!service_valid(i))
-               return -EINVAL;
-       struct rte_service_spec_impl *s = &rte_services[i];
+       if (!s)
+               SERVICE_VALID_GET_OR_ERR_RET(i, s, -EINVAL);
+

No need to check the service if we ensure that the passed index is valid. See below.

Quoted 38 lines
lib/librte_eal/common/rte_service.c +5 -7
        if (s->comp_runstate != RUNSTATE_RUNNING ||
                        s->app_runstate != RUNSTATE_RUNNING ||
                        !(service_mask & (UINT64_C(1) << i))) {
@@ -383,7 +384,7 @@ rte_service_may_be_active(uint32_t id)
        int32_t lcore_count = rte_service_lcore_list(ids, RTE_MAX_LCORE);
        int i;
 
-       if (!service_valid(id))
+       if (id >= RTE_SERVICE_NUM_MAX || !service_valid(id))
                return -EINVAL;
 
        for (i = 0; i < lcore_count; i++) {
@@ -397,12 +398,10 @@ rte_service_may_be_active(uint32_t id)
 int32_t
 rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe)
 {
-       /* run service on calling core, using all-ones as the service mask */
-       if (!service_valid(id))
-               return -EINVAL;
-
        struct core_state *cs = &lcore_states[rte_lcore_id()];
-       struct rte_service_spec_impl *s = &rte_services[id];
+       struct rte_service_spec_impl *s;
+
+       SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL);
 
        /* Atomically add this core to the mapped cores first, then examine if
         * we can run the service. This avoids a race condition between
@@ -418,7 +417,7 @@ rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe)
                return -EBUSY;
        }
 
-       int ret = service_run(id, cs, UINT64_MAX);
+       int ret = service_run(id, cs, UINT64_MAX, s);
 
        if (serialize_mt_unsafe)
                rte_atomic32_dec(&s->num_mapped_cores);

lib/librte_eal/common/rte_service.c rte_service_runner_func(void *arg) · line 440

Quoted 3 lines
lib/librte_eal/common/rte_service.c +0 -0
@@ -439,7 +438,7 @@ rte_service_runner_func(void *arg)
 
                for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) {
                        /* return value ignored as no change to code flow */

if (!service_valid(idx))

    continue;

Plus, if we add this check here, thenall loops in this file are consistent. WDYT?

Signature · 1 line
David Marchand
#3 lore raw
Quoted 25 lines · David Marchand
Quoted 24 lines · Aaron Conole

The service_valid call is used without properly bounds checking the input parameter. Almost all instances of the service_valid call are inside a for() loop that prevents excessive walks, but some of the public APIs don't bounds check and will pass invalid arguments.

Prevent this by using SERVICE_GET_OR_ERR_RET where it makes sense, and adding a bounds check to one service_valid() use.

Fixes 8d39d3e237c2 ("service: fix race in service on app lcore function")
Fixes e9139a32f6e8 ("service: add function to run on app lcore")
Fixes e30dd31847d2 ("service: add mechanism for quiescing")
---
 lib/librte_eal/common/rte_service.c | 23 +++++++++++------------
 1 file changed, 11 insertions(+), 12 deletions(-)
lib/librte_eal/common/rte_service.c +0 -0
index 79235c03f8..73de7bbade 100644
@@ -345,11 +345,12 @@ rte_service_runner_do_callback(struct rte_service_spec_impl *s,
 
 
Quoted 14 lines

lib/librte_eal/common/rte_service.c rte_service_runner_do_callback(struct rte_service_spec_impl *s, · line 353

Quoted 11 lines
lib/librte_eal/common/rte_service.c +5 -4
 static inline int32_t
-service_run(uint32_t i, struct core_state *cs, uint64_t service_mask)
+service_run(uint32_t i, struct core_state *cs, uint64_t service_mask,
+           struct rte_service_spec_impl *s)
 {
-       if (!service_valid(i))
-               return -EINVAL;
-       struct rte_service_spec_impl *s = &rte_services[i];
+       if (!s)
+               SERVICE_VALID_GET_OR_ERR_RET(i, s, -EINVAL);
+

No need to check the service if we ensure that the passed index is valid. See below.

Okay. I will document that then ;)

Quoted 40 lines
Quoted 40 lines
lib/librte_eal/common/rte_service.c +5 -7
        if (s->comp_runstate != RUNSTATE_RUNNING ||
                        s->app_runstate != RUNSTATE_RUNNING ||
                        !(service_mask & (UINT64_C(1) << i))) {
@@ -383,7 +384,7 @@ rte_service_may_be_active(uint32_t id)
        int32_t lcore_count = rte_service_lcore_list(ids, RTE_MAX_LCORE);
        int i;
 
-       if (!service_valid(id))
+       if (id >= RTE_SERVICE_NUM_MAX || !service_valid(id))
                return -EINVAL;
 
        for (i = 0; i < lcore_count; i++) {
@@ -397,12 +398,10 @@ rte_service_may_be_active(uint32_t id)
 int32_t
 rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe)
 {
-       /* run service on calling core, using all-ones as the service mask */
-       if (!service_valid(id))
-               return -EINVAL;
-
        struct core_state *cs = &lcore_states[rte_lcore_id()];
-       struct rte_service_spec_impl *s = &rte_services[id];
+       struct rte_service_spec_impl *s;
+
+       SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL);
 
        /* Atomically add this core to the mapped cores first, then examine if
         * we can run the service. This avoids a race condition between
@@ -418,7 +417,7 @@ rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe)
                return -EBUSY;
        }
 
-       int ret = service_run(id, cs, UINT64_MAX);
+       int ret = service_run(id, cs, UINT64_MAX, s);
 
        if (serialize_mt_unsafe)
                rte_atomic32_dec(&s->num_mapped_cores);
@@ -439,7 +438,7 @@ rte_service_runner_func(void *arg)
 
Quoted 8 lines

lib/librte_eal/common/rte_service.c rte_service_runner_func(void *arg) · line 440

Quoted 2 lines
lib/librte_eal/common/rte_service.c +0 -0
                for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) {
                        /* return value ignored as no change to code flow */

if (!service_valid(idx))

    continue;

Plus, if we add this check here, thenall loops in this file are consistent. WDYT?

Agreed - it's better. Okay.

#4 lore raw

PATCH v2 1/1 service: don't walk out of bounds when checking services

In reply to Aaron Conole

The service_valid call is used without properly bounds checking the input parameter. Almost all instances of the service_valid call are inside a for() loop that prevents excessive walks, but some of the public APIs don't bounds check and will pass invalid arguments.

Prevent this by using SERVICE_GET_OR_ERR_RET where it makes sense, and adding a bounds check to one service_valid() use.

Fixes 8d39d3e237c2 ("service: fix race in service on app lcore function")
Fixes e9139a32f6e8 ("service: add function to run on app lcore")
Fixes e30dd31847d2 ("service: add mechanism for quiescing")

---
v2:
- make for() loop consistent and service_run() always require a valid 's' pointer
- introduce service_get() for a future patch to clean up the bare references to 'rte_services[i]'
- remove some useless 'inline' specifiers on functions (they aren't needed in .c files). a future patch can clean up the others.

 lib/librte_eal/common/rte_service.c | 32 ++++++++++++++++++-----------
 1 file changed, 20 insertions(+), 12 deletions(-)
lib/librte_eal/common/rte_service.c +20 -12
index 79235c03f8..7e537b8cd2 100644
@@ -137,6 +137,12 @@ service_valid(uint32_t id)
 	return !!(rte_services[id].internal_flags & SERVICE_F_REGISTERED);
 }
 
+static struct rte_service_spec_impl *
+service_get(uint32_t id)
+{
+	return &rte_services[id];
+}
+
 /* validate ID and retrieve service pointer, or return error value */
 #define SERVICE_VALID_GET_OR_ERR_RET(id, service, retval) do {          \
 	if (id >= RTE_SERVICE_NUM_MAX || !service_valid(id))            \
@@ -344,12 +350,14 @@ rte_service_runner_do_callback(struct rte_service_spec_impl *s,
 }
 
 
-static inline int32_t
-service_run(uint32_t i, struct core_state *cs, uint64_t service_mask)
+/* Expects the service 's' is valid. */
+static int32_t
+service_run(uint32_t i, struct core_state *cs, uint64_t service_mask,
+	    struct rte_service_spec_impl *s)
 {
-	if (!service_valid(i))
+	if (!s)
 		return -EINVAL;
-	struct rte_service_spec_impl *s = &rte_services[i];
+
 	if (s->comp_runstate != RUNSTATE_RUNNING ||
 			s->app_runstate != RUNSTATE_RUNNING ||
 			!(service_mask & (UINT64_C(1) << i))) {
@@ -383,7 +391,7 @@ rte_service_may_be_active(uint32_t id)
 	int32_t lcore_count = rte_service_lcore_list(ids, RTE_MAX_LCORE);
 	int i;
 
-	if (!service_valid(id))
+	if (id >= RTE_SERVICE_NUM_MAX || !service_valid(id))
 		return -EINVAL;
 
 	for (i = 0; i < lcore_count; i++) {
@@ -397,12 +405,10 @@ rte_service_may_be_active(uint32_t id)
 int32_t
 rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe)
 {
-	/* run service on calling core, using all-ones as the service mask */
-	if (!service_valid(id))
-		return -EINVAL;
-
 	struct core_state *cs = &lcore_states[rte_lcore_id()];
-	struct rte_service_spec_impl *s = &rte_services[id];
+	struct rte_service_spec_impl *s;
+
+	SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL);
 
 	/* Atomically add this core to the mapped cores first, then examine if
 	 * we can run the service. This avoids a race condition between
@@ -418,7 +424,7 @@ rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe)
 		return -EBUSY;
 	}
 
-	int ret = service_run(id, cs, UINT64_MAX);
+	int ret = service_run(id, cs, UINT64_MAX, s);
 
 	if (serialize_mt_unsafe)
 		rte_atomic32_dec(&s->num_mapped_cores);
@@ -438,8 +444,10 @@ rte_service_runner_func(void *arg)
 		const uint64_t service_mask = cs->service_mask;
 
 		for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) {
+			if (!service_valid(i))
+				continue;
 			/* return value ignored as no change to code flow */
-			service_run(i, cs, service_mask);
+			service_run(i, cs, service_mask, service_get(i));
 		}
 
 		cs->loops++;
Signature · 1 line
2.21.0
Reviewed-by on v2 1/1
#5 lore raw
Quoted 6 lines · Aaron Conole

The service_valid call is used without properly bounds checking the input parameter. Almost all instances of the service_valid call are inside a for() loop that prevents excessive walks, but some of the public APIs don't bounds check and will pass invalid arguments.

Quoted 7 lines

Prevent this by using SERVICE_GET_OR_ERR_RET where it makes sense, and adding a bounds check to one service_valid() use.

Fixes 8d39d3e237c2 ("service: fix race in service on app lcore function")
Fixes e9139a32f6e8 ("service: add function to run on app lcore")
Fixes e30dd31847d2 ("service: add mechanism for quiescing")
Signature · 1 line
David Marchand
#6 lore raw
Quoted 8 lines · David Marchand
Quoted 6 lines · Aaron Conole

The service_valid call is used without properly bounds checking the input parameter. Almost all instances of the service_valid call are inside a for() loop that prevents excessive walks, but some of the public APIs don't bounds check and will pass invalid arguments.

Quoted 6 lines
Quoted 6 lines

Prevent this by using SERVICE_GET_OR_ERR_RET where it makes sense, and adding a bounds check to one service_valid() use.

Fixes 8d39d3e237c2 ("service: fix race in service on app lcore function")
Fixes e9139a32f6e8 ("service: add function to run on app lcore")
Fixes e30dd31847d2 ("service: add mechanism for quiescing")
Quoted 3 lines
Quoted 1 line
Signature · 1 line
David Marchand
#7 lore raw
Quoted 10 lines · David Marchand
Quoted 8 lines · David Marchand
Quoted 6 lines · Aaron Conole

The service_valid call is used without properly bounds checking the input parameter. Almost all instances of the service_valid call are inside a for() loop that prevents excessive walks, but some of the public APIs don't bounds check and will pass invalid arguments.

Quoted 11 lines
Quoted 6 lines
Quoted 6 lines

Prevent this by using SERVICE_GET_OR_ERR_RET where it makes sense, and adding a bounds check to one service_valid() use.

Fixes 8d39d3e237c2 ("service: fix race in service on app lcore function")
Fixes e9139a32f6e8 ("service: add function to run on app lcore")
Fixes e30dd31847d2 ("service: add mechanism for quiescing")
Quoted 3 lines
Quoted 1 line

Applied, thanks.

--
David Marchand

#8 lore raw
Quoted 11 lines · David Marchand
Quoted 10 lines · David Marchand
Quoted 8 lines · David Marchand
Quoted 6 lines · Aaron Conole

The service_valid call is used without properly bounds checking the input parameter. Almost all instances of the service_valid call are inside a for() loop that prevents excessive walks, but some of the public APIs don't bounds check and will pass invalid arguments.

Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 6 lines

Prevent this by using SERVICE_GET_OR_ERR_RET where it makes sense, and adding a bounds check to one service_valid() use.

Fixes 8d39d3e237c2 ("service: fix race in service on app lcore function")
Fixes e9139a32f6e8 ("service: add function to run on app lcore")
Fixes e30dd31847d2 ("service: add mechanism for quiescing")

With the commit below, this patch will apply cleanly on 18.11.

Seems ok to me to add below commit, wdyt?

commit e484ccddbe1b41886fef1e445ef2fdfa55086198

Author: Nikhil Rao <[email protected]>
Date:   Mon Sep 16 15:31:02 2019 +0530

service: avoid false sharing on core state

Quoted 10 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

Applied, thanks.

--
David Marchand

#9 lore raw
Quoted 18 lines · Kevin Traynor
Quoted 17 lines · David Marchand
Quoted 16 lines · David Marchand
Quoted 14 lines · David Marchand
Quoted 12 lines · Aaron Conole

The service_valid call is used without properly bounds checking the input parameter. Almost all instances of the service_valid call are inside a for() loop that prevents excessive walks, but some of the public APIs don't bounds check and will pass invalid arguments.

Prevent this by using SERVICE_GET_OR_ERR_RET where it makes sense, and adding a bounds check to one service_valid() use.

Fixes 8d39d3e237c2 ("service: fix race in service on app lcore function")
Fixes e9139a32f6e8 ("service: add function to run on app lcore")
Fixes e30dd31847d2 ("service: add mechanism for quiescing")
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines

With the commit below, this patch will apply cleanly on 18.11.

Seems ok to me to add below commit, wdyt?

If I'm reading it correctly, the move is for an internal data structure in librte_eal, so I think it shouldn't be an ABI breakage.

Looks safe to me as well.

Quoted 17 lines

commit e484ccddbe1b41886fef1e445ef2fdfa55086198

Author: Nikhil Rao <[email protected]>
Date:   Mon Sep 16 15:31:02 2019 +0530

service: avoid false sharing on core state

Quoted 10 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

Applied, thanks.

--
David Marchand

#10 lore raw
Quoted 23 lines · Aaron Conole
Quoted 21 lines · Kevin Traynor
Quoted 19 lines · David Marchand
Quoted 18 lines · David Marchand
Quoted 8 lines · David Marchand
Quoted 6 lines · Aaron Conole

The service_valid call is used without properly bounds checking the input parameter. Almost all instances of the service_valid call are inside a for() loop that prevents excessive walks, but some of the public APIs don't bounds check and will pass invalid arguments.

Quoted 6 lines
Quoted 6 lines

Prevent this by using SERVICE_GET_OR_ERR_RET where it makes sense, and adding a bounds check to one service_valid() use.

Fixes 8d39d3e237c2 ("service: fix race in service on app lcore function")
Fixes e9139a32f6e8 ("service: add function to run on app lcore")
Fixes e30dd31847d2 ("service: add mechanism for quiescing")
Quoted 9 lines
Quoted 3 lines

With the commit below, this patch will apply cleanly on 18.11.

Seems ok to me to add below commit, wdyt?

If I'm reading it correctly, the move is for an internal data structure in librte_eal, so I think it shouldn't be an ABI breakage.

Looks safe to me as well.

Thanks, both patches applied.

Quoted 17 lines
Quoted 17 lines

commit e484ccddbe1b41886fef1e445ef2fdfa55086198

Author: Nikhil Rao <[email protected]>
Date:   Mon Sep 16 15:31:02 2019 +0530

service: avoid false sharing on core state

Quoted 10 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

Applied, thanks.

--
David Marchand