remoteproc: Add support for predefined notifyids
PATCH 1/1 remoteproc: Add support for predefined notifyids
In order to support preallocated notify ids, if their value is equal to FW_RSC_NOTIFY_ID_ANY, then do no allocate a notify id dynamically but try to allocate the requested one. This is useful when using custom ids to bind them to custom vendor resources. For instance, it allow to assign a group of queues to a specific interrupti in order to dispatch notifications.
--- drivers/remoteproc/remoteproc_core.c | 27 +++++++++++++++++++-------- include/linux/remoteproc.h | 1 + 2 files changed, 20 insertions(+), 8 deletions(-)
drivers/remoteproc/remoteproc_core.c +19 -8
@@ -351,14 +351,27 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)/* * Assign an rproc-wide unique index for this vring * TODO: assign a notifyid for rvdev updates as well -* TODO: support predefined notifyids (via resource table) */ - ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); -if (ret <0 ) { -dev_err (dev,"idr_alloc failed: %d\n" , ret); -return ret; +if (rsc->vring[i].notifyid ==FW_RSC_NOTIFY_ID_ANY ) { + ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } + notifyid = ret; + +/* Let the rproc know the notifyid of this vring.*/ + rsc->vring[i].notifyid = notifyid; + }else { +/* Reserve requested notify_id */ + notifyid = rsc->vring[i].notifyid; + ret =idr_alloc (&rproc->notifyids, rvring, notifyid, + notifyid +1 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } } - notifyid = ret;/* Potentially bump max_notifyid */ if (notifyid > rproc->max_notifyid)
@@ -366,8 +379,6 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) rvring->notifyid = notifyid; -/* Let the rproc know the notifyid of this vring.*/ - rsc->vring[i].notifyid = notifyid;return 0 ; }
include/linux/remoteproc.h +1 -0
@@ -123,6 +123,7 @@ enum fw_resource_type { };#define FW_RSC_ADDR_ANY (-1) +#define FW_RSC_NOTIFY_ID_ANY (-1)/** * struct fw_rsc_carveout - physically contiguous memory request
Signature · 1 line
2.15.0.276.g89ea799
Hi Clément,
Quoted 52 lines · Clement Leger
In order to support preallocated notify ids, if their value is equal to
FW_RSC_NOTIFY_ID_ANY, then do no allocate a notify id dynamically but try to allocate the requested one. This is useful when using custom ids to bind them to custom vendor resources. For instance, it allow to assign a group of queues to a specific interrupti in order to dispatch notifications.Signed-off-by Clement Leger <[email protected]>--- drivers/remoteproc/remoteproc_core.c | 27 +++++++++++++++++++-------- include/linux/remoteproc.h | 1 + 2 files changed, 20 insertions(+), 8 deletions(-)drivers/remoteproc/remoteproc_core.c +19
-6@@ -351,14 +351,27 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)/* * Assign an rproc-wide unique index for this vring * TODO: assign a notifyid for rvdev updates as well -* TODO: support predefined notifyids (via resource table) */ - ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); -if (ret <0 ) { -dev_err (dev,"idr_alloc failed: %d\n" , ret); -return ret; +if (rsc->vring[i].notifyid ==FW_RSC_NOTIFY_ID_ANY ) { + ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } + notifyid = ret; + +/* Let the rproc know the notifyid of this vring.*/ + rsc->vring[i].notifyid = notifyid; + }else { +/* Reserve requested notify_id */ + notifyid = rsc->vring[i].notifyid; + ret =idr_alloc (&rproc->notifyids, rvring, notifyid, + notifyid +1 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } } - notifyid = ret;/* Potentially bump max_notifyid */ if (notifyid > rproc->max_notifyid)
drivers/remoteproc/remoteproc_core.c int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) · line 383
Quoted 7 lines
drivers/remoteproc/remoteproc_core.c +0
-2@@ -366,8 +379,6 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) rvring->notifyid = notifyid; -/* Let the rproc know the notifyid of this vring.*/ - rsc->vring[i].notifyid = notifyid;return 0 ; }
The rproc_free_vring function resets the notifyid to -1 on free. This could generate a side effect if the resource table is not reloaded.
include/linux/remoteproc.h enum fw_resource_type { · line 126
Quoted 10 lines
include/linux/remoteproc.h +1
-0@@ -123,6 +123,7 @@ enum fw_resource_type { };#define FW_RSC_ADDR_ANY (-1) +#define FW_RSC_NOTIFY_ID_ANY (-1)This define can also be used in rproc_free_vring
Regards,
Arnaud
Quoted 4 lines
include/linux/remoteproc.h +0
-0/** * struct fw_rsc_carveout - physically contiguous memory request
Hi Arnaud,
Quoted 57 lines · ----- On 15 Jan, 2020, at 15:06, Arnaud Pouliquen [email protected] wrote:
Hi Clément,
Quoted 54 lines · Clement Leger
In order to support preallocated notify ids, if their value is equal to
FW_RSC_NOTIFY_ID_ANY, then do no allocate a notify id dynamically but try to allocate the requested one. This is useful when using custom ids to bind them to custom vendor resources. For instance, it allow to assign a group of queues to a specific interrupti in order to dispatch notifications.Signed-off-by Clement Leger <[email protected]>--- drivers/remoteproc/remoteproc_core.c | 27 +++++++++++++++++++-------- include/linux/remoteproc.h | 1 + 2 files changed, 20 insertions(+), 8 deletions(-)drivers/remoteproc/remoteproc_core.c +19
-6@@ -351,14 +351,27 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)/* * Assign an rproc-wide unique index for this vring * TODO: assign a notifyid for rvdev updates as well -* TODO: support predefined notifyids (via resource table) */ - ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); -if (ret <0 ) { -dev_err (dev,"idr_alloc failed: %d\n" , ret); -return ret; +if (rsc->vring[i].notifyid ==FW_RSC_NOTIFY_ID_ANY ) { + ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } + notifyid = ret; + +/* Let the rproc know the notifyid of this vring.*/ + rsc->vring[i].notifyid = notifyid; + }else { +/* Reserve requested notify_id */ + notifyid = rsc->vring[i].notifyid; + ret =idr_alloc (&rproc->notifyids, rvring, notifyid, + notifyid +1 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } } - notifyid = ret;/* Potentially bump max_notifyid */ if (notifyid > rproc->max_notifyid)@@ -366,8 +379,6 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)
Quoted 8 lines
drivers/remoteproc/remoteproc_core.c int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) · line 383
Quoted 6 lines
drivers/remoteproc/remoteproc_core.c +0
-2rvring->notifyid = notifyid; -/* Let the rproc know the notifyid of this vring.*/ - rsc->vring[i].notifyid = notifyid;return 0 ; }The
rproc_free_vringfunction resets the notifyid to -1 on free. This could generate a side effect if the resource table is not reloaded.
Oh indeed, I did not thought of that. What would you recommend ? If using -1 in free vring, notify ids will be reallocated at next round.
I was also worried that it would break some existing user applications which uses "0" as a notify id in vring but expect the id to be allocated dynamically. With my modification, it means it will try to use "0" as a predefined id, leading to allocation failure.
Quoted 9 lines
Quoted 9 lines
include/linux/remoteproc.h +0
-0@@ -123,6 +123,7 @@ enum fw_resource_type { };
Quoted 3 lines
Quoted 3 lines
include/linux/remoteproc.h +1
-0#define FW_RSC_ADDR_ANY (-1) +#define FW_RSC_NOTIFY_ID_ANY (-1)This define can also be used in
rproc_free_vring
Indeed.
Thanks for your review.
Regards,
Clément
Quoted 6 lines
Regards,
ArnaudQuoted 3 lines
/**
* structfw_rsc_carveout- physically contiguous memory request
Quoted 70 lines · Clément Leger
Hi Arnaud,
Quoted 57 lines · ----- On 15 Jan, 2020, at 15:06, Arnaud Pouliquen [email protected] wrote:
Hi Clément,
Quoted 54 lines · Clement Leger
In order to support preallocated notify ids, if their value is equal to
FW_RSC_NOTIFY_ID_ANY, then do no allocate a notify id dynamically but try to allocate the requested one. This is useful when using custom ids to bind them to custom vendor resources. For instance, it allow to assign a group of queues to a specific interrupti in order to dispatch notifications.Signed-off-by Clement Leger <[email protected]>--- drivers/remoteproc/remoteproc_core.c | 27 +++++++++++++++++++-------- include/linux/remoteproc.h | 1 + 2 files changed, 20 insertions(+), 8 deletions(-)drivers/remoteproc/remoteproc_core.c +19
-6@@ -351,14 +351,27 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)/* * Assign an rproc-wide unique index for this vring * TODO: assign a notifyid for rvdev updates as well -* TODO: support predefined notifyids (via resource table) */ - ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); -if (ret <0 ) { -dev_err (dev,"idr_alloc failed: %d\n" , ret); -return ret; +if (rsc->vring[i].notifyid ==FW_RSC_NOTIFY_ID_ANY ) { + ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } + notifyid = ret; + +/* Let the rproc know the notifyid of this vring.*/ + rsc->vring[i].notifyid = notifyid; + }else { +/* Reserve requested notify_id */ + notifyid = rsc->vring[i].notifyid; + ret =idr_alloc (&rproc->notifyids, rvring, notifyid, + notifyid +1 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } } - notifyid = ret;/* Potentially bump max_notifyid */ if (notifyid > rproc->max_notifyid)@@ -366,8 +379,6 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)Quoted 8 lines
drivers/remoteproc/remoteproc_core.c int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) · line 383
Quoted 6 lines
drivers/remoteproc/remoteproc_core.c +0
-2rvring->notifyid = notifyid; -/* Let the rproc know the notifyid of this vring.*/ - rsc->vring[i].notifyid = notifyid;return 0 ; }The
rproc_free_vringfunction resets the notifyid to -1 on free. This could generate a side effect if the resource table is not reloaded.
Quoted 3 lines
Oh indeed, I did not thought of that. What would you recommend ? If using -1 in free vring, notify ids will be reallocated at next round.
Regarding the code i'm not sure that it is useful to reset the notifyID to -1 on free. In current version, on alloc, the notifyID is overwriten without check.
And as vdev status is updated, vring struct in resource table should be considered as invalid Except if i missed a usecase/race condition...
Quoted 6 lines
I was also worried that it would break some existing user applications which uses "0" as a notify id in vring but expect the id to be allocated dynamically. With my modification, it means it will try to use "0" as a predefined id, leading to allocation failure.
Yes this could introduce regression for firmware that sets 0 as default value. Probably better to introduce this patch with a new version of the resource table :)
Regards
Arnaud
Quoted 27 lines
Quoted 9 lines
Quoted 9 lines
include/linux/remoteproc.h +0
-0@@ -123,6 +123,7 @@ enum fw_resource_type { };Quoted 3 lines
Quoted 3 lines
include/linux/remoteproc.h +1
-0#define FW_RSC_ADDR_ANY (-1) +#define FW_RSC_NOTIFY_ID_ANY (-1)This define can also be used in
rproc_free_vringIndeed.
Thanks for your review.
Regards,
Clément
Quoted 6 lines
Regards,
ArnaudQuoted 3 lines
/**
* structfw_rsc_carveout- physically contiguous memory request
Quoted 82 lines · ----- On 15 Jan, 2020, at 16:09, Arnaud Pouliquen [email protected] wrote:
Quoted 70 lines · Clément Leger
Hi Arnaud,
Quoted 57 lines · ----- On 15 Jan, 2020, at 15:06, Arnaud Pouliquen [email protected] wrote:
Hi Clément,
Quoted 54 lines · Clement Leger
In order to support preallocated notify ids, if their value is equal to
FW_RSC_NOTIFY_ID_ANY, then do no allocate a notify id dynamically but try to allocate the requested one. This is useful when using custom ids to bind them to custom vendor resources. For instance, it allow to assign a group of queues to a specific interrupti in order to dispatch notifications.Signed-off-by Clement Leger <[email protected]>--- drivers/remoteproc/remoteproc_core.c | 27 +++++++++++++++++++-------- include/linux/remoteproc.h | 1 + 2 files changed, 20 insertions(+), 8 deletions(-)drivers/remoteproc/remoteproc_core.c +19
-6@@ -351,14 +351,27 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)/* * Assign an rproc-wide unique index for this vring * TODO: assign a notifyid for rvdev updates as well -* TODO: support predefined notifyids (via resource table) */ - ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); -if (ret <0 ) { -dev_err (dev,"idr_alloc failed: %d\n" , ret); -return ret; +if (rsc->vring[i].notifyid ==FW_RSC_NOTIFY_ID_ANY ) { + ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } + notifyid = ret; + +/* Let the rproc know the notifyid of this vring.*/ + rsc->vring[i].notifyid = notifyid; + }else { +/* Reserve requested notify_id */ + notifyid = rsc->vring[i].notifyid; + ret =idr_alloc (&rproc->notifyids, rvring, notifyid, + notifyid +1 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } } - notifyid = ret;/* Potentially bump max_notifyid */ if (notifyid > rproc->max_notifyid)@@ -366,8 +379,6 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)Quoted 8 lines
drivers/remoteproc/remoteproc_core.c int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) · line 383
Quoted 6 lines
drivers/remoteproc/remoteproc_core.c +0
-2rvring->notifyid = notifyid; -/* Let the rproc know the notifyid of this vring.*/ - rsc->vring[i].notifyid = notifyid;return 0 ; }The
rproc_free_vringfunction resets the notifyid to -1 on free. This could generate a side effect if the resource table is not reloaded.Quoted 3 lines
Oh indeed, I did not thought of that. What would you recommend ? If using -1 in free vring, notify ids will be reallocated at next round.
Regarding the code i'm not sure that it is useful to reset the notifyID to -1 on free.
In current version, on alloc, the notifyID is overwriten without check. And as vdev status is updated, vring struct in resource table should be considered as invalid
Except if i missed a usecase/race condition...Quoted 1 line
Quoted 8 lines
Quoted 5 lines
I was also worried that it would break some existing user applications which uses "0" as a notify id in vring but expect the id to be allocated dynamically. With my modification, it means it will try to use "0" as a predefined id, leading to allocation failure.
Yes this could introduce regression for firmware that sets 0 as default value. Probably better to introduce this patch with a new version of the resource table :)
Understood ;)
Regards,
Clément
Quoted 30 lines
Regards
ArnaudQuoted 27 lines
Quoted 9 lines
Quoted 9 lines
include/linux/remoteproc.h +0
-0@@ -123,6 +123,7 @@ enum fw_resource_type { };Quoted 3 lines
Quoted 3 lines
include/linux/remoteproc.h +1
-0#define FW_RSC_ADDR_ANY (-1) +#define FW_RSC_NOTIFY_ID_ANY (-1)This define can also be used in
rproc_free_vringIndeed.
Thanks for your review.
Regards,
Clément
Quoted 6 lines
Regards,
ArnaudQuoted 3 lines
/**
* structfw_rsc_carveout- physically contiguous memory request
Mathieu Poirier
maintainerHey guys,
Quoted 75 lines · Cl�ment Leger
Quoted 71 lines · ----- On 15 Jan, 2020, at 16:09, Arnaud Pouliquen [email protected] wrote:
Quoted 70 lines · Cl�ment Leger
Hi Arnaud,
Quoted 65 lines · ----- On 15 Jan, 2020, at 15:06, Arnaud Pouliquen [email protected] wrote:
Hi Cl�ment,
Quoted 53 lines · Clement Leger
In order to support preallocated notify ids, if their value is equal to
FW_RSC_NOTIFY_ID_ANY, then do no allocate a notify id dynamically but try to allocate the requested one. This is useful when using custom ids to bind them to custom vendor resources. For instance, it allow to assign a group of queues to a specific interrupti in order to dispatch notifications.Signed-off-by Clement Leger <[email protected]>--- drivers/remoteproc/remoteproc_core.c | 27 +++++++++++++++++++-------- include/linux/remoteproc.h | 1 + 2 files changed, 20 insertions(+), 8 deletions(-)drivers/remoteproc/remoteproc_core.c +19
-6@@ -351,14 +351,27 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)/* * Assign an rproc-wide unique index for this vring * TODO: assign a notifyid for rvdev updates as well -* TODO: support predefined notifyids (via resource table) */ - ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); -if (ret <0 ) { -dev_err (dev,"idr_alloc failed: %d\n" , ret); -return ret; +if (rsc->vring[i].notifyid ==FW_RSC_NOTIFY_ID_ANY ) { + ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } + notifyid = ret; + +/* Let the rproc know the notifyid of this vring.*/ + rsc->vring[i].notifyid = notifyid; + }else { +/* Reserve requested notify_id */ + notifyid = rsc->vring[i].notifyid; + ret =idr_alloc (&rproc->notifyids, rvring, notifyid, + notifyid +1 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } } - notifyid = ret;/* Potentially bump max_notifyid */ if (notifyid > rproc->max_notifyid)drivers/remoteproc/remoteproc_core.c int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) · line 383
Quoted 7 lines
drivers/remoteproc/remoteproc_core.c +0
-2@@ -366,8 +379,6 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) rvring->notifyid = notifyid; -/* Let the rproc know the notifyid of this vring.*/ - rsc->vring[i].notifyid = notifyid;return 0 ; }The
rproc_free_vringfunction resets the notifyid to -1 on free. This could generate a side effect if the resource table is not reloaded.
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Oh indeed, I did not thought of that. What would you recommend ? If using -1 in free vring, notify ids will be reallocated at next round.
Regarding the code i'm not sure that it is useful to reset the notifyID to -1 on free.
I'm not sure setting notifyid to -1 in rproc_free_vring() is such a big problem. No matter the code path I look at, if rproc_free_vring() is called something serious has happened and the resource table will be reloaded if another attempt at booting the remote processor is done. It can also be that a graceful shutdown is underway, in which case the resource table will be reloaded anyway if/when the slave is brought back in service.
Let me know if I'm missing a scenario.
To me the real problem is if a FW image has set the notifyids in the resource table to 0xffffffff, thinking they will be overwritten. In that case things will really south.
Quoted 6 lines
Quoted 6 lines
In current version, on alloc, the notifyID is overwriten without check. And as vdev status is updated, vring struct in resource table should be considered as invalid
Except if i missed a usecase/race condition...Quoted 1 line
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
I was also worried that it would break some existing user applications which uses "0" as a notify id in vring but expect the id to be allocated dynamically. With my modification, it means it will try to use "0" as a predefined id, leading to allocation failure.
From my point of view they will have been lucky for all this time. Even with
a new version of the resource table (which I think is the right way go) cases like this will break.
Thanks,
Mathieu
Quoted 41 lines
Quoted 4 lines
Quoted 1 line
Yes this could introduce regression for firmware that sets 0 as default value. Probably better to introduce this patch with a new version of the resource table :)
Understood ;)
Regards,
Cl�ment
Quoted 30 lines
Regards
ArnaudQuoted 27 lines
Quoted 9 lines
Quoted 9 lines
include/linux/remoteproc.h +0
-0@@ -123,6 +123,7 @@ enum fw_resource_type { };Quoted 3 lines
Quoted 3 lines
include/linux/remoteproc.h +1
-0#define FW_RSC_ADDR_ANY (-1) +#define FW_RSC_NOTIFY_ID_ANY (-1)This define can also be used in
rproc_free_vringIndeed.
Thanks for your review.
Regards,
Cl�ment
Quoted 6 lines
Regards,
ArnaudQuoted 3 lines
/**
* structfw_rsc_carveout- physically contiguous memory request
Hi Mathieu,
Quoted 84 lines · ----- On 17 Jan, 2020, at 23:52, Mathieu Poirier [email protected]…
Hey guys,
Quoted 75 lines · Clément Leger
Quoted 71 lines · ----- On 15 Jan, 2020, at 16:09, Arnaud Pouliquen [email protected] wrote:
Quoted 70 lines · Clément Leger
Hi Arnaud,
Quoted 65 lines · ----- On 15 Jan, 2020, at 15:06, Arnaud Pouliquen [email protected] wrote:
Hi Clément,
Quoted 53 lines · Clement Leger
In order to support preallocated notify ids, if their value is equal to
FW_RSC_NOTIFY_ID_ANY, then do no allocate a notify id dynamically but try to allocate the requested one. This is useful when using custom ids to bind them to custom vendor resources. For instance, it allow to assign a group of queues to a specific interrupti in order to dispatch notifications.Signed-off-by Clement Leger <[email protected]>--- drivers/remoteproc/remoteproc_core.c | 27 +++++++++++++++++++-------- include/linux/remoteproc.h | 1 + 2 files changed, 20 insertions(+), 8 deletions(-)drivers/remoteproc/remoteproc_core.c +19
-6@@ -351,14 +351,27 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)/* * Assign an rproc-wide unique index for this vring * TODO: assign a notifyid for rvdev updates as well -* TODO: support predefined notifyids (via resource table) */ - ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); -if (ret <0 ) { -dev_err (dev,"idr_alloc failed: %d\n" , ret); -return ret; +if (rsc->vring[i].notifyid ==FW_RSC_NOTIFY_ID_ANY ) { + ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } + notifyid = ret; + +/* Let the rproc know the notifyid of this vring.*/ + rsc->vring[i].notifyid = notifyid; + }else { +/* Reserve requested notify_id */ + notifyid = rsc->vring[i].notifyid; + ret =idr_alloc (&rproc->notifyids, rvring, notifyid, + notifyid +1 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } } - notifyid = ret;/* Potentially bump max_notifyid */ if (notifyid > rproc->max_notifyid)drivers/remoteproc/remoteproc_core.c int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) · line 383
Quoted 7 lines
drivers/remoteproc/remoteproc_core.c +0
-2@@ -366,8 +379,6 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) rvring->notifyid = notifyid; -/* Let the rproc know the notifyid of this vring.*/ - rsc->vring[i].notifyid = notifyid;return 0 ; }The
rproc_free_vringfunction resets the notifyid to -1 on free. This could generate a side effect if the resource table is not reloaded.Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Oh indeed, I did not thought of that. What would you recommend ? If using -1 in free vring, notify ids will be reallocated at next round.
Regarding the code i'm not sure that it is useful to reset the notifyID to -1 on free.
Quoted 8 lines
I'm not sure setting notifyid to -1 in
rproc_free_vring()is such a big problem. No matter the code path I look at, ifrproc_free_vring()is called something serious has happened and the resource table will be reloaded if another attempt at booting the remote processor is done. It can also be that a graceful shutdown is underway, in which case the resource table will be reloaded anyway if/when the slave is brought back in service.Let me know if I'm missing a scenario.
No, you are actually right
Quoted 4 lines
To me the real problem is if a FW image has set the notifyids in the resource table to
0xffffffff, thinking they will be overwritten. In that case things will really south.
Hum, if set to 0xFFFFFFFF, then they will be assigned dynamically and updated in the resource table (with this patch). But your probably mean existing code, right ?
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
In current version, on alloc, the notifyID is overwriten without check. And as vdev status is updated, vring struct in resource table should be considered as invalid
Except if i missed a usecase/race condition...Quoted 1 line
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
I was also worried that it would break some existing user applications which uses "0" as a notify id in vring but expect the id to be allocated dynamically. With my modification, it means it will try to use "0" as a predefined id, leading to allocation failure.
From my point of view they will have been lucky for all this time. Even with a new version of the resource table (which I think is the right way go) cases like this will break.
Agreed, and actually, I may have missread some code. I can't find were I read that. By the way, is there any documentation which state the allowed values of notify ids ?
Quoted 45 lines
Thanks,
MathieuQuoted 41 lines
Quoted 4 lines
Quoted 1 line
Yes this could introduce regression for firmware that sets 0 as default value. Probably better to introduce this patch with a new version of the resource table :)
Understood ;)
Regards,
Clément
Quoted 30 lines
Regards
ArnaudQuoted 27 lines
Quoted 9 lines
Quoted 9 lines
include/linux/remoteproc.h +0
-0@@ -123,6 +123,7 @@ enum fw_resource_type { };Quoted 3 lines
Quoted 3 lines
include/linux/remoteproc.h +1
-0#define FW_RSC_ADDR_ANY (-1) +#define FW_RSC_NOTIFY_ID_ANY (-1)This define can also be used in
rproc_free_vringIndeed.
Thanks for your review.
Regards,
Clément
Quoted 6 lines
Regards,
ArnaudQuoted 3 lines
/**
* structfw_rsc_carveout- physically contiguous memory request
Hi,
Quoted 95 lines · Clément Leger
Hi Mathieu,
Quoted 91 lines · ----- On 17 Jan, 2020, at 23:52, Mathieu Poirier [email protected]…
Hey guys,
Quoted 75 lines · Clément Leger
Quoted 71 lines · ----- On 15 Jan, 2020, at 16:09, Arnaud Pouliquen [email protected] wrote:
Quoted 70 lines · Clément Leger
Hi Arnaud,
Quoted 65 lines · ----- On 15 Jan, 2020, at 15:06, Arnaud Pouliquen [email protected] wrote:
Hi Clément,
Quoted 53 lines · Clement Leger
In order to support preallocated notify ids, if their value is equal to
FW_RSC_NOTIFY_ID_ANY, then do no allocate a notify id dynamically but try to allocate the requested one. This is useful when using custom ids to bind them to custom vendor resources. For instance, it allow to assign a group of queues to a specific interrupti in order to dispatch notifications.Signed-off-by Clement Leger <[email protected]>--- drivers/remoteproc/remoteproc_core.c | 27 +++++++++++++++++++-------- include/linux/remoteproc.h | 1 + 2 files changed, 20 insertions(+), 8 deletions(-)drivers/remoteproc/remoteproc_core.c +19
-6@@ -351,14 +351,27 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)/* * Assign an rproc-wide unique index for this vring * TODO: assign a notifyid for rvdev updates as well -* TODO: support predefined notifyids (via resource table) */ - ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); -if (ret <0 ) { -dev_err (dev,"idr_alloc failed: %d\n" , ret); -return ret; +if (rsc->vring[i].notifyid ==FW_RSC_NOTIFY_ID_ANY ) { + ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } + notifyid = ret; + +/* Let the rproc know the notifyid of this vring.*/ + rsc->vring[i].notifyid = notifyid; + }else { +/* Reserve requested notify_id */ + notifyid = rsc->vring[i].notifyid; + ret =idr_alloc (&rproc->notifyids, rvring, notifyid, + notifyid +1 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } } - notifyid = ret;/* Potentially bump max_notifyid */ if (notifyid > rproc->max_notifyid)drivers/remoteproc/remoteproc_core.c int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) · line 383
Quoted 7 lines
drivers/remoteproc/remoteproc_core.c +0
-2@@ -366,8 +379,6 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) rvring->notifyid = notifyid; -/* Let the rproc know the notifyid of this vring.*/ - rsc->vring[i].notifyid = notifyid;return 0 ; }The
rproc_free_vringfunction resets the notifyid to -1 on free. This could generate a side effect if the resource table is not reloaded.Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Oh indeed, I did not thought of that. What would you recommend ? If using -1 in free vring, notify ids will be reallocated at next round.
Regarding the code i'm not sure that it is useful to reset the notifyID to -1 on free.
I'm not sure setting notifyid to -1 in
rproc_free_vring()is such a big problem. No matter the code path I look at, ifrproc_free_vring()is called something serious has happened and the resource table will be reloaded if another attempt at booting the remote processor is done. It can also be that a graceful shutdown is underway, in which case the resource table will be reloaded anyway if/when the slave is brought back in service.
Quoted 3 lines
Quoted 1 line
Let me know if I'm missing a scenario.
No, you are actually right
My concern here is that the resource table can be uncorrelated from the Firmware.
For instance the remote proc firmware could run autonomously.
In this case only the resource table will be managed.
Should we consider that this resource table must be reloaded on a stop/start from application? Now regarding the da that is also reset to 0 (instead of FW_RSC_ADDR_ANY) in rproc_free_vring, i suppose that my scenario is not realistic...on restart rproc would try to allocate vring at address 0, Leading to a crash.
Quoted 9 lines
Quoted 5 lines
To me the real problem is if a FW image has set the notifyids in the resource table to
0xffffffff, thinking they will be overwritten. In that case things will really south.Hum, if set to
0xFFFFFFFF, then they will be assigned dynamically and updated in the resource table (with this patch). But your probably mean existing code, right ?
point also not clear for me. In existing code the notifyids is always overwritten. Mathieu could you details what you have in mind here?
Quoted 13 lines
Quoted 13 lines
Quoted 12 lines
Quoted 12 lines
In current version, on alloc, the notifyID is overwriten without check. And as vdev status is updated, vring struct in resource table should be considered as invalid
Except if i missed a usecase/race condition...Quoted 5 lines
I was also worried that it would break some existing user applications which uses "0" as a notify id in vring but expect the id to be allocated dynamically. With my modification, it means it will try to use "0" as a predefined id, leading to allocation failure.
Quoted 7 lines
Quoted 3 lines
From my point of view they will have been lucky for all this time. Even with a new version of the resource table (which I think is the right way go) cases like this will break.
Agreed, and actually, I may have missread some code. I can't find were I read that. By the way, is there any documentation which state the allowed values of notify ids ?
0 seems a valid value. I'm not aware about a constraint on the notify ID.
With the current implementation, the initial values are simply ignored, therefore do not seem to be a problem.
With Clément's patch, the issue is for legacy firmwares with both vrings IDs initialized to the same value (0 or another value different from -1). The rproc_alloc_vring fails while trying to reserve the same IDs for both vring.
If this patch is introduced with a new version of the resource table a constraint on the notiifyID seems acceptable.
Thanks,
Arnaud
Quoted 46 lines
Quoted 46 lines
Thanks,
MathieuQuoted 41 lines
Quoted 4 lines
Quoted 1 line
Yes this could introduce regression for firmware that sets 0 as default value. Probably better to introduce this patch with a new version of the resource table :)
Understood ;)
Regards,
Clément
Quoted 30 lines
Regards
ArnaudQuoted 27 lines
Quoted 9 lines
Quoted 9 lines
include/linux/remoteproc.h +0
-0@@ -123,6 +123,7 @@ enum fw_resource_type { };Quoted 3 lines
Quoted 3 lines
include/linux/remoteproc.h +1
-0#define FW_RSC_ADDR_ANY (-1) +#define FW_RSC_NOTIFY_ID_ANY (-1)This define can also be used in
rproc_free_vringIndeed.
Thanks for your review.
Regards,
Clément
Quoted 6 lines
Regards,
ArnaudQuoted 3 lines
/**
* structfw_rsc_carveout- physically contiguous memory request
Mathieu Poirier
maintainerQuoted 101 lines · Clément Leger
Hi Mathieu,
Quoted 84 lines · ----- On 17 Jan, 2020, at 23:52, Mathieu Poirier [email protected]…
Hey guys,
Quoted 75 lines · Clément Leger
Quoted 71 lines · ----- On 15 Jan, 2020, at 16:09, Arnaud Pouliquen [email protected] wrote:
Quoted 70 lines · Clément Leger
Hi Arnaud,
Quoted 65 lines · ----- On 15 Jan, 2020, at 15:06, Arnaud Pouliquen [email protected] wrote:
Hi Clément,
Quoted 53 lines · Clement Leger
In order to support preallocated notify ids, if their value is equal to
FW_RSC_NOTIFY_ID_ANY, then do no allocate a notify id dynamically but try to allocate the requested one. This is useful when using custom ids to bind them to custom vendor resources. For instance, it allow to assign a group of queues to a specific interrupti in order to dispatch notifications.Signed-off-by Clement Leger <[email protected]>--- drivers/remoteproc/remoteproc_core.c | 27 +++++++++++++++++++-------- include/linux/remoteproc.h | 1 + 2 files changed, 20 insertions(+), 8 deletions(-)drivers/remoteproc/remoteproc_core.c +19
-6@@ -351,14 +351,27 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)/* * Assign an rproc-wide unique index for this vring * TODO: assign a notifyid for rvdev updates as well -* TODO: support predefined notifyids (via resource table) */ - ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); -if (ret <0 ) { -dev_err (dev,"idr_alloc failed: %d\n" , ret); -return ret; +if (rsc->vring[i].notifyid ==FW_RSC_NOTIFY_ID_ANY ) { + ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } + notifyid = ret; + +/* Let the rproc know the notifyid of this vring.*/ + rsc->vring[i].notifyid = notifyid; + }else { +/* Reserve requested notify_id */ + notifyid = rsc->vring[i].notifyid; + ret =idr_alloc (&rproc->notifyids, rvring, notifyid, + notifyid +1 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } } - notifyid = ret;/* Potentially bump max_notifyid */ if (notifyid > rproc->max_notifyid)drivers/remoteproc/remoteproc_core.c int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) · line 383
Quoted 7 lines
drivers/remoteproc/remoteproc_core.c +0
-2@@ -366,8 +379,6 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) rvring->notifyid = notifyid; -/* Let the rproc know the notifyid of this vring.*/ - rsc->vring[i].notifyid = notifyid;return 0 ; }The
rproc_free_vringfunction resets the notifyid to -1 on free. This could generate a side effect if the resource table is not reloaded.Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Oh indeed, I did not thought of that. What would you recommend ? If using -1 in free vring, notify ids will be reallocated at next round.
Regarding the code i'm not sure that it is useful to reset the notifyID to -1 on free.
Quoted 8 lines
I'm not sure setting notifyid to -1 in
rproc_free_vring()is such a big problem. No matter the code path I look at, ifrproc_free_vring()is called something serious has happened and the resource table will be reloaded if another attempt at booting the remote processor is done. It can also be that a graceful shutdown is underway, in which case the resource table will be reloaded anyway if/when the slave is brought back in service.Let me know if I'm missing a scenario.
No, you are actually right
Quoted 1 line
Quoted 7 lines
Quoted 3 lines
To me the real problem is if a FW image has set the notifyids in the resource table to
0xffffffff, thinking they will be overwritten. In that case things will really south.Hum, if set to
0xFFFFFFFF, then they will be assigned dynamically and updated in the resource table (with this patch). But your probably mean existing code, right ?
My apologies for not expressing myself clearly here - let me try again.
At this time notifyids in the firmware's resource table can be set to anything because the code will overwrite them. With this patch firmware images that don't have their notifyids set to -1 will see a change in how ids are assigned, something that has the potential to break user space.
Regards,
Mathieu
Quoted 66 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
In current version, on alloc, the notifyID is overwriten without check. And as vdev status is updated, vring struct in resource table should be considered as invalid
Except if i missed a usecase/race condition...Quoted 1 line
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
I was also worried that it would break some existing user applications which uses "0" as a notify id in vring but expect the id to be allocated dynamically. With my modification, it means it will try to use "0" as a predefined id, leading to allocation failure.
From my point of view they will have been lucky for all this time. Even with a new version of the resource table (which I think is the right way go) cases like this will break.
Agreed, and actually, I may have missread some code. I can't find were I read that. By the way, is there any documentation which state the allowed values of notify ids ?
Quoted 45 lines
Thanks,
MathieuQuoted 41 lines
Quoted 4 lines
Quoted 1 line
Yes this could introduce regression for firmware that sets 0 as default value. Probably better to introduce this patch with a new version of the resource table :)
Understood ;)
Regards,
Clément
Quoted 30 lines
Regards
ArnaudQuoted 27 lines
Quoted 9 lines
Quoted 9 lines
include/linux/remoteproc.h +0
-0@@ -123,6 +123,7 @@ enum fw_resource_type { };Quoted 3 lines
Quoted 3 lines
include/linux/remoteproc.h +1
-0#define FW_RSC_ADDR_ANY (-1) +#define FW_RSC_NOTIFY_ID_ANY (-1)This define can also be used in
rproc_free_vringIndeed.
Thanks for your review.
Regards,
Clément
Quoted 6 lines
Regards,
ArnaudQuoted 3 lines
/**
* structfw_rsc_carveout- physically contiguous memory request
Quoted 110 lines · Mathieu Poirier
Quoted 101 lines · Cl�ment Leger
Hi Mathieu,
Quoted 84 lines · ----- On 17 Jan, 2020, at 23:52, Mathieu Poirier [email protected]…
Hey guys,
Quoted 75 lines · Cl�ment Leger
Quoted 71 lines · ----- On 15 Jan, 2020, at 16:09, Arnaud Pouliquen [email protected] wrote:
Quoted 70 lines · Cl�ment Leger
Hi Arnaud,
Quoted 65 lines · ----- On 15 Jan, 2020, at 15:06, Arnaud Pouliquen [email protected] wrote:
Hi Cl�ment,
Quoted 53 lines · Clement Leger
In order to support preallocated notify ids, if their value is equal to
FW_RSC_NOTIFY_ID_ANY, then do no allocate a notify id dynamically but try to allocate the requested one. This is useful when using custom ids to bind them to custom vendor resources. For instance, it allow to assign a group of queues to a specific interrupti in order to dispatch notifications.Signed-off-by Clement Leger <[email protected]>--- drivers/remoteproc/remoteproc_core.c | 27 +++++++++++++++++++-------- include/linux/remoteproc.h | 1 + 2 files changed, 20 insertions(+), 8 deletions(-)drivers/remoteproc/remoteproc_core.c +19
-6@@ -351,14 +351,27 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)/* * Assign an rproc-wide unique index for this vring * TODO: assign a notifyid for rvdev updates as well -* TODO: support predefined notifyids (via resource table) */ - ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); -if (ret <0 ) { -dev_err (dev,"idr_alloc failed: %d\n" , ret); -return ret; +if (rsc->vring[i].notifyid ==FW_RSC_NOTIFY_ID_ANY ) { + ret =idr_alloc (&rproc->notifyids, rvring,0 ,0 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } + notifyid = ret; + +/* Let the rproc know the notifyid of this vring.*/ + rsc->vring[i].notifyid = notifyid; + }else { +/* Reserve requested notify_id */ + notifyid = rsc->vring[i].notifyid; + ret =idr_alloc (&rproc->notifyids, rvring, notifyid, + notifyid +1 ,GFP_KERNEL ); +if (ret <0 ) { +dev_err (dev,"idr_alloc failed: %d\n" , ret); +return ret; + } } - notifyid = ret;/* Potentially bump max_notifyid */ if (notifyid > rproc->max_notifyid)drivers/remoteproc/remoteproc_core.c int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) · line 383
Quoted 7 lines
drivers/remoteproc/remoteproc_core.c +0
-2@@ -366,8 +379,6 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i) rvring->notifyid = notifyid; -/* Let the rproc know the notifyid of this vring.*/ - rsc->vring[i].notifyid = notifyid;return 0 ; }The
rproc_free_vringfunction resets the notifyid to -1 on free. This could generate a side effect if the resource table is not reloaded.Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Oh indeed, I did not thought of that. What would you recommend ? If using -1 in free vring, notify ids will be reallocated at next round.
Regarding the code i'm not sure that it is useful to reset the notifyID to -1 on free.
Quoted 8 lines
I'm not sure setting notifyid to -1 in
rproc_free_vring()is such a big problem. No matter the code path I look at, ifrproc_free_vring()is called something serious has happened and the resource table will be reloaded if another attempt at booting the remote processor is done. It can also be that a graceful shutdown is underway, in which case the resource table will be reloaded anyway if/when the slave is brought back in service.Let me know if I'm missing a scenario.
No, you are actually right
Quoted 1 line
Quoted 7 lines
Quoted 3 lines
To me the real problem is if a FW image has set the notifyids in the resource table to
0xffffffff, thinking they will be overwritten. In that case things will really south.Hum, if set to
0xFFFFFFFF, then they will be assigned dynamically and updated in the resource table (with this patch). But your probably mean existing code, right ?
Quoted 8 lines
My apologies for not expressing myself clearly here - let me try again.
At this time notifyids in the firmware's resource table can be set to anything because the code will overwrite them. With this patch firmware images that don't have their notifyids set to -1 will see a change in how ids are assigned, something that has the potential to break user space.
Right, we should have had a check in place to ensure that we only accepted -1 here, as that's what the code relies upon.
If a conclusion on the new resource table format is imminent it sounds reasonable to document the behaviour for the current version and make it behave as as expected in the new one.
Regards,
Bjorn