sctp: make sctp_setsockopt_events() less strict about the option length
Subsystems (3)
PATCH 1/1 sctp: make sctp_setsockopt_events() less strict about the option length
Make sctp_setsockopt_events() able to accept sctp_event_subscribe structures longer than the current definitions.
This should prevent unjustified setsockopt() failures due to struct sctp_event_subscribe extensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.
--- net/sctp/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/sctp/socket.c +1 -1
@@ -2311,7 +2311,7 @@ static int sctp_setsockopt_events(struct sock *sk, char __user *optval,int i;if (optlen >sizeof (struct sctp_event_subscribe )) -return -EINVAL ; + optlen =sizeof (struct sctp_event_subscribe );if (copy_from_user (&subscribe, optval, optlen))return -EFAULT ;
Signature · 1 line
2.20.1
Marcelo Ricardo Leitner
maintainerQuoted 7 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.
Not sure if we support backwards compatibility like this?
My issue with this change is that by doing this, application will have no clue if the new bits were ignored or not and it may think that an event is enabled while it is not.
A workaround would be to do a getsockopt and check the size that was returned. But then, it might as well use the right struct here in the first place.
I'm seeing current implementation as an implicitly versioned argument:
it will always accept setsockopt calls with an old struct (v4.11 or v4.12), but if the user tries to use v3 on a v1-only system, it will be rejected. Pretty much like using a newer setsockopt on an old system.
Quoted 22 lines
Signed-off-by Julien Gomes <[email protected]>--- net/sctp/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sctp/socket.c +1
-1@@ -2311,7 +2311,7 @@ static int sctp_setsockopt_events(struct sock *sk, char __user *optval,int i;if (optlen >sizeof (struct sctp_event_subscribe )) -return -EINVAL ; + optlen =sizeof (struct sctp_event_subscribe );if (copy_from_user (&subscribe, optval, optlen))return -EFAULT ;Signature · 1 line
2.20.1
Quoted 19 lines · Marcelo Ricardo Leitner
Quoted 7 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
My issue with this change is that by doing this, application will have no clue if the new bits were ignored or not and it may think that an event is enabled while it is not.
A workaround would be to do a getsockopt and check the size that was returned. But then, it might as well use the right struct here in the first place.
Quoted 5 lines
I'm seeing current implementation as an implicitly versioned argument:
it will always accept setsockopt calls with an old struct (v4.11 or v4.12), but if the user tries to use v3 on a v1-only system, it will be rejected. Pretty much like using a newer setsockopt on an old system.
With the current implementation, given sources that say are supposed to run on a 4.9 kernel (no use of any newer field added in 4.11 or 4.12), we can't rebuild the exact same sources on a 4.19 kernel and still run them on 4.9 without messing with structures re-definition.
I understand your point, but this still looks like a sort of uapi breakage to me.
I also had another way to work-around this in mind, by copying optlen bytes and checking that any additional field (not included in the "current" kernel structure definition) is not set, returning EINVAL in such case to keep a similar to current behavior.
The issue with this is that I didn't find a suitable (ie not totally arbitrary such as "twice the existing structure size") upper limit to optlen.
Quoted 23 lines
Quoted 23 lines
Signed-off-by Julien Gomes <[email protected]>--- net/sctp/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sctp/socket.c +1
-1@@ -2311,7 +2311,7 @@ static int sctp_setsockopt_events(struct sock *sk, char __user *optval,int i;if (optlen >sizeof (struct sctp_event_subscribe )) -return -EINVAL ; + optlen =sizeof (struct sctp_event_subscribe );if (copy_from_user (&subscribe, optval, optlen))return -EFAULT ;Signature · 1 line
2.20.1
Marcelo Ricardo Leitner
maintainerQuoted 28 lines · Julien Gomes
Quoted 19 lines · Marcelo Ricardo Leitner
Quoted 7 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
My issue with this change is that by doing this, application will have no clue if the new bits were ignored or not and it may think that an event is enabled while it is not.
A workaround would be to do a getsockopt and check the size that was returned. But then, it might as well use the right struct here in the first place.
Quoted 5 lines
I'm seeing current implementation as an implicitly versioned argument:
it will always accept setsockopt calls with an old struct (v4.11 or v4.12), but if the user tries to use v3 on a v1-only system, it will be rejected. Pretty much like using a newer setsockopt on an old system.
Quoted 4 lines
With the current implementation, given sources that say are supposed to run on a 4.9 kernel (no use of any newer field added in 4.11 or 4.12), we can't rebuild the exact same sources on a 4.19 kernel and still run them on 4.9 without messing with structures re-definition.
Maybe what we want(ed) here then is explicit versioning, to have the 3 definitions available. Then the application is able to use, say struct sctp_event_subscribe, and be happy with it, while there is struct sctp_event_subscribe_v2 and struct sctp_event_subscribe_v3 there too.
But it's too late for that now because that would break applications already using the new fields in sctp_event_subscribe.
Quoted 3 lines
I understand your point, but this still looks like a sort of uapi breakage to me.
Not disagreeing. I really just don't know how supported that is. Willing to know so I can pay more attention to this on future changes.
Btw, is this the only occurrence?
Quoted 9 lines
I also had another way to work-around this in mind, by copying optlen bytes and checking that any additional field (not included in the "current" kernel structure definition) is not set, returning EINVAL in such case to keep a similar to current behavior.
The issue with this is that I didn't find a suitable (ie not totally arbitrary such as "twice the existing structure size") upper limit to optlen.
Seems interesting. Why would it need that upper limit to optlen?
Say struct v1 had 4 bytes, v3 now had 12. The user supplies 12 bytes to the kernel that only knows about 4 bytes. It can check that (12-4) bytes in the end, make sure no bit is on and use only the first 4.
The fact that it was 12 or 200 shouldn't matter, should it? As long as the (200-4) bytes are 0'ed, only the first 4 will be used and it should be ok, otherwise EINVAL. No need to know how big the current current actually is because it wouldn't be validating that here: just that it can safely use the first 4 bytes.
Quoted 25 lines
Quoted 24 lines
Quoted 24 lines
Signed-off-by Julien Gomes <[email protected]>--- net/sctp/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sctp/socket.c +1
-1@@ -2311,7 +2311,7 @@ static int sctp_setsockopt_events(struct sock *sk, char __user *optval,int i;if (optlen >sizeof (struct sctp_event_subscribe )) -return -EINVAL ; + optlen =sizeof (struct sctp_event_subscribe );if (copy_from_user (&subscribe, optval, optlen))return -EFAULT ;Signature · 1 line
2.20.1
Quoted 34 lines · Marcelo Ricardo Leitner
Quoted 28 lines · Julien Gomes
Quoted 19 lines · Marcelo Ricardo Leitner
Quoted 7 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
My issue with this change is that by doing this, application will have no clue if the new bits were ignored or not and it may think that an event is enabled while it is not.
A workaround would be to do a getsockopt and check the size that was returned. But then, it might as well use the right struct here in the first place.
Quoted 5 lines
I'm seeing current implementation as an implicitly versioned argument:
it will always accept setsockopt calls with an old struct (v4.11 or v4.12), but if the user tries to use v3 on a v1-only system, it will be rejected. Pretty much like using a newer setsockopt on an old system.Quoted 4 lines
With the current implementation, given sources that say are supposed to run on a 4.9 kernel (no use of any newer field added in 4.11 or 4.12), we can't rebuild the exact same sources on a 4.19 kernel and still run them on 4.9 without messing with structures re-definition.
Quoted 8 lines
Maybe what we want(ed) here then is explicit versioning, to have the 3 definitions available. Then the application is able to use, say struct
sctp_event_subscribe, and be happy with it, while there is structsctp_event_subscribe_v2and structsctp_event_subscribe_v3there too.But it's too late for that now because that would break applications already using the new fields in
sctp_event_subscribe.
Yeah, I'm not supportive of codifying that knoweldge in the kernel. If we were to support bi-directional versioning, I would encode it into lksctp-tools rather than the kernel.
Quoted 9 lines
Quoted 3 lines
I understand your point, but this still looks like a sort of uapi breakage to me.
Not disagreeing. I really just don't know how supported that is. Willing to know so I can pay more attention to this on future changes.
Btw, is this the only occurrence?
No, I think there are a few others (maybe paddrparams?)
Quoted 10 lines
Quoted 9 lines
I also had another way to work-around this in mind, by copying optlen bytes and checking that any additional field (not included in the "current" kernel structure definition) is not set, returning EINVAL in such case to keep a similar to current behavior.
The issue with this is that I didn't find a suitable (ie not totally arbitrary such as "twice the existing structure size") upper limit to optlen.
Quoted 2 lines
Seems interesting. Why would it need that upper limit to optlen?
I think the thought was to differentiate between someone passing a legit larger structure from a newer UAPI, from someone just passing in a massive inappropriately sized buffer (even if the return on both is the same).
Quoted 10 lines
Say struct v1 had 4 bytes, v3 now had 12. The user supplies 12 bytes to the kernel that only knows about 4 bytes. It can check that (12-4) bytes in the end, make sure no bit is on and use only the first 4.
The fact that it was 12 or 200 shouldn't matter, should it? As long as the (200-4) bytes are 0'ed, only the first 4 will be used and it should be ok, otherwise EINVAL. No need to know how big the current current actually is because it wouldn't be validating that here: just that it can safely use the first 4 bytes.
I'm less than excited about making the kernel check an unbounded user space buffer, thats seems like a potential DOS attack from an unpriviledged user to me. I'm also still hung up on the notion that, despite how we do this, this patch is going into the latest kernel, so it will only work on a kernel that already understands the most recent set of subscriptions. It would work if we, again someday in the future extended this struct, someone built against that newer UAPI, and then tried to run it on a kernel that had this patch.
FWIW, there is an existing implied method to determine the available subscription events. sctp_getsockopt_events does clamp the size of the output buffer, and returns that information in the optlen field via put_user. An application that was build against UAPIs from 4.19 could pass in the 4.19 sctp_event_subscribe struct to sctp_getsockopt_events, and read the output length, whcih would inform the application of the events that the kernel is capable of reporting, and limit itself to only using those events. Its not a perfect solution, but its direct, understandable and portable.
Neil
Quoted 26 lines
Quoted 25 lines
Quoted 24 lines
Quoted 24 lines
Signed-off-by Julien Gomes <[email protected]>--- net/sctp/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sctp/socket.c +1
-1@@ -2311,7 +2311,7 @@ static int sctp_setsockopt_events(struct sock *sk, char __user *optval,int i;if (optlen >sizeof (struct sctp_event_subscribe )) -return -EINVAL ; + optlen =sizeof (struct sctp_event_subscribe );if (copy_from_user (&subscribe, optval, optlen))return -EFAULT ;Signature · 1 line
2.20.1
Quoted 40 lines · Neil Horman
Quoted 39 lines · Marcelo Ricardo Leitner
Quoted 28 lines · Julien Gomes
Quoted 19 lines · Marcelo Ricardo Leitner
Quoted 7 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
My issue with this change is that by doing this, application will have no clue if the new bits were ignored or not and it may think that an event is enabled while it is not.
A workaround would be to do a getsockopt and check the size that was returned. But then, it might as well use the right struct here in the first place.
Quoted 5 lines
I'm seeing current implementation as an implicitly versioned argument:
it will always accept setsockopt calls with an old struct (v4.11 or v4.12), but if the user tries to use v3 on a v1-only system, it will be rejected. Pretty much like using a newer setsockopt on an old system.Quoted 4 lines
With the current implementation, given sources that say are supposed to run on a 4.9 kernel (no use of any newer field added in 4.11 or 4.12), we can't rebuild the exact same sources on a 4.19 kernel and still run them on 4.9 without messing with structures re-definition.
Maybe what we want(ed) here then is explicit versioning, to have the 3 definitions available. Then the application is able to use, say struct
sctp_event_subscribe, and be happy with it, while there is structsctp_event_subscribe_v2and structsctp_event_subscribe_v3there too.
Quoted 6 lines
Quoted 3 lines
But it's too late for that now because that would break applications already using the new fields in
sctp_event_subscribe.Yeah, I'm not supportive of codifying that knoweldge in the kernel. If we were to support bi-directional versioning, I would encode it into lksctp-tools rather than the kernel.
I'm not sure that forcing a library on users is a good reason to break UAPI.
Quoted 38 lines
Quoted 10 lines
Quoted 4 lines
I understand your point, but this still looks like a sort of uapi breakage to me.
Not disagreeing. I really just don't know how supported that is. Willing to know so I can pay more attention to this on future changes.
Btw, is this the only occurrence?
No, I think there are a few others (maybe paddrparams?)
Quoted 10 lines
Quoted 9 lines
I also had another way to work-around this in mind, by copying optlen bytes and checking that any additional field (not included in the "current" kernel structure definition) is not set, returning EINVAL in such case to keep a similar to current behavior.
The issue with this is that I didn't find a suitable (ie not totally arbitrary such as "twice the existing structure size") upper limit to optlen.Quoted 2 lines
Seems interesting. Why would it need that upper limit to optlen?
I think the thought was to differentiate between someone passing a legit larger structure from a newer UAPI, from someone just passing in a massive inappropriately sized buffer (even if the return on both is the same).
Quoted 10 lines
Say struct v1 had 4 bytes, v3 now had 12. The user supplies 12 bytes to the kernel that only knows about 4 bytes. It can check that (12-4) bytes in the end, make sure no bit is on and use only the first 4.
The fact that it was 12 or 200 shouldn't matter, should it? As long as the (200-4) bytes are 0'ed, only the first 4 will be used and it should be ok, otherwise EINVAL. No need to know how big the current current actually is because it wouldn't be validating that here: just that it can safely use the first 4 bytes.
Quoted 7 lines
I'm less than excited about making the kernel check an unbounded user space buffer, thats seems like a potential DOS attack from an unpriviledged user to me. I'm also still hung up on the notion that, despite how we do this, this patch is going into the latest kernel, so it will only work on a kernel that already understands the most recent set of subscriptions. It would work if we, again someday in the future extended this struct, someone built against that newer UAPI, and then tried to run it on a kernel that had this patch.
The patch is going into the latest, but can also be backported on future stables.
I don't think "not fixing it because it's not fixed yet" is a good reason to keep things the way they are. But maybe that's just me. Given that the structure has already been extended several times, there is pretty much nothing to keep this from happening again and again.
Quoted 38 lines
FWIW, there is an existing implied method to determine the available subscription events.
sctp_getsockopt_eventsdoes clamp the size of the output buffer, and returns that information in the optlen field viaput_user. An application that was build against UAPIs from 4.19 could pass in the 4.19sctp_event_subscribestruct tosctp_getsockopt_events, and read the output length, whcih would inform the application of the events that the kernel is capable of reporting, and limit itself to only using those events. Its not a perfect solution, but its direct, understandable and portable.Neil
Quoted 26 lines
Quoted 25 lines
Quoted 24 lines
Quoted 24 lines
Signed-off-by Julien Gomes <[email protected]>--- net/sctp/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sctp/socket.c +1
-1@@ -2311,7 +2311,7 @@ static int sctp_setsockopt_events(struct sock *sk, char __user *optval,int i;if (optlen >sizeof (struct sctp_event_subscribe )) -return -EINVAL ; + optlen =sizeof (struct sctp_event_subscribe );if (copy_from_user (&subscribe, optval, optlen))return -EFAULT ;Signature · 1 line
2.20.1
Signature · 1 line
Julien Gomes
Quoted 43 lines · Julien Gomes
Quoted 40 lines · Neil Horman
Quoted 39 lines · Marcelo Ricardo Leitner
Quoted 28 lines · Julien Gomes
Quoted 19 lines · Marcelo Ricardo Leitner
Quoted 7 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
My issue with this change is that by doing this, application will have no clue if the new bits were ignored or not and it may think that an event is enabled while it is not.
A workaround would be to do a getsockopt and check the size that was returned. But then, it might as well use the right struct here in the first place.
Quoted 5 lines
I'm seeing current implementation as an implicitly versioned argument:
it will always accept setsockopt calls with an old struct (v4.11 or v4.12), but if the user tries to use v3 on a v1-only system, it will be rejected. Pretty much like using a newer setsockopt on an old system.Quoted 4 lines
With the current implementation, given sources that say are supposed to run on a 4.9 kernel (no use of any newer field added in 4.11 or 4.12), we can't rebuild the exact same sources on a 4.19 kernel and still run them on 4.9 without messing with structures re-definition.
Maybe what we want(ed) here then is explicit versioning, to have the 3 definitions available. Then the application is able to use, say struct
sctp_event_subscribe, and be happy with it, while there is structsctp_event_subscribe_v2and structsctp_event_subscribe_v3there too.
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
But it's too late for that now because that would break applications already using the new fields in
sctp_event_subscribe.Yeah, I'm not supportive of codifying that knoweldge in the kernel. If we were to support bi-directional versioning, I would encode it into lksctp-tools rather than the kernel.
I'm not sure that forcing a library on users is a good reason to break UAPI.
Thats a misleading statement. We've never supported running newer applications on older kernels, and no one is forcing anyone to use the lksctp-tools library, I was only suggesting that, if we were to support this compatibility, that might be a place to offer it.
Its also worth noting that we have precident for this. If you look at the git log, this particular structure has been extended about 6 times in the life of sctp.
Quoted 46 lines
Quoted 38 lines
Quoted 10 lines
Quoted 4 lines
I understand your point, but this still looks like a sort of uapi breakage to me.
Not disagreeing. I really just don't know how supported that is. Willing to know so I can pay more attention to this on future changes.
Btw, is this the only occurrence?
No, I think there are a few others (maybe paddrparams?)
Quoted 10 lines
Quoted 9 lines
I also had another way to work-around this in mind, by copying optlen bytes and checking that any additional field (not included in the "current" kernel structure definition) is not set, returning EINVAL in such case to keep a similar to current behavior.
The issue with this is that I didn't find a suitable (ie not totally arbitrary such as "twice the existing structure size") upper limit to optlen.Quoted 2 lines
Seems interesting. Why would it need that upper limit to optlen?
I think the thought was to differentiate between someone passing a legit larger structure from a newer UAPI, from someone just passing in a massive inappropriately sized buffer (even if the return on both is the same).
Quoted 10 lines
Say struct v1 had 4 bytes, v3 now had 12. The user supplies 12 bytes to the kernel that only knows about 4 bytes. It can check that (12-4) bytes in the end, make sure no bit is on and use only the first 4.
The fact that it was 12 or 200 shouldn't matter, should it? As long as the (200-4) bytes are 0'ed, only the first 4 will be used and it should be ok, otherwise EINVAL. No need to know how big the current current actually is because it wouldn't be validating that here: just that it can safely use the first 4 bytes.
Quoted 7 lines
I'm less than excited about making the kernel check an unbounded user space buffer, thats seems like a potential DOS attack from an unpriviledged user to me. I'm also still hung up on the notion that, despite how we do this, this patch is going into the latest kernel, so it will only work on a kernel that already understands the most recent set of subscriptions. It would work if we, again someday in the future extended this struct, someone built against that newer UAPI, and then tried to run it on a kernel that had this patch.
Quoted 7 lines
The patch is going into the latest, but can also be backported on future stables.
I don't think "not fixing it because it's not fixed yet" is a good reason to keep things the way they are. But maybe that's just me. Given that the structure has already been extended several times, there is pretty much nothing to keep this from happening again and again.
Also misleading, as it assumes that we're not intentionally doing this. I get wanting to support running applications built for newer kernels on older kernels, but thats just not something that we do, and to say thats broken is misleading. Older applications are required to run on newer kernels, but not vice versa, which is what you are asking for.
And yes, this patch can be backported to older stable kernels, but by that same token, so can the patches that extend the struct, which would also fix the problem, while supporting the newer features, which seems to me to be the better solution for applications which are looking for that support.
Quoted 42 lines
Quoted 38 lines
FWIW, there is an existing implied method to determine the available subscription events.
sctp_getsockopt_eventsdoes clamp the size of the output buffer, and returns that information in the optlen field viaput_user. An application that was build against UAPIs from 4.19 could pass in the 4.19sctp_event_subscribestruct tosctp_getsockopt_events, and read the output length, whcih would inform the application of the events that the kernel is capable of reporting, and limit itself to only using those events. Its not a perfect solution, but its direct, understandable and portable.Neil
Quoted 26 lines
Quoted 25 lines
Quoted 24 lines
Quoted 24 lines
Signed-off-by Julien Gomes <[email protected]>--- net/sctp/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sctp/socket.c +1
-1@@ -2311,7 +2311,7 @@ static int sctp_setsockopt_events(struct sock *sk, char __user *optval,int i;if (optlen >sizeof (struct sctp_event_subscribe )) -return -EINVAL ; + optlen =sizeof (struct sctp_event_subscribe );if (copy_from_user (&subscribe, optval, optlen))return -EFAULT ;Signature · 1 line
2.20.1Signature · 1 line
Julien Gomes
Quoted 34 lines · Marcelo Ricardo Leitner
Quoted 28 lines · Julien Gomes
Quoted 19 lines · Marcelo Ricardo Leitner
Quoted 7 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
My issue with this change is that by doing this, application will have no clue if the new bits were ignored or not and it may think that an event is enabled while it is not.
A workaround would be to do a getsockopt and check the size that was returned. But then, it might as well use the right struct here in the first place.
Quoted 5 lines
I'm seeing current implementation as an implicitly versioned argument:
it will always accept setsockopt calls with an old struct (v4.11 or v4.12), but if the user tries to use v3 on a v1-only system, it will be rejected. Pretty much like using a newer setsockopt on an old system.Quoted 4 lines
With the current implementation, given sources that say are supposed to run on a 4.9 kernel (no use of any newer field added in 4.11 or 4.12), we can't rebuild the exact same sources on a 4.19 kernel and still run them on 4.9 without messing with structures re-definition.
Quoted 7 lines
Maybe what we want(ed) here then is explicit versioning, to have the 3 definitions available. Then the application is able to use, say struct
sctp_event_subscribe, and be happy with it, while there is structsctp_event_subscribe_v2and structsctp_event_subscribe_v3there too.But it's too late for that now because that would break applications already using the new fields in
sctp_event_subscribe.
Right.
Quoted 9 lines
Quoted 4 lines
I understand your point, but this still looks like a sort of uapi breakage to me.
Not disagreeing. I really just don't know how supported that is. Willing to know so I can pay more attention to this on future changes.
Btw, is this the only occurrence?
Can't really say, this is one I witnessed, I haven't really looked for others.
Quoted 17 lines
Quoted 10 lines
I also had another way to work-around this in mind, by copying optlen bytes and checking that any additional field (not included in the "current" kernel structure definition) is not set, returning EINVAL in such case to keep a similar to current behavior.
The issue with this is that I didn't find a suitable (ie not totally arbitrary such as "twice the existing structure size") upper limit to optlen.Seems interesting. Why would it need that upper limit to optlen?
Say struct v1 had 4 bytes, v3 now had 12. The user supplies 12 bytes to the kernel that only knows about 4 bytes. It can check that (12-4) bytes in the end, make sure no bit is on and use only the first 4.
Quoted 5 lines
The fact that it was 12 or 200 shouldn't matter, should it? As long as the (200-4) bytes are 0'ed, only the first 4 will be used and it should be ok, otherwise EINVAL. No need to know how big the current current actually is because it wouldn't be validating that here: just that it can safely use the first 4 bytes.
The upper limit concern is more regarding the call to copy_from_user with an unrestricted/unchecked value.
I am not sure of how much of a risk/how exploitable this could be, that's why I cautiously wanted to limit it in the first place just in case.
Quoted 26 lines
Quoted 26 lines
Quoted 25 lines
Quoted 25 lines
Signed-off-by Julien Gomes <[email protected]>--- net/sctp/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sctp/socket.c +1
-1@@ -2311,7 +2311,7 @@ static int sctp_setsockopt_events(struct sock *sk, char __user *optval,int i;if (optlen >sizeof (struct sctp_event_subscribe )) -return -EINVAL ; + optlen =sizeof (struct sctp_event_subscribe );if (copy_from_user (&subscribe, optval, optlen))return -EFAULT ;Signature · 1 line
2.20.1
Signature · 1 line
Julien Gomes
Quoted 77 lines · Julien Gomes
Quoted 34 lines · Marcelo Ricardo Leitner
Quoted 28 lines · Julien Gomes
Quoted 19 lines · Marcelo Ricardo Leitner
Quoted 7 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
My issue with this change is that by doing this, application will have no clue if the new bits were ignored or not and it may think that an event is enabled while it is not.
A workaround would be to do a getsockopt and check the size that was returned. But then, it might as well use the right struct here in the first place.
Quoted 5 lines
I'm seeing current implementation as an implicitly versioned argument:
it will always accept setsockopt calls with an old struct (v4.11 or v4.12), but if the user tries to use v3 on a v1-only system, it will be rejected. Pretty much like using a newer setsockopt on an old system.Quoted 4 lines
With the current implementation, given sources that say are supposed to run on a 4.9 kernel (no use of any newer field added in 4.11 or 4.12), we can't rebuild the exact same sources on a 4.19 kernel and still run them on 4.9 without messing with structures re-definition.
Quoted 7 lines
Maybe what we want(ed) here then is explicit versioning, to have the 3 definitions available. Then the application is able to use, say struct
sctp_event_subscribe, and be happy with it, while there is structsctp_event_subscribe_v2and structsctp_event_subscribe_v3there too.But it's too late for that now because that would break applications already using the new fields in
sctp_event_subscribe.Right.
Quoted 9 lines
Quoted 4 lines
I understand your point, but this still looks like a sort of uapi breakage to me.
Not disagreeing. I really just don't know how supported that is. Willing to know so I can pay more attention to this on future changes.
Btw, is this the only occurrence?
Can't really say, this is one I witnessed, I haven't really looked for others.
Quoted 17 lines
Quoted 10 lines
I also had another way to work-around this in mind, by copying optlen bytes and checking that any additional field (not included in the "current" kernel structure definition) is not set, returning EINVAL in such case to keep a similar to current behavior.
The issue with this is that I didn't find a suitable (ie not totally arbitrary such as "twice the existing structure size") upper limit to optlen.Seems interesting. Why would it need that upper limit to optlen?
Say struct v1 had 4 bytes, v3 now had 12. The user supplies 12 bytes to the kernel that only knows about 4 bytes. It can check that (12-4) bytes in the end, make sure no bit is on and use only the first 4.
Quoted 8 lines
Quoted 5 lines
The fact that it was 12 or 200 shouldn't matter, should it? As long as the (200-4) bytes are 0'ed, only the first 4 will be used and it should be ok, otherwise EINVAL. No need to know how big the current current actually is because it wouldn't be validating that here: just that it can safely use the first 4 bytes.
The upper limit concern is more regarding the call to
copy_from_userwith an unrestricted/unchecked value.
Copy_from_user should be safe to copy an arbitrary amount, the only restriction is that optlen can't exceed the size of the buffer receiving the data in the kernel. From that standpoint your patch is safe. However, that exposes the problem of checking any tail data on the userspace buffer. That is to say, if you want to ensure that any extra data that gets sent from userspace isn't 'set', you would have to copy that extra data in consumable chunks and check them individaully, and that screams DOS to me (i.e. imagine a user passing in a 4GB buffer, and having to wait for the kernel to copy each X sized chunk, looking for non-zero values).
Quoted 33 lines
I am not sure of how much of a risk/how exploitable this could be, that's why I cautiously wanted to limit it in the first place just in case.
Quoted 26 lines
Quoted 26 lines
Quoted 25 lines
Quoted 25 lines
Signed-off-by Julien Gomes <[email protected]>--- net/sctp/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sctp/socket.c +1
-1@@ -2311,7 +2311,7 @@ static int sctp_setsockopt_events(struct sock *sk, char __user *optval,int i;if (optlen >sizeof (struct sctp_event_subscribe )) -return -EINVAL ; + optlen =sizeof (struct sctp_event_subscribe );if (copy_from_user (&subscribe, optval, optlen))return -EFAULT ;Signature · 1 line
2.20.1Signature · 1 line
Julien Gomes
Quoted 86 lines · Neil Horman
Quoted 77 lines · Julien Gomes
Quoted 34 lines · Marcelo Ricardo Leitner
Quoted 28 lines · Julien Gomes
Quoted 19 lines · Marcelo Ricardo Leitner
Quoted 7 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
My issue with this change is that by doing this, application will have no clue if the new bits were ignored or not and it may think that an event is enabled while it is not.
A workaround would be to do a getsockopt and check the size that was returned. But then, it might as well use the right struct here in the first place.
Quoted 5 lines
I'm seeing current implementation as an implicitly versioned argument:
it will always accept setsockopt calls with an old struct (v4.11 or v4.12), but if the user tries to use v3 on a v1-only system, it will be rejected. Pretty much like using a newer setsockopt on an old system.Quoted 4 lines
With the current implementation, given sources that say are supposed to run on a 4.9 kernel (no use of any newer field added in 4.11 or 4.12), we can't rebuild the exact same sources on a 4.19 kernel and still run them on 4.9 without messing with structures re-definition.
Quoted 7 lines
Maybe what we want(ed) here then is explicit versioning, to have the 3 definitions available. Then the application is able to use, say struct
sctp_event_subscribe, and be happy with it, while there is structsctp_event_subscribe_v2and structsctp_event_subscribe_v3there too.But it's too late for that now because that would break applications already using the new fields in
sctp_event_subscribe.Right.
Quoted 9 lines
Quoted 4 lines
I understand your point, but this still looks like a sort of uapi breakage to me.
Not disagreeing. I really just don't know how supported that is. Willing to know so I can pay more attention to this on future changes.
Btw, is this the only occurrence?
Can't really say, this is one I witnessed, I haven't really looked for others.
Quoted 17 lines
Quoted 10 lines
I also had another way to work-around this in mind, by copying optlen bytes and checking that any additional field (not included in the "current" kernel structure definition) is not set, returning EINVAL in such case to keep a similar to current behavior.
The issue with this is that I didn't find a suitable (ie not totally arbitrary such as "twice the existing structure size") upper limit to optlen.Seems interesting. Why would it need that upper limit to optlen?
Say struct v1 had 4 bytes, v3 now had 12. The user supplies 12 bytes to the kernel that only knows about 4 bytes. It can check that (12-4) bytes in the end, make sure no bit is on and use only the first 4.
Quoted 8 lines
Quoted 5 lines
The fact that it was 12 or 200 shouldn't matter, should it? As long as the (200-4) bytes are 0'ed, only the first 4 will be used and it should be ok, otherwise EINVAL. No need to know how big the current current actually is because it wouldn't be validating that here: just that it can safely use the first 4 bytes.
The upper limit concern is more regarding the call to
copy_from_userwith an unrestricted/unchecked value.
Quoted 9 lines
Copy_from_usershould be safe to copy an arbitrary amount, the only restriction is that optlen can't exceed the size of the buffer receiving the data in the kernel. From that standpoint your patch is safe. However, that exposes the problem of checking any tail data on the userspace buffer. That is to say, if you want to ensure that any extra data that gets sent from userspace isn't 'set', you would have to copy that extra data in consumable chunks and check them individaully, and that screams DOS to me (i.e. imagine a user passing in a 4GB buffer, and having to wait for the kernel to copy each X sized chunk, looking for non-zero values).
There probably is a decent compromise to find between "not accepting a single additional byte" and accepting several GB.
For example how likely is it that the growth of this structure make it go over a page? I would hope not at all.
By choosing a large but decent high limit, I think we can find a future-compatible compromise that doesn't rely on a preliminary getsockopt() just for structure trucation decision...
Quoted 34 lines
Quoted 34 lines
I am not sure of how much of a risk/how exploitable this could be, that's why I cautiously wanted to limit it in the first place just in case.
Quoted 26 lines
Quoted 26 lines
Quoted 25 lines
Quoted 25 lines
Signed-off-by Julien Gomes <[email protected]>--- net/sctp/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sctp/socket.c +1
-1@@ -2311,7 +2311,7 @@ static int sctp_setsockopt_events(struct sock *sk, char __user *optval,int i;if (optlen >sizeof (struct sctp_event_subscribe )) -return -EINVAL ; + optlen =sizeof (struct sctp_event_subscribe );if (copy_from_user (&subscribe, optval, optlen))return -EFAULT ;Signature · 1 line
2.20.1Signature · 1 line
Julien Gomes
Signature · 1 line
Julien Gomes
Quoted 99 lines · Julien Gomes
Quoted 86 lines · Neil Horman
Quoted 77 lines · Julien Gomes
Quoted 34 lines · Marcelo Ricardo Leitner
Quoted 28 lines · Julien Gomes
Quoted 19 lines · Marcelo Ricardo Leitner
Quoted 7 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
My issue with this change is that by doing this, application will have no clue if the new bits were ignored or not and it may think that an event is enabled while it is not.
A workaround would be to do a getsockopt and check the size that was returned. But then, it might as well use the right struct here in the first place.
Quoted 5 lines
I'm seeing current implementation as an implicitly versioned argument:
it will always accept setsockopt calls with an old struct (v4.11 or v4.12), but if the user tries to use v3 on a v1-only system, it will be rejected. Pretty much like using a newer setsockopt on an old system.Quoted 4 lines
With the current implementation, given sources that say are supposed to run on a 4.9 kernel (no use of any newer field added in 4.11 or 4.12), we can't rebuild the exact same sources on a 4.19 kernel and still run them on 4.9 without messing with structures re-definition.
Quoted 7 lines
Maybe what we want(ed) here then is explicit versioning, to have the 3 definitions available. Then the application is able to use, say struct
sctp_event_subscribe, and be happy with it, while there is structsctp_event_subscribe_v2and structsctp_event_subscribe_v3there too.But it's too late for that now because that would break applications already using the new fields in
sctp_event_subscribe.Right.
Quoted 9 lines
Quoted 4 lines
I understand your point, but this still looks like a sort of uapi breakage to me.
Not disagreeing. I really just don't know how supported that is. Willing to know so I can pay more attention to this on future changes.
Btw, is this the only occurrence?
Can't really say, this is one I witnessed, I haven't really looked for others.
Quoted 17 lines
Quoted 10 lines
I also had another way to work-around this in mind, by copying optlen bytes and checking that any additional field (not included in the "current" kernel structure definition) is not set, returning EINVAL in such case to keep a similar to current behavior.
The issue with this is that I didn't find a suitable (ie not totally arbitrary such as "twice the existing structure size") upper limit to optlen.Seems interesting. Why would it need that upper limit to optlen?
Say struct v1 had 4 bytes, v3 now had 12. The user supplies 12 bytes to the kernel that only knows about 4 bytes. It can check that (12-4) bytes in the end, make sure no bit is on and use only the first 4.
Quoted 8 lines
Quoted 5 lines
The fact that it was 12 or 200 shouldn't matter, should it? As long as the (200-4) bytes are 0'ed, only the first 4 will be used and it should be ok, otherwise EINVAL. No need to know how big the current current actually is because it wouldn't be validating that here: just that it can safely use the first 4 bytes.
The upper limit concern is more regarding the call to
copy_from_userwith an unrestricted/unchecked value.Quoted 9 lines
Copy_from_usershould be safe to copy an arbitrary amount, the only restriction is that optlen can't exceed the size of the buffer receiving the data in the kernel. From that standpoint your patch is safe. However, that exposes the problem of checking any tail data on the userspace buffer. That is to say, if you want to ensure that any extra data that gets sent from userspace isn't 'set', you would have to copy that extra data in consumable chunks and check them individaully, and that screams DOS to me (i.e. imagine a user passing in a 4GB buffer, and having to wait for the kernel to copy each X sized chunk, looking for non-zero values).
Quoted 8 lines
There probably is a decent compromise to find between "not accepting a single additional byte" and accepting several GB.
For example how likely is it that the growth of this structure make it go over a page? I would hope not at all.By choosing a large but decent high limit, I think we can find a future-compatible compromise that doesn't rely on a preliminary
getsockopt()just for structure trucation decision...
And I was just reminded about huge pages.
But still, my point of finding a compromise still stands.
Quoted 36 lines
Quoted 35 lines
Quoted 35 lines
I am not sure of how much of a risk/how exploitable this could be, that's why I cautiously wanted to limit it in the first place just in case.
Quoted 26 lines
Quoted 26 lines
Quoted 25 lines
Quoted 25 lines
Signed-off-by Julien Gomes <[email protected]>--- net/sctp/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sctp/socket.c +1
-1@@ -2311,7 +2311,7 @@ static int sctp_setsockopt_events(struct sock *sk, char __user *optval,int i;if (optlen >sizeof (struct sctp_event_subscribe )) -return -EINVAL ; + optlen =sizeof (struct sctp_event_subscribe );if (copy_from_user (&subscribe, optval, optlen))return -EFAULT ;Signature · 1 line
2.20.1Signature · 1 line
Julien Gomes
Quoted 99 lines · Julien Gomes
Quoted 86 lines · Neil Horman
Quoted 77 lines · Julien Gomes
Quoted 34 lines · Marcelo Ricardo Leitner
Quoted 28 lines · Julien Gomes
Quoted 19 lines · Marcelo Ricardo Leitner
Quoted 7 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
My issue with this change is that by doing this, application will have no clue if the new bits were ignored or not and it may think that an event is enabled while it is not.
A workaround would be to do a getsockopt and check the size that was returned. But then, it might as well use the right struct here in the first place.
Quoted 5 lines
I'm seeing current implementation as an implicitly versioned argument:
it will always accept setsockopt calls with an old struct (v4.11 or v4.12), but if the user tries to use v3 on a v1-only system, it will be rejected. Pretty much like using a newer setsockopt on an old system.Quoted 4 lines
With the current implementation, given sources that say are supposed to run on a 4.9 kernel (no use of any newer field added in 4.11 or 4.12), we can't rebuild the exact same sources on a 4.19 kernel and still run them on 4.9 without messing with structures re-definition.
Quoted 7 lines
Maybe what we want(ed) here then is explicit versioning, to have the 3 definitions available. Then the application is able to use, say struct
sctp_event_subscribe, and be happy with it, while there is structsctp_event_subscribe_v2and structsctp_event_subscribe_v3there too.But it's too late for that now because that would break applications already using the new fields in
sctp_event_subscribe.Right.
Quoted 9 lines
Quoted 4 lines
I understand your point, but this still looks like a sort of uapi breakage to me.
Not disagreeing. I really just don't know how supported that is. Willing to know so I can pay more attention to this on future changes.
Btw, is this the only occurrence?
Can't really say, this is one I witnessed, I haven't really looked for others.
Quoted 17 lines
Quoted 10 lines
I also had another way to work-around this in mind, by copying optlen bytes and checking that any additional field (not included in the "current" kernel structure definition) is not set, returning EINVAL in such case to keep a similar to current behavior.
The issue with this is that I didn't find a suitable (ie not totally arbitrary such as "twice the existing structure size") upper limit to optlen.Seems interesting. Why would it need that upper limit to optlen?
Say struct v1 had 4 bytes, v3 now had 12. The user supplies 12 bytes to the kernel that only knows about 4 bytes. It can check that (12-4) bytes in the end, make sure no bit is on and use only the first 4.
Quoted 8 lines
Quoted 5 lines
The fact that it was 12 or 200 shouldn't matter, should it? As long as the (200-4) bytes are 0'ed, only the first 4 will be used and it should be ok, otherwise EINVAL. No need to know how big the current current actually is because it wouldn't be validating that here: just that it can safely use the first 4 bytes.
The upper limit concern is more regarding the call to
copy_from_userwith an unrestricted/unchecked value.Quoted 9 lines
Copy_from_usershould be safe to copy an arbitrary amount, the only restriction is that optlen can't exceed the size of the buffer receiving the data in the kernel. From that standpoint your patch is safe. However, that exposes the problem of checking any tail data on the userspace buffer. That is to say, if you want to ensure that any extra data that gets sent from userspace isn't 'set', you would have to copy that extra data in consumable chunks and check them individaully, and that screams DOS to me (i.e. imagine a user passing in a 4GB buffer, and having to wait for the kernel to copy each X sized chunk, looking for non-zero values).
Quoted 5 lines
There probably is a decent compromise to find between "not accepting a single additional byte" and accepting several GB.
For example how likely is it that the growth of this structure make it go over a page? I would hope not at all.
Probably not, but I'm not going to pick a magic number to gate whats ok and whats not for sockopt validation.
Quoted 4 lines
By choosing a large but decent high limit, I think we can find a future-compatible compromise that doesn't rely on a preliminary
getsockopt()just for structure trucation decision...
We really don't have to, I refer you to my previous not referencing the fact that the getsockopt variant of this call will return the expected length of this option for the running kernel, allowing userspace to know explicitly what the buffer size should be, and by extension, what options are supported
Neil
Quoted 38 lines
Quoted 34 lines
Quoted 34 lines
I am not sure of how much of a risk/how exploitable this could be, that's why I cautiously wanted to limit it in the first place just in case.
Quoted 26 lines
Quoted 26 lines
Quoted 25 lines
Quoted 25 lines
Signed-off-by Julien Gomes <[email protected]>--- net/sctp/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sctp/socket.c +1
-1@@ -2311,7 +2311,7 @@ static int sctp_setsockopt_events(struct sock *sk, char __user *optval,int i;if (optlen >sizeof (struct sctp_event_subscribe )) -return -EINVAL ; + optlen =sizeof (struct sctp_event_subscribe );if (copy_from_user (&subscribe, optval, optlen))return -EFAULT ;Signature · 1 line
2.20.1Signature · 1 line
Julien GomesSignature · 1 line
Julien Gomes
From: Marcelo Ricardo Leitner
Quoted 36 lines
Sent: 06 February 2019 21:07
Quoted 28 lines · Julien Gomes
Quoted 19 lines · Marcelo Ricardo Leitner
Quoted 7 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
My issue with this change is that by doing this, application will have no clue if the new bits were ignored or not and it may think that an event is enabled while it is not.
A workaround would be to do a getsockopt and check the size that was returned. But then, it might as well use the right struct here in the first place.
Quoted 5 lines
I'm seeing current implementation as an implicitly versioned argument:
it will always accept setsockopt calls with an old struct (v4.11 or v4.12), but if the user tries to use v3 on a v1-only system, it will be rejected. Pretty much like using a newer setsockopt on an old system.Quoted 4 lines
With the current implementation, given sources that say are supposed to run on a 4.9 kernel (no use of any newer field added in 4.11 or 4.12), we can't rebuild the exact same sources on a 4.19 kernel and still run them on 4.9 without messing with structures re-definition.
Quoted 7 lines
Maybe what we want(ed) here then is explicit versioning, to have the 3 definitions available. Then the application is able to use, say struct
sctp_event_subscribe, and be happy with it, while there is structsctp_event_subscribe_v2and structsctp_event_subscribe_v3there too.But it's too late for that now because that would break applications already using the new fields in
sctp_event_subscribe.
It is probably better to break the recompilation of the few programs that use the new fields (and have them not work on old kernels) than to stop recompilations of old programs stop working on old kernels or have requested new options silently ignored.
There are all sorts of reasons why programs get built on systems that are newer than the ones they need to run on.
I'm currently planning to get around the glibc 'memcpy()' fubar so I can retire some very old build systems before their disks die.
Fortunately our sctp code is in the kernel - so has to be compiled with the correct headers.
Quoted 5 lines
Quoted 2 lines
I understand your point, but this still looks like a sort of uapi breakage to me.
Not disagreeing. I really just don't know how supported that is. Willing to know so I can pay more attention to this on future changes.
Agreed, these structures should never be changed.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
'Marcelo Ricardo Leitner'
maintainerQuoted 42 lines · David Laight
From: Marcelo Ricardo Leitner
Quoted 41 lines
Sent: 06 February 2019 21:07
Quoted 28 lines · Julien Gomes
Quoted 19 lines · Marcelo Ricardo Leitner
Quoted 7 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
My issue with this change is that by doing this, application will have no clue if the new bits were ignored or not and it may think that an event is enabled while it is not.
A workaround would be to do a getsockopt and check the size that was returned. But then, it might as well use the right struct here in the first place.
Quoted 5 lines
I'm seeing current implementation as an implicitly versioned argument:
it will always accept setsockopt calls with an old struct (v4.11 or v4.12), but if the user tries to use v3 on a v1-only system, it will be rejected. Pretty much like using a newer setsockopt on an old system.Quoted 4 lines
With the current implementation, given sources that say are supposed to run on a 4.9 kernel (no use of any newer field added in 4.11 or 4.12), we can't rebuild the exact same sources on a 4.19 kernel and still run them on 4.9 without messing with structures re-definition.
Maybe what we want(ed) here then is explicit versioning, to have the 3 definitions available. Then the application is able to use, say struct
sctp_event_subscribe, and be happy with it, while there is structsctp_event_subscribe_v2and structsctp_event_subscribe_v3there too.
Quoted 7 lines
Quoted 2 lines
But it's too late for that now because that would break applications already using the new fields in
sctp_event_subscribe.It is probably better to break the recompilation of the few programs that use the new fields (and have them not work on old kernels) than to stop recompilations of old programs stop working on old kernels or have requested new options silently ignored.
I got confused here, not sure what you mean. Seems there is one "stop" word too many.
Quoted 5 lines
There are all sorts of reasons why programs get built on systems that are newer than the ones they need to run on.
I'm currently planning to get around the glibc 'memcpy()' fubar so I can retire some very old build systems before their disks die.
You can virtualize those. That's not really a good reason for building with newer kernel and running on old systems, as virtually any old system can be virtualized.
Marcelo
Quoted 18 lines
Fortunately our sctp code is in the kernel - so has to be compiled with the correct headers.
Quoted 5 lines
Quoted 2 lines
I understand your point, but this still looks like a sort of uapi breakage to me.
Not disagreeing. I really just don't know how supported that is. Willing to know so I can pay more attention to this on future changes.
Agreed, these structures should never be changed.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
From: 'Marcelo Ricardo Leitner'
Quoted 1 line
Sent: 07 February 2019 17:47
...
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Maybe what we want(ed) here then is explicit versioning, to have the 3 definitions available. Then the application is able to use, say struct
sctp_event_subscribe, and be happy with it, while there is structsctp_event_subscribe_v2and structsctp_event_subscribe_v3there too.But it's too late for that now because that would break applications already using the new fields in
sctp_event_subscribe.
Quoted 7 lines
Quoted 4 lines
It is probably better to break the recompilation of the few programs that use the new fields (and have them not work on old kernels) than to stop recompilations of old programs stop working on old kernels or have requested new options silently ignored.
I got confused here, not sure what you mean. Seems there is one "stop" word too many.
More confusing than I intended...
With the current kernel and headers a 'new program' (one that needs the new options) will fail to run on an old kernel - which is good. However a recompilation of an 'old program' (that doesn't use the new options) will also fail to run on an old kernel - which is bad.
Changing the kernel to ignore extra events flags breaks the 'new' program.
Versioning the structure now (even though it should have been done earlier) won't change the behaviour of existing binaries.
However a recompilation of an 'old' program would use the 'old' structure and work on old kernels.
Attempts to recompile a 'new' program will fail - until the structure name (or some #define to enable the extra fields) is changed.
Breaking compilations is much better than unexpected run-time behaviour.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Quoted 19 lines · David Laight
From: 'Marcelo Ricardo Leitner'
Quoted 1 line
Sent: 07 February 2019 17:47
...
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Maybe what we want(ed) here then is explicit versioning, to have the 3 definitions available. Then the application is able to use, say struct
sctp_event_subscribe, and be happy with it, while there is structsctp_event_subscribe_v2and structsctp_event_subscribe_v3there too.But it's too late for that now because that would break applications already using the new fields in
sctp_event_subscribe.Quoted 7 lines
Quoted 4 lines
It is probably better to break the recompilation of the few programs that use the new fields (and have them not work on old kernels) than to stop recompilations of old programs stop working on old kernels or have requested new options silently ignored.
I got confused here, not sure what you mean. Seems there is one "stop" word too many.
Quoted 7 lines
More confusing than I intended...
With the current kernel and headers a 'new program' (one that needs the new options) will fail to run on an old kernel - which is good. However a recompilation of an 'old program' (that doesn't use the new options) will also fail to run on an old kernel - which is bad.
I disagree with this, at least as a unilateral statement. I would assert that an old program, within the constraints of the issue being discussed here, will run perfectly well, when built and run against a new kernel.
At issue is the size of the structure sctp_event_subscribe, and the fact that in several instances over the last few years, its been extended to be larger and encompass more events to subscribe to.
Nominally an application will use this structure (roughly) as follows:
...
struct sctp_event_subscribe events;size_t evsize = sizeof(events);
memset(&events, 0, sizeof(events));
events.sctp_send_failure_event = 1; /*example event subscription*/
if (sctp_setsocktpt(sctp_fd, SOL_SCTP, SCTP_EVENTS, &events, &evsize) < 0) {
/* do error recovery */
}....
Assume this code will be built and run against kernel versions A and B, in which:
A) has a struct sctp_event_subscribe with a size of 9 bytes
B) has a struct sctp_event_subscribe with a size of 10 bytes (due to the added field sctp_sender_dry_event)
That gives us 4 cases to handle
1) Application build against kernel A and run on kernel A. This works fine, the sizes of the struct in question will always match
2) Application is built against kernel A and run on kernel B. In this case, everything will work because the application passes a buffer of size 9, and the kernel accepts it, because it allows for buffers to be shorter than the current struct sctp_event_subscribe size. The kernel simply operates on the options available in the buffer. The application is none the wiser, because it has no knoweldge of the new option, nor should it because it was built against kernel A, that never offered that option
3) Application is built against kernel B and run on kernel B. This works fine for the same reason as (1).
4) Application is built against kernel B and run on kernel A. This will break because the application is passing a buffer that is larger than what the kernel
expects, and rightly so. The application is passing in a buffer that is
incompatible with what the running kernel expects.
We could look into ways in which to detect the cases in which this might be 'ok', but I don't see why we should bother, because at some point its still an error to pass in an incompatible buffer. In my mind this is no different than trying to run a program that allocates hugepages on a kernel that doesn't support hugepages (just to make up an example). Applications built against newer kernel can't expect all the features/semantics/etc to be identical to older kernels.
Quoted 3 lines
Changing the kernel to ignore extra events flags breaks the 'new' program.
It shouldn't. Assuming you have a program built against headers from kernel B (above), if you set a field in the structure that only exists in kernel B, and try to run it on kernel A, you will get an EINVAL return, which is correct behavior because you are attempting to deliver information to the kernel that kernel A (the running kernel) doesn't know about. Thats correct behavior.
Quoted 3 lines
Versioning the structure now (even though it should have been done earlier) won't change the behaviour of existing binaries.
I won't disagree about the niceness of versioning, but that ship has sailed.
Quoted 2 lines
However a recompilation of an 'old' program would use the 'old' structure and work on old kernels.
To be clear, this is situation (1) above, and yeah, running on the kernel you built your application against should always work from a compatibility standpoint.
Quoted 3 lines
Attempts to recompile a 'new' program will fail - until the structure name (or some #define to enable the extra fields) is changed.
Yes, but this is alawys the case for structures that change. If you have an application built against kernel (B), and uses structure fields that only exist in that version of the kernel (and not earlier) will fail to compile when built against kernel (A) headers, and thats expected. This happens with any kernel api that exists in a newer kernel but not an older kernel.
Quoted 3 lines
Breaking compilations is much better than unexpected run-time behaviour.
Any time you make a system call to the kernel, you have to be prepared to handle the resulting error condition, thats not unexpected. To assume that a system call will always work is bad programming practice.
Neil
Quoted 7 lines
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Quoted 22 lines · Julien Gomes
Quoted 19 lines · Marcelo Ricardo Leitner
Quoted 7 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
My issue with this change is that by doing this, application will have no clue if the new bits were ignored or not and it may think that an event is enabled while it is not.
A workaround would be to do a getsockopt and check the size that was returned. But then, it might as well use the right struct here in the first place.
Quoted 8 lines
Quoted 5 lines
I'm seeing current implementation as an implicitly versioned argument:
it will always accept setsockopt calls with an old struct (v4.11 or v4.12), but if the user tries to use v3 on a v1-only system, it will be rejected. Pretty much like using a newer setsockopt on an old system.With the current implementation, given sources that say are supposed to run on a 4.9 kernel (no use of any newer field added in 4.11 or 4.12),
What given sources say that? I understand it might be expected, but this is an common concern with setsockopt method on many protocols, it just so happens that sctp extends them more than other protocols.
Quoted 3 lines
we can't rebuild the exact same sources on a 4.19 kernel and still run them on 4.9 without messing with structures re-definition.
Right, put another way, we support backward compatibility with older userspace applications, but not newer one. I.e. if you build an application against the 4.9 SCTP API, it should work with the 4.19 UAPI, but not vice versa, which it seems is like what you are trying to do here.
Quoted 4 lines
I understand your point, but this still looks like a sort of uapi breakage to me.
Quoted 8 lines
I also had another way to work-around this in mind, by copying optlen bytes and checking that any additional field (not included in the "current" kernel structure definition) is not set, returning EINVAL in such case to keep a similar to current behavior.
The issue with this is that I didn't find a suitable (ie not totally arbitrary such as "twice the existing structure size") upper limit to optlen.
There is no real uppper limit to the size of the structure in this case, and IIRC this isn't the only sockopt structure that can be exentded for SCTP in this way.
I really don't see a sane way to allow newer userspaces to be compatible with older kernels here. If we were to do it I would suggest moving the responsibility for that feature into lksctp-tools, versioning that library such that correlary symbols are versioned to translate the application view of the socket options structs to the size and format that the running kernel undertands. Note that I'm not really advocating for that, as it seems like a fast moving target, but if we were to do it I think that would be the most sane way to handle it.
Neil
Quoted 25 lines
Quoted 23 lines
Quoted 23 lines
Signed-off-by Julien Gomes <[email protected]>--- net/sctp/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sctp/socket.c +1
-1@@ -2311,7 +2311,7 @@ static int sctp_setsockopt_events(struct sock *sk, char __user *optval,int i;if (optlen >sizeof (struct sctp_event_subscribe )) -return -EINVAL ; + optlen =sizeof (struct sctp_event_subscribe );if (copy_from_user (&subscribe, optval, optlen))return -EFAULT ;Signature · 1 line
2.20.1
Marcelo Ricardo Leitner
maintainerQuoted 35 lines · Neil Horman
Quoted 22 lines · Julien Gomes
Quoted 19 lines · Marcelo Ricardo Leitner
Quoted 7 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
My issue with this change is that by doing this, application will have no clue if the new bits were ignored or not and it may think that an event is enabled while it is not.
A workaround would be to do a getsockopt and check the size that was returned. But then, it might as well use the right struct here in the first place.
Quoted 8 lines
Quoted 5 lines
I'm seeing current implementation as an implicitly versioned argument:
it will always accept setsockopt calls with an old struct (v4.11 or v4.12), but if the user tries to use v3 on a v1-only system, it will be rejected. Pretty much like using a newer setsockopt on an old system.With the current implementation, given sources that say are supposed to run on a 4.9 kernel (no use of any newer field added in 4.11 or 4.12),
What given sources say that? I understand it might be expected, but this is an common concern with setsockopt method on many protocols, it just so happens that sctp extends them more than other protocols.
Quoted 7 lines
Quoted 3 lines
we can't rebuild the exact same sources on a 4.19 kernel and still run them on 4.9 without messing with structures re-definition.
Right, put another way, we support backward compatibility with older userspace applications, but not newer one. I.e. if you build an application against the 4.9 SCTP API, it should work with the 4.19 UAPI, but not vice versa, which it seems is like what you are trying to do here.
Was looking for that. Thanks.
Quoted 17 lines
Quoted 5 lines
I understand your point, but this still looks like a sort of uapi breakage to me.
Quoted 8 lines
I also had another way to work-around this in mind, by copying optlen bytes and checking that any additional field (not included in the "current" kernel structure definition) is not set, returning EINVAL in such case to keep a similar to current behavior.
The issue with this is that I didn't find a suitable (ie not totally arbitrary such as "twice the existing structure size") upper limit to optlen.There is no real uppper limit to the size of the structure in this case, and IIRC this isn't the only sockopt structure that can be exentded for SCTP in this way.
Quoted 8 lines
I really don't see a sane way to allow newer userspaces to be compatible with older kernels here. If we were to do it I would suggest moving the responsibility for that feature into lksctp-tools, versioning that library such that correlary symbols are versioned to translate the application view of the socket options structs to the size and format that the running kernel undertands. Note that I'm not really advocating for that, as it seems like a fast moving target, but if we were to do it I think that would be the most sane way to handle it.
Speaking of that, recent lksctp-tools got some defines to help knowing which features the available kernel headers have as it now probes if specific struct members are available or not. Though yeah, it also wouldn't help in this case, just mentioning it.
Quoted 28 lines
Neil
Quoted 25 lines
Quoted 23 lines
Quoted 23 lines
Signed-off-by Julien Gomes <[email protected]>--- net/sctp/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sctp/socket.c +1
-1@@ -2311,7 +2311,7 @@ static int sctp_setsockopt_events(struct sock *sk, char __user *optval,int i;if (optlen >sizeof (struct sctp_event_subscribe )) -return -EINVAL ; + optlen =sizeof (struct sctp_event_subscribe );if (copy_from_user (&subscribe, optval, optlen))return -EFAULT ;Signature · 1 line
2.20.1
David Miller
maintainerFrom: Marcelo Ricardo Leitner <[email protected]>
Date: Wed, 6 Feb 2019 18:37:54 -0200
Quoted 10 lines · Julien Gomes
Quoted 7 lines
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
What a complete mess we have here.
Use new socket option numbers next time, do not change the size and/or layout of existing socket options.
This whole thread, if you read it, is basically "if we compatability this way, that breaks, and if we do compatability this other way oh shit this other thing doesn't work."
I think we really need to specifically check for the difference sizes that existed one by one, clear out the part not given by the user, and backport this as far back as possible in a way that in the older kernels we see if the user is actually trying to use the new features and if so error out.
Which, btw, is terrible behavior. Newly compiled apps should work on older kernels if they don't try to use the new features, and if they can the ones that want to try to use the new features should be able to fall back when that feature isn't available in a non-ambiguous and precisely defined way.
The fact that the use of the new feature is hidden in the new structure elements is really rotten.
This patch, at best, needs some work and definitely a longer and more detailed commit message.
Marcelo Ricardo Leitner
maintainerQuoted 12 lines · David Miller
From: Marcelo Ricardo Leitner <[email protected]>
Date: Wed, 6 Feb 2019 18:37:54 -0200Quoted 9 lines · Julien Gomes
Quoted 7 lines
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.
Quoted 6 lines
Quoted 1 line
Not sure if we support backwards compatibility like this?
What a complete mess we have here.
Use new socket option numbers next time, do not change the size and/or layout of existing socket options.
What about reusing the same socket option, but defining a new struct? Say, MYSOCKOPT supports struct mysockopt, struct mysockopt2, struct mysockopt3...
That way we have a clear definition of the user's intent.
Quoted 10 lines
This whole thread, if you read it, is basically "if we compatability this way, that breaks, and if we do compatability this other way oh shit this other thing doesn't work."
I think we really need to specifically check for the difference sizes that existed one by one, clear out the part not given by the user, and backport this as far back as possible in a way that in the older kernels we see if the user is actually trying to use the new features and if so error out.
I'm afraid clearing out may not be enough, though seems it's the best we can do so far. If the struct is allocated but not fully initialized via a memset, but by setting its fields one by one, the remaining new fields will be left uninitinialized.
Quoted 3 lines
Which, btw, is terrible behavior. Newly compiled apps should work on older kernels if they don't try to use the new features, and if they
One use case here is: a given distro is using kernel X and app Foo is built against it. Then upgrades to X+1, Foo is patched to fix an issue and is rebuilt against X+1. The user upgrades Foo package but for whatever reason, doesn't upgrade kernel or reboot the system. Here, Foo doesn't work anymore until the new kernel is also running.
Quoted 9 lines
can the ones that want to try to use the new features should be able to fall back when that feature isn't available in a non-ambiguous and precisely defined way.
The fact that the use of the new feature is hidden in the new structure elements is really rotten.
This patch, at best, needs some work and definitely a longer and more detailed commit message.
Marcelo Ricardo Leitner
maintainerQuoted 37 lines · Marcelo Ricardo Leitner
Quoted 12 lines · David Miller
From: Marcelo Ricardo Leitner <[email protected]>
Date: Wed, 6 Feb 2019 18:37:54 -0200Quoted 9 lines · Julien Gomes
Quoted 7 lines
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Quoted 6 lines
Quoted 1 line
Not sure if we support backwards compatibility like this?
What a complete mess we have here.
Use new socket option numbers next time, do not change the size and/or layout of existing socket options.
What about reusing the same socket option, but defining a new struct? Say, MYSOCKOPT supports struct mysockopt, struct mysockopt2, struct mysockopt3...
That way we have a clear definition of the user's intent.
Quoted 10 lines
This whole thread, if you read it, is basically "if we compatability this way, that breaks, and if we do compatability this other way oh shit this other thing doesn't work."
I think we really need to specifically check for the difference sizes that existed one by one, clear out the part not given by the user, and backport this as far back as possible in a way that in the older kernels we see if the user is actually trying to use the new features and if so error out.
Quoted 4 lines
I'm afraid clearing out may not be enough, though seems it's the best we can do so far. If the struct is allocated but not fully initialized via a memset, but by setting its fields one by one, the remaining new fields will be left uninitinialized.
Need to clarify the "clearing out", I think it was meant differently. It was more about on how to ensure that the 16-bytes long of the v3 supplied to a v1-only kernel is compatible with the 12-bytes long v1. The kernel would have to check the trailing 4 bytes after v1-size and make sure they are all zeroed in order for the old kernel to accept it as a v1. But, as I said above, there are situations that this will not be enough.
Quoted 21 lines
Quoted 4 lines
Which, btw, is terrible behavior. Newly compiled apps should work on older kernels if they don't try to use the new features, and if they
One use case here is: a given distro is using kernel X and app Foo is built against it. Then upgrades to X+1, Foo is patched to fix an issue and is rebuilt against X+1. The user upgrades Foo package but for whatever reason, doesn't upgrade kernel or reboot the system. Here, Foo doesn't work anymore until the new kernel is also running.
Quoted 9 lines
can the ones that want to try to use the new features should be able to fall back when that feature isn't available in a non-ambiguous and precisely defined way.
The fact that the use of the new feature is hidden in the new structure elements is really rotten.
This patch, at best, needs some work and definitely a longer and more detailed commit message.
We have issues on read path too. 52ccb8e90c0a ("[SCTP]: Update SCTP_PEER_ADDR_PARAMS socket option to the latest api draft.") extended struct sctp_paddrparams and its getsockopt goes with:
sctp_getsockopt_peer_addr_params()
...
if (len < sizeof(struct sctp_paddrparams))
return -EINVAL;
len = sizeof(struct sctp_paddrparams);By then, we didn't have the /uapi/ folder yet. There may other cases.
From: Marcelo Ricardo Leitner
Quoted 1 line
Sent: 10 February 2019 20:16
...
Quoted 3 lines
We have issues on read path too.
52ccb8e90c0a("[SCTP]: UpdateSCTP_PEER_ADDR_PARAMSsocket option to the latest api draft.") extended structsctp_paddrparamsand its getsockopt goes with:
The API shouldn't change like this at all.
Is this from the RFC or elsewhere??
If the structure changes the socket option name and value should also change.
IMHO large chunks of the sctp rfc are just horrid.
In particular all the places where is states that API functions are implemented using setsockopt() - that should be an implementation detail. Also ISTR that some of the structures are defined to have holes in them...
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
'Marcelo Ricardo Leitner'
maintainerQuoted 9 lines · David Laight
From: Marcelo Ricardo Leitner
Quoted 1 line
Sent: 10 February 2019 20:16
...
Quoted 3 lines
We have issues on read path too.
52ccb8e90c0a("[SCTP]: UpdateSCTP_PEER_ADDR_PARAMSsocket option to the latest api draft.") extended structsctp_paddrparamsand its getsockopt goes with:The API shouldn't change like this at all.
Is this from the RFC or elsewhere??
I would think so. That commit is from 2005, pretty close to initial SCTP RFCs.
Quoted 3 lines
If the structure changes the socket option name and value should also change.
That's what is at the core of this thread.
Marcelo
Quoted 12 lines
IMHO large chunks of the sctp rfc are just horrid.
In particular all the places where is states that API functions are implemented usingsetsockopt()- that should be an implementation detail. Also ISTR that some of the structures are defined to have holes in them...David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Quoted 17 lines · Marcelo Ricardo Leitner
Quoted 16 lines · David Miller
From: Marcelo Ricardo Leitner <[email protected]>
Date: Wed, 6 Feb 2019 18:37:54 -0200Quoted 10 lines · Julien Gomes
Quoted 7 lines
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Not sure if we support backwards compatibility like this?
What a complete mess we have here.
Quoted 9 lines
Quoted 2 lines
Use new socket option numbers next time, do not change the size and/or layout of existing socket options.
What about reusing the same socket option, but defining a new struct? Say, MYSOCKOPT supports struct mysockopt, struct mysockopt2, struct mysockopt3...
That way we have a clear definition of the user's intent.
Thats possible, but I think thats pretty equivalaent to what daves saying, in that he wants us to identify all the sizes of this struct and the git history and act on them accordingly. Having internal versions of the struct seems like a fine way to get there, but I think we need to consider how we got to this situations before we go down the implementation path.
Quoted 11 lines
Quoted 10 lines
This whole thread, if you read it, is basically "if we compatability this way, that breaks, and if we do compatability this other way oh shit this other thing doesn't work."
I think we really need to specifically check for the difference sizes that existed one by one, clear out the part not given by the user, and backport this as far back as possible in a way that in the older kernels we see if the user is actually trying to use the new features and if so error out.
Quoted 5 lines
I'm afraid clearing out may not be enough, though seems it's the best we can do so far. If the struct is allocated but not fully initialized via a memset, but by setting its fields one by one, the remaining new fields will be left uninitinialized.
I'm not sure this even makes sense. Currently (as I understood it), the issue we are facing is the one in which an application is built against a newer kernel and run on an older one, the implication there being that the application will pass in a buffer that is larger than what the kernel expects. In that situation, clearing isn't needed, all thats needed (I think), is a memcmp of the space between the sizeof(kernel struct version), and sizeof(userspace struct version) to see if any bits are non-zero. If they are, we error out, otherwise, we ignore the space and move forward as though that overage doesn't exist.
Mind you, I'm not (yet) advocating for that approach, just trying to clarify whats needed.
Quoted 10 lines
Quoted 3 lines
Which, btw, is terrible behavior. Newly compiled apps should work on older kernels if they don't try to use the new features, and if they
One use case here is: a given distro is using kernel X and app Foo is built against it. Then upgrades to X+1, Foo is patched to fix an issue and is rebuilt against X+1. The user upgrades Foo package but for whatever reason, doesn't upgrade kernel or reboot the system. Here, Foo doesn't work anymore until the new kernel is also running.
Yes, thats the use case that we're trying to address.
Quoted 10 lines
Quoted 9 lines
can the ones that want to try to use the new features should be able to fall back when that feature isn't available in a non-ambiguous and precisely defined way.
The fact that the use of the new feature is hidden in the new structure elements is really rotten.
This patch, at best, needs some work and definitely a longer and more detailed commit message.
FWIW, before we decide on a course of action, I think I need to point out that, over the last 10 years, we've extended this structure 6 times, in the following commits:0f3fffd8ab1db7e8616d8e7731e1cdd553d482c35ea82d611da5c95129d127c6db444153fb5a64
The first two I believe were modifications during a period when sctp was actually getting integrated to the kernel, but the last 4 were definately done during more recent development periods and wen't in without any commentary about the impact to UAPI compatibility. The check for optlen > sizeof(struct sctp_event_subscribe) was made back in 2008, and while not spelled out, seems pretty clearly directed at enforcing compatibility with older appliations, not compatibility with newer applications running on older kernels.
I really worry about situations in which we need to support applications expecting features that the running kernel doesn't have. In this particular situation it seems like a fixable thing, but I could envision situations in which we just can't do it, and I don't want to set that expectation when we can't consistently meet it.
So, if the consensus is that we need to support applications built on newer kernels, but run on older kernels (and I'd like to get verbal consensus on that), then we need to identify a method to fix this. I'm still hesitant to do anything that involves us accepting any size buffer over the kernel expected size, as that puts us in a position to have to read large amounts of user data (i.e. possible DOS), and just picking an arbitrary large number to limit the buffer size seems wrong. What if, on receipt of a structure from a newer kernel (implying a size larger than what the kernel expects), we clamp optlen to the kernel size, and put_user it back to the application? i.e. we don't check any data above and beyond what the the kernel knows about, but we use the optlen as an indicator to user space that not all the data was processed? That allows the kernel to ignore the overage safely, and while its not in the socket api extension RFC, its not violating anything, and is something we can document in the sctp(7) man page as a linux only behavior.
Thoughts?
Neil
Marcelo Ricardo Leitner
maintainerQuoted 25 lines · Neil Horman
Quoted 24 lines · Marcelo Ricardo Leitner
Quoted 12 lines · David Miller
From: Marcelo Ricardo Leitner <[email protected]>
Date: Wed, 6 Feb 2019 18:37:54 -0200Quoted 9 lines · Julien Gomes
Quoted 7 lines
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Quoted 6 lines
Quoted 1 line
Not sure if we support backwards compatibility like this?
What a complete mess we have here.
Use new socket option numbers next time, do not change the size and/or layout of existing socket options.
What about reusing the same socket option, but defining a new struct? Say, MYSOCKOPT supports struct mysockopt, struct mysockopt2, struct mysockopt3...
Quoted 7 lines
Quoted 2 lines
That way we have a clear definition of the user's intent.
Thats possible, but I think thats pretty equivalaent to what daves saying, in that he wants us to identify all the sizes of this struct and the git history and act on them accordingly. Having internal versions of the struct seems like a fine way to get there, but I think we need to consider how we got to this situations before we go down the implementation path.
I was more referring to future stuff, but yes. I find it a bit easier to handle than having to switch the sockopt too and so far I couldn't find drawbacks to it.
That is, when using a new sockopt, we could accept a buffer larger than the needed, but I'm not considering that as a valid point anymore. Putting this compatibility aside for a moment, that pretty much means the user doesn't know what it wants and so we also don't.
Quoted 18 lines
Quoted 12 lines
Quoted 11 lines
This whole thread, if you read it, is basically "if we compatability this way, that breaks, and if we do compatability this other way oh shit this other thing doesn't work."
I think we really need to specifically check for the difference sizes that existed one by one, clear out the part not given by the user, and backport this as far back as possible in a way that in the older kernels we see if the user is actually trying to use the new features and if so error out.
Quoted 5 lines
I'm afraid clearing out may not be enough, though seems it's the best we can do so far. If the struct is allocated but not fully initialized via a memset, but by setting its fields one by one, the remaining new fields will be left uninitinialized.
Quoted 8 lines
I'm not sure this even makes sense. Currently (as I understood it), the issue we are facing is the one in which an application is built against a newer kernel and run on an older one, the implication there being that the application will pass in a buffer that is larger than what the kernel expects. In that situation, clearing isn't needed, all thats needed (I think), is a memcmp of the space between the sizeof(kernel struct version), and sizeof(userspace struct version) to see if any bits are non-zero. If they are, we error out, otherwise, we ignore the space and move forward as though that overage doesn't exist.
That's exactly what I tried to mean. :-)
Quoted 3 lines
Mind you, I'm not (yet) advocating for that approach, just trying to clarify whats needed.
Ok.
Quoted 40 lines
Quoted 10 lines
Quoted 3 lines
Which, btw, is terrible behavior. Newly compiled apps should work on older kernels if they don't try to use the new features, and if they
One use case here is: a given distro is using kernel X and app Foo is built against it. Then upgrades to X+1, Foo is patched to fix an issue and is rebuilt against X+1. The user upgrades Foo package but for whatever reason, doesn't upgrade kernel or reboot the system. Here, Foo doesn't work anymore until the new kernel is also running.
Yes, thats the use case that we're trying to address.
Quoted 10 lines
Quoted 9 lines
can the ones that want to try to use the new features should be able to fall back when that feature isn't available in a non-ambiguous and precisely defined way.
The fact that the use of the new feature is hidden in the new structure elements is really rotten.
This patch, at best, needs some work and definitely a longer and more detailed commit message.
FWIW, before we decide on a course of action, I think I need to point out that, over the last 10 years, we've extended this structure 6 times, in the following commits:
0f3fffd8ab1db7e8616d8e7731e1cdd553d482c35ea82d611da5c95129d127c6db444153fb5a64The first two I believe were modifications during a period when sctp was actually getting integrated to the kernel, but the last 4 were definately done during more recent development periods and wen't in without any commentary about the impact to UAPI compatibility. The check for optlen >
sizeof(struct sctp_event_subscribe)was made back in 2008, and while not spelled out, seems pretty clearly directed at enforcing compatibility with older appliations, not compatibility with newer applications running on older kernels.
Quoted 8 lines
I really worry about situations in which we need to support applications expecting features that the running kernel doesn't have. In this particular situation it seems like a fixable thing, but I could envision situations in which we just can't do it, and I don't want to set that expectation when we can't consistently meet it.
So, if the consensus is that we need to support applications built on newer kernels, but run on older kernels (and I'd like to get verbal consensus on
Yes from my side.
Quoted 7 lines
that), then we need to identify a method to fix this. I'm still hesitant to do anything that involves us accepting any size buffer over the kernel expected size, as that puts us in a position to have to read large amounts of user data (i.e. possible DOS), and just picking an arbitrary large number to limit the buffer size seems wrong. What if, on receipt of a structure from a newer kernel (implying a size larger than what the kernel expects), we clamp optlen to the kernel size, and
put_userit back to the application? i.e. we don't check any
We can't do that on setsockopt calls, as optlen is R/O there. Returning > 0 is not specified on setsockopt(2).
Quoted 8 lines
data above and beyond what the the kernel knows about, but we use the optlen as an indicator to user space that not all the data was processed? That allows the kernel to ignore the overage safely, and while its not in the socket api extension RFC, its not violating anything, and is something we can document in the sctp(7) man page as a linux only behavior.
Thoughts?
Neil
I also need to dig deeper on this, but in general what if we draw a line based on the current implementation:
- Current struct is X bytes long
- Patch current and older kernels to accept up to X bytes, as long as the trailing bytes are zeroed. Otherwise, EINVAL.
X may be a magic number for old kernel, but this way we avoid unbounded buffers and the limit is not random.
- On further changes, create a new, explicitly versioned struct. Older kernels will EINVAL if this new struct is used, which is expected.
Newer kernels will then have to cope with the different sizes/structs accordingly.
Marcelo
Quoted 18 lines · Julien Gomes
Make
sctp_setsockopt_events()able to acceptsctp_event_subscribestructures longer than the current definitions.This should prevent unjustified
setsockopt()failures due to structsctp_event_subscribeextensions (as in 4.11 and 4.12) when using binaries that should be compatible, but were built with later kernel uapi headers.Signed-off-by Julien Gomes <[email protected]>--- net/sctp/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sctp/socket.c +0
-0
net/sctp/socket.c static int sctp_setsockopt_events(struct sock *sk, char __user *optval, · line 2315
Quoted 6 lines
net/sctp/socket.c +1
-1@@ -2311,7 +2311,7 @@ static int sctp_setsockopt_events(struct sock *sk, char __user *optval,int i;if (optlen >sizeof (struct sctp_event_subscribe )) -return -EINVAL ; + optlen =sizeof (struct sctp_event_subscribe );
I'm not sure I like this. If you have a userspace application built against more recent uapi headers than the kernel you are actually running on, then by defintion you won't have this check in place, and you'll get EINVAL returns anyway. If you just backport this patch to an older kernel, you'll not get the EINVAL return, but you will get silent failures on event subscriptions that your application thinks exists, but the kernel doesn't recognize.
This would make sense if you had a way to communicate back to user space the unrecognized options, but since we don't (currently) have that, I would rather see the EINVAL returned than just have things not work.
Neil
Quoted 6 lines
net/sctp/socket.c +0
-0if (copy_from_user (&subscribe, optval, optlen))return -EFAULT ;Signature · 1 line
2.20.1