docs: Introduce deprecated APIs list
PATCH 1/1 docs: Introduce deprecated APIs list
As discussed in the "API replacement/deprecation" thread[1], this makes an effort to document what things shouldn't get (re)added to the kernel, by introducing Documentation/process/deprecated.rst. It also adds the overflow kerndoc to ReST output, and tweaks the struct_size() documentation to parse correctly.
[1] https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2018-September/005282.html
--- Documentation/driver-api/basics.rst | 3 + Documentation/process/deprecated.rst | 99 ++++++++++++++++++++++++++++ Documentation/process/index.rst | 1 + include/linux/overflow.h | 2 +- 4 files changed, 104 insertions(+), 1 deletion(-) create mode 100644 Documentation/process/deprecated.rst
Documentation/driver-api/basics.rst +3 -0
@@ -121,6 +121,9 @@ Kernel utility functions .. kernel-doc:: kernel/rcu/update.c :export: +.. kernel-doc:: include/linux/overflow.h + :internal: + Device Resource Management --------------------------
Documentation/process/deprecated.rst new file +99 -0
99 lines not on this page · raw message
Documentation/process/index.rst +1 -0
@@ -41,6 +41,7 @@ Other guides to the community that are of interest to most developers are: stable-kernel-rules submit-checklist kernel-docs + deprecated These are some overall technical guides that have been put here for now for lack of a better place.
include/linux/overflow.h +1 -1
@@ -291,7 +291,7 @@ static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) }/** -* struct_size() - Calculate size of structure with trailing array. +* function struct_size() - Calculate size of structure with trailing array. * @p: Pointer to the structure. * @member: Name of the array member. * @n: Number of elements in the array.
Signature · 6 lines
2.17.1 -- Kees Cook Pixel Security
Randy Dunlap
reviewerQuoted 23 lines · Kees Cook
As discussed in the "API replacement/deprecation" thread[1], this makes an effort to document what things shouldn't get (re)added to the kernel, by introducing
Documentation/process/deprecated.rst. It also adds the overflow kerndoc to ReST output, and tweaks thestruct_size()documentation to parse correctly.[1] https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2018-September/005282.html
Signed-off-by Kees Cook <[email protected]>--- Documentation/driver-api/basics.rst | 3 + Documentation/process/deprecated.rst | 99 ++++++++++++++++++++++++++++ Documentation/process/index.rst | 1 + include/linux/overflow.h | 2 +- 4 files changed, 104 insertions(+), 1 deletion(-) create mode 100644 Documentation/process/deprecated.rstinclude/linux/overflow.h +0
-0
include/linux/overflow.h static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) · line 294
Quoted 5 lines
include/linux/overflow.h +1
-1@@ -291,7 +291,7 @@ static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) }/** -* struct_size() - Calculate size of structure with trailing array. +* function struct_size() - Calculate size of structure with trailing array.
That syntax is not explained nor documented in Documentation/doc-guide/kernel-doc.rst.
Is the root problem that the function name begins with "struct"? Please explain in the patch description.
include/linux/overflow.h static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) · line 297
Quoted 3 lines
include/linux/overflow.h +0
-0* @p: Pointer to the structure. * @member: Name of the array member. * @n: Number of elements in the array.
thanks.
Signature · 1 line
~Randy
Quoted 26 lines · Randy Dunlap
Quoted 25 lines · Kees Cook
As discussed in the "API replacement/deprecation" thread[1], this makes an effort to document what things shouldn't get (re)added to the kernel, by introducing
Documentation/process/deprecated.rst. It also adds the overflow kerndoc to ReST output, and tweaks thestruct_size()documentation to parse correctly.[1] https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2018-September/005282.html
Signed-off-by Kees Cook <[email protected]>--- Documentation/driver-api/basics.rst | 3 + Documentation/process/deprecated.rst | 99 ++++++++++++++++++++++++++++ Documentation/process/index.rst | 1 + include/linux/overflow.h | 2 +- 4 files changed, 104 insertions(+), 1 deletion(-) create mode 100644 Documentation/process/deprecated.rstinclude/linux/overflow.h +0
-0@@ -291,7 +291,7 @@ static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) }
Quoted 8 lines
include/linux/overflow.h static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) · line 294
Quoted 3 lines
include/linux/overflow.h +1
-1/** -* struct_size() - Calculate size of structure with trailing array. +* function struct_size() - Calculate size of structure with trailing array. That syntax is not explained nor documented in
Documentation/doc-guide/kernel-doc.rst.Is the root problem that the function name begins with "struct"? Please explain in the patch description.
Indeed, shouldn't be needed.
BR,
Jani.
Quoted 7 lines
include/linux/overflow.h static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) · line 297
Quoted 4 lines
include/linux/overflow.h +0
-0* @p: Pointer to the structure. * @member: Name of the array member. * @n: Number of elements in the array. thanks.
Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
Randy Dunlap
reviewerQuoted 33 lines · Jani Nikula
Quoted 32 lines · Randy Dunlap
Quoted 23 lines · Kees Cook
As discussed in the "API replacement/deprecation" thread[1], this makes an effort to document what things shouldn't get (re)added to the kernel, by introducing
Documentation/process/deprecated.rst. It also adds the overflow kerndoc to ReST output, and tweaks thestruct_size()documentation to parse correctly.[1] https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2018-September/005282.html
Signed-off-by Kees Cook <[email protected]>--- Documentation/driver-api/basics.rst | 3 + Documentation/process/deprecated.rst | 99 ++++++++++++++++++++++++++++ Documentation/process/index.rst | 1 + include/linux/overflow.h | 2 +- 4 files changed, 104 insertions(+), 1 deletion(-) create mode 100644 Documentation/process/deprecated.rstinclude/linux/overflow.h +0
-0include/linux/overflow.h static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) · line 294
Quoted 5 lines
include/linux/overflow.h +1
-1@@ -291,7 +291,7 @@ static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) }/** -* struct_size() - Calculate size of structure with trailing array. +* function struct_size() - Calculate size of structure with trailing array. That syntax is not explained nor documented in
Documentation/doc-guide/kernel-doc.rst.
Quoted 15 lines
Quoted 2 lines
Is the root problem that the function name begins with "struct"? Please explain in the patch description.
Indeed, shouldn't be needed.
BR,
Jani.Quoted 7 lines
include/linux/overflow.h static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) · line 297
Quoted 4 lines
include/linux/overflow.h +0
-0* @p: Pointer to the structure. * @member: Name of the array member. * @n: Number of elements in the array. thanks.
Well, this is just a guess (no testing), but in scripts/kernel-doc (at line 1907 in 4.19-rc8), we can see:
if ($identifier =~ m/^struct/) {
$decl_type = 'struct';
} elsif ($identifier =~ m/^union/) {
$decl_type = 'union';
} elsif ($identifier =~ m/^enum/) {
$decl_type = 'enum';
} elsif ($identifier =~ m/^typedef/) {
$decl_type = 'typedef';
} else {
$decl_type = 'function';
}I wouldn't be surprised if a function named "struct_size" looks like a type struct. Maybe it needs to be more strict, with either a space or word boundary at the end of each type string. E.g.:
if ($identifier =~ m/^struct\b/) {
$decl_type = 'struct';
} elsif ($identifier =~ m/^union\b/) {
$decl_type = 'union';
} elsif ($identifier =~ m/^enum\b/) {
$decl_type = 'enum';
} elsif ($identifier =~ m/^typedef\b/) {
$decl_type = 'typedef';
} else {
$decl_type = 'function';
}Signature · 1 line
~Randy
Quoted 34 lines · Randy Dunlap
Quoted 33 lines · Jani Nikula
Quoted 32 lines · Randy Dunlap
Quoted 23 lines · Kees Cook
As discussed in the "API replacement/deprecation" thread[1], this makes an effort to document what things shouldn't get (re)added to the kernel, by introducing
Documentation/process/deprecated.rst. It also adds the overflow kerndoc to ReST output, and tweaks thestruct_size()documentation to parse correctly.[1] https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2018-September/005282.html
Signed-off-by Kees Cook <[email protected]>--- Documentation/driver-api/basics.rst | 3 + Documentation/process/deprecated.rst | 99 ++++++++++++++++++++++++++++ Documentation/process/index.rst | 1 + include/linux/overflow.h | 2 +- 4 files changed, 104 insertions(+), 1 deletion(-) create mode 100644 Documentation/process/deprecated.rstinclude/linux/overflow.h +0
-0include/linux/overflow.h static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) · line 294
Quoted 5 lines
include/linux/overflow.h +1
-1@@ -291,7 +291,7 @@ static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) }/** -* struct_size() - Calculate size of structure with trailing array. +* function struct_size() - Calculate size of structure with trailing array. That syntax is not explained nor documented in
Documentation/doc-guide/kernel-doc.rst.
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Is the root problem that the function name begins with "struct"? Please explain in the patch description.
Indeed, shouldn't be needed.
I actually thought the problem was with it not knowing how to deal with struct_size() being a macro instead of a real function.
Quoted 19 lines
Well, this is just a guess (no testing), but in
scripts/kernel-doc(at line 1907 in 4.19-rc8), we can see:if ($identifier =~ m/^struct/) { $decl_type = 'struct'; } elsif ($identifier =~ m/^union/) { $decl_type = 'union'; } elsif ($identifier =~ m/^enum/) { $decl_type = 'enum'; } elsif ($identifier =~ m/^typedef/) { $decl_type = 'typedef'; } else { $decl_type = 'function'; }I wouldn't be surprised if a function named "
struct_size" looks like a type struct. Maybe it needs to be more strict, with either a space or word boundary at the end of each type string. E.g.:
Quoted 11 lines
if ($identifier =~ m/^struct\b/) {
$decl_type = 'struct';
} elsif ($identifier =~ m/^union\b/) {
$decl_type = 'union';
} elsif ($identifier =~ m/^enum\b/) {
$decl_type = 'enum';
} elsif ($identifier =~ m/^typedef\b/) {
$decl_type = 'typedef';
} else {
$decl_type = 'function';
}But I see it's actually the prefix! :P
Using the above code fixes it for me. Can you send this fix with my Tested-by, and I'll spin a v2 of my "deprecated.rst" patch without the overflow.h change?
-Kees
Signature · 2 lines
Kees Cook Pixel Security
Randy Dunlap
reviewerQuoted 74 lines · Kees Cook
Quoted 34 lines · Randy Dunlap
Quoted 33 lines · Jani Nikula
Quoted 32 lines · Randy Dunlap
Quoted 23 lines · Kees Cook
As discussed in the "API replacement/deprecation" thread[1], this makes an effort to document what things shouldn't get (re)added to the kernel, by introducing
Documentation/process/deprecated.rst. It also adds the overflow kerndoc to ReST output, and tweaks thestruct_size()documentation to parse correctly.[1] https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2018-September/005282.html
Signed-off-by Kees Cook <[email protected]>--- Documentation/driver-api/basics.rst | 3 + Documentation/process/deprecated.rst | 99 ++++++++++++++++++++++++++++ Documentation/process/index.rst | 1 + include/linux/overflow.h | 2 +- 4 files changed, 104 insertions(+), 1 deletion(-) create mode 100644 Documentation/process/deprecated.rstinclude/linux/overflow.h +0
-0include/linux/overflow.h static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) · line 294
Quoted 5 lines
include/linux/overflow.h +1
-1@@ -291,7 +291,7 @@ static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) }/** -* struct_size() - Calculate size of structure with trailing array. +* function struct_size() - Calculate size of structure with trailing array. That syntax is not explained nor documented in
Documentation/doc-guide/kernel-doc.rst.Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Is the root problem that the function name begins with "struct"? Please explain in the patch description.
Indeed, shouldn't be needed.
I actually thought the problem was with it not knowing how to deal with
struct_size()being a macro instead of a real function.Quoted 19 lines
Well, this is just a guess (no testing), but in
scripts/kernel-doc(at line 1907 in 4.19-rc8), we can see:if ($identifier =~ m/^struct/) { $decl_type = 'struct'; } elsif ($identifier =~ m/^union/) { $decl_type = 'union'; } elsif ($identifier =~ m/^enum/) { $decl_type = 'enum'; } elsif ($identifier =~ m/^typedef/) { $decl_type = 'typedef'; } else { $decl_type = 'function'; }I wouldn't be surprised if a function named "
struct_size" looks like a type struct. Maybe it needs to be more strict, with either a space or word boundary at the end of each type string. E.g.:Quoted 11 lines
if ($identifier =~ m/^struct\b/) { $decl_type = 'struct'; } elsif ($identifier =~ m/^union\b/) { $decl_type = 'union'; } elsif ($identifier =~ m/^enum\b/) { $decl_type = 'enum'; } elsif ($identifier =~ m/^typedef\b/) { $decl_type = 'typedef'; } else { $decl_type = 'function'; }
Quoted 5 lines
But I see it's actually the prefix! :P
Using the above code fixes it for me. Can you send this fix with my Tested-by, and I'll spin a v2 of my "
deprecated.rst" patch without theoverflow.hchange?
OK, but I'll have to first test for unexpected consequences of it. (i.e., run full docs build with and without the patch)
Signature · 1 line
~Randy
Randy Dunlap
reviewerQuoted 75 lines · Randy Dunlap
Quoted 74 lines · Kees Cook
Quoted 34 lines · Randy Dunlap
Quoted 33 lines · Jani Nikula
Quoted 32 lines · Randy Dunlap
Quoted 23 lines · Kees Cook
As discussed in the "API replacement/deprecation" thread[1], this makes an effort to document what things shouldn't get (re)added to the kernel, by introducing
Documentation/process/deprecated.rst. It also adds the overflow kerndoc to ReST output, and tweaks thestruct_size()documentation to parse correctly.[1] https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2018-September/005282.html
Signed-off-by Kees Cook <[email protected]>--- Documentation/driver-api/basics.rst | 3 + Documentation/process/deprecated.rst | 99 ++++++++++++++++++++++++++++ Documentation/process/index.rst | 1 + include/linux/overflow.h | 2 +- 4 files changed, 104 insertions(+), 1 deletion(-) create mode 100644 Documentation/process/deprecated.rstinclude/linux/overflow.h +0
-0include/linux/overflow.h static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) · line 294
Quoted 5 lines
include/linux/overflow.h +1
-1@@ -291,7 +291,7 @@ static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c) }/** -* struct_size() - Calculate size of structure with trailing array. +* function struct_size() - Calculate size of structure with trailing array. That syntax is not explained nor documented in
Documentation/doc-guide/kernel-doc.rst.Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Is the root problem that the function name begins with "struct"? Please explain in the patch description.
Indeed, shouldn't be needed.
I actually thought the problem was with it not knowing how to deal with
struct_size()being a macro instead of a real function.Quoted 30 lines
Well, this is just a guess (no testing), but in
scripts/kernel-doc(at line 1907 in 4.19-rc8), we can see:if ($identifier =~ m/^struct/) { $decl_type = 'struct'; } elsif ($identifier =~ m/^union/) { $decl_type = 'union'; } elsif ($identifier =~ m/^enum/) { $decl_type = 'enum'; } elsif ($identifier =~ m/^typedef/) { $decl_type = 'typedef'; } else { $decl_type = 'function'; }I wouldn't be surprised if a function named "
struct_size" looks like a type struct. Maybe it needs to be more strict, with either a space or word boundary at the end of each type string. E.g.:if ($identifier =~ m/^struct\b/) { $decl_type = 'struct'; } elsif ($identifier =~ m/^union\b/) { $decl_type = 'union'; } elsif ($identifier =~ m/^enum\b/) { $decl_type = 'enum'; } elsif ($identifier =~ m/^typedef\b/) { $decl_type = 'typedef'; } else { $decl_type = 'function'; }
Quoted 8 lines
Quoted 5 lines
But I see it's actually the prefix! :P
Using the above code fixes it for me. Can you send this fix with my Tested-by, and I'll spin a v2 of my "
deprecated.rst" patch without theoverflow.hchange?OK, but I'll have to first test for unexpected consequences of it. (i.e., run full docs build with and without the patch)
I've tested it successfully. I'll send it out later tonight. (-EBUSY now)
Signature · 1 line
~Randy
Jonathan Corbet
maintainerQuoted 5 lines · On Tue, 16 Oct 2018 19:17:08 -0700 Kees Cook <[email protected]> wrote:
As discussed in the "API replacement/deprecation" thread[1], this makes an effort to document what things shouldn't get (re)added to the kernel, by introducing
Documentation/process/deprecated.rst. It also adds the overflow kerndoc to ReST output, and tweaks thestruct_size()documentation to parse correctly.
Seems like a good idea overall...a couple of quick comments
Quoted 7 lines
[1] https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2018-September/005282.html
Signed-off-by Kees Cook <[email protected]>--- Documentation/driver-api/basics.rst | 3 + Documentation/process/deprecated.rst | 99 ++++++++++++++++++++++++++++ Documentation/process/index.rst | 1 +
I wonder if "process" is the right place, or core-api? I guess we have a lot of similar stuff in process now.
[...]
Quoted 7 lines
Documentation/process/deprecated.rst new file +7
-0+open-coded arithmetic in allocator arguments +-------------------------------------------- +Dynamic size calculations (especially multiplication) +should not be performed in memory allocator (or similar) +function arguments. + +For example, do not use ``rows * cols`` as an argument, as in:
Documentation/process/deprecated.rst line 46
Quoted 10 lines
Documentation/process/deprecated.rst new file +10
-0+``kmalloc(rows * cols, GFP_KERNEL)``. +Instead, the 2-factor form of the allocator should be used: +``kmalloc_array(rows, cols, GFP_KERNEL)``. +If no 2-factor form is available, the saturate-on-overflow helpers should +be used: +``vmalloc(array_size(rows, cols))``. + +See :c:func:`array_size`, :c:func:`array3_size`, and :c:func:`struct_size`, +for more details as well as the related :c:func:`check_add_overflow` and +:c:func:`check_mul_overflow` family of functions.
I think this should say *why* developers are being told not to do it. Does this advice hold even in cases where the dimensions are known, under the kernel's control, and guaranteed not to overflow even on Alan's port to eight-bit CPUs?
To me it's also a bit confusing to present the arguments to kmalloc_array() as "rows" and "cols" when they are really "n" and "size".
Thanks,
jon
Quoted 13 lines · Jonathan Corbet
Quoted 5 lines · On Tue, 16 Oct 2018 19:17:08 -0700 Kees Cook <[email protected]> wrote:
As discussed in the "API replacement/deprecation" thread[1], this makes an effort to document what things shouldn't get (re)added to the kernel, by introducing
Documentation/process/deprecated.rst. It also adds the overflow kerndoc to ReST output, and tweaks thestruct_size()documentation to parse correctly.Seems like a good idea overall...a couple of quick comments
Quoted 8 lines
Quoted 5 lines
Signed-off-by Kees Cook <[email protected]>--- Documentation/driver-api/basics.rst | 3 + Documentation/process/deprecated.rst | 99 ++++++++++++++++++++++++++++ Documentation/process/index.rst | 1 +I wonder if "process" is the right place, or core-api? I guess we have a lot of similar stuff in process now.
Totally up to you. It seemed better suited for "process" (here's things NOT to do) than "core-api" (here's things to use).
Quoted 17 lines
[...]
Quoted 14 lines
Documentation/process/deprecated.rst new file +14
-0+open-coded arithmetic in allocator arguments +-------------------------------------------- +Dynamic size calculations (especially multiplication) +should not be performed in memory allocator (or similar) +function arguments. + +For example, do not use ``rows * cols`` as an argument, as in: +``kmalloc(rows * cols, GFP_KERNEL)``. +Instead, the 2-factor form of the allocator should be used: +``kmalloc_array(rows, cols, GFP_KERNEL)``. +If no 2-factor form is available, the saturate-on-overflow helpers should +be used: +``vmalloc(array_size(rows, cols))``. +
Quoted 8 lines
Documentation/process/deprecated.rst line 46
Quoted 3 lines
Documentation/process/deprecated.rst new file +3
-0+See :c:func:`array_size`, :c:func:`array3_size`, and :c:func:`struct_size`, +for more details as well as the related :c:func:`check_add_overflow` and +:c:func:`check_mul_overflow` family of functions.I think this should say *why* developers are being told not to do it. Does this advice hold even in cases where the dimensions are known, under the kernel's control, and guaranteed not to overflow even on Alan's port to eight-bit CPUs?
I will attempt to explain this better. When all factors are constants, the compiler will warn if there is an overflow. If, however, anything is a dynamic size, we run the risk of overflow. It's not true in all cases (e.g. u8 var * sizeof()) but it's more robust to globally avoid it. (What happens when that u8 becomes a u64 at some future time?)
Quoted 2 lines
To me it's also a bit confusing to present the arguments to
kmalloc_array()as "rows" and "cols" when they are really "n" and "size".
That's true, though I used rows * cols as an example because people aren't always thinking about their multiplications as having n-many sizes. e.g. "I just want a full screen of pixels." Let me see if I can adjust it...
Thanks for the feedback!
-Kees
Signature · 2 lines
Kees Cook Pixel Security
Quoted 6 lines · Kees Cook
As discussed in the "API replacement/deprecation" thread[1], this makes an effort to document what things shouldn't get (re)added to the kernel, by introducing
Documentation/process/deprecated.rst. It also adds the overflow kerndoc to ReST output, and tweaks thestruct_size()documentation to parse correctly.
If you remember, please CC me as well, since I would like to be involved in this as well, given that it touches the compiler attributes :-)
Also, I will send a patch (for the deprecated.rst file, adding a section about fallthrough comments) later on.
Thanks for adding the file!
Cheers,
Miguel