flexible-array member conversion patches for 5.8-rc2
The following changes since commit b3a9e3b9622ae10064826dccb4f7a52bd88c7407:
Linux 5.8-rc1 (2020-06-14 12:45:04 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/gustavoars/linux.git tags/flex-array-conversions-5.8-rc2
for you to fetch changes up to 4caeb3acd21ffa2496fc3e94398019d4b3415523:
w1: Replace zero-length array with flexible-array (2020-06-15 15:20:19 -0500)
----------------------------------------------------------------
flexible-array member conversion patches for 5.8-rc2
Hi Linus,
Please, pull the following patches that replace zero-length arrays with flexible-array members.
Notice that all of these patches have been baking in linux-next for two development cycles now.
There is a regular need in the kernel to provide a way to declare having a dynamically sized set of trailing elements in a structure. Kernel code should always use “flexible array members”[1] for these cases. The older style of one-element or zero-length arrays should no longer be used.
In older C code, dynamically sized trailing elements were done by specifying a one-element array at the end of a structure:
struct something {
size_t count;
struct foo items[1];
};This led to fragile size calculations via sizeof() (which would need to remove the size of the single trailing element to get a correct size of the “header”). A GNU C extension[2] was introduced to allow for zero-length arrays, to avoid these kinds of size problems:
struct something {
size_t count;
struct foo items[0];
};But this led to other problems, and didn’t solve some problems shared by both styles, like not being able to detect when such an array is accidentally being used _not_ at the end of a structure (which could happen directly, or when such a struct was in unions, structs of structs, etc).
C99 introduced “flexible array members”, which lacks a numeric size for the array declaration entirely:
struct something {
size_t count;
struct foo items[];
};This is the way the kernel expects dynamically sized trailing elements to be declared. It allows the compiler to generate errors when the flexible array does not occur last in the structure, which helps to prevent some kind of undefined behavior[3] bugs from being inadvertently introduced to the codebase. It also allows the compiler to correctly analyze array sizes (via sizeof(), CONFIG_FORTIFY_SOURCE, and CONFIG_UBSAN_BOUNDS). For instance, there is no mechanism that warns us that the following application of the sizeof() operator to a zero-length array always results in zero:
struct something {
size_t count;
struct foo items[0];
};struct something *instance;
instance = kmalloc(struct_size(instance, items, count), GFP_KERNEL); instance->count = count;
size = sizeof(instance->items) * instance->count;memcpy(instance->items, source, size);
At the last line of code above, size turns out to be zero, when one might have thought it represents the total size in bytes of the dynamic memory recently allocated for the trailing array items. Here are a couple examples of this issue[4][5]. Instead, flexible array members have incomplete type, and so the sizeof() operator may not be applied[2], so any misuse of such operators will be immediately noticed at build time.
With respect to one-element arrays, one has to be acutely aware that such arrays occupy at least as much space as a single object of the type[2], hence they contribute to the size of the enclosing structure. This is prone to error every time people want to calculate the total size of dynamic memory to allocate for a structure containing an array of this kind as a member:
struct something {
size_t count;
struct foo items[1];
};struct something *instance;
instance = kmalloc(struct_size(instance, items, count - 1), GFP_KERNEL); instance->count = count;
size = sizeof(instance->items) * instance->count;memcpy(instance->items, source, size);
In the example above, we had to remember to calculate count - 1 when using the struct_size() helper, otherwise we would have –unintentionally– allocated memory for one too many items objects. The cleanest and least error-prone way to implement this is through the use of a flexible array member:
struct something {
size_t count;
struct foo items[];
};struct something *instance;
instance = kmalloc(struct_size(instance, items, count), GFP_KERNEL); instance->count = count;
size = sizeof(instance->items[0]) * instance->count; memcpy(instance->items, source, size);
Thanks
--
Gustavo
[1] https://en.wikipedia.org/wiki/Flexible_array_member
[2] https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html
[3] https://git.kernel.org/linus/76497732932f15e7323dc805e8ea8dc11bb587cf
[4] https://git.kernel.org/linus/f2cd32a443da694ac4e28fbf4ac6f9d5cc63a539
[5] https://git.kernel.org/linus/ab91c2a89f86be2898cee208d492816ec238b2cf
----------------------------------------------------------------
Gustavo A. R. Silva (41):
firmware: dmi-sysfs: Replace zero-length array with flexible-array member
firmware: google: memconsole: Replace zero-length array with flexible-array member
firmware: google: vpd: Replace zero-length array with flexible-array member
aio: Replace zero-length array with flexible-array
ARM: tegra: Replace zero-length array with flexible-array
dmaengine: Replace zero-length array with flexible-array
can: peak_canfd: Replace zero-length array with flexible-array
can: Replace zero-length array with flexible-array
crypto: Replace zero-length array with flexible-array
drbd: Replace zero-length array with flexible-array
drm/edid: Replace zero-length array with flexible-array
cb710: Replace zero-length array with flexible-array
firewire: ohci: Replace zero-length array with flexible-array
FS-Cache: Replace zero-length array with flexible-array
ia64: kernel: unwind_i.h: Replace zero-length array with flexible-array
samples: mei: Replace zero-length array with flexible-array
ibft: Replace zero-length array with flexible-array
jffs2: Replace zero-length array with flexible-array
KVM: Replace zero-length array with flexible-array
kexec: Replace zero-length array with flexible-array
keys: encrypted-type: Replace zero-length array with flexible-array
kprobes: Replace zero-length array with flexible-array
libata: Replace zero-length array with flexible-array tools/testing/nvdimm: Replace zero-length array with flexible-array
block: Replace zero-length array with flexible-array
oprofile: Replace zero-length array with flexible-array
firmware: pcdp: Replace zero-length array with flexible-array
media: pwc: Replace zero-length array with flexible-array
rapidio: Replace zero-length array with flexible-array
RxRPC: Replace zero-length array with flexible-array
phy: samsung: Replace zero-length array with flexible-array
sctp: Replace zero-length array with flexible-array
ima: Replace zero-length array with flexible-array
ASoC: SOF: Replace zero-length array with flexible-array
Squashfs: Replace zero-length array with flexible-array
stm class: Replace zero-length array with flexible-array
dmaengine: tegra-apb: Replace zero-length array with flexible-array
tifm: Replace zero-length array with flexible-array
soc: ti: Replace zero-length array with flexible-array tracing/probe: Replace zero-length array with flexible-array
w1: Replace zero-length array with flexible-array
arch/ia64/kernel/unwind_i.h | 2 +- block/partitions/ldm.h | 2 +- drivers/amba/tegra-ahb.c | 2 +- drivers/block/drbd/drbd_int.h | 2 +- drivers/block/drbd/drbd_protocol.h | 8 +++--- drivers/crypto/chelsio/chcr_crypto.h | 8 +++--- drivers/dma/milbeaut-hdmac.c | 2 +- drivers/dma/milbeaut-xdmac.c | 2 +- drivers/dma/moxart-dma.c | 2 +- drivers/dma/tegra20-apb-dma.c | 2 +- drivers/dma/ti/edma.c | 2 +- drivers/dma/ti/k3-udma.c | 2 +- drivers/dma/timb_dma.c | 2 +- drivers/firewire/core-cdev.c | 2 +- drivers/firewire/core-transaction.c | 2 +- drivers/firewire/core.h | 2 +- drivers/firewire/nosy.c | 2 +- drivers/firewire/ohci.c | 2 +- drivers/firmware/dmi-sysfs.c | 2 +- drivers/firmware/google/memconsole-coreboot.c | 2 +- drivers/firmware/google/vpd.c | 2 +- drivers/firmware/iscsi_ibft.c | 2 +- drivers/firmware/pcdp.h | 2 +- drivers/hwtracing/stm/policy.c | 2 +- drivers/hwtracing/stm/stm.h | 4 +-- drivers/media/usb/pwc/pwc.h | 2 +- drivers/net/can/peak_canfd/peak_pciefd_main.c | 4 +-- drivers/oprofile/cpu_buffer.h | 2 +- drivers/phy/samsung/phy-samsung-usb2.h | 2 +- drivers/rapidio/rio-scan.c | 2 +- drivers/soc/ti/knav_qmss.h | 2 +- drivers/w1/w1_netlink.h | 4 +-- fs/aio.c | 2 +- fs/jffs2/nodelist.h | 2 +- fs/jffs2/summary.h | 4 +-- fs/squashfs/squashfs_fs.h | 16 ++++++------ include/drm/drm_displayid.h | 2 +- include/keys/encrypted-type.h | 2 +- include/keys/rxrpc-type.h | 4 +-- include/linux/can/skb.h | 2 +- include/linux/cb710.h | 2 +- include/linux/dmaengine.h | 4 +-- include/linux/fscache-cache.h | 2 +- include/linux/kexec.h | 2 +- include/linux/kprobes.h | 2 +- include/linux/kvm_host.h | 2 +- include/linux/libata.h | 2 +- include/linux/sctp.h | 36 +++++++++++++-------------- include/linux/tifm.h | 2 +- kernel/trace/trace_probe.h | 2 +- samples/mei/mei-amt-version.c | 2 +- security/integrity/integrity.h | 4 +-- sound/soc/sof/probe.h | 8 +++--- tools/testing/nvdimm/test/nfit_test.h | 6 ++--- 54 files changed, 96 insertions(+), 96 deletions(-)
Quoted 2 lines · Gustavo A. R. Silva
flexible-array member conversion patches for 5.8-rc2
Gaah. I merged this, and even spent some time trying to make the merge message look better, and then when I started looking at the end result, I ended up unpulling it.
It's all been recently rebased, which I might be willing to ignore for something like this, but all these one-liner patches then come with a boiler-plate commit message that is something like 50 lines long and is actively misleading as it talks about "this tree-wide patch" and just repeats the generic issues that were presented (better) in the merge message anyway.
The individual commit messages literally add nothing specific to that commit AT ALL except for the very first line, which is also boiler-plate but at least mentions the subsystem.
End result: when I do "git log" after having merged this, the result is basically two *thousand* lines of repeated noise.
Some repetition is fine. But 2000 lines of repeated boiler-plate makes me go "that's actively detrimental to any use of 'git log'" and I undid my pull.
Linus
Quoted 20 lines · Linus Torvalds
Quoted 2 lines · Gustavo A. R. Silva
flexible-array member conversion patches for 5.8-rc2
Gaah. I merged this, and even spent some time trying to make the merge message look better, and then when I started looking at the end result, I ended up unpulling it.
It's all been recently rebased, which I might be willing to ignore for something like this, but all these one-liner patches then come with a boiler-plate commit message that is something like 50 lines long and is actively misleading as it talks about "this tree-wide patch" and just repeats the generic issues that were presented (better) in the merge message anyway.
The individual commit messages literally add nothing specific to that commit AT ALL except for the very first line, which is also boiler-plate but at least mentions the subsystem.
Quoted 7 lines
End result: when I do "git log" after having merged this, the result is basically two *thousand* lines of repeated noise.
Some repetition is fine. But 2000 lines of repeated boiler-plate makes me go "that's actively detrimental to any use of 'git log'" and I undid my pull.
Yep, you're right. We made the mistake of not having something to point people to, from the beginning; which we are trying to fix by adding the following documentation to deprecated.rst:
https://lore.kernel.org/lkml/20200608213711.GA22271@embeddedor/
That's the reason why the log for these sorts of patches ended up being so repetitive: as a way to compensate for not having proper documentation in place from the beginning.
And yes, it was my mistake not to remove part of the log from the changelog text when I turned the treewide patch into separate per-subsystem patches --for they to be properly tested by the 0-day CI Kernel Test Service.
I will modify the changelog text of the separate patches and send you the pull-request once again.
BTW, the commit message is the exact same text we are planning to add to the deprecated.rst file, which will be used for both zero-length and one-element array conversions. It might be better to to remove the part for the one-element arrays, in this case.
Thanks
--
Gustavo