util-linux add dmesg caller_id Issue: 2666 patch 1 of 4
PATCH 1/1 util-linux add dmesg caller_id Issue: 2666 patch 1 of 4
Submission to Project: util-linux
Open Incident: #2609 at github.com/util-linux/util-linux/issues/2609
Component: util-linux/sys-utils
File: dmesg.c
Code level patch applied against: 2.39.3 - latest code pulled from
git.github.com:util-linux/util-linux.git
Revision: #1 on 2023/12/08 per Review from Karel Zak
Revision: #2 on 2023/12/12 Adjust line offsets for master update and
Add caller_id_size init to dmesg -K
Revision: #3 on 2023/12/12 Use of sizeof for cidbuf and limit search
for caller_id to dmesg prefix to msg text
Revision: #4 on 2023/12/15 Ensure SYSLOG and kmsg inputs have
caller_id_size set appropriately
Revision: #5 on 2023/12/24 Make caller_id width consistent with
makedumpfile
Revision: #6 on 2023/12/30 Use updated test naming convention
Revision: #7 on 2024/01/04 Normalize kmsg caller_id spacing for test
platforms by removing caller_id padding in test generated output.
Add support to standard dmesg command for the optional Linux Kernel debug CONFIG option PRINTK_CALLER which adds an optional dmesg field that contains the Thread Id or CPU Id that is issuing the printk to add the message to the kernel ring buffer. This makes debugging simpler as dmesg entries for a specific thread or CPU can be recognized.
The dmesg -S using the old syslog interface supports printing the PRINTK_CALLER field but currently standard dmesg does not support printing the field if present. There are utilities that use dmesg and so it would be optimal if dmesg supported PRINTK_CALLER as well.
The additional field provided by PRINTK_CALLER is only present if it was configured for the Linux kernel where the dmesg command is run. It is a debug option and not configured by default so the dmesg output will only change for those kernels where the option was configured when the kernel was built. For users who went to the trouble to configure PRINTK_CALLER and have the extra field available for debugging, having dmesg print the field is very helpful.
Size of the PRINTK_CALLER field is determined by the maximum number tasks that can be run on the system which is limited by the value of /proc/sys/kernel/pid_max as pid values are from 0 to value - 1. This value determines the number of id digits needed by the caller id. The PRINTK_CALLER field is printed as T<id> for a Task Id or C<id> for a CPU Id for a printk in CPU context. The values are left space
padded and enclosed in parentheses such as: [ T123] or [ C16]
For consistency with dmesg -S which supports the PRINTK_CALLER field the field is printed followed by a space. For JSON format output the PRINTK_CALLER field is identified as caller as that is consistent with it's naming in /dev/kmsg. No space padding is used to reduce space consumed by the JSON output. So the output from the command on a system with PRINTK_CALLER field enabled in the Linux .config file the dmesg output appears as:
Quoted 1 line
dmesg
...
[ 520.897104] [ T3919] usb 3-3: Product: USB 2.0 Hub
and
Quoted 1 line
dmesg -x
...
kern :info : [ 520.897104] [ T3919] usb 3-3: Product: USB 2.0 Hub
and
Quoted 1 line
dmesg -J
...
},{
"pri": 6,
"time": 520.897104,
"caller": "T3919",
"msg": "usb 3-3: Product: USB 2.0 Hub"
},{and
Quoted 1 line
dmesg -J -x
...
},{
"fac": "kern",
"pri": "info",
"time": 520.897104,
"caller": "T3919",
"msg": "usb 3-3: Product: USB 2.0 Hub"
},{Quoted 1 line
For comparison:
Quoted 1 line
dmesg -S
...
[ 520.897104] [ T3919] usb 3-3: Product: USB 2.0 Hub
and
Quoted 1 line
dmesg -S -x
...
kern :info : [ 520.897104] [ T3919] usb 3-3: Product: USB 2.0 Hub
Note: When dmesg uses the old syslog interface the reserved space for
the PRINTK_CALLER field is capped at 5 digits because 32-bit kernels are capped at 32768 as the max number of PIDs. However, 64-bit systems are currently capped at 2^22 PIDs (0 - 4194303). The PID cap is set by PID_MAX_LIMIT but the system limit can be less so we use /proc/sys/kernel/pid_max to determine the size needed to hold the maximum PID value size for the current system. Many 64-bit systems support 2^22 PIDs (0 - 4194303) and you see:
Quoted 1 line
dmesg -x
...
kern :info : [ 520.899558] [ T3919] hub 3-3:1.0: USB hub found
...
kern :info : [ 9830.456898] [ T98982] cgroup: fork rejected by pids ... kern :info : [14301.158878] [ T137336] cgroup: fork rejected by pids ... kern :info : [18980.803190] [T1637865] cgroup: fork rejected by pids ...
Quoted 1 line
dmesg -S -x
...
kern :info : [ 520.899558] [ T3919] hub 3-3:1.0: USB hub found
...
kern :info : [ 9830.456898] [T98982] cgroup: fork rejected by pids ... kern :info : [14301.158878] [T137336] cgroup: fork rejected by pids ... kern :info : [18980.803190] [T1637865] cgroup: fork rejected by pids ...
This is the only difference seen with PRINTK_CALLER configured and printing between the dmesg /dev/kmsg interface and the dmesg -S syslog interface.
Tests naming has been revised based on naming convention Thomas used to introduce dmest json tests. The naming of test and input files that reside in tests/ts/dmeg include:
<name> are existing dmesg syslog interface tests and input files. cid-<name> are dmesg syslog interface caller_id tests and input files. json-<name> are dmesg kmsg interface tests and input files. cid-json-<name> are dmesg kmsg interface caller_id tests and input files.
Resulting expected files match the test names.
--- include/pathnames.h | 3 + sys-utils/dmesg.c | 128 ++++- tests/expected/dmesg/cid-json | 535 +++++++++++++++++++ tests/expected/dmesg/cid-kmsg-colors | 59 ++ tests/expected/dmesg/cid-kmsg-console-levels | 146 +++++ tests/expected/dmesg/cid-kmsg-decode | 59 ++ tests/expected/dmesg/cid-kmsg-delta | 59 ++ tests/expected/dmesg/cid-kmsg-facilities | 73 +++ tests/expected/dmesg/cid-kmsg-indentation | 28 + tests/expected/dmesg/cid-kmsg-json | 295 ++++++++++ tests/expected/dmesg/cid-kmsg-limit | 31 ++ tests/expected/dmesg/kmsg-file | 126 ++++- tests/ts/dmesg/cid-input | 106 ++++ tests/ts/dmesg/cid-json | 28 + tests/ts/dmesg/cid-kmsg-colors | 34 ++ tests/ts/dmesg/cid-kmsg-console-levels | 49 ++ tests/ts/dmesg/cid-kmsg-decode | 33 ++ tests/ts/dmesg/cid-kmsg-delta | 33 ++ tests/ts/dmesg/cid-kmsg-facilities | 36 ++ tests/ts/dmesg/cid-kmsg-indentation | 45 ++ tests/ts/dmesg/cid-kmsg-input | Bin 0 -> 5464 bytes tests/ts/dmesg/cid-kmsg-json | 28 + tests/ts/dmesg/cid-kmsg-limit | 34 ++ tests/ts/dmesg/cid-kmsg-newlines | Bin 0 -> 152 bytes tests/ts/dmesg/kmsg-input | Bin 1955 -> 3944 bytes 25 files changed, 1948 insertions(+), 20 deletions(-) create mode 100644 tests/expected/dmesg/cid-json create mode 100644 tests/expected/dmesg/cid-kmsg-colors create mode 100644 tests/expected/dmesg/cid-kmsg-console-levels create mode 100644 tests/expected/dmesg/cid-kmsg-decode create mode 100644 tests/expected/dmesg/cid-kmsg-delta create mode 100644 tests/expected/dmesg/cid-kmsg-facilities create mode 100644 tests/expected/dmesg/cid-kmsg-indentation create mode 100644 tests/expected/dmesg/cid-kmsg-json create mode 100644 tests/expected/dmesg/cid-kmsg-limit create mode 100644 tests/ts/dmesg/cid-input create mode 100755 tests/ts/dmesg/cid-json create mode 100755 tests/ts/dmesg/cid-kmsg-colors create mode 100755 tests/ts/dmesg/cid-kmsg-console-levels create mode 100755 tests/ts/dmesg/cid-kmsg-decode create mode 100755 tests/ts/dmesg/cid-kmsg-delta create mode 100755 tests/ts/dmesg/cid-kmsg-facilities create mode 100755 tests/ts/dmesg/cid-kmsg-indentation create mode 100644 tests/ts/dmesg/cid-kmsg-input create mode 100755 tests/ts/dmesg/cid-kmsg-json create mode 100755 tests/ts/dmesg/cid-kmsg-limit create mode 100644 tests/ts/dmesg/cid-kmsg-newlines
include/pathnames.h +3 -0
@@ -230,4 +230,7 @@/* cgroup path */ #define _PATH_SYS_CGROUP "/sys/fs/cgroup" +/* Maximum number of PIDs system supports */ +#define _PATH_PROC_PIDMAX "/proc/sys/kernel/pid_max" +#endif /* PATHNAMES_H */
sys-utils/dmesg.c +125 -3
223 lines not on this page · raw message
tests/expected/dmesg/cid-json new file +535 -0
535 lines not on this page · raw message
tests/expected/dmesg/cid-kmsg-colors new file +59 -0
59 lines not on this page · raw message
tests/expected/dmesg/cid-kmsg-console-levels new file +146 -0
146 lines not on this page · raw message
tests/expected/dmesg/cid-kmsg-decode new file +59 -0
59 lines not on this page · raw message
tests/expected/dmesg/cid-kmsg-delta new file +59 -0
59 lines not on this page · raw message
tests/expected/dmesg/cid-kmsg-facilities new file +73 -0
73 lines not on this page · raw message
tests/expected/dmesg/cid-kmsg-indentation new file +28 -0
@@ -0,0 +1,28 @@ +[ 0.000000] [T0] line zero +[ 1.000000] [T1] new +[ 2.000000] [T2] two +[ 3.000000] [T3] three +[ 0.000000] [T0] line zero +[ 1.000000] [T1] new +[ 2.000000] [T2] two +[ 3.000000] [T3] three +[< 0.000000>] [T0] line zero +[< 0.000000>] [T1] new +[< 1.000000>] [T2] two +[< 1.000000>] [T3] three +[T0] line zero +[T1] new +[T2] two +[T3] three +[Feb13 23:31] [T0] line zero +[ +0.000000] [T1] new +[ +1.000000] [T2] two +[ +1.000000] [T3] three +[Fri Feb 13 23:31:30 2009] [T0] line zero +[Fri Feb 13 23:31:31 2009] [T1] new +[Fri Feb 13 23:31:32 2009] [T2] two +[Fri Feb 13 23:31:33 2009] [T3] three +2009-02-13T23:31:30,123456+00:00 [T0] line zero +2009-02-13T23:31:31,123456+00:00 [T1] new +2009-02-13T23:31:32,123456+00:00 [T2] two +2009-02-13T23:31:33,123456+00:00 [T3] three
tests/expected/dmesg/cid-kmsg-json new file +295 -0
295 lines not on this page · raw message
tests/expected/dmesg/cid-kmsg-limit new file +31 -0
@@ -0,0 +1,31 @@ +[ 0.201607] [T12] smp: Bringing up secondary CPUs ... +[ 0.201607] [T13] smpboot: x86: Booting SMP configuration: +[ 0.209670] [T14] #1 #3 #5 #7 +[ 0.212630] [T15] smp: Brought up 1 node, 16 CPUs +[ 0.215936] [T16] audit: type=2000 audit(1702926179.015:1): state=initialized audit_enabled=0 res=1 +[ 0.215937] [T17] thermal_sys: Registered thermal governor 'fair_share' +[ 0.215966] [T18] ENERGY_PERF_BIAS: Set to 'normal', was 'performance' +[ 0.367657] [T19] ACPI: \_SB_.PCI0.GP19.NHI1.PWRS: New power resource +[ 0.368615] [T20] ACPI: \_SB_.PCI0.GP19.XHC4.PWRS: New power resource +[ 0.376316] [T21] ACPI: \_SB_.PRWL: New power resource +[ 0.376343] [T22] ACPI: \_SB_.PRWB: New power resource +[ 0.377373] [T23] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) +[ 0.377378] [T24] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] +[ 0.377569] [T25] acpi PNP0A08:00: _OSC: platform does not support [SHPCHotplug AER] +[ 0.377933] [T26] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME PCIeCapability LTR DPC] +[ 0.378458] [T27] PCI host bridge to bus 0000:00 +[ 0.378459] [T28] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] +[ 0.378461] [T29] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] +[ 9.398562] [T30] user network daemon initialization complete +[ 10.441520] [T31] systemd[1]: systemd 254.7-1.fc39 running in system mode +[ 11.441524] [T32] systemd[1]: Detected architecture x86-64. +[ 12.441525] [T33] systemd[1]: Running in initrd. +[ 13.541598] [T34] systemd[1]: Hostname set to <catalina>. +[ 15.641860] [T35] usb 3-3: New USB device found, idVendor=1a40, idProduct=0101, bcdDevice= 1.11 +[ 16.690000] [T36] Serial bus multi instantiate pseudo device driver INT3515:00: error -ENXIO: IRQ index 1 not found. +[ 17.710000] [T37] snd_hda_intel 0000:00:1f.3: CORB reset timeout#2, CORBRP = 65535 +[ 18.720000] [T38] systemd-journald[723]: Received client request to flush runtime journal. +[ 20.740000] [T39] systemd-journald[723]: File /var/log/journal/a124ea923b144109a12d557d5ac53179/system.journal corrupted or uncleanly shut down, renaming and replacing. +[ 21.752348] [T40] systemd-journald[723]: /var/log/journal/ad7a2547ac0e4342a342e62a34a3eae4/user-1000.journal: Journal file uses a different sequence number ID, rotating. +[ 24.761100] [T41] PEFILE: Unsigned PE binary +[ 35.768091] [T42] snd_hda_intel 0000:00:1f.3: CORB reset timeout#2, CORBRP = 65535
tests/expected/dmesg/kmsg-file +109 -17
164 lines not on this page · raw message
tests/ts/dmesg/cid-input new file +106 -0
106 lines not on this page · raw message
tests/ts/dmesg/cid-json new file +28 -0
@@ -0,0 +1,28 @@ +#!/bin/bash + +# This file is part of util-linux. +# +# This file is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This file is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +TS_TOPDIR="${0%/*}/../.." +TS_DESC="cid-json" + +. "$TS_TOPDIR"/functions.sh +ts_init "$*" + +ts_check_test_command "$TS_HELPER_DMESG" + +export TZ="GMT" +export DMESG_TEST_BOOTIME="1234567890.123456" + +$TS_HELPER_DMESG -J -F $TS_SELF/cid-input >> $TS_OUTPUT 2>/dev/null + +ts_finalize
tests/ts/dmesg/cid-kmsg-colors new file +34 -0
@@ -0,0 +1,34 @@ +#!/bin/bash + +# This file is part of util-linux. +# +# This file is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This file is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +TS_TOPDIR="${0%/*}/../.." +TS_DESC="cid-kmsg-colors" + +. "$TS_TOPDIR"/functions.sh +ts_init "$*" + +ts_check_test_command "$TS_HELPER_DMESG" +ts_inhibit_custom_colorscheme + +export TZ="GMT" +export DMESG_TEST_BOOTIME="1234567890.123456" + +$TS_HELPER_DMESG --color=always -K $TS_SELF/cid-kmsg-input -x >> $TS_OUTPUT 2>/dev/null + +# dmesg kmsg output can have varying number of caller_id padding spaces +# normalize caller_id padding by removing leading spaces +sed -i 's/\[ *T/\[T/g' $TS_OUTPUT +sed -i 's/\[ *C/\[C/g' $TS_OUTPUT + +ts_finalize
tests/ts/dmesg/cid-kmsg-console-levels new file +49 -0
49 lines not on this page · raw message
tests/ts/dmesg/cid-kmsg-decode new file +33 -0
@@ -0,0 +1,33 @@ +#!/bin/bash + +# This file is part of util-linux. +# +# This file is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This file is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +TS_TOPDIR="${0%/*}/../.." +TS_DESC="cid-kmsg-decode" + +. "$TS_TOPDIR"/functions.sh +ts_init "$*" + +ts_check_test_command "$TS_HELPER_DMESG" + +export TZ="GMT" +export DMESG_TEST_BOOTIME="1234567890.123456" + +$TS_HELPER_DMESG -x -K $TS_SELF/cid-kmsg-input >> $TS_OUTPUT 2>/dev/null + +# dmesg kmsg output can have varying number of caller_id padding spaces +# normalize caller_id padding by removing leading spaces +sed -i 's/\[ *T/\[T/g' $TS_OUTPUT +sed -i 's/\[ *C/\[C/g' $TS_OUTPUT + +ts_finalize
tests/ts/dmesg/cid-kmsg-delta new file +33 -0
@@ -0,0 +1,33 @@ +#!/bin/bash + +# This file is part of util-linux. +# +# This file is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This file is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +TS_TOPDIR="${0%/*}/../.." +TS_DESC="cid-kmsg-delta" + +. "$TS_TOPDIR"/functions.sh +ts_init "$*" + +ts_check_test_command "$TS_HELPER_DMESG" + +export TZ="GMT" +export DMESG_TEST_BOOTIME="1234567890.123456" + +$TS_HELPER_DMESG -d -K $TS_SELF/cid-kmsg-input >> $TS_OUTPUT 2>/dev/null + +# dmesg kmsg output can have varying number of caller_id padding spaces +# normalize caller_id padding by removing leading spaces +sed -i 's/\[ *T/\[T/g' $TS_OUTPUT +sed -i 's/\[ *C/\[C/g' $TS_OUTPUT + +ts_finalize
tests/ts/dmesg/cid-kmsg-facilities new file +36 -0
@@ -0,0 +1,36 @@ +#!/bin/bash + +# This file is part of util-linux. +# +# This file is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This file is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +TS_TOPDIR="${0%/*}/../.." +TS_DESC="cid-kmsg-facilities" + +. "$TS_TOPDIR"/functions.sh +ts_init "$*" + +ts_check_test_command "$TS_HELPER_DMESG" + +export TZ="GMT" +export DMESG_TEST_BOOTIME="1234567890.123456" + +for I in {-1..12}; do + echo "Display facility list: $I" >> $TS_OUTPUT + $TS_HELPER_DMESG -K $TS_SELF/cid-kmsg-input -f $I -x >> $TS_OUTPUT 2>/dev/null +done + +# dmesg kmsg output can have varying number of caller_id padding spaces +# normalize caller_id padding by removing leading spaces +sed -i 's/\[ *T/\[T/g' $TS_OUTPUT +sed -i 's/\[ *C/\[C/g' $TS_OUTPUT + +ts_finalize
tests/ts/dmesg/cid-kmsg-indentation new file +45 -0
45 lines not on this page · raw message
tests/ts/dmesg/cid-kmsg-input new file binary +0 -0
tests/ts/dmesg/cid-kmsg-json new file +28 -0
@@ -0,0 +1,28 @@ +#!/bin/bash + +# This file is part of util-linux. +# +# This file is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This file is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +TS_TOPDIR="${0%/*}/../.." +TS_DESC="cid-kmsg-json" + +. "$TS_TOPDIR"/functions.sh +ts_init "$*" + +ts_check_test_command "$TS_HELPER_DMESG" + +export TZ="GMT" +export DMESG_TEST_BOOTIME="1234567890.123456" + +$TS_HELPER_DMESG -J -K $TS_SELF/cid-kmsg-input >> $TS_OUTPUT 2>/dev/null + +ts_finalize
tests/ts/dmesg/cid-kmsg-limit new file +34 -0
@@ -0,0 +1,34 @@ +#!/bin/bash + +# This file is part of util-linux. +# +# This file is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This file is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +TS_TOPDIR="${0%/*}/../.." +TS_DESC="cid-kmsg-limit" + +. "$TS_TOPDIR"/functions.sh +ts_init "$*" + +ts_check_test_command "$TS_HELPER_DMESG" + +export TZ="GMT" +export DMESG_TEST_BOOTIME="1234567890.123456" + +$TS_HELPER_DMESG --since @1234567890.124 --until @1234567991 -K $TS_SELF/cid-kmsg-input \ + >> $TS_OUTPUT 2> $TS_ERRLOG + +# dmesg kmsg output can have varying number of caller_id padding spaces +# normalize caller_id padding by removing leading spaces +sed -i 's/\[ *T/\[T/g' $TS_OUTPUT +sed -i 's/\[ *C/\[C/g' $TS_OUTPUT + +ts_finalize
tests/ts/dmesg/cid-kmsg-newlines new file binary +0 -0
tests/ts/dmesg/kmsg-input binary +0 -0
Signature · 1 line
2.43.0