Skip to contentSkip to conversation

Run as user, and python3 support

PATCH New Kieran Bingham 1 participant 4 messages Sep 17, 2020

lore
Share & tools
lore
#0 lore raw

v2 cover Run as user, and python3 support

My development target now runs as a user, rather than root and fails to obtain timestamps through the /proc/timer_list which requires root access, and generates a warning on every log output.

Whilst not fatal, this can be fixed by using a c-based implementation to read the monotonic timestamps without parsing proc manually with awk.

This has the extra advantage of not spawning extra processes for every line that is logged, and simplifies the timestamp handling.

Furthermore, python2 is no longer available on my platforms so the gen-lut.py script is updated to run as python3.

I am aware that there is a second python2 tool in this repository, for converting histograms, however I have not yet identified what files this process so I have not completed the python3 migration for that file yet.

If someone wants to complete this, I have the initial conversion, or alternatively - if someone has a set of appropriate histograms to give me I can update and validate the tool myself.

Version two updated with comments from Laurent, and collects his tags.

--
Regards

Kieran

Kieran Bingham (3):
gen-lut: Update for python3
src: monotonic-ts: Monotonic timestamp logging
scripts/logger: Use new monotonic-ts tool

 data/frames/gen-lut.py | 18 +++++++++---------
 scripts/logger.sh      | 20 ++++----------------
 src/Makefile           |  9 +++++++--
 src/monotonic-ts.c     | 36 ++++++++++++++++++++++++++++++++++++
 4 files changed, 56 insertions(+), 27 deletions(-)
 create mode 100644 src/monotonic-ts.c
Signature · 1 line
2.25.1
#1 lore raw

PATCH v2 1/3 gen-lut: Update for python3

Python2 has now gone end-of-life and is discontinued.

Update the gen-lut utility to use python3 directly, converting xrange usages to range, and using bytearray to store the tables and write them directly removing the discontinued file object.

---
 data/frames/gen-lut.py | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
data/frames/gen-lut.py +9 -9
index 07889b11f4ac..335b9f1613bc 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 # SPDX-License-Identifier: GPL-2.0-or-later
 # SPDX-FileCopyrightText: 2016 Renesas Electronics Corporation
 
@@ -49,26 +49,26 @@ def clu_value(x, y, z, scale, a, freq, weights):
 	return (z, y, x, 0)
 
 def generate_clu(config):
-	clu = []
+	clu = bytearray()
 
-	for z in xrange(17):
-		for y in xrange(17):
-			for x in xrange(17):
+	for z in range(17):
+		for y in range(17):
+			for x in range(17):
 				clu.extend(clu_value(x, y, z, **config[1]))
 
-	file('clu-%s.bin' % config[0], 'wb').write(''.join([chr(c) for c in clu]))
+	open('clu-%s.bin' % config[0], 'wb').write(clu)
 
 
 def gamma(vin, gamma, scale):
 	return int(255 * scale * math.pow(vin / 255., gamma))
 
 def generate_lut(config):
-	lut = []
-	for i in xrange(256):
+	lut = bytearray()
+	for i in range(256):
 		lut.extend([gamma(i, g, config[1]) for g in config[2:]])
 		lut.append(0)
 
-	file('lut-%s.bin' % config[0], 'wb').write(''.join([chr(c) for c in lut]))
+	open('lut-%s.bin' % config[0], 'wb').write(lut)
 
 
 def main(argv):
Signature · 1 line
2.25.1
#2 lore raw

PATCH v2 2/3 src: monotonic-ts: Monotonic timestamp logging

In reply to Kieran Bingham

Introduce a new utility which prefixes a monotonic timestamp rendered in the same format as the kernel logs to all lines fed in through stdin.

---
v2:
- Clean up Makefile
- Use separate line for each installed binary
- Remove blank line from c file

 src/Makefile       |  9 +++++++--
 src/monotonic-ts.c | 36 ++++++++++++++++++++++++++++++++++++
 2 files changed, 43 insertions(+), 2 deletions(-)
 create mode 100644 src/monotonic-ts.c
src/Makefile +7 -2
index d7f901f58be6..c40e95afca50 100644
@@ -7,18 +7,23 @@ CFLAGS	?= -O0 -g -W -Wall -Wno-unused-parameter -Iinclude
 LDFLAGS	?=
 LIBS	:= -lm
 GEN-IMAGE := gen-image
+MONOTONIC-TS := monotonic-ts
 
 %.o : %.c
 	$(CC) $(CFLAGS) -c -o $@ $<
 
-all: $(GEN-IMAGE)
+all: $(GEN-IMAGE) $(MONOTONIC-TS)
 
 $(GEN-IMAGE): gen-image.o
 	$(CC) $(LDFLAGS) -o $@ $^ $(LIBS)
 
+$(MONOTONIC-TS): monotonic-ts.o
+	$(CC) $(LDFLAGS) -o $@ $^ $(LIBS)
+
 clean:
 	-rm -f *.o
-	-rm -f $(GEN-IMAGE)
+	-rm -f $(GEN-IMAGE) $(MONOTONIC_TS)
 
 install:
 	cp $(GEN-IMAGE) $(INSTALL_DIR)/
+	cp $(MONOTONIC-TS) $(INSTALL_DIR)/
src/monotonic-ts.c new file +36 -0
new file mode 100644
index 000000000000..69a1809de1d1
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/* SPDX-FileCopyrightText: 2020 Kieran Bingham <[email protected]> */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <time.h>
+
+int main(int argc, char ** argv)
+{
+	struct timespec tp;
+	char *line = NULL;
+	size_t size = 0;
+	const char *label = "";
+
+	if (argc > 1)
+		label = argv[1];
+
+	/*
+	 * Explicitly set line buffering on stdin to be sure it is delivered
+	 * in a timely fashion for our timestamping purposes when data is fed
+	 * through a pipe.
+	 */
+	setlinebuf(stdin);
+
+	do {
+		if (getline(&line, &size, stdin) <= 0)
+			break;
+
+		clock_gettime(CLOCK_MONOTONIC, &tp);
+		printf("[%ld.%.9ld]%s %s", tp.tv_sec, tp.tv_nsec, label, line);
+	} while (!feof(stdin));
+
+	free(line);
+
+	return 0;
+}
Signature · 1 line
2.25.1
#3 lore raw

PATCH v2 3/3 scripts/logger: Use new monotonic-ts tool

In reply to Kieran Bingham

Utilise the new monotonic timestamping tool to remove the manual parsing of timestamps via /proc/timer_list which can only be read by root.

This also simplifies the processing required and contains all timestamping actions within a single process space.

---
v2:
- Fix shellcheck issues

 scripts/logger.sh | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)
scripts/logger.sh +4 -16
index 97e1f582da2b..75426a51961a 100755
@@ -2,23 +2,11 @@
 # SPDX-License-Identifier: GPL-2.0-or-later
 # SPDX-FileCopyrightText: 2016 Renesas Electronics Corporation
 
-now() {
-	awk '/^now/ {time=$3; printf("[%u.%06u]", time / 1000000000, (time % 1000000000) / 1000) ; exit}' /proc/timer_list
-}
-
 label=${1:+ [$1]}
 
 TRACE_MARKER=/sys/kernel/debug/tracing/trace_marker
-if [ -e $TRACE_MARKER ]; then
-	extra_log_files=$TRACE_MARKER
+if [ -e $TRACE_MARKER ] && [ "$(id -u)" = 0 ]; then
+	./monotonic-ts "$label" | tee -a $TRACE_MARKER
+else
+	./monotonic-ts "$label"
 fi
-
-while read line ; do
-	newline="$(now)$label $line"
-
-	echo "$newline"
-
-	for f in $extra_log_files; do
-		echo "$newline" >> $f;
-	done;
-done
Signature · 1 line
2.25.1