Skip to contentSkip to conversation

drm: add kernel-log renderer

RFC In review David Herrmann 3 participants 7 messages Mar 6, 2014 – Mar 7, 2014

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

RFC 1/1 drm: add kernel-log renderer

On modern linux user-space, the VT subsystem is no longer needed for system consoles. Although most DEs will fail if VTs are disabled, there are several gfx-systems that support this mode. Especially the lower system stack has been extended to work without VTs.

However, there is one major drawback if VTs are disabled: You don't get oops/panic-screens nor early boot-debugging. The VT subsystem registers a console-driver, thus displays the kernel log and oops/panic screens in those situations. This patch introduces a fallback for CONFIG_VT=n.

A new DRM-Log core is added. At its heart, DRM-Log maintains a log-buffer of kernel-log messages. It registers a console-driver and pushes new messages into this buffer whenever they appear. The size of the log-buffer can be changed via drm_log_ensure_size(). Initially, a suitable buffer is chosen, but whenever drivers register high-res CRTCs, they ought to increase that buffer to guarantee there's always enough data to render the whole screen.
This log-buffer is managed at the character-level, not pixel-level. It is shared across all users, supports parallel, atomic readers and writers and supports seamless resizing.

Additionally to the log-buffer handling, a generic software renderer is introduced. drm_log_draw() renders the current log-buffer into any memory-mapped framebuffer you want. Note that it supports splitting lines in case your framebuffer is smaller than the log-buffer, but also merging continuation lines in case your framebuffer is wider. This allows rendering an 80x25 log-buffer in full-width on high-res displays. On the other hand, 800x250 log-buffers can be rendered without information loss (apart from a shorter backlog) on low-res displays.

No high-level API is introduced, yet. So for now drivers need to decide when and where to render on their own. A generic interface with panic-handlers, scheduled redrawing and automatic modesetting will be added in follow-ups.

A few more notes:
- This is for *debugging*, and for debugging only! You can use this renderer to get a kernel-log onto your framebuffers. However, the renderer is horribly slow! There is no intention to speed it up. No-one cares whether it takes 500ms to render your panic-screen!
- If you enable the log-renderer for a continous boot-log, you're supposed to redraw on vblanks. The renderer does not support damage-tracking as each new-line causes a scroll-up, anyway.
- The renderer works bottom-up and right-to-left. This is required to avoid pre-calculating log sizes.
- The renderer supports multiple columns. This is quite handy on high-res screens where you can get bigger backlogs this way. Usually 80% of the screen are black. Now you can reuse this for the next log-column.
- I have some wrappers to use this in user-space. I did several tests with resizing and different buffer-sizes and valgrind didn't complain. Results looked all fine.
- The renderer supports *any* RGB target, from 8bit to 32bit with big-endian and little-endian support. The related pixel-renderer will probably never win a beauty-contest, but it works.. Again, who cares for debug-log rendering speed?
- If you want a normal boot-log, use a user-space log-renderer (like experimential systemd-er (no merged, yet)). They can render with proper speed, can do hw-scrolling and more.
Or: Enable VTs! Seriously, if you want all the VT features, use VTs! I will not tell you to disable them.

On a last note:
With this (and the high-level integration) in place, you can easily disable CONFIG_FB, CONFIG_VT and CONFIG_FRAMEBUFFER_CONSOLE. Combined with a proper user-space system-console, you will end up with more features, less bugs and definitely some happy kernel maintainers.

---
 drivers/gpu/drm/Kconfig   |    6 +
 drivers/gpu/drm/Makefile  |    1 +
 drivers/gpu/drm/drm_drv.c |    4 +
 drivers/gpu/drm/drm_log.c | 1188 +++++++++++++++++++++++++++++++++++++++++++++
 include/drm/drm_log.h     |   62 +++
 5 files changed, 1261 insertions(+)
 create mode 100644 drivers/gpu/drm/drm_log.c
 create mode 100644 include/drm/drm_log.h
drivers/gpu/drm/Kconfig +6 -0
index 8e7fa4d..4d9b7fa 100644
@@ -45,6 +45,12 @@ config DRM_KMS_FB_HELPER
 	help
 	  FBDEV helpers for KMS drivers.
 
+config DRM_LOG
+	bool
+	depends on DRM && !VT
+	select FONT_SUPPORT
+	default y
+
 config DRM_LOAD_EDID_FIRMWARE
 	bool "Allow to specify an EDID data set instead of probing for it"
 	depends on DRM_KMS_HELPER
drivers/gpu/drm/Makefile +1 -0
index 292a79d..9c97b58 100644
@@ -19,6 +19,7 @@ drm-$(CONFIG_COMPAT) += drm_ioc32.o
 drm-$(CONFIG_DRM_GEM_CMA_HELPER) += drm_gem_cma_helper.o
 drm-$(CONFIG_PCI) += ati_pcigart.o
 drm-$(CONFIG_DRM_PANEL) += drm_panel.o
+drm-$(CONFIG_DRM_LOG) += drm_log.o
 
 drm-usb-y   := drm_usb.o
 
drivers/gpu/drm/drm_drv.c +4 -0
index 345be03..310f99d 100644
@@ -51,6 +51,7 @@
 #include <linux/export.h>
 #include <drm/drmP.h>
 #include <drm/drm_core.h>
+#include <drm/drm_log.h>
 
 
 static int drm_version(struct drm_device *dev, void *data,
@@ -185,6 +186,7 @@ static int __init drm_core_init(void)
 	drm_global_init();
 	drm_connector_ida_init();
 	idr_init(&drm_minors_idr);
+	drm_log_init();
 
 	if (register_chrdev(DRM_MAJOR, "drm", &drm_stub_fops))
 		goto err_p1;
@@ -213,6 +215,7 @@ err_p2:
 
 	idr_destroy(&drm_minors_idr);
 err_p1:
+	drm_log_exit();
 	return ret;
 }
 
@@ -223,6 +226,7 @@ static void __exit drm_core_exit(void)
 
 	unregister_chrdev(DRM_MAJOR, "drm");
 
+	drm_log_exit();
 	drm_connector_ida_destroy();
 	idr_destroy(&drm_minors_idr);
 }
drivers/gpu/drm/drm_log.c new file +1188 -0

1,188 lines not on this page · raw message

include/drm/drm_log.h new file +62 -0

62 lines not on this page · raw message

Signature · 1 line
1.9.0
#1 lore raw

Hi David,

Quoted 10 lines · On Thu, 06 March 2014 David Herrmann <[email protected]> wrote:

On modern linux user-space, the VT subsystem is no longer needed for system consoles. Although most DEs will fail if VTs are disabled, there are several gfx-systems that support this mode. Especially the lower system stack has been extended to work without VTs.

However, there is one major drawback if VTs are disabled: You don't get oops/panic-screens nor early boot-debugging. The VT subsystem registers a console-driver, thus displays the kernel log and oops/panic screens in those situations. This patch introduces a fallback for CONFIG_VT=n.

Quoted 10 lines

A new DRM-Log core is added. At its heart, DRM-Log maintains a log-buffer of kernel-log messages. It registers a console-driver and pushes new messages into this buffer whenever they appear. The size of the log-buffer can be changed via drm_log_ensure_size(). Initially, a suitable buffer is chosen, but whenever drivers register high-res CRTCs, they ought to increase that buffer to guarantee there's always enough data to render the whole screen.
This log-buffer is managed at the character-level, not pixel-level. It is shared across all users, supports parallel, atomic readers and writers and supports seamless resizing.

Does it make sense to express drm_log_ensure_size() with pixel arguments and have the buffer below operated in characters? Shouldn't the pixel->char calculation be done at a higher level? (see also below regarding high-DPI)

Quoted 8 lines

Additionally to the log-buffer handling, a generic software renderer is introduced. drm_log_draw() renders the current log-buffer into any memory-mapped framebuffer you want. Note that it supports splitting lines in case your framebuffer is smaller than the log-buffer, but also merging continuation lines in case your framebuffer is wider. This allows rendering an 80x25 log-buffer in full-width on high-res displays. On the other hand, 800x250 log-buffers can be rendered without information loss (apart from a shorter backlog) on low-res displays.

Would it be possible to pass the dlog_font to drm_log_draw() so that DRM driver or high-level code could choose a larger font on high-DPI monitors (think retina-like)?

Without that option kernel would need to be built specifically for a given DPI range and not be able to provide readable debug output on multiple CRTCs with (very) different DPIs.

Bruno

Quoted 36 lines

No high-level API is introduced, yet. So for now drivers need to decide when and where to render on their own. A generic interface with panic-handlers, scheduled redrawing and automatic modesetting will be added in follow-ups.

A few more notes:
- This is for *debugging*, and for debugging only! You can use this renderer to get a kernel-log onto your framebuffers. However, the renderer is horribly slow! There is no intention to speed it up. No-one cares whether it takes 500ms to render your panic-screen!
- If you enable the log-renderer for a continous boot-log, you're supposed to redraw on vblanks. The renderer does not support damage-tracking as each new-line causes a scroll-up, anyway.
- The renderer works bottom-up and right-to-left. This is required to avoid pre-calculating log sizes.
- The renderer supports multiple columns. This is quite handy on high-res screens where you can get bigger backlogs this way. Usually 80% of the screen are black. Now you can reuse this for the next log-column.
- I have some wrappers to use this in user-space. I did several tests with resizing and different buffer-sizes and valgrind didn't complain. Results looked all fine.
- The renderer supports *any* RGB target, from 8bit to 32bit with big-endian and little-endian support. The related pixel-renderer will probably never win a beauty-contest, but it works.. Again, who cares for debug-log rendering speed?
- If you want a normal boot-log, use a user-space log-renderer (like experimential systemd-er (no merged, yet)). They can render with proper speed, can do hw-scrolling and more.
Or: Enable VTs! Seriously, if you want all the VT features, use VTs! I will not tell you to disable them.

On a last note:
With this (and the high-level integration) in place, you can easily disable CONFIG_FB, CONFIG_VT and CONFIG_FRAMEBUFFER_CONSOLE. Combined with a proper user-space system-console, you will end up with more features, less bugs and definitely some happy kernel maintainers.

#2 lore raw

Hi

Quoted 24 lines · Bruno Prémont

Hi David,

Quoted 10 lines · On Thu, 06 March 2014 David Herrmann <[email protected]> wrote:

On modern linux user-space, the VT subsystem is no longer needed for system consoles. Although most DEs will fail if VTs are disabled, there are several gfx-systems that support this mode. Especially the lower system stack has been extended to work without VTs.

However, there is one major drawback if VTs are disabled: You don't get oops/panic-screens nor early boot-debugging. The VT subsystem registers a console-driver, thus displays the kernel log and oops/panic screens in those situations. This patch introduces a fallback for CONFIG_VT=n.

Quoted 10 lines

A new DRM-Log core is added. At its heart, DRM-Log maintains a log-buffer of kernel-log messages. It registers a console-driver and pushes new messages into this buffer whenever they appear. The size of the log-buffer can be changed via drm_log_ensure_size(). Initially, a suitable buffer is chosen, but whenever drivers register high-res CRTCs, they ought to increase that buffer to guarantee there's always enough data to render the whole screen.
This log-buffer is managed at the character-level, not pixel-level. It is shared across all users, supports parallel, atomic readers and writers and supports seamless resizing.

Quoted 3 lines

Does it make sense to express drm_log_ensure_size() with pixel arguments and have the buffer below operated in characters? Shouldn't the pixel->char calculation be done at a higher level? (see also below regarding high-DPI)

I don't want to expose "character" sizes. Any external API should be dealing with pixels and only pixels. There is no reason to let them deal with glyphs and text.. Regarding DPI, see below.

Quoted 9 lines
Quoted 8 lines

Additionally to the log-buffer handling, a generic software renderer is introduced. drm_log_draw() renders the current log-buffer into any memory-mapped framebuffer you want. Note that it supports splitting lines in case your framebuffer is smaller than the log-buffer, but also merging continuation lines in case your framebuffer is wider. This allows rendering an 80x25 log-buffer in full-width on high-res displays. On the other hand, 800x250 log-buffers can be rendered without information loss (apart from a shorter backlog) on low-res displays.

Quoted 7 lines

Would it be possible to pass the dlog_font to drm_log_draw() so that DRM driver or high-level code could choose a larger font on high-DPI monitors (think retina-like)?

Without that option kernel would need to be built specifically for a given DPI range and not be able to provide readable debug output on multiple CRTCs with (very) different DPIs.

First of all, I won't support any fancy high-DPI features, that's just wrong for debugging stuff. What I can do (and what we do in fallback consoles in user-space) is to add an "dpi" or "scale" argument to drm_log_draw() which does _integer_ scaling of glyphs. This allows you to render glyphs 2x or 3x .. as big as usual.
I don't think it makes sense to modify drm_log_ensure_size(). I mean, the worst that can happen is that the *text*-backlog is twice as big as required. But if you have a high-dpi display, you already require like 10x as much space for each framebuffer than for the entire log-buffer. The log-buffer requires 1 byte per *char*. The frambuffer requires at least 8*16*4=512 bytes per char.

Anyhow, integer-scaling should be fairly easy to get done.

Thanks
David

#3 lore raw

Hi David,

Quoted 48 lines · On Fri, 7 Mar 2014 00:41:05 +0100 David Herrmann wrote:
Quoted 22 lines · Bruno Prémont
Quoted 10 lines · On Thu, 06 March 2014 David Herrmann wrote:

On modern linux user-space, the VT subsystem is no longer needed for system consoles. Although most DEs will fail if VTs are disabled, there are several gfx-systems that support this mode. Especially the lower system stack has been extended to work without VTs.

However, there is one major drawback if VTs are disabled: You don't get oops/panic-screens nor early boot-debugging. The VT subsystem registers a console-driver, thus displays the kernel log and oops/panic screens in those situations. This patch introduces a fallback for CONFIG_VT=n.

Quoted 10 lines

A new DRM-Log core is added. At its heart, DRM-Log maintains a log-buffer of kernel-log messages. It registers a console-driver and pushes new messages into this buffer whenever they appear. The size of the log-buffer can be changed via drm_log_ensure_size(). Initially, a suitable buffer is chosen, but whenever drivers register high-res CRTCs, they ought to increase that buffer to guarantee there's always enough data to render the whole screen.
This log-buffer is managed at the character-level, not pixel-level. It is shared across all users, supports parallel, atomic readers and writers and supports seamless resizing.

Quoted 3 lines

Does it make sense to express drm_log_ensure_size() with pixel arguments and have the buffer below operated in characters? Shouldn't the pixel->char calculation be done at a higher level? (see also below regarding high-DPI)

I don't want to expose "character" sizes. Any external API should be dealing with pixels and only pixels. There is no reason to let them deal with glyphs and text.. Regarding DPI, see below.

Quoted 9 lines
Quoted 8 lines

Additionally to the log-buffer handling, a generic software renderer is introduced. drm_log_draw() renders the current log-buffer into any memory-mapped framebuffer you want. Note that it supports splitting lines in case your framebuffer is smaller than the log-buffer, but also merging continuation lines in case your framebuffer is wider. This allows rendering an 80x25 log-buffer in full-width on high-res displays. On the other hand, 800x250 log-buffers can be rendered without information loss (apart from a shorter backlog) on low-res displays.

Quoted 7 lines

Would it be possible to pass the dlog_font to drm_log_draw() so that DRM driver or high-level code could choose a larger font on high-DPI monitors (think retina-like)?

Without that option kernel would need to be built specifically for a given DPI range and not be able to provide readable debug output on multiple CRTCs with (very) different DPIs.

Quoted 5 lines

First of all, I won't support any fancy high-DPI features, that's just wrong for debugging stuff. What I can do (and what we do in fallback consoles in user-space) is to add an "dpi" or "scale" argument to drm_log_draw() which does _integer_ scaling of glyphs. This allows you to render glyphs 2x or 3x .. as big as usual.

Integer scaling would do as well. Just some way to avoid tiny glyphs on high-DPI monitors (so that it can be applied per-CRTC) is needed.

Quoted 6 lines

I don't think it makes sense to modify drm_log_ensure_size(). I mean, the worst that can happen is that the *text*-backlog is twice as big as required. But if you have a high-dpi display, you already require like 10x as much space for each framebuffer than for the entire log-buffer. The log-buffer requires 1 byte per *char*. The frambuffer requires at least 8*16*4=512 bytes per char.

That's fair, especially as drm_log_ensure_size() is grow-only.

Quoted 1 line

Anyhow, integer-scaling should be fairly easy to get done.

I thought providing the font to drm_log_draw() and express drm_log_ensure_size() in chars - or pass font to it as well - would be easier than scaling.

Bruno

Quoted 2 lines

Thanks
David

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel

#4 lore raw

Hi

Quoted 7 lines
Quoted 6 lines

I don't think it makes sense to modify drm_log_ensure_size(). I mean, the worst that can happen is that the *text*-backlog is twice as big as required. But if you have a high-dpi display, you already require like 10x as much space for each framebuffer than for the entire log-buffer. The log-buffer requires 1 byte per *char*. The frambuffer requires at least 8*16*4=512 bytes per char.

Quoted 7 lines

That's fair, especially as drm_log_ensure_size() is grow-only.

Quoted 1 line

Anyhow, integer-scaling should be fairly easy to get done.

I thought providing the font to drm_log_draw() and express drm_log_ensure_size() in chars - or pass font to it as well - would be easier than scaling.

There are no real high-res fonts in ./lib/fonts/ so I don't think that's an option. Besides, we don't want high-res fonts, we just want bigger fonts so scaling seems fine.

Thanks
David

#5 lore raw
In reply to David Herrmann
Quoted 4 lines

- The renderer supports *any* RGB target, from 8bit to 32bit with big-endian and little-endian support. The related pixel-renderer will probably never win a beauty-contest, but it works.. Again, who cares for debug-log rendering speed?

Debug log writing performance is extremely important when you are using the log for debugging work rather than for panic display. I don't think that is a show stopper because having a logger makes sense and someone can write a better render engine if they care.

Supporting any target is also going to be useful but you do need to support non linear framebuffers. A GMA500 with something like an X display in use for example does not have a linear mapping of the framebuffer memory. I don't believe its exactly unique in this.

What I am more dubious about is tying it to DRM. Yes it uses DRM constants but it doesn't appear functionally to have a single tie to either DRM or even framebuffer.

It's potentially useful in cases where neither framebuffer or DRM are compiled into the system.

Alan

#6 lore raw

Hi

Quoted 9 lines · One Thousand Gnomes
Quoted 4 lines

- The renderer supports *any* RGB target, from 8bit to 32bit with big-endian and little-endian support. The related pixel-renderer will probably never win a beauty-contest, but it works.. Again, who cares for debug-log rendering speed?

Debug log writing performance is extremely important when you are using the log for debugging work rather than for panic display. I don't think that is a show stopper because having a logger makes sense and someone can write a better render engine if they care.

I don't do damage-based rendering (useless if you don't do hw-scrolling), therefore you can easily configure the higher-level API to render in a 50ms (or even 500ms) interval if anything changed. This should be suitable for other use-cases and avoids hogging the CPU like VTs do. Furthermore, you can just avoid rendering until something useful happened that you *want* to see. Last but not least, when something goes wrong, the panic-handler will always print the latest log, so that should contain all useful information.
Anyhow, I need specific use-cases to comment on that.

Quoted 4 lines

Supporting any target is also going to be useful but you do need to support non linear framebuffers. A GMA500 with something like an X display in use for example does not have a linear mapping of the framebuffer memory. I don't believe its exactly unique in this.

Should be fairly easy to do. But as long as you can atomically flip to a linear FB, that's not needed as you can use a separate FB for the render-log (which is also what we currently do for fbdev-fallbacks). But imho most drivers allow you to easily map it linearly via the GTT, so I haven't bothered supported tiled setups. If drivers need that, they can tell me what formats exactly to support and I can add that.

Quoted 6 lines

What I am more dubious about is tying it to DRM. Yes it uses DRM constants but it doesn't appear functionally to have a single tie to either DRM or even framebuffer.

It's potentially useful in cases where neither framebuffer or DRM are compiled into the system.

Using the four-cc constants makes sense, imho. Apart from that, drm_log.c will stay generic as it is now. There is nothing DRM-specific in there. All the higher-level helpers are separate in drm_log_helpers.c (which is not posted here). However, I've spent enough time trying to get the attention of core maintainers for simple fixes, I really don't want to waste my time pinging on feature-patches every 5 days to get any attention. If someone outside of DRM wants to use it, I'd be happy to discuss any code-sharing. Until then, I'd like to keep it here as people are willing to take it through their tree.

Cheers
David