drm/fbdev-generic: Fix framebuffer on big endian devices
Subsystems (2)
PATCH 1/1 drm/fbdev-generic: Fix framebuffer on big endian devices
Starting with kernel 6.7, the framebuffer text console is not working anymore with the virtio-gpu device on s390x hosts. Such big endian fb devices are usinga different pixel ordering than little endian devices, e.g. DRM_FORMAT_BGRX8888 instead of DRM_FORMAT_XRGB8888.
This used to work fine as long as drm_client_buffer_addfb() was still calling drm_mode_addfb() which called drm_driver_legacy_fb_format() internally to get the right format. But drm_client_buffer_addfb() has recently been reworked to call drm_mode_addfb2() instead with the format value that has been passed to it as a parameter (see commit 6ae2ff23aa43 ("drm/client: Convert drm_client_buffer_addfb() to drm_mode_addfb2()").
That format parameter is determined in drm_fbdev_generic_helper_fb_probe() via the drm_mode_legacy_fb_format() function - which only generates formats suitable for little endian devices. So to fix this issue switch to drm_driver_legacy_fb_format() here instead to take the device endianness into consideration.
--- drivers/gpu/drm/drm_fbdev_generic.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/drm_fbdev_generic.c +2 -1
@@ -84,7 +84,8 @@ static int drm_fbdev_generic_helper_fb_probe(struct drm_fb_helper *fb_helper, sizes->surface_width, sizes->surface_height, sizes->surface_bpp); - format =drm_mode_legacy_fb_format (sizes->surface_bpp, sizes->surface_depth); + format =drm_driver_legacy_fb_format (dev, sizes->surface_bpp, + sizes->surface_depth); buffer =drm_client_framebuffer_create (client, sizes->surface_width, sizes->surface_height, format);if (IS_ERR (buffer))
Signature · 1 line
2.45.2
Thomas Huth <[email protected]> writes:
Hello Thomas,
Quoted 30 lines
Starting with kernel 6.7, the framebuffer text console is not working anymore with the virtio-gpu device on s390x hosts. Such big endian fb devices are usinga different pixel ordering than little endian devices, e.g.
DRM_FORMAT_BGRX8888instead ofDRM_FORMAT_XRGB8888.This used to work fine as long as
drm_client_buffer_addfb()was still callingdrm_mode_addfb()which calleddrm_driver_legacy_fb_format()internally to get the right format. Butdrm_client_buffer_addfb()has recently been reworked to calldrm_mode_addfb2()instead with the format value that has been passed to it as a parameter (see commit6ae2ff23aa43("drm/client: Convertdrm_client_buffer_addfb()todrm_mode_addfb2()").That format parameter is determined in
drm_fbdev_generic_helper_fb_probe()via thedrm_mode_legacy_fb_format()function - which only generates formats suitable for little endian devices. So to fix this issue switch todrm_driver_legacy_fb_format()here instead to take the device endianness into consideration.Fixes 6ae2ff23aa43 ("drm/client: Convert drm_client_buffer_addfb() to drm_mode_addfb2()")Signed-off-by Thomas Huth <[email protected]>--- drivers/gpu/drm/drm_fbdev_generic.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/gpu/drm/drm_fbdev_generic.c +0
-0
drivers/gpu/drm/drm_fbdev_generic.c static int drm_fbdev_generic_helper_fb_probe(struct drm_fb_helper *fb_helper, · line 91
Quoted 9 lines
drivers/gpu/drm/drm_fbdev_generic.c +2
-1@@ -84,7 +84,8 @@ static int drm_fbdev_generic_helper_fb_probe(struct drm_fb_helper *fb_helper, sizes->surface_width, sizes->surface_height, sizes->surface_bpp); - format =drm_mode_legacy_fb_format (sizes->surface_bpp, sizes->surface_depth); + format =drm_driver_legacy_fb_format (dev, sizes->surface_bpp, + sizes->surface_depth); buffer =drm_client_framebuffer_create (client, sizes->surface_width, sizes->surface_height, format);if (IS_ERR (buffer))
Indeed. Thanks a lot for the detailed explanation and the patch. Your fix makes sense to me.
Signature · 5 lines
Best regards, Javier Martinez Canillas Core Platforms Red Hat
Hi
Am 27.06.24 um 19:35 schrieb Thomas Huth:
Quoted 18 lines
Starting with kernel 6.7, the framebuffer text console is not working anymore with the virtio-gpu device on s390x hosts. Such big endian fb devices are usinga different pixel ordering than little endian devices, e.g.
DRM_FORMAT_BGRX8888instead ofDRM_FORMAT_XRGB8888.This used to work fine as long as
drm_client_buffer_addfb()was still callingdrm_mode_addfb()which calleddrm_driver_legacy_fb_format()internally to get the right format. Butdrm_client_buffer_addfb()has recently been reworked to calldrm_mode_addfb2()instead with the format value that has been passed to it as a parameter (see commit6ae2ff23aa43("drm/client: Convertdrm_client_buffer_addfb()todrm_mode_addfb2()").That format parameter is determined in
drm_fbdev_generic_helper_fb_probe()via thedrm_mode_legacy_fb_format()function - which only generates formats suitable for little endian devices. So to fix this issue switch todrm_driver_legacy_fb_format()here instead to take the device endianness into consideration.
Quoted 3 lines
Fixes 6ae2ff23aa43 ("drm/client: Convert drm_client_buffer_addfb() to drm_mode_addfb2()")Signed-off-by Thomas Huth <[email protected]>
Quoted 2 lines
--- drivers/gpu/drm/drm_fbdev_generic.c | 3 ++-
This file is now called drm_fbdev_ttm.c in drm-misc-next. And a similar patch might be necessary for drm_fbdev_dma.c. The code in drm_fbdev_shmem.c apparently has it already.
Best regards
Thomas
Quoted 16 lines
1 file changed, 2 insertions(+), 1 deletion(-)drivers/gpu/drm/drm_fbdev_generic.c +2
-1@@ -84,7 +84,8 @@ static int drm_fbdev_generic_helper_fb_probe(struct drm_fb_helper *fb_helper, sizes->surface_width, sizes->surface_height, sizes->surface_bpp); - format =drm_mode_legacy_fb_format (sizes->surface_bpp, sizes->surface_depth); + format =drm_driver_legacy_fb_format (dev, sizes->surface_bpp, + sizes->surface_depth); buffer =drm_client_framebuffer_create (client, sizes->surface_width, sizes->surface_height, format);if (IS_ERR (buffer))
Signature · 7 lines
-- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Frankenstrasse 146, 90461 Nuernberg, Germany GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman HRB 36809 (AG Nuernberg)
Hi Thomas,
Quoted 26 lines · Thomas Zimmermann
Am 27.06.24 um 19:35 schrieb Thomas Huth:
Quoted 18 lines
Starting with kernel 6.7, the framebuffer text console is not working anymore with the virtio-gpu device on s390x hosts. Such big endian fb devices are usinga different pixel ordering than little endian devices, e.g.
DRM_FORMAT_BGRX8888instead ofDRM_FORMAT_XRGB8888.This used to work fine as long as
drm_client_buffer_addfb()was still callingdrm_mode_addfb()which calleddrm_driver_legacy_fb_format()internally to get the right format. Butdrm_client_buffer_addfb()has recently been reworked to calldrm_mode_addfb2()instead with the format value that has been passed to it as a parameter (see commit6ae2ff23aa43("drm/client: Convertdrm_client_buffer_addfb()todrm_mode_addfb2()").That format parameter is determined in
drm_fbdev_generic_helper_fb_probe()via thedrm_mode_legacy_fb_format()function - which only generates formats suitable for little endian devices. So to fix this issue switch todrm_driver_legacy_fb_format()here instead to take the device endianness into consideration.Quoted 3 lines
Fixes 6ae2ff23aa43 ("drm/client: Convert drm_client_buffer_addfb() to drm_mode_addfb2()")Signed-off-by Thomas Huth <[email protected]>
Quoted 6 lines
Quoted 2 lines
--- drivers/gpu/drm/drm_fbdev_generic.c | 3 ++-This file is now called
drm_fbdev_ttm.cin drm-misc-next. And a similar patch might be necessary fordrm_fbdev_dma.c. The code indrm_fbdev_shmem.capparently has it already.
We are getting too many copies of this logic... (yup, had to fix them all up in my WIP support for R1 ;-)
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Quoted 25 lines · Thomas Zimmermann
Hi
Am 27.06.24 um 19:35 schrieb Thomas Huth:
Quoted 18 lines
Starting with kernel 6.7, the framebuffer text console is not working anymore with the virtio-gpu device on s390x hosts. Such big endian fb devices are usinga different pixel ordering than little endian devices, e.g.
DRM_FORMAT_BGRX8888instead ofDRM_FORMAT_XRGB8888.This used to work fine as long as
drm_client_buffer_addfb()was still callingdrm_mode_addfb()which calleddrm_driver_legacy_fb_format()internally to get the right format. Butdrm_client_buffer_addfb()has recently been reworked to calldrm_mode_addfb2()instead with the format value that has been passed to it as a parameter (see commit6ae2ff23aa43("drm/client: Convertdrm_client_buffer_addfb()todrm_mode_addfb2()").That format parameter is determined in
drm_fbdev_generic_helper_fb_probe()via thedrm_mode_legacy_fb_format()function - which only generates formats suitable for little endian devices. So to fix this issue switch todrm_driver_legacy_fb_format()here instead to take the device endianness into consideration.Quoted 3 lines
Fixes 6ae2ff23aa43 ("drm/client: Convert drm_client_buffer_addfb() to drm_mode_addfb2()")Signed-off-by Thomas Huth <[email protected]>
Quoted 7 lines
Quoted 2 lines
---
drivers/gpu/drm/drm_fbdev_generic.c| 3 ++-This file is now called
drm_fbdev_ttm.cin drm-misc-next.
Oh, ok, shall I send a v2 that is adjusted to that change, or can it be fixed while applying my patch?
Quoted 1 line
And a similar patch might be necessary for
drm_fbdev_dma.c.
Looks similar, indeed. Shall I send a patch for that one, too? ... I currently don't have a setup for testing that, though...
Thomas
Quoted 23 lines
The code in
drm_fbdev_shmem.capparently has it already.Best regards
ThomasQuoted 17 lines
1 file changed, 2 insertions(+), 1 deletion(-)drivers/gpu/drm/drm_fbdev_generic.c +2
-1@@ -84,7 +84,8 @@ static int drm_fbdev_generic_helper_fb_probe(structdrm_fb_helper *fb_helper, sizes->surface_width, sizes->surface_height, sizes->surface_bpp); - format = drm_mode_legacy_fb_format(sizes->surface_bpp, sizes->surface_depth); + format = drm_driver_legacy_fb_format(dev, sizes->surface_bpp, + sizes->surface_depth); buffer = drm_client_framebuffer_create(client, sizes->surface_width, sizes->surface_height, format);if (
IS_ERR(buffer))
Hi Thomas,
Quoted 29 lines · Thomas Huth
Quoted 28 lines · Thomas Zimmermann
Am 27.06.24 um 19:35 schrieb Thomas Huth:
Quoted 19 lines
Starting with kernel 6.7, the framebuffer text console is not working anymore with the virtio-gpu device on s390x hosts. Such big endian fb devices are usinga different pixel ordering than little endian devices, e.g.
DRM_FORMAT_BGRX8888instead ofDRM_FORMAT_XRGB8888.This used to work fine as long as
drm_client_buffer_addfb()was still callingdrm_mode_addfb()which calleddrm_driver_legacy_fb_format()internally to get the right format. Butdrm_client_buffer_addfb()has recently been reworked to calldrm_mode_addfb2()instead with the format value that has been passed to it as a parameter (see commit6ae2ff23aa43("drm/client: Convertdrm_client_buffer_addfb()todrm_mode_addfb2()").That format parameter is determined in
drm_fbdev_generic_helper_fb_probe()via thedrm_mode_legacy_fb_format()function - which only generates formats suitable for little endian devices. So to fix this issue switch todrm_driver_legacy_fb_format()here instead to take the device endianness into consideration.Quoted 4 lines
Fixes 6ae2ff23aa43 ("drm/client: Convert drm_client_buffer_addfb() to
drm_mode_addfb2()")Signed-off-by Thomas Huth <[email protected]>
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
--- drivers/gpu/drm/drm_fbdev_generic.c | 3 ++-This file is now called
drm_fbdev_ttm.cin drm-misc-next.Oh, ok, shall I send a v2 that is adjusted to that change, or can it be fixed while applying my patch?
As this is a regression in mainline, which needs to be backported, too, it's best to apply your fix to v6.10-rc6, which does not have drm_fbdev_ttm.c yet.
Quoted 4 lines
Quoted 1 line
And a similar patch might be necessary for
drm_fbdev_dma.c.Looks similar, indeed. Shall I send a patch for that one, too? ... I currently don't have a setup for testing that, though...
Obviously these need to be fixed, too.
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Thomas Zimmermann
maintainerHi
Am 01.07.24 um 10:54 schrieb Geert Uytterhoeven:
Quoted 31 lines
Hi Thomas,
Quoted 28 lines · Thomas Huth
Quoted 27 lines · Thomas Zimmermann
Am 27.06.24 um 19:35 schrieb Thomas Huth:
Quoted 19 lines
Starting with kernel 6.7, the framebuffer text console is not working anymore with the virtio-gpu device on s390x hosts. Such big endian fb devices are usinga different pixel ordering than little endian devices, e.g.
DRM_FORMAT_BGRX8888instead ofDRM_FORMAT_XRGB8888.This used to work fine as long as
drm_client_buffer_addfb()was still callingdrm_mode_addfb()which calleddrm_driver_legacy_fb_format()internally to get the right format. Butdrm_client_buffer_addfb()has recently been reworked to calldrm_mode_addfb2()instead with the format value that has been passed to it as a parameter (see commit6ae2ff23aa43("drm/client: Convertdrm_client_buffer_addfb()todrm_mode_addfb2()").That format parameter is determined in
drm_fbdev_generic_helper_fb_probe()via thedrm_mode_legacy_fb_format()function - which only generates formats suitable for little endian devices. So to fix this issue switch todrm_driver_legacy_fb_format()here instead to take the device endianness into consideration.Quoted 4 lines
Fixes 6ae2ff23aa43 ("drm/client: Convert drm_client_buffer_addfb() to
drm_mode_addfb2()")Signed-off-by Thomas Huth <[email protected]>
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Quoted 2 lines
--- drivers/gpu/drm/drm_fbdev_generic.c | 3 ++-This file is now called
drm_fbdev_ttm.cin drm-misc-next.Oh, ok, shall I send a v2 that is adjusted to that change, or can it be fixed while applying my patch?
As this is a regression in mainline, which needs to be backported, too, it's best to apply your fix to v6.10-rc6, which does not have
drm_fbdev_ttm.cyet.
I guess we can put the current fix into drm-misc-fixes.
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
And a similar patch might be necessary for
drm_fbdev_dma.c.Looks similar, indeed. Shall I send a patch for that one, too? ... I currently don't have a setup for testing that, though...
It's up to you. You'd need drm-misc-next from https://gitlab.freedesktop.org/drm/misc/kernel
Best regards
Thomas
Quoted 6 lines
Obviously these need to be fixed, too.
Gr{oetje,eeting}s,
Geert
Signature · 7 lines
-- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Frankenstrasse 146, 90461 Nuernberg, Germany GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman HRB 36809 (AG Nuernberg)
Hi Thomas,
CC Christian
Quoted 18 lines · Thomas Huth
Starting with kernel 6.7, the framebuffer text console is not working anymore with the virtio-gpu device on s390x hosts. Such big endian fb devices are usinga different pixel ordering than little endian devices, e.g.
DRM_FORMAT_BGRX8888instead ofDRM_FORMAT_XRGB8888.This used to work fine as long as
drm_client_buffer_addfb()was still callingdrm_mode_addfb()which calleddrm_driver_legacy_fb_format()internally to get the right format. Butdrm_client_buffer_addfb()has recently been reworked to calldrm_mode_addfb2()instead with the format value that has been passed to it as a parameter (see commit6ae2ff23aa43("drm/client: Convertdrm_client_buffer_addfb()todrm_mode_addfb2()").That format parameter is determined in
drm_fbdev_generic_helper_fb_probe()via thedrm_mode_legacy_fb_format()function - which only generates formats suitable for little endian devices. So to fix this issue switch todrm_driver_legacy_fb_format()here instead to take the device endianness into consideration.
Quoted 3 lines
Fixes 6ae2ff23aa43 ("drm/client: Convert drm_client_buffer_addfb() to drm_mode_addfb2()")Signed-off-by Thomas Huth <[email protected]>
works fine on m68k-virt, so:
Christian had reported a similar issue before[1].
I submitted a different solution fixing virtio[2] instead, but that caused issues with virtio-mouse-pci cursor...
drivers/gpu/drm/drm_fbdev_generic.c static int drm_fbdev_generic_helper_fb_probe(struct drm_fb_helper *fb_helper, · line 91
Quoted 12 lines
drivers/gpu/drm/drm_fbdev_generic.c +2
-1@@ -84,7 +84,8 @@ static int drm_fbdev_generic_helper_fb_probe(struct drm_fb_helper *fb_helper, sizes->surface_width, sizes->surface_height, sizes->surface_bpp); - format =drm_mode_legacy_fb_format (sizes->surface_bpp, sizes->surface_depth); + format =drm_driver_legacy_fb_format (dev, sizes->surface_bpp, + sizes->surface_depth); buffer =drm_client_framebuffer_create (client, sizes->surface_width, sizes->surface_height, format);if (IS_ERR (buffer))
[1] https://lore.kernel.org/[email protected]/
[2] "[PATCH v2] drm/virtio: Add suppport for non-native buffer formats"
https://lore.kernel.org/47a81d2e0e47b1715718779b6978a8b595cc7c5d.1700140609.git.geert@linux-m68k.org
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Merged into drm-misc-fixes.
Am 27.06.24 um 19:35 schrieb Thomas Huth:
Quoted 39 lines
Starting with kernel 6.7, the framebuffer text console is not working anymore with the virtio-gpu device on s390x hosts. Such big endian fb devices are usinga different pixel ordering than little endian devices, e.g.
DRM_FORMAT_BGRX8888instead ofDRM_FORMAT_XRGB8888.This used to work fine as long as
drm_client_buffer_addfb()was still callingdrm_mode_addfb()which calleddrm_driver_legacy_fb_format()internally to get the right format. Butdrm_client_buffer_addfb()has recently been reworked to calldrm_mode_addfb2()instead with the format value that has been passed to it as a parameter (see commit6ae2ff23aa43("drm/client: Convertdrm_client_buffer_addfb()todrm_mode_addfb2()").That format parameter is determined in
drm_fbdev_generic_helper_fb_probe()via thedrm_mode_legacy_fb_format()function - which only generates formats suitable for little endian devices. So to fix this issue switch todrm_driver_legacy_fb_format()here instead to take the device endianness into consideration.Fixes 6ae2ff23aa43 ("drm/client: Convert drm_client_buffer_addfb() to drm_mode_addfb2()")Signed-off-by Thomas Huth <[email protected]>--- drivers/gpu/drm/drm_fbdev_generic.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/gpu/drm/drm_fbdev_generic.c +2
-1@@ -84,7 +84,8 @@ static int drm_fbdev_generic_helper_fb_probe(struct drm_fb_helper *fb_helper, sizes->surface_width, sizes->surface_height, sizes->surface_bpp); - format =drm_mode_legacy_fb_format (sizes->surface_bpp, sizes->surface_depth); + format =drm_driver_legacy_fb_format (dev, sizes->surface_bpp, + sizes->surface_depth); buffer =drm_client_framebuffer_create (client, sizes->surface_width, sizes->surface_height, format);if (IS_ERR (buffer))
Signature · 7 lines
-- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Frankenstrasse 146, 90461 Nuernberg, Germany GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman HRB 36809 (AG Nuernberg)