Skip to contentSkip to conversation

drm: byteorder fixes

PATCH Superseded Gerd Hoffmann 3 participants 18 messages Sep 5, 2018 – Jan 11, 2019

Subsystems (5)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Gerd Hoffmann

author maintainer
#0 lore raw

v2 cover drm: byteorder fixes

Patch series adds some convinience #defines for host byteoder drm formats. It fixes drm_mode_addfb() behavior on bigendian machines. For bug compatibility reasons a mode_config quirk activates the fix. bochs and virtio-gpu drivers are updated to use the new #defines, set the new driver feature flag and fix some issues.

Gerd Hoffmann (6):
drm: replace DRIVER_PREFER_XBGR_30BPP driver flag with mode_config
quirk
drm: byteorder: add DRM_FORMAT_HOST_*
drm: do not mask out DRM_FORMAT_BIG_ENDIAN
drm: fix drm_mode_addfb() on big endian machines. drm/bochs: fix DRM_FORMAT_* handling for big endian machines. drm/virtio: fix DRM_FORMAT_* handling

 include/drm/drm_drv.h                    |  1 -
 include/drm/drm_fourcc.h                 | 22 +++++++++++++
 include/drm/drm_mode_config.h            | 15 +++++++++
 drivers/gpu/drm/bochs/bochs_fbdev.c      |  5 ++-
 drivers/gpu/drm/bochs/bochs_kms.c        | 34 +++++++++++++++++++-
 drivers/gpu/drm/bochs/bochs_mm.c         |  2 +-
 drivers/gpu/drm/drm_framebuffer.c        | 17 ++++++++--
 drivers/gpu/drm/nouveau/dispnv50/disp.c  |  2 +-
 drivers/gpu/drm/virtio/virtgpu_display.c |  5 +++
 drivers/gpu/drm/virtio/virtgpu_fb.c      |  2 +-
 drivers/gpu/drm/virtio/virtgpu_gem.c     |  7 +++--
 drivers/gpu/drm/virtio/virtgpu_plane.c   | 54 ++------------------------------
 12 files changed, 101 insertions(+), 65 deletions(-)
Signature · 6 lines
2.9.3

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
author maintainer
#1 lore raw

PATCH v2 1/6 drm: replace DRIVER_PREFER_XBGR_30BPP driver flag with mode_config quirk

---
 include/drm/drm_drv.h                   | 1 -
 include/drm/drm_mode_config.h           | 1 +
 drivers/gpu/drm/drm_framebuffer.c       | 4 ++--
 drivers/gpu/drm/nouveau/dispnv50/disp.c | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)
include/drm/drm_drv.h +0 -1
index 46a8009784..23b9678137 100644
@@ -56,7 +56,6 @@ struct drm_printer;
 #define DRIVER_ATOMIC			0x10000
 #define DRIVER_KMS_LEGACY_CONTEXT	0x20000
 #define DRIVER_SYNCOBJ                  0x40000
-#define DRIVER_PREFER_XBGR_30BPP        0x80000
 
 /**
  * struct drm_driver - DRM driver structure
include/drm/drm_mode_config.h +1 -0
index a0b202e1d6..5d29f4ba6f 100644
@@ -809,6 +809,7 @@ struct drm_mode_config {
 
 	/* dumb ioctl parameters */
 	uint32_t preferred_depth, prefer_shadow;
+	bool quirk_addfb_prefer_xbgr_30bpp;
 
 	/**
 	 * @async_page_flip: Does this device support async flips on the primary
drivers/gpu/drm/drm_framebuffer.c +2 -2
index 781af1d42d..17b7b8944d 100644
@@ -120,8 +120,8 @@ int drm_mode_addfb(struct drm_device *dev, struct drm_mode_fb_cmd *or,
 	r.pixel_format = drm_mode_legacy_fb_format(or->bpp, or->depth);
 	r.handles[0] = or->handle;
 
-	if (r.pixel_format == DRM_FORMAT_XRGB2101010 &&
-	    dev->driver->driver_features & DRIVER_PREFER_XBGR_30BPP)
+	if (dev->mode_config.quirk_addfb_prefer_xbgr_30bpp &&
+	    r.pixel_format == DRM_FORMAT_XRGB2101010)
 		r.pixel_format = DRM_FORMAT_XBGR2101010;
 
 	ret = drm_mode_addfb2(dev, &r, file_priv);
drivers/gpu/drm/nouveau/dispnv50/disp.c +1 -1
index 8412119bd9..a9bb656058 100644
@@ -2174,7 +2174,7 @@ nv50_display_create(struct drm_device *dev)
 	nouveau_display(dev)->fini = nv50_display_fini;
 	disp->disp = &nouveau_display(dev)->disp;
 	dev->mode_config.funcs = &nv50_disp_func;
-	dev->driver->driver_features |= DRIVER_PREFER_XBGR_30BPP;
+	dev->mode_config.quirk_addfb_prefer_xbgr_30bpp = true;
 
 	/* small shared memory area we use for notifiers and semaphores */
 	ret = nouveau_bo_new(&drm->client, 4096, 0x1000, TTM_PL_FLAG_VRAM,
Signature · 6 lines
2.9.3

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
author maintainer
#2 lore raw

PATCH v2 2/6 drm: byteorder: add DRM_FORMAT_HOST_*

In reply to Gerd Hoffmann

Add fourcc variants in host byte order. With these at hand we don't need #ifdefs in drivers which support framebuffers in cpu endianess.

---
 include/drm/drm_fourcc.h | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
include/drm/drm_fourcc.h +22 -0
index f9c15845f4..fac831c401 100644
@@ -25,6 +25,28 @@
 #include <linux/types.h>
 #include <uapi/drm/drm_fourcc.h>
 
+/*
+ * DRM formats are little endian.  Define host endian variants for the
+ * most common formats here, to reduce the #ifdefs needed in drivers.
+ *
+ * Note that the DRM_FORMAT_BIG_ENDIAN flag should only be used in
+ * case the format can't be specified otherwise, so we don't end up
+ * with two values describing the same format.
+ */
+#ifdef __BIG_ENDIAN
+# define DRM_FORMAT_HOST_XRGB1555     (DRM_FORMAT_XRGB1555         |	\
+				       DRM_FORMAT_BIG_ENDIAN)
+# define DRM_FORMAT_HOST_RGB565       (DRM_FORMAT_RGB565           |	\
+				       DRM_FORMAT_BIG_ENDIAN)
+# define DRM_FORMAT_HOST_XRGB8888     DRM_FORMAT_BGRX8888
+# define DRM_FORMAT_HOST_ARGB8888     DRM_FORMAT_BGRA8888
+#else
+# define DRM_FORMAT_HOST_XRGB1555     DRM_FORMAT_XRGB1555
+# define DRM_FORMAT_HOST_RGB565       DRM_FORMAT_RGB565
+# define DRM_FORMAT_HOST_XRGB8888     DRM_FORMAT_XRGB8888
+# define DRM_FORMAT_HOST_ARGB8888     DRM_FORMAT_ARGB8888
+#endif
+
 struct drm_device;
 struct drm_mode_fb_cmd2;
 
Signature · 1 line
2.9.3
author maintainer
#3 lore raw

PATCH v2 3/6 drm: do not mask out DRM_FORMAT_BIG_ENDIAN

In reply to Gerd Hoffmann

framebuffer_check() expects that drm_get_format_info() will not fail if the __drm_format_info() call was successful. That'll work only in case both are called with the same pixel_format value, so masking out the DRM_FORMAT_BIG_ENDIAN flag isn't a good idea.

---
 drivers/gpu/drm/drm_framebuffer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/drm_framebuffer.c +1 -1
index 17b7b8944d..888c4d53cf 100644
@@ -164,7 +164,7 @@ static int framebuffer_check(struct drm_device *dev,
 	int i;
 
 	/* check if the format is supported at all */
-	info = __drm_format_info(r->pixel_format & ~DRM_FORMAT_BIG_ENDIAN);
+	info = __drm_format_info(r->pixel_format);
 	if (!info) {
 		struct drm_format_name_buf format_name;
 
Signature · 6 lines
2.9.3

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
author maintainer
#4 lore raw

PATCH v2 4/6 drm: fix drm_mode_addfb() on big endian machines.

In reply to Gerd Hoffmann

Userspace on big endian machhines typically expects the ADDFB ioctl returns a big endian framebuffer. drm_mode_addfb() will call drm_mode_addfb2() unconditionally with little endian DRM_FORMAT_* values though, which is wrong. This patch fixes that.

Drivers (both kernel and xorg) have quirks in place to deal with the broken drm_mode_addfb() behavior. Because of this we can't just change drm_mode_addfb() behavior for everybody without breaking things. Add the quirk_addfb_prefer_host_byte_order field to mode_config, so drivers can opt-in.

---
 include/drm/drm_mode_config.h     | 14 ++++++++++++++
 drivers/gpu/drm/drm_framebuffer.c | 11 +++++++++++
 2 files changed, 25 insertions(+)
include/drm/drm_mode_config.h +14 -0
index 5d29f4ba6f..65020086c9 100644
@@ -811,6 +811,20 @@ struct drm_mode_config {
 	uint32_t preferred_depth, prefer_shadow;
 	bool quirk_addfb_prefer_xbgr_30bpp;
 
+	/*
+	 * @quirk_addfb_prefer_host_byte_order:
+	 *
+	 * When set to true drm_mode_addfb() will pick host byte order
+	 * pixel_format when calling drm_mode_addfb2().  This is how
+	 * drm_mode_addfb() should have worked from day one.  It
+	 * didn't though, so we ended up with quirks in both kernel
+	 * and userspace drivers to deal with the broken behavior.
+	 * Simply fixing drm_mode_addfb() unconditionally would break
+	 * these drivers, so add a quirk bit here to allow drivers
+	 * opt-in.
+	 */
+	bool quirk_addfb_prefer_host_byte_order;
+
 	/**
 	 * @async_page_flip: Does this device support async flips on the primary
 	 * plane?
drivers/gpu/drm/drm_framebuffer.c +11 -0
index 888c4d53cf..f863f8a20f 100644
@@ -124,6 +124,17 @@ int drm_mode_addfb(struct drm_device *dev, struct drm_mode_fb_cmd *or,
 	    r.pixel_format == DRM_FORMAT_XRGB2101010)
 		r.pixel_format = DRM_FORMAT_XBGR2101010;
 
+	if (dev->mode_config.quirk_addfb_prefer_host_byte_order) {
+		if (r.pixel_format == DRM_FORMAT_XRGB8888)
+			r.pixel_format = DRM_FORMAT_HOST_XRGB8888;
+		if (r.pixel_format == DRM_FORMAT_ARGB8888)
+			r.pixel_format = DRM_FORMAT_HOST_ARGB8888;
+		if (r.pixel_format == DRM_FORMAT_RGB565)
+			r.pixel_format = DRM_FORMAT_HOST_RGB565;
+		if (r.pixel_format == DRM_FORMAT_XRGB1555)
+			r.pixel_format = DRM_FORMAT_HOST_XRGB1555;
+	}
+
 	ret = drm_mode_addfb2(dev, &r, file_priv);
 	if (ret)
 		return ret;
Signature · 6 lines
2.9.3

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Reviewed-by on v2 4/6
#5 lore raw

Re: [PATCH v2 4/6] drm: fix drm_mode_addfb() on big endian machines.

Quoted 22 lines · Gerd Hoffmann

Userspace on big endian machhines typically expects the ADDFB ioctl returns a big endian framebuffer. drm_mode_addfb() will call drm_mode_addfb2() unconditionally with little endian DRM_FORMAT_* values though, which is wrong. This patch fixes that.

Drivers (both kernel and xorg) have quirks in place to deal with the broken drm_mode_addfb() behavior. Because of this we can't just change drm_mode_addfb() behavior for everybody without breaking things. Add the quirk_addfb_prefer_host_byte_order field to mode_config, so drivers can opt-in.

---
 include/drm/drm_mode_config.h     | 14 ++++++++++++++
 drivers/gpu/drm/drm_framebuffer.c | 11 +++++++++++
 2 files changed, 25 insertions(+)
include/drm/drm_mode_config.h +0 -0
index 5d29f4ba6f..65020086c9 100644

include/drm/drm_mode_config.h struct drm_mode_config { · line 814

Quoted 4 lines
include/drm/drm_mode_config.h +1 -0
@@ -811,6 +811,20 @@ struct drm_mode_config {
 	uint32_t preferred_depth, prefer_shadow;
 	bool quirk_addfb_prefer_xbgr_30bpp;
 
+	/*

This doesn't parse, needs /**

$ make htmldocs

to check it.

include/drm/drm_mode_config.h struct drm_mode_config { · line 826

Quoted 12 lines
include/drm/drm_mode_config.h +12 -0
+	 * @quirk_addfb_prefer_host_byte_order:
+	 *
+	 * When set to true drm_mode_addfb() will pick host byte order
+	 * pixel_format when calling drm_mode_addfb2().  This is how
+	 * drm_mode_addfb() should have worked from day one.  It
+	 * didn't though, so we ended up with quirks in both kernel
+	 * and userspace drivers to deal with the broken behavior.
+	 * Simply fixing drm_mode_addfb() unconditionally would break
+	 * these drivers, so add a quirk bit here to allow drivers
+	 * opt-in.
+	 */
+	bool quirk_addfb_prefer_host_byte_order;

I wonder whether we should reject addFB2 on big endian hosts if this isn't set. Otherwise we need addfb3, aka will never be able to fix this. Anyway, that might be good work for a follow-up ...

Up to this patch series (with the kerneldoc fixed):

Quoted 33 lines
include/drm/drm_mode_config.h +1 -0
+
 	/**
 	 * @async_page_flip: Does this device support async flips on the primary
 	 * plane?
drivers/gpu/drm/drm_framebuffer.c +11 -0
index 888c4d53cf..f863f8a20f 100644
@@ -124,6 +124,17 @@ int drm_mode_addfb(struct drm_device *dev, struct drm_mode_fb_cmd *or,
 	    r.pixel_format == DRM_FORMAT_XRGB2101010)
 		r.pixel_format = DRM_FORMAT_XBGR2101010;
 
+	if (dev->mode_config.quirk_addfb_prefer_host_byte_order) {
+		if (r.pixel_format == DRM_FORMAT_XRGB8888)
+			r.pixel_format = DRM_FORMAT_HOST_XRGB8888;
+		if (r.pixel_format == DRM_FORMAT_ARGB8888)
+			r.pixel_format = DRM_FORMAT_HOST_ARGB8888;
+		if (r.pixel_format == DRM_FORMAT_RGB565)
+			r.pixel_format = DRM_FORMAT_HOST_RGB565;
+		if (r.pixel_format == DRM_FORMAT_XRGB1555)
+			r.pixel_format = DRM_FORMAT_HOST_XRGB1555;
+	}
+
 	ret = drm_mode_addfb2(dev, &r, file_priv);
 	if (ret)
 		return ret;
Signature · 6 lines
2.9.3

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
author maintainer
#6 lore raw

Re: [PATCH v2 4/6] drm: fix drm_mode_addfb() on big endian machines.

Hi,

Quoted 5 lines

include/drm/drm_mode_config.h struct drm_mode_config { · line 826

Quoted 1 line
include/drm/drm_mode_config.h +1 -0
+	bool quirk_addfb_prefer_host_byte_order;

I wonder whether we should reject addFB2 on big endian hosts if this isn't set. Otherwise we need addfb3, aka will never be able to fix this. Anyway, that might be good work for a follow-up ...

Sounds sensible, I'll send a patch.

Quoted 3 lines

Up to this patch series (with the kerneldoc fixed):

Fixed, patches 1-4 pushed.

Any takers for 5+6 review? Dave?

thanks,
Gerd

author maintainer
#7 lore raw

PATCH v2 5/6 drm/bochs: fix DRM_FORMAT_* handling for big endian machines.

In reply to Gerd Hoffmann

Use DRM_FORMAT_HOST_XRGB8888, so we are using the correct format code on bigendian machines. Also set the quirk_addfb_prefer_host_byte_order mode_config bit so drm_mode_addfb() asks for the correct format code.

Create our own plane and use drm_crtc_init_with_planes() instead of depending on the default created by drm_crtc_init(). That way the plane format list is correct on bigendian machines.

With this patch applied both ADDFB and ADDFB2 ioctls work correctly in the bochs-drm.ko driver on big endian machines. Without the patch only ADDFB (which still seems to be used by the majority of userspace) works correctly.

---
 drivers/gpu/drm/bochs/bochs_fbdev.c |  5 ++---
 drivers/gpu/drm/bochs/bochs_kms.c   | 34 +++++++++++++++++++++++++++++++++-
 drivers/gpu/drm/bochs/bochs_mm.c    |  2 +-
 3 files changed, 36 insertions(+), 5 deletions(-)
drivers/gpu/drm/bochs/bochs_fbdev.c +2 -3
index 14eb8d0d5a..bf728790fa 100644
@@ -64,9 +64,8 @@ static int bochsfb_create(struct drm_fb_helper *helper,
 
 	mode_cmd.width = sizes->surface_width;
 	mode_cmd.height = sizes->surface_height;
-	mode_cmd.pitches[0] = mode_cmd.width * ((sizes->surface_bpp + 7) / 8);
-	mode_cmd.pixel_format = drm_mode_legacy_fb_format(sizes->surface_bpp,
-							  sizes->surface_depth);
+	mode_cmd.pitches[0] = sizes->surface_width * 4;
+	mode_cmd.pixel_format = DRM_FORMAT_HOST_XRGB8888;
 	size = mode_cmd.pitches[0] * mode_cmd.height;
 
 	/* alloc, pin & map bo */
drivers/gpu/drm/bochs/bochs_kms.c +33 -1
index ca5a9afdd5..b40077d04b 100644
@@ -129,12 +129,43 @@ static const struct drm_crtc_helper_funcs bochs_helper_funcs = {
 	.commit = bochs_crtc_commit,
 };
 
+static const uint32_t bochs_formats[] = {
+	DRM_FORMAT_HOST_XRGB8888,
+};
+
+static struct drm_plane *bochs_primary_plane(struct drm_device *dev)
+{
+	struct drm_plane *primary;
+	int ret;
+
+	primary = kzalloc(sizeof(*primary), GFP_KERNEL);
+	if (primary == NULL) {
+		DRM_DEBUG_KMS("Failed to allocate primary plane\n");
+		return NULL;
+	}
+
+	ret = drm_universal_plane_init(dev, primary, 0,
+				       &drm_primary_helper_funcs,
+				       bochs_formats,
+				       ARRAY_SIZE(bochs_formats),
+				       NULL,
+				       DRM_PLANE_TYPE_PRIMARY, NULL);
+	if (ret) {
+		kfree(primary);
+		primary = NULL;
+	}
+
+	return primary;
+}
+
 static void bochs_crtc_init(struct drm_device *dev)
 {
 	struct bochs_device *bochs = dev->dev_private;
 	struct drm_crtc *crtc = &bochs->crtc;
+	struct drm_plane *primary = bochs_primary_plane(dev);
 
-	drm_crtc_init(dev, crtc, &bochs_crtc_funcs);
+	drm_crtc_init_with_planes(dev, crtc, primary, NULL,
+				  &bochs_crtc_funcs, NULL);
 	drm_crtc_helper_add(crtc, &bochs_helper_funcs);
 }
 
@@ -253,6 +284,7 @@ int bochs_kms_init(struct bochs_device *bochs)
 	bochs->dev->mode_config.fb_base = bochs->fb_base;
 	bochs->dev->mode_config.preferred_depth = 24;
 	bochs->dev->mode_config.prefer_shadow = 0;
+	bochs->dev->mode_config.quirk_addfb_prefer_host_byte_order = true;
 
 	bochs->dev->mode_config.funcs = &bochs_mode_funcs;
 
drivers/gpu/drm/bochs/bochs_mm.c +1 -1
index c9c7097030..fdf151fbdb 100644
@@ -506,7 +506,7 @@ bochs_user_framebuffer_create(struct drm_device *dev,
 	       (mode_cmd->pixel_format >> 16) & 0xff,
 	       (mode_cmd->pixel_format >> 24) & 0xff);
 
-	if (mode_cmd->pixel_format != DRM_FORMAT_XRGB8888)
+	if (mode_cmd->pixel_format != DRM_FORMAT_HOST_XRGB8888)
 		return ERR_PTR(-ENOENT);
 
 	obj = drm_gem_object_lookup(filp, mode_cmd->handles[0]);
Signature · 1 line
2.9.3
author maintainer
#8 lore raw

PATCH v2 5/6 drm/bochs: fix DRM_FORMAT_* handling for big endian machines.

In reply to Gerd Hoffmann

Use DRM_FORMAT_HOST_XRGB8888, so we are using the correct format code on bigendian machines. Also set the quirk_addfb_prefer_host_byte_order mode_config bit so drm_mode_addfb() asks for the correct format code.

Create our own plane and use drm_crtc_init_with_planes() instead of depending on the default created by drm_crtc_init(). That way the plane format list is correct on bigendian machines.

With this patch applied both ADDFB and ADDFB2 ioctls work correctly in the bochs-drm.ko driver on big endian machines. Without the patch only ADDFB (which still seems to be used by the majority of userspace) works correctly.

---
 drivers/gpu/drm/bochs/bochs_fbdev.c |  5 ++---
 drivers/gpu/drm/bochs/bochs_kms.c   | 34 +++++++++++++++++++++++++++++++++-
 drivers/gpu/drm/bochs/bochs_mm.c    |  2 +-
 3 files changed, 36 insertions(+), 5 deletions(-)
drivers/gpu/drm/bochs/bochs_fbdev.c +2 -3
index 14eb8d0d5a..bf728790fa 100644
@@ -64,9 +64,8 @@ static int bochsfb_create(struct drm_fb_helper *helper,
 
 	mode_cmd.width = sizes->surface_width;
 	mode_cmd.height = sizes->surface_height;
-	mode_cmd.pitches[0] = mode_cmd.width * ((sizes->surface_bpp + 7) / 8);
-	mode_cmd.pixel_format = drm_mode_legacy_fb_format(sizes->surface_bpp,
-							  sizes->surface_depth);
+	mode_cmd.pitches[0] = sizes->surface_width * 4;
+	mode_cmd.pixel_format = DRM_FORMAT_HOST_XRGB8888;
 	size = mode_cmd.pitches[0] * mode_cmd.height;
 
 	/* alloc, pin & map bo */
drivers/gpu/drm/bochs/bochs_kms.c +33 -1
index ca5a9afdd5..b40077d04b 100644
@@ -129,12 +129,43 @@ static const struct drm_crtc_helper_funcs bochs_helper_funcs = {
 	.commit = bochs_crtc_commit,
 };
 
+static const uint32_t bochs_formats[] = {
+	DRM_FORMAT_HOST_XRGB8888,
+};
+
+static struct drm_plane *bochs_primary_plane(struct drm_device *dev)
+{
+	struct drm_plane *primary;
+	int ret;
+
+	primary = kzalloc(sizeof(*primary), GFP_KERNEL);
+	if (primary == NULL) {
+		DRM_DEBUG_KMS("Failed to allocate primary plane\n");
+		return NULL;
+	}
+
+	ret = drm_universal_plane_init(dev, primary, 0,
+				       &drm_primary_helper_funcs,
+				       bochs_formats,
+				       ARRAY_SIZE(bochs_formats),
+				       NULL,
+				       DRM_PLANE_TYPE_PRIMARY, NULL);
+	if (ret) {
+		kfree(primary);
+		primary = NULL;
+	}
+
+	return primary;
+}
+
 static void bochs_crtc_init(struct drm_device *dev)
 {
 	struct bochs_device *bochs = dev->dev_private;
 	struct drm_crtc *crtc = &bochs->crtc;
+	struct drm_plane *primary = bochs_primary_plane(dev);
 
-	drm_crtc_init(dev, crtc, &bochs_crtc_funcs);
+	drm_crtc_init_with_planes(dev, crtc, primary, NULL,
+				  &bochs_crtc_funcs, NULL);
 	drm_crtc_helper_add(crtc, &bochs_helper_funcs);
 }
 
@@ -253,6 +284,7 @@ int bochs_kms_init(struct bochs_device *bochs)
 	bochs->dev->mode_config.fb_base = bochs->fb_base;
 	bochs->dev->mode_config.preferred_depth = 24;
 	bochs->dev->mode_config.prefer_shadow = 0;
+	bochs->dev->mode_config.quirk_addfb_prefer_host_byte_order = true;
 
 	bochs->dev->mode_config.funcs = &bochs_mode_funcs;
 
drivers/gpu/drm/bochs/bochs_mm.c +1 -1
index c9c7097030..fdf151fbdb 100644
@@ -506,7 +506,7 @@ bochs_user_framebuffer_create(struct drm_device *dev,
 	       (mode_cmd->pixel_format >> 16) & 0xff,
 	       (mode_cmd->pixel_format >> 24) & 0xff);
 
-	if (mode_cmd->pixel_format != DRM_FORMAT_XRGB8888)
+	if (mode_cmd->pixel_format != DRM_FORMAT_HOST_XRGB8888)
 		return ERR_PTR(-ENOENT);
 
 	obj = drm_gem_object_lookup(filp, mode_cmd->handles[0]);
Signature · 1 line
2.9.3
author maintainer
#9 lore raw

PATCH v2 6/6 drm/virtio: fix DRM_FORMAT_* handling

In reply to Gerd Hoffmann

Use DRM_FORMAT_HOST_XRGB8888, so we are using the correct format code on bigendian machines. Also set the quirk_addfb_prefer_host_byte_order mode_config bit so drm_mode_addfb() asks for the correct format code.

Both DRM_FORMAT_* and VIRTIO_GPU_FORMAT_* are defined to be little endian, so using a different mapping on bigendian machines is wrong. It's there because of broken drm_mode_addfb() behavior. So with drm_mode_addfb() being fixed we can fix this too.

While wading through the code I've noticed we have a little issue in
virtio: We attach a format to the bo when it is created (DRM_IOCTL_MODE_CREATE_DUMB), not when we map it as framebuffer (DRM_IOCTL_MODE_ADDFB). Easy way out: Support a single format only. Pick DRM_FORMAT_HOST_XRGB8888, it is the only one actually used in practice. Drop unused mappings in virtio_gpu_translate_format().

With this patch applied both ADDFB and ADDFB2 ioctls work correctly in the virtio-gpu.ko driver on big endian machines. Without the patch only ADDFB (which still seems to be used by the majority of userspace) works correctly.

---
 drivers/gpu/drm/virtio/virtgpu_display.c |  5 +++
 drivers/gpu/drm/virtio/virtgpu_fb.c      |  2 +-
 drivers/gpu/drm/virtio/virtgpu_gem.c     |  7 +++--
 drivers/gpu/drm/virtio/virtgpu_plane.c   | 54 ++------------------------------
 4 files changed, 13 insertions(+), 55 deletions(-)
drivers/gpu/drm/virtio/virtgpu_display.c +5 -0
index 25503b9335..14a13edc02 100644
@@ -306,6 +306,10 @@ virtio_gpu_user_framebuffer_create(struct drm_device *dev,
 	struct virtio_gpu_framebuffer *virtio_gpu_fb;
 	int ret;
 
+	if (mode_cmd->pixel_format != DRM_FORMAT_HOST_XRGB8888 &&
+	    mode_cmd->pixel_format != DRM_FORMAT_HOST_ARGB8888)
+		return ERR_PTR(-ENOENT);
+
 	/* lookup object associated with res handle */
 	obj = drm_gem_object_lookup(file_priv, mode_cmd->handles[0]);
 	if (!obj)
@@ -354,6 +358,7 @@ int virtio_gpu_modeset_init(struct virtio_gpu_device *vgdev)
 	int i;
 
 	drm_mode_config_init(vgdev->ddev);
+	vgdev->ddev->mode_config.quirk_addfb_prefer_host_byte_order = true;
 	vgdev->ddev->mode_config.funcs = &virtio_gpu_mode_funcs;
 	vgdev->ddev->mode_config.helper_private = &virtio_mode_config_helpers;
 
drivers/gpu/drm/virtio/virtgpu_fb.c +1 -1
index a121b1c795..9d87ebd645 100644
@@ -233,7 +233,7 @@ static int virtio_gpufb_create(struct drm_fb_helper *helper,
 	mode_cmd.width = sizes->surface_width;
 	mode_cmd.height = sizes->surface_height;
 	mode_cmd.pitches[0] = mode_cmd.width * 4;
-	mode_cmd.pixel_format = drm_mode_legacy_fb_format(32, 24);
+	mode_cmd.pixel_format = DRM_FORMAT_HOST_XRGB8888;
 
 	format = virtio_gpu_translate_format(mode_cmd.pixel_format);
 	if (format == 0)
drivers/gpu/drm/virtio/virtgpu_gem.c +5 -2
index 0f2768eaca..82c817f37c 100644
@@ -90,7 +90,10 @@ int virtio_gpu_mode_dumb_create(struct drm_file *file_priv,
 	uint32_t resid;
 	uint32_t format;
 
-	pitch = args->width * ((args->bpp + 1) / 8);
+	if (args->bpp != 32)
+		return -EINVAL;
+
+	pitch = args->width * 4;
 	args->size = pitch * args->height;
 	args->size = ALIGN(args->size, PAGE_SIZE);
 
@@ -99,7 +102,7 @@ int virtio_gpu_mode_dumb_create(struct drm_file *file_priv,
 	if (ret)
 		goto fail;
 
-	format = virtio_gpu_translate_format(DRM_FORMAT_XRGB8888);
+	format = virtio_gpu_translate_format(DRM_FORMAT_HOST_XRGB8888);
 	virtio_gpu_resource_id_get(vgdev, &resid);
 	virtio_gpu_cmd_create_resource(vgdev, resid, format,
 				       args->width, args->height);
drivers/gpu/drm/virtio/virtgpu_plane.c +2 -52

75 lines not on this page · raw message

Signature · 1 line
2.9.3
author maintainer
#10 lore raw

PATCH v2 6/6 drm/virtio: fix DRM_FORMAT_* handling

In reply to Gerd Hoffmann

Use DRM_FORMAT_HOST_XRGB8888, so we are using the correct format code on bigendian machines. Also set the quirk_addfb_prefer_host_byte_order mode_config bit so drm_mode_addfb() asks for the correct format code.

Both DRM_FORMAT_* and VIRTIO_GPU_FORMAT_* are defined to be little endian, so using a different mapping on bigendian machines is wrong. It's there because of broken drm_mode_addfb() behavior. So with drm_mode_addfb() being fixed we can fix this too.

While wading through the code I've noticed we have a little issue in
virtio: We attach a format to the bo when it is created (DRM_IOCTL_MODE_CREATE_DUMB), not when we map it as framebuffer (DRM_IOCTL_MODE_ADDFB). Easy way out: Support a single format only. Pick DRM_FORMAT_HOST_XRGB8888, it is the only one actually used in practice. Drop unused mappings in virtio_gpu_translate_format().

With this patch applied both ADDFB and ADDFB2 ioctls work correctly in the virtio-gpu.ko driver on big endian machines. Without the patch only ADDFB (which still seems to be used by the majority of userspace) works correctly.

---
 drivers/gpu/drm/virtio/virtgpu_display.c |  5 +++
 drivers/gpu/drm/virtio/virtgpu_fb.c      |  2 +-
 drivers/gpu/drm/virtio/virtgpu_gem.c     |  7 +++--
 drivers/gpu/drm/virtio/virtgpu_plane.c   | 54 ++------------------------------
 4 files changed, 13 insertions(+), 55 deletions(-)
drivers/gpu/drm/virtio/virtgpu_display.c +5 -0
index 25503b9335..14a13edc02 100644
@@ -306,6 +306,10 @@ virtio_gpu_user_framebuffer_create(struct drm_device *dev,
 	struct virtio_gpu_framebuffer *virtio_gpu_fb;
 	int ret;
 
+	if (mode_cmd->pixel_format != DRM_FORMAT_HOST_XRGB8888 &&
+	    mode_cmd->pixel_format != DRM_FORMAT_HOST_ARGB8888)
+		return ERR_PTR(-ENOENT);
+
 	/* lookup object associated with res handle */
 	obj = drm_gem_object_lookup(file_priv, mode_cmd->handles[0]);
 	if (!obj)
@@ -354,6 +358,7 @@ int virtio_gpu_modeset_init(struct virtio_gpu_device *vgdev)
 	int i;
 
 	drm_mode_config_init(vgdev->ddev);
+	vgdev->ddev->mode_config.quirk_addfb_prefer_host_byte_order = true;
 	vgdev->ddev->mode_config.funcs = &virtio_gpu_mode_funcs;
 	vgdev->ddev->mode_config.helper_private = &virtio_mode_config_helpers;
 
drivers/gpu/drm/virtio/virtgpu_fb.c +1 -1
index a121b1c795..9d87ebd645 100644
@@ -233,7 +233,7 @@ static int virtio_gpufb_create(struct drm_fb_helper *helper,
 	mode_cmd.width = sizes->surface_width;
 	mode_cmd.height = sizes->surface_height;
 	mode_cmd.pitches[0] = mode_cmd.width * 4;
-	mode_cmd.pixel_format = drm_mode_legacy_fb_format(32, 24);
+	mode_cmd.pixel_format = DRM_FORMAT_HOST_XRGB8888;
 
 	format = virtio_gpu_translate_format(mode_cmd.pixel_format);
 	if (format == 0)
drivers/gpu/drm/virtio/virtgpu_gem.c +5 -2
index 0f2768eaca..82c817f37c 100644
@@ -90,7 +90,10 @@ int virtio_gpu_mode_dumb_create(struct drm_file *file_priv,
 	uint32_t resid;
 	uint32_t format;
 
-	pitch = args->width * ((args->bpp + 1) / 8);
+	if (args->bpp != 32)
+		return -EINVAL;
+
+	pitch = args->width * 4;
 	args->size = pitch * args->height;
 	args->size = ALIGN(args->size, PAGE_SIZE);
 
@@ -99,7 +102,7 @@ int virtio_gpu_mode_dumb_create(struct drm_file *file_priv,
 	if (ret)
 		goto fail;
 
-	format = virtio_gpu_translate_format(DRM_FORMAT_XRGB8888);
+	format = virtio_gpu_translate_format(DRM_FORMAT_HOST_XRGB8888);
 	virtio_gpu_resource_id_get(vgdev, &resid);
 	virtio_gpu_cmd_create_resource(vgdev, resid, format,
 				       args->width, args->height);
drivers/gpu/drm/virtio/virtgpu_plane.c +2 -52

75 lines not on this page · raw message

Signature · 1 line
2.9.3
#11 lore raw
In reply to Gerd Hoffmann

Hi Gerd,

What happened with this series (and the next one)?

Semi-relatedly, I wonder if it wouldn't be better to just dump the BIG_ENDIAN flag and just define the "host" format to be the right one for a consistent little-endian interpretation.

Cheers,

-ilia

Quoted 30 lines · Gerd Hoffmann

Patch series adds some convinience #defines for host byteoder drm formats. It fixes drm_mode_addfb() behavior on bigendian machines. For bug compatibility reasons a mode_config quirk activates the fix. bochs and virtio-gpu drivers are updated to use the new #defines, set the new driver feature flag and fix some issues.

Gerd Hoffmann (6):
drm: replace DRIVER_PREFER_XBGR_30BPP driver flag with mode_config
quirk
drm: byteorder: add DRM_FORMAT_HOST_*
drm: do not mask out DRM_FORMAT_BIG_ENDIAN
drm: fix drm_mode_addfb() on big endian machines. drm/bochs: fix DRM_FORMAT_* handling for big endian machines. drm/virtio: fix DRM_FORMAT_* handling

 include/drm/drm_drv.h                    |  1 -
 include/drm/drm_fourcc.h                 | 22 +++++++++++++
 include/drm/drm_mode_config.h            | 15 +++++++++
 drivers/gpu/drm/bochs/bochs_fbdev.c      |  5 ++-
 drivers/gpu/drm/bochs/bochs_kms.c        | 34 +++++++++++++++++++-
 drivers/gpu/drm/bochs/bochs_mm.c         |  2 +-
 drivers/gpu/drm/drm_framebuffer.c        | 17 ++++++++--
 drivers/gpu/drm/nouveau/dispnv50/disp.c  |  2 +-
 drivers/gpu/drm/virtio/virtgpu_display.c |  5 +++
 drivers/gpu/drm/virtio/virtgpu_fb.c      |  2 +-
 drivers/gpu/drm/virtio/virtgpu_gem.c     |  7 +++--
 drivers/gpu/drm/virtio/virtgpu_plane.c   | 54 ++------------------------------
 12 files changed, 101 insertions(+), 65 deletions(-)
Quoted 3 lines

--
2.9.3

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

author maintainer
#12 lore raw
Quoted 3 lines · Ilia Mirkin

Hi Gerd,

What happened with this series (and the next one)?

Landed upstream in 4.20.

Quoted 3 lines

Semi-relatedly, I wonder if it wouldn't be better to just dump the BIG_ENDIAN flag and just define the "host" format to be the right one for a consistent little-endian interpretation.

Not fully sure what you mean here.

The big endian flag is needed to specify RGB565 or XRGB1555 format in big endian byte order. For 32 bit depth we don't need it because XRGB8888 big endian == BGRX8888 little endian (see also include/drm/drm_fourcc.h).

cheers,
Gerd

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#13 lore raw
Quoted 12 lines · Gerd Hoffmann
Quoted 3 lines · Ilia Mirkin

Hi Gerd,

What happened with this series (and the next one)?

Landed upstream in 4.20.

Quoted 3 lines

Semi-relatedly, I wonder if it wouldn't be better to just dump the BIG_ENDIAN flag and just define the "host" format to be the right one for a consistent little-endian interpretation.

Quoted 6 lines

Not fully sure what you mean here.

The big endian flag is needed to specify RGB565 or XRGB1555 format in big endian byte order. For 32 bit depth we don't need it because XRGB8888 big endian == BGRX8888 little endian (see also include/drm/drm_fourcc.h).

Yeah we'd need to add a few more fourcc codes for the missing big-endian versions. Aside from that I like Ilia's idea of removing the BIG_ENDIAN flag, it causes lots of aliasing among formats, doesn't change anything with others, and we probably have a huge mess already
...
-Daniel

Quoted 8 lines

cheers,
Gerd

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
author maintainer
#14 lore raw
Quoted 15 lines · Daniel Vetter
Quoted 14 lines · Gerd Hoffmann
Quoted 3 lines · Ilia Mirkin

Hi Gerd,

What happened with this series (and the next one)?

Landed upstream in 4.20.

Quoted 3 lines

Semi-relatedly, I wonder if it wouldn't be better to just dump the BIG_ENDIAN flag and just define the "host" format to be the right one for a consistent little-endian interpretation.

Not fully sure what you mean here.

Quoted 7 lines
Quoted 4 lines

The big endian flag is needed to specify RGB565 or XRGB1555 format in big endian byte order. For 32 bit depth we don't need it because XRGB8888 big endian == BGRX8888 little endian (see also include/drm/drm_fourcc.h).

Yeah we'd need to add a few more fourcc codes for the missing big-endian versions.

If we do that then yes we can drop the BIG_ENDIAN flag.

Not sure what the userspace API implications are,
ADDFB2 ioctl comes to mind.

Quoted 4 lines

Aside from that I like Ilia's idea of removing
the BIG_ENDIAN flag, it causes lots of aliasing among formats, doesn't change anything with others, and we probably have a huge mess already
...

Shouldn't be too messy. The place where the DRM_FORMAT_HOST_* formats are defined (include/drm/drm_fourcc.h) is almost the only place where DRM_FORMAT_BIG_ENDIAN is used, and for the DRM_FORMAT_HOST_* users the switch should be transparent.

cheers,
Gerd

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#15 lore raw
Quoted 34 lines · Gerd Hoffmann
Quoted 15 lines · Daniel Vetter
Quoted 14 lines · Gerd Hoffmann
Quoted 3 lines · Ilia Mirkin

Hi Gerd,

What happened with this series (and the next one)?

Landed upstream in 4.20.

Quoted 3 lines

Semi-relatedly, I wonder if it wouldn't be better to just dump the BIG_ENDIAN flag and just define the "host" format to be the right one for a consistent little-endian interpretation.

Not fully sure what you mean here.

Quoted 7 lines
Quoted 4 lines

The big endian flag is needed to specify RGB565 or XRGB1555 format in big endian byte order. For 32 bit depth we don't need it because XRGB8888 big endian == BGRX8888 little endian (see also include/drm/drm_fourcc.h).

Yeah we'd need to add a few more fourcc codes for the missing big-endian versions.

If we do that then yes we can drop the BIG_ENDIAN flag.

Not sure what the userspace API implications are,
ADDFB2 ioctl comes to mind.

Quoted 4 lines

Aside from that I like Ilia's idea of removing
the BIG_ENDIAN flag, it causes lots of aliasing among formats, doesn't change anything with others, and we probably have a huge mess already
...

Quoted 4 lines

Shouldn't be too messy. The place where the DRM_FORMAT_HOST_* formats are defined (include/drm/drm_fourcc.h) is almost the only place where DRM_FORMAT_BIG_ENDIAN is used, and for the DRM_FORMAT_HOST_* users the switch should be transparent.

Yeah we'd need to review current userspace, but I suspect the overlap of "uses addfb2" and "runs on BE platform" is 0.
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#16 lore raw
Quoted 12 lines · On Fri, Jan 11, 2019, 05:26 Daniel Vetter <[email protected] wrote:
Quoted 2 lines · Gerd Hoffmann
Quoted 1 line · Daniel Vetter

On Fri, Jan 11, 2019 at 10:08 AM Gerd Hoffmann <[email protected]>

Quoted 7 lines · wrote:
Quoted 7 lines
Quoted 7 lines · Ilia Mirkin
Quoted 3 lines

Hi Gerd,

What happened with this series (and the next one)?

Landed upstream in 4.20.

Huh. I must have been looking at an option tree.

Quoted 13 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Semi-relatedly, I wonder if it wouldn't be better to just dump the BIG_ENDIAN flag and just define the "host" format to be the right

one

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line

for a consistent little-endian interpretation.

Not fully sure what you mean here.

The big endian flag is needed to specify RGB565 or XRGB1555 format in big endian byte order. For 32 bit depth we don't need it because XRGB8888 big endian == BGRX8888 little endian (see also include/drm/drm_fourcc.h).

Bleargh. I was thinking xrgb1555 le == bgrx5551 be, but that's clearly false. Made sense to me last night though. That makes my suggestion considerably less appealing.

Quoted 14 lines
Quoted 14 lines
Quoted 3 lines

Yeah we'd need to add a few more fourcc codes for the missing big-endian versions.

If we do that then yes we can drop the BIG_ENDIAN flag.

Not sure what the userspace API implications are,
ADDFB2 ioctl comes to mind.

Quoted 4 lines

Aside from that I like Ilia's idea of removing
the BIG_ENDIAN flag, it causes lots of aliasing among formats, doesn't change anything with others, and we probably have a huge mess already
...

Quoted 8 lines
Quoted 4 lines

Shouldn't be too messy. The place where the DRM_FORMAT_HOST_* formats are defined (include/drm/drm_fourcc.h) is almost the only place where DRM_FORMAT_BIG_ENDIAN is used, and for the DRM_FORMAT_HOST_* users the switch should be transparent.

Yeah we'd need to review current userspace, but I suspect the overlap of "uses addfb2" and "runs on BE platform" is 0.

Most importantly, "uses bigendian flag" is 0 since we would previously error out on such modesets.

Although the need to double up on the 16bpp formats makes this not really worth it. Ohhh well.

-ilia

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#17 lore raw
Quoted 53 lines · Ilia Mirkin
Quoted 11 lines · On Fri, Jan 11, 2019, 05:26 Daniel Vetter <[email protected] wrote:
Quoted 9 lines · Gerd Hoffmann
Quoted 8 lines · Daniel Vetter
Quoted 7 lines · Gerd Hoffmann
Quoted 3 lines · Ilia Mirkin

Hi Gerd,

What happened with this series (and the next one)?

Landed upstream in 4.20.

Huh. I must have been looking at an option tree.

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 4 lines

Semi-relatedly, I wonder if it wouldn't be better to just dump the BIG_ENDIAN flag and just define the "host" format to be the right one for a consistent little-endian interpretation.

Not fully sure what you mean here.

The big endian flag is needed to specify RGB565 or XRGB1555 format in big endian byte order. For 32 bit depth we don't need it because XRGB8888 big endian == BGRX8888 little endian (see also include/drm/drm_fourcc.h).

Bleargh. I was thinking xrgb1555 le == bgrx5551 be, but that's clearly false. Made sense to me last night though. That makes my suggestion considerably less appealing.

Quoted 19 lines
Quoted 18 lines
Quoted 3 lines

Yeah we'd need to add a few more fourcc codes for the missing big-endian versions.

If we do that then yes we can drop the BIG_ENDIAN flag.

Not sure what the userspace API implications are,
ADDFB2 ioctl comes to mind.

Quoted 4 lines

Aside from that I like Ilia's idea of removing
the BIG_ENDIAN flag, it causes lots of aliasing among formats, doesn't change anything with others, and we probably have a huge mess already
...

Shouldn't be too messy. The place where the DRM_FORMAT_HOST_* formats are defined (include/drm/drm_fourcc.h) is almost the only place where DRM_FORMAT_BIG_ENDIAN is used, and for the DRM_FORMAT_HOST_* users the switch should be transparent.

Quoted 7 lines
Quoted 2 lines

Yeah we'd need to review current userspace, but I suspect the overlap of "uses addfb2" and "runs on BE platform" is 0.

Most importantly, "uses bigendian flag" is 0 since we would previously error out on such modesets.

Although the need to double up on the 16bpp formats makes this not really worth it. Ohhh well.

I think even with a few formats doubled up it'll be worth it, since we remove all the confusion around the formats where the BE flag is not needed/confusion/causes aliasing with another format. Also given that even ppc is switching to LE I think making LE the forced default assumption for everything will work much better long term. At least with the explicit fourcc code, legacy addfb will pick native endianess (at least since 4.20).
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel