drm: Only try to set formats supported by the hardware

When userland uses DRM_IOCTL_MODE_ADDFB to add a framebuffer, the DRM subsystem tries to find a pixel format from the supplied depth and bpp-values. It does this by calling drm_driver_legacy_fb_format(). Unfortunately drm_driver_legacy_fb_format() can return formats not supported by the underlying hardware. This series of patches remedies this problem in patch 1.

In order to use the same logic for determining the pixel format, when a fbdev adds a framebuffer as userland does, patches 2 to 11 migrates fbdev users of drm_mode_legacy_fb_format() to drm_driver_legacy_fb_format().

This series has been tested with the nouveau and modesetting drivers on a NVIDIA NV96, the modesetting driver on Beagleboard Black, and with the Intel and modesetting drivers on an Intel HD Graphics 4000 chipset.

This is an evolved version of the changes proposed in "drm: Don't return unsupported formats in drm_mode_legacy_fb_format" [1] following the suggestions of Thomas Zimmermann.

[1] https://lore.kernel.org/all/[email protected]/

Frej Drejhammar (11):
drm: Only return supported formats from drm_driver_legacy_fb_format drm/fbdev_generic: Use drm_driver_legacy_fb_format() for fbdev drm/armada: Use drm_driver_legacy_fb_format() for fbdev drm/exynos: Use drm_driver_legacy_fb_format() for fbdev drm/gma500: Use drm_driver_legacy_fb_format() for fbdev drm/i915: Use drm_driver_legacy_fb_format() for fbdev drm/msm: Use drm_driver_legacy_fb_format() for fbdev drm/omapdrm: Use drm_driver_legacy_fb_format() for fbdev drm/radeon: Use drm_driver_legacy_fb_format() for fbdev drm/tegra: Use drm_driver_legacy_fb_format() for fbdev drm/xe: Use drm_driver_legacy_fb_format() for fbdev

 drivers/gpu/drm/armada/armada_fbdev.c         |  5 +-
 drivers/gpu/drm/drm_fb_helper.c               |  2 +-
 drivers/gpu/drm/drm_fbdev_dma.c               |  4 +-
 drivers/gpu/drm/drm_fbdev_generic.c           |  4 +-
 drivers/gpu/drm/drm_fourcc.c                  | 83 +++++++++++++++++++
 drivers/gpu/drm/exynos/exynos_drm_fbdev.c     |  6 +-
 drivers/gpu/drm/gma500/fbdev.c                |  2 +-
 drivers/gpu/drm/i915/display/intel_fbdev_fb.c |  6 +-
 drivers/gpu/drm/msm/msm_fbdev.c               |  4 +-
 drivers/gpu/drm/omapdrm/omap_fbdev.c          |  6 +-
 drivers/gpu/drm/radeon/radeon_fbdev.c         |  6 +-
 drivers/gpu/drm/tegra/fbdev.c                 |  5 +-
 drivers/gpu/drm/xe/display/intel_fbdev_fb.c   |  5 +-
 13 files changed, 119 insertions(+), 19 deletions(-)

base-commit: 119b225f01e4d3ce974cd3b4d982c76a380c796d

2.44.0