Skip to contentSkip to conversation

drm/i915: Fix global state use-after-frees with a refcount

PATCH In review Ville Syrjala 3 participants 8 messages May 27, 2020 – Jun 1, 2020

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/i915: Fix global state use-after-frees with a refcount

From: Ville Syrjälä <[email protected]>

While the current locking/serialization of the global state suffices for protecting the obj->state access and the actual hardware reprogramming, we do have a problem with accessing the old/new states during nonblocking commits.

The state computation and swap will be protected by the crtc locks, but the commit_tails can finish out of order, thus also causing the atomic states to be cleaned up out of order. This would mean the commit that started first but finished last has had its new state freed as the no-longer-needed old state by the other commit.

To fix this let's just refcount the states. obj->state amounts to one reference, and the intel_atomic_state holds extra references to both its new and old global obj states.

Fixes 0ef1905ecf2e ("drm/i915: Introduce better global state handling")
---
 .../gpu/drm/i915/display/intel_global_state.c | 45 ++++++++++++++++---
 .../gpu/drm/i915/display/intel_global_state.h |  3 ++
 2 files changed, 42 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/display/intel_global_state.c +39 -6
index 212d4ee68205..7a19215ad844 100644
@@ -10,6 +10,28 @@
 #include "intel_display_types.h"
 #include "intel_global_state.h"
 
+static void __intel_atomic_global_state_free(struct kref *kref)
+{
+	struct intel_global_state *obj_state =
+		container_of(kref, struct intel_global_state, ref);
+	struct intel_global_obj *obj = obj_state->obj;
+
+	obj->funcs->atomic_destroy_state(obj, obj_state);
+}
+
+static void intel_atomic_global_state_put(struct intel_global_state *obj_state)
+{
+	kref_put(&obj_state->ref, __intel_atomic_global_state_free);
+}
+
+static struct intel_global_state *
+intel_atomic_global_state_get(struct intel_global_state *obj_state)
+{
+	kref_get(&obj_state->ref);
+
+	return obj_state;
+}
+
 void intel_atomic_global_obj_init(struct drm_i915_private *dev_priv,
 				  struct intel_global_obj *obj,
 				  struct intel_global_state *state,
@@ -17,6 +39,10 @@ void intel_atomic_global_obj_init(struct drm_i915_private *dev_priv,
 {
 	memset(obj, 0, sizeof(*obj));
 
+	state->obj = obj;
+
+	kref_init(&state->ref);
+
 	obj->state = state;
 	obj->funcs = funcs;
 	list_add_tail(&obj->head, &dev_priv->global_obj_list);
@@ -28,7 +54,9 @@ void intel_atomic_global_obj_cleanup(struct drm_i915_private *dev_priv)
 
 	list_for_each_entry_safe(obj, next, &dev_priv->global_obj_list, head) {
 		list_del(&obj->head);
-		obj->funcs->atomic_destroy_state(obj, obj->state);
+
+		drm_WARN_ON(&dev_priv->drm, kref_read(&obj->state->ref) != 1);
+		intel_atomic_global_state_put(obj->state);
 	}
 }
 
@@ -97,10 +125,14 @@ intel_atomic_get_global_obj_state(struct intel_atomic_state *state,
 	if (!obj_state)
 		return ERR_PTR(-ENOMEM);
 
+	obj_state->obj = obj;
 	obj_state->changed = false;
 
+	kref_init(&obj_state->ref);
+
 	state->global_objs[index].state = obj_state;
-	state->global_objs[index].old_state = obj->state;
+	state->global_objs[index].old_state =
+		intel_atomic_global_state_get(obj->state);
 	state->global_objs[index].new_state = obj_state;
 	state->global_objs[index].ptr = obj;
 	obj_state->state = state;
@@ -163,7 +195,9 @@ void intel_atomic_swap_global_state(struct intel_atomic_state *state)
 		new_obj_state->state = NULL;
 
 		state->global_objs[i].state = old_obj_state;
-		obj->state = new_obj_state;
+
+		intel_atomic_global_state_put(obj->state);
+		obj->state = intel_atomic_global_state_get(new_obj_state);
 	}
 }
 
@@ -172,10 +206,9 @@ void intel_atomic_clear_global_state(struct intel_atomic_state *state)
 	int i;
 
 	for (i = 0; i < state->num_global_objs; i++) {
-		struct intel_global_obj *obj = state->global_objs[i].ptr;
+		intel_atomic_global_state_put(state->global_objs[i].old_state);
+		intel_atomic_global_state_put(state->global_objs[i].new_state);
 
-		obj->funcs->atomic_destroy_state(obj,
-						 state->global_objs[i].state);
 		state->global_objs[i].ptr = NULL;
 		state->global_objs[i].state = NULL;
 		state->global_objs[i].old_state = NULL;
drivers/gpu/drm/i915/display/intel_global_state.h +3 -0
index e6163a469029..1f16fa3073c9 100644
@@ -6,6 +6,7 @@
 #ifndef __INTEL_GLOBAL_STATE_H__
 #define __INTEL_GLOBAL_STATE_H__
 
+#include <linux/kref.h>
 #include <linux/list.h>
 
 struct drm_i915_private;
@@ -54,7 +55,9 @@ struct intel_global_obj {
 		for_each_if(obj)
 
 struct intel_global_state {
+	struct intel_global_obj *obj;
 	struct intel_atomic_state *state;
+	struct kref ref;
 	bool changed;
 };
 
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

[Intel-gfx] ✓ Fi.CI.BAT: success for drm/i915: Fix global state use-after-frees with a refcount

== Series Details ==

== Series Details ==

Series: drm/i915: Fix global state use-after-frees with a refcount
URL : https://patchwork.freedesktop.org/series/77715/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_8544 -> Patchwork_17796
====================================================

Summary
-------

**SUCCESS**

No regressions found.

External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/index.html

Known issues
------------

Here are the changes found in Patchwork_17796 that come from known issues:

### IGT changes ###

#### Possible fixes ####

* igt@kms_chamelium@dp-crc-fast:
- fi-icl-u2: [FAIL][1] ([i915#262]) -> [PASS][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/fi-icl-u2/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/fi-icl-u2/igt@[email protected]

#### Warnings ####

* igt@i915_pm_rpm@module-reload:
- fi-kbl-x1275: [SKIP][3] ([fdo#109271]) -> [FAIL][4] ([i915#62])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/fi-kbl-x1275/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/fi-kbl-x1275/igt@[email protected]

[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[i915#262]: https://gitlab.freedesktop.org/drm/intel/issues/262
[i915#62]: https://gitlab.freedesktop.org/drm/intel/issues/62

Participating hosts (49 -> 43)
------------------------------

Additional (1): fi-kbl-7560u
Missing (7): fi-ilk-m540 fi-hsw-4200u fi-byt-squawks fi-bsw-cyan fi-ctg-p8600 fi-byt-clapper fi-bdw-samus

Build changes
-------------

* Linux: CI_DRM_8544 -> Patchwork_17796

CI-20190529: 20190529
CI_DRM_8544: c6c0a18e985d7a3fd4451e0e786e6522371ea9ee @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_5680: f7e3772175c53f0c910f4513831791cb5bdcab04 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
Patchwork_17796: 1fac3818bc7097e659306e2db4a082c9e1099dd2 @ git://anongit.freedesktop.org/gfx-ci/linux

== Linux commits ==

1fac3818bc70 drm/i915: Fix global state use-after-frees with a refcount

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/index.html
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#2 lore raw

[Intel-gfx] ✗ Fi.CI.IGT: failure for drm/i915: Fix global state use-after-frees with a refcount

In reply to Ville Syrjala
== Series Details ==

== Series Details ==

Series: drm/i915: Fix global state use-after-frees with a refcount
URL : https://patchwork.freedesktop.org/series/77715/
State : failure

== Summary ==

CI Bug Log - changes from CI_DRM_8544_full -> Patchwork_17796_full
====================================================

Summary
-------

**FAILURE**

Serious unknown changes coming with Patchwork_17796_full absolutely need to be verified manually.

If you think the reported changes have nothing to do with the changes introduced in Patchwork_17796_full, please notify your bug team to allow them to document this new failure mode, which will reduce false positives in CI.

Possible new issues
-------------------

Here are the unknown changes that may have been introduced in Patchwork_17796_full:

### IGT changes ###

#### Possible regressions ####

* igt@kms_plane_alpha_blend@pipe-b-alpha-basic:
- shard-glk: NOTRUN -> [TIMEOUT][1] +1 similar issue
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-glk2/igt@[email protected]

* igt@runner@aborted:
- shard-hsw: NOTRUN -> [FAIL][2]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-hsw8/igt@[email protected]

Known issues
------------

Here are the changes found in Patchwork_17796_full that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@gen9_exec_parse@allowed-all:
- shard-apl: [PASS][3] -> [DMESG-WARN][4] ([i915#1436] / [i915#716])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-apl4/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-apl7/igt@[email protected]

* igt@kms_big_fb@y-tiled-64bpp-rotate-0:
- shard-glk: [PASS][5] -> [FAIL][6] ([i915#1119] / [i915#118] / [i915#95])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-glk2/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-glk8/igt@[email protected]

* igt@kms_cursor_crc@pipe-c-cursor-64x64-offscreen:
- shard-hsw: [PASS][7] -> [INCOMPLETE][8] ([i915#1927] / [i915#61])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-hsw2/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-hsw8/igt@[email protected]

* igt@kms_cursor_crc@pipe-c-cursor-suspend:
- shard-skl: [PASS][9] -> [INCOMPLETE][10] ([i915#300])
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-skl4/igt@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-skl1/igt@[email protected]
- shard-kbl: [PASS][11] -> [DMESG-WARN][12] ([i915#180]) +4 similar issues
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-kbl1/igt@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-kbl3/igt@[email protected]

* igt@kms_dp_dsc@basic-dsc-enable-edp:
- shard-iclb: [PASS][13] -> [SKIP][14] ([fdo#109349])
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-iclb2/igt@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-iclb7/igt@[email protected]

* igt@kms_frontbuffer_tracking@fbc-suspend:
- shard-kbl: [PASS][15] -> [DMESG-WARN][16] ([i915#180] / [i915#93] / [i915#95])
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-kbl2/igt@[email protected]
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-kbl3/igt@[email protected]

* igt@kms_plane_alpha_blend@pipe-a-coverage-7efc:
- shard-skl: [PASS][17] -> [FAIL][18] ([fdo#108145] / [i915#265]) +1 similar issue
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-skl4/igt@[email protected]
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-skl4/igt@[email protected]

* igt@kms_psr@psr2_cursor_render:
- shard-iclb: [PASS][19] -> [SKIP][20] ([fdo#109441]) +2 similar issues
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-iclb2/igt@kms_psr@psr2_cursor_render.html
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-iclb3/igt@kms_psr@psr2_cursor_render.html

* igt@kms_setmode@basic:
- shard-apl: [PASS][21] -> [FAIL][22] ([i915#31])
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-apl2/igt@[email protected]
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-apl2/igt@[email protected]

#### Possible fixes ####

* {igt@gem_exec_reloc@basic-concurrent0}:
- shard-glk: [FAIL][23] ([i915#1930]) -> [PASS][24]
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-glk7/igt@[email protected]
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-glk4/igt@[email protected]

* igt@i915_pm_rpm@system-suspend-execbuf:
- shard-kbl: [INCOMPLETE][25] ([i915#151] / [i915#155]) -> [PASS][26]
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-kbl1/igt@[email protected]
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-kbl6/igt@[email protected]

* igt@i915_suspend@forcewake:
- shard-skl: [INCOMPLETE][27] ([i915#636] / [i915#69]) -> [PASS][28]
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-skl5/igt@[email protected]
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-skl9/igt@[email protected]

* igt@kms_big_fb@linear-64bpp-rotate-180:
- shard-glk: [FAIL][29] ([i915#1119] / [i915#118] / [i915#95]) -> [PASS][30]
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-glk8/igt@[email protected]
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-glk6/igt@[email protected]

* igt@kms_cursor_crc@pipe-a-cursor-suspend:
- shard-kbl: [DMESG-WARN][31] ([i915#180]) -> [PASS][32] +3 similar issues
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-kbl4/igt@[email protected]
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-kbl2/igt@[email protected]

* igt@kms_cursor_crc@pipe-b-cursor-suspend:
- shard-apl: [DMESG-WARN][33] ([i915#180]) -> [PASS][34]
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-apl6/igt@[email protected]
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-apl4/igt@[email protected]

* {igt@kms_flip@2x-flip-vs-expired-vblank@bc-hdmi-a1-hdmi-a2}:
- shard-glk: [FAIL][35] ([i915#79]) -> [PASS][36]
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-glk5/igt@kms_flip@[email protected]
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-glk7/igt@kms_flip@[email protected]

* {igt@kms_flip@flip-vs-suspend@c-hdmi-a1}:
- shard-hsw: [INCOMPLETE][37] ([i915#61]) -> [PASS][38]
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-hsw8/igt@kms_flip@[email protected]
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-hsw8/igt@kms_flip@[email protected]

* igt@kms_hdr@bpc-switch-dpms:
- shard-skl: [FAIL][39] ([i915#1188]) -> [PASS][40] +1 similar issue
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-skl5/igt@[email protected]
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-skl1/igt@[email protected]

* igt@kms_psr@psr2_sprite_mmap_gtt:
- shard-iclb: [SKIP][41] ([fdo#109441]) -> [PASS][42] +3 similar issues
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-iclb1/igt@kms_psr@psr2_sprite_mmap_gtt.html
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-iclb2/igt@kms_psr@psr2_sprite_mmap_gtt.html

* {igt@perf@blocking-parameterized}:
- shard-hsw: [FAIL][43] ([i915#1542]) -> [PASS][44]
[43]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-hsw7/igt@[email protected]
[44]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-hsw2/igt@[email protected]

#### Warnings ####

* igt@kms_content_protection@atomic:
- shard-apl: [FAIL][45] ([fdo#110321] / [fdo#110336]) -> [TIMEOUT][46] ([i915#1319] / [i915#1635])
[45]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-apl6/igt@[email protected]
[46]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-apl4/igt@[email protected]

* igt@kms_content_protection@atomic-dpms:
- shard-apl: [TIMEOUT][47] ([i915#1319] / [i915#1635]) -> [FAIL][48] ([fdo#110321] / [fdo#110336] / [i915#95])
[47]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8544/shard-apl3/igt@[email protected]
[48]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/shard-apl1/igt@[email protected]

{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).

[fdo#108145]: https://bugs.freedesktop.org/show_bug.cgi?id=108145
[fdo#109349]: https://bugs.freedesktop.org/show_bug.cgi?id=109349
[fdo#109441]: https://bugs.freedesktop.org/show_bug.cgi?id=109441
[fdo#110321]: https://bugs.freedesktop.org/show_bug.cgi?id=110321
[fdo#110336]: https://bugs.freedesktop.org/show_bug.cgi?id=110336
[i915#1119]: https://gitlab.freedesktop.org/drm/intel/issues/1119
[i915#118]: https://gitlab.freedesktop.org/drm/intel/issues/118
[i915#1188]: https://gitlab.freedesktop.org/drm/intel/issues/1188
[i915#1319]: https://gitlab.freedesktop.org/drm/intel/issues/1319
[i915#1436]: https://gitlab.freedesktop.org/drm/intel/issues/1436
[i915#151]: https://gitlab.freedesktop.org/drm/intel/issues/151
[i915#1542]: https://gitlab.freedesktop.org/drm/intel/issues/1542
[i915#155]: https://gitlab.freedesktop.org/drm/intel/issues/155
[i915#1635]: https://gitlab.freedesktop.org/drm/intel/issues/1635
[i915#180]: https://gitlab.freedesktop.org/drm/intel/issues/180
[i915#1927]: https://gitlab.freedesktop.org/drm/intel/issues/1927
[i915#1930]: https://gitlab.freedesktop.org/drm/intel/issues/1930
[i915#265]: https://gitlab.freedesktop.org/drm/intel/issues/265
[i915#300]: https://gitlab.freedesktop.org/drm/intel/issues/300
[i915#31]: https://gitlab.freedesktop.org/drm/intel/issues/31
[i915#61]: https://gitlab.freedesktop.org/drm/intel/issues/61
[i915#636]: https://gitlab.freedesktop.org/drm/intel/issues/636
[i915#69]: https://gitlab.freedesktop.org/drm/intel/issues/69
[i915#716]: https://gitlab.freedesktop.org/drm/intel/issues/716
[i915#79]: https://gitlab.freedesktop.org/drm/intel/issues/79
[i915#93]: https://gitlab.freedesktop.org/drm/intel/issues/93
[i915#95]: https://gitlab.freedesktop.org/drm/intel/issues/95

Participating hosts (11 -> 11)
------------------------------

No changes in participating hosts

Build changes
-------------

* Linux: CI_DRM_8544 -> Patchwork_17796

CI-20190529: 20190529
CI_DRM_8544: c6c0a18e985d7a3fd4451e0e786e6522371ea9ee @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_5680: f7e3772175c53f0c910f4513831791cb5bdcab04 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
Patchwork_17796: 1fac3818bc7097e659306e2db4a082c9e1099dd2 @ git://anongit.freedesktop.org/gfx-ci/linux
piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_17796/index.html
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#3 lore raw
In reply to Ville Syrjala
Quoted 108 lines · Ville Syrjala

From: Ville Syrjälä <[email protected]>

While the current locking/serialization of the global state suffices for protecting the obj->state access and the actual hardware reprogramming, we do have a problem with accessing the old/new states during nonblocking commits.

The state computation and swap will be protected by the crtc locks, but the commit_tails can finish out of order, thus also causing the atomic states to be cleaned up out of order. This would mean the commit that started first but finished last has had its new state freed as the no-longer-needed old state by the other commit.

To fix this let's just refcount the states. obj->state amounts to one reference, and the intel_atomic_state holds extra references to both its new and old global obj states.

Fixes 0ef1905ecf2e ("drm/i915: Introduce better global state handling")
---
 .../gpu/drm/i915/display/intel_global_state.c | 45 ++++++++++++++++---
 .../gpu/drm/i915/display/intel_global_state.h |  3 ++
 2 files changed, 42 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/display/intel_global_state.c +37 -3
index 212d4ee68205..7a19215ad844 100644
@@ -10,6 +10,28 @@
 #include "intel_display_types.h"
 #include "intel_global_state.h"
 
+static void __intel_atomic_global_state_free(struct kref *kref)
+{
+	struct intel_global_state *obj_state =
+		container_of(kref, struct intel_global_state, ref);
+	struct intel_global_obj *obj = obj_state->obj;
+
+	obj->funcs->atomic_destroy_state(obj, obj_state);
+}
+
+static void intel_atomic_global_state_put(struct intel_global_state *obj_state)
+{
+	kref_put(&obj_state->ref, __intel_atomic_global_state_free);
+}
+
+static struct intel_global_state *
+intel_atomic_global_state_get(struct intel_global_state *obj_state)
+{
+	kref_get(&obj_state->ref);
+
+	return obj_state;
+}
+
 void intel_atomic_global_obj_init(struct drm_i915_private *dev_priv,
 				  struct intel_global_obj *obj,
 				  struct intel_global_state *state,
@@ -17,6 +39,10 @@ void intel_atomic_global_obj_init(struct drm_i915_private *dev_priv,
 {
 	memset(obj, 0, sizeof(*obj));
 
+	state->obj = obj;
+
+	kref_init(&state->ref);
+
 	obj->state = state;
 	obj->funcs = funcs;
 	list_add_tail(&obj->head, &dev_priv->global_obj_list);
@@ -28,7 +54,9 @@ void intel_atomic_global_obj_cleanup(struct drm_i915_private *dev_priv)
 
 	list_for_each_entry_safe(obj, next, &dev_priv->global_obj_list, head) {
 		list_del(&obj->head);
-		obj->funcs->atomic_destroy_state(obj, obj->state);
+
+		drm_WARN_ON(&dev_priv->drm, kref_read(&obj->state->ref) != 1);
+		intel_atomic_global_state_put(obj->state);
 	}
 }
 
@@ -97,10 +125,14 @@ intel_atomic_get_global_obj_state(struct intel_atomic_state *state,
 	if (!obj_state)
 		return ERR_PTR(-ENOMEM);
 
+	obj_state->obj = obj;
 	obj_state->changed = false;
 
+	kref_init(&obj_state->ref);
+
 	state->global_objs[index].state = obj_state;
-	state->global_objs[index].old_state = obj->state;
+	state->global_objs[index].old_state =
+		intel_atomic_global_state_get(obj->state);
 	state->global_objs[index].new_state = obj_state;
 	state->global_objs[index].ptr = obj;
 	obj_state->state = state;
@@ -163,7 +195,9 @@ void intel_atomic_swap_global_state(struct intel_atomic_state *state)
 		new_obj_state->state = NULL;
 
 		state->global_objs[i].state = old_obj_state;
-		obj->state = new_obj_state;
+
+		intel_atomic_global_state_put(obj->state);
+		obj->state = intel_atomic_global_state_get(new_obj_state);
 	}
 }
 

drivers/gpu/drm/i915/display/intel_global_state.c void intel_atomic_clear_global_state(struct intel_atomic_state *state) · line 210

Quoted 6 lines
drivers/gpu/drm/i915/display/intel_global_state.c +2 -1
@@ -172,10 +206,9 @@ void intel_atomic_clear_global_state(struct intel_atomic_state *state)
 	int i;
 
 	for (i = 0; i < state->num_global_objs; i++) {
-		struct intel_global_obj *obj = state->global_objs[i].ptr;
+		intel_atomic_global_state_put(state->global_objs[i].old_state);
+		intel_atomic_global_state_put(state->global_objs[i].new_state);

Shouldn't we clean old_state only?

As I understand in absence of any transaction you now have a pool of global_obj each has a state with single kref taken.

So when we are going to get a new state, we do +1 kref to old_state(which is current global obj->state) in order to prevent it being cleared by competing commit.
However the new state doesn't have any kref taken by that moment.
Then you swap do -1 kref for the old state and do +1 kref for new state,
which means that when you -1 kref again for old state in atomic_clear also,
it will be destroyed, however regarding the new state, as I understand
it still has only single kref grabbed when it was swapped,
so isn't it going to be now removed? unless we are lucky and somebody
haven't grabbed it already as an old_state in the next commit?

Stan

Quoted 35 lines
drivers/gpu/drm/i915/display/intel_global_state.c +0 -2
-		obj->funcs->atomic_destroy_state(obj,
-						 state->global_objs[i].state);
 		state->global_objs[i].ptr = NULL;
 		state->global_objs[i].state = NULL;
 		state->global_objs[i].old_state = NULL;
drivers/gpu/drm/i915/display/intel_global_state.h +3 -0
index e6163a469029..1f16fa3073c9 100644
@@ -6,6 +6,7 @@
 #ifndef __INTEL_GLOBAL_STATE_H__
 #define __INTEL_GLOBAL_STATE_H__
 
+#include <linux/kref.h>
 #include <linux/list.h>
 
 struct drm_i915_private;
@@ -54,7 +55,9 @@ struct intel_global_obj {
 		for_each_if(obj)
 
 struct intel_global_state {
+	struct intel_global_obj *obj;
 	struct intel_atomic_state *state;
+	struct kref ref;
 	bool changed;
 };
 
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#4 lore raw
Quoted 131 lines · Stanislav
Quoted 108 lines · Ville Syrjala

From: Ville Syrjälä <[email protected]>

While the current locking/serialization of the global state suffices for protecting the obj->state access and the actual hardware reprogramming, we do have a problem with accessing the old/new states during nonblocking commits.

The state computation and swap will be protected by the crtc locks, but the commit_tails can finish out of order, thus also causing the atomic states to be cleaned up out of order. This would mean the commit that started first but finished last has had its new state freed as the no-longer-needed old state by the other commit.

To fix this let's just refcount the states. obj->state amounts to one reference, and the intel_atomic_state holds extra references to both its new and old global obj states.

Fixes 0ef1905ecf2e ("drm/i915: Introduce better global state handling")
---
 .../gpu/drm/i915/display/intel_global_state.c | 45 ++++++++++++++++---
 .../gpu/drm/i915/display/intel_global_state.h |  3 ++
 2 files changed, 42 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/display/intel_global_state.c +37 -3
index 212d4ee68205..7a19215ad844 100644
@@ -10,6 +10,28 @@
 #include "intel_display_types.h"
 #include "intel_global_state.h"
 
+static void __intel_atomic_global_state_free(struct kref *kref)
+{
+	struct intel_global_state *obj_state =
+		container_of(kref, struct intel_global_state, ref);
+	struct intel_global_obj *obj = obj_state->obj;
+
+	obj->funcs->atomic_destroy_state(obj, obj_state);
+}
+
+static void intel_atomic_global_state_put(struct intel_global_state *obj_state)
+{
+	kref_put(&obj_state->ref, __intel_atomic_global_state_free);
+}
+
+static struct intel_global_state *
+intel_atomic_global_state_get(struct intel_global_state *obj_state)
+{
+	kref_get(&obj_state->ref);
+
+	return obj_state;
+}
+
 void intel_atomic_global_obj_init(struct drm_i915_private *dev_priv,
 				  struct intel_global_obj *obj,
 				  struct intel_global_state *state,
@@ -17,6 +39,10 @@ void intel_atomic_global_obj_init(struct drm_i915_private *dev_priv,
 {
 	memset(obj, 0, sizeof(*obj));
 
+	state->obj = obj;
+
+	kref_init(&state->ref);
+
 	obj->state = state;
 	obj->funcs = funcs;
 	list_add_tail(&obj->head, &dev_priv->global_obj_list);
@@ -28,7 +54,9 @@ void intel_atomic_global_obj_cleanup(struct drm_i915_private *dev_priv)
 
 	list_for_each_entry_safe(obj, next, &dev_priv->global_obj_list, head) {
 		list_del(&obj->head);
-		obj->funcs->atomic_destroy_state(obj, obj->state);
+
+		drm_WARN_ON(&dev_priv->drm, kref_read(&obj->state->ref) != 1);
+		intel_atomic_global_state_put(obj->state);
 	}
 }
 
@@ -97,10 +125,14 @@ intel_atomic_get_global_obj_state(struct intel_atomic_state *state,
 	if (!obj_state)
 		return ERR_PTR(-ENOMEM);
 
+	obj_state->obj = obj;
 	obj_state->changed = false;
 
+	kref_init(&obj_state->ref);
+
 	state->global_objs[index].state = obj_state;
-	state->global_objs[index].old_state = obj->state;
+	state->global_objs[index].old_state =
+		intel_atomic_global_state_get(obj->state);
 	state->global_objs[index].new_state = obj_state;
 	state->global_objs[index].ptr = obj;
 	obj_state->state = state;
@@ -163,7 +195,9 @@ void intel_atomic_swap_global_state(struct intel_atomic_state *state)
 		new_obj_state->state = NULL;
 
 		state->global_objs[i].state = old_obj_state;
-		obj->state = new_obj_state;
+
+		intel_atomic_global_state_put(obj->state);
+		obj->state = intel_atomic_global_state_get(new_obj_state);
 	}
 }
 

drivers/gpu/drm/i915/display/intel_global_state.c void intel_atomic_clear_global_state(struct intel_atomic_state *state) · line 210

Quoted 6 lines
drivers/gpu/drm/i915/display/intel_global_state.c +2 -1
@@ -172,10 +206,9 @@ void intel_atomic_clear_global_state(struct intel_atomic_state *state)
 	int i;
 
 	for (i = 0; i < state->num_global_objs; i++) {
-		struct intel_global_obj *obj = state->global_objs[i].ptr;
+		intel_atomic_global_state_put(state->global_objs[i].old_state);
+		intel_atomic_global_state_put(state->global_objs[i].new_state);

Shouldn't we clean old_state only?

As I understand in absence of any transaction you now have a pool of global_obj each has a state with single kref taken.

So when we are going to get a new state, we do +1 kref to old_state(which is current global obj->state) in order to prevent it being cleared by competing commit.
However the new state doesn't have any kref taken by that moment.
Then you swap do -1 kref for the old state and do +1 kref for new state,
which means that when you -1 kref again for old state in atomic_clear also,
it will be destroyed, however regarding the new state, as I understand
it still has only single kref grabbed when it was swapped,
so isn't it going to be now removed? unless we are lucky and somebody
haven't grabbed it already as an old_state in the next commit?

Quoted 1 line

Stan

Ah actually I got it - forgot that kref is init as 1.
But then you probably don't even need to increment kref for new state when swapping.
Before assigning new obj->state you release one kref in swap(which makes sense) Then you just do only intel_atomic_global_state_put(old_state) in atomic_clear and then no need in doing intel_atomic_global_state_get(new_state) during swap.
I.e we always call intel_atomic_global_state_get/put only regarding "old" obj->state and each new_state will be disposed when it becomes old_state.

Stan

Quoted 31 lines
Quoted 31 lines
drivers/gpu/drm/i915/display/intel_global_state.c +0 -2
-		obj->funcs->atomic_destroy_state(obj,
-						 state->global_objs[i].state);
 		state->global_objs[i].ptr = NULL;
 		state->global_objs[i].state = NULL;
 		state->global_objs[i].old_state = NULL;
drivers/gpu/drm/i915/display/intel_global_state.h +3 -0
index e6163a469029..1f16fa3073c9 100644
@@ -6,6 +6,7 @@
 #ifndef __INTEL_GLOBAL_STATE_H__
 #define __INTEL_GLOBAL_STATE_H__
 
+#include <linux/kref.h>
 #include <linux/list.h>
 
 struct drm_i915_private;
@@ -54,7 +55,9 @@ struct intel_global_obj {
 		for_each_if(obj)
 
 struct intel_global_state {
+	struct intel_global_obj *obj;
 	struct intel_atomic_state *state;
+	struct kref ref;
 	bool changed;
 };
 
Signature · 1 line
2.26.2
Quoted 8 lines
Quoted 4 lines

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#5 lore raw
Quoted 134 lines · Stanislav
Quoted 131 lines · Stanislav
Quoted 108 lines · Ville Syrjala

From: Ville Syrjälä <[email protected]>

While the current locking/serialization of the global state suffices for protecting the obj->state access and the actual hardware reprogramming, we do have a problem with accessing the old/new states during nonblocking commits.

The state computation and swap will be protected by the crtc locks, but the commit_tails can finish out of order, thus also causing the atomic states to be cleaned up out of order. This would mean the commit that started first but finished last has had its new state freed as the no-longer-needed old state by the other commit.

To fix this let's just refcount the states. obj->state amounts to one reference, and the intel_atomic_state holds extra references to both its new and old global obj states.

Fixes 0ef1905ecf2e ("drm/i915: Introduce better global state handling")
---
 .../gpu/drm/i915/display/intel_global_state.c | 45 ++++++++++++++++---
 .../gpu/drm/i915/display/intel_global_state.h |  3 ++
 2 files changed, 42 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/display/intel_global_state.c +37 -3
index 212d4ee68205..7a19215ad844 100644
@@ -10,6 +10,28 @@
 #include "intel_display_types.h"
 #include "intel_global_state.h"
 
+static void __intel_atomic_global_state_free(struct kref *kref)
+{
+	struct intel_global_state *obj_state =
+		container_of(kref, struct intel_global_state, ref);
+	struct intel_global_obj *obj = obj_state->obj;
+
+	obj->funcs->atomic_destroy_state(obj, obj_state);
+}
+
+static void intel_atomic_global_state_put(struct intel_global_state *obj_state)
+{
+	kref_put(&obj_state->ref, __intel_atomic_global_state_free);
+}
+
+static struct intel_global_state *
+intel_atomic_global_state_get(struct intel_global_state *obj_state)
+{
+	kref_get(&obj_state->ref);
+
+	return obj_state;
+}
+
 void intel_atomic_global_obj_init(struct drm_i915_private *dev_priv,
 				  struct intel_global_obj *obj,
 				  struct intel_global_state *state,
@@ -17,6 +39,10 @@ void intel_atomic_global_obj_init(struct drm_i915_private *dev_priv,
 {
 	memset(obj, 0, sizeof(*obj));
 
+	state->obj = obj;
+
+	kref_init(&state->ref);
+
 	obj->state = state;
 	obj->funcs = funcs;
 	list_add_tail(&obj->head, &dev_priv->global_obj_list);
@@ -28,7 +54,9 @@ void intel_atomic_global_obj_cleanup(struct drm_i915_private *dev_priv)
 
 	list_for_each_entry_safe(obj, next, &dev_priv->global_obj_list, head) {
 		list_del(&obj->head);
-		obj->funcs->atomic_destroy_state(obj, obj->state);
+
+		drm_WARN_ON(&dev_priv->drm, kref_read(&obj->state->ref) != 1);
+		intel_atomic_global_state_put(obj->state);
 	}
 }
 
@@ -97,10 +125,14 @@ intel_atomic_get_global_obj_state(struct intel_atomic_state *state,
 	if (!obj_state)
 		return ERR_PTR(-ENOMEM);
 
+	obj_state->obj = obj;
 	obj_state->changed = false;
 
+	kref_init(&obj_state->ref);
+
 	state->global_objs[index].state = obj_state;
-	state->global_objs[index].old_state = obj->state;
+	state->global_objs[index].old_state =
+		intel_atomic_global_state_get(obj->state);
 	state->global_objs[index].new_state = obj_state;
 	state->global_objs[index].ptr = obj;
 	obj_state->state = state;
@@ -163,7 +195,9 @@ void intel_atomic_swap_global_state(struct intel_atomic_state *state)
 		new_obj_state->state = NULL;
 
 		state->global_objs[i].state = old_obj_state;
-		obj->state = new_obj_state;
+
+		intel_atomic_global_state_put(obj->state);
+		obj->state = intel_atomic_global_state_get(new_obj_state);
 	}
 }
 

drivers/gpu/drm/i915/display/intel_global_state.c void intel_atomic_clear_global_state(struct intel_atomic_state *state) · line 210

Quoted 6 lines
drivers/gpu/drm/i915/display/intel_global_state.c +2 -1
@@ -172,10 +206,9 @@ void intel_atomic_clear_global_state(struct intel_atomic_state *state)
 	int i;
 
 	for (i = 0; i < state->num_global_objs; i++) {
-		struct intel_global_obj *obj = state->global_objs[i].ptr;
+		intel_atomic_global_state_put(state->global_objs[i].old_state);
+		intel_atomic_global_state_put(state->global_objs[i].new_state);

Shouldn't we clean old_state only?

As I understand in absence of any transaction you now have a pool of global_obj each has a state with single kref taken.

So when we are going to get a new state, we do +1 kref to old_state(which is current global obj->state) in order to prevent it being cleared by competing commit.
However the new state doesn't have any kref taken by that moment.
Then you swap do -1 kref for the old state and do +1 kref for new state,
which means that when you -1 kref again for old state in atomic_clear also,
it will be destroyed, however regarding the new state, as I understand
it still has only single kref grabbed when it was swapped,
so isn't it going to be now removed? unless we are lucky and somebody
haven't grabbed it already as an old_state in the next commit?

Quoted 1 line

Stan

Quoted 9 lines

Ah actually I got it - forgot that kref is init as 1.
But then you probably don't even need to increment kref for new state when swapping.
Before assigning new obj->state you release one kref in swap(which makes sense) Then you just do only intel_atomic_global_state_put(old_state) in atomic_clear and then no need in doing intel_atomic_global_state_get(new_state) during swap.
I.e we always call intel_atomic_global_state_get/put only regarding "old" obj->state and each new_state will be disposed when it becomes old_state.

IMO the approach of handing off references is just hard to follow. Better to just get/put explicitly whenever you assign a pointer. I already dislike handing off the original kref_init() reference, and almost added a get+put there too. Maybe I really should do that...

Quoted 42 lines

Stan

Quoted 39 lines
Quoted 35 lines
drivers/gpu/drm/i915/display/intel_global_state.c +0 -2
-		obj->funcs->atomic_destroy_state(obj,
-						 state->global_objs[i].state);
 		state->global_objs[i].ptr = NULL;
 		state->global_objs[i].state = NULL;
 		state->global_objs[i].old_state = NULL;
drivers/gpu/drm/i915/display/intel_global_state.h +3 -0
index e6163a469029..1f16fa3073c9 100644
@@ -6,6 +6,7 @@
 #ifndef __INTEL_GLOBAL_STATE_H__
 #define __INTEL_GLOBAL_STATE_H__
 
+#include <linux/kref.h>
 #include <linux/list.h>
 
 struct drm_i915_private;
@@ -54,7 +55,9 @@ struct intel_global_obj {
 		for_each_if(obj)
 
 struct intel_global_state {
+	struct intel_global_obj *obj;
 	struct intel_atomic_state *state;
+	struct kref ref;
 	bool changed;
 };
 
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Signature · 6 lines
Ville Syrjälä
Intel
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#6 lore raw
Quoted 146 lines · Ville Syrjälä
Quoted 134 lines · Stanislav
Quoted 131 lines · Stanislav
Quoted 108 lines · Ville Syrjala

From: Ville Syrjälä <[email protected]>

While the current locking/serialization of the global state suffices for protecting the obj->state access and the actual hardware reprogramming, we do have a problem with accessing the old/new states during nonblocking commits.

The state computation and swap will be protected by the crtc locks, but the commit_tails can finish out of order, thus also causing the atomic states to be cleaned up out of order. This would mean the commit that started first but finished last has had its new state freed as the no-longer-needed old state by the other commit.

To fix this let's just refcount the states. obj->state amounts to one reference, and the intel_atomic_state holds extra references to both its new and old global obj states.

Fixes 0ef1905ecf2e ("drm/i915: Introduce better global state handling")
---
 .../gpu/drm/i915/display/intel_global_state.c | 45 ++++++++++++++++---
 .../gpu/drm/i915/display/intel_global_state.h |  3 ++
 2 files changed, 42 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/display/intel_global_state.c +37 -3
index 212d4ee68205..7a19215ad844 100644
@@ -10,6 +10,28 @@
 #include "intel_display_types.h"
 #include "intel_global_state.h"
 
+static void __intel_atomic_global_state_free(struct kref *kref)
+{
+	struct intel_global_state *obj_state =
+		container_of(kref, struct intel_global_state, ref);
+	struct intel_global_obj *obj = obj_state->obj;
+
+	obj->funcs->atomic_destroy_state(obj, obj_state);
+}
+
+static void intel_atomic_global_state_put(struct intel_global_state *obj_state)
+{
+	kref_put(&obj_state->ref, __intel_atomic_global_state_free);
+}
+
+static struct intel_global_state *
+intel_atomic_global_state_get(struct intel_global_state *obj_state)
+{
+	kref_get(&obj_state->ref);
+
+	return obj_state;
+}
+
 void intel_atomic_global_obj_init(struct drm_i915_private *dev_priv,
 				  struct intel_global_obj *obj,
 				  struct intel_global_state *state,
@@ -17,6 +39,10 @@ void intel_atomic_global_obj_init(struct drm_i915_private *dev_priv,
 {
 	memset(obj, 0, sizeof(*obj));
 
+	state->obj = obj;
+
+	kref_init(&state->ref);
+
 	obj->state = state;
 	obj->funcs = funcs;
 	list_add_tail(&obj->head, &dev_priv->global_obj_list);
@@ -28,7 +54,9 @@ void intel_atomic_global_obj_cleanup(struct drm_i915_private *dev_priv)
 
 	list_for_each_entry_safe(obj, next, &dev_priv->global_obj_list, head) {
 		list_del(&obj->head);
-		obj->funcs->atomic_destroy_state(obj, obj->state);
+
+		drm_WARN_ON(&dev_priv->drm, kref_read(&obj->state->ref) != 1);
+		intel_atomic_global_state_put(obj->state);
 	}
 }
 
@@ -97,10 +125,14 @@ intel_atomic_get_global_obj_state(struct intel_atomic_state *state,
 	if (!obj_state)
 		return ERR_PTR(-ENOMEM);
 
+	obj_state->obj = obj;
 	obj_state->changed = false;
 
+	kref_init(&obj_state->ref);
+
 	state->global_objs[index].state = obj_state;
-	state->global_objs[index].old_state = obj->state;
+	state->global_objs[index].old_state =
+		intel_atomic_global_state_get(obj->state);
 	state->global_objs[index].new_state = obj_state;
 	state->global_objs[index].ptr = obj;
 	obj_state->state = state;
@@ -163,7 +195,9 @@ void intel_atomic_swap_global_state(struct intel_atomic_state *state)
 		new_obj_state->state = NULL;
 
 		state->global_objs[i].state = old_obj_state;
-		obj->state = new_obj_state;
+
+		intel_atomic_global_state_put(obj->state);
+		obj->state = intel_atomic_global_state_get(new_obj_state);
 	}
 }
 

drivers/gpu/drm/i915/display/intel_global_state.c void intel_atomic_clear_global_state(struct intel_atomic_state *state) · line 210

Quoted 6 lines
drivers/gpu/drm/i915/display/intel_global_state.c +2 -1
@@ -172,10 +206,9 @@ void intel_atomic_clear_global_state(struct intel_atomic_state *state)
 	int i;
 
 	for (i = 0; i < state->num_global_objs; i++) {
-		struct intel_global_obj *obj = state->global_objs[i].ptr;
+		intel_atomic_global_state_put(state->global_objs[i].old_state);
+		intel_atomic_global_state_put(state->global_objs[i].new_state);

Shouldn't we clean old_state only?

As I understand in absence of any transaction you now have a pool of global_obj each has a state with single kref taken.

So when we are going to get a new state, we do +1 kref to old_state(which is current global obj->state) in order to prevent it being cleared by competing commit.
However the new state doesn't have any kref taken by that moment.
Then you swap do -1 kref for the old state and do +1 kref for new state,
which means that when you -1 kref again for old state in atomic_clear also,
it will be destroyed, however regarding the new state, as I understand
it still has only single kref grabbed when it was swapped,
so isn't it going to be now removed? unless we are lucky and somebody
haven't grabbed it already as an old_state in the next commit?

Quoted 1 line

Stan

Quoted 9 lines

Ah actually I got it - forgot that kref is init as 1.
But then you probably don't even need to increment kref for new state when swapping.
Before assigning new obj->state you release one kref in swap(which makes sense) Then you just do only intel_atomic_global_state_put(old_state) in atomic_clear and then no need in doing intel_atomic_global_state_get(new_state) during swap.
I.e we always call intel_atomic_global_state_get/put only regarding "old" obj->state and each new_state will be disposed when it becomes old_state.

Quoted 4 lines

IMO the approach of handing off references is just hard to follow. Better to just get/put explicitly whenever you assign a pointer. I already dislike handing off the original kref_init() reference, and almost added a get+put there too. Maybe I really should do that...

Agree, tbh I don't like the idea that kref_init already implicitly holds a reference - it even confused me initially.
Typical smartpointer usually increments the ref only when assignment is done.

Quoted 35 lines
Quoted 35 lines

Stan

Quoted 31 lines
Quoted 31 lines
drivers/gpu/drm/i915/display/intel_global_state.c +0 -2
-		obj->funcs->atomic_destroy_state(obj,
-						 state->global_objs[i].state);
 		state->global_objs[i].ptr = NULL;
 		state->global_objs[i].state = NULL;
 		state->global_objs[i].old_state = NULL;
drivers/gpu/drm/i915/display/intel_global_state.h +3 -0
index e6163a469029..1f16fa3073c9 100644
@@ -6,6 +6,7 @@
 #ifndef __INTEL_GLOBAL_STATE_H__
 #define __INTEL_GLOBAL_STATE_H__
 
+#include <linux/kref.h>
 #include <linux/list.h>
 
 struct drm_i915_private;
@@ -54,7 +55,9 @@ struct intel_global_obj {
 		for_each_if(obj)
 
 struct intel_global_state {
+	struct intel_global_obj *obj;
 	struct intel_atomic_state *state;
+	struct kref ref;
 	bool changed;
 };
 
Signature · 1 line
2.26.2
Quoted 12 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Signature · 2 lines
Ville Syrjälä
Intel

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#7 lore raw
Quoted 152 lines · Stanislav
Quoted 146 lines · Ville Syrjälä
Quoted 134 lines · Stanislav
Quoted 131 lines · Stanislav
Quoted 108 lines · Ville Syrjala

From: Ville Syrjälä <[email protected]>

While the current locking/serialization of the global state suffices for protecting the obj->state access and the actual hardware reprogramming, we do have a problem with accessing the old/new states during nonblocking commits.

The state computation and swap will be protected by the crtc locks, but the commit_tails can finish out of order, thus also causing the atomic states to be cleaned up out of order. This would mean the commit that started first but finished last has had its new state freed as the no-longer-needed old state by the other commit.

To fix this let's just refcount the states. obj->state amounts to one reference, and the intel_atomic_state holds extra references to both its new and old global obj states.

Fixes 0ef1905ecf2e ("drm/i915: Introduce better global state handling")
---
 .../gpu/drm/i915/display/intel_global_state.c | 45 ++++++++++++++++---
 .../gpu/drm/i915/display/intel_global_state.h |  3 ++
 2 files changed, 42 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/display/intel_global_state.c +37 -3
index 212d4ee68205..7a19215ad844 100644
@@ -10,6 +10,28 @@
 #include "intel_display_types.h"
 #include "intel_global_state.h"
 
+static void __intel_atomic_global_state_free(struct kref *kref)
+{
+	struct intel_global_state *obj_state =
+		container_of(kref, struct intel_global_state, ref);
+	struct intel_global_obj *obj = obj_state->obj;
+
+	obj->funcs->atomic_destroy_state(obj, obj_state);
+}
+
+static void intel_atomic_global_state_put(struct intel_global_state *obj_state)
+{
+	kref_put(&obj_state->ref, __intel_atomic_global_state_free);
+}
+
+static struct intel_global_state *
+intel_atomic_global_state_get(struct intel_global_state *obj_state)
+{
+	kref_get(&obj_state->ref);
+
+	return obj_state;
+}
+
 void intel_atomic_global_obj_init(struct drm_i915_private *dev_priv,
 				  struct intel_global_obj *obj,
 				  struct intel_global_state *state,
@@ -17,6 +39,10 @@ void intel_atomic_global_obj_init(struct drm_i915_private *dev_priv,
 {
 	memset(obj, 0, sizeof(*obj));
 
+	state->obj = obj;
+
+	kref_init(&state->ref);
+
 	obj->state = state;
 	obj->funcs = funcs;
 	list_add_tail(&obj->head, &dev_priv->global_obj_list);
@@ -28,7 +54,9 @@ void intel_atomic_global_obj_cleanup(struct drm_i915_private *dev_priv)
 
 	list_for_each_entry_safe(obj, next, &dev_priv->global_obj_list, head) {
 		list_del(&obj->head);
-		obj->funcs->atomic_destroy_state(obj, obj->state);
+
+		drm_WARN_ON(&dev_priv->drm, kref_read(&obj->state->ref) != 1);
+		intel_atomic_global_state_put(obj->state);
 	}
 }
 
@@ -97,10 +125,14 @@ intel_atomic_get_global_obj_state(struct intel_atomic_state *state,
 	if (!obj_state)
 		return ERR_PTR(-ENOMEM);
 
+	obj_state->obj = obj;
 	obj_state->changed = false;
 
+	kref_init(&obj_state->ref);
+
 	state->global_objs[index].state = obj_state;
-	state->global_objs[index].old_state = obj->state;
+	state->global_objs[index].old_state =
+		intel_atomic_global_state_get(obj->state);
 	state->global_objs[index].new_state = obj_state;
 	state->global_objs[index].ptr = obj;
 	obj_state->state = state;
@@ -163,7 +195,9 @@ void intel_atomic_swap_global_state(struct intel_atomic_state *state)
 		new_obj_state->state = NULL;
 
 		state->global_objs[i].state = old_obj_state;
-		obj->state = new_obj_state;
+
+		intel_atomic_global_state_put(obj->state);
+		obj->state = intel_atomic_global_state_get(new_obj_state);
 	}
 }
 

drivers/gpu/drm/i915/display/intel_global_state.c void intel_atomic_clear_global_state(struct intel_atomic_state *state) · line 210

Quoted 6 lines
drivers/gpu/drm/i915/display/intel_global_state.c +2 -1
@@ -172,10 +206,9 @@ void intel_atomic_clear_global_state(struct intel_atomic_state *state)
 	int i;
 
 	for (i = 0; i < state->num_global_objs; i++) {
-		struct intel_global_obj *obj = state->global_objs[i].ptr;
+		intel_atomic_global_state_put(state->global_objs[i].old_state);
+		intel_atomic_global_state_put(state->global_objs[i].new_state);

Shouldn't we clean old_state only?

As I understand in absence of any transaction you now have a pool of global_obj each has a state with single kref taken.

So when we are going to get a new state, we do +1 kref to old_state(which is current global obj->state) in order to prevent it being cleared by competing commit.
However the new state doesn't have any kref taken by that moment.
Then you swap do -1 kref for the old state and do +1 kref for new state,
which means that when you -1 kref again for old state in atomic_clear also,
it will be destroyed, however regarding the new state, as I understand
it still has only single kref grabbed when it was swapped,
so isn't it going to be now removed? unless we are lucky and somebody
haven't grabbed it already as an old_state in the next commit?

Quoted 1 line

Stan

Quoted 9 lines

Ah actually I got it - forgot that kref is init as 1.
But then you probably don't even need to increment kref for new state when swapping.
Before assigning new obj->state you release one kref in swap(which makes sense) Then you just do only intel_atomic_global_state_put(old_state) in atomic_clear and then no need in doing intel_atomic_global_state_get(new_state) during swap.
I.e we always call intel_atomic_global_state_get/put only regarding "old" obj->state and each new_state will be disposed when it becomes old_state.

Quoted 4 lines

IMO the approach of handing off references is just hard to follow. Better to just get/put explicitly whenever you assign a pointer. I already dislike handing off the original kref_init() reference, and almost added a get+put there too. Maybe I really should do that...

Quoted 7 lines

Agree, tbh I don't like the idea that kref_init already implicitly holds a reference - it even confused me initially.
Typical smartpointer usually increments the ref only when assignment is done.

Ta. Pushed. Hopefully few rounds of ci will show whether this fixes things. Though I've also seen some vma related use-after-frees in the logs as well, so there may be further problems elsewhere...

Signature · 6 lines
Ville Syrjälä
Intel
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx