Skip to contentSkip to conversation

iGVT-g implementation in i915

RFC In review Zhi Wang 7 participants 49 messages Jan 28, 2016 – Feb 16, 2016 Page 1 of 2

Subsystems (7)
DRM drivers x86 DRM misc Intel i915 +3 more subsystems
lore
Share & tools
lore

Zhi Wang

author
#0 lore raw

cover iGVT-g implementation in i915

This patchset implements the core part of Intel GVT-g [1] for graphics virtualization. The patchset consists of two subsets: One is the device model implementation of vGPUs for virtual machines (patch 09 -
29) Another is the needed i915 native driver changes for that implementation (patch 01 - 08). We hope to get feedbacks of the native driver changes from i915 developers, while the device model part can be a reference to see how we utilize those changes.

Background
----------

Back to 1 year ago, we ever sent RFC of iGVT-g changes to native i915 drivers, so called "host mediation"[2]. The architecture is that host i915 driver runs on top of iGVT-g device model, and device model controls hardware directly if it is enabled. After discussion, we agreed an architecture change to let native driver on host controls hardware directly, and integrate virtualization changes into i915.

Last year we followed this direction to remove host mediation, and move some virtualization related logics into i915. For instance, the interrupt part [3]. Later we found that the i915 changes can be much reduced if we virtualize more resources for VMs. Now we are happy to see that the i915 changes are very limited.

Major i915 changes
------------------

In the new architecture, device-model works as a client. We do not mediate any i915 accesses of hardware resources. VM's vGPU resources keep virtual as much as possible. One example is interrupt. vGPU's interrupt will be triggered either through virtual timer (vblnk) or workload status changes (ctx_switch), not directly bound to hardware interrupts. Hence we do not need to touch i915 interrupt framework at all.

The remaining i915 changes are:

- vGPU guest framebuffer GEM object support

Wrap a guest framebuffer into a host i915 GEM object. So host graphics application could be able to access guest display surface easily via DRM/GL APIs.

Related commits:

drm/i915: Support vGPU guest framebuffer GEM object

- Host i915 GGTT graphics memory space/fence registers balloon

i915 host owns limited GGTT graphics memory space/fence registers under GVT environment, others are managed by GVT resource allocator, which allocates/de-allocates these resources for GVT guest.

Related commits:

drm/i915: Introduce host graphics memory balloon

- GVT context creation

GVT workload scheduler needs special host LRC contexts, so called "shadow LRC context" to submit guest workload to host i915. During the guest workload submission. GVT fills the shadow LRC context with the content guest LRC context: engine context is copied without changes, ring context is mostly owned by host i915, except the PPGTT root pointers, which will be filled with the shadow PPGTT page table root pointers managed by GVT-g.

Related commits:

drm/i915: gem: Introduce i915_gem_create_gvt_context() drm/i915: Ondemand populate context addressing mode bit drm/i915: Do not initialize the engine state of GVT context drm/i915: Do not populate PPGTT root pointers for GVT context

- GVT context scheduling

Since GVT guest workload will also be dispatched through i915 GEM submission system, the GVT context mentioned above is also scheduled by host i915 LRC routines. The workload inside a GVT context may come from different GVT guests, some render MMIOs have to be configured with the guest render MMIOs before the GVT context gets kicked out. It's hard to submit a GVT context with a host LRC context in the same ELSP combo.

The LRC context submission policy for GVT context is adjusted as:

* If ELSP submission routines pick a GVT context, then stop to pick next context. (Single ELSP submission - for easy render MMIOs save/restore)

* If ELSP submission routines pick a i915 context, and the next context is a GVT context, then stop pick the next context. (Single ELSP submission - we cannot skip the GVT context, as the seqno of workload in GVT context may be passed, but the workload in GVT context haven't be executed yet)

Related commits:
drm/i915: GVT context scheduling

Difference from community release
---------------------------------

This patchset is different from regular iGVT-g code release[4], which is still based on old host-mediated architecture. Furthermore, this patchset only supports BDW whereas code release supports HSW/BDW/SKL. We will add SKL support later based on this RFC code and HSW support will be dropped.

Internally we tested this RFC patchset with both linux and windows VM and the architecture changes work fine.

Acknowledgement
---------------

iGVT-g implementation is several years effort and many people contributed to the code. There names are not here yet. In later formal patchset we will reflect individual's contribution.

Meanwhile, in the previous iGVT-g related discussion, Daniel, Chris and Joonas ever gave very good inputs. We appreciate them and look forward to more comments/suggestions from community.

We are trying to get more familiar with i915 and willing to adopt suggestions to keep improving. We hope to work with community together to make iGVT-g a great component in i915 to support graphics virtualization. Thanks!

Reference
---------

[1] https://01.org/igvt-g
[2] http://lists.freedesktop.org/archives/intel-gfx/2014-September/053098.html
[3] http://lists.freedesktop.org/archives/intel-gfx/2015-September/075397.html
[4] http://lists.freedesktop.org/archives/intel-gfx/2016-January/086372.html
*** BLURB HERE ***

Bing Niu (2):
drm/i915: Introduce host graphics memory balloon for gvt drm/i915: gvt: Full display virtualization

Niu Bing (1):
drm/i915: gvt: vGPU framebuffer format decoder

Niu,Bing (1):
drm/i915: gvt: vGPU MMIO register emulation

Yi Sun (1):
drm/i915: gvt: Introduce GVT control interface

Yulei Zhang (2):
drm/i915: gvt: Resource allocator
drm/i915: gvt: vGPU command scanner

Zhi Wang (21):
drm/i915/gvt: Introduce the basic architecture of GVT-g drm/i915: Introduce GVT context creation API drm/i915: Ondemand populate context addressing mode bit drm/i915: Do not populate PPGTT root pointers for GVT context drm/i915: Do not initialize the engine state of GVT context drm/i915: GVT context scheduling
drm/i915: gvt: Basic mmio emulation state
drm/i915: gvt: update PVINFO page definition in i915_vgpu.h drm/i915: gvt: vGPU life cycle management
drm/i915: gvt: trace stub
drm/i915: gvt: vGPU interrupt emulation framework drm/i915: gvt: vGPU graphics memory emulation framework drm/i915: gvt: Generic MPT framework
gvt: Xen hypervisor GVT-g MPT module
drm/i915: gvt: vGPU configuration emulation drm/i915: gvt: vGPU OpRegion emulation
drm/i915: gvt: Full execlist status emulation drm/i915: gvt: vGPU execlist workload submission drm/i915: gvt: workload scheduler
drm/i915: gvt: vGPU schedule policy framework drm/i915: gvt: vGPU context switch

Zhiyuan Lv (1):
drm/i915: Support vGPU guest framebuffer GEM object

 arch/x86/include/asm/xen/hypercall.h      |    7 +
 arch/x86/include/asm/xen/interface.h      |    4 +
 arch/x86/xen/mmu.c                        |   83 ++
 drivers/gpu/drm/i915/Kconfig              |   16 +
 drivers/gpu/drm/i915/Makefile             |    3 +
 drivers/gpu/drm/i915/gvt/Makefile         |    8 +
 drivers/gpu/drm/i915/gvt/aperture_gm.c    |  225 +++
 drivers/gpu/drm/i915/gvt/cfg_space.c      |  190 +++
 drivers/gpu/drm/i915/gvt/cfg_space.h      |   33 +
 drivers/gpu/drm/i915/gvt/cmd_parser.c     | 2018 +++++++++++++++++++++++++++
 drivers/gpu/drm/i915/gvt/cmd_parser.h     |  466 +++++++
 drivers/gpu/drm/i915/gvt/control.c        |  181 +++
 drivers/gpu/drm/i915/gvt/debug.h          |  103 ++
 drivers/gpu/drm/i915/gvt/display.c        |  233 ++++
 drivers/gpu/drm/i915/gvt/display.h        |  129 ++
 drivers/gpu/drm/i915/gvt/edid.c           |  493 +++++++
 drivers/gpu/drm/i915/gvt/edid.h           |  184 +++
 drivers/gpu/drm/i915/gvt/execlist.c       |  612 +++++++++
 drivers/gpu/drm/i915/gvt/execlist.h       |  179 +++
 drivers/gpu/drm/i915/gvt/fb_decoder.c     |  325 +++++
 drivers/gpu/drm/i915/gvt/fb_decoder.h     |  110 ++
 drivers/gpu/drm/i915/gvt/gtt.c            | 1912 ++++++++++++++++++++++++++
 drivers/gpu/drm/i915/gvt/gtt.h            |  305 ++++
 drivers/gpu/drm/i915/gvt/gvt.c            |  446 ++++++
 drivers/gpu/drm/i915/gvt/gvt.h            |  715 ++++++++++
 drivers/gpu/drm/i915/gvt/handlers.c       | 2141 +++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/gvt/hypercall.h      |   56 +
 drivers/gpu/drm/i915/gvt/instance.c       |  272 ++++
 drivers/gpu/drm/i915/gvt/interrupt.c      |  721 ++++++++++
 drivers/gpu/drm/i915/gvt/interrupt.h      |  257 ++++
 drivers/gpu/drm/i915/gvt/mmio.c           |  545 ++++++++
 drivers/gpu/drm/i915/gvt/mmio.h           |  105 ++
 drivers/gpu/drm/i915/gvt/mpt.h            |  169 +++
 drivers/gpu/drm/i915/gvt/opregion.c       |  314 +++++
 drivers/gpu/drm/i915/gvt/opregion.h       |   34 +
 drivers/gpu/drm/i915/gvt/params.c         |   54 +
 drivers/gpu/drm/i915/gvt/params.h         |   49 +
 drivers/gpu/drm/i915/gvt/perf.h           |   69 +
 drivers/gpu/drm/i915/gvt/reg.h            |  889 ++++++++++++
 drivers/gpu/drm/i915/gvt/render.c         |  104 ++
 drivers/gpu/drm/i915/gvt/render.h         |   31 +
 drivers/gpu/drm/i915/gvt/sched_policy.c   |  295 ++++
 drivers/gpu/drm/i915/gvt/sched_policy.h   |   48 +
 drivers/gpu/drm/i915/gvt/scheduler.c      |  501 +++++++
 drivers/gpu/drm/i915/gvt/scheduler.h      |   85 ++
 drivers/gpu/drm/i915/gvt/trace.h          |  260 ++++
 drivers/gpu/drm/i915/gvt/trace_points.c   |   29 +
 drivers/gpu/drm/i915/gvt/utility.c        |   91 ++
 drivers/gpu/drm/i915/i915_dma.c           |   20 +
 drivers/gpu/drm/i915/i915_drv.h           |   29 +
 drivers/gpu/drm/i915/i915_gem.c           |    3 +
 drivers/gpu/drm/i915/i915_gem_context.c   |   38 +
 drivers/gpu/drm/i915/i915_gem_gtt.c       |    4 +-
 drivers/gpu/drm/i915/i915_gem_gvtbuffer.c |  247 ++++
 drivers/gpu/drm/i915/i915_vgpu.c          |   16 +-
 drivers/gpu/drm/i915/i915_vgpu.h          |   94 +-
 drivers/gpu/drm/i915/intel_lrc.c          |   64 +-
 drivers/xen/Kconfig                       |    5 +
 drivers/xen/Makefile                      |    6 +
 drivers/xen/xengt.c                       | 1153 ++++++++++++++++
 include/uapi/drm/i915_drm.h               |   38 +
 include/xen/interface/hvm/hvm_op.h        |  177 ++-
 include/xen/interface/hvm/ioreq.h         |  132 ++
 include/xen/interface/memory.h            |   28 +
 include/xen/interface/xen.h               |  107 ++
 include/xen/xen-ops.h                     |    5 +
 66 files changed, 18222 insertions(+), 43 deletions(-)
 create mode 100644 drivers/gpu/drm/i915/gvt/Makefile
 create mode 100644 drivers/gpu/drm/i915/gvt/aperture_gm.c
 create mode 100644 drivers/gpu/drm/i915/gvt/cfg_space.c
 create mode 100644 drivers/gpu/drm/i915/gvt/cfg_space.h
 create mode 100644 drivers/gpu/drm/i915/gvt/cmd_parser.c
 create mode 100644 drivers/gpu/drm/i915/gvt/cmd_parser.h
 create mode 100644 drivers/gpu/drm/i915/gvt/control.c
 create mode 100644 drivers/gpu/drm/i915/gvt/debug.h
 create mode 100644 drivers/gpu/drm/i915/gvt/display.c
 create mode 100644 drivers/gpu/drm/i915/gvt/display.h
 create mode 100644 drivers/gpu/drm/i915/gvt/edid.c
 create mode 100644 drivers/gpu/drm/i915/gvt/edid.h
 create mode 100644 drivers/gpu/drm/i915/gvt/execlist.c
 create mode 100644 drivers/gpu/drm/i915/gvt/execlist.h
 create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.c
 create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.h
 create mode 100644 drivers/gpu/drm/i915/gvt/gtt.c
 create mode 100644 drivers/gpu/drm/i915/gvt/gtt.h
 create mode 100644 drivers/gpu/drm/i915/gvt/gvt.c
 create mode 100644 drivers/gpu/drm/i915/gvt/gvt.h
 create mode 100644 drivers/gpu/drm/i915/gvt/handlers.c
 create mode 100644 drivers/gpu/drm/i915/gvt/hypercall.h
 create mode 100644 drivers/gpu/drm/i915/gvt/instance.c
 create mode 100644 drivers/gpu/drm/i915/gvt/interrupt.c
 create mode 100644 drivers/gpu/drm/i915/gvt/interrupt.h
 create mode 100644 drivers/gpu/drm/i915/gvt/mmio.c
 create mode 100644 drivers/gpu/drm/i915/gvt/mmio.h
 create mode 100644 drivers/gpu/drm/i915/gvt/mpt.h
 create mode 100644 drivers/gpu/drm/i915/gvt/opregion.c
 create mode 100644 drivers/gpu/drm/i915/gvt/opregion.h
 create mode 100644 drivers/gpu/drm/i915/gvt/params.c
 create mode 100644 drivers/gpu/drm/i915/gvt/params.h
 create mode 100644 drivers/gpu/drm/i915/gvt/perf.h
 create mode 100644 drivers/gpu/drm/i915/gvt/reg.h
 create mode 100644 drivers/gpu/drm/i915/gvt/render.c
 create mode 100644 drivers/gpu/drm/i915/gvt/render.h
 create mode 100644 drivers/gpu/drm/i915/gvt/sched_policy.c
 create mode 100644 drivers/gpu/drm/i915/gvt/sched_policy.h
 create mode 100644 drivers/gpu/drm/i915/gvt/scheduler.c
 create mode 100644 drivers/gpu/drm/i915/gvt/scheduler.h
 create mode 100644 drivers/gpu/drm/i915/gvt/trace.h
 create mode 100644 drivers/gpu/drm/i915/gvt/trace_points.c
 create mode 100644 drivers/gpu/drm/i915/gvt/utility.c
 create mode 100644 drivers/gpu/drm/i915/i915_gem_gvtbuffer.c
 create mode 100644 drivers/xen/xengt.c
 create mode 100644 include/xen/interface/hvm/ioreq.h
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#1 lore raw

RFC 1/29 drm/i915/gvt: Introduce the basic architecture of GVT-g

This patch introduces the very basic framework of GVT-g device model, includes basic prototypes, definitions, initialization.

---
 arch/x86/include/asm/xen/interface.h  |   3 +
 drivers/gpu/drm/i915/Kconfig          |  16 ++
 drivers/gpu/drm/i915/Makefile         |   2 +
 drivers/gpu/drm/i915/gvt/Makefile     |   5 +
 drivers/gpu/drm/i915/gvt/debug.h      |  72 +++++++
 drivers/gpu/drm/i915/gvt/fb_decoder.c |  34 ++++
 drivers/gpu/drm/i915/gvt/fb_decoder.h | 110 ++++++++++
 drivers/gpu/drm/i915/gvt/gvt.c        | 366 ++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/gvt/gvt.h        | 130 ++++++++++++
 drivers/gpu/drm/i915/gvt/hypercall.h  |  30 +++
 drivers/gpu/drm/i915/gvt/mpt.h        |  97 +++++++++
 drivers/gpu/drm/i915/gvt/params.c     |  29 +++
 drivers/gpu/drm/i915/gvt/params.h     |  34 ++++
 drivers/gpu/drm/i915/gvt/reg.h        |  34 ++++
 drivers/gpu/drm/i915/i915_dma.c       |  19 ++
 drivers/gpu/drm/i915/i915_drv.h       |   6 +
 drivers/gpu/drm/i915/i915_vgpu.h      |   3 +
 include/xen/interface/xen.h           |   1 +
 18 files changed, 991 insertions(+)
 create mode 100644 drivers/gpu/drm/i915/gvt/Makefile
 create mode 100644 drivers/gpu/drm/i915/gvt/debug.h
 create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.c
 create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.h
 create mode 100644 drivers/gpu/drm/i915/gvt/gvt.c
 create mode 100644 drivers/gpu/drm/i915/gvt/gvt.h
 create mode 100644 drivers/gpu/drm/i915/gvt/hypercall.h
 create mode 100644 drivers/gpu/drm/i915/gvt/mpt.h
 create mode 100644 drivers/gpu/drm/i915/gvt/params.c
 create mode 100644 drivers/gpu/drm/i915/gvt/params.h
 create mode 100644 drivers/gpu/drm/i915/gvt/reg.h
arch/x86/include/asm/xen/interface.h +3 -0
index 62ca03e..6ff4986 100644
@@ -73,6 +73,9 @@
 #endif
 
 #ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+
 /* Explicitly size integers that represent pfns in the public interface
  * with Xen so that on ARM we can have one ABI that works for 32 and 64
  * bit guests. */
drivers/gpu/drm/i915/Kconfig +16 -0
index 051eab3..89ff723 100644
@@ -47,3 +47,19 @@ config DRM_I915_PRELIMINARY_HW_SUPPORT
 	  option changes the default for that module option.
 
 	  If in doubt, say "N".
+
+config I915_GVT
+        tristate "GVT-g host driver"
+        depends on DRM_I915
+        select IRQ_WORK
+        default y
+        help
+          Enabling GVT-g mediated graphics passthrough technique for Intel i915
+          based integrated graphics card. With GVT-g, it's possible to have one
+          integrated i915 device shared by multiple VMs. Performance critical
+          opterations such as apperture accesses and ring buffer operations
+          are pass-throughed to VM, with a minimal set of conflicting resources
+          (e.g. display settings) mediated by vGT driver. The benefit of vGT
+          is on both the performance, given that each VM could directly operate
+          its aperture space and submit commands like running on native, and
+          the feature completeness, given that a true GEN hardware is exposed.
drivers/gpu/drm/i915/Makefile +2 -0
index 0851de07..d4df410 100644
@@ -91,6 +91,8 @@ i915-y += dvo_ch7017.o \
 	  intel_sdvo.o \
 	  intel_tv.o
 
+obj-$(CONFIG_I915_GVT)  += gvt/
+
 # virtual gpu code
 i915-y += i915_vgpu.o
 
drivers/gpu/drm/i915/gvt/Makefile new file +5 -0
new file mode 100644
index 0000000..6935b78
@@ -0,0 +1,5 @@
+GVT_SOURCE := gvt.o params.o fb_decoder.o
+
+ccflags-y			+= -I$(src) -I$(src)/.. -Wall -Werror -Wno-unused-function
+i915_gvt-y			:= $(GVT_SOURCE)
+obj-$(CONFIG_I915_GVT)		+= i915_gvt.o
drivers/gpu/drm/i915/gvt/debug.h new file +72 -0

72 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/fb_decoder.c new file +34 -0
new file mode 100644
index 0000000..a219819
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include "gvt.h"
+
+int gvt_decode_fb_format(struct pgt_device *pdev, int vmid, struct gvt_fb_format *fb)
+{
+	return 0;
+}
+
+int gvt_fb_notifier_call_chain(unsigned long val, void *data)
+{
+	return 0;
+}
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +110 -0

110 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/gvt.c new file +366 -0

366 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/gvt.h new file +130 -0

130 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/hypercall.h new file +30 -0
new file mode 100644
index 0000000..0a41874
@@ -0,0 +1,30 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_HYPERCALL_H_
+#define _GVT_HYPERCALL_H_
+
+struct gvt_kernel_dm {
+};
+
+#endif /* _GVT_HYPERCALL_H_ */
drivers/gpu/drm/i915/gvt/mpt.h new file +97 -0

97 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/params.c new file +29 -0
new file mode 100644
index 0000000..dfc33c3
@@ -0,0 +1,29 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include "gvt.h"
+
+struct gvt_kernel_params gvt = {
+	.enable = true,
+	.debug = 0,
+};
drivers/gpu/drm/i915/gvt/params.h new file +34 -0
new file mode 100644
index 0000000..d2955b9
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_PARAMS_H_
+#define _GVT_PARAMS_H_
+
+struct gvt_kernel_params {
+	bool enable;
+	int debug;
+};
+
+extern struct gvt_kernel_params gvt;
+
+#endif
drivers/gpu/drm/i915/gvt/reg.h new file +34 -0
new file mode 100644
index 0000000..d363b74
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_REG_H
+#define _GVT_REG_H
+
+#define GVT_CFG_SPACE_SZ	256
+#define GVT_BAR_NUM		4
+
+#define GVT_REG_CFG_SPACE_BAR0	0x10
+#define GVT_REG_CFG_SPACE_BAR1	0x18
+#define GVT_REG_CFG_SPACE_BAR2	0x20
+
+#endif
drivers/gpu/drm/i915/i915_dma.c +19 -0
index 4725e8d..eca8e50 100644
@@ -943,6 +943,10 @@ int i915_driver_load(struct drm_device *dev, unsigned long flags)
 
 	intel_uncore_init(dev);
 
+	dev_priv->vgpu.host_private_data = gvt_create_pgt_device(dev_priv);
+	if(intel_gvt_host_active(dev))
+		DRM_INFO("GVT-g is running in host mode\n");
+
 	ret = i915_gem_gtt_init(dev);
 	if (ret)
 		goto out_freecsr;
@@ -1067,6 +1071,13 @@ int i915_driver_load(struct drm_device *dev, unsigned long flags)
 		goto out_power_well;
 	}
 
+	if (intel_gvt_host_active(dev)) {
+		if (!gvt_post_init_pgt_device(dev_priv->vgpu.host_private_data)) {
+			DRM_ERROR("failed to post init pgt device\n");
+			goto out_power_well;
+		}
+	}
+
 	/*
 	 * Notify a valid surface after modesetting,
 	 * when running inside a VM.
@@ -1117,6 +1128,10 @@ out_gtt:
 	i915_global_gtt_cleanup(dev);
 out_freecsr:
 	intel_csr_ucode_fini(dev_priv);
+	if (intel_gvt_host_active(dev)) {
+		gvt_destroy_pgt_device(dev_priv->vgpu.host_private_data);
+		dev_priv->vgpu.host_private_data = NULL;
+	}
 	intel_uncore_fini(dev);
 	pci_iounmap(dev->pdev, dev_priv->regs);
 put_bridge:
@@ -1165,6 +1180,10 @@ int i915_driver_unload(struct drm_device *dev)
 
 	intel_modeset_cleanup(dev);
 
+	if (intel_gvt_host_active(dev)) {
+		gvt_destroy_pgt_device(dev_priv->vgpu.host_private_data);
+		dev_priv->vgpu.host_private_data = NULL;
+	}
 	/*
 	 * free the memory space allocated for the child device
 	 * config parsed from VBT
drivers/gpu/drm/i915/i915_drv.h +6 -0
index 01cc982..db3c79b 100644
@@ -1673,6 +1673,7 @@ struct i915_workarounds {
 
 struct i915_virtual_gpu {
 	bool active;
+	void *host_private_data;
 };
 
 struct i915_execbuffer_params {
@@ -2747,6 +2748,11 @@ static inline bool intel_vgpu_active(struct drm_device *dev)
 	return to_i915(dev)->vgpu.active;
 }
 
+static inline bool intel_gvt_host_active(struct drm_device *dev)
+{
+	return to_i915(dev)->vgpu.host_private_data ? true : false;
+}
+
 void
 i915_enable_pipestat(struct drm_i915_private *dev_priv, enum pipe pipe,
 		     u32 status_mask);
drivers/gpu/drm/i915/i915_vgpu.h +3 -0
index 3c83b47..942490a 100644
@@ -113,5 +113,8 @@ struct vgt_if {
 extern void i915_check_vgpu(struct drm_device *dev);
 extern int intel_vgt_balloon(struct drm_device *dev);
 extern void intel_vgt_deballoon(void);
+extern void *gvt_create_pgt_device(struct drm_i915_private *dev_priv);
+extern bool gvt_post_init_pgt_device(void *private_data);
+extern void gvt_destroy_pgt_device(void *private_data);
 
 #endif /* _I915_VGPU_H_ */
include/xen/interface/xen.h +1 -0
index d133112..78a38f1 100644
@@ -28,6 +28,7 @@
 #define __XEN_PUBLIC_XEN_H__
 
 #include <asm/xen/interface.h>
+#include <linux/types.h>
 
 /*
  * XEN "SYSTEM CALLS" (a.k.a. HYPERCALLS).
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer
#2 lore raw

Re: [RFC 01/29] drm/i915/gvt: Introduce the basic architecture of GVT-g

Hi,

TL;DR Overall, we have same problem as with the scheduler series, there is too much placeholder stuff for easy review. Just squash enough code into one commit so it actually does something logical that can be reviewed and then extend it later. Then it can be reviewed and pushed. Just splitting the code down to achieve smaller patches is not the right thing to do.

Comments on the overall code: You need to document all header file functions (in the source files), and it is good to document the static functions within a file too, to make future maintenance easier.

It is not about splitting the code down to small chunks, but splitting it down to small *logical* chunks. It doesn't make sense to commit dozens of empty bodied functions for review, and then later add their code.

If you add functions, only add them at a patch that takes them into use too, unless we're talking about general purpose shared code. And also remember to add the function body and documentation header. If you simply add a "return 0;" or similar function body, do add a comment to why the function does not exist and when it will.

Then, there is a trend of having a boolean return values in the code. When possible, it should rather be int and the cause for failure should be propagated from the last level all the way to up (-ENOMEN etc.). This way debugging becomes easier and if new error conditions appear, there is less of a maintenance burden to add the propagation later.

Finally, make sure to look at the existing driver parts and
https://www.kernel.org/doc/Documentation/CodingStyle
for proper coding style. There are lots of whitespace fixes needed in this series, like array initializations.

I hope to see this first patch rerolled so that you squash some of the later commits into it so that all functions have a body and you add documentation for the functions so I can both see what it should do and what it actually does. Only reroll the first patch, to keep the iterative step smaller. Lets only then continue with the rest of the series once we've reached a consensus on the formatting and style basics.

See more comments below.

Quoted 42 lines · Zhi Wang

This patch introduces the very basic framework of GVT-g device model, includes basic prototypes, definitions, initialization.
---
 arch/x86/include/asm/xen/interface.h  |   3 + drivers/gpu/drm/i915/Kconfig          |  16 ++ drivers/gpu/drm/i915/Makefile         |   2 + drivers/gpu/drm/i915/gvt/Makefile     |   5 + drivers/gpu/drm/i915/gvt/debug.h      |  72 +++++++ drivers/gpu/drm/i915/gvt/fb_decoder.c |  34 ++++ drivers/gpu/drm/i915/gvt/fb_decoder.h | 110 ++++++++++ drivers/gpu/drm/i915/gvt/gvt.c        | 366 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/gvt/gvt.h        | 130 ++++++++++++ drivers/gpu/drm/i915/gvt/hypercall.h  |  30 +++ drivers/gpu/drm/i915/gvt/mpt.h        |  97 +++++++++ drivers/gpu/drm/i915/gvt/params.c     |  29 +++ drivers/gpu/drm/i915/gvt/params.h     |  34 ++++ drivers/gpu/drm/i915/gvt/reg.h        |  34 ++++ drivers/gpu/drm/i915/i915_dma.c       |  19 ++ drivers/gpu/drm/i915/i915_drv.h       |   6 + drivers/gpu/drm/i915/i915_vgpu.h      |   3 + include/xen/interface/xen.h           |   1 +

 18 files changed, 991 insertions(+)

 create mode 100644 drivers/gpu/drm/i915/gvt/Makefile create mode 100644 drivers/gpu/drm/i915/gvt/debug.h create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.c create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.h create mode 100644 drivers/gpu/drm/i915/gvt/gvt.c create mode 100644 drivers/gpu/drm/i915/gvt/gvt.h create mode 100644 drivers/gpu/drm/i915/gvt/hypercall.h create mode 100644 drivers/gpu/drm/i915/gvt/mpt.h create mode 100644 drivers/gpu/drm/i915/gvt/params.c create mode 100644 drivers/gpu/drm/i915/gvt/params.h create mode 100644 drivers/gpu/drm/i915/gvt/reg.h

arch/x86/include/asm/xen/interface.h +0 -0
index 62ca03e..6ff4986 100644
@@ -73,6 +73,9 @@
  #endif


 #ifndef __ASSEMBLY__

arch/x86/include/asm/xen/interface.h line 78

Quoted 3 lines
arch/x86/include/asm/xen/interface.h +3 -0
+
+#include 
+

I don't follow why this would need to be added if the file is not modified otherwise. Each header should only include what they use.

If this is an existing bug (that xen/interface.h can not be included without including linux/types.h), it should be split to a separate patch and sent to Xen team. Same for include/xen/interface/xen.h.

Quoted 11 lines

 /* Explicitly size integers that represent pfns in the public interface
  * with Xen so that on ARM we can have one ABI that works for 32 and 64
  * bit guests. */

drivers/gpu/drm/i915/Kconfig +0 -0
index 051eab3..89ff723 100644
@@ -47,3 +47,19 @@ config DRM_I915_PRELIMINARY_HW_SUPPORT

    option changes the default for that module option.
    If in doubt, say "N".

drivers/gpu/drm/i915/Kconfig config DRM_I915_PRELIMINARY_HW_SUPPORT · line 55

Quoted 6 lines
drivers/gpu/drm/i915/Kconfig +6 -0
+
+config I915_GVT
+        tristate "GVT-g host driver"
+        depends on DRM_I915
+        select IRQ_WORK
+        default y

Defaulting to "n" would make sense initially.

Quoted 72 lines

+        help
+          Enabling GVT-g mediated graphics passthrough technique for Intel i915 +          based integrated graphics card. With GVT-g, it's possible to have one +          integrated i915 device shared by multiple VMs. Performance critical +          opterations such as apperture accesses and ring buffer operations +          are pass-throughed to VM, with a minimal set of conflicting resources +          (e.g. display settings) mediated by vGT driver. The benefit of vGT +          is on both the performance, given that each VM could directly operate +          its aperture space and submit commands like running on native, and +          the feature completeness, given that a true GEN hardware is exposed.

drivers/gpu/drm/i915/Makefile +0 -0
index 0851de07..d4df410 100644
@@ -91,6 +91,8 @@ i915-y += dvo_ch7017.o \
  	  intel_sdvo.o \

    intel_tv.o

drivers/gpu/drm/i915/Makefile +2 -0
+obj-$(CONFIG_I915_GVT)  += gvt/
+
 # virtual gpu code
 i915-y += i915_vgpu.o
 
drivers/gpu/drm/i915/gvt/Makefile new file +5 -0
new file mode 100644
index 0000000..6935b78
@@ -0,0 +1,5 @@
+GVT_SOURCE := gvt.o params.o fb_decoder.o
+
+ccflags-y			+= -I$(src) -I$(src)/.. -Wall -Werror -Wno-unused-function
+i915_gvt-y			:= $(GVT_SOURCE)
+obj-$(CONFIG_I915_GVT)		+= i915_gvt.o
drivers/gpu/drm/i915/gvt/debug.h new file +32 -0
new file mode 100644
index 0000000..18e1467
@@ -0,0 +1,72 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef __GVT_DEBUG_H__
+#define __GVT_DEBUG_H__
+
+#define ASSERT(x)                                                       \
+        do {                                                            \
+                if (!(x)) {                                             \
+                        printk("Assert at %s line %d\n",                \
+                                __FILE__, __LINE__);                    \
+                }                                                       \

drivers/gpu/drm/i915/gvt/debug.h line 42

Quoted 10 lines
drivers/gpu/drm/i915/gvt/debug.h new file +10 -0
+        } while (0);
+
+#define ASSERT_NUM(x, y)                                                \
+        do {                                                            \
+                if (!(x)) {                                             \
+                        printk("Assert at %s line %d para 0x%llx\n",    \
+                                __FILE__, __LINE__, (u64)y);            \
+                }                                                       \
+        } while (0);
+

There already is WARN_ON (and i915_drv.h modifies it a little). Do not introduce custom functions like this, if the existing ones need improvement, improve them.

Quoted 20 lines
drivers/gpu/drm/i915/gvt/debug.h new file +20 -0
+#define gvt_info(fmt, args...) \
+	printk(KERN_INFO"[GVT-g] "fmt"\n", ##args)
+
+#define gvt_err(fmt, args...) \
+	printk(KERN_ERR"%s() - %d: "fmt"\n", __func__, __LINE__, ##args)
+
+#define gvt_warn(fmt, args...) \
+	printk(KERN_WARNING"%s() - %d: "fmt"\n", __func__, __LINE__, ##args)
+
+#define gvt_dbg(level, fmt, args...) do { \
+		if (gvt.debug & level) \
+			printk(KERN_DEBUG"%s() - %d: "fmt"\n", __func__, __LINE__, ##args); \
+	}while(0)
+
+enum {
+	GVT_DBG_CORE = (1 << 0),
+	GVT_DBG_MM = (1 << 1),
+	GVT_DBG_IRQ = (1 << 2),
+};
+

drivers/gpu/drm/i915/gvt/debug.h line 72

Quoted 10 lines
drivers/gpu/drm/i915/gvt/debug.h new file +10 -0
+#define gvt_dbg_core(fmt, args...) \
+	gvt_dbg(GVT_DBG_CORE, fmt, ##args)
+
+#define gvt_dbg_mm(fmt, args...) \
+	gvt_dbg(GVT_DBG_MM, fmt, ##args)
+
+#define gvt_dbg_irq(fmt, args...) \
+	gvt_dbg(GVT_DBG_IRQ, fmt, ##args)
+
+#endif

This should be integrated better to DRM debugging options, custom debugging code only for i915 was rejected a while ago.

Quoted 30 lines
drivers/gpu/drm/i915/gvt/fb_decoder.c new file +24 -0
new file mode 100644
index 0000000..a219819
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include "gvt.h"

drivers/gpu/drm/i915/gvt/fb_decoder.c line 34

Quoted 10 lines
drivers/gpu/drm/i915/gvt/fb_decoder.c new file +10 -0
+
+int gvt_decode_fb_format(struct pgt_device *pdev, int vmid, struct gvt_fb_format *fb)
+{
+	return 0;
+}
+
+int gvt_fb_notifier_call_chain(unsigned long val, void *data)
+{
+	return 0;
+}

Kerneldoc missing for these functions. It is all the same to squash later patches to introduce the code to these functions already, reviewing utility functions with no kerneldoc and no body makes it somewhat difficult to see the big picture.

Quoted 90 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +84 -0
new file mode 100644
index 0000000..2c29ed4
@@ -0,0 +1,110 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_FB_DECODER_H_
+#define _GVT_FB_DECODER_H_
+
+typedef enum {
+	FB_MODE_SET_START = 1,
+	FB_MODE_SET_END,
+	FB_DISPLAY_FLIP,
+}gvt_fb_event_t;
+
+typedef enum {
+	DDI_PORT_NONE	= 0,
+	DDI_PORT_B	= 1,
+	DDI_PORT_C	= 2,
+	DDI_PORT_D	= 3,
+	DDI_PORT_E	= 4
+} ddi_port_t;
+
+struct pgt_device;
+
+struct gvt_fb_notify_msg {
+	unsigned vm_id;
+	unsigned pipe_id; /* id starting from 0 */
+	unsigned plane_id; /* primary, cursor, or sprite */
+};
+
+/* color space conversion and gamma correction are not included */
+struct gvt_primary_plane_format {
+	u8	enabled;	/* plane is enabled */
+	u8	tiled;		/* X-tiled */
+	u8	bpp;		/* bits per pixel */
+	u32	hw_format;	/* format field in the PRI_CTL register */
+	u32	drm_format;	/* format in DRM definition */
+	u32	base;		/* framebuffer base in graphics memory */
+	u32	x_offset;	/* in pixels */
+	u32	y_offset;	/* in lines */
+	u32	width;		/* in pixels */
+	u32	height;		/* in lines */
+	u32	stride;		/* in bytes */
+};
+
+struct gvt_sprite_plane_format {
+	u8	enabled;	/* plane is enabled */
+	u8	tiled;		/* X-tiled */
+	u8	bpp;		/* bits per pixel */
+	u32	hw_format;	/* format field in the SPR_CTL register */
+	u32	drm_format;	/* format in DRM definition */
+	u32	base;		/* sprite base in graphics memory */
+	u32	x_pos;		/* in pixels */
+	u32	y_pos;		/* in lines */
+	u32	x_offset;	/* in pixels */
+	u32	y_offset;	/* in lines */
+	u32	width;		/* in pixels */
+	u32	height;		/* in lines */
+};
+
+struct gvt_cursor_plane_format {
+	u8	enabled;
+	u8	mode;		/* cursor mode select */
+	u8	bpp;		/* bits per pixel */
+	u32	drm_format;	/* format in DRM definition */
+	u32	base;		/* cursor base in graphics memory */

drivers/gpu/drm/i915/gvt/fb_decoder.h line 94

Quoted 10 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +10 -0
+	u32	x_pos;		/* in pixels */
+	u32	y_pos;		/* in lines */
+	u8	x_sign;		/* X Position Sign */
+	u8	y_sign;		/* Y Position Sign */
+	u32	width;		/* in pixels */
+	u32	height;		/* in lines */
+	u32	x_hot;		/* in pixels */
+	u32	y_hot;		/* in pixels */
+};
+

The above structs have a lot in common, would it make sense to have the common members + plane type and then union for the plane type specific data. I suspect having it all split this way will lead to more utility functions somewhere.

Quoted 46 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +16 -0
+struct gvt_pipe_format {
+	struct gvt_primary_plane_format	primary;
+	struct gvt_sprite_plane_format	sprite;
+	struct gvt_cursor_plane_format	cursor;
+	ddi_port_t ddi_port;  /* the DDI port that the pipe is connected to */
+};
+
+struct gvt_fb_format{
+	struct gvt_pipe_format	pipes[I915_MAX_PIPES];
+};
+
+extern int gvt_fb_notifier_call_chain(unsigned long val, void *data);
+extern int gvt_decode_fb_format(struct pgt_device *pdev, int vmid,
+				struct gvt_fb_format *fb);
+
+#endif
drivers/gpu/drm/i915/gvt/gvt.c new file +24 -0
new file mode 100644
index 0000000..041d10f
@@ -0,0 +1,366 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include 

drivers/gpu/drm/i915/gvt/gvt.c line 34

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +10 -0
+#include 
+
+#include "gvt.h"
+
+struct gvt_host gvt_host;
+
+extern struct gvt_kernel_dm xengt_kdm;
+extern struct gvt_kernel_dm kvmgt_kdm;
+
+static const char *supported_hypervisors[] = {

should be "static const char * const".

Quoted 9 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +9 -0
+	[GVT_HYPERVISOR_TYPE_XEN] = "Xen Hypervisor",
+	[GVT_HYPERVISOR_TYPE_KVM] = "KVM",
+};
+
+static bool gvt_init_host(void)
+{
+	struct gvt_host *host = &gvt_host;
+
+	if (!gvt.enable) {

drivers/gpu/drm/i915/gvt/gvt.c line 53

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +10 -0
+		gvt_dbg_core("GVT-g has been disabled by kernel parameter");
+		return false;
+	}
+
+	if (host->initialized) {
+		gvt_err("GVT-g has already been initialized!");
+		return false;
+	}
+
+	if (xen_initial_domain()) {

Shouldn't Xen code be CONFIG_XEN_DOM0 and CONFIG_XEN #ifdef protected?

drivers/gpu/drm/i915/gvt/gvt.c line 65

Quoted 12 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +12 -0
+		/* Xen Dom0 */
+		host->kdm = try_then_request_module(symbol_get(xengt_kdm), "xengt");
+		host->hypervisor_type = GVT_HYPERVISOR_TYPE_XEN;
+	} else if(xen_domain()) {
+		/* Xen DomU */
+		return false;
+	} else {
+		/* not in Xen. Try KVMGT */
+		host->kdm = try_then_request_module(symbol_get(kvmgt_kdm), "kvm");
+		host->hypervisor_type = GVT_HYPERVISOR_TYPE_KVM;
+	}
+

Why do we need to keep track of hypervisor type, are there plenty of hypervisor specific behaviour differences? If it is just for printing the below debug, I think it's better to move the debug into above code that detects the hypervisor, wrap them with appropriate CONFIG #ifdefs.

Quoted 31 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +31 -0
+	if (!host->kdm)
+		return false;
+
+	if (!hypervisor_detect_host())
+		return false;
+
+	gvt_info("Running with hypervisor %s in host mode",
+			supported_hypervisors[host->hypervisor_type]);
+
+	idr_init(&host->device_idr);
+	mutex_init(&host->device_idr_lock);
+
+	host->initialized = true;
+	return true;
+}
+
+static bool init_device_info(struct pgt_device *pdev)
+{
+	struct gvt_device_info *info = &pdev->device_info;
+
+	if (!IS_BROADWELL(pdev->dev_priv)) {
+		gvt_err("Unsupported GEN device");
+		return false;
+	}
+
+	if (IS_BROADWELL(pdev->dev_priv)) {
+		info->max_gtt_gm_sz = (1UL << 32);
+		/*
+		 * The layout of BAR0 in BDW:
+		 * |< - MMIO 2MB ->|<- Reserved 6MB ->|<- MAX GTT 8MB->|
+		 *

drivers/gpu/drm/i915/gvt/gvt.c line 106

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +10 -0
+		 * GTT offset in BAR0 starts from 8MB to 16MB, and
+		 * Whatever GTT size is configured in BIOS,
+		 * the size of BAR0 is always 16MB. The actual configured
+		 * GTT size can be found in GMCH_CTRL.
+		 */
+		info->gtt_start_offset = (1UL << 23);
+		info->max_gtt_size = (1UL << 23);
+		info->gtt_entry_size = 8;
+		info->gtt_entry_size_shift = 3;
+		info->gmadr_bytes_in_cmd = 8;

Would this information be useful in a header as #defines too?

drivers/gpu/drm/i915/gvt/gvt.c line 116

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +10 -0
+	}
+
+	gvt_info("Device info:");
+	printk("        max_gtt_gm_sz: %llx\n", info->max_gtt_gm_sz);
+	printk("        max_gtt_size: %x\n", info->max_gtt_size);
+	printk("        gtt_size_entry: %x\n", info->gtt_entry_size);
+	printk("        gtt_entry_size_shift: %x\n", info->gtt_entry_size_shift);
+	printk("        gtt_start_offset: %x\n", info->gtt_start_offset);
+	printk("        gtt_end_offset: %x\n", info->gtt_end_offset);
+

Just put this to kind of stuff to i915_debugfs.c.

drivers/gpu/drm/i915/gvt/gvt.c line 127

Quoted 11 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +11 -0
+	return true;
+}
+
+static void init_initial_cfg_space_state(struct pgt_device *pdev)
+{
+	struct pci_dev *pci_dev = pdev->dev_priv->dev->pdev;
+	int i;
+
+	gvt_dbg_core("init initial cfg space, id %d", pdev->id);
+
+	for (i = 0; i < GVT_CFG_SPACE_SZ; i += 4)

+= sizeof(...)

drivers/gpu/drm/i915/gvt/gvt.c line 131

Quoted 4 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +4 -0
+		pci_read_config_dword(pci_dev, i,
+				(u32 *)&pdev->initial_cfg_space[i]);
+
+	for (i = 0; i < 3; i++) {

No magic numbers make a #define for 3 and give it a descriptive name.

Quoted 22 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +22 -0
+		pdev->bar_size[i] = pci_resource_len(pci_dev, i * 2);
+		gvt_info("bar %d size: %llx", i, pdev->bar_size[i]);
+	}
+}
+
+static void clean_initial_mmio_state(struct pgt_device *pdev)
+{
+	if (pdev->gttmmio_va) {
+		iounmap(pdev->gttmmio_va);
+		pdev->gttmmio_va = NULL;
+	}
+
+	if (pdev->gmadr_va) {
+		iounmap(pdev->gmadr_va);
+		pdev->gmadr_va = NULL;
+	}
+}
+
+static bool init_initial_mmio_state(struct pgt_device *pdev)
+{
+	u64 bar0, bar1;
+

drivers/gpu/drm/i915/gvt/gvt.c line 163

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +10 -0
+	gvt_dbg_core("init initial mmio state, id %d", pdev->id);
+
+	bar0 = *(u64 *)&pdev->initial_cfg_space[GVT_REG_CFG_SPACE_BAR0];
+	bar1 = *(u64 *)&pdev->initial_cfg_space[GVT_REG_CFG_SPACE_BAR1];
+
+	pdev->gttmmio_base = bar0 & ~0xf;
+	pdev->mmio_size = 2 * 1024 * 1024;
+	pdev->reg_num = pdev->mmio_size / 4;
+	pdev->gmadr_base = bar1 & ~0xf;
+

Many magic numbers.

drivers/gpu/drm/i915/gvt/gvt.c line 166

Quoted 3 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +3 -0
+	pdev->gttmmio_va = ioremap(pdev->gttmmio_base, pdev->bar_size[0]);
+	if (!pdev->gttmmio_va) {
+		gvt_err("fail to map GTTMMIO BAR.");

These should be if(WARN_ON(...))

Quoted 27 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +27 -0
+		return false;
+	}
+
+	pdev->gmadr_va = ioremap(pdev->gmadr_base, pdev->bar_size[2]);
+	if (!pdev->gmadr_va) {
+		gvt_err("fail to map GMADR BAR.");
+		goto err;
+	}
+
+	gvt_info("bar0: 0x%llx, bar1: 0x%llx", bar0, bar1);
+	gvt_info("mmio size: %x", pdev->mmio_size);
+	gvt_info("gttmmio: 0x%llx, gmadr: 0x%llx", pdev->gttmmio_base, pdev->gmadr_base);
+	gvt_info("gttmmio_va: %p", pdev->gttmmio_va);
+	gvt_info("gmadr_va: %p", pdev->gmadr_va);
+
+	return true;
+err:
+	clean_initial_mmio_state(pdev);
+	return false;
+}
+
+static int gvt_service_thread(void *data)
+{
+	struct pgt_device *pdev = (struct pgt_device *)data;
+	int r;
+
+	gvt_dbg_core("service thread start, pgt %d", pdev->id);

drivers/gpu/drm/i915/gvt/gvt.c line 203

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +10 -0
+
+	while(!kthread_should_stop()) {
+		r = wait_event_interruptible(pdev->service_thread_wq,
+				kthread_should_stop() || pdev->service_request);
+
+		if (kthread_should_stop())
+			break;
+
+		if (r) {
+			gvt_warn("service thread is waken up by unexpected signal.");

Should be WARN_ONCE, to avoid future disasters with CI.

Quoted 256 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +163 -0
+			continue;
+		}
+	}
+	return 0;
+}
+
+static void clean_service_thread(struct pgt_device *pdev)
+{
+	if (pdev->service_thread) {
+		kthread_stop(pdev->service_thread);
+		pdev->service_thread = NULL;
+	}
+}
+
+static bool init_service_thread(struct pgt_device *pdev)
+{
+	init_waitqueue_head(&pdev->service_thread_wq);
+
+	pdev->service_thread = kthread_run(gvt_service_thread,
+			pdev, "gvt_service_thread%d", pdev->id);
+
+	if (!pdev->service_thread) {
+		gvt_err("fail to start service thread.");
+		return false;
+	}
+
+	return true;
+}
+
+static void clean_pgt_device(struct pgt_device *pdev)
+{
+	clean_service_thread(pdev);
+	clean_initial_mmio_state(pdev);
+}
+
+static bool init_pgt_device(struct pgt_device *pdev, struct drm_i915_private *dev_priv)
+{
+	if (!init_device_info(pdev))
+		return false;
+
+	init_initial_cfg_space_state(pdev);
+
+	if (!init_initial_mmio_state(pdev))
+		goto err;
+
+	if (!init_service_thread(pdev))
+		goto err;
+
+	return true;
+err:
+	clean_pgt_device(pdev);
+	return false;
+}
+
+static bool post_init_pgt_device(struct pgt_device *pdev)
+{
+	return true;
+}
+
+static void free_pgt_device(struct pgt_device *pdev)
+{
+	struct gvt_host *host = &gvt_host;
+
+	mutex_lock(&host->device_idr_lock);
+	idr_remove(&host->device_idr, pdev->id);
+	mutex_unlock(&host->device_idr_lock);
+
+	vfree(pdev);
+}
+
+static struct pgt_device *alloc_pgt_device(struct drm_i915_private *dev_priv)
+{
+	struct gvt_host *host = &gvt_host;
+	struct pgt_device *pdev = NULL;
+
+	pdev = vzalloc(sizeof(*pdev));
+	if (!pdev) {
+		gvt_err("fail to allocate memory for pgt device.");
+		return NULL;
+	}
+
+	mutex_lock(&host->device_idr_lock);
+	pdev->id = idr_alloc(&host->device_idr, pdev, 0, 0, GFP_KERNEL);
+	mutex_unlock(&host->device_idr_lock);
+
+	if (pdev->id < 0) {
+		gvt_err("fail to allocate pgt device id.");
+		goto err;
+	}
+
+	mutex_init(&pdev->lock);
+	pdev->dev_priv = dev_priv;
+	idr_init(&pdev->instance_idr);
+
+	return pdev;
+err:
+	free_pgt_device(pdev);
+	return NULL;
+}
+
+void gvt_destroy_pgt_device(void *private_data)
+{
+	struct pgt_device *pdev = (struct pgt_device *)private_data;
+
+	clean_pgt_device(pdev);
+	free_pgt_device(pdev);
+}
+
+void *gvt_create_pgt_device(struct drm_i915_private *dev_priv)
+{
+	struct pgt_device *pdev = NULL;
+	struct gvt_host *host = &gvt_host;
+
+	if (!host->initialized && !gvt_init_host()) {
+		gvt_err("gvt_init_host fail");
+		return NULL;
+	}
+
+	gvt_dbg_core("create new pgt device, i915 dev_priv: %p", dev_priv);
+
+	pdev = alloc_pgt_device(dev_priv);
+	if (!pdev) {
+		gvt_err("fail to allocate memory for pgt device.");
+		goto err;
+	}
+
+	gvt_dbg_core("init pgt device, id %d", pdev->id);
+
+	if (!init_pgt_device(pdev, dev_priv)) {
+		gvt_err("fail to init physical device state.");
+		goto err;
+	}
+
+	gvt_dbg_core("pgt device creation done, id %d", pdev->id);
+
+	return pdev;
+err:
+	if (pdev) {
+		gvt_destroy_pgt_device(pdev);
+		pdev = NULL;
+	}
+	return NULL;
+}
+
+bool gvt_post_init_pgt_device(void *private_data)
+{
+	struct pgt_device *pdev = (struct pgt_device *)private_data;
+	struct gvt_host *host = &gvt_host;
+
+	if (!host->initialized) {
+		gvt_err("gvt_host haven't been initialized.");
+		return false;
+	}
+
+	gvt_dbg_core("post init pgt device %d", pdev->id);
+
+	if (!post_init_pgt_device(pdev)) {
+		gvt_err("fail to post init physical device state.");
+		return false;
+	}
+
+	return true;
+}
drivers/gpu/drm/i915/gvt/gvt.h new file +87 -0
new file mode 100644
index 0000000..6c85bba
@@ -0,0 +1,130 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_H_
+#define _GVT_H_
+
+#include "i915_drv.h"
+#include "i915_vgpu.h"
+
+#include "debug.h"
+#include "params.h"
+#include "reg.h"
+#include "hypercall.h"
+#include "mpt.h"
+#include "fb_decoder.h"
+
+#define GVT_MAX_VGPU 8
+
+enum {
+	GVT_HYPERVISOR_TYPE_XEN = 0,
+	GVT_HYPERVISOR_TYPE_KVM,
+};
+
+struct gvt_host {
+	bool initialized;
+	int hypervisor_type;
+	struct mutex device_idr_lock;
+	struct idr device_idr;
+	struct gvt_kernel_dm *kdm;
+};
+
+extern struct gvt_host gvt_host;
+
+/* Describe the limitation of HW.*/
+struct gvt_device_info {
+	u64 max_gtt_gm_sz;
+	u32 gtt_start_offset;
+	u32 gtt_end_offset;
+	u32 max_gtt_size;
+	u32 gtt_entry_size;
+	u32 gtt_entry_size_shift;
+	u32 gmadr_bytes_in_cmd;
+};
+
+struct vgt_device {
+	int id;
+	int vm_id;
+	struct pgt_device *pdev;
+	bool warn_untrack;
+};
+
+struct pgt_device {
+	struct mutex lock;
+	int id;
+
+	struct drm_i915_private *dev_priv;
+	struct idr instance_idr;
+
+	struct gvt_device_info device_info;
+
+	u8 initial_cfg_space[GVT_CFG_SPACE_SZ];
+	u64 bar_size[GVT_BAR_NUM];
+
+	u64 gttmmio_base;
+	void *gttmmio_va;
+
+	u64 gmadr_base;

drivers/gpu/drm/i915/gvt/gvt.h line 97

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +10 -0
+	void *gmadr_va;
+
+	u32 mmio_size;
+	u32 reg_num;
+
+	wait_queue_head_t service_thread_wq;
+	struct task_struct *service_thread;
+	unsigned long service_request;
+};
+

Here-->

Quoted 22 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +22 -0
+static inline u32 gvt_mmio_read(struct pgt_device *pdev,
+		u32 reg)
+{
+	struct drm_i915_private *dev_priv = pdev->dev_priv;
+	i915_reg_t tmp = {.reg = reg};
+	return I915_READ(tmp);
+}
+
+static inline void gvt_mmio_write(struct pgt_device *pdev,
+		u32 reg, u32 val)
+{
+	struct drm_i915_private *dev_priv = pdev->dev_priv;
+	i915_reg_t tmp = {.reg = reg};
+	I915_WRITE(tmp, val);
+}
+
+static inline u64 gvt_mmio_read64(struct pgt_device *pdev,
+		u32 reg)
+{
+	struct drm_i915_private *dev_priv = pdev->dev_priv;
+	i915_reg_t tmp = {.reg = reg};
+	return I915_READ64(tmp);

drivers/gpu/drm/i915/gvt/gvt.h line 129

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +10 -0
+}
+
+static inline void gvt_mmio_write64(struct pgt_device *pdev,
+		u32 reg, u64 val)
+{
+	struct drm_i915_private *dev_priv = pdev->dev_priv;
+	i915_reg_t tmp = {.reg = reg};
+	I915_WRITE64(tmp, val);
+}
+

<-- Why? The i915_reg_t type was added to avoid problems, this code is not used anywhere, and it has no documentation, so I can not review it.

I wrote comments at the top of the post.

Regards, Joonas

Quoted 355 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +1 -0
+#endif
drivers/gpu/drm/i915/gvt/hypercall.h new file +30 -0
new file mode 100644
index 0000000..0a41874
@@ -0,0 +1,30 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_HYPERCALL_H_
+#define _GVT_HYPERCALL_H_
+
+struct gvt_kernel_dm {
+};
+
+#endif /* _GVT_HYPERCALL_H_ */
drivers/gpu/drm/i915/gvt/mpt.h new file +97 -0
new file mode 100644
index 0000000..bbe4465
@@ -0,0 +1,97 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_MPT_H_
+#define _GVT_MPT_H_
+
+struct vgt_device;
+
+static inline unsigned long hypervisor_g2m_pfn(struct vgt_device *vgt,
+	unsigned long g_pfn)
+{
+	return 0;
+}
+
+static inline int hypervisor_pause_domain(struct vgt_device *vgt)
+{
+	return 0;
+}
+
+static inline int hypervisor_shutdown_domain(struct vgt_device *vgt)
+{
+	return 0;
+}
+
+static inline int hypervisor_set_trap_area(struct vgt_device *vgt,
+	uint64_t start, uint64_t end, bool map)
+{
+	return 0;
+}
+
+static inline bool hypervisor_detect_host(void)
+{
+	return false;
+}
+
+static inline int hypervisor_virt_to_mfn(void *addr)
+{
+	return 0;
+}
+
+static inline void *hypervisor_mfn_to_virt(int mfn)
+{
+	return NULL;
+}
+
+static inline void hypervisor_inject_msi(struct vgt_device *vgt)
+{
+	return;
+}
+
+static inline int hypervisor_hvm_init(struct vgt_device *vgt)
+{
+	return 0;
+}
+
+static inline void hypervisor_hvm_exit(struct vgt_device *vgt)
+{
+}
+
+static inline void *hypervisor_gpa_to_va(struct vgt_device *vgt, unsigned long gpa)
+{
+	return NULL;
+}
+
+static inline bool hypervisor_read_va(struct vgt_device *vgt, void *va,
+		void *val, int len, int atomic)
+{
+	return false;
+}
+
+static inline bool hypervisor_write_va(struct vgt_device *vgt, void *va,
+		void *val, int len, int atomic)
+{
+	return false;
+}
+
+#endif /* _GVT_MPT_H_ */
drivers/gpu/drm/i915/gvt/params.c new file +29 -0
new file mode 100644
index 0000000..dfc33c3
@@ -0,0 +1,29 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include "gvt.h"
+
+struct gvt_kernel_params gvt = {
+	.enable = true,
+	.debug = 0,
+};
drivers/gpu/drm/i915/gvt/params.h new file +34 -0
new file mode 100644
index 0000000..d2955b9
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_PARAMS_H_
+#define _GVT_PARAMS_H_
+
+struct gvt_kernel_params {
+	bool enable;
+	int debug;
+};
+
+extern struct gvt_kernel_params gvt;
+
+#endif
drivers/gpu/drm/i915/gvt/reg.h new file +34 -0
new file mode 100644
index 0000000..d363b74
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_REG_H
+#define _GVT_REG_H
+
+#define GVT_CFG_SPACE_SZ	256
+#define GVT_BAR_NUM		4
+
+#define GVT_REG_CFG_SPACE_BAR0	0x10
+#define GVT_REG_CFG_SPACE_BAR1	0x18
+#define GVT_REG_CFG_SPACE_BAR2	0x20
+
+#endif
drivers/gpu/drm/i915/i915_dma.c +0 -0
index 4725e8d..eca8e50 100644
@@ -943,6 +943,10 @@ int i915_driver_load(struct drm_device *dev, unsigned long flags)


  intel_uncore_init(dev);

drivers/gpu/drm/i915/i915_dma.c +4 -0
+	dev_priv->vgpu.host_private_data = gvt_create_pgt_device(dev_priv);
+	if(intel_gvt_host_active(dev))
+		DRM_INFO("GVT-g is running in host mode\n");
+

  ret = i915_gem_gtt_init(dev);
  if (ret)
  goto out_freecsr;

drivers/gpu/drm/i915/i915_dma.c +0 -0
@@ -1067,6 +1071,13 @@ int i915_driver_load(struct drm_device *dev, unsigned long flags)
  		goto out_power_well;
  	}

drivers/gpu/drm/i915/i915_dma.c +7 -0
+	if (intel_gvt_host_active(dev)) {
+		if (!gvt_post_init_pgt_device(dev_priv->vgpu.host_private_data)) {
+			DRM_ERROR("failed to post init pgt device\n");
+			goto out_power_well;
+		}
+	}
+

  /*
   * Notify a valid surface after modesetting,
   * when running inside a VM.

drivers/gpu/drm/i915/i915_dma.c +0 -0
@@ -1117,6 +1128,10 @@ out_gtt:
  	i915_global_gtt_cleanup(dev);

 out_freecsr:
  intel_csr_ucode_fini(dev_priv);

drivers/gpu/drm/i915/i915_dma.c +4 -0
+	if (intel_gvt_host_active(dev)) {
+		gvt_destroy_pgt_device(dev_priv->vgpu.host_private_data);
+		dev_priv->vgpu.host_private_data = NULL;
+	}

  intel_uncore_fini(dev);
  pci_iounmap(dev->pdev, dev_priv->regs);
 put_bridge:

drivers/gpu/drm/i915/i915_dma.c +0 -0
@@ -1165,6 +1180,10 @@ int i915_driver_unload(struct drm_device *dev)


  intel_modeset_cleanup(dev);

drivers/gpu/drm/i915/i915_dma.c +4 -0
+	if (intel_gvt_host_active(dev)) {
+		gvt_destroy_pgt_device(dev_priv->vgpu.host_private_data);
+		dev_priv->vgpu.host_private_data = NULL;
+	}

  /*
   * free the memory space allocated for the child device
   * config parsed from VBT

drivers/gpu/drm/i915/i915_drv.h +0 -0
index 01cc982..db3c79b 100644
@@ -1673,6 +1673,7 @@ struct i915_workarounds {


 struct i915_virtual_gpu {
  bool active;

drivers/gpu/drm/i915/i915_drv.h +1 -0
+	void *host_private_data;

 };

 struct i915_execbuffer_params {

drivers/gpu/drm/i915/i915_drv.h +0 -0
@@ -2747,6 +2748,11 @@ static inline bool intel_vgpu_active(struct drm_device *dev)
  	return to_i915(dev)->vgpu.active;
  }

drivers/gpu/drm/i915/i915_drv.h +5 -0
+static inline bool intel_gvt_host_active(struct drm_device *dev)
+{
+	return to_i915(dev)->vgpu.host_private_data ? true : false;
+}
+

 void
 i915_enable_pipestat(struct drm_i915_private *dev_priv, enum pipe pipe, u32 status_mask);

drivers/gpu/drm/i915/i915_vgpu.h +3 -0
index 3c83b47..942490a 100644
@@ -113,5 +113,8 @@ struct vgt_if {
  extern void i915_check_vgpu(struct drm_device *dev);
  extern int intel_vgt_balloon(struct drm_device *dev);
  extern void intel_vgt_deballoon(void);
+extern void *gvt_create_pgt_device(struct drm_i915_private *dev_priv);
+extern bool gvt_post_init_pgt_device(void *private_data);
+extern void gvt_destroy_pgt_device(void *private_data);


 #endif /* _I915_VGPU_H_ */

include/xen/interface/xen.h +0 -0
index d133112..78a38f1 100644
@@ -28,6 +28,7 @@
  #define __XEN_PUBLIC_XEN_H__


 #include
+#include

 /*
  * XEN "SYSTEM CALLS" (a.k.a. HYPERCALLS).

Signature · 7 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw

Re: [RFC 01/29] drm/i915/gvt: Introduce the basic architecture of GVT-g

Quoted 43 lines · Joonas Lahtinen

Hi,

TL;DR Overall, we have same problem as with the scheduler series, there is too much placeholder stuff for easy review. Just squash enough code into one commit so it actually does something logical that can be reviewed and then extend it later. Then it can be reviewed and pushed. Just splitting the code down to achieve smaller patches is not the right thing to do.

Comments on the overall code: You need to document all header file functions (in the source files), and it is good to document the static functions within a file too, to make future maintenance easier.

It is not about splitting the code down to small chunks, but splitting it down to small *logical* chunks. It doesn't make sense to commit dozens of empty bodied functions for review, and then later add their code.

If you add functions, only add them at a patch that takes them into use too, unless we're talking about general purpose shared code. And also remember to add the function body and documentation header. If you simply add a "return 0;" or similar function body, do add a comment to why the function does not exist and when it will.

Then, there is a trend of having a boolean return values in the code. When possible, it should rather be int and the cause for failure should be propagated from the last level all the way to up (-ENOMEN etc.). This way debugging becomes easier and if new error conditions appear, there is less of a maintenance burden to add the propagation later.

Finally, make sure to look at the existing driver parts and
https://www.kernel.org/doc/Documentation/CodingStyle
for proper coding style. There are lots of whitespace fixes needed in this series, like array initializations.

I hope to see this first patch rerolled so that you squash some of the later commits into it so that all functions have a body and you add documentation for the functions so I can both see what it should do and what it actually does. Only reroll the first patch, to keep the iterative step smaller. Lets only then continue with the rest of the series once we've reached a consensus on the formatting and style basics.

Quoted 1 line

See more comments below.

I'm glad you did the nitpicking. As far as the integration goes, on the whole I'm happy with the way it is structured and the reuse of existing code. I tried to attack various aspects of the GVT contexts and came to the conclusion I couldn't suggest a better approach (though maybe tomorrow!). A few bits and pieces I got lost trying to pull together (in particular like how we do is read back through the GTT entries performed, the hypervisor_read_va abstraction iirc) and would appreciate having a branch available to get the complete picture. -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#4 lore raw

Re: [RFC 01/29] drm/i915/gvt: Introduce the basic architecture of GVT-g

Hi Chris:
Thanks for your feedback. :) Currently there are some host i915
changes we need to discussion, need your advice and feedback. For GVT LRC context, as you see, we need it to do shadow context. All the modifications of LRC are for it. Need inputs to think a better approach or code placement. :)

---

The GGTT page table is shadowed in GVT-g. When guest writes the GTT MMIOs, this write will be trapped by hypervisor, GVT-g will check if it's a MMIO access which needs to be emulated, or a GGTT page table entry which needs to be shadowed, then the shadowed GGTT page table entry will be written into pGGTT page table. So that the mapping looks like:

From guest point of view:

guest PFN -> guest GGTT page table entry -> guest GGTT MMIO

The GVT-g shadow process looks like:

guest PFN -> guest GGTT page table entry -> GVT-g translated guest PFN to machine PFN -> GVT-g writes the translated entry into physical GGTT MMIO.

---

The hypervisor_{read, write}_va() abstraction layer is to cover the different approaches between hypervisors to let the host i915 to access guest memory.

For example, guest writes the vELSP to submit a workload, then hypervisor traps these writes and forward them to GVT-g core logic. Actually, if GVT-g wants to access the guest context, only LRCA (GGTT address) could be extracted from guest context descriptor.

First GVT-g will try to get the guest PFN from guest GGTT page table (You can see the GGTT/PPGTT page table entry extraction routines in gtt.c),

Second GVT-g will try to get the host VA with the guest PFN via hypervisor specific routines. (The function gvt_gma_to_va())

Then call hypervisor_{read,write}_va() to let hypervisor specific routines to read/write the data from/to guest via hypervisor specific routines.

This is a typical scenario of how GVT-g access a guest GGTT mapped memory. For PPGTT, it becomes much more complicated.(Also in gvt_gma_to_va())

Quoted 46 lines · Chris Wilson
Quoted 43 lines · Joonas Lahtinen

Hi,

TL;DR Overall, we have same problem as with the scheduler series, there is too much placeholder stuff for easy review. Just squash enough code into one commit so it actually does something logical that can be reviewed and then extend it later. Then it can be reviewed and pushed. Just splitting the code down to achieve smaller patches is not the right thing to do.

Comments on the overall code: You need to document all header file functions (in the source files), and it is good to document the static functions within a file too, to make future maintenance easier.

It is not about splitting the code down to small chunks, but splitting it down to small *logical* chunks. It doesn't make sense to commit dozens of empty bodied functions for review, and then later add their code.

If you add functions, only add them at a patch that takes them into use too, unless we're talking about general purpose shared code. And also remember to add the function body and documentation header. If you simply add a "return 0;" or similar function body, do add a comment to why the function does not exist and when it will.

Then, there is a trend of having a boolean return values in the code. When possible, it should rather be int and the cause for failure should be propagated from the last level all the way to up (-ENOMEN etc.). This way debugging becomes easier and if new error conditions appear, there is less of a maintenance burden to add the propagation later.

Finally, make sure to look at the existing driver parts and
https://www.kernel.org/doc/Documentation/CodingStyle
for proper coding style. There are lots of whitespace fixes needed in this series, like array initializations.

I hope to see this first patch rerolled so that you squash some of the later commits into it so that all functions have a body and you add documentation for the functions so I can both see what it should do and what it actually does. Only reroll the first patch, to keep the iterative step smaller. Lets only then continue with the rest of the series once we've reached a consensus on the formatting and style basics.

Quoted 1 line

See more comments below.

Quoted 10 lines

I'm glad you did the nitpicking. As far as the integration goes, on the whole I'm happy with the way it is structured and the reuse of existing code. I tried to attack various aspects of the GVT contexts and came to the conclusion I couldn't suggest a better approach (though maybe tomorrow!). A few bits and pieces I got lost trying to pull together (in particular like how we do is read back through the GTT entries performed, the hypervisor_read_va abstraction iirc) and would appreciate having a branch available to get the complete picture. -Chris

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#5 lore raw

Re: [RFC 01/29] drm/i915/gvt: Introduce the basic architecture of GVT-g

In reply to Chris Wilson

Hi Chris,

Quoted 46 lines · Chris Wilson
Quoted 43 lines · Joonas Lahtinen

Hi,

TL;DR Overall, we have same problem as with the scheduler series, there is too much placeholder stuff for easy review. Just squash enough code into one commit so it actually does something logical that can be reviewed and then extend it later. Then it can be reviewed and pushed. Just splitting the code down to achieve smaller patches is not the right thing to do.

Comments on the overall code: You need to document all header file functions (in the source files), and it is good to document the static functions within a file too, to make future maintenance easier.

It is not about splitting the code down to small chunks, but splitting it down to small *logical* chunks. It doesn't make sense to commit dozens of empty bodied functions for review, and then later add their code.

If you add functions, only add them at a patch that takes them into use too, unless we're talking about general purpose shared code. And also remember to add the function body and documentation header. If you simply add a "return 0;" or similar function body, do add a comment to why the function does not exist and when it will.

Then, there is a trend of having a boolean return values in the code. When possible, it should rather be int and the cause for failure should be propagated from the last level all the way to up (-ENOMEN etc.). This way debugging becomes easier and if new error conditions appear, there is less of a maintenance burden to add the propagation later.

Finally, make sure to look at the existing driver parts and
https://www.kernel.org/doc/Documentation/CodingStyle
for proper coding style. There are lots of whitespace fixes needed in this series, like array initializations.

I hope to see this first patch rerolled so that you squash some of the later commits into it so that all functions have a body and you add documentation for the functions so I can both see what it should do and what it actually does. Only reroll the first patch, to keep the iterative step smaller. Lets only then continue with the rest of the series once we've reached a consensus on the formatting and style basics.

Quoted 1 line

See more comments below.

Quoted 8 lines

I'm glad you did the nitpicking. As far as the integration goes, on the whole I'm happy with the way it is structured and the reuse of existing code. I tried to attack various aspects of the GVT contexts and came to the conclusion I couldn't suggest a better approach (though maybe tomorrow!). A few bits and pieces I got lost trying to pull together (in particular like how we do is read back through the GTT entries performed, the hypervisor_read_va abstraction iirc) and would appreciate having a branch available to get the complete picture.

I pushed the RFC code into below repo:

https://github.com/01org/Igvtg-kernel.git

Branch: gvt-upstream-rfc

Thanks for review and look forward to more comments!

Regards,
-Zhiyuan

Quoted 3 lines

-Chris

Signature · 1 line
Chris Wilson, Intel Open Source Technology Centre

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

author
#6 lore raw

Re: [RFC 01/29] drm/i915/gvt: Introduce the basic architecture of GVT-g

In reply to Joonas Lahtinen

Hi Joonas:
Thanks you very much! We're very excited for receiving your advice
and continue to be open to any comments. :)

I'm supposed that we should make the agreements on i915 host change at the very beginning, as I'm concerned during the discussion of i915 host change, you know, maybe some code of GVT-g needs to be refined or re-designed. So we put the i915 host changes to the beginning of the patch set for the convenience of discussion.

I summarize your comments as below, very applicate. :)

- Replace boolean return value with int as much as possible.
- Replace customized debug ASSERT() function & macros with DRM_DEBUG_*
- Document all non-static functions like i915
- Fix all whitespace via scripts/cleanpatch.pl
- Commit function structure refinement.
- Change the register access behavior just like what i915 does.

For other comments, see my comments below. :)

Quoted 109 lines · Joonas Lahtinen

Hi,

TL;DR Overall, we have same problem as with the scheduler series, there is too much placeholder stuff for easy review. Just squash enough code into one commit so it actually does something logical that can be reviewed and then extend it later. Then it can be reviewed and pushed. Just splitting the code down to achieve smaller patches is not the right thing to do.

Comments on the overall code: You need to document all header file functions (in the source files), and it is good to document the static functions within a file too, to make future maintenance easier.

It is not about splitting the code down to small chunks, but splitting it down to small *logical* chunks. It doesn't make sense to commit dozens of empty bodied functions for review, and then later add their code.

If you add functions, only add them at a patch that takes them into use too, unless we're talking about general purpose shared code. And also remember to add the function body and documentation header. If you simply add a "return 0;" or similar function body, do add a comment to why the function does not exist and when it will.

Then, there is a trend of having a boolean return values in the code. When possible, it should rather be int and the cause for failure should be propagated from the last level all the way to up (-ENOMEN etc.). This way debugging becomes easier and if new error conditions appear, there is less of a maintenance burden to add the propagation later.

Finally, make sure to look at the existing driver parts and
https://www.kernel.org/doc/Documentation/CodingStyle
for proper coding style. There are lots of whitespace fixes needed in this series, like array initializations.

I hope to see this first patch rerolled so that you squash some of the later commits into it so that all functions have a body and you add documentation for the functions so I can both see what it should do and what it actually does. Only reroll the first patch, to keep the iterative step smaller. Lets only then continue with the rest of the series once we've reached a consensus on the formatting and style basics.

See more comments below.

Quoted 39 lines · Zhi Wang

This patch introduces the very basic framework of GVT-g device model, includes basic prototypes, definitions, initialization.

---
  arch/x86/include/asm/xen/interface.h  |   3 +
  drivers/gpu/drm/i915/Kconfig          |  16 ++
  drivers/gpu/drm/i915/Makefile         |   2 +
  drivers/gpu/drm/i915/gvt/Makefile     |   5 +
  drivers/gpu/drm/i915/gvt/debug.h      |  72 +++++++
  drivers/gpu/drm/i915/gvt/fb_decoder.c |  34 ++++
  drivers/gpu/drm/i915/gvt/fb_decoder.h | 110 ++++++++++
  drivers/gpu/drm/i915/gvt/gvt.c        | 366 ++++++++++++++++++++++++++++++++++
  drivers/gpu/drm/i915/gvt/gvt.h        | 130 ++++++++++++
  drivers/gpu/drm/i915/gvt/hypercall.h  |  30 +++
  drivers/gpu/drm/i915/gvt/mpt.h        |  97 +++++++++
  drivers/gpu/drm/i915/gvt/params.c     |  29 +++
  drivers/gpu/drm/i915/gvt/params.h     |  34 ++++
  drivers/gpu/drm/i915/gvt/reg.h        |  34 ++++
  drivers/gpu/drm/i915/i915_dma.c       |  19 ++
  drivers/gpu/drm/i915/i915_drv.h       |   6 +
  drivers/gpu/drm/i915/i915_vgpu.h      |   3 +
  include/xen/interface/xen.h           |   1 +
  18 files changed, 991 insertions(+)

create mode 100644 drivers/gpu/drm/i915/gvt/Makefile create mode 100644 drivers/gpu/drm/i915/gvt/debug.h create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.c create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.h create mode 100644 drivers/gpu/drm/i915/gvt/gvt.c create mode 100644 drivers/gpu/drm/i915/gvt/gvt.h create mode 100644 drivers/gpu/drm/i915/gvt/hypercall.h create mode 100644 drivers/gpu/drm/i915/gvt/mpt.h create mode 100644 drivers/gpu/drm/i915/gvt/params.c create mode 100644 drivers/gpu/drm/i915/gvt/params.h create mode 100644 drivers/gpu/drm/i915/gvt/reg.h

arch/x86/include/asm/xen/interface.h +0 -0
index 62ca03e..6ff4986 100644

arch/x86/include/asm/xen/interface.h line 78

Quoted 6 lines
arch/x86/include/asm/xen/interface.h +3 -0
@@ -73,6 +73,9 @@
  #endif
 
  #ifndef __ASSEMBLY__
+
+#include
+

I don't follow why this would need to be added if the file is not modified otherwise. Each header should only include what they use.

If this is an existing bug (that xen/interface.h can not be included without including linux/types.h), it should be split to a separate patch and sent to Xen team. Same for include/xen/interface/xen.h.

Quoted 10 lines
arch/x86/include/asm/xen/interface.h +0 -0
  /* Explicitly size integers that represent pfns in the public interface
   * with Xen so that on ARM we can have one ABI that works for 32 and 64
   * bit guests. */
drivers/gpu/drm/i915/Kconfig +0 -0
index 051eab3..89ff723 100644
@@ -47,3 +47,19 @@ config DRM_I915_PRELIMINARY_HW_SUPPORT
  	  option changes the default for that module option.
 
Quoted 10 lines

drivers/gpu/drm/i915/Kconfig config DRM_I915_PRELIMINARY_HW_SUPPORT · line 55

Quoted 7 lines
drivers/gpu/drm/i915/Kconfig +6 -0
  	  If in doubt, say "N".
+
+config I915_GVT
+        tristate "GVT-g host driver"
+        depends on DRM_I915
+        select IRQ_WORK
+        default y

Defaulting to "n" would make sense initially.

[Zhi] Sure, will do.

Quoted 78 lines
Quoted 78 lines
drivers/gpu/drm/i915/Kconfig +10 -0
+        help
+          Enabling GVT-g mediated graphics passthrough technique for Intel i915
+          based integrated graphics card. With GVT-g, it's possible to have one
+          integrated i915 device shared by multiple VMs. Performance critical
+          opterations such as apperture accesses and ring buffer operations
+          are pass-throughed to VM, with a minimal set of conflicting resources
+          (e.g. display settings) mediated by vGT driver. The benefit of vGT
+          is on both the performance, given that each VM could directly operate
+          its aperture space and submit commands like running on native, and
+          the feature completeness, given that a true GEN hardware is exposed.
drivers/gpu/drm/i915/Makefile +2 -0
index 0851de07..d4df410 100644
@@ -91,6 +91,8 @@ i915-y += dvo_ch7017.o \
  	  intel_sdvo.o \
  	  intel_tv.o
 
+obj-$(CONFIG_I915_GVT)  += gvt/
+
  # virtual gpu code
  i915-y += i915_vgpu.o
 
drivers/gpu/drm/i915/gvt/Makefile new file +5 -0
new file mode 100644
index 0000000..6935b78
@@ -0,0 +1,5 @@
+GVT_SOURCE := gvt.o params.o fb_decoder.o
+
+ccflags-y			+= -I$(src) -I$(src)/.. -Wall -Werror -Wno-unused-function
+i915_gvt-y			:= $(GVT_SOURCE)
+obj-$(CONFIG_I915_GVT)		+= i915_gvt.o
drivers/gpu/drm/i915/gvt/debug.h new file +38 -0
new file mode 100644
index 0000000..18e1467
@@ -0,0 +1,72 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef __GVT_DEBUG_H__
+#define __GVT_DEBUG_H__
+
+#define ASSERT(x)                                                       \
+        do {                                                            \
+                if (!(x)) {                                             \
+                        printk("Assert at %s line %d\n",                \
+                                __FILE__, __LINE__);                    \
+                }                                                       \
+        } while (0);
+
+#define ASSERT_NUM(x, y)                                                \
+        do {                                                            \
+                if (!(x)) {                                             \
+                        printk("Assert at %s line %d para 0x%llx\n",    \
Quoted 9 lines

drivers/gpu/drm/i915/gvt/debug.h line 42

Quoted 4 lines
drivers/gpu/drm/i915/gvt/debug.h new file +4 -0
+                                __FILE__, __LINE__, (u64)y);            \
+                }                                                       \
+        } while (0);
+

There already is WARN_ON (and i915_drv.h modifies it a little). Do not introduce custom functions like this, if the existing ones need improvement, improve them.

[Zhi] OK. Will do as what i915 does

Quoted 25 lines
Quoted 25 lines
drivers/gpu/drm/i915/gvt/debug.h new file +25 -0
+#define gvt_info(fmt, args...) \
+	printk(KERN_INFO"[GVT-g] "fmt"\n", ##args)
+
+#define gvt_err(fmt, args...) \
+	printk(KERN_ERR"%s() - %d: "fmt"\n", __func__, __LINE__, ##args)
+
+#define gvt_warn(fmt, args...) \
+	printk(KERN_WARNING"%s() - %d: "fmt"\n", __func__, __LINE__, ##args)
+
+#define gvt_dbg(level, fmt, args...) do { \
+		if (gvt.debug & level) \
+			printk(KERN_DEBUG"%s() - %d: "fmt"\n", __func__, __LINE__, ##args); \
+	}while(0)
+
+enum {
+	GVT_DBG_CORE = (1 << 0),
+	GVT_DBG_MM = (1 << 1),
+	GVT_DBG_IRQ = (1 << 2),
+};
+
+#define gvt_dbg_core(fmt, args...) \
+	gvt_dbg(GVT_DBG_CORE, fmt, ##args)
+
+#define gvt_dbg_mm(fmt, args...) \
+	gvt_dbg(GVT_DBG_MM, fmt, ##args)
Quoted 9 lines

drivers/gpu/drm/i915/gvt/debug.h line 72

Quoted 5 lines
drivers/gpu/drm/i915/gvt/debug.h new file +5 -0
+
+#define gvt_dbg_irq(fmt, args...) \
+	gvt_dbg(GVT_DBG_IRQ, fmt, ##args)
+
+#endif

This should be integrated better to DRM debugging options, custom debugging code only for i915 was rejected a while ago.

[ZHI] Thanks for sharing the history. :) Will change that.

Quoted 37 lines
Quoted 37 lines
drivers/gpu/drm/i915/gvt/fb_decoder.c new file +31 -0
new file mode 100644
index 0000000..a219819
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include "gvt.h"
+
+int gvt_decode_fb_format(struct pgt_device *pdev, int vmid, struct gvt_fb_format *fb)
+{
+	return 0;
+}
+
+int gvt_fb_notifier_call_chain(unsigned long val, void *data)
Quoted 9 lines

drivers/gpu/drm/i915/gvt/fb_decoder.c line 29

Quoted 3 lines
drivers/gpu/drm/i915/gvt/fb_decoder.c new file +3 -0
+{
+	return 0;
+}

Kerneldoc missing for these functions. It is all the same to squash later patches to introduce the code to these functions already, reviewing utility functions with no kerneldoc and no body makes it somewhat difficult to see the big picture.

[Zhi] One question. I saw i915 put some function instruction in *.c. Is it also OK for kernel doc generating the proper doc files?

Quoted 97 lines
Quoted 97 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +91 -0
new file mode 100644
index 0000000..2c29ed4
@@ -0,0 +1,110 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_FB_DECODER_H_
+#define _GVT_FB_DECODER_H_
+
+typedef enum {
+	FB_MODE_SET_START = 1,
+	FB_MODE_SET_END,
+	FB_DISPLAY_FLIP,
+}gvt_fb_event_t;
+
+typedef enum {
+	DDI_PORT_NONE	= 0,
+	DDI_PORT_B	= 1,
+	DDI_PORT_C	= 2,
+	DDI_PORT_D	= 3,
+	DDI_PORT_E	= 4
+} ddi_port_t;
+
+struct pgt_device;
+
+struct gvt_fb_notify_msg {
+	unsigned vm_id;
+	unsigned pipe_id; /* id starting from 0 */
+	unsigned plane_id; /* primary, cursor, or sprite */
+};
+
+/* color space conversion and gamma correction are not included */
+struct gvt_primary_plane_format {
+	u8	enabled;	/* plane is enabled */
+	u8	tiled;		/* X-tiled */
+	u8	bpp;		/* bits per pixel */
+	u32	hw_format;	/* format field in the PRI_CTL register */
+	u32	drm_format;	/* format in DRM definition */
+	u32	base;		/* framebuffer base in graphics memory */
+	u32	x_offset;	/* in pixels */
+	u32	y_offset;	/* in lines */
+	u32	width;		/* in pixels */
+	u32	height;		/* in lines */
+	u32	stride;		/* in bytes */
+};
+
+struct gvt_sprite_plane_format {
+	u8	enabled;	/* plane is enabled */
+	u8	tiled;		/* X-tiled */
+	u8	bpp;		/* bits per pixel */
+	u32	hw_format;	/* format field in the SPR_CTL register */
+	u32	drm_format;	/* format in DRM definition */
+	u32	base;		/* sprite base in graphics memory */
+	u32	x_pos;		/* in pixels */
+	u32	y_pos;		/* in lines */
+	u32	x_offset;	/* in pixels */
+	u32	y_offset;	/* in lines */
+	u32	width;		/* in pixels */
+	u32	height;		/* in lines */
+};
+
+struct gvt_cursor_plane_format {
+	u8	enabled;
+	u8	mode;		/* cursor mode select */
+	u8	bpp;		/* bits per pixel */
+	u32	drm_format;	/* format in DRM definition */
+	u32	base;		/* cursor base in graphics memory */
+	u32	x_pos;		/* in pixels */
+	u32	y_pos;		/* in lines */
+	u8	x_sign;		/* X Position Sign */
+	u8	y_sign;		/* Y Position Sign */
+	u32	width;		/* in pixels */
+	u32	height;		/* in lines */
+	u32	x_hot;		/* in pixels */
Quoted 9 lines

drivers/gpu/drm/i915/gvt/fb_decoder.h line 94

Quoted 3 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +3 -0
+	u32	y_hot;		/* in pixels */
+};
+

The above structs have a lot in common, would it make sense to have the common members + plane type and then union for the plane type specific data. I suspect having it all split this way will lead to more utility functions somewhere.

[Zhi] Thanks. Will do that.

Quoted 72 lines
Quoted 46 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +16 -0
+struct gvt_pipe_format {
+	struct gvt_primary_plane_format	primary;
+	struct gvt_sprite_plane_format	sprite;
+	struct gvt_cursor_plane_format	cursor;
+	ddi_port_t ddi_port;  /* the DDI port that the pipe is connected to */
+};
+
+struct gvt_fb_format{
+	struct gvt_pipe_format	pipes[I915_MAX_PIPES];
+};
+
+extern int gvt_fb_notifier_call_chain(unsigned long val, void *data);
+extern int gvt_decode_fb_format(struct pgt_device *pdev, int vmid,
+				struct gvt_fb_format *fb);
+
+#endif
drivers/gpu/drm/i915/gvt/gvt.c new file +24 -0
new file mode 100644
index 0000000..041d10f
@@ -0,0 +1,366 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include

drivers/gpu/drm/i915/gvt/gvt.c line 34

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +10 -0
+#include
+
+#include "gvt.h"
+
+struct gvt_host gvt_host;
+
+extern struct gvt_kernel_dm xengt_kdm;
+extern struct gvt_kernel_dm kvmgt_kdm;
+
+static const char *supported_hypervisors[] = {

should be "static const char * const".

Quoted 13 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +13 -0
+	[GVT_HYPERVISOR_TYPE_XEN] = "Xen Hypervisor",
+	[GVT_HYPERVISOR_TYPE_KVM] = "KVM",
+};
+
+static bool gvt_init_host(void)
+{
+	struct gvt_host *host = &gvt_host;
+
+	if (!gvt.enable) {
+		gvt_dbg_core("GVT-g has been disabled by kernel parameter");
+		return false;
+	}
+
Quoted 9 lines

drivers/gpu/drm/i915/gvt/gvt.c line 53

Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +6 -0
+	if (host->initialized) {
+		gvt_err("GVT-g has already been initialized!");
+		return false;
+	}
+
+	if (xen_initial_domain()) {

Shouldn't Xen code be CONFIG_XEN_DOM0 and CONFIG_XEN #ifdef protected?

[Zhi] The following code piece shows xen_initial_domain()/xen_domain() could also be used when CONFIG_XEN_DOM0 is not set.

#ifdef CONFIG_XEN
extern enum xen_domain_type xen_domain_type;
#else
#define xen_domain_type XEN_NATIVE
#endif

#define xen_domain() (xen_domain_type != XEN_NATIVE)

#define xen_pv_domain()         (xen_domain() &&                        \
                                 xen_domain_type == XEN_PV_DOMAIN)
#define xen_hvm_domain()        (xen_domain() &&                        \
                                 xen_domain_type == XEN_HVM_DOMAIN)

#ifdef CONFIG_XEN_DOM0
#include <xen/interface/xen.h>
#include <asm/xen/hypervisor.h>

#define xen_initial_domain()    (xen_domain() && \
                                 xen_start_info && 

xen_start_info->flags & SIF_INITDOMAIN)
#else /* !CONFIG_XEN_DOM0 */
#define xen_initial_domain() (0)
#endif /* CONFIG_XEN_DOM0 */

Quoted 7 lines
Quoted 7 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +7 -0
+		/* Xen Dom0 */
+		host->kdm = try_then_request_module(symbol_get(xengt_kdm), "xengt");
+		host->hypervisor_type = GVT_HYPERVISOR_TYPE_XEN;
+	} else if(xen_domain()) {
+		/* Xen DomU */
+		return false;
+	} else {
Quoted 8 lines

drivers/gpu/drm/i915/gvt/gvt.c line 65

Quoted 5 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +5 -0
+		/* not in Xen. Try KVMGT */
+		host->kdm = try_then_request_module(symbol_get(kvmgt_kdm), "kvm");
+		host->hypervisor_type = GVT_HYPERVISOR_TYPE_KVM;
+	}
+

Why do we need to keep track of hypervisor type, are there plenty of hypervisor specific behaviour differences?

[Zhi] As GVT-g needs some hypervisor services to work, we write a abstraction layer to connect GVT-g to different hypervisor. But still there are some emulation logic couldn't be fitted into that abstraction layer, like OpRegion emulation. In Xen, we emulates it in kernel space, while we emulates it in Qemu under KVM. I also agreed that we should find some approach to improve it just like what you said.

If it is just for printing

Quoted 38 lines

the below debug, I think it's better to move the debug into above code that detects the hypervisor, wrap them with appropriate CONFIG #ifdefs.

Quoted 35 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +35 -0
+	if (!host->kdm)
+		return false;
+
+	if (!hypervisor_detect_host())
+		return false;
+
+	gvt_info("Running with hypervisor %s in host mode",
+			supported_hypervisors[host->hypervisor_type]);
+
+	idr_init(&host->device_idr);
+	mutex_init(&host->device_idr_lock);
+
+	host->initialized = true;
+	return true;
+}
+
+static bool init_device_info(struct pgt_device *pdev)
+{
+	struct gvt_device_info *info = &pdev->device_info;
+
+	if (!IS_BROADWELL(pdev->dev_priv)) {
+		gvt_err("Unsupported GEN device");
+		return false;
+	}
+
+	if (IS_BROADWELL(pdev->dev_priv)) {
+		info->max_gtt_gm_sz = (1UL << 32);
+		/*
+		 * The layout of BAR0 in BDW:
+		 * |< - MMIO 2MB ->|<- Reserved 6MB ->|<- MAX GTT 8MB->|
+		 *
+		 * GTT offset in BAR0 starts from 8MB to 16MB, and
+		 * Whatever GTT size is configured in BIOS,
+		 * the size of BAR0 is always 16MB. The actual configured
+		 * GTT size can be found in GMCH_CTRL.
Quoted 9 lines

drivers/gpu/drm/i915/gvt/gvt.c line 106

Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +6 -0
+		 */
+		info->gtt_start_offset = (1UL << 23);
+		info->max_gtt_size = (1UL << 23);
+		info->gtt_entry_size = 8;
+		info->gtt_entry_size_shift = 3;
+		info->gmadr_bytes_in_cmd = 8;

Would this information be useful in a header as #defines too?

[Zhi] Probably. Consider that one GVT-g featured kernel could be able to run on different platforms, the informaion data structure could be different. But I agree to define the magic numbers in a header files. :)

Quoted 4 lines
Quoted 4 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +4 -0
+	}
+
+	gvt_info("Device info:");
+	printk("        max_gtt_gm_sz: %llx\n", info->max_gtt_gm_sz);
Quoted 9 lines

drivers/gpu/drm/i915/gvt/gvt.c line 116

Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +6 -0
+	printk("        max_gtt_size: %x\n", info->max_gtt_size);
+	printk("        gtt_size_entry: %x\n", info->gtt_entry_size);
+	printk("        gtt_entry_size_shift: %x\n", info->gtt_entry_size_shift);
+	printk("        gtt_start_offset: %x\n", info->gtt_start_offset);
+	printk("        gtt_end_offset: %x\n", info->gtt_end_offset);
+

Just put this to kind of stuff to i915_debugfs.c.

[Zhi] Thanks.

Quoted 93 lines

drivers/gpu/drm/i915/gvt/gvt.c line 127

Quoted 11 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +11 -0
+	return true;
+}
+
+static void init_initial_cfg_space_state(struct pgt_device *pdev)
+{
+	struct pci_dev *pci_dev = pdev->dev_priv->dev->pdev;
+	int i;
+
+	gvt_dbg_core("init initial cfg space, id %d", pdev->id);
+
+	for (i = 0; i < GVT_CFG_SPACE_SZ; i += 4)

+= sizeof(...)

drivers/gpu/drm/i915/gvt/gvt.c line 131

Quoted 4 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +4 -0
+		pci_read_config_dword(pci_dev, i,
+				(u32 *)&pdev->initial_cfg_space[i]);
+
+	for (i = 0; i < 3; i++) {

No magic numbers make a #define for 3 and give it a descriptive name.

Quoted 22 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +22 -0
+		pdev->bar_size[i] = pci_resource_len(pci_dev, i * 2);
+		gvt_info("bar %d size: %llx", i, pdev->bar_size[i]);
+	}
+}
+
+static void clean_initial_mmio_state(struct pgt_device *pdev)
+{
+	if (pdev->gttmmio_va) {
+		iounmap(pdev->gttmmio_va);
+		pdev->gttmmio_va = NULL;
+	}
+
+	if (pdev->gmadr_va) {
+		iounmap(pdev->gmadr_va);
+		pdev->gmadr_va = NULL;
+	}
+}
+
+static bool init_initial_mmio_state(struct pgt_device *pdev)
+{
+	u64 bar0, bar1;
+

drivers/gpu/drm/i915/gvt/gvt.c line 163

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +10 -0
+	gvt_dbg_core("init initial mmio state, id %d", pdev->id);
+
+	bar0 = *(u64 *)&pdev->initial_cfg_space[GVT_REG_CFG_SPACE_BAR0];
+	bar1 = *(u64 *)&pdev->initial_cfg_space[GVT_REG_CFG_SPACE_BAR1];
+
+	pdev->gttmmio_base = bar0 & ~0xf;
+	pdev->mmio_size = 2 * 1024 * 1024;
+	pdev->reg_num = pdev->mmio_size / 4;
+	pdev->gmadr_base = bar1 & ~0xf;
+

Many magic numbers.

drivers/gpu/drm/i915/gvt/gvt.c line 166

Quoted 3 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +3 -0
+	pdev->gttmmio_va = ioremap(pdev->gttmmio_base, pdev->bar_size[0]);
+	if (!pdev->gttmmio_va) {
+		gvt_err("fail to map GTTMMIO BAR.");

These should be if(WARN_ON(...))

Quoted 31 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +31 -0
+		return false;
+	}
+
+	pdev->gmadr_va = ioremap(pdev->gmadr_base, pdev->bar_size[2]);
+	if (!pdev->gmadr_va) {
+		gvt_err("fail to map GMADR BAR.");
+		goto err;
+	}
+
+	gvt_info("bar0: 0x%llx, bar1: 0x%llx", bar0, bar1);
+	gvt_info("mmio size: %x", pdev->mmio_size);
+	gvt_info("gttmmio: 0x%llx, gmadr: 0x%llx", pdev->gttmmio_base, pdev->gmadr_base);
+	gvt_info("gttmmio_va: %p", pdev->gttmmio_va);
+	gvt_info("gmadr_va: %p", pdev->gmadr_va);
+
+	return true;
+err:
+	clean_initial_mmio_state(pdev);
+	return false;
+}
+
+static int gvt_service_thread(void *data)
+{
+	struct pgt_device *pdev = (struct pgt_device *)data;
+	int r;
+
+	gvt_dbg_core("service thread start, pgt %d", pdev->id);
+
+	while(!kthread_should_stop()) {
+		r = wait_event_interruptible(pdev->service_thread_wq,
+				kthread_should_stop() || pdev->service_request);
Quoted 9 lines

drivers/gpu/drm/i915/gvt/gvt.c line 203

Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +6 -0
+
+		if (kthread_should_stop())
+			break;
+
+		if (r) {
+			gvt_warn("service thread is waken up by unexpected signal.");

Should be WARN_ONCE, to avoid future disasters with CI.

[Zhi] Thanks.

Quoted 305 lines
Quoted 256 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +163 -0
+			continue;
+		}
+	}
+	return 0;
+}
+
+static void clean_service_thread(struct pgt_device *pdev)
+{
+	if (pdev->service_thread) {
+		kthread_stop(pdev->service_thread);
+		pdev->service_thread = NULL;
+	}
+}
+
+static bool init_service_thread(struct pgt_device *pdev)
+{
+	init_waitqueue_head(&pdev->service_thread_wq);
+
+	pdev->service_thread = kthread_run(gvt_service_thread,
+			pdev, "gvt_service_thread%d", pdev->id);
+
+	if (!pdev->service_thread) {
+		gvt_err("fail to start service thread.");
+		return false;
+	}
+
+	return true;
+}
+
+static void clean_pgt_device(struct pgt_device *pdev)
+{
+	clean_service_thread(pdev);
+	clean_initial_mmio_state(pdev);
+}
+
+static bool init_pgt_device(struct pgt_device *pdev, struct drm_i915_private *dev_priv)
+{
+	if (!init_device_info(pdev))
+		return false;
+
+	init_initial_cfg_space_state(pdev);
+
+	if (!init_initial_mmio_state(pdev))
+		goto err;
+
+	if (!init_service_thread(pdev))
+		goto err;
+
+	return true;
+err:
+	clean_pgt_device(pdev);
+	return false;
+}
+
+static bool post_init_pgt_device(struct pgt_device *pdev)
+{
+	return true;
+}
+
+static void free_pgt_device(struct pgt_device *pdev)
+{
+	struct gvt_host *host = &gvt_host;
+
+	mutex_lock(&host->device_idr_lock);
+	idr_remove(&host->device_idr, pdev->id);
+	mutex_unlock(&host->device_idr_lock);
+
+	vfree(pdev);
+}
+
+static struct pgt_device *alloc_pgt_device(struct drm_i915_private *dev_priv)
+{
+	struct gvt_host *host = &gvt_host;
+	struct pgt_device *pdev = NULL;
+
+	pdev = vzalloc(sizeof(*pdev));
+	if (!pdev) {
+		gvt_err("fail to allocate memory for pgt device.");
+		return NULL;
+	}
+
+	mutex_lock(&host->device_idr_lock);
+	pdev->id = idr_alloc(&host->device_idr, pdev, 0, 0, GFP_KERNEL);
+	mutex_unlock(&host->device_idr_lock);
+
+	if (pdev->id < 0) {
+		gvt_err("fail to allocate pgt device id.");
+		goto err;
+	}
+
+	mutex_init(&pdev->lock);
+	pdev->dev_priv = dev_priv;
+	idr_init(&pdev->instance_idr);
+
+	return pdev;
+err:
+	free_pgt_device(pdev);
+	return NULL;
+}
+
+void gvt_destroy_pgt_device(void *private_data)
+{
+	struct pgt_device *pdev = (struct pgt_device *)private_data;
+
+	clean_pgt_device(pdev);
+	free_pgt_device(pdev);
+}
+
+void *gvt_create_pgt_device(struct drm_i915_private *dev_priv)
+{
+	struct pgt_device *pdev = NULL;
+	struct gvt_host *host = &gvt_host;
+
+	if (!host->initialized && !gvt_init_host()) {
+		gvt_err("gvt_init_host fail");
+		return NULL;
+	}
+
+	gvt_dbg_core("create new pgt device, i915 dev_priv: %p", dev_priv);
+
+	pdev = alloc_pgt_device(dev_priv);
+	if (!pdev) {
+		gvt_err("fail to allocate memory for pgt device.");
+		goto err;
+	}
+
+	gvt_dbg_core("init pgt device, id %d", pdev->id);
+
+	if (!init_pgt_device(pdev, dev_priv)) {
+		gvt_err("fail to init physical device state.");
+		goto err;
+	}
+
+	gvt_dbg_core("pgt device creation done, id %d", pdev->id);
+
+	return pdev;
+err:
+	if (pdev) {
+		gvt_destroy_pgt_device(pdev);
+		pdev = NULL;
+	}
+	return NULL;
+}
+
+bool gvt_post_init_pgt_device(void *private_data)
+{
+	struct pgt_device *pdev = (struct pgt_device *)private_data;
+	struct gvt_host *host = &gvt_host;
+
+	if (!host->initialized) {
+		gvt_err("gvt_host haven't been initialized.");
+		return false;
+	}
+
+	gvt_dbg_core("post init pgt device %d", pdev->id);
+
+	if (!post_init_pgt_device(pdev)) {
+		gvt_err("fail to post init physical device state.");
+		return false;
+	}
+
+	return true;
+}
drivers/gpu/drm/i915/gvt/gvt.h new file +87 -0
new file mode 100644
index 0000000..6c85bba
@@ -0,0 +1,130 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_H_
+#define _GVT_H_
+
+#include "i915_drv.h"
+#include "i915_vgpu.h"
+
+#include "debug.h"
+#include "params.h"
+#include "reg.h"
+#include "hypercall.h"
+#include "mpt.h"
+#include "fb_decoder.h"
+
+#define GVT_MAX_VGPU 8
+
+enum {
+	GVT_HYPERVISOR_TYPE_XEN = 0,
+	GVT_HYPERVISOR_TYPE_KVM,
+};
+
+struct gvt_host {
+	bool initialized;
+	int hypervisor_type;
+	struct mutex device_idr_lock;
+	struct idr device_idr;
+	struct gvt_kernel_dm *kdm;
+};
+
+extern struct gvt_host gvt_host;
+
+/* Describe the limitation of HW.*/
+struct gvt_device_info {
+	u64 max_gtt_gm_sz;
+	u32 gtt_start_offset;
+	u32 gtt_end_offset;
+	u32 max_gtt_size;
+	u32 gtt_entry_size;
+	u32 gtt_entry_size_shift;
+	u32 gmadr_bytes_in_cmd;
+};
+
+struct vgt_device {
+	int id;
+	int vm_id;
+	struct pgt_device *pdev;
+	bool warn_untrack;
+};
+
+struct pgt_device {
+	struct mutex lock;
+	int id;
+
+	struct drm_i915_private *dev_priv;
+	struct idr instance_idr;
+
+	struct gvt_device_info device_info;
+
+	u8 initial_cfg_space[GVT_CFG_SPACE_SZ];
+	u64 bar_size[GVT_BAR_NUM];
+
+	u64 gttmmio_base;
+	void *gttmmio_va;
+
+	u64 gmadr_base;

drivers/gpu/drm/i915/gvt/gvt.h line 97

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +10 -0
+	void *gmadr_va;
+
+	u32 mmio_size;
+	u32 reg_num;
+
+	wait_queue_head_t service_thread_wq;
+	struct task_struct *service_thread;
+	unsigned long service_request;
+};
+

Here-->

Quoted 22 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +22 -0
+static inline u32 gvt_mmio_read(struct pgt_device *pdev,
+		u32 reg)
+{
+	struct drm_i915_private *dev_priv = pdev->dev_priv;
+	i915_reg_t tmp = {.reg = reg};
+	return I915_READ(tmp);
+}
+
+static inline void gvt_mmio_write(struct pgt_device *pdev,
+		u32 reg, u32 val)
+{
+	struct drm_i915_private *dev_priv = pdev->dev_priv;
+	i915_reg_t tmp = {.reg = reg};
+	I915_WRITE(tmp, val);
+}
+
+static inline u64 gvt_mmio_read64(struct pgt_device *pdev,
+		u32 reg)
+{
+	struct drm_i915_private *dev_priv = pdev->dev_priv;
+	i915_reg_t tmp = {.reg = reg};
+	return I915_READ64(tmp);

drivers/gpu/drm/i915/gvt/gvt.h line 129

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +10 -0
+}
+
+static inline void gvt_mmio_write64(struct pgt_device *pdev,
+		u32 reg, u64 val)
+{
+	struct drm_i915_private *dev_priv = pdev->dev_priv;
+	i915_reg_t tmp = {.reg = reg};
+	I915_WRITE64(tmp, val);
+}
+

<-- Why? The i915_reg_t type was added to avoid problems, this code is not used anywhere, and it has no documentation, so I can not review it.

Quoted 2 lines

I wrote comments at the top of the post.

[Zhi] Thanks, I will check that.

Quoted 352 lines

Regards, Joonas

Quoted 350 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +1 -0
+#endif
drivers/gpu/drm/i915/gvt/hypercall.h new file +30 -0
new file mode 100644
index 0000000..0a41874
@@ -0,0 +1,30 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_HYPERCALL_H_
+#define _GVT_HYPERCALL_H_
+
+struct gvt_kernel_dm {
+};
+
+#endif /* _GVT_HYPERCALL_H_ */
drivers/gpu/drm/i915/gvt/mpt.h new file +97 -0
new file mode 100644
index 0000000..bbe4465
@@ -0,0 +1,97 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_MPT_H_
+#define _GVT_MPT_H_
+
+struct vgt_device;
+
+static inline unsigned long hypervisor_g2m_pfn(struct vgt_device *vgt,
+	unsigned long g_pfn)
+{
+	return 0;
+}
+
+static inline int hypervisor_pause_domain(struct vgt_device *vgt)
+{
+	return 0;
+}
+
+static inline int hypervisor_shutdown_domain(struct vgt_device *vgt)
+{
+	return 0;
+}
+
+static inline int hypervisor_set_trap_area(struct vgt_device *vgt,
+	uint64_t start, uint64_t end, bool map)
+{
+	return 0;
+}
+
+static inline bool hypervisor_detect_host(void)
+{
+	return false;
+}
+
+static inline int hypervisor_virt_to_mfn(void *addr)
+{
+	return 0;
+}
+
+static inline void *hypervisor_mfn_to_virt(int mfn)
+{
+	return NULL;
+}
+
+static inline void hypervisor_inject_msi(struct vgt_device *vgt)
+{
+	return;
+}
+
+static inline int hypervisor_hvm_init(struct vgt_device *vgt)
+{
+	return 0;
+}
+
+static inline void hypervisor_hvm_exit(struct vgt_device *vgt)
+{
+}
+
+static inline void *hypervisor_gpa_to_va(struct vgt_device *vgt, unsigned long gpa)
+{
+	return NULL;
+}
+
+static inline bool hypervisor_read_va(struct vgt_device *vgt, void *va,
+		void *val, int len, int atomic)
+{
+	return false;
+}
+
+static inline bool hypervisor_write_va(struct vgt_device *vgt, void *va,
+		void *val, int len, int atomic)
+{
+	return false;
+}
+
+#endif /* _GVT_MPT_H_ */
drivers/gpu/drm/i915/gvt/params.c new file +29 -0
new file mode 100644
index 0000000..dfc33c3
@@ -0,0 +1,29 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include "gvt.h"
+
+struct gvt_kernel_params gvt = {
+	.enable = true,
+	.debug = 0,
+};
drivers/gpu/drm/i915/gvt/params.h new file +34 -0
new file mode 100644
index 0000000..d2955b9
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_PARAMS_H_
+#define _GVT_PARAMS_H_
+
+struct gvt_kernel_params {
+	bool enable;
+	int debug;
+};
+
+extern struct gvt_kernel_params gvt;
+
+#endif
drivers/gpu/drm/i915/gvt/reg.h new file +34 -0
new file mode 100644
index 0000000..d363b74
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_REG_H
+#define _GVT_REG_H
+
+#define GVT_CFG_SPACE_SZ	256
+#define GVT_BAR_NUM		4
+
+#define GVT_REG_CFG_SPACE_BAR0	0x10
+#define GVT_REG_CFG_SPACE_BAR1	0x18
+#define GVT_REG_CFG_SPACE_BAR2	0x20
+
+#endif
drivers/gpu/drm/i915/i915_dma.c +19 -0
index 4725e8d..eca8e50 100644
@@ -943,6 +943,10 @@ int i915_driver_load(struct drm_device *dev, unsigned long flags)
 
  	intel_uncore_init(dev);
 
+	dev_priv->vgpu.host_private_data = gvt_create_pgt_device(dev_priv);
+	if(intel_gvt_host_active(dev))
+		DRM_INFO("GVT-g is running in host mode\n");
+
  	ret = i915_gem_gtt_init(dev);
  	if (ret)
  		goto out_freecsr;
@@ -1067,6 +1071,13 @@ int i915_driver_load(struct drm_device *dev, unsigned long flags)
  		goto out_power_well;
  	}
 
+	if (intel_gvt_host_active(dev)) {
+		if (!gvt_post_init_pgt_device(dev_priv->vgpu.host_private_data)) {
+			DRM_ERROR("failed to post init pgt device\n");
+			goto out_power_well;
+		}
+	}
+
  	/*
  	 * Notify a valid surface after modesetting,
  	 * when running inside a VM.
@@ -1117,6 +1128,10 @@ out_gtt:
  	i915_global_gtt_cleanup(dev);
  out_freecsr:
  	intel_csr_ucode_fini(dev_priv);
+	if (intel_gvt_host_active(dev)) {
+		gvt_destroy_pgt_device(dev_priv->vgpu.host_private_data);
+		dev_priv->vgpu.host_private_data = NULL;
+	}
  	intel_uncore_fini(dev);
  	pci_iounmap(dev->pdev, dev_priv->regs);
  put_bridge:
@@ -1165,6 +1180,10 @@ int i915_driver_unload(struct drm_device *dev)
 
  	intel_modeset_cleanup(dev);
 
+	if (intel_gvt_host_active(dev)) {
+		gvt_destroy_pgt_device(dev_priv->vgpu.host_private_data);
+		dev_priv->vgpu.host_private_data = NULL;
+	}
  	/*
  	 * free the memory space allocated for the child device
  	 * config parsed from VBT
drivers/gpu/drm/i915/i915_drv.h +6 -0
index 01cc982..db3c79b 100644
@@ -1673,6 +1673,7 @@ struct i915_workarounds {
 
  struct i915_virtual_gpu {
  	bool active;
+	void *host_private_data;
  };
 
  struct i915_execbuffer_params {
@@ -2747,6 +2748,11 @@ static inline bool intel_vgpu_active(struct drm_device *dev)
  	return to_i915(dev)->vgpu.active;
  }
 
+static inline bool intel_gvt_host_active(struct drm_device *dev)
+{
+	return to_i915(dev)->vgpu.host_private_data ? true : false;
+}
+
  void
  i915_enable_pipestat(struct drm_i915_private *dev_priv, enum pipe pipe,
  		     u32 status_mask);
drivers/gpu/drm/i915/i915_vgpu.h +3 -0
index 3c83b47..942490a 100644
@@ -113,5 +113,8 @@ struct vgt_if {
  extern void i915_check_vgpu(struct drm_device *dev);
  extern int intel_vgt_balloon(struct drm_device *dev);
  extern void intel_vgt_deballoon(void);
+extern void *gvt_create_pgt_device(struct drm_i915_private *dev_priv);
+extern bool gvt_post_init_pgt_device(void *private_data);
+extern void gvt_destroy_pgt_device(void *private_data);
 
  #endif /* _I915_VGPU_H_ */
include/xen/interface/xen.h +0 -0
index d133112..78a38f1 100644
@@ -28,6 +28,7 @@
  #define __XEN_PUBLIC_XEN_H__
 
Quoted 5 lines

include/xen/interface/xen.h line 34

Quoted 5 lines
include/xen/interface/xen.h +1 -0
  #include
+#include
 
  /*
   * XEN "SYSTEM CALLS" (a.k.a. HYPERCALLS).

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#7 lore raw

Re: [RFC 01/29] drm/i915/gvt: Introduce the basic architecture of GVT-g

Quoted 22 lines · Zhi Wang

Hi Joonas:
Thanks you very much! We're very excited for receiving your
advice and continue to be open to any comments. :)

I'm supposed that we should make the agreements on i915 host change at the very beginning, as I'm concerned during the discussion of i915 host change, you know, maybe some code of GVT-g needs to be refined or re-designed. So we put the i915 host changes to the beginning of the patch set for the convenience of discussion.

I summarize your comments as below, very applicate. :)

- Replace boolean return value with int as much as possible.
- Replace customized debug ASSERT() function & macros with DRM_DEBUG_*
- Document all non-static functions like i915
- Fix all whitespace via scripts/cleanpatch.pl
- Commit function structure refinement.
- Change the register access behavior just like what i915 does.

For other comments, see my comments below. :)

On 01/29/16 21:57, Joonas Lahtinen wrote:

snip

Quoted 16 lines
Quoted 7 lines
Quoted 7 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +7 -0
+		/* Xen Dom0 */
+		host->kdm = try_then_request_module(symbol_get(xengt_kdm), "xengt");
+		host->hypervisor_type = GVT_HYPERVISOR_TYPE_XEN;
+	} else if(xen_domain()) {
+		/* Xen DomU */
+		return false;
+	} else {
Quoted 8 lines

drivers/gpu/drm/i915/gvt/gvt.c line 65

Quoted 5 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +5 -0
+		/* not in Xen. Try KVMGT */
+		host->kdm = try_then_request_module(symbol_get(kvmgt_kdm), "kvm");
+		host->hypervisor_type = GVT_HYPERVISOR_TYPE_KVM;
+	}
+

Why do we need to keep track of hypervisor type, are there plenty of hypervisor specific behaviour differences?

Quoted 5 lines

[Zhi] As GVT-g needs some hypervisor services to work, we write a abstraction layer to connect GVT-g to different hypervisor. But still there are some emulation logic couldn't be fitted into that abstraction layer, like OpRegion emulation. In Xen, we emulates it in kernel space, while we emulates it in Qemu under KVM. I also

Just a small clarify: most likely the cfg-emulation for KVMGT will be in kernel as well. This part is under discussion. But still we may have code related to hypervisor type in device model, for instance, VFIO based implementation for KVM only. Thanks!

Regards,
-Zhiyuan
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

maintainer
#8 lore raw

Re: [RFC 01/29] drm/i915/gvt: Introduce the basic architecture of GVT-g

In reply to Zhi Wang

Hi,

Quoted 312 lines · Zhi Wang

Hi Joonas:
     Thanks you very much! We're very excited for receiving your advice and continue to be open to any comments. :)

I'm supposed that we should make the agreements on i915 host change at the very beginning, as I'm concerned during the discussion of i915 host change, you know, maybe some code of GVT-g needs to be refined or re-designed. So we put the i915 host changes to the beginning of the patch set for the convenience of discussion.

I summarize your comments as below, very applicate. :)

- Replace boolean return value with int as much as possible.
- Replace customized debug ASSERT() function & macros with DRM_DEBUG_*
- Document all non-static functions like i915
- Fix all whitespace via scripts/cleanpatch.pl
- Commit function structure refinement.
- Change the register access behavior just like what i915 does.

For other comments, see my comments below. :)

Quoted 109 lines · Joonas Lahtinen

Hi,

TL;DR Overall, we have same problem as with the scheduler series, there is too much placeholder stuff for easy review. Just squash enough code into one commit so it actually does something logical that can be reviewed and then extend it later. Then it can be reviewed and pushed. Just splitting the code down to achieve smaller patches is not the right thing to do.

Comments on the overall code: You need to document all header file functions (in the source files), and it is good to document the static functions within a file too, to make future maintenance easier.

It is not about splitting the code down to small chunks, but splitting it down to small *logical* chunks. It doesn't make sense to commit dozens of empty bodied functions for review, and then later add their code.

If you add functions, only add them at a patch that takes them into use too, unless we're talking about general purpose shared code. And also remember to add the function body and documentation header. If you simply add a "return 0;" or similar function body, do add a comment to why the function does not exist and when it will.

Then, there is a trend of having a boolean return values in the code. When possible, it should rather be int and the cause for failure should be propagated from the last level all the way to up (-ENOMEN etc.). This way debugging becomes easier and if new error conditions appear, there is less of a maintenance burden to add the propagation later.

Finally, make sure to look at the existing driver parts and
https://www.kernel.org/doc/Documentation/CodingStyle
for proper coding style. There are lots of whitespace fixes needed in this series, like array initializations.

I hope to see this first patch rerolled so that you squash some of the later commits into it so that all functions have a body and you add documentation for the functions so I can both see what it should do and what it actually does. Only reroll the first patch, to keep the iterative step smaller. Lets only then continue with the rest of the series once we've reached a consensus on the formatting and style basics.

See more comments below.

Quoted 39 lines · Zhi Wang

This patch introduces the very basic framework of GVT-g device model, includes basic prototypes, definitions, initialization.
---
  arch/x86/include/asm/xen/interface.h  |   3 + drivers/gpu/drm/i915/Kconfig          |  16 ++ drivers/gpu/drm/i915/Makefile         |   2 + drivers/gpu/drm/i915/gvt/Makefile     |   5 + drivers/gpu/drm/i915/gvt/debug.h      |  72 +++++++ drivers/gpu/drm/i915/gvt/fb_decoder.c |  34 ++++ drivers/gpu/drm/i915/gvt/fb_decoder.h | 110 ++++++++++ drivers/gpu/drm/i915/gvt/gvt.c        | 366 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/gvt/gvt.h        | 130 ++++++++++++ drivers/gpu/drm/i915/gvt/hypercall.h  |  30 +++ drivers/gpu/drm/i915/gvt/mpt.h        |  97 +++++++++ drivers/gpu/drm/i915/gvt/params.c     |  29 +++ drivers/gpu/drm/i915/gvt/params.h     |  34 ++++ drivers/gpu/drm/i915/gvt/reg.h        |  34 ++++ drivers/gpu/drm/i915/i915_dma.c       |  19 ++ drivers/gpu/drm/i915/i915_drv.h       |   6 + drivers/gpu/drm/i915/i915_vgpu.h      |   3 + include/xen/interface/xen.h           |   1 +

  18 files changed, 991 insertions(+)

  create mode 100644 drivers/gpu/drm/i915/gvt/Makefile create mode 100644 drivers/gpu/drm/i915/gvt/debug.h create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.c create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.h create mode 100644 drivers/gpu/drm/i915/gvt/gvt.c create mode 100644 drivers/gpu/drm/i915/gvt/gvt.h create mode 100644 drivers/gpu/drm/i915/gvt/hypercall.h create mode 100644 drivers/gpu/drm/i915/gvt/mpt.h create mode 100644 drivers/gpu/drm/i915/gvt/params.c create mode 100644 drivers/gpu/drm/i915/gvt/params.h create mode 100644 drivers/gpu/drm/i915/gvt/reg.h

arch/x86/include/asm/xen/interface.h +0 -0
index 62ca03e..6ff4986 100644

arch/x86/include/asm/xen/interface.h line 78

Quoted 6 lines
arch/x86/include/asm/xen/interface.h +3 -0
@@ -73,6 +73,9 @@
   #endif
 
   #ifndef __ASSEMBLY__
+
+#include
+

I don't follow why this would need to be added if the file is not modified otherwise. Each header should only include what they use.

If this is an existing bug (that xen/interface.h can not be included without including linux/types.h), it should be split to a separate patch and sent to Xen team. Same for include/xen/interface/xen.h.

Quoted 10 lines
arch/x86/include/asm/xen/interface.h +0 -0
   /* Explicitly size integers that represent pfns in the public interface
    * with Xen so that on ARM we can have one ABI that works for 32 and 64
    * bit guests. */
drivers/gpu/drm/i915/Kconfig +0 -0
index 051eab3..89ff723 100644
@@ -47,3 +47,19 @@ config DRM_I915_PRELIMINARY_HW_SUPPORT

     option changes the default for that module option.

Quoted 10 lines

drivers/gpu/drm/i915/Kconfig config DRM_I915_PRELIMINARY_HW_SUPPORT · line 55

Quoted 7 lines

     If in doubt, say "N".

drivers/gpu/drm/i915/Kconfig +6 -0
+
+config I915_GVT
+        tristate "GVT-g host driver"
+        depends on DRM_I915
+        select IRQ_WORK
+        default y

Defaulting to "n" would make sense initially.

[Zhi] Sure, will do.

Quoted 78 lines
Quoted 78 lines

+        help
+          Enabling GVT-g mediated graphics passthrough technique for Intel i915 +          based integrated graphics card. With GVT-g, it's possible to have one +          integrated i915 device shared by multiple VMs. Performance critical +          opterations such as apperture accesses and ring buffer operations +          are pass-throughed to VM, with a minimal set of conflicting resources +          (e.g. display settings) mediated by vGT driver. The benefit of vGT +          is on both the performance, given that each VM could directly operate +          its aperture space and submit commands like running on native, and +          the feature completeness, given that a true GEN hardware is exposed.

drivers/gpu/drm/i915/Makefile +0 -0
index 0851de07..d4df410 100644
@@ -91,6 +91,8 @@ i915-y += dvo_ch7017.o \
   	  intel_sdvo.o \

     intel_tv.o

drivers/gpu/drm/i915/Makefile +2 -0
+obj-$(CONFIG_I915_GVT)  += gvt/
+
  # virtual gpu code
  i915-y += i915_vgpu.o
drivers/gpu/drm/i915/gvt/Makefile new file +5 -0
new file mode 100644
index 0000000..6935b78
@@ -0,0 +1,5 @@
+GVT_SOURCE := gvt.o params.o fb_decoder.o
+
+ccflags-y			+= -I$(src) -I$(src)/.. -Wall -Werror -Wno-unused-function
+i915_gvt-y			:= $(GVT_SOURCE)
+obj-$(CONFIG_I915_GVT)		+= i915_gvt.o
drivers/gpu/drm/i915/gvt/debug.h new file +38 -0
new file mode 100644
index 0000000..18e1467
@@ -0,0 +1,72 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef __GVT_DEBUG_H__
+#define __GVT_DEBUG_H__
+
+#define ASSERT(x)                                                       \
+        do {                                                            \
+                if (!(x)) {                                             \
+                        printk("Assert at %s line %d\n",                \
+                                __FILE__, __LINE__);                    \
+                }                                                       \
+        } while (0);
+
+#define ASSERT_NUM(x, y)                                                \
+        do {                                                            \
+                if (!(x)) {                                             \
+                        printk("Assert at %s line %d para 0x%llx\n",    \
Quoted 9 lines

drivers/gpu/drm/i915/gvt/debug.h line 42

Quoted 4 lines
drivers/gpu/drm/i915/gvt/debug.h new file +4 -0
+                                __FILE__, __LINE__, (u64)y);            \
+                }                                                       \
+        } while (0);
+

There already is WARN_ON (and i915_drv.h modifies it a little). Do not introduce custom functions like this, if the existing ones need improvement, improve them.

[Zhi] OK. Will do as what i915 does

Quoted 25 lines
Quoted 25 lines
drivers/gpu/drm/i915/gvt/debug.h new file +25 -0
+#define gvt_info(fmt, args...) \
+	printk(KERN_INFO"[GVT-g] "fmt"\n", ##args)
+
+#define gvt_err(fmt, args...) \
+	printk(KERN_ERR"%s() - %d: "fmt"\n", __func__, __LINE__, ##args)
+
+#define gvt_warn(fmt, args...) \
+	printk(KERN_WARNING"%s() - %d: "fmt"\n", __func__, __LINE__, ##args)
+
+#define gvt_dbg(level, fmt, args...) do { \
+		if (gvt.debug & level) \
+			printk(KERN_DEBUG"%s() - %d: "fmt"\n", __func__, __LINE__, ##args); \
+	}while(0)
+
+enum {
+	GVT_DBG_CORE = (1 << 0),
+	GVT_DBG_MM = (1 << 1),
+	GVT_DBG_IRQ = (1 << 2),
+};
+
+#define gvt_dbg_core(fmt, args...) \
+	gvt_dbg(GVT_DBG_CORE, fmt, ##args)
+
+#define gvt_dbg_mm(fmt, args...) \
+	gvt_dbg(GVT_DBG_MM, fmt, ##args)
Quoted 9 lines

drivers/gpu/drm/i915/gvt/debug.h line 72

Quoted 5 lines
drivers/gpu/drm/i915/gvt/debug.h new file +5 -0
+
+#define gvt_dbg_irq(fmt, args...) \
+	gvt_dbg(GVT_DBG_IRQ, fmt, ##args)
+
+#endif

This should be integrated better to DRM debugging options, custom debugging code only for i915 was rejected a while ago.

[ZHI] Thanks for sharing the history. :) Will change that.

Quoted 37 lines
Quoted 37 lines
drivers/gpu/drm/i915/gvt/fb_decoder.c new file +31 -0
new file mode 100644
index 0000000..a219819
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include "gvt.h"
+
+int gvt_decode_fb_format(struct pgt_device *pdev, int vmid, struct gvt_fb_format *fb)
+{
+	return 0;
+}
+
+int gvt_fb_notifier_call_chain(unsigned long val, void *data)
Quoted 9 lines

drivers/gpu/drm/i915/gvt/fb_decoder.c line 29

Quoted 3 lines
drivers/gpu/drm/i915/gvt/fb_decoder.c new file +3 -0
+{
+	return 0;
+}

Kerneldoc missing for these functions. It is all the same to squash later patches to introduce the code to these functions already, reviewing utility functions with no kerneldoc and no body makes it somewhat difficult to see the big picture.

Quoted 2 lines

[Zhi] One question. I saw i915 put some function instruction in *.c. Is it also OK for kernel doc generating the proper doc files?

Yes, (again referring to history) there was a suggestion to move all the documentation to header files and it was decided on that the documentation should RATHER be in the implementation than header files. Then it's closer to the implementation when on is changing the function and one has a higher chance of updating it, rather than switching between header and implementation.

Quoted 190 lines
Quoted 98 lines
Quoted 98 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +91 -0
new file mode 100644
index 0000000..2c29ed4
@@ -0,0 +1,110 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_FB_DECODER_H_
+#define _GVT_FB_DECODER_H_
+
+typedef enum {
+	FB_MODE_SET_START = 1,
+	FB_MODE_SET_END,
+	FB_DISPLAY_FLIP,
+}gvt_fb_event_t;
+
+typedef enum {
+	DDI_PORT_NONE	= 0,
+	DDI_PORT_B	= 1,
+	DDI_PORT_C	= 2,
+	DDI_PORT_D	= 3,
+	DDI_PORT_E	= 4
+} ddi_port_t;
+
+struct pgt_device;
+
+struct gvt_fb_notify_msg {
+	unsigned vm_id;
+	unsigned pipe_id; /* id starting from 0 */
+	unsigned plane_id; /* primary, cursor, or sprite */
+};
+
+/* color space conversion and gamma correction are not included */
+struct gvt_primary_plane_format {
+	u8	enabled;	/* plane is enabled */
+	u8	tiled;		/* X-tiled */
+	u8	bpp;		/* bits per pixel */
+	u32	hw_format;	/* format field in the PRI_CTL register */
+	u32	drm_format;	/* format in DRM definition */
+	u32	base;		/* framebuffer base in graphics memory */
+	u32	x_offset;	/* in pixels */
+	u32	y_offset;	/* in lines */
+	u32	width;		/* in pixels */
+	u32	height;		/* in lines */
+	u32	stride;		/* in bytes */
+};
+
+struct gvt_sprite_plane_format {
+	u8	enabled;	/* plane is enabled */
+	u8	tiled;		/* X-tiled */
+	u8	bpp;		/* bits per pixel */
+	u32	hw_format;	/* format field in the SPR_CTL register */
+	u32	drm_format;	/* format in DRM definition */
+	u32	base;		/* sprite base in graphics memory */
+	u32	x_pos;		/* in pixels */
+	u32	y_pos;		/* in lines */
+	u32	x_offset;	/* in pixels */
+	u32	y_offset;	/* in lines */
+	u32	width;		/* in pixels */
+	u32	height;		/* in lines */
+};
+
+struct gvt_cursor_plane_format {
+	u8	enabled;
+	u8	mode;		/* cursor mode select */
+	u8	bpp;		/* bits per pixel */
+	u32	drm_format;	/* format in DRM definition */
+	u32	base;		/* cursor base in graphics memory */
+	u32	x_pos;		/* in pixels */
+	u32	y_pos;		/* in lines */
+	u8	x_sign;		/* X Position Sign */
+	u8	y_sign;		/* Y Position Sign */
+	u32	width;		/* in pixels */
+	u32	height;		/* in lines */
+	u32	x_hot;		/* in pixels */
Quoted 9 lines

drivers/gpu/drm/i915/gvt/fb_decoder.h line 94

Quoted 3 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +3 -0
+	u32	y_hot;		/* in pixels */
+};
+

The above structs have a lot in common, would it make sense to have the common members + plane type and then union for the plane type specific data. I suspect having it all split this way will lead to more utility functions somewhere.

[Zhi] Thanks. Will do that.

Quoted 72 lines
Quoted 46 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +16 -0
+struct gvt_pipe_format {
+	struct gvt_primary_plane_format	primary;
+	struct gvt_sprite_plane_format	sprite;
+	struct gvt_cursor_plane_format	cursor;
+	ddi_port_t ddi_port;  /* the DDI port that the pipe is connected to */
+};
+
+struct gvt_fb_format{
+	struct gvt_pipe_format	pipes[I915_MAX_PIPES];
+};
+
+extern int gvt_fb_notifier_call_chain(unsigned long val, void *data);
+extern int gvt_decode_fb_format(struct pgt_device *pdev, int vmid,
+				struct gvt_fb_format *fb);
+
+#endif
drivers/gpu/drm/i915/gvt/gvt.c new file +24 -0
new file mode 100644
index 0000000..041d10f
@@ -0,0 +1,366 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include

drivers/gpu/drm/i915/gvt/gvt.c line 34

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +10 -0
+#include
+
+#include "gvt.h"
+
+struct gvt_host gvt_host;
+
+extern struct gvt_kernel_dm xengt_kdm;
+extern struct gvt_kernel_dm kvmgt_kdm;
+
+static const char *supported_hypervisors[] = {

should be "static const char * const".

Quoted 13 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +13 -0
+	[GVT_HYPERVISOR_TYPE_XEN] = "Xen Hypervisor",
+	[GVT_HYPERVISOR_TYPE_KVM] = "KVM",
+};
+
+static bool gvt_init_host(void)
+{
+	struct gvt_host *host = &gvt_host;
+
+	if (!gvt.enable) {
+		gvt_dbg_core("GVT-g has been disabled by kernel parameter");
+		return false;
+	}
+
Quoted 9 lines

drivers/gpu/drm/i915/gvt/gvt.c line 53

Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +6 -0
+	if (host->initialized) {
+		gvt_err("GVT-g has already been initialized!");
+		return false;
+	}
+
+	if (xen_initial_domain()) {

Shouldn't Xen code be CONFIG_XEN_DOM0 and CONFIG_XEN #ifdef protected?

Quoted 9 lines

[Zhi] The following code piece shows xen_initial_domain()/xen_domain() could also be used when CONFIG_XEN_DOM0 is not set.

#ifdef CONFIG_XEN
extern enum xen_domain_type xen_domain_type;
#else
#define xen_domain_type         XEN_NATIVE
#endif

Ok, then it's cool. I missed this construct :)

Quoted 33 lines

#define xen_domain()            (xen_domain_type != XEN_NATIVE) #define xen_pv_domain()         (xen_domain() &&                        \
                                  xen_domain_type == XEN_PV_DOMAIN) #define xen_hvm_domain()        (xen_domain() &&                        \
                                  xen_domain_type == XEN_HVM_DOMAIN)

#ifdef CONFIG_XEN_DOM0
#include
#include

#define xen_initial_domain()    (xen_domain() && \
                                  xen_start_info && xen_start_info->flags & SIF_INITDOMAIN) #else  /* !CONFIG_XEN_DOM0 */
#define xen_initial_domain()    (0)
#endif  /* CONFIG_XEN_DOM0 */

Quoted 7 lines
Quoted 7 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +7 -0
+		/* Xen Dom0 */
+		host->kdm = try_then_request_module(symbol_get(xengt_kdm), "xengt");
+		host->hypervisor_type = GVT_HYPERVISOR_TYPE_XEN;
+	} else if(xen_domain()) {
+		/* Xen DomU */
+		return false;
+	} else {
Quoted 8 lines

drivers/gpu/drm/i915/gvt/gvt.c line 65

Quoted 5 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +5 -0
+		/* not in Xen. Try KVMGT */
+		host->kdm = try_then_request_module(symbol_get(kvmgt_kdm), "kvm");
+		host->hypervisor_type = GVT_HYPERVISOR_TYPE_KVM;
+	}
+

Why do we need to keep track of hypervisor type, are there plenty of hypervisor specific behaviour differences?

Quoted 7 lines

[Zhi] As GVT-g needs some hypervisor services to work, we write a abstraction layer to connect GVT-g to different hypervisor. But still there are some emulation logic couldn't be fitted into that abstraction layer, like OpRegion emulation. In Xen, we emulates it in kernel space, while we emulates it in Qemu under KVM. I also agreed that we should find some approach to improve it just like what you said.

Yep, if correcting it is not currently possible, I can live with it. But host->hypervisor_type dependent code should be kept to absolute minimum.

Regards, Joonas

Quoted 836 lines
Quoted 39 lines

If it is just for printing
the below debug, I think it's better to move the debug into above code that detects the hypervisor, wrap them with appropriate CONFIG #ifdefs.

Quoted 35 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +35 -0
+	if (!host->kdm)
+		return false;
+
+	if (!hypervisor_detect_host())
+		return false;
+
+	gvt_info("Running with hypervisor %s in host mode",
+			supported_hypervisors[host->hypervisor_type]);
+
+	idr_init(&host->device_idr);
+	mutex_init(&host->device_idr_lock);
+
+	host->initialized = true;
+	return true;
+}
+
+static bool init_device_info(struct pgt_device *pdev)
+{
+	struct gvt_device_info *info = &pdev->device_info;
+
+	if (!IS_BROADWELL(pdev->dev_priv)) {
+		gvt_err("Unsupported GEN device");
+		return false;
+	}
+
+	if (IS_BROADWELL(pdev->dev_priv)) {
+		info->max_gtt_gm_sz = (1UL << 32);
+		/*
+		 * The layout of BAR0 in BDW:
+		 * |< - MMIO 2MB ->|<- Reserved 6MB ->|<- MAX GTT 8MB->|
+		 *
+		 * GTT offset in BAR0 starts from 8MB to 16MB, and
+		 * Whatever GTT size is configured in BIOS,
+		 * the size of BAR0 is always 16MB. The actual configured
+		 * GTT size can be found in GMCH_CTRL.
Quoted 9 lines

drivers/gpu/drm/i915/gvt/gvt.c line 106

Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +6 -0
+		 */
+		info->gtt_start_offset = (1UL << 23);
+		info->max_gtt_size = (1UL << 23);
+		info->gtt_entry_size = 8;
+		info->gtt_entry_size_shift = 3;
+		info->gmadr_bytes_in_cmd = 8;

Would this information be useful in a header as #defines too?

[Zhi] Probably. Consider that one GVT-g featured kernel could be able to run on different platforms, the informaion data structure could be different. But I agree to define the magic numbers in a header files. :)

Quoted 4 lines
Quoted 4 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +4 -0
+	}
+
+	gvt_info("Device info:");
+	printk("        max_gtt_gm_sz: %llx\n", info->max_gtt_gm_sz);
Quoted 9 lines

drivers/gpu/drm/i915/gvt/gvt.c line 116

Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +6 -0
+	printk("        max_gtt_size: %x\n", info->max_gtt_size);
+	printk("        gtt_size_entry: %x\n", info->gtt_entry_size);
+	printk("        gtt_entry_size_shift: %x\n", info->gtt_entry_size_shift);
+	printk("        gtt_start_offset: %x\n", info->gtt_start_offset);
+	printk("        gtt_end_offset: %x\n", info->gtt_end_offset);
+

Just put this to kind of stuff to i915_debugfs.c.

[Zhi] Thanks.

Quoted 93 lines

drivers/gpu/drm/i915/gvt/gvt.c line 127

Quoted 11 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +11 -0
+	return true;
+}
+
+static void init_initial_cfg_space_state(struct pgt_device *pdev)
+{
+	struct pci_dev *pci_dev = pdev->dev_priv->dev->pdev;
+	int i;
+
+	gvt_dbg_core("init initial cfg space, id %d", pdev->id);
+
+	for (i = 0; i < GVT_CFG_SPACE_SZ; i += 4)

+= sizeof(...)

drivers/gpu/drm/i915/gvt/gvt.c line 131

Quoted 4 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +4 -0
+		pci_read_config_dword(pci_dev, i,
+				(u32 *)&pdev->initial_cfg_space[i]);
+
+	for (i = 0; i < 3; i++) {

No magic numbers make a #define for 3 and give it a descriptive name.

Quoted 22 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +22 -0
+		pdev->bar_size[i] = pci_resource_len(pci_dev, i * 2);
+		gvt_info("bar %d size: %llx", i, pdev->bar_size[i]);
+	}
+}
+
+static void clean_initial_mmio_state(struct pgt_device *pdev)
+{
+	if (pdev->gttmmio_va) {
+		iounmap(pdev->gttmmio_va);
+		pdev->gttmmio_va = NULL;
+	}
+
+	if (pdev->gmadr_va) {
+		iounmap(pdev->gmadr_va);
+		pdev->gmadr_va = NULL;
+	}
+}
+
+static bool init_initial_mmio_state(struct pgt_device *pdev)
+{
+	u64 bar0, bar1;
+

drivers/gpu/drm/i915/gvt/gvt.c line 163

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +10 -0
+	gvt_dbg_core("init initial mmio state, id %d", pdev->id);
+
+	bar0 = *(u64 *)&pdev->initial_cfg_space[GVT_REG_CFG_SPACE_BAR0];
+	bar1 = *(u64 *)&pdev->initial_cfg_space[GVT_REG_CFG_SPACE_BAR1];
+
+	pdev->gttmmio_base = bar0 & ~0xf;
+	pdev->mmio_size = 2 * 1024 * 1024;
+	pdev->reg_num = pdev->mmio_size / 4;
+	pdev->gmadr_base = bar1 & ~0xf;
+

Many magic numbers.

drivers/gpu/drm/i915/gvt/gvt.c line 166

Quoted 3 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +3 -0
+	pdev->gttmmio_va = ioremap(pdev->gttmmio_base, pdev->bar_size[0]);
+	if (!pdev->gttmmio_va) {
+		gvt_err("fail to map GTTMMIO BAR.");

These should be if(WARN_ON(...))

Quoted 31 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +31 -0
+		return false;
+	}
+
+	pdev->gmadr_va = ioremap(pdev->gmadr_base, pdev->bar_size[2]);
+	if (!pdev->gmadr_va) {
+		gvt_err("fail to map GMADR BAR.");
+		goto err;
+	}
+
+	gvt_info("bar0: 0x%llx, bar1: 0x%llx", bar0, bar1);
+	gvt_info("mmio size: %x", pdev->mmio_size);
+	gvt_info("gttmmio: 0x%llx, gmadr: 0x%llx", pdev->gttmmio_base, pdev->gmadr_base);
+	gvt_info("gttmmio_va: %p", pdev->gttmmio_va);
+	gvt_info("gmadr_va: %p", pdev->gmadr_va);
+
+	return true;
+err:
+	clean_initial_mmio_state(pdev);
+	return false;
+}
+
+static int gvt_service_thread(void *data)
+{
+	struct pgt_device *pdev = (struct pgt_device *)data;
+	int r;
+
+	gvt_dbg_core("service thread start, pgt %d", pdev->id);
+
+	while(!kthread_should_stop()) {
+		r = wait_event_interruptible(pdev->service_thread_wq,
+				kthread_should_stop() || pdev->service_request);
Quoted 9 lines

drivers/gpu/drm/i915/gvt/gvt.c line 203

Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +6 -0
+
+		if (kthread_should_stop())
+			break;
+
+		if (r) {
+			gvt_warn("service thread is waken up by unexpected signal.");

Should be WARN_ONCE, to avoid future disasters with CI.

[Zhi] Thanks.

Quoted 305 lines
Quoted 256 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +163 -0
+			continue;
+		}
+	}
+	return 0;
+}
+
+static void clean_service_thread(struct pgt_device *pdev)
+{
+	if (pdev->service_thread) {
+		kthread_stop(pdev->service_thread);
+		pdev->service_thread = NULL;
+	}
+}
+
+static bool init_service_thread(struct pgt_device *pdev)
+{
+	init_waitqueue_head(&pdev->service_thread_wq);
+
+	pdev->service_thread = kthread_run(gvt_service_thread,
+			pdev, "gvt_service_thread%d", pdev->id);
+
+	if (!pdev->service_thread) {
+		gvt_err("fail to start service thread.");
+		return false;
+	}
+
+	return true;
+}
+
+static void clean_pgt_device(struct pgt_device *pdev)
+{
+	clean_service_thread(pdev);
+	clean_initial_mmio_state(pdev);
+}
+
+static bool init_pgt_device(struct pgt_device *pdev, struct drm_i915_private *dev_priv)
+{
+	if (!init_device_info(pdev))
+		return false;
+
+	init_initial_cfg_space_state(pdev);
+
+	if (!init_initial_mmio_state(pdev))
+		goto err;
+
+	if (!init_service_thread(pdev))
+		goto err;
+
+	return true;
+err:
+	clean_pgt_device(pdev);
+	return false;
+}
+
+static bool post_init_pgt_device(struct pgt_device *pdev)
+{
+	return true;
+}
+
+static void free_pgt_device(struct pgt_device *pdev)
+{
+	struct gvt_host *host = &gvt_host;
+
+	mutex_lock(&host->device_idr_lock);
+	idr_remove(&host->device_idr, pdev->id);
+	mutex_unlock(&host->device_idr_lock);
+
+	vfree(pdev);
+}
+
+static struct pgt_device *alloc_pgt_device(struct drm_i915_private *dev_priv)
+{
+	struct gvt_host *host = &gvt_host;
+	struct pgt_device *pdev = NULL;
+
+	pdev = vzalloc(sizeof(*pdev));
+	if (!pdev) {
+		gvt_err("fail to allocate memory for pgt device.");
+		return NULL;
+	}
+
+	mutex_lock(&host->device_idr_lock);
+	pdev->id = idr_alloc(&host->device_idr, pdev, 0, 0, GFP_KERNEL);
+	mutex_unlock(&host->device_idr_lock);
+
+	if (pdev->id < 0) {
+		gvt_err("fail to allocate pgt device id.");
+		goto err;
+	}
+
+	mutex_init(&pdev->lock);
+	pdev->dev_priv = dev_priv;
+	idr_init(&pdev->instance_idr);
+
+	return pdev;
+err:
+	free_pgt_device(pdev);
+	return NULL;
+}
+
+void gvt_destroy_pgt_device(void *private_data)
+{
+	struct pgt_device *pdev = (struct pgt_device *)private_data;
+
+	clean_pgt_device(pdev);
+	free_pgt_device(pdev);
+}
+
+void *gvt_create_pgt_device(struct drm_i915_private *dev_priv)
+{
+	struct pgt_device *pdev = NULL;
+	struct gvt_host *host = &gvt_host;
+
+	if (!host->initialized && !gvt_init_host()) {
+		gvt_err("gvt_init_host fail");
+		return NULL;
+	}
+
+	gvt_dbg_core("create new pgt device, i915 dev_priv: %p", dev_priv);
+
+	pdev = alloc_pgt_device(dev_priv);
+	if (!pdev) {
+		gvt_err("fail to allocate memory for pgt device.");
+		goto err;
+	}
+
+	gvt_dbg_core("init pgt device, id %d", pdev->id);
+
+	if (!init_pgt_device(pdev, dev_priv)) {
+		gvt_err("fail to init physical device state.");
+		goto err;
+	}
+
+	gvt_dbg_core("pgt device creation done, id %d", pdev->id);
+
+	return pdev;
+err:
+	if (pdev) {
+		gvt_destroy_pgt_device(pdev);
+		pdev = NULL;
+	}
+	return NULL;
+}
+
+bool gvt_post_init_pgt_device(void *private_data)
+{
+	struct pgt_device *pdev = (struct pgt_device *)private_data;
+	struct gvt_host *host = &gvt_host;
+
+	if (!host->initialized) {
+		gvt_err("gvt_host haven't been initialized.");
+		return false;
+	}
+
+	gvt_dbg_core("post init pgt device %d", pdev->id);
+
+	if (!post_init_pgt_device(pdev)) {
+		gvt_err("fail to post init physical device state.");
+		return false;
+	}
+
+	return true;
+}
drivers/gpu/drm/i915/gvt/gvt.h new file +87 -0
new file mode 100644
index 0000000..6c85bba
@@ -0,0 +1,130 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_H_
+#define _GVT_H_
+
+#include "i915_drv.h"
+#include "i915_vgpu.h"
+
+#include "debug.h"
+#include "params.h"
+#include "reg.h"
+#include "hypercall.h"
+#include "mpt.h"
+#include "fb_decoder.h"
+
+#define GVT_MAX_VGPU 8
+
+enum {
+	GVT_HYPERVISOR_TYPE_XEN = 0,
+	GVT_HYPERVISOR_TYPE_KVM,
+};
+
+struct gvt_host {
+	bool initialized;
+	int hypervisor_type;
+	struct mutex device_idr_lock;
+	struct idr device_idr;
+	struct gvt_kernel_dm *kdm;
+};
+
+extern struct gvt_host gvt_host;
+
+/* Describe the limitation of HW.*/
+struct gvt_device_info {
+	u64 max_gtt_gm_sz;
+	u32 gtt_start_offset;
+	u32 gtt_end_offset;
+	u32 max_gtt_size;
+	u32 gtt_entry_size;
+	u32 gtt_entry_size_shift;
+	u32 gmadr_bytes_in_cmd;
+};
+
+struct vgt_device {
+	int id;
+	int vm_id;
+	struct pgt_device *pdev;
+	bool warn_untrack;
+};
+
+struct pgt_device {
+	struct mutex lock;
+	int id;
+
+	struct drm_i915_private *dev_priv;
+	struct idr instance_idr;
+
+	struct gvt_device_info device_info;
+
+	u8 initial_cfg_space[GVT_CFG_SPACE_SZ];
+	u64 bar_size[GVT_BAR_NUM];
+
+	u64 gttmmio_base;
+	void *gttmmio_va;
+
+	u64 gmadr_base;

drivers/gpu/drm/i915/gvt/gvt.h line 97

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +10 -0
+	void *gmadr_va;
+
+	u32 mmio_size;
+	u32 reg_num;
+
+	wait_queue_head_t service_thread_wq;
+	struct task_struct *service_thread;
+	unsigned long service_request;
+};
+

Here-->

Quoted 22 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +22 -0
+static inline u32 gvt_mmio_read(struct pgt_device *pdev,
+		u32 reg)
+{
+	struct drm_i915_private *dev_priv = pdev->dev_priv;
+	i915_reg_t tmp = {.reg = reg};
+	return I915_READ(tmp);
+}
+
+static inline void gvt_mmio_write(struct pgt_device *pdev,
+		u32 reg, u32 val)
+{
+	struct drm_i915_private *dev_priv = pdev->dev_priv;
+	i915_reg_t tmp = {.reg = reg};
+	I915_WRITE(tmp, val);
+}
+
+static inline u64 gvt_mmio_read64(struct pgt_device *pdev,
+		u32 reg)
+{
+	struct drm_i915_private *dev_priv = pdev->dev_priv;
+	i915_reg_t tmp = {.reg = reg};
+	return I915_READ64(tmp);

drivers/gpu/drm/i915/gvt/gvt.h line 129

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +10 -0
+}
+
+static inline void gvt_mmio_write64(struct pgt_device *pdev,
+		u32 reg, u64 val)
+{
+	struct drm_i915_private *dev_priv = pdev->dev_priv;
+	i915_reg_t tmp = {.reg = reg};
+	I915_WRITE64(tmp, val);
+}
+

<-- Why? The i915_reg_t type was added to avoid problems, this code is not used anywhere, and it has no documentation, so I can not review it.

Quoted 2 lines

I wrote comments at the top of the post.

[Zhi] Thanks, I will check that.

Quoted 357 lines

Regards, Joonas

Quoted 355 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +1 -0
+#endif
drivers/gpu/drm/i915/gvt/hypercall.h new file +30 -0
new file mode 100644
index 0000000..0a41874
@@ -0,0 +1,30 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_HYPERCALL_H_
+#define _GVT_HYPERCALL_H_
+
+struct gvt_kernel_dm {
+};
+
+#endif /* _GVT_HYPERCALL_H_ */
drivers/gpu/drm/i915/gvt/mpt.h new file +97 -0
new file mode 100644
index 0000000..bbe4465
@@ -0,0 +1,97 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_MPT_H_
+#define _GVT_MPT_H_
+
+struct vgt_device;
+
+static inline unsigned long hypervisor_g2m_pfn(struct vgt_device *vgt,
+	unsigned long g_pfn)
+{
+	return 0;
+}
+
+static inline int hypervisor_pause_domain(struct vgt_device *vgt)
+{
+	return 0;
+}
+
+static inline int hypervisor_shutdown_domain(struct vgt_device *vgt)
+{
+	return 0;
+}
+
+static inline int hypervisor_set_trap_area(struct vgt_device *vgt,
+	uint64_t start, uint64_t end, bool map)
+{
+	return 0;
+}
+
+static inline bool hypervisor_detect_host(void)
+{
+	return false;
+}
+
+static inline int hypervisor_virt_to_mfn(void *addr)
+{
+	return 0;
+}
+
+static inline void *hypervisor_mfn_to_virt(int mfn)
+{
+	return NULL;
+}
+
+static inline void hypervisor_inject_msi(struct vgt_device *vgt)
+{
+	return;
+}
+
+static inline int hypervisor_hvm_init(struct vgt_device *vgt)
+{
+	return 0;
+}
+
+static inline void hypervisor_hvm_exit(struct vgt_device *vgt)
+{
+}
+
+static inline void *hypervisor_gpa_to_va(struct vgt_device *vgt, unsigned long gpa)
+{
+	return NULL;
+}
+
+static inline bool hypervisor_read_va(struct vgt_device *vgt, void *va,
+		void *val, int len, int atomic)
+{
+	return false;
+}
+
+static inline bool hypervisor_write_va(struct vgt_device *vgt, void *va,
+		void *val, int len, int atomic)
+{
+	return false;
+}
+
+#endif /* _GVT_MPT_H_ */
drivers/gpu/drm/i915/gvt/params.c new file +29 -0
new file mode 100644
index 0000000..dfc33c3
@@ -0,0 +1,29 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include "gvt.h"
+
+struct gvt_kernel_params gvt = {
+	.enable = true,
+	.debug = 0,
+};
drivers/gpu/drm/i915/gvt/params.h new file +34 -0
new file mode 100644
index 0000000..d2955b9
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_PARAMS_H_
+#define _GVT_PARAMS_H_
+
+struct gvt_kernel_params {
+	bool enable;
+	int debug;
+};
+
+extern struct gvt_kernel_params gvt;
+
+#endif
drivers/gpu/drm/i915/gvt/reg.h new file +34 -0
new file mode 100644
index 0000000..d363b74
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_REG_H
+#define _GVT_REG_H
+
+#define GVT_CFG_SPACE_SZ	256
+#define GVT_BAR_NUM		4
+
+#define GVT_REG_CFG_SPACE_BAR0	0x10
+#define GVT_REG_CFG_SPACE_BAR1	0x18
+#define GVT_REG_CFG_SPACE_BAR2	0x20
+
+#endif
drivers/gpu/drm/i915/i915_dma.c +11 -0
index 4725e8d..eca8e50 100644
@@ -943,6 +943,10 @@ int i915_driver_load(struct drm_device *dev, unsigned long flags)
 
   	intel_uncore_init(dev);
 
+	dev_priv->vgpu.host_private_data = gvt_create_pgt_device(dev_priv);
+	if(intel_gvt_host_active(dev))
+		DRM_INFO("GVT-g is running in host mode\n");
+
   	ret = i915_gem_gtt_init(dev);
   	if (ret)
   		goto out_freecsr;
@@ -1067,6 +1071,13 @@ int i915_driver_load(struct drm_device *dev, unsigned long flags)
   		goto out_power_well;
   	}
 
+	if (intel_gvt_host_active(dev)) {
+		if (!gvt_post_init_pgt_device(dev_priv->vgpu.host_private_data)) {
+			DRM_ERROR("failed to post init pgt device\n");
+			goto out_power_well;
+		}
+	}
+
   	/*
   	 * Notify a valid surface after modesetting,
   	 * when running inside a VM.
@@ -1117,6 +1128,10 @@ out_gtt:
   	i915_global_gtt_cleanup(dev);

  out_freecsr:
   intel_csr_ucode_fini(dev_priv);

drivers/gpu/drm/i915/i915_dma.c +4 -0
+	if (intel_gvt_host_active(dev)) {
+		gvt_destroy_pgt_device(dev_priv->vgpu.host_private_data);
+		dev_priv->vgpu.host_private_data = NULL;
+	}

   intel_uncore_fini(dev);
   pci_iounmap(dev->pdev, dev_priv->regs);
  put_bridge:

drivers/gpu/drm/i915/i915_dma.c +4 -0
@@ -1165,6 +1180,10 @@ int i915_driver_unload(struct drm_device *dev)
 
   	intel_modeset_cleanup(dev);
 
+	if (intel_gvt_host_active(dev)) {
+		gvt_destroy_pgt_device(dev_priv->vgpu.host_private_data);
+		dev_priv->vgpu.host_private_data = NULL;
+	}
   	/*
   	 * free the memory space allocated for the child device
   	 * config parsed from VBT
drivers/gpu/drm/i915/i915_drv.h +6 -0
index 01cc982..db3c79b 100644
@@ -1673,6 +1673,7 @@ struct i915_workarounds {
 
   struct i915_virtual_gpu {
   	bool active;
+	void *host_private_data;
   };
 
   struct i915_execbuffer_params {
@@ -2747,6 +2748,11 @@ static inline bool intel_vgpu_active(struct drm_device *dev)
   	return to_i915(dev)->vgpu.active;
   }
 
+static inline bool intel_gvt_host_active(struct drm_device *dev)
+{
+	return to_i915(dev)->vgpu.host_private_data ? true : false;
+}
+

  void
  i915_enable_pipestat(struct drm_i915_private *dev_priv, enum pipe pipe, u32 status_mask);

drivers/gpu/drm/i915/i915_vgpu.h +3 -0
index 3c83b47..942490a 100644
@@ -113,5 +113,8 @@ struct vgt_if {
   extern void i915_check_vgpu(struct drm_device *dev);
   extern int intel_vgt_balloon(struct drm_device *dev);
   extern void intel_vgt_deballoon(void);
+extern void *gvt_create_pgt_device(struct drm_i915_private *dev_priv);
+extern bool gvt_post_init_pgt_device(void *private_data);
+extern void gvt_destroy_pgt_device(void *private_data);
 
   #endif /* _I915_VGPU_H_ */
include/xen/interface/xen.h +1 -0
index d133112..78a38f1 100644
@@ -28,6 +28,7 @@
   #define __XEN_PUBLIC_XEN_H__
 
   #include
+#include
 
   /*
    * XEN "SYSTEM CALLS" (a.k.a. HYPERCALLS).
Signature · 8 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#9 lore raw

Re: [RFC 01/29] drm/i915/gvt: Introduce the basic architecture of GVT-g

In reply to Zhi Wang

Hi Joonas:
For the debug function/macros, could we use DRM_DEBUG_DRIVER as
output, but keep our internal debug switch & marcos, like:

#define gvt_dbg_xxx(xxx, xxx) \
	if (gvt_debug & xxx) \
		DRM_DEBUG_DRIVER(xxxxxx)

Is it OK for the maintainers? Or we have to use DRM_DEBUG_DRIVER directly :(.

Thanks,
Zhi.

Quoted 1,573 lines · Zhi Wang

Hi Joonas:
Thanks you very much! We're very excited for receiving your advice
and continue to be open to any comments. :)

I'm supposed that we should make the agreements on i915 host change at the very beginning, as I'm concerned during the discussion of i915 host change, you know, maybe some code of GVT-g needs to be refined or re-designed. So we put the i915 host changes to the beginning of the patch set for the convenience of discussion.

I summarize your comments as below, very applicate. :)

- Replace boolean return value with int as much as possible.
- Replace customized debug ASSERT() function & macros with DRM_DEBUG_*
- Document all non-static functions like i915
- Fix all whitespace via scripts/cleanpatch.pl
- Commit function structure refinement.
- Change the register access behavior just like what i915 does.

For other comments, see my comments below. :)

Quoted 113 lines · Joonas Lahtinen

Hi,

TL;DR Overall, we have same problem as with the scheduler series, there is too much placeholder stuff for easy review. Just squash enough code into one commit so it actually does something logical that can be reviewed and then extend it later. Then it can be reviewed and pushed. Just splitting the code down to achieve smaller patches is not the right thing to do.

Comments on the overall code: You need to document all header file functions (in the source files), and it is good to document the static functions within a file too, to make future maintenance easier.

It is not about splitting the code down to small chunks, but splitting it down to small *logical* chunks. It doesn't make sense to commit dozens of empty bodied functions for review, and then later add their code.

If you add functions, only add them at a patch that takes them into use too, unless we're talking about general purpose shared code. And also remember to add the function body and documentation header. If you simply add a "return 0;" or similar function body, do add a comment to why the function does not exist and when it will.

Then, there is a trend of having a boolean return values in the code. When possible, it should rather be int and the cause for failure should be propagated from the last level all the way to up (-ENOMEN etc.). This way debugging becomes easier and if new error conditions appear, there is less of a maintenance burden to add the propagation later.

Finally, make sure to look at the existing driver parts and
https://www.kernel.org/doc/Documentation/CodingStyle
for proper coding style. There are lots of whitespace fixes needed in this series, like array initializations.

I hope to see this first patch rerolled so that you squash some of the later commits into it so that all functions have a body and you add documentation for the functions so I can both see what it should do and what it actually does. Only reroll the first patch, to keep the iterative step smaller. Lets only then continue with the rest of the series once we've reached a consensus on the formatting and style basics.

See more comments below.

Quoted 41 lines · Zhi Wang

This patch introduces the very basic framework of GVT-g device model, includes basic prototypes, definitions, initialization.

---
  arch/x86/include/asm/xen/interface.h  |   3 +
  drivers/gpu/drm/i915/Kconfig          |  16 ++
  drivers/gpu/drm/i915/Makefile         |   2 +
  drivers/gpu/drm/i915/gvt/Makefile     |   5 +
  drivers/gpu/drm/i915/gvt/debug.h      |  72 +++++++
  drivers/gpu/drm/i915/gvt/fb_decoder.c |  34 ++++
  drivers/gpu/drm/i915/gvt/fb_decoder.h | 110 ++++++++++
  drivers/gpu/drm/i915/gvt/gvt.c        | 366
++++++++++++++++++++++++++++++++++
  drivers/gpu/drm/i915/gvt/gvt.h        | 130 ++++++++++++
  drivers/gpu/drm/i915/gvt/hypercall.h  |  30 +++
  drivers/gpu/drm/i915/gvt/mpt.h        |  97 +++++++++
  drivers/gpu/drm/i915/gvt/params.c     |  29 +++
  drivers/gpu/drm/i915/gvt/params.h     |  34 ++++
  drivers/gpu/drm/i915/gvt/reg.h        |  34 ++++
  drivers/gpu/drm/i915/i915_dma.c       |  19 ++
  drivers/gpu/drm/i915/i915_drv.h       |   6 +
  drivers/gpu/drm/i915/i915_vgpu.h      |   3 +
  include/xen/interface/xen.h           |   1 +
  18 files changed, 991 insertions(+)

create mode 100644 drivers/gpu/drm/i915/gvt/Makefile create mode 100644 drivers/gpu/drm/i915/gvt/debug.h create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.c create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.h create mode 100644 drivers/gpu/drm/i915/gvt/gvt.c create mode 100644 drivers/gpu/drm/i915/gvt/gvt.h create mode 100644 drivers/gpu/drm/i915/gvt/hypercall.h create mode 100644 drivers/gpu/drm/i915/gvt/mpt.h create mode 100644 drivers/gpu/drm/i915/gvt/params.c create mode 100644 drivers/gpu/drm/i915/gvt/params.h create mode 100644 drivers/gpu/drm/i915/gvt/reg.h

arch/x86/include/asm/xen/interface.h +0 -0
index 62ca03e..6ff4986 100644

arch/x86/include/asm/xen/interface.h line 78

Quoted 6 lines
arch/x86/include/asm/xen/interface.h +3 -0
@@ -73,6 +73,9 @@
  #endif
 
  #ifndef __ASSEMBLY__
+
+#include
+

I don't follow why this would need to be added if the file is not modified otherwise. Each header should only include what they use.

If this is an existing bug (that xen/interface.h can not be included without including linux/types.h), it should be split to a separate patch and sent to Xen team. Same for include/xen/interface/xen.h.

Quoted 12 lines
arch/x86/include/asm/xen/interface.h +0 -0
  /* Explicitly size integers that represent pfns in the public

interface
* with Xen so that on ARM we can have one ABI that works for 32
and 64
* bit guests. */

drivers/gpu/drm/i915/Kconfig +0 -0
index 051eab3..89ff723 100644
@@ -47,3 +47,19 @@ config DRM_I915_PRELIMINARY_HW_SUPPORT
        option changes the default for that module option.
 
Quoted 10 lines

drivers/gpu/drm/i915/Kconfig config DRM_I915_PRELIMINARY_HW_SUPPORT · line 55

Quoted 7 lines
drivers/gpu/drm/i915/Kconfig +6 -0
        If in doubt, say "N".
+
+config I915_GVT
+        tristate "GVT-g host driver"
+        depends on DRM_I915
+        select IRQ_WORK
+        default y

Defaulting to "n" would make sense initially.

[Zhi] Sure, will do.

Quoted 116 lines
Quoted 116 lines
drivers/gpu/drm/i915/Kconfig +2 -0
+        help
+          Enabling GVT-g mediated graphics passthrough technique for

Intel i915
+ based integrated graphics card. With GVT-g, it's possible
to have one
+ integrated i915 device shared by multiple VMs. Performance
critical
+ opterations such as apperture accesses and ring buffer
operations
+ are pass-throughed to VM, with a minimal set of
conflicting resources
+ (e.g. display settings) mediated by vGT driver. The
benefit of vGT
+ is on both the performance, given that each VM could
directly operate
+ its aperture space and submit commands like running on
native, and
+ the feature completeness, given that a true GEN hardware
is exposed.

drivers/gpu/drm/i915/Makefile +2 -0
index 0851de07..d4df410 100644
@@ -91,6 +91,8 @@ i915-y += dvo_ch7017.o \
        intel_sdvo.o \
        intel_tv.o
 
+obj-$(CONFIG_I915_GVT)  += gvt/
+
  # virtual gpu code
  i915-y += i915_vgpu.o
 
drivers/gpu/drm/i915/gvt/Makefile new file +5 -1
new file mode 100644
index 0000000..6935b78
@@ -0,0 +1,5 @@
+GVT_SOURCE := gvt.o params.o fb_decoder.o
+
+ccflags-y            += -I$(src) -I$(src)/.. -Wall -Werror
-Wno-unused-function
+i915_gvt-y            := $(GVT_SOURCE)
+obj-$(CONFIG_I915_GVT)        += i915_gvt.o
drivers/gpu/drm/i915/gvt/debug.h new file +4 -0
new file mode 100644
index 0000000..18e1467
@@ -0,0 +1,72 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/debug.h new file +5 -0
+
+#ifndef __GVT_DEBUG_H__
+#define __GVT_DEBUG_H__
+
+#define

ASSERT(x) \
+ do
{ \
+ if (!(x))
{ \
+ printk("Assert at %s line
%d\n", \
+ __FILE__,
__LINE__); \
+
} \

drivers/gpu/drm/i915/gvt/debug.h new file +3 -0
+        } while (0);
+
+#define ASSERT_NUM(x,

y) \
+ do
{ \
+ if (!(x))
{ \
+ printk("Assert at %s line %d para
0x%llx\n", \
+ __FILE__, __LINE__,
(u64)y); \

Quoted 9 lines

drivers/gpu/drm/i915/gvt/debug.h

Quoted 4 lines

+
} \

drivers/gpu/drm/i915/gvt/debug.h new file +2 -0
+        } while (0);
+

There already is WARN_ON (and i915_drv.h modifies it a little). Do not introduce custom functions like this, if the existing ones need improvement, improve them.

[Zhi] OK. Will do as what i915 does

Quoted 27 lines
Quoted 27 lines
drivers/gpu/drm/i915/gvt/debug.h new file +8 -0
+#define gvt_info(fmt, args...) \
+    printk(KERN_INFO"[GVT-g] "fmt"\n", ##args)
+
+#define gvt_err(fmt, args...) \
+    printk(KERN_ERR"%s() - %d: "fmt"\n", __func__, __LINE__, ##args)
+
+#define gvt_warn(fmt, args...) \
+    printk(KERN_WARNING"%s() - %d: "fmt"\n", __func__, __LINE__,

##args)

drivers/gpu/drm/i915/gvt/debug.h new file +4 -0
+
+#define gvt_dbg(level, fmt, args...) do { \
+        if (gvt.debug & level) \
+            printk(KERN_DEBUG"%s() - %d: "fmt"\n", __func__,

__LINE__, ##args); \

drivers/gpu/drm/i915/gvt/debug.h new file +13 -0
+    }while(0)
+
+enum {
+    GVT_DBG_CORE = (1 << 0),
+    GVT_DBG_MM = (1 << 1),
+    GVT_DBG_IRQ = (1 << 2),
+};
+
+#define gvt_dbg_core(fmt, args...) \
+    gvt_dbg(GVT_DBG_CORE, fmt, ##args)
+
+#define gvt_dbg_mm(fmt, args...) \
+    gvt_dbg(GVT_DBG_MM, fmt, ##args)
Quoted 9 lines

drivers/gpu/drm/i915/gvt/debug.h line 72

Quoted 5 lines
drivers/gpu/drm/i915/gvt/debug.h new file +5 -0
+
+#define gvt_dbg_irq(fmt, args...) \
+    gvt_dbg(GVT_DBG_IRQ, fmt, ##args)
+
+#endif

This should be integrated better to DRM debugging options, custom debugging code only for i915 was rejected a while ago.

[ZHI] Thanks for sharing the history. :) Will change that.

Quoted 52 lines
Quoted 52 lines
drivers/gpu/drm/i915/gvt/fb_decoder.c new file +4 -0
new file mode 100644
index 0000000..a219819
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/fb_decoder.c new file +4 -0
+
+#include "gvt.h"
+
+int gvt_decode_fb_format(struct pgt_device *pdev, int vmid, struct

gvt_fb_format *fb)

drivers/gpu/drm/i915/gvt/fb_decoder.c new file +5 -0
+{
+    return 0;
+}
+
+int gvt_fb_notifier_call_chain(unsigned long val, void *data)
Quoted 9 lines

drivers/gpu/drm/i915/gvt/fb_decoder.c line 29

Quoted 3 lines
drivers/gpu/drm/i915/gvt/fb_decoder.c new file +3 -0
+{
+    return 0;
+}

Kerneldoc missing for these functions. It is all the same to squash later patches to introduce the code to these functions already, reviewing utility functions with no kerneldoc and no body makes it somewhat difficult to see the big picture.

[Zhi] One question. I saw i915 put some function instruction in *.c. Is it also OK for kernel doc generating the proper doc files?

Quoted 111 lines
Quoted 111 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +4 -0
new file mode 100644
index 0000000..2c29ed4
@@ -0,0 +1,110 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/fb_decoder.h new file +69 -0
+
+#ifndef _GVT_FB_DECODER_H_
+#define _GVT_FB_DECODER_H_
+
+typedef enum {
+    FB_MODE_SET_START = 1,
+    FB_MODE_SET_END,
+    FB_DISPLAY_FLIP,
+}gvt_fb_event_t;
+
+typedef enum {
+    DDI_PORT_NONE    = 0,
+    DDI_PORT_B    = 1,
+    DDI_PORT_C    = 2,
+    DDI_PORT_D    = 3,
+    DDI_PORT_E    = 4
+} ddi_port_t;
+
+struct pgt_device;
+
+struct gvt_fb_notify_msg {
+    unsigned vm_id;
+    unsigned pipe_id; /* id starting from 0 */
+    unsigned plane_id; /* primary, cursor, or sprite */
+};
+
+/* color space conversion and gamma correction are not included */
+struct gvt_primary_plane_format {
+    u8    enabled;    /* plane is enabled */
+    u8    tiled;        /* X-tiled */
+    u8    bpp;        /* bits per pixel */
+    u32    hw_format;    /* format field in the PRI_CTL register */
+    u32    drm_format;    /* format in DRM definition */
+    u32    base;        /* framebuffer base in graphics memory */
+    u32    x_offset;    /* in pixels */
+    u32    y_offset;    /* in lines */
+    u32    width;        /* in pixels */
+    u32    height;        /* in lines */
+    u32    stride;        /* in bytes */
+};
+
+struct gvt_sprite_plane_format {
+    u8    enabled;    /* plane is enabled */
+    u8    tiled;        /* X-tiled */
+    u8    bpp;        /* bits per pixel */
+    u32    hw_format;    /* format field in the SPR_CTL register */
+    u32    drm_format;    /* format in DRM definition */
+    u32    base;        /* sprite base in graphics memory */
+    u32    x_pos;        /* in pixels */
+    u32    y_pos;        /* in lines */
+    u32    x_offset;    /* in pixels */
+    u32    y_offset;    /* in lines */
+    u32    width;        /* in pixels */
+    u32    height;        /* in lines */
+};
+
+struct gvt_cursor_plane_format {
+    u8    enabled;
+    u8    mode;        /* cursor mode select */
+    u8    bpp;        /* bits per pixel */
+    u32    drm_format;    /* format in DRM definition */
+    u32    base;        /* cursor base in graphics memory */
+    u32    x_pos;        /* in pixels */
+    u32    y_pos;        /* in lines */
+    u8    x_sign;        /* X Position Sign */
+    u8    y_sign;        /* Y Position Sign */
+    u32    width;        /* in pixels */
+    u32    height;        /* in lines */
+    u32    x_hot;        /* in pixels */
Quoted 9 lines

drivers/gpu/drm/i915/gvt/fb_decoder.h line 94

Quoted 3 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +3 -0
+    u32    y_hot;        /* in pixels */
+};
+

The above structs have a lot in common, would it make sense to have the common members + plane type and then union for the plane type specific data. I suspect having it all split this way will lead to more utility functions somewhere.

[Zhi] Thanks. Will do that.

Quoted 87 lines
Quoted 61 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +5 -0
+struct gvt_pipe_format {
+    struct gvt_primary_plane_format    primary;
+    struct gvt_sprite_plane_format    sprite;
+    struct gvt_cursor_plane_format    cursor;
+    ddi_port_t ddi_port;  /* the DDI port that the pipe is connected

to */

drivers/gpu/drm/i915/gvt/fb_decoder.h new file +11 -0
+};
+
+struct gvt_fb_format{
+    struct gvt_pipe_format    pipes[I915_MAX_PIPES];
+};
+
+extern int gvt_fb_notifier_call_chain(unsigned long val, void *data);
+extern int gvt_decode_fb_format(struct pgt_device *pdev, int vmid,
+                struct gvt_fb_format *fb);
+
+#endif
drivers/gpu/drm/i915/gvt/gvt.c new file +4 -0
new file mode 100644
index 0000000..041d10f
@@ -0,0 +1,366 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/gvt.c new file +2 -0
+
+#include

drivers/gpu/drm/i915/gvt/gvt.c line 34

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +10 -0
+#include
+
+#include "gvt.h"
+
+struct gvt_host gvt_host;
+
+extern struct gvt_kernel_dm xengt_kdm;
+extern struct gvt_kernel_dm kvmgt_kdm;
+
+static const char *supported_hypervisors[] = {

should be "static const char * const".

Quoted 13 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +13 -0
+    [GVT_HYPERVISOR_TYPE_XEN] = "Xen Hypervisor",
+    [GVT_HYPERVISOR_TYPE_KVM] = "KVM",
+};
+
+static bool gvt_init_host(void)
+{
+    struct gvt_host *host = &gvt_host;
+
+    if (!gvt.enable) {
+        gvt_dbg_core("GVT-g has been disabled by kernel parameter");
+        return false;
+    }
+
Quoted 9 lines

drivers/gpu/drm/i915/gvt/gvt.c line 53

Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +6 -0
+    if (host->initialized) {
+        gvt_err("GVT-g has already been initialized!");
+        return false;
+    }
+
+    if (xen_initial_domain()) {

Shouldn't Xen code be CONFIG_XEN_DOM0 and CONFIG_XEN #ifdef protected?

[Zhi] The following code piece shows xen_initial_domain()/xen_domain() could also be used when CONFIG_XEN_DOM0 is not set.

#ifdef CONFIG_XEN
extern enum xen_domain_type xen_domain_type;
#else
#define xen_domain_type XEN_NATIVE
#endif

#define xen_domain() (xen_domain_type != XEN_NATIVE)

#define xen_pv_domain()         (xen_domain() &&                        \
                                  xen_domain_type == XEN_PV_DOMAIN)
#define xen_hvm_domain()        (xen_domain() &&                        \
                                  xen_domain_type == XEN_HVM_DOMAIN)

#ifdef CONFIG_XEN_DOM0
#include <xen/interface/xen.h>
#include <asm/xen/hypervisor.h>

#define xen_initial_domain()    (xen_domain() && \
                                  xen_start_info &&

xen_start_info->flags & SIF_INITDOMAIN)
#else /* !CONFIG_XEN_DOM0 */
#define xen_initial_domain() (0)
#endif /* CONFIG_XEN_DOM0 */

Quoted 15 lines
Quoted 11 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +2 -0
+        /* Xen Dom0 */
+        host->kdm = try_then_request_module(symbol_get(xengt_kdm),

"xengt");

drivers/gpu/drm/i915/gvt/gvt.c new file +7 -0
+        host->hypervisor_type = GVT_HYPERVISOR_TYPE_XEN;
+    } else if(xen_domain()) {
+        /* Xen DomU */
+        return false;
+    } else {
+        /* not in Xen. Try KVMGT */
+        host->kdm = try_then_request_module(symbol_get(kvmgt_kdm),

"kvm");

drivers/gpu/drm/i915/gvt/gvt.c line 65

Quoted 3 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +3 -0
+        host->hypervisor_type = GVT_HYPERVISOR_TYPE_KVM;
+    }
+
Quoted 2 lines

Why do we need to keep track of hypervisor type, are there plenty of hypervisor specific behaviour differences?

[Zhi] As GVT-g needs some hypervisor services to work, we write a abstraction layer to connect GVT-g to different hypervisor. But still there are some emulation logic couldn't be fitted into that abstraction layer, like OpRegion emulation. In Xen, we emulates it in kernel space, while we emulates it in Qemu under KVM. I also agreed that we should find some approach to improve it just like what you said.

If it is just for printing

Quoted 38 lines

the below debug, I think it's better to move the debug into above code that detects the hypervisor, wrap them with appropriate CONFIG #ifdefs.

Quoted 35 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +35 -0
+    if (!host->kdm)
+        return false;
+
+    if (!hypervisor_detect_host())
+        return false;
+
+    gvt_info("Running with hypervisor %s in host mode",
+            supported_hypervisors[host->hypervisor_type]);
+
+    idr_init(&host->device_idr);
+    mutex_init(&host->device_idr_lock);
+
+    host->initialized = true;
+    return true;
+}
+
+static bool init_device_info(struct pgt_device *pdev)
+{
+    struct gvt_device_info *info = &pdev->device_info;
+
+    if (!IS_BROADWELL(pdev->dev_priv)) {
+        gvt_err("Unsupported GEN device");
+        return false;
+    }
+
+    if (IS_BROADWELL(pdev->dev_priv)) {
+        info->max_gtt_gm_sz = (1UL << 32);
+        /*
+         * The layout of BAR0 in BDW:
+         * |< - MMIO 2MB ->|<- Reserved 6MB ->|<- MAX GTT 8MB->|
+         *
+         * GTT offset in BAR0 starts from 8MB to 16MB, and
+         * Whatever GTT size is configured in BIOS,
+         * the size of BAR0 is always 16MB. The actual configured
+         * GTT size can be found in GMCH_CTRL.
Quoted 9 lines

drivers/gpu/drm/i915/gvt/gvt.c line 106

Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +6 -0
+         */
+        info->gtt_start_offset = (1UL << 23);
+        info->max_gtt_size = (1UL << 23);
+        info->gtt_entry_size = 8;
+        info->gtt_entry_size_shift = 3;
+        info->gmadr_bytes_in_cmd = 8;

Would this information be useful in a header as #defines too?

[Zhi] Probably. Consider that one GVT-g featured kernel could be able to run on different platforms, the informaion data structure could be different. But I agree to define the magic numbers in a header files. :)

Quoted 5 lines
Quoted 5 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +5 -0
+    }
+
+    gvt_info("Device info:");
+    printk("        max_gtt_gm_sz: %llx\n", info->max_gtt_gm_sz);
+    printk("        max_gtt_size: %x\n", info->max_gtt_size);
Quoted 9 lines

drivers/gpu/drm/i915/gvt/gvt.c line 116

Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +2 -0
+    printk("        gtt_size_entry: %x\n", info->gtt_entry_size);
+    printk("        gtt_entry_size_shift: %x\n",

info->gtt_entry_size_shift);

drivers/gpu/drm/i915/gvt/gvt.c new file +3 -0
+    printk("        gtt_start_offset: %x\n", info->gtt_start_offset);
+    printk("        gtt_end_offset: %x\n", info->gtt_end_offset);
+

Just put this to kind of stuff to i915_debugfs.c.

[Zhi] Thanks.

Quoted 95 lines

drivers/gpu/drm/i915/gvt/gvt.c line 127

Quoted 11 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +11 -0
+    return true;
+}
+
+static void init_initial_cfg_space_state(struct pgt_device *pdev)
+{
+    struct pci_dev *pci_dev = pdev->dev_priv->dev->pdev;
+    int i;
+
+    gvt_dbg_core("init initial cfg space, id %d", pdev->id);
+
+    for (i = 0; i < GVT_CFG_SPACE_SZ; i += 4)

+= sizeof(...)

drivers/gpu/drm/i915/gvt/gvt.c line 131

Quoted 4 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +4 -0
+        pci_read_config_dword(pci_dev, i,
+                (u32 *)&pdev->initial_cfg_space[i]);
+
+    for (i = 0; i < 3; i++) {

No magic numbers make a #define for 3 and give it a descriptive name.

Quoted 22 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +22 -0
+        pdev->bar_size[i] = pci_resource_len(pci_dev, i * 2);
+        gvt_info("bar %d size: %llx", i, pdev->bar_size[i]);
+    }
+}
+
+static void clean_initial_mmio_state(struct pgt_device *pdev)
+{
+    if (pdev->gttmmio_va) {
+        iounmap(pdev->gttmmio_va);
+        pdev->gttmmio_va = NULL;
+    }
+
+    if (pdev->gmadr_va) {
+        iounmap(pdev->gmadr_va);
+        pdev->gmadr_va = NULL;
+    }
+}
+
+static bool init_initial_mmio_state(struct pgt_device *pdev)
+{
+    u64 bar0, bar1;
+

drivers/gpu/drm/i915/gvt/gvt.c line 163

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +10 -0
+    gvt_dbg_core("init initial mmio state, id %d", pdev->id);
+
+    bar0 = *(u64 *)&pdev->initial_cfg_space[GVT_REG_CFG_SPACE_BAR0];
+    bar1 = *(u64 *)&pdev->initial_cfg_space[GVT_REG_CFG_SPACE_BAR1];
+
+    pdev->gttmmio_base = bar0 & ~0xf;
+    pdev->mmio_size = 2 * 1024 * 1024;
+    pdev->reg_num = pdev->mmio_size / 4;
+    pdev->gmadr_base = bar1 & ~0xf;
+

Many magic numbers.

drivers/gpu/drm/i915/gvt/gvt.c line 166

Quoted 3 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +3 -0
+    pdev->gttmmio_va = ioremap(pdev->gttmmio_base, pdev->bar_size[0]);
+    if (!pdev->gttmmio_va) {
+        gvt_err("fail to map GTTMMIO BAR.");

These should be if(WARN_ON(...))

Quoted 33 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +12 -0
+        return false;
+    }
+
+    pdev->gmadr_va = ioremap(pdev->gmadr_base, pdev->bar_size[2]);
+    if (!pdev->gmadr_va) {
+        gvt_err("fail to map GMADR BAR.");
+        goto err;
+    }
+
+    gvt_info("bar0: 0x%llx, bar1: 0x%llx", bar0, bar1);
+    gvt_info("mmio size: %x", pdev->mmio_size);
+    gvt_info("gttmmio: 0x%llx, gmadr: 0x%llx", pdev->gttmmio_base,

pdev->gmadr_base);

drivers/gpu/drm/i915/gvt/gvt.c new file +20 -0
+    gvt_info("gttmmio_va: %p", pdev->gttmmio_va);
+    gvt_info("gmadr_va: %p", pdev->gmadr_va);
+
+    return true;
+err:
+    clean_initial_mmio_state(pdev);
+    return false;
+}
+
+static int gvt_service_thread(void *data)
+{
+    struct pgt_device *pdev = (struct pgt_device *)data;
+    int r;
+
+    gvt_dbg_core("service thread start, pgt %d", pdev->id);
+
+    while(!kthread_should_stop()) {
+        r = wait_event_interruptible(pdev->service_thread_wq,
+                kthread_should_stop() || pdev->service_request);
+
Quoted 9 lines
Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +5 -0
+        if (kthread_should_stop())
+            break;
+
+        if (r) {
+            gvt_warn("service thread is waken up by unexpected

signal.");

Should be WARN_ONCE, to avoid future disasters with CI.

[Zhi] Thanks.

Quoted 321 lines
Quoted 272 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +36 -0
+            continue;
+        }
+    }
+    return 0;
+}
+
+static void clean_service_thread(struct pgt_device *pdev)
+{
+    if (pdev->service_thread) {
+        kthread_stop(pdev->service_thread);
+        pdev->service_thread = NULL;
+    }
+}
+
+static bool init_service_thread(struct pgt_device *pdev)
+{
+    init_waitqueue_head(&pdev->service_thread_wq);
+
+    pdev->service_thread = kthread_run(gvt_service_thread,
+            pdev, "gvt_service_thread%d", pdev->id);
+
+    if (!pdev->service_thread) {
+        gvt_err("fail to start service thread.");
+        return false;
+    }
+
+    return true;
+}
+
+static void clean_pgt_device(struct pgt_device *pdev)
+{
+    clean_service_thread(pdev);
+    clean_initial_mmio_state(pdev);
+}
+
+static bool init_pgt_device(struct pgt_device *pdev, struct

drm_i915_private *dev_priv)

drivers/gpu/drm/i915/gvt/gvt.c new file +35 -0
+{
+    if (!init_device_info(pdev))
+        return false;
+
+    init_initial_cfg_space_state(pdev);
+
+    if (!init_initial_mmio_state(pdev))
+        goto err;
+
+    if (!init_service_thread(pdev))
+        goto err;
+
+    return true;
+err:
+    clean_pgt_device(pdev);
+    return false;
+}
+
+static bool post_init_pgt_device(struct pgt_device *pdev)
+{
+    return true;
+}
+
+static void free_pgt_device(struct pgt_device *pdev)
+{
+    struct gvt_host *host = &gvt_host;
+
+    mutex_lock(&host->device_idr_lock);
+    idr_remove(&host->device_idr, pdev->id);
+    mutex_unlock(&host->device_idr_lock);
+
+    vfree(pdev);
+}
+
+static struct pgt_device *alloc_pgt_device(struct drm_i915_private

*dev_priv)

drivers/gpu/drm/i915/gvt/gvt.c new file +92 -0
+{
+    struct gvt_host *host = &gvt_host;
+    struct pgt_device *pdev = NULL;
+
+    pdev = vzalloc(sizeof(*pdev));
+    if (!pdev) {
+        gvt_err("fail to allocate memory for pgt device.");
+        return NULL;
+    }
+
+    mutex_lock(&host->device_idr_lock);
+    pdev->id = idr_alloc(&host->device_idr, pdev, 0, 0, GFP_KERNEL);
+    mutex_unlock(&host->device_idr_lock);
+
+    if (pdev->id < 0) {
+        gvt_err("fail to allocate pgt device id.");
+        goto err;
+    }
+
+    mutex_init(&pdev->lock);
+    pdev->dev_priv = dev_priv;
+    idr_init(&pdev->instance_idr);
+
+    return pdev;
+err:
+    free_pgt_device(pdev);
+    return NULL;
+}
+
+void gvt_destroy_pgt_device(void *private_data)
+{
+    struct pgt_device *pdev = (struct pgt_device *)private_data;
+
+    clean_pgt_device(pdev);
+    free_pgt_device(pdev);
+}
+
+void *gvt_create_pgt_device(struct drm_i915_private *dev_priv)
+{
+    struct pgt_device *pdev = NULL;
+    struct gvt_host *host = &gvt_host;
+
+    if (!host->initialized && !gvt_init_host()) {
+        gvt_err("gvt_init_host fail");
+        return NULL;
+    }
+
+    gvt_dbg_core("create new pgt device, i915 dev_priv: %p", dev_priv);
+
+    pdev = alloc_pgt_device(dev_priv);
+    if (!pdev) {
+        gvt_err("fail to allocate memory for pgt device.");
+        goto err;
+    }
+
+    gvt_dbg_core("init pgt device, id %d", pdev->id);
+
+    if (!init_pgt_device(pdev, dev_priv)) {
+        gvt_err("fail to init physical device state.");
+        goto err;
+    }
+
+    gvt_dbg_core("pgt device creation done, id %d", pdev->id);
+
+    return pdev;
+err:
+    if (pdev) {
+        gvt_destroy_pgt_device(pdev);
+        pdev = NULL;
+    }
+    return NULL;
+}
+
+bool gvt_post_init_pgt_device(void *private_data)
+{
+    struct pgt_device *pdev = (struct pgt_device *)private_data;
+    struct gvt_host *host = &gvt_host;
+
+    if (!host->initialized) {
+        gvt_err("gvt_host haven't been initialized.");
+        return false;
+    }
+
+    gvt_dbg_core("post init pgt device %d", pdev->id);
+
+    if (!post_init_pgt_device(pdev)) {
+        gvt_err("fail to post init physical device state.");
+        return false;
+    }
+
+    return true;
+}
drivers/gpu/drm/i915/gvt/gvt.h new file +4 -0
new file mode 100644
index 0000000..6c85bba
@@ -0,0 +1,130 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/gvt.h new file +65 -0
+
+#ifndef _GVT_H_
+#define _GVT_H_
+
+#include "i915_drv.h"
+#include "i915_vgpu.h"
+
+#include "debug.h"
+#include "params.h"
+#include "reg.h"
+#include "hypercall.h"
+#include "mpt.h"
+#include "fb_decoder.h"
+
+#define GVT_MAX_VGPU 8
+
+enum {
+    GVT_HYPERVISOR_TYPE_XEN = 0,
+    GVT_HYPERVISOR_TYPE_KVM,
+};
+
+struct gvt_host {
+    bool initialized;
+    int hypervisor_type;
+    struct mutex device_idr_lock;
+    struct idr device_idr;
+    struct gvt_kernel_dm *kdm;
+};
+
+extern struct gvt_host gvt_host;
+
+/* Describe the limitation of HW.*/
+struct gvt_device_info {
+    u64 max_gtt_gm_sz;
+    u32 gtt_start_offset;
+    u32 gtt_end_offset;
+    u32 max_gtt_size;
+    u32 gtt_entry_size;
+    u32 gtt_entry_size_shift;
+    u32 gmadr_bytes_in_cmd;
+};
+
+struct vgt_device {
+    int id;
+    int vm_id;
+    struct pgt_device *pdev;
+    bool warn_untrack;
+};
+
+struct pgt_device {
+    struct mutex lock;
+    int id;
+
+    struct drm_i915_private *dev_priv;
+    struct idr instance_idr;
+
+    struct gvt_device_info device_info;
+
+    u8 initial_cfg_space[GVT_CFG_SPACE_SZ];
+    u64 bar_size[GVT_BAR_NUM];
+
+    u64 gttmmio_base;
+    void *gttmmio_va;
+
+    u64 gmadr_base;

drivers/gpu/drm/i915/gvt/gvt.h line 97

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +10 -0
+    void *gmadr_va;
+
+    u32 mmio_size;
+    u32 reg_num;
+
+    wait_queue_head_t service_thread_wq;
+    struct task_struct *service_thread;
+    unsigned long service_request;
+};
+

Here-->

Quoted 22 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +22 -0
+static inline u32 gvt_mmio_read(struct pgt_device *pdev,
+        u32 reg)
+{
+    struct drm_i915_private *dev_priv = pdev->dev_priv;
+    i915_reg_t tmp = {.reg = reg};
+    return I915_READ(tmp);
+}
+
+static inline void gvt_mmio_write(struct pgt_device *pdev,
+        u32 reg, u32 val)
+{
+    struct drm_i915_private *dev_priv = pdev->dev_priv;
+    i915_reg_t tmp = {.reg = reg};
+    I915_WRITE(tmp, val);
+}
+
+static inline u64 gvt_mmio_read64(struct pgt_device *pdev,
+        u32 reg)
+{
+    struct drm_i915_private *dev_priv = pdev->dev_priv;
+    i915_reg_t tmp = {.reg = reg};
+    return I915_READ64(tmp);

drivers/gpu/drm/i915/gvt/gvt.h line 129

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +10 -0
+}
+
+static inline void gvt_mmio_write64(struct pgt_device *pdev,
+        u32 reg, u64 val)
+{
+    struct drm_i915_private *dev_priv = pdev->dev_priv;
+    i915_reg_t tmp = {.reg = reg};
+    I915_WRITE64(tmp, val);
+}
+

<-- Why? The i915_reg_t type was added to avoid problems, this code is not used anywhere, and it has no documentation, so I can not review it.

Quoted 2 lines

I wrote comments at the top of the post.

[Zhi] Thanks, I will check that.

Quoted 432 lines

Regards, Joonas

Quoted 430 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +1 -0
+#endif
drivers/gpu/drm/i915/gvt/hypercall.h new file +4 -0
new file mode 100644
index 0000000..0a41874
@@ -0,0 +1,30 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/hypercall.h new file +8 -0
+
+#ifndef _GVT_HYPERCALL_H_
+#define _GVT_HYPERCALL_H_
+
+struct gvt_kernel_dm {
+};
+
+#endif /* _GVT_HYPERCALL_H_ */
drivers/gpu/drm/i915/gvt/mpt.h new file +4 -0
new file mode 100644
index 0000000..bbe4465
@@ -0,0 +1,97 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/mpt.h new file +58 -0
+
+#ifndef _GVT_MPT_H_
+#define _GVT_MPT_H_
+
+struct vgt_device;
+
+static inline unsigned long hypervisor_g2m_pfn(struct vgt_device *vgt,
+    unsigned long g_pfn)
+{
+    return 0;
+}
+
+static inline int hypervisor_pause_domain(struct vgt_device *vgt)
+{
+    return 0;
+}
+
+static inline int hypervisor_shutdown_domain(struct vgt_device *vgt)
+{
+    return 0;
+}
+
+static inline int hypervisor_set_trap_area(struct vgt_device *vgt,
+    uint64_t start, uint64_t end, bool map)
+{
+    return 0;
+}
+
+static inline bool hypervisor_detect_host(void)
+{
+    return false;
+}
+
+static inline int hypervisor_virt_to_mfn(void *addr)
+{
+    return 0;
+}
+
+static inline void *hypervisor_mfn_to_virt(int mfn)
+{
+    return NULL;
+}
+
+static inline void hypervisor_inject_msi(struct vgt_device *vgt)
+{
+    return;
+}
+
+static inline int hypervisor_hvm_init(struct vgt_device *vgt)
+{
+    return 0;
+}
+
+static inline void hypervisor_hvm_exit(struct vgt_device *vgt)
+{
+}
+
+static inline void *hypervisor_gpa_to_va(struct vgt_device *vgt,

unsigned long gpa)

drivers/gpu/drm/i915/gvt/mpt.h new file +11 -0
+{
+    return NULL;
+}
+
+static inline bool hypervisor_read_va(struct vgt_device *vgt, void *va,
+        void *val, int len, int atomic)
+{
+    return false;
+}
+
+static inline bool hypervisor_write_va(struct vgt_device *vgt, void

*va,

drivers/gpu/drm/i915/gvt/mpt.h new file +6 -0
+        void *val, int len, int atomic)
+{
+    return false;
+}
+
+#endif /* _GVT_MPT_H_ */
drivers/gpu/drm/i915/gvt/params.c new file +4 -0
new file mode 100644
index 0000000..dfc33c3
@@ -0,0 +1,29 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/params.c new file +7 -0
+
+#include "gvt.h"
+
+struct gvt_kernel_params gvt = {
+    .enable = true,
+    .debug = 0,
+};
drivers/gpu/drm/i915/gvt/params.h new file +4 -0
new file mode 100644
index 0000000..d2955b9
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/params.h new file +12 -0
+
+#ifndef _GVT_PARAMS_H_
+#define _GVT_PARAMS_H_
+
+struct gvt_kernel_params {
+    bool enable;
+    int debug;
+};
+
+extern struct gvt_kernel_params gvt;
+
+#endif
drivers/gpu/drm/i915/gvt/reg.h new file +4 -0
new file mode 100644
index 0000000..d363b74
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/reg.h new file +12 -0
+
+#ifndef _GVT_REG_H
+#define _GVT_REG_H
+
+#define GVT_CFG_SPACE_SZ    256
+#define GVT_BAR_NUM        4
+
+#define GVT_REG_CFG_SPACE_BAR0    0x10
+#define GVT_REG_CFG_SPACE_BAR1    0x18
+#define GVT_REG_CFG_SPACE_BAR2    0x20
+
+#endif
drivers/gpu/drm/i915/i915_dma.c +4 -0
index 4725e8d..eca8e50 100644
@@ -943,6 +943,10 @@ int i915_driver_load(struct drm_device *dev,
 unsigned long flags)
 
      intel_uncore_init(dev);
 
+    dev_priv->vgpu.host_private_data = gvt_create_pgt_device(dev_priv);
+    if(intel_gvt_host_active(dev))
+        DRM_INFO("GVT-g is running in host mode\n");
+
      ret = i915_gem_gtt_init(dev);
      if (ret)
          goto out_freecsr;
drivers/gpu/drm/i915/i915_dma.c +7 -0
@@ -1067,6 +1071,13 @@ int i915_driver_load(struct drm_device *dev,
 unsigned long flags)
          goto out_power_well;
      }
 
+    if (intel_gvt_host_active(dev)) {
+        if
 (!gvt_post_init_pgt_device(dev_priv->vgpu.host_private_data)) {
+            DRM_ERROR("failed to post init pgt device\n");
+            goto out_power_well;
+        }
+    }
+
      /*

* Notify a valid surface after modesetting,
* when running inside a VM.

drivers/gpu/drm/i915/i915_dma.c +8 -0
@@ -1117,6 +1128,10 @@ out_gtt:
      i915_global_gtt_cleanup(dev);
  out_freecsr:
      intel_csr_ucode_fini(dev_priv);
+    if (intel_gvt_host_active(dev)) {
+        gvt_destroy_pgt_device(dev_priv->vgpu.host_private_data);
+        dev_priv->vgpu.host_private_data = NULL;
+    }
      intel_uncore_fini(dev);
      pci_iounmap(dev->pdev, dev_priv->regs);
  put_bridge:
@@ -1165,6 +1180,10 @@ int i915_driver_unload(struct drm_device *dev)
 
      intel_modeset_cleanup(dev);
 
+    if (intel_gvt_host_active(dev)) {
+        gvt_destroy_pgt_device(dev_priv->vgpu.host_private_data);
+        dev_priv->vgpu.host_private_data = NULL;
+    }
      /*
       * free the memory space allocated for the child device
       * config parsed from VBT
drivers/gpu/drm/i915/i915_drv.h +6 -0
index 01cc982..db3c79b 100644
@@ -1673,6 +1673,7 @@ struct i915_workarounds {
 
  struct i915_virtual_gpu {
      bool active;
+    void *host_private_data;
  };
 
  struct i915_execbuffer_params {
@@ -2747,6 +2748,11 @@ static inline bool intel_vgpu_active(struct
 drm_device *dev)
      return to_i915(dev)->vgpu.active;
  }
 
+static inline bool intel_gvt_host_active(struct drm_device *dev)
+{
+    return to_i915(dev)->vgpu.host_private_data ? true : false;
+}
+
  void
  i915_enable_pipestat(struct drm_i915_private *dev_priv, enum pipe

pipe,

               u32 status_mask);
drivers/gpu/drm/i915/i915_vgpu.h +3 -0
index 3c83b47..942490a 100644
@@ -113,5 +113,8 @@ struct vgt_if {
  extern void i915_check_vgpu(struct drm_device *dev);
  extern int intel_vgt_balloon(struct drm_device *dev);
  extern void intel_vgt_deballoon(void);
+extern void *gvt_create_pgt_device(struct drm_i915_private *dev_priv);
+extern bool gvt_post_init_pgt_device(void *private_data);
+extern void gvt_destroy_pgt_device(void *private_data);
 
  #endif /* _I915_VGPU_H_ */
include/xen/interface/xen.h +0 -0
index d133112..78a38f1 100644
@@ -28,6 +28,7 @@
  #define __XEN_PUBLIC_XEN_H__
 
Quoted 5 lines
Quoted 5 lines

include/xen/interface/xen.h line 34

Quoted 5 lines
include/xen/interface/xen.h +1 -0
  #include
+#include
 
  /*
   * XEN "SYSTEM CALLS" (a.k.a. HYPERCALLS).

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

maintainer
#10 lore raw

Re: [RFC 01/29] drm/i915/gvt: Introduce the basic architecture of GVT-g

Quoted 10 lines · Zhi Wang

Hi Joonas:
For the debug function/macros, could we use DRM_DEBUG_DRIVER as
output, but keep our internal debug switch & marcos, like:

#define gvt_dbg_xxx(xxx, xxx) \
	if (gvt_debug & xxx) \
		DRM_DEBUG_DRIVER(xxxxxx)

Is it OK for the maintainers? Or we have to use DRM_DEBUG_DRIVER directly :(.

Rather than introducing new debug macros of your own, I think I'd rather see any combination of the following:

* DRM_DEBUG_*, DRM_ERROR, and friends.

* dev_dbg, dev_info, dev_err, and friends. See the dynamic debug facilities for fine grained control of what is printed.

* pr_debug, pr_info, pr_err, and friends. See pr_fmt for defining custom prefixes for what is printed.

BR,
Jani.

Quoted 1,587 lines

Thanks,
Zhi.

Quoted 1,573 lines · Zhi Wang

Hi Joonas:
Thanks you very much! We're very excited for receiving your advice
and continue to be open to any comments. :)

I'm supposed that we should make the agreements on i915 host change at the very beginning, as I'm concerned during the discussion of i915 host change, you know, maybe some code of GVT-g needs to be refined or re-designed. So we put the i915 host changes to the beginning of the patch set for the convenience of discussion.

I summarize your comments as below, very applicate. :)

- Replace boolean return value with int as much as possible.
- Replace customized debug ASSERT() function & macros with DRM_DEBUG_*
- Document all non-static functions like i915
- Fix all whitespace via scripts/cleanpatch.pl
- Commit function structure refinement.
- Change the register access behavior just like what i915 does.

For other comments, see my comments below. :)

Quoted 113 lines · Joonas Lahtinen

Hi,

TL;DR Overall, we have same problem as with the scheduler series, there is too much placeholder stuff for easy review. Just squash enough code into one commit so it actually does something logical that can be reviewed and then extend it later. Then it can be reviewed and pushed. Just splitting the code down to achieve smaller patches is not the right thing to do.

Comments on the overall code: You need to document all header file functions (in the source files), and it is good to document the static functions within a file too, to make future maintenance easier.

It is not about splitting the code down to small chunks, but splitting it down to small *logical* chunks. It doesn't make sense to commit dozens of empty bodied functions for review, and then later add their code.

If you add functions, only add them at a patch that takes them into use too, unless we're talking about general purpose shared code. And also remember to add the function body and documentation header. If you simply add a "return 0;" or similar function body, do add a comment to why the function does not exist and when it will.

Then, there is a trend of having a boolean return values in the code. When possible, it should rather be int and the cause for failure should be propagated from the last level all the way to up (-ENOMEN etc.). This way debugging becomes easier and if new error conditions appear, there is less of a maintenance burden to add the propagation later.

Finally, make sure to look at the existing driver parts and
https://www.kernel.org/doc/Documentation/CodingStyle
for proper coding style. There are lots of whitespace fixes needed in this series, like array initializations.

I hope to see this first patch rerolled so that you squash some of the later commits into it so that all functions have a body and you add documentation for the functions so I can both see what it should do and what it actually does. Only reroll the first patch, to keep the iterative step smaller. Lets only then continue with the rest of the series once we've reached a consensus on the formatting and style basics.

See more comments below.

Quoted 41 lines · Zhi Wang

This patch introduces the very basic framework of GVT-g device model, includes basic prototypes, definitions, initialization.

---
  arch/x86/include/asm/xen/interface.h  |   3 +
  drivers/gpu/drm/i915/Kconfig          |  16 ++
  drivers/gpu/drm/i915/Makefile         |   2 +
  drivers/gpu/drm/i915/gvt/Makefile     |   5 +
  drivers/gpu/drm/i915/gvt/debug.h      |  72 +++++++
  drivers/gpu/drm/i915/gvt/fb_decoder.c |  34 ++++
  drivers/gpu/drm/i915/gvt/fb_decoder.h | 110 ++++++++++
  drivers/gpu/drm/i915/gvt/gvt.c        | 366
++++++++++++++++++++++++++++++++++
  drivers/gpu/drm/i915/gvt/gvt.h        | 130 ++++++++++++
  drivers/gpu/drm/i915/gvt/hypercall.h  |  30 +++
  drivers/gpu/drm/i915/gvt/mpt.h        |  97 +++++++++
  drivers/gpu/drm/i915/gvt/params.c     |  29 +++
  drivers/gpu/drm/i915/gvt/params.h     |  34 ++++
  drivers/gpu/drm/i915/gvt/reg.h        |  34 ++++
  drivers/gpu/drm/i915/i915_dma.c       |  19 ++
  drivers/gpu/drm/i915/i915_drv.h       |   6 +
  drivers/gpu/drm/i915/i915_vgpu.h      |   3 +
  include/xen/interface/xen.h           |   1 +
  18 files changed, 991 insertions(+)

create mode 100644 drivers/gpu/drm/i915/gvt/Makefile create mode 100644 drivers/gpu/drm/i915/gvt/debug.h create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.c create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.h create mode 100644 drivers/gpu/drm/i915/gvt/gvt.c create mode 100644 drivers/gpu/drm/i915/gvt/gvt.h create mode 100644 drivers/gpu/drm/i915/gvt/hypercall.h create mode 100644 drivers/gpu/drm/i915/gvt/mpt.h create mode 100644 drivers/gpu/drm/i915/gvt/params.c create mode 100644 drivers/gpu/drm/i915/gvt/params.h create mode 100644 drivers/gpu/drm/i915/gvt/reg.h

arch/x86/include/asm/xen/interface.h +0 -0
index 62ca03e..6ff4986 100644

arch/x86/include/asm/xen/interface.h line 78

Quoted 6 lines
arch/x86/include/asm/xen/interface.h +3 -0
@@ -73,6 +73,9 @@
  #endif
 
  #ifndef __ASSEMBLY__
+
+#include
+

I don't follow why this would need to be added if the file is not modified otherwise. Each header should only include what they use.

If this is an existing bug (that xen/interface.h can not be included without including linux/types.h), it should be split to a separate patch and sent to Xen team. Same for include/xen/interface/xen.h.

Quoted 12 lines
arch/x86/include/asm/xen/interface.h +0 -0
  /* Explicitly size integers that represent pfns in the public

interface
* with Xen so that on ARM we can have one ABI that works for 32
and 64
* bit guests. */

drivers/gpu/drm/i915/Kconfig +0 -0
index 051eab3..89ff723 100644
@@ -47,3 +47,19 @@ config DRM_I915_PRELIMINARY_HW_SUPPORT
        option changes the default for that module option.
 
Quoted 10 lines

drivers/gpu/drm/i915/Kconfig config DRM_I915_PRELIMINARY_HW_SUPPORT · line 55

Quoted 7 lines
drivers/gpu/drm/i915/Kconfig +6 -0
        If in doubt, say "N".
+
+config I915_GVT
+        tristate "GVT-g host driver"
+        depends on DRM_I915
+        select IRQ_WORK
+        default y

Defaulting to "n" would make sense initially.

[Zhi] Sure, will do.

Quoted 116 lines
Quoted 116 lines
drivers/gpu/drm/i915/Kconfig +2 -0
+        help
+          Enabling GVT-g mediated graphics passthrough technique for

Intel i915
+ based integrated graphics card. With GVT-g, it's possible
to have one
+ integrated i915 device shared by multiple VMs. Performance
critical
+ opterations such as apperture accesses and ring buffer
operations
+ are pass-throughed to VM, with a minimal set of
conflicting resources
+ (e.g. display settings) mediated by vGT driver. The
benefit of vGT
+ is on both the performance, given that each VM could
directly operate
+ its aperture space and submit commands like running on
native, and
+ the feature completeness, given that a true GEN hardware
is exposed.

drivers/gpu/drm/i915/Makefile +2 -0
index 0851de07..d4df410 100644
@@ -91,6 +91,8 @@ i915-y += dvo_ch7017.o \
        intel_sdvo.o \
        intel_tv.o
 
+obj-$(CONFIG_I915_GVT)  += gvt/
+
  # virtual gpu code
  i915-y += i915_vgpu.o
 
drivers/gpu/drm/i915/gvt/Makefile new file +5 -1
new file mode 100644
index 0000000..6935b78
@@ -0,0 +1,5 @@
+GVT_SOURCE := gvt.o params.o fb_decoder.o
+
+ccflags-y            += -I$(src) -I$(src)/.. -Wall -Werror
-Wno-unused-function
+i915_gvt-y            := $(GVT_SOURCE)
+obj-$(CONFIG_I915_GVT)        += i915_gvt.o
drivers/gpu/drm/i915/gvt/debug.h new file +4 -0
new file mode 100644
index 0000000..18e1467
@@ -0,0 +1,72 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/debug.h new file +5 -0
+
+#ifndef __GVT_DEBUG_H__
+#define __GVT_DEBUG_H__
+
+#define

ASSERT(x) \
+ do
{ \
+ if (!(x))
{ \
+ printk("Assert at %s line
%d\n", \
+ __FILE__,
__LINE__); \
+
} \

drivers/gpu/drm/i915/gvt/debug.h new file +3 -0
+        } while (0);
+
+#define ASSERT_NUM(x,

y) \
+ do
{ \
+ if (!(x))
{ \
+ printk("Assert at %s line %d para
0x%llx\n", \
+ __FILE__, __LINE__,
(u64)y); \

Quoted 9 lines

drivers/gpu/drm/i915/gvt/debug.h

Quoted 4 lines

+
} \

drivers/gpu/drm/i915/gvt/debug.h new file +2 -0
+        } while (0);
+

There already is WARN_ON (and i915_drv.h modifies it a little). Do not introduce custom functions like this, if the existing ones need improvement, improve them.

[Zhi] OK. Will do as what i915 does

Quoted 27 lines
Quoted 27 lines
drivers/gpu/drm/i915/gvt/debug.h new file +8 -0
+#define gvt_info(fmt, args...) \
+    printk(KERN_INFO"[GVT-g] "fmt"\n", ##args)
+
+#define gvt_err(fmt, args...) \
+    printk(KERN_ERR"%s() - %d: "fmt"\n", __func__, __LINE__, ##args)
+
+#define gvt_warn(fmt, args...) \
+    printk(KERN_WARNING"%s() - %d: "fmt"\n", __func__, __LINE__,

##args)

drivers/gpu/drm/i915/gvt/debug.h new file +4 -0
+
+#define gvt_dbg(level, fmt, args...) do { \
+        if (gvt.debug & level) \
+            printk(KERN_DEBUG"%s() - %d: "fmt"\n", __func__,

__LINE__, ##args); \

drivers/gpu/drm/i915/gvt/debug.h new file +13 -0
+    }while(0)
+
+enum {
+    GVT_DBG_CORE = (1 << 0),
+    GVT_DBG_MM = (1 << 1),
+    GVT_DBG_IRQ = (1 << 2),
+};
+
+#define gvt_dbg_core(fmt, args...) \
+    gvt_dbg(GVT_DBG_CORE, fmt, ##args)
+
+#define gvt_dbg_mm(fmt, args...) \
+    gvt_dbg(GVT_DBG_MM, fmt, ##args)
Quoted 9 lines

drivers/gpu/drm/i915/gvt/debug.h line 72

Quoted 5 lines
drivers/gpu/drm/i915/gvt/debug.h new file +5 -0
+
+#define gvt_dbg_irq(fmt, args...) \
+    gvt_dbg(GVT_DBG_IRQ, fmt, ##args)
+
+#endif

This should be integrated better to DRM debugging options, custom debugging code only for i915 was rejected a while ago.

[ZHI] Thanks for sharing the history. :) Will change that.

Quoted 52 lines
Quoted 52 lines
drivers/gpu/drm/i915/gvt/fb_decoder.c new file +4 -0
new file mode 100644
index 0000000..a219819
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/fb_decoder.c new file +4 -0
+
+#include "gvt.h"
+
+int gvt_decode_fb_format(struct pgt_device *pdev, int vmid, struct

gvt_fb_format *fb)

drivers/gpu/drm/i915/gvt/fb_decoder.c new file +5 -0
+{
+    return 0;
+}
+
+int gvt_fb_notifier_call_chain(unsigned long val, void *data)
Quoted 9 lines

drivers/gpu/drm/i915/gvt/fb_decoder.c line 29

Quoted 3 lines
drivers/gpu/drm/i915/gvt/fb_decoder.c new file +3 -0
+{
+    return 0;
+}

Kerneldoc missing for these functions. It is all the same to squash later patches to introduce the code to these functions already, reviewing utility functions with no kerneldoc and no body makes it somewhat difficult to see the big picture.

[Zhi] One question. I saw i915 put some function instruction in *.c. Is it also OK for kernel doc generating the proper doc files?

Quoted 111 lines
Quoted 111 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +4 -0
new file mode 100644
index 0000000..2c29ed4
@@ -0,0 +1,110 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/fb_decoder.h new file +69 -0
+
+#ifndef _GVT_FB_DECODER_H_
+#define _GVT_FB_DECODER_H_
+
+typedef enum {
+    FB_MODE_SET_START = 1,
+    FB_MODE_SET_END,
+    FB_DISPLAY_FLIP,
+}gvt_fb_event_t;
+
+typedef enum {
+    DDI_PORT_NONE    = 0,
+    DDI_PORT_B    = 1,
+    DDI_PORT_C    = 2,
+    DDI_PORT_D    = 3,
+    DDI_PORT_E    = 4
+} ddi_port_t;
+
+struct pgt_device;
+
+struct gvt_fb_notify_msg {
+    unsigned vm_id;
+    unsigned pipe_id; /* id starting from 0 */
+    unsigned plane_id; /* primary, cursor, or sprite */
+};
+
+/* color space conversion and gamma correction are not included */
+struct gvt_primary_plane_format {
+    u8    enabled;    /* plane is enabled */
+    u8    tiled;        /* X-tiled */
+    u8    bpp;        /* bits per pixel */
+    u32    hw_format;    /* format field in the PRI_CTL register */
+    u32    drm_format;    /* format in DRM definition */
+    u32    base;        /* framebuffer base in graphics memory */
+    u32    x_offset;    /* in pixels */
+    u32    y_offset;    /* in lines */
+    u32    width;        /* in pixels */
+    u32    height;        /* in lines */
+    u32    stride;        /* in bytes */
+};
+
+struct gvt_sprite_plane_format {
+    u8    enabled;    /* plane is enabled */
+    u8    tiled;        /* X-tiled */
+    u8    bpp;        /* bits per pixel */
+    u32    hw_format;    /* format field in the SPR_CTL register */
+    u32    drm_format;    /* format in DRM definition */
+    u32    base;        /* sprite base in graphics memory */
+    u32    x_pos;        /* in pixels */
+    u32    y_pos;        /* in lines */
+    u32    x_offset;    /* in pixels */
+    u32    y_offset;    /* in lines */
+    u32    width;        /* in pixels */
+    u32    height;        /* in lines */
+};
+
+struct gvt_cursor_plane_format {
+    u8    enabled;
+    u8    mode;        /* cursor mode select */
+    u8    bpp;        /* bits per pixel */
+    u32    drm_format;    /* format in DRM definition */
+    u32    base;        /* cursor base in graphics memory */
+    u32    x_pos;        /* in pixels */
+    u32    y_pos;        /* in lines */
+    u8    x_sign;        /* X Position Sign */
+    u8    y_sign;        /* Y Position Sign */
+    u32    width;        /* in pixels */
+    u32    height;        /* in lines */
+    u32    x_hot;        /* in pixels */
Quoted 9 lines

drivers/gpu/drm/i915/gvt/fb_decoder.h line 94

Quoted 3 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +3 -0
+    u32    y_hot;        /* in pixels */
+};
+

The above structs have a lot in common, would it make sense to have the common members + plane type and then union for the plane type specific data. I suspect having it all split this way will lead to more utility functions somewhere.

[Zhi] Thanks. Will do that.

Quoted 87 lines
Quoted 61 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +5 -0
+struct gvt_pipe_format {
+    struct gvt_primary_plane_format    primary;
+    struct gvt_sprite_plane_format    sprite;
+    struct gvt_cursor_plane_format    cursor;
+    ddi_port_t ddi_port;  /* the DDI port that the pipe is connected

to */

drivers/gpu/drm/i915/gvt/fb_decoder.h new file +11 -0
+};
+
+struct gvt_fb_format{
+    struct gvt_pipe_format    pipes[I915_MAX_PIPES];
+};
+
+extern int gvt_fb_notifier_call_chain(unsigned long val, void *data);
+extern int gvt_decode_fb_format(struct pgt_device *pdev, int vmid,
+                struct gvt_fb_format *fb);
+
+#endif
drivers/gpu/drm/i915/gvt/gvt.c new file +4 -0
new file mode 100644
index 0000000..041d10f
@@ -0,0 +1,366 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/gvt.c new file +2 -0
+
+#include

drivers/gpu/drm/i915/gvt/gvt.c line 34

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +10 -0
+#include
+
+#include "gvt.h"
+
+struct gvt_host gvt_host;
+
+extern struct gvt_kernel_dm xengt_kdm;
+extern struct gvt_kernel_dm kvmgt_kdm;
+
+static const char *supported_hypervisors[] = {

should be "static const char * const".

Quoted 13 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +13 -0
+    [GVT_HYPERVISOR_TYPE_XEN] = "Xen Hypervisor",
+    [GVT_HYPERVISOR_TYPE_KVM] = "KVM",
+};
+
+static bool gvt_init_host(void)
+{
+    struct gvt_host *host = &gvt_host;
+
+    if (!gvt.enable) {
+        gvt_dbg_core("GVT-g has been disabled by kernel parameter");
+        return false;
+    }
+
Quoted 9 lines

drivers/gpu/drm/i915/gvt/gvt.c line 53

Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +6 -0
+    if (host->initialized) {
+        gvt_err("GVT-g has already been initialized!");
+        return false;
+    }
+
+    if (xen_initial_domain()) {

Shouldn't Xen code be CONFIG_XEN_DOM0 and CONFIG_XEN #ifdef protected?

[Zhi] The following code piece shows xen_initial_domain()/xen_domain() could also be used when CONFIG_XEN_DOM0 is not set.

#ifdef CONFIG_XEN
extern enum xen_domain_type xen_domain_type;
#else
#define xen_domain_type XEN_NATIVE
#endif

#define xen_domain() (xen_domain_type != XEN_NATIVE)

#define xen_pv_domain()         (xen_domain() &&                        \
                                  xen_domain_type == XEN_PV_DOMAIN)
#define xen_hvm_domain()        (xen_domain() &&                        \
                                  xen_domain_type == XEN_HVM_DOMAIN)

#ifdef CONFIG_XEN_DOM0
#include <xen/interface/xen.h>
#include <asm/xen/hypervisor.h>

#define xen_initial_domain()    (xen_domain() && \
                                  xen_start_info &&

xen_start_info->flags & SIF_INITDOMAIN)
#else /* !CONFIG_XEN_DOM0 */
#define xen_initial_domain() (0)
#endif /* CONFIG_XEN_DOM0 */

Quoted 15 lines
Quoted 11 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +2 -0
+        /* Xen Dom0 */
+        host->kdm = try_then_request_module(symbol_get(xengt_kdm),

"xengt");

drivers/gpu/drm/i915/gvt/gvt.c new file +7 -0
+        host->hypervisor_type = GVT_HYPERVISOR_TYPE_XEN;
+    } else if(xen_domain()) {
+        /* Xen DomU */
+        return false;
+    } else {
+        /* not in Xen. Try KVMGT */
+        host->kdm = try_then_request_module(symbol_get(kvmgt_kdm),

"kvm");

drivers/gpu/drm/i915/gvt/gvt.c line 65

Quoted 3 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +3 -0
+        host->hypervisor_type = GVT_HYPERVISOR_TYPE_KVM;
+    }
+
Quoted 2 lines

Why do we need to keep track of hypervisor type, are there plenty of hypervisor specific behaviour differences?

[Zhi] As GVT-g needs some hypervisor services to work, we write a abstraction layer to connect GVT-g to different hypervisor. But still there are some emulation logic couldn't be fitted into that abstraction layer, like OpRegion emulation. In Xen, we emulates it in kernel space, while we emulates it in Qemu under KVM. I also agreed that we should find some approach to improve it just like what you said.

If it is just for printing

Quoted 38 lines

the below debug, I think it's better to move the debug into above code that detects the hypervisor, wrap them with appropriate CONFIG #ifdefs.

Quoted 35 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +35 -0
+    if (!host->kdm)
+        return false;
+
+    if (!hypervisor_detect_host())
+        return false;
+
+    gvt_info("Running with hypervisor %s in host mode",
+            supported_hypervisors[host->hypervisor_type]);
+
+    idr_init(&host->device_idr);
+    mutex_init(&host->device_idr_lock);
+
+    host->initialized = true;
+    return true;
+}
+
+static bool init_device_info(struct pgt_device *pdev)
+{
+    struct gvt_device_info *info = &pdev->device_info;
+
+    if (!IS_BROADWELL(pdev->dev_priv)) {
+        gvt_err("Unsupported GEN device");
+        return false;
+    }
+
+    if (IS_BROADWELL(pdev->dev_priv)) {
+        info->max_gtt_gm_sz = (1UL << 32);
+        /*
+         * The layout of BAR0 in BDW:
+         * |< - MMIO 2MB ->|<- Reserved 6MB ->|<- MAX GTT 8MB->|
+         *
+         * GTT offset in BAR0 starts from 8MB to 16MB, and
+         * Whatever GTT size is configured in BIOS,
+         * the size of BAR0 is always 16MB. The actual configured
+         * GTT size can be found in GMCH_CTRL.
Quoted 9 lines

drivers/gpu/drm/i915/gvt/gvt.c line 106

Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +6 -0
+         */
+        info->gtt_start_offset = (1UL << 23);
+        info->max_gtt_size = (1UL << 23);
+        info->gtt_entry_size = 8;
+        info->gtt_entry_size_shift = 3;
+        info->gmadr_bytes_in_cmd = 8;

Would this information be useful in a header as #defines too?

[Zhi] Probably. Consider that one GVT-g featured kernel could be able to run on different platforms, the informaion data structure could be different. But I agree to define the magic numbers in a header files. :)

Quoted 5 lines
Quoted 5 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +5 -0
+    }
+
+    gvt_info("Device info:");
+    printk("        max_gtt_gm_sz: %llx\n", info->max_gtt_gm_sz);
+    printk("        max_gtt_size: %x\n", info->max_gtt_size);
Quoted 9 lines

drivers/gpu/drm/i915/gvt/gvt.c line 116

Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +2 -0
+    printk("        gtt_size_entry: %x\n", info->gtt_entry_size);
+    printk("        gtt_entry_size_shift: %x\n",

info->gtt_entry_size_shift);

drivers/gpu/drm/i915/gvt/gvt.c new file +3 -0
+    printk("        gtt_start_offset: %x\n", info->gtt_start_offset);
+    printk("        gtt_end_offset: %x\n", info->gtt_end_offset);
+

Just put this to kind of stuff to i915_debugfs.c.

[Zhi] Thanks.

Quoted 95 lines

drivers/gpu/drm/i915/gvt/gvt.c line 127

Quoted 11 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +11 -0
+    return true;
+}
+
+static void init_initial_cfg_space_state(struct pgt_device *pdev)
+{
+    struct pci_dev *pci_dev = pdev->dev_priv->dev->pdev;
+    int i;
+
+    gvt_dbg_core("init initial cfg space, id %d", pdev->id);
+
+    for (i = 0; i < GVT_CFG_SPACE_SZ; i += 4)

+= sizeof(...)

drivers/gpu/drm/i915/gvt/gvt.c line 131

Quoted 4 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +4 -0
+        pci_read_config_dword(pci_dev, i,
+                (u32 *)&pdev->initial_cfg_space[i]);
+
+    for (i = 0; i < 3; i++) {

No magic numbers make a #define for 3 and give it a descriptive name.

Quoted 22 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +22 -0
+        pdev->bar_size[i] = pci_resource_len(pci_dev, i * 2);
+        gvt_info("bar %d size: %llx", i, pdev->bar_size[i]);
+    }
+}
+
+static void clean_initial_mmio_state(struct pgt_device *pdev)
+{
+    if (pdev->gttmmio_va) {
+        iounmap(pdev->gttmmio_va);
+        pdev->gttmmio_va = NULL;
+    }
+
+    if (pdev->gmadr_va) {
+        iounmap(pdev->gmadr_va);
+        pdev->gmadr_va = NULL;
+    }
+}
+
+static bool init_initial_mmio_state(struct pgt_device *pdev)
+{
+    u64 bar0, bar1;
+

drivers/gpu/drm/i915/gvt/gvt.c line 163

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +10 -0
+    gvt_dbg_core("init initial mmio state, id %d", pdev->id);
+
+    bar0 = *(u64 *)&pdev->initial_cfg_space[GVT_REG_CFG_SPACE_BAR0];
+    bar1 = *(u64 *)&pdev->initial_cfg_space[GVT_REG_CFG_SPACE_BAR1];
+
+    pdev->gttmmio_base = bar0 & ~0xf;
+    pdev->mmio_size = 2 * 1024 * 1024;
+    pdev->reg_num = pdev->mmio_size / 4;
+    pdev->gmadr_base = bar1 & ~0xf;
+

Many magic numbers.

drivers/gpu/drm/i915/gvt/gvt.c line 166

Quoted 3 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +3 -0
+    pdev->gttmmio_va = ioremap(pdev->gttmmio_base, pdev->bar_size[0]);
+    if (!pdev->gttmmio_va) {
+        gvt_err("fail to map GTTMMIO BAR.");

These should be if(WARN_ON(...))

Quoted 33 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +12 -0
+        return false;
+    }
+
+    pdev->gmadr_va = ioremap(pdev->gmadr_base, pdev->bar_size[2]);
+    if (!pdev->gmadr_va) {
+        gvt_err("fail to map GMADR BAR.");
+        goto err;
+    }
+
+    gvt_info("bar0: 0x%llx, bar1: 0x%llx", bar0, bar1);
+    gvt_info("mmio size: %x", pdev->mmio_size);
+    gvt_info("gttmmio: 0x%llx, gmadr: 0x%llx", pdev->gttmmio_base,

pdev->gmadr_base);

drivers/gpu/drm/i915/gvt/gvt.c new file +20 -0
+    gvt_info("gttmmio_va: %p", pdev->gttmmio_va);
+    gvt_info("gmadr_va: %p", pdev->gmadr_va);
+
+    return true;
+err:
+    clean_initial_mmio_state(pdev);
+    return false;
+}
+
+static int gvt_service_thread(void *data)
+{
+    struct pgt_device *pdev = (struct pgt_device *)data;
+    int r;
+
+    gvt_dbg_core("service thread start, pgt %d", pdev->id);
+
+    while(!kthread_should_stop()) {
+        r = wait_event_interruptible(pdev->service_thread_wq,
+                kthread_should_stop() || pdev->service_request);
+
Quoted 9 lines
Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +5 -0
+        if (kthread_should_stop())
+            break;
+
+        if (r) {
+            gvt_warn("service thread is waken up by unexpected

signal.");

Should be WARN_ONCE, to avoid future disasters with CI.

[Zhi] Thanks.

Quoted 321 lines
Quoted 272 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +36 -0
+            continue;
+        }
+    }
+    return 0;
+}
+
+static void clean_service_thread(struct pgt_device *pdev)
+{
+    if (pdev->service_thread) {
+        kthread_stop(pdev->service_thread);
+        pdev->service_thread = NULL;
+    }
+}
+
+static bool init_service_thread(struct pgt_device *pdev)
+{
+    init_waitqueue_head(&pdev->service_thread_wq);
+
+    pdev->service_thread = kthread_run(gvt_service_thread,
+            pdev, "gvt_service_thread%d", pdev->id);
+
+    if (!pdev->service_thread) {
+        gvt_err("fail to start service thread.");
+        return false;
+    }
+
+    return true;
+}
+
+static void clean_pgt_device(struct pgt_device *pdev)
+{
+    clean_service_thread(pdev);
+    clean_initial_mmio_state(pdev);
+}
+
+static bool init_pgt_device(struct pgt_device *pdev, struct

drm_i915_private *dev_priv)

drivers/gpu/drm/i915/gvt/gvt.c new file +35 -0
+{
+    if (!init_device_info(pdev))
+        return false;
+
+    init_initial_cfg_space_state(pdev);
+
+    if (!init_initial_mmio_state(pdev))
+        goto err;
+
+    if (!init_service_thread(pdev))
+        goto err;
+
+    return true;
+err:
+    clean_pgt_device(pdev);
+    return false;
+}
+
+static bool post_init_pgt_device(struct pgt_device *pdev)
+{
+    return true;
+}
+
+static void free_pgt_device(struct pgt_device *pdev)
+{
+    struct gvt_host *host = &gvt_host;
+
+    mutex_lock(&host->device_idr_lock);
+    idr_remove(&host->device_idr, pdev->id);
+    mutex_unlock(&host->device_idr_lock);
+
+    vfree(pdev);
+}
+
+static struct pgt_device *alloc_pgt_device(struct drm_i915_private

*dev_priv)

drivers/gpu/drm/i915/gvt/gvt.c new file +92 -0
+{
+    struct gvt_host *host = &gvt_host;
+    struct pgt_device *pdev = NULL;
+
+    pdev = vzalloc(sizeof(*pdev));
+    if (!pdev) {
+        gvt_err("fail to allocate memory for pgt device.");
+        return NULL;
+    }
+
+    mutex_lock(&host->device_idr_lock);
+    pdev->id = idr_alloc(&host->device_idr, pdev, 0, 0, GFP_KERNEL);
+    mutex_unlock(&host->device_idr_lock);
+
+    if (pdev->id < 0) {
+        gvt_err("fail to allocate pgt device id.");
+        goto err;
+    }
+
+    mutex_init(&pdev->lock);
+    pdev->dev_priv = dev_priv;
+    idr_init(&pdev->instance_idr);
+
+    return pdev;
+err:
+    free_pgt_device(pdev);
+    return NULL;
+}
+
+void gvt_destroy_pgt_device(void *private_data)
+{
+    struct pgt_device *pdev = (struct pgt_device *)private_data;
+
+    clean_pgt_device(pdev);
+    free_pgt_device(pdev);
+}
+
+void *gvt_create_pgt_device(struct drm_i915_private *dev_priv)
+{
+    struct pgt_device *pdev = NULL;
+    struct gvt_host *host = &gvt_host;
+
+    if (!host->initialized && !gvt_init_host()) {
+        gvt_err("gvt_init_host fail");
+        return NULL;
+    }
+
+    gvt_dbg_core("create new pgt device, i915 dev_priv: %p", dev_priv);
+
+    pdev = alloc_pgt_device(dev_priv);
+    if (!pdev) {
+        gvt_err("fail to allocate memory for pgt device.");
+        goto err;
+    }
+
+    gvt_dbg_core("init pgt device, id %d", pdev->id);
+
+    if (!init_pgt_device(pdev, dev_priv)) {
+        gvt_err("fail to init physical device state.");
+        goto err;
+    }
+
+    gvt_dbg_core("pgt device creation done, id %d", pdev->id);
+
+    return pdev;
+err:
+    if (pdev) {
+        gvt_destroy_pgt_device(pdev);
+        pdev = NULL;
+    }
+    return NULL;
+}
+
+bool gvt_post_init_pgt_device(void *private_data)
+{
+    struct pgt_device *pdev = (struct pgt_device *)private_data;
+    struct gvt_host *host = &gvt_host;
+
+    if (!host->initialized) {
+        gvt_err("gvt_host haven't been initialized.");
+        return false;
+    }
+
+    gvt_dbg_core("post init pgt device %d", pdev->id);
+
+    if (!post_init_pgt_device(pdev)) {
+        gvt_err("fail to post init physical device state.");
+        return false;
+    }
+
+    return true;
+}
drivers/gpu/drm/i915/gvt/gvt.h new file +4 -0
new file mode 100644
index 0000000..6c85bba
@@ -0,0 +1,130 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/gvt.h new file +65 -0
+
+#ifndef _GVT_H_
+#define _GVT_H_
+
+#include "i915_drv.h"
+#include "i915_vgpu.h"
+
+#include "debug.h"
+#include "params.h"
+#include "reg.h"
+#include "hypercall.h"
+#include "mpt.h"
+#include "fb_decoder.h"
+
+#define GVT_MAX_VGPU 8
+
+enum {
+    GVT_HYPERVISOR_TYPE_XEN = 0,
+    GVT_HYPERVISOR_TYPE_KVM,
+};
+
+struct gvt_host {
+    bool initialized;
+    int hypervisor_type;
+    struct mutex device_idr_lock;
+    struct idr device_idr;
+    struct gvt_kernel_dm *kdm;
+};
+
+extern struct gvt_host gvt_host;
+
+/* Describe the limitation of HW.*/
+struct gvt_device_info {
+    u64 max_gtt_gm_sz;
+    u32 gtt_start_offset;
+    u32 gtt_end_offset;
+    u32 max_gtt_size;
+    u32 gtt_entry_size;
+    u32 gtt_entry_size_shift;
+    u32 gmadr_bytes_in_cmd;
+};
+
+struct vgt_device {
+    int id;
+    int vm_id;
+    struct pgt_device *pdev;
+    bool warn_untrack;
+};
+
+struct pgt_device {
+    struct mutex lock;
+    int id;
+
+    struct drm_i915_private *dev_priv;
+    struct idr instance_idr;
+
+    struct gvt_device_info device_info;
+
+    u8 initial_cfg_space[GVT_CFG_SPACE_SZ];
+    u64 bar_size[GVT_BAR_NUM];
+
+    u64 gttmmio_base;
+    void *gttmmio_va;
+
+    u64 gmadr_base;

drivers/gpu/drm/i915/gvt/gvt.h line 97

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +10 -0
+    void *gmadr_va;
+
+    u32 mmio_size;
+    u32 reg_num;
+
+    wait_queue_head_t service_thread_wq;
+    struct task_struct *service_thread;
+    unsigned long service_request;
+};
+

Here-->

Quoted 22 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +22 -0
+static inline u32 gvt_mmio_read(struct pgt_device *pdev,
+        u32 reg)
+{
+    struct drm_i915_private *dev_priv = pdev->dev_priv;
+    i915_reg_t tmp = {.reg = reg};
+    return I915_READ(tmp);
+}
+
+static inline void gvt_mmio_write(struct pgt_device *pdev,
+        u32 reg, u32 val)
+{
+    struct drm_i915_private *dev_priv = pdev->dev_priv;
+    i915_reg_t tmp = {.reg = reg};
+    I915_WRITE(tmp, val);
+}
+
+static inline u64 gvt_mmio_read64(struct pgt_device *pdev,
+        u32 reg)
+{
+    struct drm_i915_private *dev_priv = pdev->dev_priv;
+    i915_reg_t tmp = {.reg = reg};
+    return I915_READ64(tmp);

drivers/gpu/drm/i915/gvt/gvt.h line 129

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +10 -0
+}
+
+static inline void gvt_mmio_write64(struct pgt_device *pdev,
+        u32 reg, u64 val)
+{
+    struct drm_i915_private *dev_priv = pdev->dev_priv;
+    i915_reg_t tmp = {.reg = reg};
+    I915_WRITE64(tmp, val);
+}
+

<-- Why? The i915_reg_t type was added to avoid problems, this code is not used anywhere, and it has no documentation, so I can not review it.

Quoted 2 lines

I wrote comments at the top of the post.

[Zhi] Thanks, I will check that.

Quoted 432 lines

Regards, Joonas

Quoted 430 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +1 -0
+#endif
drivers/gpu/drm/i915/gvt/hypercall.h new file +4 -0
new file mode 100644
index 0000000..0a41874
@@ -0,0 +1,30 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/hypercall.h new file +8 -0
+
+#ifndef _GVT_HYPERCALL_H_
+#define _GVT_HYPERCALL_H_
+
+struct gvt_kernel_dm {
+};
+
+#endif /* _GVT_HYPERCALL_H_ */
drivers/gpu/drm/i915/gvt/mpt.h new file +4 -0
new file mode 100644
index 0000000..bbe4465
@@ -0,0 +1,97 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/mpt.h new file +58 -0
+
+#ifndef _GVT_MPT_H_
+#define _GVT_MPT_H_
+
+struct vgt_device;
+
+static inline unsigned long hypervisor_g2m_pfn(struct vgt_device *vgt,
+    unsigned long g_pfn)
+{
+    return 0;
+}
+
+static inline int hypervisor_pause_domain(struct vgt_device *vgt)
+{
+    return 0;
+}
+
+static inline int hypervisor_shutdown_domain(struct vgt_device *vgt)
+{
+    return 0;
+}
+
+static inline int hypervisor_set_trap_area(struct vgt_device *vgt,
+    uint64_t start, uint64_t end, bool map)
+{
+    return 0;
+}
+
+static inline bool hypervisor_detect_host(void)
+{
+    return false;
+}
+
+static inline int hypervisor_virt_to_mfn(void *addr)
+{
+    return 0;
+}
+
+static inline void *hypervisor_mfn_to_virt(int mfn)
+{
+    return NULL;
+}
+
+static inline void hypervisor_inject_msi(struct vgt_device *vgt)
+{
+    return;
+}
+
+static inline int hypervisor_hvm_init(struct vgt_device *vgt)
+{
+    return 0;
+}
+
+static inline void hypervisor_hvm_exit(struct vgt_device *vgt)
+{
+}
+
+static inline void *hypervisor_gpa_to_va(struct vgt_device *vgt,

unsigned long gpa)

drivers/gpu/drm/i915/gvt/mpt.h new file +11 -0
+{
+    return NULL;
+}
+
+static inline bool hypervisor_read_va(struct vgt_device *vgt, void *va,
+        void *val, int len, int atomic)
+{
+    return false;
+}
+
+static inline bool hypervisor_write_va(struct vgt_device *vgt, void

*va,

drivers/gpu/drm/i915/gvt/mpt.h new file +6 -0
+        void *val, int len, int atomic)
+{
+    return false;
+}
+
+#endif /* _GVT_MPT_H_ */
drivers/gpu/drm/i915/gvt/params.c new file +4 -0
new file mode 100644
index 0000000..dfc33c3
@@ -0,0 +1,29 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/params.c new file +7 -0
+
+#include "gvt.h"
+
+struct gvt_kernel_params gvt = {
+    .enable = true,
+    .debug = 0,
+};
drivers/gpu/drm/i915/gvt/params.h new file +4 -0
new file mode 100644
index 0000000..d2955b9
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/params.h new file +12 -0
+
+#ifndef _GVT_PARAMS_H_
+#define _GVT_PARAMS_H_
+
+struct gvt_kernel_params {
+    bool enable;
+    int debug;
+};
+
+extern struct gvt_kernel_params gvt;
+
+#endif
drivers/gpu/drm/i915/gvt/reg.h new file +4 -0
new file mode 100644
index 0000000..d363b74
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/reg.h new file +12 -0
+
+#ifndef _GVT_REG_H
+#define _GVT_REG_H
+
+#define GVT_CFG_SPACE_SZ    256
+#define GVT_BAR_NUM        4
+
+#define GVT_REG_CFG_SPACE_BAR0    0x10
+#define GVT_REG_CFG_SPACE_BAR1    0x18
+#define GVT_REG_CFG_SPACE_BAR2    0x20
+
+#endif
drivers/gpu/drm/i915/i915_dma.c +4 -0
index 4725e8d..eca8e50 100644
@@ -943,6 +943,10 @@ int i915_driver_load(struct drm_device *dev,
 unsigned long flags)
 
      intel_uncore_init(dev);
 
+    dev_priv->vgpu.host_private_data = gvt_create_pgt_device(dev_priv);
+    if(intel_gvt_host_active(dev))
+        DRM_INFO("GVT-g is running in host mode\n");
+
      ret = i915_gem_gtt_init(dev);
      if (ret)
          goto out_freecsr;
drivers/gpu/drm/i915/i915_dma.c +7 -0
@@ -1067,6 +1071,13 @@ int i915_driver_load(struct drm_device *dev,
 unsigned long flags)
          goto out_power_well;
      }
 
+    if (intel_gvt_host_active(dev)) {
+        if
 (!gvt_post_init_pgt_device(dev_priv->vgpu.host_private_data)) {
+            DRM_ERROR("failed to post init pgt device\n");
+            goto out_power_well;
+        }
+    }
+
      /*

* Notify a valid surface after modesetting,
* when running inside a VM.

drivers/gpu/drm/i915/i915_dma.c +8 -0
@@ -1117,6 +1128,10 @@ out_gtt:
      i915_global_gtt_cleanup(dev);
  out_freecsr:
      intel_csr_ucode_fini(dev_priv);
+    if (intel_gvt_host_active(dev)) {
+        gvt_destroy_pgt_device(dev_priv->vgpu.host_private_data);
+        dev_priv->vgpu.host_private_data = NULL;
+    }
      intel_uncore_fini(dev);
      pci_iounmap(dev->pdev, dev_priv->regs);
  put_bridge:
@@ -1165,6 +1180,10 @@ int i915_driver_unload(struct drm_device *dev)
 
      intel_modeset_cleanup(dev);
 
+    if (intel_gvt_host_active(dev)) {
+        gvt_destroy_pgt_device(dev_priv->vgpu.host_private_data);
+        dev_priv->vgpu.host_private_data = NULL;
+    }
      /*
       * free the memory space allocated for the child device
       * config parsed from VBT
drivers/gpu/drm/i915/i915_drv.h +6 -0
index 01cc982..db3c79b 100644
@@ -1673,6 +1673,7 @@ struct i915_workarounds {
 
  struct i915_virtual_gpu {
      bool active;
+    void *host_private_data;
  };
 
  struct i915_execbuffer_params {
@@ -2747,6 +2748,11 @@ static inline bool intel_vgpu_active(struct
 drm_device *dev)
      return to_i915(dev)->vgpu.active;
  }
 
+static inline bool intel_gvt_host_active(struct drm_device *dev)
+{
+    return to_i915(dev)->vgpu.host_private_data ? true : false;
+}
+
  void
  i915_enable_pipestat(struct drm_i915_private *dev_priv, enum pipe

pipe,

               u32 status_mask);
drivers/gpu/drm/i915/i915_vgpu.h +3 -0
index 3c83b47..942490a 100644
@@ -113,5 +113,8 @@ struct vgt_if {
  extern void i915_check_vgpu(struct drm_device *dev);
  extern int intel_vgt_balloon(struct drm_device *dev);
  extern void intel_vgt_deballoon(void);
+extern void *gvt_create_pgt_device(struct drm_i915_private *dev_priv);
+extern bool gvt_post_init_pgt_device(void *private_data);
+extern void gvt_destroy_pgt_device(void *private_data);
 
  #endif /* _I915_VGPU_H_ */
include/xen/interface/xen.h +0 -0
index d133112..78a38f1 100644
@@ -28,6 +28,7 @@
  #define __XEN_PUBLIC_XEN_H__
 
Quoted 5 lines
Quoted 5 lines

include/xen/interface/xen.h line 34

Quoted 5 lines
include/xen/interface/xen.h +1 -0
  #include
+#include
 
  /*
   * XEN "SYSTEM CALLS" (a.k.a. HYPERCALLS).

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Signature · 5 lines
Jani Nikula, Intel Open Source Technology Center
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer
#11 lore raw

Re: [RFC 01/29] drm/i915/gvt: Introduce the basic architecture of GVT-g

In reply to Zhi Wang

Hi,

Quoted 11 lines · Zhi Wang

Hi Joonas:
     For the debug function/macros, could we use DRM_DEBUG_DRIVER as output, but keep our internal debug switch & marcos, like:

#define gvt_dbg_xxx(xxx, xxx) \
	if (gvt_debug & xxx) \
		DRM_DEBUG_DRIVER(xxxxxx)

Is it OK for the maintainers? Or we have to use DRM_DEBUG_DRIVER directly :(.

I think from upstream kernel perspective the right answer is that code like above really needs to use dynamic debugging instead of self-baked system. When using dynamic debugging framework, debugging could be enabled in different granularity levels (module, file, function, line, or format). More at: https://lwn.net/Articles/434833/

Passing dynamic debug filter 'format "i915: gvt: core: " +p' would then enable GVT core debugging.

On the other hand, Daniel does not want us to break drm.debug or add i915.debug (so I guess i915.gvt_debug is not fine either). Discussion occurred in this patch:

https://patchwork.freedesktop.org/patch/71412/

So I think best you can currently do is:

#define gvt_dbg_xxx(fmt, xxx) \
	DRM_DEBUG_DRIVER("gvt: xxx: " fmt, ...)>

And then we wait for DRM to get converted to dynamic debug.

I discussed this with Daniel, and it should be doable. Only down-fall is that in order to make drm.debug=0x0e syntax still work, we need to make make ddebug_exec_queries symbol exported and add code like this for backwards compatability;

if (drm.debug & DRM_UT_CORE)

	ddebug_exec_queries("format \"drm: core: \" +p", ...);
if (drm.debug & DRM_UT_DRIVER)
	ddebug_exec_queries("format \"drm: drv: \" +p", ...);

What do you guys think? One could export more symbols and add the filters in a more direct manner, but I'm not sure if it's worth the effort.

Regards, Joonas

Quoted 1,818 lines

Thanks,
Zhi.

Quoted 1,814 lines · Zhi Wang

Hi Joonas:
     Thanks you very much! We're very excited for receiving your advice
and continue to be open to any comments. :)

I'm supposed that we should make the agreements on i915 host change at
the very beginning, as I'm concerned during the discussion of i915 host
change, you know, maybe some code of GVT-g needs to be refined or re-designed. So we put the i915 host changes to the beginning of the
patch set for the convenience of discussion.

I summarize your comments as below, very applicate. :)

- Replace boolean return value with int as much as possible.
- Replace customized debug ASSERT() function & macros with DRM_DEBUG_*
- Document all non-static functions like i915
- Fix all whitespace via scripts/cleanpatch.pl
- Commit function structure refinement.
- Change the register access behavior just like what i915 does.

For other comments, see my comments below. :)

Quoted 142 lines · Joonas Lahtinen

Hi,

TL;DR Overall, we have same problem as with the scheduler series, there
is too much placeholder stuff for easy review. Just squash enough code
into one commit so it actually does something logical that can be reviewed and then extend it later. Then it can be reviewed and pushed.
Just splitting the code down to achieve smaller patches is not the
right thing to do.

Comments on the overall code: You need to document all header file
functions (in the source files), and it is good to document the static
functions within a file too, to make future maintenance easier.

It is not about splitting the code down to small chunks, but splitting
it down to small *logical* chunks. It doesn't make sense to commit
dozens of empty bodied functions for review, and then later add their
code.

If you add functions, only add them at a patch that takes them into use
too, unless we're talking about general purpose shared code. And also
remember to add the function body and documentation header. If you
simply add a "return 0;" or similar function body, do add a comment to
why the function does not exist and when it will.

Then, there is a trend of having a boolean return values in the code.
When possible, it should rather be int and the cause for failure should
be propagated from the last level all the way to up (-ENOMEN etc.).
This way debugging becomes easier and if new error conditions appear,
there is less of a maintenance burden to add the propagation later.

Finally, make sure to look at the existing driver parts and
    https://www.kernel.org/doc/Documentation/CodingStyle
for proper coding style. There are lots of whitespace fixes needed in
this series, like array initializations.

I hope to see this first patch rerolled so that you squash some of the
later commits into it so that all functions have a body and you add
documentation for the functions so I can both see what it should do and
what it actually does. Only reroll the first patch, to keep the iterative step smaller. Lets only then continue with the rest of the
series once we've reached a consensus on the formatting and style basics.

See more comments below.

Quoted 42 lines · Zhi Wang

This patch introduces the very basic framework of GVT-g device model,
includes basic prototypes, definitions, initialization.
---
  arch/x86/include/asm/xen/interface.h  |   3 + drivers/gpu/drm/i915/Kconfig          |  16 ++ drivers/gpu/drm/i915/Makefile         |   2 + drivers/gpu/drm/i915/gvt/Makefile     |   5 + drivers/gpu/drm/i915/gvt/debug.h      |  72 +++++++ drivers/gpu/drm/i915/gvt/fb_decoder.c |  34 ++++ drivers/gpu/drm/i915/gvt/fb_decoder.h | 110 ++++++++++ drivers/gpu/drm/i915/gvt/gvt.c        | 366
++++++++++++++++++++++++++++++++++
  drivers/gpu/drm/i915/gvt/gvt.h        | 130 ++++++++++++ drivers/gpu/drm/i915/gvt/hypercall.h  |  30 +++ drivers/gpu/drm/i915/gvt/mpt.h        |  97 +++++++++ drivers/gpu/drm/i915/gvt/params.c     |  29 +++ drivers/gpu/drm/i915/gvt/params.h     |  34 ++++ drivers/gpu/drm/i915/gvt/reg.h        |  34 ++++ drivers/gpu/drm/i915/i915_dma.c       |  19 ++ drivers/gpu/drm/i915/i915_drv.h       |   6 + drivers/gpu/drm/i915/i915_vgpu.h      |   3 + include/xen/interface/xen.h           |   1 +

  18 files changed, 991 insertions(+)

  create mode 100644 drivers/gpu/drm/i915/gvt/Makefile create mode 100644 drivers/gpu/drm/i915/gvt/debug.h create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.c create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.h create mode 100644 drivers/gpu/drm/i915/gvt/gvt.c create mode 100644 drivers/gpu/drm/i915/gvt/gvt.h create mode 100644 drivers/gpu/drm/i915/gvt/hypercall.h create mode 100644 drivers/gpu/drm/i915/gvt/mpt.h create mode 100644 drivers/gpu/drm/i915/gvt/params.c create mode 100644 drivers/gpu/drm/i915/gvt/params.h create mode 100644 drivers/gpu/drm/i915/gvt/reg.h

arch/x86/include/asm/xen/interface.h +0 -0
index 62ca03e..6ff4986 100644

arch/x86/include/asm/xen/interface.h line 78

Quoted 6 lines
arch/x86/include/asm/xen/interface.h +3 -0
@@ -73,6 +73,9 @@
   #endif
 
   #ifndef __ASSEMBLY__
+
+#include
+

I don't follow why this would need to be added if the file is not modified otherwise. Each header should only include what they use.

If this is an existing bug (that xen/interface.h can not be included
without including linux/types.h), it should be split to a separate
patch and sent to Xen team. Same for include/xen/interface/xen.h.

Quoted 14 lines
arch/x86/include/asm/xen/interface.h +0 -0
   /* Explicitly size integers that represent pfns in the public

interface
   * with Xen so that on ARM we can have one ABI that works for 32
and 64
   * bit guests. */

drivers/gpu/drm/i915/Kconfig +0 -0
index 051eab3..89ff723 100644
@@ -47,3 +47,19 @@ config DRM_I915_PRELIMINARY_HW_SUPPORT

        option changes the default for that module option.

Quoted 10 lines

drivers/gpu/drm/i915/Kconfig config DRM_I915_PRELIMINARY_HW_SUPPORT · line 55

Quoted 7 lines

        If in doubt, say "N".

drivers/gpu/drm/i915/Kconfig +6 -0
+
+config I915_GVT
+        tristate "GVT-g host driver"
+        depends on DRM_I915
+        select IRQ_WORK
+        default y

Defaulting to "n" would make sense initially.

[Zhi] Sure, will do.

Quoted 136 lines
Quoted 136 lines

+        help
+          Enabling GVT-g mediated graphics passthrough technique for
Intel i915
+          based integrated graphics card. With GVT-g, it's possible
to have one
+          integrated i915 device shared by multiple VMs. Performance
critical
+          opterations such as apperture accesses and ring buffer
operations
+          are pass-throughed to VM, with a minimal set of conflicting resources
+          (e.g. display settings) mediated by vGT driver. The benefit of vGT
+          is on both the performance, given that each VM could directly operate
+          its aperture space and submit commands like running on
native, and
+          the feature completeness, given that a true GEN hardware
is exposed.

drivers/gpu/drm/i915/Makefile +0 -0
index 0851de07..d4df410 100644
@@ -91,6 +91,8 @@ i915-y += dvo_ch7017.o \
         intel_sdvo.o \

        intel_tv.o

drivers/gpu/drm/i915/Makefile +2 -0
+obj-$(CONFIG_I915_GVT)  += gvt/
+
  # virtual gpu code
  i915-y += i915_vgpu.o
drivers/gpu/drm/i915/gvt/Makefile new file +5 -1
new file mode 100644
index 0000000..6935b78
@@ -0,0 +1,5 @@
+GVT_SOURCE := gvt.o params.o fb_decoder.o
+
+ccflags-y            += -I$(src) -I$(src)/.. -Wall -Werror
-Wno-unused-function
+i915_gvt-y            := $(GVT_SOURCE)
+obj-$(CONFIG_I915_GVT)        += i915_gvt.o
drivers/gpu/drm/i915/gvt/debug.h new file +2 -0
new file mode 100644
index 0000000..18e1467
@@ -0,0 +1,72 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights

reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
"Software"),
+ * to deal in the Software without restriction, including without
limitation
+ * the rights to use, copy, modify, merge, publish, distribute,
sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom
the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including
the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO
EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/debug.h new file +5 -0
+
+#ifndef __GVT_DEBUG_H__
+#define __GVT_DEBUG_H__
+
+#define

ASSERT(x)
 \
+        do
{                                                            \
+                if (!(x))
{                                             \
+                        printk("Assert at %s line %d\n",                \
+                                __FILE__, __LINE__);                    \
+
}                                                       \
+        } while (0);

drivers/gpu/drm/i915/gvt/debug.h new file +2 -0
+
+#define ASSERT_NUM(x,

y)                                                \
+        do
{                                                            \
+                if (!(x))
{                                             \
+                        printk("Assert at %s line %d para 0x%llx\n",    \
+                                __FILE__, __LINE__, (u64)y);            \
+

Quoted 9 lines
Quoted 3 lines

}                                                       \
+        } while (0);
+

There already is WARN_ON (and i915_drv.h modifies it a little). Do not
introduce custom functions like this, if the existing ones need improvement, improve them.

[Zhi] OK. Will do as what i915 does

Quoted 29 lines
Quoted 29 lines
drivers/gpu/drm/i915/gvt/debug.h new file +5 -0
+#define gvt_info(fmt, args...) \
+    printk(KERN_INFO"[GVT-g] "fmt"\n", ##args)
+
+#define gvt_err(fmt, args...) \
+    printk(KERN_ERR"%s() - %d: "fmt"\n", __func__, __LINE__,

##args)

drivers/gpu/drm/i915/gvt/debug.h new file +3 -0
+
+#define gvt_warn(fmt, args...) \
+    printk(KERN_WARNING"%s() - %d: "fmt"\n", __func__,

__LINE__,
##args)

drivers/gpu/drm/i915/gvt/debug.h new file +4 -0
+
+#define gvt_dbg(level, fmt, args...) do { \
+        if (gvt.debug & level) \
+            printk(KERN_DEBUG"%s() - %d: "fmt"\n", __func__,

__LINE__, ##args); \
+    }while(0)

drivers/gpu/drm/i915/gvt/debug.h new file +12 -0
+
+enum {
+    GVT_DBG_CORE = (1 << 0),
+    GVT_DBG_MM = (1 << 1),
+    GVT_DBG_IRQ = (1 << 2),
+};
+
+#define gvt_dbg_core(fmt, args...) \
+    gvt_dbg(GVT_DBG_CORE, fmt, ##args)
+
+#define gvt_dbg_mm(fmt, args...) \
+    gvt_dbg(GVT_DBG_MM, fmt, ##args)
Quoted 9 lines

drivers/gpu/drm/i915/gvt/debug.h line 72

Quoted 5 lines
drivers/gpu/drm/i915/gvt/debug.h new file +5 -0
+
+#define gvt_dbg_irq(fmt, args...) \
+    gvt_dbg(GVT_DBG_IRQ, fmt, ##args)
+
+#endif

This should be integrated better to DRM debugging options, custom debugging code only for i915 was rejected a while ago.

[ZHI] Thanks for sharing the history. :) Will change that.

Quoted 67 lines
Quoted 67 lines
drivers/gpu/drm/i915/gvt/fb_decoder.c new file +2 -0
new file mode 100644
index 0000000..a219819
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights

reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
"Software"),
+ * to deal in the Software without restriction, including without
limitation
+ * the rights to use, copy, modify, merge, publish, distribute,
sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom
the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including
the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO
EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/fb_decoder.c new file +4 -0
+
+#include "gvt.h"
+
+int gvt_decode_fb_format(struct pgt_device *pdev, int vmid,

struct
gvt_fb_format *fb)

drivers/gpu/drm/i915/gvt/fb_decoder.c new file +7 -0
+{
+    return 0;
+}
+
+int gvt_fb_notifier_call_chain(unsigned long val, void *data)
+{
+    return 0;
Quoted 9 lines
Quoted 1 line

+}

Kerneldoc missing for these functions. It is all the same to squash
later patches to introduce the code to these functions already, reviewing utility functions with no kerneldoc and no body makes it
somewhat difficult to see the big picture.

[Zhi] One question. I saw i915 put some function instruction in *.c. Is
it also OK for kernel doc generating the proper doc files?

Quoted 131 lines
Quoted 120 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +2 -0
new file mode 100644
index 0000000..2c29ed4
@@ -0,0 +1,110 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights

reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
"Software"),
+ * to deal in the Software without restriction, including without
limitation
+ * the rights to use, copy, modify, merge, publish, distribute,
sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom
the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including
the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO
EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/fb_decoder.h new file +27 -0
+
+#ifndef _GVT_FB_DECODER_H_
+#define _GVT_FB_DECODER_H_
+
+typedef enum {
+    FB_MODE_SET_START = 1,
+    FB_MODE_SET_END,
+    FB_DISPLAY_FLIP,
+}gvt_fb_event_t;
+
+typedef enum {
+    DDI_PORT_NONE    = 0,
+    DDI_PORT_B    = 1,
+    DDI_PORT_C    = 2,
+    DDI_PORT_D    = 3,
+    DDI_PORT_E    = 4
+} ddi_port_t;
+
+struct pgt_device;
+
+struct gvt_fb_notify_msg {
+    unsigned vm_id;
+    unsigned pipe_id; /* id starting from 0 */
+    unsigned plane_id; /* primary, cursor, or sprite */
+};
+
+/* color space conversion and gamma correction are not

included */
+struct gvt_primary_plane_format {
+    u8    enabled;    /* plane is enabled */
+    u8    tiled;        /* X-tiled */
+    u8    bpp;        /* bits per pixel */
+    u32    hw_format;    /* format field in the PRI_CTL register */
+    u32    drm_format;    /* format in DRM definition */ +    u32    base;        /* framebuffer base in graphics memory */
+    u32    x_offset;    /* in pixels */
+    u32    y_offset;    /* in lines */
+    u32    width;        /* in pixels */
+    u32    height;        /* in lines */
+    u32    stride;        /* in bytes */

drivers/gpu/drm/i915/gvt/fb_decoder.h new file +7 -0
+};
+
+struct gvt_sprite_plane_format {
+    u8    enabled;    /* plane is enabled */
+    u8    tiled;        /* X-tiled */
+    u8    bpp;        /* bits per pixel */
+    u32    hw_format;    /* format field in the SPR_CTL

register */
+    u32    drm_format;    /* format in DRM definition */ +    u32    base;        /* sprite base in graphics memory */ +    u32    x_pos;        /* in pixels */
+    u32    y_pos;        /* in lines */
+    u32    x_offset;    /* in pixels */
+    u32    y_offset;    /* in lines */
+    u32    width;        /* in pixels */
+    u32    height;        /* in lines */

drivers/gpu/drm/i915/gvt/fb_decoder.h new file +8 -0
+};
+
+struct gvt_cursor_plane_format {
+    u8    enabled;
+    u8    mode;        /* cursor mode select */
+    u8    bpp;        /* bits per pixel */
+    u32    drm_format;    /* format in DRM definition */
+    u32    base;        /* cursor base in graphics memory */

drivers/gpu/drm/i915/gvt/fb_decoder.h line 94

Quoted 10 lines
drivers/gpu/drm/i915/gvt/fb_decoder.h new file +10 -0
+    u32    x_pos;        /* in pixels */
+    u32    y_pos;        /* in lines */
+    u8    x_sign;        /* X Position Sign */
+    u8    y_sign;        /* Y Position Sign */
+    u32    width;        /* in pixels */
+    u32    height;        /* in lines */
+    u32    x_hot;        /* in pixels */
+    u32    y_hot;        /* in pixels */
+};
+
Quoted 8 lines

The above structs have a lot in common, would it make sense to have the
common members + plane type and then union for the plane type specific
data. I suspect having it all split this way will lead to more utility
functions somewhere.

[Zhi] Thanks. Will do that.

Quoted 104 lines
Quoted 76 lines

+struct gvt_pipe_format {
+    struct gvt_primary_plane_format    primary;
+    struct gvt_sprite_plane_format    sprite;
+    struct gvt_cursor_plane_format    cursor;
+    ddi_port_t ddi_port;  /* the DDI port that the pipe is connected
to */

drivers/gpu/drm/i915/gvt/fb_decoder.h new file +7 -0
+};
+
+struct gvt_fb_format{
+    struct gvt_pipe_format    pipes[I915_MAX_PIPES];
+};
+
+extern int gvt_fb_notifier_call_chain(unsigned long val, void

*data);
+extern int gvt_decode_fb_format(struct pgt_device *pdev, int vmid,
+                struct gvt_fb_format *fb);

drivers/gpu/drm/i915/gvt/fb_decoder.h new file +2 -0
+
+#endif
drivers/gpu/drm/i915/gvt/gvt.c new file +2 -0
new file mode 100644
index 0000000..041d10f
@@ -0,0 +1,366 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights

reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
"Software"),
+ * to deal in the Software without restriction, including without
limitation
+ * the rights to use, copy, modify, merge, publish, distribute,
sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom
the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including
the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO
EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/gvt.c new file +2 -0
+
+#include

drivers/gpu/drm/i915/gvt/gvt.c line 34

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +10 -0
+#include
+
+#include "gvt.h"
+
+struct gvt_host gvt_host;
+
+extern struct gvt_kernel_dm xengt_kdm;
+extern struct gvt_kernel_dm kvmgt_kdm;
+
+static const char *supported_hypervisors[] = {

should be "static const char * const".

Quoted 15 lines

+    [GVT_HYPERVISOR_TYPE_XEN] = "Xen Hypervisor", +    [GVT_HYPERVISOR_TYPE_KVM] = "KVM",

drivers/gpu/drm/i915/gvt/gvt.c new file +8 -0
+};
+
+static bool gvt_init_host(void)
+{
+    struct gvt_host *host = &gvt_host;
+
+    if (!gvt.enable) {
+        gvt_dbg_core("GVT-g has been disabled by kernel

parameter");
+        return false;
+    }
+
+    if (host->initialized) {

Quoted 9 lines
Quoted 5 lines

+        gvt_err("GVT-g has already been initialized!"); +        return false;
+    }
+
+    if (xen_initial_domain()) {

Shouldn't Xen code be CONFIG_XEN_DOM0 and CONFIG_XEN #ifdef protected?

[Zhi] The following code piece shows xen_initial_domain()/xen_domain()
could also be used when CONFIG_XEN_DOM0 is not set.

#ifdef CONFIG_XEN
extern enum xen_domain_type xen_domain_type;
#else
#define xen_domain_type         XEN_NATIVE
#endif

#define xen_domain()            (xen_domain_type != XEN_NATIVE) #define xen_pv_domain()         (xen_domain()
&&                        \
                                  xen_domain_type == XEN_PV_DOMAIN) #define xen_hvm_domain()        (xen_domain()
&&                        \
                                  xen_domain_type == XEN_HVM_DOMAIN)

#ifdef CONFIG_XEN_DOM0
#include <xen/interface/xen.h>
#include <asm/xen/hypervisor.h>

#define xen_initial_domain()    (xen_domain() && \
                                  xen_start_info && xen_start_info->flags & SIF_INITDOMAIN) #else  /* !CONFIG_XEN_DOM0 */
#define xen_initial_domain()    (0)
#endif  /* CONFIG_XEN_DOM0 */

Quoted 17 lines
Quoted 8 lines

+        /* Xen Dom0 */
+        host->kdm = try_then_request_module(symbol_get(xengt_kdm), "xengt");
+        host->hypervisor_type = GVT_HYPERVISOR_TYPE_XEN; +    } else if(xen_domain()) {
+        /* Xen DomU */
+        return false;

Quoted 8 lines

+    } else {
+        /* not in Xen. Try KVMGT */
+        host->kdm = try_then_request_module(symbol_get(kvmgt_kdm), "kvm");
+        host->hypervisor_type = GVT_HYPERVISOR_TYPE_KVM; +    }
+

Quoted 3 lines

Why do we need to keep track of hypervisor type, are there plenty of
hypervisor specific behaviour differences?

[Zhi] As GVT-g needs some hypervisor services to work, we write a abstraction layer to connect GVT-g to different hypervisor. But still
there are some emulation logic couldn't be fitted into that abstraction
layer, like OpRegion emulation. In Xen, we emulates it in kernel space,
while we emulates it in Qemu under KVM. I also agreed that we should
find some approach to improve it just like what you said.

If it is just for printing

Quoted 49 lines

the below debug, I think it's better to move the debug into above code
that detects the hypervisor, wrap them with appropriate CONFIG #ifdefs.

Quoted 31 lines

+    if (!host->kdm)
+        return false;

drivers/gpu/drm/i915/gvt/gvt.c new file +28 -0
+
+    if (!hypervisor_detect_host())
+        return false;
+
+    gvt_info("Running with hypervisor %s in host mode",
+            supported_hypervisors[host->hypervisor_type]);
+
+    idr_init(&host->device_idr);
+    mutex_init(&host->device_idr_lock);
+
+    host->initialized = true;
+    return true;
+}
+
+static bool init_device_info(struct pgt_device *pdev)
+{
+    struct gvt_device_info *info = &pdev->device_info;
+
+    if (!IS_BROADWELL(pdev->dev_priv)) {
+        gvt_err("Unsupported GEN device");
+        return false;
+    }
+
+    if (IS_BROADWELL(pdev->dev_priv)) {
+        info->max_gtt_gm_sz = (1UL << 32);
+        /*
+         * The layout of BAR0 in BDW:
+         * |< - MMIO 2MB ->|<- Reserved 6MB ->|<- MAX GTT 8MB-
Quoted 1 line

|

Quoted 12 lines

+         *
+         * GTT offset in BAR0 starts from 8MB to 16MB, and +         * Whatever GTT size is configured in BIOS, +         * the size of BAR0 is always 16MB. The actual configured
+         * GTT size can be found in GMCH_CTRL.
+         */
+        info->gtt_start_offset = (1UL << 23);
+        info->max_gtt_size = (1UL << 23);
+        info->gtt_entry_size = 8;
+        info->gtt_entry_size_shift = 3;
+        info->gmadr_bytes_in_cmd = 8;

Quoted 2 lines

Would this information be useful in a header as #defines too?

[Zhi] Probably. Consider that one GVT-g featured kernel could be able to
run on different platforms, the informaion data structure could be different. But I agree to define the magic numbers in a header files. :)

Quoted 11 lines
Quoted 11 lines

+    }
+
+    gvt_info("Device info:");
+    printk("        max_gtt_gm_sz: %llx\n", info-

Quoted 1 line

max_gtt_gm_sz);

+    printk("        max_gtt_size: %x\n", info->max_gtt_size); +    printk("        gtt_size_entry: %x\n", info-

Quoted 1 line

gtt_entry_size);

+    printk("        gtt_entry_size_shift: %x\n", info->gtt_entry_size_shift);
+    printk("        gtt_start_offset: %x\n", info-

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

gtt_start_offset);

+    printk("        gtt_end_offset: %x\n", info-

Quoted 1 line

gtt_end_offset);

+

Just put this to kind of stuff to i915_debugfs.c.

[Zhi] Thanks.

Quoted 102 lines

drivers/gpu/drm/i915/gvt/gvt.c line 127

Quoted 12 lines

+    return true;

drivers/gpu/drm/i915/gvt/gvt.c new file +3 -0
+}
+
+static void init_initial_cfg_space_state(struct pgt_device

*pdev)

drivers/gpu/drm/i915/gvt/gvt.c new file +7 -0
+{
+    struct pci_dev *pci_dev = pdev->dev_priv->dev->pdev;
+    int i;
+
+    gvt_dbg_core("init initial cfg space, id %d", pdev->id);
+
+    for (i = 0; i < GVT_CFG_SPACE_SZ; i += 4)

+= sizeof(...)

Quoted 4 lines

+        pci_read_config_dword(pci_dev, i,
+                (u32 *)&pdev->initial_cfg_space[i]); +
+    for (i = 0; i < 3; i++) {

No magic numbers make a #define for 3 and give it a descriptive name.

Quoted 28 lines

+        pdev->bar_size[i] = pci_resource_len(pci_dev, i * 2); +        gvt_info("bar %d size: %llx", i, pdev->bar_size[i]); +    }

drivers/gpu/drm/i915/gvt/gvt.c new file +22 -0
+}
+
+static void clean_initial_mmio_state(struct pgt_device *pdev)
+{
+    if (pdev->gttmmio_va) {
+        iounmap(pdev->gttmmio_va);
+        pdev->gttmmio_va = NULL;
+    }
+
+    if (pdev->gmadr_va) {
+        iounmap(pdev->gmadr_va);
+        pdev->gmadr_va = NULL;
+    }
+}
+
+static bool init_initial_mmio_state(struct pgt_device *pdev)
+{
+    u64 bar0, bar1;
+
+    gvt_dbg_core("init initial mmio state, id %d", pdev->id);
+
+    bar0 = *(u64 *)&pdev-
Quoted 1 line

initial_cfg_space[GVT_REG_CFG_SPACE_BAR0];

+    bar1 = *(u64 *)&pdev-

Quoted 1 line

initial_cfg_space[GVT_REG_CFG_SPACE_BAR1];

drivers/gpu/drm/i915/gvt/gvt.c line 163

Quoted 6 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +6 -0
+
+    pdev->gttmmio_base = bar0 & ~0xf;
+    pdev->mmio_size = 2 * 1024 * 1024;
+    pdev->reg_num = pdev->mmio_size / 4;
+    pdev->gmadr_base = bar1 & ~0xf;
+

Many magic numbers.

Quoted 4 lines

+    pdev->gttmmio_va = ioremap(pdev->gttmmio_base, pdev-

Quoted 1 line

bar_size[0]);

+    if (!pdev->gttmmio_va) {
+        gvt_err("fail to map GTTMMIO BAR.");

These should be if(WARN_ON(...))

Quoted 35 lines

+        return false;
+    }
+
+    pdev->gmadr_va = ioremap(pdev->gmadr_base, pdev-

Quoted 1 line

bar_size[2]);

+    if (!pdev->gmadr_va) {
+        gvt_err("fail to map GMADR BAR.");
+        goto err;
+    }
+
+    gvt_info("bar0: 0x%llx, bar1: 0x%llx", bar0, bar1); +    gvt_info("mmio size: %x", pdev->mmio_size); +    gvt_info("gttmmio: 0x%llx, gmadr: 0x%llx", pdev-

Quoted 1 line

gttmmio_base,

pdev->gmadr_base);
+    gvt_info("gttmmio_va: %p", pdev->gttmmio_va); +    gvt_info("gmadr_va: %p", pdev->gmadr_va);

drivers/gpu/drm/i915/gvt/gvt.c new file +17 -0
+
+    return true;
+err:
+    clean_initial_mmio_state(pdev);
+    return false;
+}
+
+static int gvt_service_thread(void *data)
+{
+    struct pgt_device *pdev = (struct pgt_device *)data;
+    int r;
+
+    gvt_dbg_core("service thread start, pgt %d", pdev->id);
+
+    while(!kthread_should_stop()) {
+        r = wait_event_interruptible(pdev->service_thread_wq,
+                kthread_should_stop() || pdev-
Quoted 1 line

service_request);

Quoted 10 lines
Quoted 7 lines
drivers/gpu/drm/i915/gvt/gvt.c new file +6 -0
+
+        if (kthread_should_stop())
+            break;
+
+        if (r) {
+            gvt_warn("service thread is waken up by unexpected

signal.");

Should be WARN_ONCE, to avoid future disasters with CI.

[Zhi] Thanks.

Quoted 340 lines
Quoted 289 lines

+            continue;
+        }
+    }
+    return 0;

drivers/gpu/drm/i915/gvt/gvt.c new file +32 -0
+}
+
+static void clean_service_thread(struct pgt_device *pdev)
+{
+    if (pdev->service_thread) {
+        kthread_stop(pdev->service_thread);
+        pdev->service_thread = NULL;
+    }
+}
+
+static bool init_service_thread(struct pgt_device *pdev)
+{
+    init_waitqueue_head(&pdev->service_thread_wq);
+
+    pdev->service_thread = kthread_run(gvt_service_thread,
+            pdev, "gvt_service_thread%d", pdev->id);
+
+    if (!pdev->service_thread) {
+        gvt_err("fail to start service thread.");
+        return false;
+    }
+
+    return true;
+}
+
+static void clean_pgt_device(struct pgt_device *pdev)
+{
+    clean_service_thread(pdev);
+    clean_initial_mmio_state(pdev);
+}
+
+static bool init_pgt_device(struct pgt_device *pdev, struct

drm_i915_private *dev_priv)

drivers/gpu/drm/i915/gvt/gvt.c new file +35 -0
+{
+    if (!init_device_info(pdev))
+        return false;
+
+    init_initial_cfg_space_state(pdev);
+
+    if (!init_initial_mmio_state(pdev))
+        goto err;
+
+    if (!init_service_thread(pdev))
+        goto err;
+
+    return true;
+err:
+    clean_pgt_device(pdev);
+    return false;
+}
+
+static bool post_init_pgt_device(struct pgt_device *pdev)
+{
+    return true;
+}
+
+static void free_pgt_device(struct pgt_device *pdev)
+{
+    struct gvt_host *host = &gvt_host;
+
+    mutex_lock(&host->device_idr_lock);
+    idr_remove(&host->device_idr, pdev->id);
+    mutex_unlock(&host->device_idr_lock);
+
+    vfree(pdev);
+}
+
+static struct pgt_device *alloc_pgt_device(struct

drm_i915_private
*dev_priv)

drivers/gpu/drm/i915/gvt/gvt.c new file +12 -0
+{
+    struct gvt_host *host = &gvt_host;
+    struct pgt_device *pdev = NULL;
+
+    pdev = vzalloc(sizeof(*pdev));
+    if (!pdev) {
+        gvt_err("fail to allocate memory for pgt device.");
+        return NULL;
+    }
+
+    mutex_lock(&host->device_idr_lock);
+    pdev->id = idr_alloc(&host->device_idr, pdev, 0, 0,

GFP_KERNEL);
+    mutex_unlock(&host->device_idr_lock);

drivers/gpu/drm/i915/gvt/gvt.c new file +19 -0
+
+    if (pdev->id < 0) {
+        gvt_err("fail to allocate pgt device id.");
+        goto err;
+    }
+
+    mutex_init(&pdev->lock);
+    pdev->dev_priv = dev_priv;
+    idr_init(&pdev->instance_idr);
+
+    return pdev;
+err:
+    free_pgt_device(pdev);
+    return NULL;
+}
+
+void gvt_destroy_pgt_device(void *private_data)
+{
+    struct pgt_device *pdev = (struct pgt_device

*)private_data;

drivers/gpu/drm/i915/gvt/gvt.c new file +16 -0
+
+    clean_pgt_device(pdev);
+    free_pgt_device(pdev);
+}
+
+void *gvt_create_pgt_device(struct drm_i915_private *dev_priv)
+{
+    struct pgt_device *pdev = NULL;
+    struct gvt_host *host = &gvt_host;
+
+    if (!host->initialized && !gvt_init_host()) {
+        gvt_err("gvt_init_host fail");
+        return NULL;
+    }
+
+    gvt_dbg_core("create new pgt device, i915 dev_priv: %p",

dev_priv);

drivers/gpu/drm/i915/gvt/gvt.c new file +28 -0
+
+    pdev = alloc_pgt_device(dev_priv);
+    if (!pdev) {
+        gvt_err("fail to allocate memory for pgt device.");
+        goto err;
+    }
+
+    gvt_dbg_core("init pgt device, id %d", pdev->id);
+
+    if (!init_pgt_device(pdev, dev_priv)) {
+        gvt_err("fail to init physical device state.");
+        goto err;
+    }
+
+    gvt_dbg_core("pgt device creation done, id %d", pdev->id);
+
+    return pdev;
+err:
+    if (pdev) {
+        gvt_destroy_pgt_device(pdev);
+        pdev = NULL;
+    }
+    return NULL;
+}
+
+bool gvt_post_init_pgt_device(void *private_data)
+{
+    struct pgt_device *pdev = (struct pgt_device

*)private_data;
+    struct gvt_host *host = &gvt_host;

drivers/gpu/drm/i915/gvt/gvt.c new file +15 -0
+
+    if (!host->initialized) {
+        gvt_err("gvt_host haven't been initialized.");
+        return false;
+    }
+
+    gvt_dbg_core("post init pgt device %d", pdev->id);
+
+    if (!post_init_pgt_device(pdev)) {
+        gvt_err("fail to post init physical device state.");
+        return false;
+    }
+
+    return true;
+}
drivers/gpu/drm/i915/gvt/gvt.h new file +2 -0
new file mode 100644
index 0000000..6c85bba
@@ -0,0 +1,130 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights

reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
"Software"),
+ * to deal in the Software without restriction, including without
limitation
+ * the rights to use, copy, modify, merge, publish, distribute,
sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom
the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including
the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO
EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/gvt.h new file +65 -0
+
+#ifndef _GVT_H_
+#define _GVT_H_
+
+#include "i915_drv.h"
+#include "i915_vgpu.h"
+
+#include "debug.h"
+#include "params.h"
+#include "reg.h"
+#include "hypercall.h"
+#include "mpt.h"
+#include "fb_decoder.h"
+
+#define GVT_MAX_VGPU 8
+
+enum {
+    GVT_HYPERVISOR_TYPE_XEN = 0,
+    GVT_HYPERVISOR_TYPE_KVM,
+};
+
+struct gvt_host {
+    bool initialized;
+    int hypervisor_type;
+    struct mutex device_idr_lock;
+    struct idr device_idr;
+    struct gvt_kernel_dm *kdm;
+};
+
+extern struct gvt_host gvt_host;
+
+/* Describe the limitation of HW.*/
+struct gvt_device_info {
+    u64 max_gtt_gm_sz;
+    u32 gtt_start_offset;
+    u32 gtt_end_offset;
+    u32 max_gtt_size;
+    u32 gtt_entry_size;
+    u32 gtt_entry_size_shift;
+    u32 gmadr_bytes_in_cmd;
+};
+
+struct vgt_device {
+    int id;
+    int vm_id;
+    struct pgt_device *pdev;
+    bool warn_untrack;
+};
+
+struct pgt_device {
+    struct mutex lock;
+    int id;
+
+    struct drm_i915_private *dev_priv;
+    struct idr instance_idr;
+
+    struct gvt_device_info device_info;
+
+    u8 initial_cfg_space[GVT_CFG_SPACE_SZ];
+    u64 bar_size[GVT_BAR_NUM];
+
+    u64 gttmmio_base;
+    void *gttmmio_va;
+
+    u64 gmadr_base;

drivers/gpu/drm/i915/gvt/gvt.h line 97

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +10 -0
+    void *gmadr_va;
+
+    u32 mmio_size;
+    u32 reg_num;
+
+    wait_queue_head_t service_thread_wq;
+    struct task_struct *service_thread;
+    unsigned long service_request;
+};
+

Here-->

Quoted 22 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +22 -0
+static inline u32 gvt_mmio_read(struct pgt_device *pdev,
+        u32 reg)
+{
+    struct drm_i915_private *dev_priv = pdev->dev_priv;
+    i915_reg_t tmp = {.reg = reg};
+    return I915_READ(tmp);
+}
+
+static inline void gvt_mmio_write(struct pgt_device *pdev,
+        u32 reg, u32 val)
+{
+    struct drm_i915_private *dev_priv = pdev->dev_priv;
+    i915_reg_t tmp = {.reg = reg};
+    I915_WRITE(tmp, val);
+}
+
+static inline u64 gvt_mmio_read64(struct pgt_device *pdev,
+        u32 reg)
+{
+    struct drm_i915_private *dev_priv = pdev->dev_priv;
+    i915_reg_t tmp = {.reg = reg};
+    return I915_READ64(tmp);

drivers/gpu/drm/i915/gvt/gvt.h line 129

Quoted 10 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +10 -0
+}
+
+static inline void gvt_mmio_write64(struct pgt_device *pdev,
+        u32 reg, u64 val)
+{
+    struct drm_i915_private *dev_priv = pdev->dev_priv;
+    i915_reg_t tmp = {.reg = reg};
+    I915_WRITE64(tmp, val);
+}
+

<-- Why? The i915_reg_t type was added to avoid problems, this code is
not used anywhere, and it has no documentation, so I can not review it.

Quoted 2 lines

I wrote comments at the top of the post.

[Zhi] Thanks, I will check that.

Quoted 517 lines

Regards, Joonas

Quoted 515 lines
drivers/gpu/drm/i915/gvt/gvt.h new file +1 -0
+#endif
drivers/gpu/drm/i915/gvt/hypercall.h new file +2 -0
new file mode 100644
index 0000000..0a41874
@@ -0,0 +1,30 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights

reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
"Software"),
+ * to deal in the Software without restriction, including without
limitation
+ * the rights to use, copy, modify, merge, publish, distribute,
sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom
the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including
the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO
EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/hypercall.h new file +8 -0
+
+#ifndef _GVT_HYPERCALL_H_
+#define _GVT_HYPERCALL_H_
+
+struct gvt_kernel_dm {
+};
+
+#endif /* _GVT_HYPERCALL_H_ */
drivers/gpu/drm/i915/gvt/mpt.h new file +2 -0
new file mode 100644
index 0000000..bbe4465
@@ -0,0 +1,97 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights

reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
"Software"),
+ * to deal in the Software without restriction, including without
limitation
+ * the rights to use, copy, modify, merge, publish, distribute,
sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom
the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including
the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO
EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/mpt.h new file +7 -0
+
+#ifndef _GVT_MPT_H_
+#define _GVT_MPT_H_
+
+struct vgt_device;
+
+static inline unsigned long hypervisor_g2m_pfn(struct

vgt_device *vgt,
+    unsigned long g_pfn)

drivers/gpu/drm/i915/gvt/mpt.h new file +5 -0
+{
+    return 0;
+}
+
+static inline int hypervisor_pause_domain(struct vgt_device

*vgt)

drivers/gpu/drm/i915/gvt/mpt.h new file +5 -0
+{
+    return 0;
+}
+
+static inline int hypervisor_shutdown_domain(struct vgt_device

*vgt)

drivers/gpu/drm/i915/gvt/mpt.h new file +5 -0
+{
+    return 0;
+}
+
+static inline int hypervisor_set_trap_area(struct vgt_device

*vgt,
+    uint64_t start, uint64_t end, bool map)

drivers/gpu/drm/i915/gvt/mpt.h new file +20 -0
+{
+    return 0;
+}
+
+static inline bool hypervisor_detect_host(void)
+{
+    return false;
+}
+
+static inline int hypervisor_virt_to_mfn(void *addr)
+{
+    return 0;
+}
+
+static inline void *hypervisor_mfn_to_virt(int mfn)
+{
+    return NULL;
+}
+
+static inline void hypervisor_inject_msi(struct vgt_device

*vgt)

drivers/gpu/drm/i915/gvt/mpt.h new file +14 -0
+{
+    return;
+}
+
+static inline int hypervisor_hvm_init(struct vgt_device *vgt)
+{
+    return 0;
+}
+
+static inline void hypervisor_hvm_exit(struct vgt_device *vgt)
+{
+}
+
+static inline void *hypervisor_gpa_to_va(struct vgt_device

*vgt,
unsigned long gpa)

drivers/gpu/drm/i915/gvt/mpt.h new file +5 -0
+{
+    return NULL;
+}
+
+static inline bool hypervisor_read_va(struct vgt_device *vgt,

void *va,
+        void *val, int len, int atomic)

drivers/gpu/drm/i915/gvt/mpt.h new file +5 -0
+{
+    return false;
+}
+
+static inline bool hypervisor_write_va(struct vgt_device *vgt,

void
*va,
+        void *val, int len, int atomic)

drivers/gpu/drm/i915/gvt/mpt.h new file +5 -0
+{
+    return false;
+}
+
+#endif /* _GVT_MPT_H_ */
drivers/gpu/drm/i915/gvt/params.c new file +2 -0
new file mode 100644
index 0000000..dfc33c3
@@ -0,0 +1,29 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights

reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
"Software"),
+ * to deal in the Software without restriction, including without
limitation
+ * the rights to use, copy, modify, merge, publish, distribute,
sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom
the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including
the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO
EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/params.c new file +7 -0
+
+#include "gvt.h"
+
+struct gvt_kernel_params gvt = {
+    .enable = true,
+    .debug = 0,
+};
drivers/gpu/drm/i915/gvt/params.h new file +2 -0
new file mode 100644
index 0000000..d2955b9
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights

reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
"Software"),
+ * to deal in the Software without restriction, including without
limitation
+ * the rights to use, copy, modify, merge, publish, distribute,
sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom
the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including
the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO
EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/params.h new file +12 -0
+
+#ifndef _GVT_PARAMS_H_
+#define _GVT_PARAMS_H_
+
+struct gvt_kernel_params {
+    bool enable;
+    int debug;
+};
+
+extern struct gvt_kernel_params gvt;
+
+#endif
drivers/gpu/drm/i915/gvt/reg.h new file +2 -0
new file mode 100644
index 0000000..d363b74
@@ -0,0 +1,34 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights

reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
"Software"),
+ * to deal in the Software without restriction, including without
limitation
+ * the rights to use, copy, modify, merge, publish, distribute,
sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom
the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including
the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO
EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE
+ * SOFTWARE.
+ */

drivers/gpu/drm/i915/gvt/reg.h new file +12 -0
+
+#ifndef _GVT_REG_H
+#define _GVT_REG_H
+
+#define GVT_CFG_SPACE_SZ    256
+#define GVT_BAR_NUM        4
+
+#define GVT_REG_CFG_SPACE_BAR0    0x10
+#define GVT_REG_CFG_SPACE_BAR1    0x18
+#define GVT_REG_CFG_SPACE_BAR2    0x20
+
+#endif
drivers/gpu/drm/i915/i915_dma.c +4 -0
index 4725e8d..eca8e50 100644
@@ -943,6 +943,10 @@ int i915_driver_load(struct drm_device
 *dev,
 unsigned long flags)
 
       intel_uncore_init(dev);
 
+    dev_priv->vgpu.host_private_data =
 gvt_create_pgt_device(dev_priv);
+    if(intel_gvt_host_active(dev))
+        DRM_INFO("GVT-g is running in host mode\n");
+

      ret = i915_gem_gtt_init(dev);
      if (ret)
          goto out_freecsr;

drivers/gpu/drm/i915/i915_dma.c +7 -0
@@ -1067,6 +1071,13 @@ int i915_driver_load(struct drm_device
 *dev,
 unsigned long flags)
           goto out_power_well;
       }
 
+    if (intel_gvt_host_active(dev)) {
+        if
 (!gvt_post_init_pgt_device(dev_priv->vgpu.host_private_data)) {
+            DRM_ERROR("failed to post init pgt device\n");
+            goto out_power_well;
+        }
+    }
+

      /*
       * Notify a valid surface after modesetting,
       * when running inside a VM.

drivers/gpu/drm/i915/i915_dma.c +0 -0
@@ -1117,6 +1128,10 @@ out_gtt:
       i915_global_gtt_cleanup(dev);

  out_freecsr:
      intel_csr_ucode_fini(dev_priv);
+    if (intel_gvt_host_active(dev)) {
+        gvt_destroy_pgt_device(dev_priv-

Quoted 1 line

vgpu.host_private_data);

+        dev_priv->vgpu.host_private_data = NULL;
+    }
      intel_uncore_fini(dev);
      pci_iounmap(dev->pdev, dev_priv->regs);
  put_bridge:

drivers/gpu/drm/i915/i915_dma.c +4 -0
@@ -1165,6 +1180,10 @@ int i915_driver_unload(struct drm_device
 *dev)
 
       intel_modeset_cleanup(dev);
 
+    if (intel_gvt_host_active(dev)) {
+        gvt_destroy_pgt_device(dev_priv-
 >vgpu.host_private_data);
+        dev_priv->vgpu.host_private_data = NULL;
+    }
       /*

       * free the memory space allocated for the child device
       * config parsed from VBT

drivers/gpu/drm/i915/i915_drv.h +1 -0
index 01cc982..db3c79b 100644
@@ -1673,6 +1673,7 @@ struct i915_workarounds {
 
   struct i915_virtual_gpu {
       bool active;
+    void *host_private_data;
   };
 
   struct i915_execbuffer_params {
@@ -2747,6 +2748,11 @@ static inline bool

intel_vgpu_active(struct
drm_device *dev)
      return to_i915(dev)->vgpu.active;
  }

+static inline bool intel_gvt_host_active(struct drm_device *dev)
+{
+    return to_i915(dev)->vgpu.host_private_data ? true :
false;

drivers/gpu/drm/i915/i915_drv.h +2 -0
+}
+

  void
  i915_enable_pipestat(struct drm_i915_private *dev_priv, enum pipe
pipe,
               u32 status_mask);

drivers/gpu/drm/i915/i915_vgpu.h +3 -0
index 3c83b47..942490a 100644
@@ -113,5 +113,8 @@ struct vgt_if {
   extern void i915_check_vgpu(struct drm_device *dev);
   extern int intel_vgt_balloon(struct drm_device *dev);
   extern void intel_vgt_deballoon(void);
+extern void *gvt_create_pgt_device(struct drm_i915_private
 *dev_priv);
+extern bool gvt_post_init_pgt_device(void *private_data);
+extern void gvt_destroy_pgt_device(void *private_data);
 

  #endif /* _I915_VGPU_H_ */

include/xen/interface/xen.h +1 -0
index d133112..78a38f1 100644
@@ -28,6 +28,7 @@
   #define __XEN_PUBLIC_XEN_H__
 
   #include
+#include
 
   /*
    * XEN "SYSTEM CALLS" (a.k.a. HYPERCALLS).
Signature · 8 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#12 lore raw

RFC 2/29 drm/i915: Introduce host graphics memory balloon for gvt

In reply to Zhi Wang

From: Bing Niu <[email protected]>

This patch introduces host graphics memory ballon when GVT-g is enabled.

As under GVT-g, i915 only owned limited graphics resources, others are managed by GVT-g resource allocator and kept for other vGPUs.

For graphics memory space partition, a typical layout looks like:

+-------+-----------------------+------+-----------------------+
|* Host |   *GVT-g Resource     |* Host|   *GVT-g Resource     |
| Owned |   Allocator Managed   | Owned|   Allocator Managed   |
|       |                       |      |                       |
+---------------+-------+----------------------+-------+-------+
|       |       |       |       |      |       |       |       |
| i915  | vm 1  | vm 2  | vm 3  | i915 | vm 1  | vm 2  | vm 3  |
|       |       |       |       |      |       |       |       |
+-------+-------+-------+--------------+-------+-------+-------+
|           Aperture            |            Hidden            |
+-------------------------------+------------------------------+
|                       GGTT memory space                      |
+--------------------------------------------------------------+

Similar with fence registers partition:

 +------ +-----------------------+
 | * Host|    GVT-g Resource     |
 | Owned |   Allocator Managed   +
 0       |                       31
 +---------------+-------+-------+
 |       |       |       |       |
 | i915  | vm 1  | vm 2  | vm 3  |
 |       |       |       |       |
 +-------+-------+-------+-------+

i915 host will read the amount of allocated resources via GVT-g kernel parameters.

---
 drivers/gpu/drm/i915/gvt/params.h   |  3 +++
 drivers/gpu/drm/i915/i915_gem.c     |  3 +++
 drivers/gpu/drm/i915/i915_gem_gtt.c |  4 ++--
 drivers/gpu/drm/i915/i915_vgpu.c    | 16 ++++++++++++----
 drivers/gpu/drm/i915/i915_vgpu.h    |  1 +
 5 files changed, 21 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/gvt/params.h +3 -0
index d2955b9..0656a98 100644
@@ -27,6 +27,9 @@
 struct gvt_kernel_params {
 	bool enable;
 	int debug;
+	int dom0_low_gm_sz;
+	int dom0_high_gm_sz;
+	int dom0_fence_sz;
 };
 
 extern struct gvt_kernel_params gvt;
drivers/gpu/drm/i915/i915_gem.c +3 -0
index 799a53a..e916e43 100644
@@ -5080,6 +5080,9 @@ i915_gem_load(struct drm_device *dev)
 	else
 		dev_priv->num_fence_regs = 8;
 
+	if(intel_gvt_host_active(dev))
+		dev_priv->num_fence_regs = gvt.dom0_fence_sz;
+
 	if (intel_vgpu_active(dev))
 		dev_priv->num_fence_regs =
 				I915_READ(vgtif_reg(avail_rs.fence_num));
drivers/gpu/drm/i915/i915_gem_gtt.c +2 -2
index 7377b67..0540de2 100644
@@ -2713,7 +2713,7 @@ static int i915_gem_setup_global_gtt(struct drm_device *dev,
 	i915_address_space_init(ggtt_vm, dev_priv);
 	ggtt_vm->total += PAGE_SIZE;
 
-	if (intel_vgpu_active(dev)) {
+	if (intel_vgpu_active(dev) || intel_gvt_host_active(dev)) {
 		ret = intel_vgt_balloon(dev);
 		if (ret)
 			return ret;
@@ -2810,7 +2810,7 @@ void i915_global_gtt_cleanup(struct drm_device *dev)
 	}
 
 	if (drm_mm_initialized(&vm->mm)) {
-		if (intel_vgpu_active(dev))
+		if (intel_vgpu_active(dev) || intel_gvt_host_active(dev))
 			intel_vgt_deballoon();
 
 		drm_mm_takedown(&vm->mm);
drivers/gpu/drm/i915/i915_vgpu.c +12 -4
index dea7429..fbe6114 100644
@@ -188,10 +188,18 @@ int intel_vgt_balloon(struct drm_device *dev)
 	unsigned long unmappable_base, unmappable_size, unmappable_end;
 	int ret;
 
-	mappable_base = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.base));
-	mappable_size = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.size));
-	unmappable_base = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.base));
-	unmappable_size = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.size));
+	if(intel_gvt_host_active(dev)) {
+		mappable_base = 0;
+		mappable_size = gvt.dom0_low_gm_sz << 20;
+		unmappable_base = dev_priv->gtt.mappable_end;
+		unmappable_size = gvt.dom0_high_gm_sz << 20;
+	} else if (intel_vgpu_active(dev)) {
+		mappable_base = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.base));
+		mappable_size = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.size));
+		unmappable_base = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.base));
+		unmappable_size = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.size));
+	} else
+		return -ENODEV;
 
 	mappable_end = mappable_base + mappable_size;
 	unmappable_end = unmappable_base + unmappable_size;
drivers/gpu/drm/i915/i915_vgpu.h +1 -0
index 942490a..b8a49e6 100644
@@ -24,6 +24,7 @@
 #ifndef _I915_VGPU_H_
 #define _I915_VGPU_H_
 
+#include "gvt/params.h"
 /* The MMIO offset of the shared info between guest and host emulator */
 #define VGT_PVINFO_PAGE	0x78000
 #define VGT_PVINFO_SIZE	0x1000
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer
#13 lore raw

Re: [RFC 02/29] drm/i915: Introduce host graphics memory balloon for gvt

Hi,

Quoted 69 lines · Zhi Wang

From: Bing Niu <[email protected]>

This patch introduces host graphics memory ballon when GVT-g is enabled.

As under GVT-g, i915 only owned limited graphics resources, others are managed by GVT-g resource allocator and kept for other vGPUs.

For graphics memory space partition, a typical layout looks like:

+-------+-----------------------+------+-----------------------+
> * Host |   *GVT-g Resource     |* Host|   *GVT-g Resource     |
> Owned |   Allocator Managed   | Owned|   Allocator Managed   |
>       |                       |      |                       |
+---------------+-------+----------------------+-------+-------+
>       |       |       |       |      |       |       |       |
> i915  | vm 1  | vm 2  | vm 3  | i915 | vm 1  | vm 2  | vm 3  |
>       |       |       |       |      |       |       |       |
+-------+-------+-------+--------------+-------+-------+-------+
>           Aperture            |            Hidden            |
+-------------------------------+------------------------------+
>                       GGTT memory space                      |
+--------------------------------------------------------------+

Similar with fence registers partition:

 +------ +-----------------------+
 | * Host|    GVT-g Resource     |
 | Owned |   Allocator Managed   +
 0       |                       31
 +---------------+-------+-------+
 |       |       |       |       |
 | i915  | vm 1  | vm 2  | vm 3  |
 |       |       |       |       |
 +-------+-------+-------+-------+

i915 host will read the amount of allocated resources via GVT-g kernel parameters.

---
 drivers/gpu/drm/i915/gvt/params.h   |  3 +++ drivers/gpu/drm/i915/i915_gem.c     |  3 +++ drivers/gpu/drm/i915/i915_gem_gtt.c |  4 ++-- drivers/gpu/drm/i915/i915_vgpu.c    | 16 ++++++++++++---- drivers/gpu/drm/i915/i915_vgpu.h    |  1 +

 5 files changed, 21 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/gvt/params.h +3 -0
index d2955b9..0656a98 100644
@@ -27,6 +27,9 @@
  struct gvt_kernel_params {
  	bool enable;
  	int debug;
+	int dom0_low_gm_sz;
+	int dom0_high_gm_sz;
+	int dom0_fence_sz;
  };


 extern struct gvt_kernel_params gvt;

drivers/gpu/drm/i915/i915_gem.c +0 -0
index 799a53a..e916e43 100644
@@ -5080,6 +5080,9 @@ i915_gem_load(struct drm_device *dev)

  else
  dev_priv->num_fence_regs = 8;

drivers/gpu/drm/i915/i915_gem.c i915_gem_load(struct drm_device *dev) · line 5083

Quoted 1 line
drivers/gpu/drm/i915/i915_gem.c +1 -0
+	if(intel_gvt_host_active(dev))

Space between "if" and "("

Quoted 5 lines
drivers/gpu/drm/i915/i915_gem.c +2 -0
+		dev_priv->num_fence_regs = gvt.dom0_fence_sz;
+

  if (intel_vgpu_active(dev))
  dev_priv->num_fence_regs = I915_READ(vgtif_reg(avail_rs.fence_num));

I'd like to see the above as "else if" construct just like you have below with the intel_vgt_balloon(). Could even do

	if (gvt_host)  {
		...
	} else if (vgpu_active) {
		...
	} else {
		per machine detection
	}

Right?

Quoted 46 lines
drivers/gpu/drm/i915/i915_gem_gtt.c +0 -0
index 7377b67..0540de2 100644
@@ -2713,7 +2713,7 @@ static int i915_gem_setup_global_gtt(struct drm_device *dev,
  	i915_address_space_init(ggtt_vm, dev_priv);
  	ggtt_vm->total += PAGE_SIZE;

drivers/gpu/drm/i915/i915_gem_gtt.c +1 -1
-	if (intel_vgpu_active(dev)) {
+	if (intel_vgpu_active(dev) || intel_gvt_host_active(dev)) {

  ret = intel_vgt_balloon(dev);
  if (ret)
  return ret;

drivers/gpu/drm/i915/i915_gem_gtt.c +0 -0
@@ -2810,7 +2810,7 @@ void i915_global_gtt_cleanup(struct drm_device *dev)
  	}


  if (drm_mm_initialized(&vm->mm)) {

drivers/gpu/drm/i915/i915_gem_gtt.c +1 -1
-		if (intel_vgpu_active(dev))
+		if (intel_vgpu_active(dev) || intel_gvt_host_active(dev))

  intel_vgt_deballoon();

  drm_mm_takedown(&vm->mm);

drivers/gpu/drm/i915/i915_vgpu.c +0 -0
index dea7429..fbe6114 100644
@@ -188,10 +188,18 @@ int intel_vgt_balloon(struct drm_device *dev)
  	unsigned long unmappable_base, unmappable_size, unmappable_end;
  	int ret;

drivers/gpu/drm/i915/i915_vgpu.c +12 -4
-	mappable_base = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.base));
-	mappable_size = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.size));
-	unmappable_base = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.base));
-	unmappable_size = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.size));
+	if(intel_gvt_host_active(dev)) {
+		mappable_base = 0;
+		mappable_size = gvt.dom0_low_gm_sz << 20;
+		unmappable_base = dev_priv->gtt.mappable_end;
+		unmappable_size = gvt.dom0_high_gm_sz << 20;
+	} else if (intel_vgpu_active(dev)) {
+		mappable_base = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.base));
+		mappable_size = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.size));
+		unmappable_base = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.base));
+		unmappable_size = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.size));
+	} else
+		return -ENODEV;
Quoted 1 line

	} else {
		return -ENODEV;
	}

This must use braces too, according to the Kernel Coding Style.

Quoted 11 lines

  mappable_end = mappable_base + mappable_size; unmappable_end = unmappable_base + unmappable_size;

drivers/gpu/drm/i915/i915_vgpu.h +0 -0
index 942490a..b8a49e6 100644
@@ -24,6 +24,7 @@
  #ifndef _I915_VGPU_H_
  #define _I915_VGPU_H_


+#include "gvt/params.h"

This is not a good way of including headers, as this header itself doesn't need it. Including unnecessary stuff in often included headers contributes to increasing project compile times and makes it harder to solve cross dependencies later. So put it into the implementation file that needs it.

Regards, Joonas

Quoted 3 lines

 /* The MMIO offset of the shared info between guest and host emulator */
 #define VGT_PVINFO_PAGE 0x78000
 #define VGT_PVINFO_SIZE 0x1000

Signature · 7 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#14 lore raw

Re: [RFC 02/29] drm/i915: Introduce host graphics memory balloon for gvt

Hi Joonas,

Thanks much for the review! We will incorporate those review comments!

Meanwhile, is it good enough to do the host ballooning like below? The pros is that it is very simple, especially consider that guest ballooning logic has been there. Thanks!

Regards,
-Zhiyuan

Quoted 67 lines · Joonas Lahtinen

Hi,

Quoted 64 lines · Zhi Wang

From: Bing Niu <[email protected]>

This patch introduces host graphics memory ballon when GVT-g is enabled.

As under GVT-g, i915 only owned limited graphics resources, others are managed by GVT-g resource allocator and kept for other vGPUs.

For graphics memory space partition, a typical layout looks like:

+-------+-----------------------+------+-----------------------+
> * Host |   *GVT-g Resource     |* Host|   *GVT-g Resource     |
> Owned |   Allocator Managed   | Owned|   Allocator Managed   |
>       |                       |      |                       |
+---------------+-------+----------------------+-------+-------+
>       |       |       |       |      |       |       |       |
> i915  | vm 1  | vm 2  | vm 3  | i915 | vm 1  | vm 2  | vm 3  |
>       |       |       |       |      |       |       |       |
+-------+-------+-------+--------------+-------+-------+-------+
>           Aperture            |            Hidden            |
+-------------------------------+------------------------------+
>                       GGTT memory space                      |
+--------------------------------------------------------------+

Similar with fence registers partition:

 +------ +-----------------------+
 | * Host|    GVT-g Resource     |
 | Owned |   Allocator Managed   +
 0       |                       31
 +---------------+-------+-------+
 |       |       |       |       |
 | i915  | vm 1  | vm 2  | vm 3  |
 |       |       |       |       |
 +-------+-------+-------+-------+

i915 host will read the amount of allocated resources via GVT-g kernel parameters.

---
 drivers/gpu/drm/i915/gvt/params.h   |  3 +++ drivers/gpu/drm/i915/i915_gem.c     |  3 +++ drivers/gpu/drm/i915/i915_gem_gtt.c |  4 ++-- drivers/gpu/drm/i915/i915_vgpu.c    | 16 ++++++++++++---- drivers/gpu/drm/i915/i915_vgpu.h    |  1 +

 5 files changed, 21 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/gvt/params.h +3 -0
index d2955b9..0656a98 100644
@@ -27,6 +27,9 @@
  struct gvt_kernel_params {
  	bool enable;
  	int debug;
+	int dom0_low_gm_sz;
+	int dom0_high_gm_sz;
+	int dom0_fence_sz;
  };


 extern struct gvt_kernel_params gvt;

drivers/gpu/drm/i915/i915_gem.c +0 -1
index 799a53a..e916e43 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
Quoted 8 lines

drivers/gpu/drm/i915/i915_gem.c i915_gem_load(struct drm_device *dev) · line 5083

Quoted 6 lines

+++ b/drivers/gpu/drm/i915/i915_gem.c

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -5080,6 +5080,9 @@ i915_gem_load(struct drm_device *dev)

  else
  dev_priv->num_fence_regs = 8;

drivers/gpu/drm/i915/i915_gem.c +1 -0
+	if(intel_gvt_host_active(dev))

Space between "if" and "("

Thanks! Will correct it.

Quoted 18 lines
Quoted 6 lines
drivers/gpu/drm/i915/i915_gem.c +2 -0
+		dev_priv->num_fence_regs = gvt.dom0_fence_sz;
+

  if (intel_vgpu_active(dev))
  dev_priv->num_fence_regs = I915_READ(vgtif_reg(avail_rs.fence_num));

I'd like to see the above as "else if" construct just like you have below with the intel_vgt_balloon(). Could even do

	if (gvt_host)  {
		...
	} else if (vgpu_active) {
		...
	} else {
		per machine detection
	}
Quoted 1 line

Right?

That looks better!

Quoted 52 lines
Quoted 47 lines
drivers/gpu/drm/i915/i915_gem_gtt.c +0 -0
index 7377b67..0540de2 100644
@@ -2713,7 +2713,7 @@ static int i915_gem_setup_global_gtt(struct drm_device *dev,
  	i915_address_space_init(ggtt_vm, dev_priv);
  	ggtt_vm->total += PAGE_SIZE;

drivers/gpu/drm/i915/i915_gem_gtt.c +1 -1
-	if (intel_vgpu_active(dev)) {
+	if (intel_vgpu_active(dev) || intel_gvt_host_active(dev)) {

  ret = intel_vgt_balloon(dev);
  if (ret)
  return ret;

drivers/gpu/drm/i915/i915_gem_gtt.c +0 -0
@@ -2810,7 +2810,7 @@ void i915_global_gtt_cleanup(struct drm_device *dev)
  	}


  if (drm_mm_initialized(&vm->mm)) {

drivers/gpu/drm/i915/i915_gem_gtt.c +1 -1
-		if (intel_vgpu_active(dev))
+		if (intel_vgpu_active(dev) || intel_gvt_host_active(dev))

  intel_vgt_deballoon();

  drm_mm_takedown(&vm->mm);

drivers/gpu/drm/i915/i915_vgpu.c +0 -0
index dea7429..fbe6114 100644
@@ -188,10 +188,18 @@ int intel_vgt_balloon(struct drm_device *dev)
  	unsigned long unmappable_base, unmappable_size, unmappable_end;
  	int ret;

drivers/gpu/drm/i915/i915_vgpu.c +12 -4
-	mappable_base = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.base));
-	mappable_size = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.size));
-	unmappable_base = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.base));
-	unmappable_size = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.size));
+	if(intel_gvt_host_active(dev)) {
+		mappable_base = 0;
+		mappable_size = gvt.dom0_low_gm_sz << 20;
+		unmappable_base = dev_priv->gtt.mappable_end;
+		unmappable_size = gvt.dom0_high_gm_sz << 20;
+	} else if (intel_vgpu_active(dev)) {
+		mappable_base = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.base));
+		mappable_size = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.size));
+		unmappable_base = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.base));
+		unmappable_size = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.size));
+	} else
+		return -ENODEV;
Quoted 1 line

	} else {
		return -ENODEV;
	}
Quoted 1 line

This must use braces too, according to the Kernel Coding Style.

That's right. Thanks for pointing it out!

Quoted 10 lines
Quoted 10 lines

  mappable_end = mappable_base + mappable_size; unmappable_end = unmappable_base + unmappable_size;

drivers/gpu/drm/i915/i915_vgpu.h +0 -0
index 942490a..b8a49e6 100644
@@ -24,6 +24,7 @@
  #ifndef _I915_VGPU_H_
  #define _I915_VGPU_H_
Quoted 9 lines
Quoted 2 lines


+#include "gvt/params.h"

This is not a good way of including headers, as this header itself doesn't need it. Including unnecessary stuff in often included headers contributes to increasing project compile times and makes it harder to solve cross dependencies later. So put it into the implementation file that needs it.

Will delete. Thanks!

Quoted 9 lines

Regards, Joonas

Quoted 3 lines

 /* The MMIO offset of the shared info between guest and host emulator */
 #define VGT_PVINFO_PAGE 0x78000
 #define VGT_PVINFO_SIZE 0x1000

Signature · 3 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

maintainer
#15 lore raw

Re: [RFC 02/29] drm/i915: Introduce host graphics memory balloon for gvt

Hi,

Quoted 8 lines · Zhiyuan Lv

Hi Joonas,

Thanks much for the review! We will incorporate those review comments!

Meanwhile, is it good enough to do the host ballooning like below? The pros is that it is very simple, especially consider that guest ballooning logic has been there. Thanks!

I think slicing it down like that is fine. Is there going to be real ballooning happening, as in, is the available memory dynamically going to change during runtime, or just be fixed size since the beginning?

Regards, Joonas

Quoted 54 lines

Regards,
-Zhiyuan

Quoted 50 lines · Joonas Lahtinen

Hi,

Quoted 47 lines · Zhi Wang

From: Bing Niu <[email protected]>

This patch introduces host graphics memory ballon when GVT-g is enabled.

As under GVT-g, i915 only owned limited graphics resources, others are managed by GVT-g resource allocator and kept for other vGPUs.

For graphics memory space partition, a typical layout looks like:

+-------+-----------------------+------+-----------------------+
> * Host |   *GVT-g Resource     |* Host|   *GVT-g Resource     |
> Owned |   Allocator Managed   | Owned|   Allocator Managed   |
>       |                       |      |                       |
+---------------+-------+----------------------+-------+-------+
>       |       |       |       |      |       |       |       |
> i915  | vm 1  | vm 2  | vm 3  | i915 | vm 1  | vm 2  | vm 3  |
>       |       |       |       |      |       |       |       |
+-------+-------+-------+--------------+-------+-------+-------+
>           Aperture            |            Hidden            |
+-------------------------------+------------------------------+
>                       GGTT memory space                      |
+--------------------------------------------------------------+

Similar with fence registers partition:

 +------ +-----------------------+
 | * Host|    GVT-g Resource     |
 | Owned |   Allocator Managed   +
 0       |                       31
 +---------------+-------+-------+
 |       |       |       |       |
 | i915  | vm 1  | vm 2  | vm 3  |
 |       |       |       |       |
 +-------+-------+-------+-------+

i915 host will read the amount of allocated resources via GVT-g kernel parameters.

---
 drivers/gpu/drm/i915/gvt/params.h   |  3 +++ drivers/gpu/drm/i915/i915_gem.c     |  3 +++ drivers/gpu/drm/i915/i915_gem_gtt.c |  4 ++-- drivers/gpu/drm/i915/i915_vgpu.c    | 16 ++++++++++++---- drivers/gpu/drm/i915/i915_vgpu.h    |  1 +

 5 files changed, 21 insertions(+), 6 deletions(-)
Quoted 11 lines
Quoted 11 lines

drivers/gpu/drm/i915/gvt/params.h line 32

Quoted 11 lines
drivers/gpu/drm/i915/gvt/params.h +3 -0
index d2955b9..0656a98 100644
@@ -27,6 +27,9 @@
  struct gvt_kernel_params {
  	bool enable;
  	int debug;
+	int dom0_low_gm_sz;
+	int dom0_high_gm_sz;
+	int dom0_fence_sz;

Some comment on the unit would be nice here, as there is a shift with << 20 later on.

Is "unsigned long" type and count in pages not acceptable? I think that's the assumption one is going to make.

Quoted 122 lines
Quoted 6 lines
Quoted 6 lines
drivers/gpu/drm/i915/gvt/params.h +0 -0
  };


 extern struct gvt_kernel_params gvt;

drivers/gpu/drm/i915/i915_gem.c +0 -1
index 799a53a..e916e43 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
Quoted 8 lines

drivers/gpu/drm/i915/i915_gem.c i915_gem_load(struct drm_device *dev) · line 5083

Quoted 6 lines

+++ b/drivers/gpu/drm/i915/i915_gem.c

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -5080,6 +5080,9 @@ i915_gem_load(struct drm_device *dev)

  else
  dev_priv->num_fence_regs = 8;

drivers/gpu/drm/i915/i915_gem.c +1 -0
+	if(intel_gvt_host_active(dev))

Space between "if" and "("

Thanks! Will correct it.

Quoted 18 lines
Quoted 6 lines
drivers/gpu/drm/i915/i915_gem.c +2 -0
+		dev_priv->num_fence_regs = gvt.dom0_fence_sz;
+

  if (intel_vgpu_active(dev))
  dev_priv->num_fence_regs = I915_READ(vgtif_reg(avail_rs.fence_num));

I'd like to see the above as "else if" construct just like you have below with the intel_vgt_balloon(). Could even do

	if (gvt_host)  {
		...
	} else if (vgpu_active) {
		...
	} else {
		per machine detection
	}
Quoted 1 line

Right?

That looks better!

Quoted 52 lines
Quoted 47 lines
drivers/gpu/drm/i915/i915_gem_gtt.c +0 -0
index 7377b67..0540de2 100644
@@ -2713,7 +2713,7 @@ static int i915_gem_setup_global_gtt(struct drm_device *dev,
  	i915_address_space_init(ggtt_vm, dev_priv);
  	ggtt_vm->total += PAGE_SIZE;

drivers/gpu/drm/i915/i915_gem_gtt.c +1 -1
-	if (intel_vgpu_active(dev)) {
+	if (intel_vgpu_active(dev) || intel_gvt_host_active(dev)) {

  ret = intel_vgt_balloon(dev);
  if (ret)
  return ret;

drivers/gpu/drm/i915/i915_gem_gtt.c +0 -0
@@ -2810,7 +2810,7 @@ void i915_global_gtt_cleanup(struct drm_device *dev)
  	}


  if (drm_mm_initialized(&vm->mm)) {

drivers/gpu/drm/i915/i915_gem_gtt.c +1 -1
-		if (intel_vgpu_active(dev))
+		if (intel_vgpu_active(dev) || intel_gvt_host_active(dev))

  intel_vgt_deballoon();

  drm_mm_takedown(&vm->mm);

drivers/gpu/drm/i915/i915_vgpu.c +0 -0
index dea7429..fbe6114 100644
@@ -188,10 +188,18 @@ int intel_vgt_balloon(struct drm_device *dev)
  	unsigned long unmappable_base, unmappable_size, unmappable_end;
  	int ret;

drivers/gpu/drm/i915/i915_vgpu.c +12 -4
-	mappable_base = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.base));
-	mappable_size = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.size));
-	unmappable_base = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.base));
-	unmappable_size = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.size));
+	if(intel_gvt_host_active(dev)) {
+		mappable_base = 0;
+		mappable_size = gvt.dom0_low_gm_sz << 20;
+		unmappable_base = dev_priv->gtt.mappable_end;
+		unmappable_size = gvt.dom0_high_gm_sz << 20;
+	} else if (intel_vgpu_active(dev)) {
+		mappable_base = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.base));
+		mappable_size = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.size));
+		unmappable_base = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.base));
+		unmappable_size = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.size));
+	} else
+		return -ENODEV;
Quoted 1 line

	} else {
		return -ENODEV;
	}
Quoted 1 line

This must use braces too, according to the Kernel Coding Style.

That's right. Thanks for pointing it out!

Quoted 10 lines
Quoted 10 lines

  mappable_end = mappable_base + mappable_size; unmappable_end = unmappable_base + unmappable_size;

drivers/gpu/drm/i915/i915_vgpu.h +0 -0
index 942490a..b8a49e6 100644
@@ -24,6 +24,7 @@
  #ifndef _I915_VGPU_H_
  #define _I915_VGPU_H_
Quoted 9 lines
Quoted 2 lines


+#include "gvt/params.h"

This is not a good way of including headers, as this header itself doesn't need it. Including unnecessary stuff in often included headers contributes to increasing project compile times and makes it harder to solve cross dependencies later. So put it into the implementation file that needs it.

Will delete. Thanks!

Quoted 5 lines

Regards, Joonas

Quoted 3 lines

 /* The MMIO offset of the shared info between guest and host emulator */
 #define VGT_PVINFO_PAGE 0x78000
 #define VGT_PVINFO_SIZE 0x1000

Signature · 8 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#16 lore raw

Re: [RFC 02/29] drm/i915: Introduce host graphics memory balloon for gvt

Hi Joonas,

Quoted 7 lines · Joonas Lahtinen

Hi,

Quoted 4 lines · Zhiyuan Lv

Hi Joonas,

Thanks much for the review! We will incorporate those review comments!

Quoted 8 lines
Quoted 4 lines

Meanwhile, is it good enough to do the host ballooning like below? The pros is that it is very simple, especially consider that guest ballooning logic has been there. Thanks!

I think slicing it down like that is fine. Is there going to be real ballooning happening, as in, is the available memory dynamically going to change during runtime, or just be fixed size since the beginning?

In our current design, once the gvt is enabled, it is fixed size of graphics memory for host since beginning. The reserved memory is all for virtual machines.

We ever considered to support dynamically enabling/disabling gvt, that host i915 could boot without gvt enabled, and can use all the hardware resources. When there is need to creat VM, we can reload i915 driver to reserve the resources. Does that sound good? Thanks!

Regards,
-Zhiyuan

Quoted 190 lines

Regards, Joonas

Quoted 54 lines

Regards,
-Zhiyuan

Quoted 50 lines · Joonas Lahtinen

Hi,

Quoted 47 lines · Zhi Wang

From: Bing Niu <[email protected]>

This patch introduces host graphics memory ballon when GVT-g is enabled.

As under GVT-g, i915 only owned limited graphics resources, others are managed by GVT-g resource allocator and kept for other vGPUs.

For graphics memory space partition, a typical layout looks like:

+-------+-----------------------+------+-----------------------+
> * Host |   *GVT-g Resource     |* Host|   *GVT-g Resource     |
> Owned |   Allocator Managed   | Owned|   Allocator Managed   |
>       |                       |      |                       |
+---------------+-------+----------------------+-------+-------+
>       |       |       |       |      |       |       |       |
> i915  | vm 1  | vm 2  | vm 3  | i915 | vm 1  | vm 2  | vm 3  |
>       |       |       |       |      |       |       |       |
+-------+-------+-------+--------------+-------+-------+-------+
>           Aperture            |            Hidden            |
+-------------------------------+------------------------------+
>                       GGTT memory space                      |
+--------------------------------------------------------------+

Similar with fence registers partition:

 +------ +-----------------------+
 | * Host|    GVT-g Resource     |
 | Owned |   Allocator Managed   +
 0       |                       31
 +---------------+-------+-------+
 |       |       |       |       |
 | i915  | vm 1  | vm 2  | vm 3  |
 |       |       |       |       |
 +-------+-------+-------+-------+

i915 host will read the amount of allocated resources via GVT-g kernel parameters.

---
 drivers/gpu/drm/i915/gvt/params.h   |  3 +++ drivers/gpu/drm/i915/i915_gem.c     |  3 +++ drivers/gpu/drm/i915/i915_gem_gtt.c |  4 ++-- drivers/gpu/drm/i915/i915_vgpu.c    | 16 ++++++++++++---- drivers/gpu/drm/i915/i915_vgpu.h    |  1 +

 5 files changed, 21 insertions(+), 6 deletions(-)
Quoted 11 lines
Quoted 11 lines

drivers/gpu/drm/i915/gvt/params.h line 32

Quoted 11 lines
drivers/gpu/drm/i915/gvt/params.h +3 -0
index d2955b9..0656a98 100644
@@ -27,6 +27,9 @@
  struct gvt_kernel_params {
  	bool enable;
  	int debug;
+	int dom0_low_gm_sz;
+	int dom0_high_gm_sz;
+	int dom0_fence_sz;

Some comment on the unit would be nice here, as there is a shift with << 20 later on.

Is "unsigned long" type and count in pages not acceptable? I think that's the assumption one is going to make.

Quoted 115 lines
Quoted 6 lines
Quoted 6 lines
drivers/gpu/drm/i915/gvt/params.h +0 -0
  };


 extern struct gvt_kernel_params gvt;

drivers/gpu/drm/i915/i915_gem.c +0 -1
index 799a53a..e916e43 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
Quoted 8 lines

drivers/gpu/drm/i915/i915_gem.c i915_gem_load(struct drm_device *dev) · line 5083

Quoted 6 lines

+++ b/drivers/gpu/drm/i915/i915_gem.c

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -5080,6 +5080,9 @@ i915_gem_load(struct drm_device *dev)

  else
  dev_priv->num_fence_regs = 8;

drivers/gpu/drm/i915/i915_gem.c +1 -0
+	if(intel_gvt_host_active(dev))

Space between "if" and "("

Thanks! Will correct it.

Quoted 18 lines
Quoted 6 lines
drivers/gpu/drm/i915/i915_gem.c +2 -0
+		dev_priv->num_fence_regs = gvt.dom0_fence_sz;
+

  if (intel_vgpu_active(dev))
  dev_priv->num_fence_regs = I915_READ(vgtif_reg(avail_rs.fence_num));

I'd like to see the above as "else if" construct just like you have below with the intel_vgt_balloon(). Could even do

	if (gvt_host)  {
		...
	} else if (vgpu_active) {
		...
	} else {
		per machine detection
	}
Quoted 1 line

Right?

That looks better!

Quoted 52 lines
Quoted 47 lines
drivers/gpu/drm/i915/i915_gem_gtt.c +0 -0
index 7377b67..0540de2 100644
@@ -2713,7 +2713,7 @@ static int i915_gem_setup_global_gtt(struct drm_device *dev,
  	i915_address_space_init(ggtt_vm, dev_priv);
  	ggtt_vm->total += PAGE_SIZE;

drivers/gpu/drm/i915/i915_gem_gtt.c +1 -1
-	if (intel_vgpu_active(dev)) {
+	if (intel_vgpu_active(dev) || intel_gvt_host_active(dev)) {

  ret = intel_vgt_balloon(dev);
  if (ret)
  return ret;

drivers/gpu/drm/i915/i915_gem_gtt.c +0 -0
@@ -2810,7 +2810,7 @@ void i915_global_gtt_cleanup(struct drm_device *dev)
  	}


  if (drm_mm_initialized(&vm->mm)) {

drivers/gpu/drm/i915/i915_gem_gtt.c +1 -1
-		if (intel_vgpu_active(dev))
+		if (intel_vgpu_active(dev) || intel_gvt_host_active(dev))

  intel_vgt_deballoon();

  drm_mm_takedown(&vm->mm);

drivers/gpu/drm/i915/i915_vgpu.c +0 -0
index dea7429..fbe6114 100644
@@ -188,10 +188,18 @@ int intel_vgt_balloon(struct drm_device *dev)
  	unsigned long unmappable_base, unmappable_size, unmappable_end;
  	int ret;

drivers/gpu/drm/i915/i915_vgpu.c +12 -4
-	mappable_base = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.base));
-	mappable_size = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.size));
-	unmappable_base = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.base));
-	unmappable_size = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.size));
+	if(intel_gvt_host_active(dev)) {
+		mappable_base = 0;
+		mappable_size = gvt.dom0_low_gm_sz << 20;
+		unmappable_base = dev_priv->gtt.mappable_end;
+		unmappable_size = gvt.dom0_high_gm_sz << 20;
+	} else if (intel_vgpu_active(dev)) {
+		mappable_base = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.base));
+		mappable_size = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.size));
+		unmappable_base = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.base));
+		unmappable_size = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.size));
+	} else
+		return -ENODEV;
Quoted 1 line

	} else {
		return -ENODEV;
	}
Quoted 1 line

This must use braces too, according to the Kernel Coding Style.

That's right. Thanks for pointing it out!

Quoted 19 lines
Quoted 10 lines

  mappable_end = mappable_base + mappable_size; unmappable_end = unmappable_base + unmappable_size;

drivers/gpu/drm/i915/i915_vgpu.h +0 -0
index 942490a..b8a49e6 100644
@@ -24,6 +24,7 @@
  #ifndef _I915_VGPU_H_
  #define _I915_VGPU_H_
Quoted 2 lines


+#include "gvt/params.h"

This is not a good way of including headers, as this header itself doesn't need it. Including unnecessary stuff in often included headers contributes to increasing project compile times and makes it harder to solve cross dependencies later. So put it into the implementation file that needs it.

Quoted 12 lines
Quoted 7 lines

Will delete. Thanks!

Quoted 5 lines

Regards, Joonas

Quoted 3 lines

 /* The MMIO offset of the shared info between guest and host emulator */
 #define VGT_PVINFO_PAGE 0x78000
 #define VGT_PVINFO_SIZE 0x1000

Signature · 3 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

maintainer
#17 lore raw

Re: [RFC 02/29] drm/i915: Introduce host graphics memory balloon for gvt

Quoted 19 lines · Zhiyuan Lv

Hi Joonas,

Quoted 7 lines · Joonas Lahtinen

Hi,

Quoted 4 lines · Zhiyuan Lv

Hi Joonas,

Thanks much for the review! We will incorporate those review comments!

Quoted 8 lines
Quoted 4 lines

Meanwhile, is it good enough to do the host ballooning like below? The pros is that it is very simple, especially consider that guest ballooning logic has been there. Thanks!

I think slicing it down like that is fine. Is there going to be real ballooning happening, as in, is the available memory dynamically going to change during runtime, or just be fixed size since the beginning?

Quoted 9 lines

In our current design, once the gvt is enabled, it is fixed size of graphics memory for host since beginning. The reserved memory is all for virtual machines.

We ever considered to support dynamically enabling/disabling gvt, that host i915 could boot without gvt enabled, and can use all the hardware resources. When there is need to creat VM, we can reload i915 driver to reserve the resources. Does that sound good? Thanks!

But is the VM graphics memory amount a fixed slice, or is there real ballooning happening in there (increasing the memory amount of the VM on demand when running)? Memory balloon ing might be the wrong term to use if a fixed slice of the graphics memory is allocated to VM when it boots.

Regards, Joonas

Quoted 200 lines

Regards,
-Zhiyuan

Quoted 197 lines

Regards, Joonas

Quoted 54 lines

Regards,
-Zhiyuan

Quoted 50 lines · Joonas Lahtinen

Hi,

Quoted 47 lines · Zhi Wang

From: Bing Niu <[email protected]>

This patch introduces host graphics memory ballon when GVT-g is enabled.

As under GVT-g, i915 only owned limited graphics resources, others are managed by GVT-g resource allocator and kept for other vGPUs.

For graphics memory space partition, a typical layout looks like:

+-------+-----------------------+------+-----------------------+
> * Host |   *GVT-g Resource     |* Host|   *GVT-g Resource     |
> Owned |   Allocator Managed   | Owned|   Allocator Managed   |
>       |                       |      |                       |
+---------------+-------+----------------------+-------+-------+
>       |       |       |       |      |       |       |       |
> i915  | vm 1  | vm 2  | vm 3  | i915 | vm 1  | vm 2  | vm 3  |
>       |       |       |       |      |       |       |       |
+-------+-------+-------+--------------+-------+-------+-------+
>           Aperture            |            Hidden            |
+-------------------------------+------------------------------+
>                       GGTT memory space                      |
+--------------------------------------------------------------+

Similar with fence registers partition:

 +------ +-----------------------+
 | * Host|    GVT-g Resource     |
 | Owned |   Allocator Managed   +
 0       |                       31
 +---------------+-------+-------+
 |       |       |       |       |
 | i915  | vm 1  | vm 2  | vm 3  |
 |       |       |       |       |
 +-------+-------+-------+-------+

i915 host will read the amount of allocated resources via GVT-g kernel parameters.

---
 drivers/gpu/drm/i915/gvt/params.h   |  3 +++ drivers/gpu/drm/i915/i915_gem.c     |  3 +++ drivers/gpu/drm/i915/i915_gem_gtt.c |  4 ++-- drivers/gpu/drm/i915/i915_vgpu.c    | 16 ++++++++++++---- drivers/gpu/drm/i915/i915_vgpu.h    |  1 +

 5 files changed, 21 insertions(+), 6 deletions(-)
Quoted 11 lines
Quoted 11 lines

drivers/gpu/drm/i915/gvt/params.h line 32

Quoted 11 lines
drivers/gpu/drm/i915/gvt/params.h +3 -0
index d2955b9..0656a98 100644
@@ -27,6 +27,9 @@
  struct gvt_kernel_params {
  	bool enable;
  	int debug;
+	int dom0_low_gm_sz;
+	int dom0_high_gm_sz;
+	int dom0_fence_sz;

Some comment on the unit would be nice here, as there is a shift with << 20 later on.

Is "unsigned long" type and count in pages not acceptable? I think that's the assumption one is going to make.

Quoted 122 lines
Quoted 6 lines
Quoted 6 lines
drivers/gpu/drm/i915/gvt/params.h +0 -0
  };


 extern struct gvt_kernel_params gvt;

drivers/gpu/drm/i915/i915_gem.c +0 -1
index 799a53a..e916e43 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
Quoted 8 lines

drivers/gpu/drm/i915/i915_gem.c i915_gem_load(struct drm_device *dev) · line 5083

Quoted 6 lines

+++ b/drivers/gpu/drm/i915/i915_gem.c

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -5080,6 +5080,9 @@ i915_gem_load(struct drm_device *dev)

  else
  dev_priv->num_fence_regs = 8;

drivers/gpu/drm/i915/i915_gem.c +1 -0
+	if(intel_gvt_host_active(dev))

Space between "if" and "("

Thanks! Will correct it.

Quoted 18 lines
Quoted 6 lines
drivers/gpu/drm/i915/i915_gem.c +2 -0
+		dev_priv->num_fence_regs = gvt.dom0_fence_sz;
+

  if (intel_vgpu_active(dev))
  dev_priv->num_fence_regs = I915_READ(vgtif_reg(avail_rs.fence_num));

I'd like to see the above as "else if" construct just like you have below with the intel_vgt_balloon(). Could even do

	if (gvt_host)  {
		...
	} else if (vgpu_active) {
		...
	} else {
		per machine detection
	}
Quoted 1 line

Right?

That looks better!

Quoted 52 lines
Quoted 47 lines
drivers/gpu/drm/i915/i915_gem_gtt.c +0 -0
index 7377b67..0540de2 100644
@@ -2713,7 +2713,7 @@ static int i915_gem_setup_global_gtt(struct drm_device *dev,
  	i915_address_space_init(ggtt_vm, dev_priv);
  	ggtt_vm->total += PAGE_SIZE;

drivers/gpu/drm/i915/i915_gem_gtt.c +1 -1
-	if (intel_vgpu_active(dev)) {
+	if (intel_vgpu_active(dev) || intel_gvt_host_active(dev)) {

  ret = intel_vgt_balloon(dev);
  if (ret)
  return ret;

drivers/gpu/drm/i915/i915_gem_gtt.c +0 -0
@@ -2810,7 +2810,7 @@ void i915_global_gtt_cleanup(struct drm_device *dev)
  	}


  if (drm_mm_initialized(&vm->mm)) {

drivers/gpu/drm/i915/i915_gem_gtt.c +1 -1
-		if (intel_vgpu_active(dev))
+		if (intel_vgpu_active(dev) || intel_gvt_host_active(dev))

  intel_vgt_deballoon();

  drm_mm_takedown(&vm->mm);

drivers/gpu/drm/i915/i915_vgpu.c +0 -0
index dea7429..fbe6114 100644
@@ -188,10 +188,18 @@ int intel_vgt_balloon(struct drm_device *dev)
  	unsigned long unmappable_base, unmappable_size, unmappable_end;
  	int ret;

drivers/gpu/drm/i915/i915_vgpu.c +12 -4
-	mappable_base = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.base));
-	mappable_size = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.size));
-	unmappable_base = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.base));
-	unmappable_size = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.size));
+	if(intel_gvt_host_active(dev)) {
+		mappable_base = 0;
+		mappable_size = gvt.dom0_low_gm_sz << 20;
+		unmappable_base = dev_priv->gtt.mappable_end;
+		unmappable_size = gvt.dom0_high_gm_sz << 20;
+	} else if (intel_vgpu_active(dev)) {
+		mappable_base = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.base));
+		mappable_size = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.size));
+		unmappable_base = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.base));
+		unmappable_size = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.size));
+	} else
+		return -ENODEV;
Quoted 1 line

	} else {
		return -ENODEV;
	}
Quoted 1 line

This must use braces too, according to the Kernel Coding Style.

That's right. Thanks for pointing it out!

Quoted 10 lines
Quoted 10 lines

  mappable_end = mappable_base + mappable_size; unmappable_end = unmappable_base + unmappable_size;

drivers/gpu/drm/i915/i915_vgpu.h +0 -0
index 942490a..b8a49e6 100644
@@ -24,6 +24,7 @@
  #ifndef _I915_VGPU_H_
  #define _I915_VGPU_H_
Quoted 9 lines
Quoted 2 lines


+#include "gvt/params.h"

This is not a good way of including headers, as this header itself doesn't need it. Including unnecessary stuff in often included headers contributes to increasing project compile times and makes it harder to solve cross dependencies later. So put it into the implementation file that needs it.

Will delete. Thanks!

Quoted 5 lines

Regards, Joonas

Quoted 3 lines

 /* The MMIO offset of the shared info between guest and host emulator */
 #define VGT_PVINFO_PAGE 0x78000
 #define VGT_PVINFO_SIZE 0x1000

Signature · 7 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#18 lore raw

Re: [RFC 02/29] drm/i915: Introduce host graphics memory balloon for gvt

In reply to Zhi Wang
Quoted 25 lines · Zhi Wang

From: Bing Niu <[email protected]>

This patch introduces host graphics memory ballon when GVT-g is enabled.

As under GVT-g, i915 only owned limited graphics resources, others are managed by GVT-g resource allocator and kept for other vGPUs.

For graphics memory space partition, a typical layout looks like:

+-------+-----------------------+------+-----------------------+
|* Host |   *GVT-g Resource     |* Host|   *GVT-g Resource     |
| Owned |   Allocator Managed   | Owned|   Allocator Managed   |
|       |                       |      |                       |
+---------------+-------+----------------------+-------+-------+
|       |       |       |       |      |       |       |       |
| i915  | vm 1  | vm 2  | vm 3  | i915 | vm 1  | vm 2  | vm 3  |
|       |       |       |       |      |       |       |       |
+-------+-------+-------+--------------+-------+-------+-------+
|           Aperture            |            Hidden            |
+-------------------------------+------------------------------+
|                       GGTT memory space                      |
+--------------------------------------------------------------+

Similar with fence registers partition:

Quoted 9 lines
 +------ +-----------------------+
 | * Host|    GVT-g Resource     |
 | Owned |   Allocator Managed   +
 0       |                       31
 +---------------+-------+-------+
 |       |       |       |       |
 | i915  | vm 1  | vm 2  | vm 3  |
 |       |       |       |       |
 +-------+-------+-------+-------+

This kind of pretty graphs should be in the kerneldoc - with the asciidoc support we have already in drm-intel-nightly (and hopefully soonish in upstream). See http://blog.ffwll.ch/2016/01/better-markup-for-kernel-gpu-docbook.html

Cheers, Daniel

Quoted 105 lines

i915 host will read the amount of allocated resources via GVT-g kernel parameters.

---
 drivers/gpu/drm/i915/gvt/params.h   |  3 +++
 drivers/gpu/drm/i915/i915_gem.c     |  3 +++
 drivers/gpu/drm/i915/i915_gem_gtt.c |  4 ++--
 drivers/gpu/drm/i915/i915_vgpu.c    | 16 ++++++++++++----
 drivers/gpu/drm/i915/i915_vgpu.h    |  1 +
 5 files changed, 21 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/gvt/params.h +3 -0
index d2955b9..0656a98 100644
@@ -27,6 +27,9 @@
 struct gvt_kernel_params {
 	bool enable;
 	int debug;
+	int dom0_low_gm_sz;
+	int dom0_high_gm_sz;
+	int dom0_fence_sz;
 };
 
 extern struct gvt_kernel_params gvt;
drivers/gpu/drm/i915/i915_gem.c +3 -0
index 799a53a..e916e43 100644
@@ -5080,6 +5080,9 @@ i915_gem_load(struct drm_device *dev)
 	else
 		dev_priv->num_fence_regs = 8;
 
+	if(intel_gvt_host_active(dev))
+		dev_priv->num_fence_regs = gvt.dom0_fence_sz;
+
 	if (intel_vgpu_active(dev))
 		dev_priv->num_fence_regs =
 				I915_READ(vgtif_reg(avail_rs.fence_num));
drivers/gpu/drm/i915/i915_gem_gtt.c +2 -2
index 7377b67..0540de2 100644
@@ -2713,7 +2713,7 @@ static int i915_gem_setup_global_gtt(struct drm_device *dev,
 	i915_address_space_init(ggtt_vm, dev_priv);
 	ggtt_vm->total += PAGE_SIZE;
 
-	if (intel_vgpu_active(dev)) {
+	if (intel_vgpu_active(dev) || intel_gvt_host_active(dev)) {
 		ret = intel_vgt_balloon(dev);
 		if (ret)
 			return ret;
@@ -2810,7 +2810,7 @@ void i915_global_gtt_cleanup(struct drm_device *dev)
 	}
 
 	if (drm_mm_initialized(&vm->mm)) {
-		if (intel_vgpu_active(dev))
+		if (intel_vgpu_active(dev) || intel_gvt_host_active(dev))
 			intel_vgt_deballoon();
 
 		drm_mm_takedown(&vm->mm);
drivers/gpu/drm/i915/i915_vgpu.c +12 -4
index dea7429..fbe6114 100644
@@ -188,10 +188,18 @@ int intel_vgt_balloon(struct drm_device *dev)
 	unsigned long unmappable_base, unmappable_size, unmappable_end;
 	int ret;
 
-	mappable_base = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.base));
-	mappable_size = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.size));
-	unmappable_base = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.base));
-	unmappable_size = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.size));
+	if(intel_gvt_host_active(dev)) {
+		mappable_base = 0;
+		mappable_size = gvt.dom0_low_gm_sz << 20;
+		unmappable_base = dev_priv->gtt.mappable_end;
+		unmappable_size = gvt.dom0_high_gm_sz << 20;
+	} else if (intel_vgpu_active(dev)) {
+		mappable_base = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.base));
+		mappable_size = I915_READ(vgtif_reg(avail_rs.mappable_gmadr.size));
+		unmappable_base = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.base));
+		unmappable_size = I915_READ(vgtif_reg(avail_rs.nonmappable_gmadr.size));
+	} else
+		return -ENODEV;
 
 	mappable_end = mappable_base + mappable_size;
 	unmappable_end = unmappable_base + unmappable_size;
drivers/gpu/drm/i915/i915_vgpu.h +1 -0
index 942490a..b8a49e6 100644
@@ -24,6 +24,7 @@
 #ifndef _I915_VGPU_H_
 #define _I915_VGPU_H_
 
+#include "gvt/params.h"
 /* The MMIO offset of the shared info between guest and host emulator */
 #define VGT_PVINFO_PAGE	0x78000
 #define VGT_PVINFO_SIZE	0x1000
Signature · 1 line
1.9.1
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#19 lore raw

RFC 3/29 drm/i915: Introduce GVT context creation API

In reply to Zhi Wang

GVT workload scheduler needs special host LRC contexts, the so called "shadow LRC context" to submit guest workload to host i915. During the guest workload submission, GVT fills the shadow LRC context with the content of guest LRC context: engine context is copied without changes, ring context is mostly owned by host i915, except the PPGTT root pointers, which will be filled with the shadow PPGTT page table root pointers managed by GVT-g.

The GVT-g workload scheduler flow:

         +-----------+                   +-----------+
         | GVT Guest |                   | GVT Guest |
         +-+-----^---+                   +-+-----^---+
           |     |                         |     |
           |     | GVT-g                   |     | GVT-g
vELSP write|     | emulates     vELSP write|     | emulates
           |     | Execlist/CSB            |     | Execlist/CSB
           |     | Status                  |     | Status
           |     |                         |     |
    +------v-----+-------------------------v-----+---------+
    |           GVT Virtual Execlist Submission            |
    +------+-------------------------------+---------------+
           |                               |
           | Per-VM/Ring Workoad Q         | Per-VM/Ring Workload Q
   +---------------------+--+      +------------------------+
       +---v--------+    ^             +---v--------+
       |GVT Workload|... |             |GVT Workload|...
       +------------+    |             +------------+
                         |
                         | Pick Workload from Q
    +--------------------+---------------------------------+
    |                GVT Workload Scheduler                |
    +--------------------+---------------------------------+
                         |         * Shadow guest LRC context
                  +------v------+  * Shadow guest ring buffer
                  | GVT Context |  * Scan/Patch guest RB instructions
                  +------+------+
                         |

v
Host i915 GEM Submission

Guest ELSP submission will be wrapped into a GVT workload data structure. When a guest is scheduled, workload scheduler picks the GVT workload from the per-VM/ring Q, then prepare to dispatch it through host i915 GEM submission.

The GVT workload lifecycle:

- Workload scheduler populates the GVT LRC context with the content of guest LRC context
- Workload scheduler populates the GVT ring buffer with the instructions from guest ring buffer
- Workload scheduler populates the PDPs in the GVT LRC context with shadow PPGTT PDPs
- Workload scheduler submits this context through i915 GEM submission interface
- Once the i915 gem request is finished, GVT-g waits until the GVT LRC context is idle.
- i915 LRC routines notifies workload scheduler the LRC context is idle.
- Workload scheduler updates the guest LRC context with the content of GVT LRC context
- Emulate CSB and context switch interrupt to guest

---
 drivers/gpu/drm/i915/i915_drv.h         | 20 +++++++++++++++++
 drivers/gpu/drm/i915/i915_gem_context.c | 38 +++++++++++++++++++++++++++++++++
 2 files changed, 58 insertions(+)
drivers/gpu/drm/i915/i915_drv.h +20 -0
index db3c79b..fc5ddee 100644
@@ -890,6 +890,24 @@ struct intel_context {
 	} engine[I915_NUM_RINGS];
 
 	struct list_head link;
+
+	/* Is a GVT context ? */
+	bool gvt_context;
+	/* Used by GVT workload scheduler. */
+	void *gvt_context_private_data[I915_NUM_RINGS];
+	/*
+	 * As GVT context may comes from different guest,
+	 * the addressing mode may be different
+	 */
+	u32 gvt_context_addressing_mode[I915_NUM_RINGS];
+	/*
+	 * Called when GVT context is scheduled-in
+	 */
+	void (*gvt_context_schedule_in)(void *data);
+	/*
+	 * Called when GVT context is scheduled-out
+	 */
+	void (*gvt_context_schedule_out)(void *data);
 };
 
 enum fb_op_origin {
@@ -2866,6 +2884,8 @@ struct drm_i915_gem_object *i915_gem_object_create_from_data(
 void i915_gem_free_object(struct drm_gem_object *obj);
 void i915_gem_vma_destroy(struct i915_vma *vma);
 
+struct intel_context * i915_gem_create_gvt_context(struct drm_device *dev);
+
 /* Flags used by pin/bind&friends. */
 #define PIN_MAPPABLE	(1<<0)
 #define PIN_NONBLOCK	(1<<1)
drivers/gpu/drm/i915/i915_gem_context.c +38 -0
index 6a4f64b..410540a 100644
@@ -260,6 +260,44 @@ err_out:
 	return ERR_PTR(ret);
 }
 
+struct intel_context *
+i915_gem_create_gvt_context(struct drm_device *dev)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct intel_context *ctx;
+	int ret;
+	int i;
+
+	mutex_lock(&dev->struct_mutex);
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (ctx == NULL)
+		return ERR_PTR(-ENOMEM);
+
+	kref_init(&ctx->ref);
+	list_add_tail(&ctx->link, &dev_priv->context_list);
+
+	ctx->i915 = dev_priv;
+	ctx->file_priv = NULL;
+	ctx->user_handle = -1;
+	ctx->remap_slice = (1 << NUM_L3_SLICES(dev)) - 1;
+	ctx->hang_stats.ban_period_seconds = DRM_I915_CTX_BAN_PERIOD;
+
+	ctx->gvt_context = true;
+
+	for (i = 0; i < I915_NUM_RINGS; i++) {
+		ret = intel_lr_context_deferred_alloc(ctx, &dev_priv->ring[i]);
+		if (ret) {
+			i915_gem_context_unreference(ctx);
+			ctx = NULL;
+			goto out;
+		}
+	}
+out:
+	mutex_unlock(&dev->struct_mutex);
+	return ctx;
+}
+
 /**
  * The default context needs to exist per ring that uses contexts. It stores the
  * context state of the GPU for applications that don't utilize HW contexts, as
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#20 lore raw

RFC 4/29 drm/i915: Ondemand populate context addressing mode bit

In reply to Zhi Wang

As GVT-g workload scheduler may submit workloads from different VMs to i915, some VM may use 32bit PPGTT addressing mode, while some VMs may use 48bit addressing mode, the context addressing mode bit in the context descriptor has to be aligned with guest workload. If the to-be-submitted context is a GVT context.

---
 drivers/gpu/drm/i915/intel_lrc.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/intel_lrc.c +9 -1
index da97bc5..48e8ca2 100644
@@ -302,7 +302,8 @@ logical_ring_init_platform_invariants(struct intel_engine_cs *ring)
  * which remains valid until the context is unpinned.
  *
  * This is what a descriptor looks like, from LSB to MSB:
- *    bits 0-11:    flags, GEN8_CTX_* (cached in ctx_desc_template)
+ *    bits 0-2,3-11: flags, GEN8_CTX_* (cached in ctx_desc_template)
+ *    bits 3:        Context addressing mode
  *    bits 12-31:    LRCA, GTT address of (the HWSP of) this context
  *    bits 32-51:    ctx ID, a globally unique tag (the LRCA again!)
  *    bits 52-63:    reserved, may encode the engine ID (for GuC)
@@ -317,6 +318,13 @@ intel_lr_context_descriptor_update(struct intel_context *ctx,
 	       LRC_PPHWSP_PN * PAGE_SIZE;
 
 	desc = ring->ctx_desc_template;			   /* bits  0-11 */
+
+	if (!ctx->gvt_context)
+		desc |= GEN8_CTX_ADDRESSING_MODE(dev) <<   /* bit 3 */
+			GEN8_CTX_ADDRESSING_MODE_SHIFT;
+	else
+		desc |= ctx->gvt_context_addressing_mode[ring->id];
+
 	desc |= lrca;					   /* bits 12-31 */
 	desc |= (lrca >> PAGE_SHIFT) << GEN8_CTX_ID_SHIFT; /* bits 32-51 */
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#21 lore raw

RFC 5/29 drm/i915: Do not populate PPGTT root pointers for GVT context

In reply to Zhi Wang

As the GVT context will use the shadow PPGTT page table generated by GVT itself from guest PPGTT page table, the GVT context doesn't need a host PPGTT page table, so host i915 doesn't need to populate PPGTT root pointers for it as well.

---
 drivers/gpu/drm/i915/intel_lrc.c | 34 ++++++++++++++++++----------------
 1 file changed, 18 insertions(+), 16 deletions(-)
drivers/gpu/drm/i915/intel_lrc.c +18 -16
index 48e8ca2..631ed96 100644
@@ -2399,22 +2399,24 @@ populate_lr_context(struct intel_context *ctx, struct drm_i915_gem_object *ctx_o
 	ASSIGN_CTX_REG(reg_state, CTX_PDP0_UDW, GEN8_RING_PDP_UDW(ring, 0), 0);
 	ASSIGN_CTX_REG(reg_state, CTX_PDP0_LDW, GEN8_RING_PDP_LDW(ring, 0), 0);
 
-	if (USES_FULL_48BIT_PPGTT(ppgtt->base.dev)) {
-		/* 64b PPGTT (48bit canonical)
-		 * PDP0_DESCRIPTOR contains the base address to PML4 and
-		 * other PDP Descriptors are ignored.
-		 */
-		ASSIGN_CTX_PML4(ppgtt, reg_state);
-	} else {
-		/* 32b PPGTT
-		 * PDP*_DESCRIPTOR contains the base address of space supported.
-		 * With dynamic page allocation, PDPs may not be allocated at
-		 * this point. Point the unallocated PDPs to the scratch page
-		 */
-		ASSIGN_CTX_PDP(ppgtt, reg_state, 3);
-		ASSIGN_CTX_PDP(ppgtt, reg_state, 2);
-		ASSIGN_CTX_PDP(ppgtt, reg_state, 1);
-		ASSIGN_CTX_PDP(ppgtt, reg_state, 0);
+	if (!ctx->gvt_context) {
+		if (USES_FULL_48BIT_PPGTT(ppgtt->base.dev)) {
+			/* 64b PPGTT (48bit canonical)
+			 * PDP0_DESCRIPTOR contains the base address to PML4 and
+			 * other PDP Descriptors are ignored.
+			 */
+			ASSIGN_CTX_PML4(ppgtt, reg_state);
+		} else {
+			/* 32b PPGTT
+			 * PDP*_DESCRIPTOR contains the base address of space supported.
+			 * With dynamic page allocation, PDPs may not be allocated at
+			 * this point. Point the unallocated PDPs to the scratch page
+			 */
+			ASSIGN_CTX_PDP(ppgtt, reg_state, 3);
+			ASSIGN_CTX_PDP(ppgtt, reg_state, 2);
+			ASSIGN_CTX_PDP(ppgtt, reg_state, 1);
+			ASSIGN_CTX_PDP(ppgtt, reg_state, 0);
+		}
 	}
 
 	if (ring->id == RCS) {
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#22 lore raw

RFC 6/29 drm/i915: Do not initialize the engine state of GVT context

In reply to Zhi Wang

As the guest context will be copied into GVT context, during the context shadow. It's not necessary for host i915 to initialize the engine state of GVT context.

---
 drivers/gpu/drm/i915/intel_lrc.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/intel_lrc.c +3 -1
index 631ed96..830133d 100644
@@ -2569,7 +2569,9 @@ int intel_lr_context_deferred_alloc(struct intel_context *ctx,
 	ctx->engine[ring->id].ringbuf = ringbuf;
 	ctx->engine[ring->id].state = ctx_obj;
 
-	if (ctx != ctx->i915->kernel_context && ring->init_context) {
+	if (ctx != ctx->i915->kernel_context &&
+		!ctx->gvt_context && ring->init_context) {
+
 		struct drm_i915_gem_request *req;
 
 		req = i915_gem_request_alloc(ring, ctx);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#23 lore raw

RFC 7/29 drm/i915: GVT context scheduling

In reply to Zhi Wang

Due to render mmio switch, GVT context cannot be scheduled with the host i915 context together in a ELSP combo. Currently we will only allow the a single submission if host i915 will schedule a GVT context.

At the time of schedule-in/out a GVT context, host i915 has to notify GVT-g the status of GVT context has been changed. so that GVT-g can copy the content of a GVT context back into the guest execlist context.

---
 drivers/gpu/drm/i915/intel_lrc.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/intel_lrc.c +15 -1
index 830133d..42638e0 100644
@@ -449,6 +449,9 @@ static void execlists_context_unqueue(struct intel_engine_cs *ring)
 				 execlist_link) {
 		if (!req0) {
 			req0 = cursor;
+			/* single submission for gvt context */
+			if (req0->ctx->gvt_context)
+				break;
 		} else if (req0->ctx == cursor->ctx) {
 			/* Same ctx: ignore first request, as second request
 			 * will update tail past first request's workload */
@@ -457,7 +460,9 @@ static void execlists_context_unqueue(struct intel_engine_cs *ring)
 				       &ring->execlist_retired_req_list);
 			req0 = cursor;
 		} else {
-			req1 = cursor;
+			if (!cursor->ctx->gvt_context)
+				req1 = cursor;
+
 			break;
 		}
 	}
@@ -484,6 +489,11 @@ static void execlists_context_unqueue(struct intel_engine_cs *ring)
 
 	WARN_ON(req1 && req1->elsp_submitted);
 
+	if (req0->ctx->gvt_context) {
+		struct intel_context *ctx = req0->ctx;
+		ctx->gvt_context_schedule_in(ctx->gvt_context_private_data[ring->id]);
+	}
+
 	execlists_submit_requests(req0, req1);
 }
 
@@ -504,6 +514,10 @@ static bool execlists_check_remove_request(struct intel_engine_cs *ring,
 			     "Never submitted head request\n");
 
 			if (--head_req->elsp_submitted <= 0) {
+				if (head_req->ctx->gvt_context) {
+					struct intel_context *ctx = head_req->ctx;
+					ctx->gvt_context_schedule_out(ctx->gvt_context_private_data[ring->id]);
+				}
 				list_move_tail(&head_req->execlist_link,
 					       &ring->execlist_retired_req_list);
 				return true;
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#24 lore raw

RFC 8/29 drm/i915: Support vGPU guest framebuffer GEM object

In reply to Zhi Wang

From: Zhiyuan Lv <[email protected]>

In the compositor mode of display, dom0/host needs to get the guest framebuffer to do more rendering, so that the guest VM's screen can show up in more fancy way, e.g., in an X window of dom0/host.

In order to do that, a new gem object type "gvtbuffer" is introduced to i915. Different from normal gem object in i915, gvtbuffer does not have its own backing storage. Instead, it borrows the page frames of guest VM's framebuffer as its own backing storage.

From high level, it works this way:
a) gvt notifies kernel/userspace the guest OS page flip by monitoring the related guest MMIO changes and commands.
b) user space issue IOCTL to create gvtbuffer gem object.
c) kernel creates the gem object, and record the guest FB base address (gfx address) from MMIO.
d) When needed, the gvtbuffer will be bound to graphics memory, and be used as normal gem object for rendering.

Guest framebuffer must be inside GGTT, whereas the gvtbuffer can be in either GGTT or PPGTT, depending on the requirement of the rendering.

Since the gvtbuffer corresponds to the guest framebuffer, which is from guest physical memory, we may not be able to get "page struct" for them. But i915 gem framework has had similar cases. A gem object can have stolen memory as its backing storage. In such case, the backing storage does not have "page struct" as well, and i915 has handled the case in the framework well.

This patch was originally from [email protected], and pretty some changes were made since then.

---
 drivers/gpu/drm/i915/Makefile             |   1 +
 drivers/gpu/drm/i915/i915_dma.c           |   1 +
 drivers/gpu/drm/i915/i915_drv.h           |   3 +
 drivers/gpu/drm/i915/i915_gem_gvtbuffer.c | 247 ++++++++++++++++++++++++++++++
 include/uapi/drm/i915_drm.h               |  38 +++++
 5 files changed, 290 insertions(+)
 create mode 100644 drivers/gpu/drm/i915/i915_gem_gvtbuffer.c
drivers/gpu/drm/i915/Makefile +1 -0
index d4df410..c4a6615 100644
@@ -33,6 +33,7 @@ i915-y += i915_cmd_parser.o \
 	  i915_gem_stolen.o \
 	  i915_gem_tiling.o \
 	  i915_gem_userptr.o \
+	  i915_gem_gvtbuffer.o \
 	  i915_gpu_error.o \
 	  i915_trace_points.o \
 	  intel_lrc.o \
drivers/gpu/drm/i915/i915_dma.c +1 -0
index eca8e50..f150602 100644
@@ -1351,6 +1351,7 @@ const struct drm_ioctl_desc i915_ioctls[] = {
 	DRM_IOCTL_DEF_DRV(I915_GEM_USERPTR, i915_gem_userptr_ioctl, DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_GETPARAM, i915_gem_context_getparam_ioctl, DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_SETPARAM, i915_gem_context_setparam_ioctl, DRM_RENDER_ALLOW),
+	DRM_IOCTL_DEF_DRV(I915_GEM_GVTBUFFER, i915_gem_gvtbuffer_ioctl, DRM_RENDER_ALLOW),
 };
 
 int i915_max_ioctl = ARRAY_SIZE(i915_ioctls);
drivers/gpu/drm/i915/i915_drv.h +3 -0
index fc5ddee..d387754 100644
@@ -3238,6 +3238,9 @@ int i915_gem_context_getparam_ioctl(struct drm_device *dev, void *data,
 int i915_gem_context_setparam_ioctl(struct drm_device *dev, void *data,
 				    struct drm_file *file_priv);
 
+int i915_gem_gvtbuffer_ioctl(struct drm_device *dev, void *data,
+			     struct drm_file *file);
+
 /* i915_gem_evict.c */
 int __must_check i915_gem_evict_something(struct drm_device *dev,
 					  struct i915_address_space *vm,
drivers/gpu/drm/i915/i915_gem_gvtbuffer.c new file +247 -0

247 lines not on this page · raw message

include/uapi/drm/i915_drm.h +38 -0
index 6a19371..5ba115a 100644
@@ -230,6 +230,7 @@ typedef struct _drm_i915_sarea {
 #define DRM_I915_GEM_USERPTR		0x33
 #define DRM_I915_GEM_CONTEXT_GETPARAM	0x34
 #define DRM_I915_GEM_CONTEXT_SETPARAM	0x35
+#define DRM_I915_GEM_GVTBUFFER          0x36
 
 #define DRM_IOCTL_I915_INIT		DRM_IOW( DRM_COMMAND_BASE + DRM_I915_INIT, drm_i915_init_t)
 #define DRM_IOCTL_I915_FLUSH		DRM_IO ( DRM_COMMAND_BASE + DRM_I915_FLUSH)
@@ -283,6 +284,7 @@ typedef struct _drm_i915_sarea {
 #define DRM_IOCTL_I915_GEM_USERPTR			DRM_IOWR (DRM_COMMAND_BASE + DRM_I915_GEM_USERPTR, struct drm_i915_gem_userptr)
 #define DRM_IOCTL_I915_GEM_CONTEXT_GETPARAM	DRM_IOWR (DRM_COMMAND_BASE + DRM_I915_GEM_CONTEXT_GETPARAM, struct drm_i915_gem_context_param)
 #define DRM_IOCTL_I915_GEM_CONTEXT_SETPARAM	DRM_IOWR (DRM_COMMAND_BASE + DRM_I915_GEM_CONTEXT_SETPARAM, struct drm_i915_gem_context_param)
+#define DRM_IOCTL_I915_GEM_GVTBUFFER		DRM_IOWR(DRM_COMMAND_BASE + DRM_I915_GEM_GVTBUFFER, struct drm_i915_gem_gvtbuffer)
 
 /* Allow drivers to submit batchbuffers directly to hardware, relying
  * on the security mechanisms provided by hardware.
@@ -1168,4 +1170,40 @@ struct drm_i915_gem_context_param {
 	__u64 value;
 };
 
+struct drm_i915_gem_gvtbuffer {
+        __u32 vmid;
+	__u32 plane_id;
+#define I915_GVT_PLANE_PRIMARY 1
+#define I915_GVT_PLANE_SPRITE 2
+#define I915_GVT_PLANE_CURSOR 3
+	__u32 pipe_id;
+	__u32 phys_pipe_id;
+	__u8  enabled;
+	__u8  tiled;
+	__u32 bpp;
+	__u32 hw_format;
+	__u32 drm_format;
+	__u32 start;
+	__u32 x_pos;
+	__u32 y_pos;
+	__u32 x_offset;
+	__u32 y_offset;
+	__u32 size;
+	__u32 width;
+	__u32 height;
+	__u32 stride;
+	__u64 user_ptr;
+	__u32 user_size;
+	__u32 flags;
+#define I915_GVTBUFFER_READ_ONLY (1<<0)
+#define I915_GVTBUFFER_QUERY_ONLY (1<<1)
+#define I915_GVTBUFFER_UNSYNCHRONIZED 0x80000000
+	/**
+	 * Returned handle for the object.
+	 *
+	 * Object handles are nonzero.
+	 */
+	__u32 handle;
+};
+
 #endif /* _UAPI_I915_DRM_H_ */
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#25 lore raw

RFC 9/29 drm/i915: gvt: Resource allocator

In reply to Zhi Wang

From: Yulei Zhang <[email protected]>

This patch introduces the GVT-g resource allocator. Under virtualization environment, GGTT and fences are partitioned. GGTT memory space and fences for i915 are limited. Only a part of GGTT memory space and fences is owned by host. The left resources are mananged by GVT-g resource allocators.

---
 drivers/gpu/drm/i915/gvt/Makefile      |   2 +-
 drivers/gpu/drm/i915/gvt/aperture_gm.c | 225 +++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/gvt/gvt.c         |   3 +
 drivers/gpu/drm/i915/gvt/gvt.h         | 105 +++++++++++++++
 drivers/gpu/drm/i915/gvt/params.c      |  12 ++
 drivers/gpu/drm/i915/gvt/params.h      |   8 ++
 6 files changed, 354 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/drm/i915/gvt/aperture_gm.c
drivers/gpu/drm/i915/gvt/Makefile +1 -1
index 6935b78..6655929 100644
@@ -1,4 +1,4 @@
-GVT_SOURCE := gvt.o params.o fb_decoder.o
+GVT_SOURCE := gvt.o params.o aperture_gm.o fb_decoder.o
 
 ccflags-y			+= -I$(src) -I$(src)/.. -Wall -Werror -Wno-unused-function
 i915_gvt-y			:= $(GVT_SOURCE)
drivers/gpu/drm/i915/gvt/aperture_gm.c new file +225 -0

225 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/gvt.c +3 -0
index 041d10f..f31e9f7 100644
@@ -234,6 +234,7 @@ static void clean_pgt_device(struct pgt_device *pdev)
 {
 	clean_service_thread(pdev);
 	clean_initial_mmio_state(pdev);
+	gvt_clean_resource_allocator(pdev);
 }
 
 static bool init_pgt_device(struct pgt_device *pdev, struct drm_i915_private *dev_priv)
@@ -246,6 +247,8 @@ static bool init_pgt_device(struct pgt_device *pdev, struct drm_i915_private *de
 	if (!init_initial_mmio_state(pdev))
 		goto err;
 
+	gvt_init_resource_allocator(pdev);
+
 	if (!init_service_thread(pdev))
 		goto err;
 
drivers/gpu/drm/i915/gvt/gvt.h +105 -0

130 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/params.c +12 -0
index dfc33c3..6cd324c 100644
@@ -26,4 +26,16 @@
 struct gvt_kernel_params gvt = {
 	.enable = true,
 	.debug = 0,
+	.dom0_low_gm_sz = 96,
+	.dom0_high_gm_sz = 384,
+	.dom0_fence_sz = 4,
 };
+
+module_param_named(dom0_low_gm_sz, gvt.dom0_low_gm_sz, int, 0600);
+MODULE_PARM_DESC(dom0_low_gm_sz, "Amount of aperture size of DOM0");
+
+module_param_named(dom0_high_gm_sz, gvt.dom0_high_gm_sz, int, 0600);
+MODULE_PARM_DESC(dom0_high_gm_sz, "Amount of high memory size of DOM0");
+
+module_param_named(dom0_fence_sz, gvt.dom0_fence_sz, int, 0600);
+MODULE_PARM_DESC(dom0_fence_sz, "Amount of fence size of DOM0");
drivers/gpu/drm/i915/gvt/params.h +8 -0
index 0656a98..0507870 100644
@@ -34,4 +34,12 @@ struct gvt_kernel_params {
 
 extern struct gvt_kernel_params gvt;
 
+struct gvt_instance_info {
+	u32 domid;
+	u32 low_gm_sz;
+	u32 high_gm_sz;
+	u32 fence_sz;
+	s32 primary;
+};
+
 #endif
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#26 lore raw

RFC 10/29 drm/i915: gvt: Basic mmio emulation state

In reply to Zhi Wang

This patch introduces the basic prototypes and definitions for GVT guest MMIO emulation.

- Save an original MMIO snaphot when system boot up, which will be used as the initial guest MMIO content.

- Introduce a framework to tracked all kinds of GEN MMIO registers, each tracked MMIO register may have a handler to emulate the HW registers change.

---
 drivers/gpu/drm/i915/gvt/Makefile   |   2 +-
 drivers/gpu/drm/i915/gvt/gvt.c      |   9 ++
 drivers/gpu/drm/i915/gvt/gvt.h      | 127 ++++++++++++++++
 drivers/gpu/drm/i915/gvt/handlers.c |  47 ++++++
 drivers/gpu/drm/i915/gvt/mmio.c     | 290 ++++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/gvt/mmio.h     |  87 +++++++++++
 drivers/gpu/drm/i915/gvt/reg.h      |   4 +
 7 files changed, 565 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/drm/i915/gvt/handlers.c
 create mode 100644 drivers/gpu/drm/i915/gvt/mmio.c
 create mode 100644 drivers/gpu/drm/i915/gvt/mmio.h
drivers/gpu/drm/i915/gvt/Makefile +1 -1
index 6655929..5d28ed1 100644
@@ -1,4 +1,4 @@
-GVT_SOURCE := gvt.o params.o aperture_gm.o fb_decoder.o
+GVT_SOURCE := gvt.o params.o aperture_gm.o mmio.o handlers.o fb_decoder.o
 
 ccflags-y			+= -I$(src) -I$(src)/.. -Wall -Werror -Wno-unused-function
 i915_gvt-y			:= $(GVT_SOURCE)
drivers/gpu/drm/i915/gvt/gvt.c +9 -0
index f31e9f7..f5066bc 100644
@@ -136,6 +136,8 @@ static void init_initial_cfg_space_state(struct pgt_device *pdev)
 
 static void clean_initial_mmio_state(struct pgt_device *pdev)
 {
+	gvt_clean_initial_mmio_state(pdev);
+
 	if (pdev->gttmmio_va) {
 		iounmap(pdev->gttmmio_va);
 		pdev->gttmmio_va = NULL;
@@ -179,6 +181,9 @@ static bool init_initial_mmio_state(struct pgt_device *pdev)
 	gvt_info("gttmmio_va: %p", pdev->gttmmio_va);
 	gvt_info("gmadr_va: %p", pdev->gmadr_va);
 
+	if (!gvt_setup_initial_mmio_state(pdev))
+		goto err;
+
 	return true;
 err:
 	clean_initial_mmio_state(pdev);
@@ -233,6 +238,7 @@ static bool init_service_thread(struct pgt_device *pdev)
 static void clean_pgt_device(struct pgt_device *pdev)
 {
 	clean_service_thread(pdev);
+	gvt_clean_mmio_emulation_state(pdev);
 	clean_initial_mmio_state(pdev);
 	gvt_clean_resource_allocator(pdev);
 }
@@ -249,6 +255,9 @@ static bool init_pgt_device(struct pgt_device *pdev, struct drm_i915_private *de
 
 	gvt_init_resource_allocator(pdev);
 
+	if (!gvt_setup_mmio_emulation_state(pdev))
+		goto err;
+
 	if (!init_service_thread(pdev))
 		goto err;
 
drivers/gpu/drm/i915/gvt/gvt.h +127 -0

156 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/handlers.c new file +47 -0

47 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/mmio.c new file +290 -0

290 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/mmio.h new file +87 -0

87 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/reg.h +4 -0
index d363b74..5682e1c 100644
@@ -31,4 +31,8 @@
 #define GVT_REG_CFG_SPACE_BAR1	0x18
 #define GVT_REG_CFG_SPACE_BAR2	0x20
 
+#define _PCH_GMBUS2			0xc5108
+
+#define _GEN6_GDRST			0x941c
+
 #endif
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#27 lore raw

RFC 11/29 drm/i915: gvt: update PVINFO page definition in i915_vgpu.h

In reply to Zhi Wang
---
 drivers/gpu/drm/i915/i915_vgpu.h | 85 +++++++++++++++++++++++++++++++++-------
 1 file changed, 70 insertions(+), 15 deletions(-)
drivers/gpu/drm/i915/i915_vgpu.h +70 -15
index b8a49e6..21c77a2 100644
@@ -42,24 +42,47 @@
 	INTEL_VGT_IF_VERSION_ENCODE(VGT_VERSION_MAJOR, VGT_VERSION_MINOR)
 
 /*
- * notifications from guest to vgpu device model
+ * The information set by the guest gfx driver, through the display_ready field
+ */
+#define    VGT_DRV_DISPLAY_NOT_READY	(0 << 0)
+#define    VGT_DRV_DISPLAY_READY	(1 << 0)	/* ready for display switch */
+#define    VGT_DRV_LEGACY_VGA_MODE	(1 << 1)	/* in the legacy VGA mode */
+
+/*
+ * guest-to-vgt notifications
  */
 enum vgt_g2v_type {
-	VGT_G2V_PPGTT_L3_PAGE_TABLE_CREATE = 2,
+	VGT_G2V_DISPLAY_REFRESH,
+	VGT_G2V_SET_POINTER_SHAPE,
+	VGT_G2V_PPGTT_L3_PAGE_TABLE_CREATE,
 	VGT_G2V_PPGTT_L3_PAGE_TABLE_DESTROY,
 	VGT_G2V_PPGTT_L4_PAGE_TABLE_CREATE,
 	VGT_G2V_PPGTT_L4_PAGE_TABLE_DESTROY,
-	VGT_G2V_EXECLIST_CONTEXT_CREATE,
-	VGT_G2V_EXECLIST_CONTEXT_DESTROY,
+	VGT_G2V_EXECLIST_CONTEXT_ELEMENT_CREATE,
+	VGT_G2V_EXECLIST_CONTEXT_ELEMENT_DESTROY,
 	VGT_G2V_MAX,
 };
 
+/*
+ * vgt-to-guest notifications
+ */
+enum vgt_v2g_type {
+	VGT_V2G_SET_HW_CURSOR,
+	VGT_V2G_SET_SW_CURSOR,
+	VGT_V2G_MAX,
+};
+
+enum vgt_caps_type {
+	VGT_CAPS_PREEMPTION = (1 << 0),
+};
+
 struct vgt_if {
 	uint64_t magic;		/* VGT_MAGIC */
 	uint16_t version_major;
 	uint16_t version_minor;
 	uint32_t vgt_id;	/* ID of vGT instance */
-	uint32_t rsv1[12];	/* pad to offset 0x40 */
+	uint32_t vgt_caps;     /* VGT capabilties */
+	uint32_t rsv1[11];	/* pad to offset 0x40 */
 	/*
 	 *  Data structure to describe the balooning info of resources.
 	 *  Each VM can only have one portion of continuous area for now.
@@ -85,13 +108,44 @@ struct vgt_if {
 	/*
 	 * The bottom half page is for response from Gfx driver to hypervisor.
 	 */
-	uint32_t rsv4;
-	uint32_t display_ready;	/* ready for display owner switch */
+	uint16_t  drv_version_major;
+	uint16_t  drv_version_minor;
+	uint32_t  display_ready;/* ready for display owner switch */
+	/*
+	 * driver reported status/error code
+	 *     0: if the avail_rs is sufficient to driver
+	 *  Bit 2,1,0 set indicating
+	 *       Insufficient low_gmadr, high_gmadr, fence resources.
+	 *  Other bits are reserved.
+	 */
+	uint32_t  rs_insufficient;
+	/*
+	 * The driver is required to update the following field with minimal
+	 * required resource size.
+	 */
+	uint32_t  min_low_gmadr;
+	uint32_t  min_high_gmadr;
+	uint32_t  min_fence_num;
+
+	/*
+	 * notifications between guest and vgt
+	 */
+	uint32_t  g2v_notify;
+	uint32_t  v2g_notify;
 
-	uint32_t rsv5[4];
+	/*
+	 * PPGTT PTE table info
+	 */
+	uint32_t  gmm_gtt_seg_base;
+	uint32_t  rsv4;
+	uint32_t  gmm_gtt_seg_size;
+	uint32_t  rsv5;
 
-	uint32_t g2v_notify;
-	uint32_t rsv6[7];
+	/*
+	 * Cursor hotspot info
+	 */
+	uint32_t  xhot;
+	uint32_t  yhot;
 
 	struct {
 		uint32_t lo;
@@ -101,16 +155,17 @@ struct vgt_if {
 	uint32_t execlist_context_descriptor_lo;
 	uint32_t execlist_context_descriptor_hi;
 
-	uint32_t  rsv7[0x200 - 24];    /* pad to one page */
+	/*
+	 * scratch space for debugging
+	 */
+	uint32_t  scratch;;
+
+	uint32_t  rsv6[0x200-25];    /* pad to one page */
 } __packed;
 
 #define vgtif_reg(x) \
 	_MMIO((VGT_PVINFO_PAGE + (long)&((struct vgt_if *)NULL)->x))
 
-/* vGPU display status to be used by the host side */
-#define VGT_DRV_DISPLAY_NOT_READY 0
-#define VGT_DRV_DISPLAY_READY     1  /* ready for display switch */
-
 extern void i915_check_vgpu(struct drm_device *dev);
 extern int intel_vgt_balloon(struct drm_device *dev);
 extern void intel_vgt_deballoon(void);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#28 lore raw

RFC 12/29 drm/i915: gvt: vGPU life cycle management

In reply to Zhi Wang

This patch introduces vGPU life cycle management framework. vGPU instance is a collection of virtual GEN hardware status, like virtual CFG/MMIO registers, how much GGTT memory space/Fence this vGPU owns, etc.

A vGPU instance consists following virtualized/limited resources:

- Configuration space(virtualized)
- MMIO registers(virtualized)
- GGTT memory space(limited)
- GGTT page table(shadowed)
- Fence(limited)

The framework is responsible for createing/destroying a vGPU instance, allocating/freeing the per-vGPU resource from GVT-g resource allocator, presenting them in the virtual PVINFO page located in virtual MMIO bar, which provides the basic foundation blocks to GVT-g CFG/MMIO emulation framework.

A big picture here looks like:

+-----------------------------------------------------------------------+
|                      XEN/KVM Hypervisor                               |
+---------------------------+--------^----------------------------------+
   CFG/MMIO emulate request |        | Emulation is done
    from hypervisor         |        | Return the result to hypervisor
                            |        |
+---------------------------v--------+-----------------------------------+
|                  GVT-g  CFG/MMIO emulation framework                   |
+-----+-----^----------------+-----^---------------------+-------^-------+
      |     |                |     |                     |       |
      |     | *vGPU instance |     |             *vGPU 2 |       |
+-----v-----+----------+-----v-----+-------------+ +-----v-------+-------+
| vConfiguration Space |vGTT/MMIO/Fence registers| |  ...                |
+----------------------+-------------------------+ +---------------------+

vGPU life cycle management

---
 drivers/gpu/drm/i915/gvt/Makefile   |   2 +-
 drivers/gpu/drm/i915/gvt/gvt.h      |  66 ++++++++++
 drivers/gpu/drm/i915/gvt/instance.c | 235 ++++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/gvt/mmio.c     |  32 +++++
 drivers/gpu/drm/i915/gvt/params.c   |   1 +
 drivers/gpu/drm/i915/gvt/params.h   |   1 +
 drivers/gpu/drm/i915/gvt/reg.h      |  29 ++++-
 drivers/gpu/drm/i915/i915_vgpu.h    |   5 +-
 8 files changed, 365 insertions(+), 6 deletions(-)
 create mode 100644 drivers/gpu/drm/i915/gvt/instance.c
drivers/gpu/drm/i915/gvt/Makefile +1 -1
index 5d28ed1..f4dcf9a 100644
@@ -1,4 +1,4 @@
-GVT_SOURCE := gvt.o params.o aperture_gm.o mmio.o handlers.o fb_decoder.o
+GVT_SOURCE := gvt.o params.o aperture_gm.o mmio.o handlers.o instance.o fb_decoder.o
 
 ccflags-y			+= -I$(src) -I$(src)/.. -Wall -Werror -Wno-unused-function
 i915_gvt-y			:= $(GVT_SOURCE)
drivers/gpu/drm/i915/gvt/gvt.h +66 -0

88 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/instance.c new file +235 -0

235 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/mmio.c +32 -0
index 0fbabd2..28e1393 100644
@@ -288,3 +288,35 @@ err:
 	gvt_clean_mmio_emulation_state(pdev);
 	return false;
 }
+
+void gvt_init_virtual_mmio_register(struct vgt_device *vgt)
+{
+	struct pgt_device *pdev = vgt->pdev;
+	int i;
+
+        for (i = 0; i < pdev->mmio_size; i += sizeof(u32)) {
+                /*
+                 * skip the area of VGT PV INFO PAGE because we need keep
+                 * its content across Dom0 S3.
+                */
+                if (i >= VGT_PVINFO_PAGE &&
+                        i < VGT_PVINFO_PAGE + VGT_PVINFO_SIZE)
+                        continue;
+
+                __vreg(vgt, i) = pdev->initial_mmio_state[REG_INDEX(i)];
+        }
+
+        /* set the bit 0:2 (Thread C-State) to C0
+         * TODO: consider other bit 3:31
+         */
+        __vreg(vgt, _GEN6_GT_THREAD_STATUS_REG) = 0;
+
+        /* set the bit 0:2(Core C-State ) to C0 */
+        __vreg(vgt, _GEN6_GT_CORE_STATUS) = 0;
+}
+
+void gvt_init_shadow_mmio_register(struct vgt_device *vgt)
+{
+	struct gvt_virtual_device_state *state = &vgt->state;
+        memcpy (state->mmio.sreg, vgt->pdev->initial_mmio_state, vgt->pdev->mmio_size);
+}
drivers/gpu/drm/i915/gvt/params.c +1 -0
index 6cd324c..fca49b0 100644
@@ -25,6 +25,7 @@
 
 struct gvt_kernel_params gvt = {
 	.enable = true,
+	.primary = true,
 	.debug = 0,
 	.dom0_low_gm_sz = 96,
 	.dom0_high_gm_sz = 384,
drivers/gpu/drm/i915/gvt/params.h +1 -0
index 0507870..80255c3 100644
@@ -26,6 +26,7 @@
 
 struct gvt_kernel_params {
 	bool enable;
+	bool primary;
 	int debug;
 	int dom0_low_gm_sz;
 	int dom0_high_gm_sz;
drivers/gpu/drm/i915/gvt/reg.h +26 -3
index 5682e1c..2edaf7c 100644
@@ -27,12 +27,35 @@
 #define GVT_CFG_SPACE_SZ	256
 #define GVT_BAR_NUM		4
 
-#define GVT_REG_CFG_SPACE_BAR0	0x10
-#define GVT_REG_CFG_SPACE_BAR1	0x18
-#define GVT_REG_CFG_SPACE_BAR2	0x20
+#define GVT_PCI_CLASS_VGA			0x03
+#define GVT_PCI_CLASS_VGA_OTHER			0x80
+
+#define GVT_REG_CFG_VENDOR_ID                   0x00
+#define GVT_REG_CFG_COMMAND                     0x04
+#define _REGBIT_CFG_COMMAND_IO                  (1 << 0)
+#define _REGBIT_CFG_COMMAND_MEMORY              (1 << 1)
+#define _REGBIT_CFG_COMMAND_MASTER              (1 << 2)
+#define GVT_REG_CFG_CLASS_PROG_IF               0x09
+#define GVT_REG_CFG_SUB_CLASS_CODE              0x0A
+#define GVT_REG_CFG_CLASS_CODE                  0x0B
+#define GVT_REG_CFG_SPACE_BAR0                  0x10
+#define GVT_REG_CFG_SPACE_BAR1                  0x18
+#define GVT_REG_CFG_SPACE_BAR2                  0x20
+#define GVT_REG_CFG_SPACE_BAR_ROM               0x30
+#define GVT_REG_CFG_SPACE_MSAC                  0x62
+#define GVT_REG_CFG_SWSCI_TRIGGER               0xE8
+#define _REGBIT_CFG_SWSCI_SCI_SELECT            (1 << 15)
+#define _REGBIT_CFG_SWSCI_SCI_TRIGGER           1
+#define GVT_REG_CFG_OPREGION                    0xFC
+
+#define _REG_GMCH_CONTROL               0x50
+#define    _REGBIT_BDW_GMCH_GMS_SHIFT   8
+#define    _REGBIT_BDW_GMCH_GMS_MASK    0xff
 
 #define _PCH_GMBUS2			0xc5108
 
 #define _GEN6_GDRST			0x941c
+#define _GEN6_GT_THREAD_STATUS_REG	0x13805c
+#define _GEN6_GT_CORE_STATUS		0x138060
 
 #endif
drivers/gpu/drm/i915/i915_vgpu.h +3 -2
index 21c77a2..b6d9fc7 100644
@@ -163,8 +163,9 @@ struct vgt_if {
 	uint32_t  rsv6[0x200-25];    /* pad to one page */
 } __packed;
 
-#define vgtif_reg(x) \
-	_MMIO((VGT_PVINFO_PAGE + (long)&((struct vgt_if *)NULL)->x))
+#define _vgtif_reg(x) \
+	((VGT_PVINFO_PAGE + (long)&((struct vgt_if *)NULL)->x))
+#define vgtif_reg(x) _MMIO(_vgtif_reg(x))
 
 extern void i915_check_vgpu(struct drm_device *dev);
 extern int intel_vgt_balloon(struct drm_device *dev);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#29 lore raw

RFC 13/29 drm/i915: gvt: trace stub

In reply to Zhi Wang
---
 drivers/gpu/drm/i915/gvt/Makefile       |  3 ++-
 drivers/gpu/drm/i915/gvt/trace.h        | 36 +++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/gvt/trace_points.c | 29 ++++++++++++++++++++++++++
 3 files changed, 67 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/drm/i915/gvt/trace.h
 create mode 100644 drivers/gpu/drm/i915/gvt/trace_points.c
drivers/gpu/drm/i915/gvt/Makefile +2 -1
index f4dcf9a..285eb84 100644
@@ -1,4 +1,5 @@
-GVT_SOURCE := gvt.o params.o aperture_gm.o mmio.o handlers.o instance.o fb_decoder.o
+GVT_SOURCE := gvt.o params.o aperture_gm.o mmio.o handlers.o instance.o \
+		trace_points.o fb_decoder.o
 
 ccflags-y			+= -I$(src) -I$(src)/.. -Wall -Werror -Wno-unused-function
 i915_gvt-y			:= $(GVT_SOURCE)
drivers/gpu/drm/i915/gvt/trace.h new file +36 -0
new file mode 100644
index 0000000..c98600a
@@ -0,0 +1,36 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of Version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
+ */
+
+#if !defined(_GVT_TRACE_H_) || defined(TRACE_HEADER_MULTI_READ)
+#define _GVT_TRACE_H_
+
+#include <linux/types.h>
+#include <linux/stringify.h>
+#include <linux/tracepoint.h>
+#include <asm/tsc.h>
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM gvt
+
+#endif /* _GVT_TRACE_H_ */
+
+/* This part must be out of protection */
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH .
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE trace
+#include <trace/define_trace.h>
drivers/gpu/drm/i915/gvt/trace_points.c new file +29 -0
new file mode 100644
index 0000000..6d0ad19
@@ -0,0 +1,29 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include "trace.h"
+
+#ifndef __CHECKER__
+#define CREATE_TRACE_POINTS
+#include "trace.h"
+#endif
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#30 lore raw

RFC 14/29 drm/i915: gvt: vGPU interrupt emulation framework

In reply to Zhi Wang

This patch introduces vGPU interrupt emulation framework.

The vGPU intrerrupt emulation framework is an event-based interrupt emulation framework. It's responsible for emulating GEN hardware interrupts demanded by other GVT-g emulation core component.

It consists serveral components:

- Descriptions of interrupt register bit definition
- Upper level <-> lower level interrupt mapping
- GEN HW IER/IMR/IIR register emulation routines
- Event-based interrupt propagation interface

The specific GVT-g component wants to emulate an interrupt to a VM just need to specify an event: e.g RCS_MI_USER_INTERRUPT to the framework. The framework will:

- Generate related virtual IIR bit according to guest virtual IER and IMRs,
- Generate related upper level interrupt virtual IIR bit accodring to the cascade interrupt mapping
- Inject a MSI to guest

Then guest GEN driver will see a virtual GEN interrupt is asserted.

---
 drivers/gpu/drm/i915/gvt/Makefile    |   2 +-
 drivers/gpu/drm/i915/gvt/gvt.c       |   4 +
 drivers/gpu/drm/i915/gvt/gvt.h       |   6 +
 drivers/gpu/drm/i915/gvt/interrupt.c | 648 +++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/gvt/interrupt.h | 232 +++++++++++++
 drivers/gpu/drm/i915/gvt/perf.h      |  39 +++
 drivers/gpu/drm/i915/gvt/reg.h       |  31 ++
 7 files changed, 961 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/drm/i915/gvt/interrupt.c
 create mode 100644 drivers/gpu/drm/i915/gvt/interrupt.h
 create mode 100644 drivers/gpu/drm/i915/gvt/perf.h
drivers/gpu/drm/i915/gvt/Makefile +1 -1
index 285eb84..78deefc 100644
@@ -1,5 +1,5 @@
 GVT_SOURCE := gvt.o params.o aperture_gm.o mmio.o handlers.o instance.o \
-		trace_points.o fb_decoder.o
+		trace_points.o interrupt.o fb_decoder.o
 
 ccflags-y			+= -I$(src) -I$(src)/.. -Wall -Werror -Wno-unused-function
 i915_gvt-y			:= $(GVT_SOURCE)
drivers/gpu/drm/i915/gvt/gvt.c +4 -0
index f5066bc..a0a9667 100644
@@ -238,6 +238,7 @@ static bool init_service_thread(struct pgt_device *pdev)
 static void clean_pgt_device(struct pgt_device *pdev)
 {
 	clean_service_thread(pdev);
+	gvt_irq_exit(pdev);
 	gvt_clean_mmio_emulation_state(pdev);
 	clean_initial_mmio_state(pdev);
 	gvt_clean_resource_allocator(pdev);
@@ -258,6 +259,9 @@ static bool init_pgt_device(struct pgt_device *pdev, struct drm_i915_private *de
 	if (!gvt_setup_mmio_emulation_state(pdev))
 		goto err;
 
+	if (!gvt_irq_init(pdev))
+		goto err;
+
 	if (!init_service_thread(pdev))
 		goto err;
 
drivers/gpu/drm/i915/gvt/gvt.h +6 -0
index c58305f..fad56b1 100644
@@ -34,6 +34,8 @@
 #include "mpt.h"
 #include "fb_decoder.h"
 #include "mmio.h"
+#include "interrupt.h"
+#include "perf.h"
 
 #define GVT_MAX_VGPU 8
 
@@ -111,6 +113,7 @@ struct vgt_device {
 	bool warn_untrack;
 	atomic_t active;
 	struct gvt_virtual_device_state state;
+	struct gvt_statistics stat;
 };
 
 struct gvt_gm_allocator {
@@ -160,6 +163,9 @@ struct pgt_device {
 	u32 aux_table_index;
 
 	DECLARE_HASHTABLE(mmio_table, GVT_HASH_BITS);
+
+	struct gvt_irq_state irq_state;
+	struct pgt_statistics stat;
 };
 
 /* definitions for physical aperture/GM space */
drivers/gpu/drm/i915/gvt/interrupt.c new file +648 -0

648 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/interrupt.h new file +232 -0

232 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/perf.h new file +39 -0
new file mode 100644
index 0000000..8f0cd15
@@ -0,0 +1,39 @@
+/*
+ * Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _GVT_PERF_H_
+#define _GVT_PERF_H_
+
+struct gvt_statistics {
+	u64	irq_num;
+	u64	events[GVT_EVENT_MAX];
+	u64	last_injection;
+};
+
+struct pgt_statistics {
+	u64	irq_num;
+	u64	irq_delay_cycles;
+	u64	events[GVT_EVENT_MAX];
+};
+
+#endif
drivers/gpu/drm/i915/gvt/reg.h +31 -0
index 2edaf7c..0e28a71 100644
@@ -58,4 +58,35 @@
 #define _GEN6_GT_THREAD_STATUS_REG	0x13805c
 #define _GEN6_GT_CORE_STATUS		0x138060
 
+#define _GEN8_DE_PORT_IMR (0x44444)
+#define _GEN8_DE_PORT_IER (0x4444c)
+#define _GEN8_DE_PORT_IIR (0x44448)
+#define _GEN8_DE_PORT_ISR (0x44440)
+
+#define _GEN8_DE_MISC_IMR (0x44464)
+#define _GEN8_DE_MISC_IER (0x4446c)
+#define _GEN8_DE_MISC_IIR (0x44468)
+#define _GEN8_DE_MISC_ISR (0x44460)
+
+#define _GEN8_PCU_IMR (0x444e4)
+#define _GEN8_PCU_IER (0x444ec)
+#define _GEN8_PCU_IIR (0x444e8)
+#define _GEN8_PCU_ISR (0x444e0)
+#define _GEN8_MASTER_IRQ			(0x44200)
+
+#define _SDEIMR  (0xc4004)
+#define _SDEIER  (0xc400c)
+#define _SDEIIR  (0xc4008)
+#define _SDEISR  (0xc4000)
+
+#define _GEN8_GT_ISR(which) (0x44300 + (0x10 * (which)))
+#define _GEN8_GT_IMR(which) (0x44304 + (0x10 * (which)))
+#define _GEN8_GT_IIR(which) (0x44308 + (0x10 * (which)))
+#define _GEN8_GT_IER(which) (0x4430c + (0x10 * (which)))
+
+#define _GEN8_DE_PIPE_ISR(pipe) (0x44400 + (0x10 * (pipe)))
+#define _GEN8_DE_PIPE_IMR(pipe) (0x44404 + (0x10 * (pipe)))
+#define _GEN8_DE_PIPE_IIR(pipe) (0x44408 + (0x10 * (pipe)))
+#define _GEN8_DE_PIPE_IER(pipe) (0x4440c + (0x10 * (pipe)))
+
 #endif
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#31 lore raw

RFC 15/29 drm/i915: gvt: vGPU graphics memory emulation framework

In reply to Zhi Wang

The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization/shadow. Under virtualization environment, the guest will populate the page table with guest page frame number(GFN). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN.

To solve this kinds of gap, shadow GGTT/PPGTT page table is introdcued.

For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers.

For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed.

vGPU graphics memory emulation framework consists:

- Per-GEN HW platform page table entry bits extract/de-extract routines.
- GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register
- PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync

---
 drivers/gpu/drm/i915/gvt/Makefile   |    2 +-
 drivers/gpu/drm/i915/gvt/gtt.c      | 1912 +++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/gvt/gtt.h      |  305 ++++++
 drivers/gpu/drm/i915/gvt/gvt.c      |    8 +
 drivers/gpu/drm/i915/gvt/gvt.h      |  148 +++
 drivers/gpu/drm/i915/gvt/instance.c |    7 +
 drivers/gpu/drm/i915/gvt/mpt.h      |   11 +
 drivers/gpu/drm/i915/gvt/params.c   |   12 +
 drivers/gpu/drm/i915/gvt/params.h   |    3 +
 drivers/gpu/drm/i915/gvt/perf.h     |   23 +
 drivers/gpu/drm/i915/gvt/trace.h    |  172 ++++
 11 files changed, 2602 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/drm/i915/gvt/gtt.c
 create mode 100644 drivers/gpu/drm/i915/gvt/gtt.h
drivers/gpu/drm/i915/gvt/Makefile +1 -1
index 78deefc..360539c 100644
@@ -1,5 +1,5 @@
 GVT_SOURCE := gvt.o params.o aperture_gm.o mmio.o handlers.o instance.o \
-		trace_points.o interrupt.o fb_decoder.o
+		trace_points.o interrupt.o gtt.o fb_decoder.o
 
 ccflags-y			+= -I$(src) -I$(src)/.. -Wall -Werror -Wno-unused-function
 i915_gvt-y			:= $(GVT_SOURCE)
drivers/gpu/drm/i915/gvt/gtt.c new file +1912 -0

1,912 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/gtt.h new file +305 -0

305 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/gvt.c +8 -0
index a0a9667..13fecdf 100644
@@ -114,6 +114,10 @@ static bool init_device_info(struct pgt_device *pdev)
 	printk("        gtt_start_offset: %x\n", info->gtt_start_offset);
 	printk("        gtt_end_offset: %x\n", info->gtt_end_offset);
 
+	if (!gvt_init_avail_gtt_size(pdev)) {
+		gvt_err("fail to get avail gtt size");
+		return false;
+	}
 	return true;
 }
 
@@ -238,6 +242,7 @@ static bool init_service_thread(struct pgt_device *pdev)
 static void clean_pgt_device(struct pgt_device *pdev)
 {
 	clean_service_thread(pdev);
+	gvt_clean_gtt(pdev);
 	gvt_irq_exit(pdev);
 	gvt_clean_mmio_emulation_state(pdev);
 	clean_initial_mmio_state(pdev);
@@ -262,6 +267,9 @@ static bool init_pgt_device(struct pgt_device *pdev, struct drm_i915_private *de
 	if (!gvt_irq_init(pdev))
 		goto err;
 
+	if (!gvt_init_gtt(pdev))
+		goto err;
+
 	if (!init_service_thread(pdev))
 		goto err;
 
drivers/gpu/drm/i915/gvt/gvt.h +148 -0

176 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/instance.c +7 -0
index 07b797a..1540661 100644
@@ -106,6 +106,7 @@ static void init_virtual_cfg_space_state(struct vgt_device *vgt,
 
 static void destroy_virtual_gm_state(struct vgt_device *vgt)
 {
+	gvt_clean_vgtt(vgt);
 	gvt_free_gm_and_fence_resource(vgt);
 }
 
@@ -151,11 +152,17 @@ static bool create_virtual_gm_state(struct vgt_device *vgt,
 
 	populate_pvinfo_page(vgt);
 
+	if (!gvt_init_vgtt(vgt)) {
+		gvt_err("fail to init vgtt");
+		return false;
+	}
+
 	return true;
 }
 
 static void destroy_virtual_device_state(struct vgt_device *vgt)
 {
+	gvt_clean_vgtt(vgt);
 	destroy_virtual_mmio_state(vgt);
 	destroy_virtual_gm_state(vgt);
 }
drivers/gpu/drm/i915/gvt/mpt.h +11 -0
index bbe4465..99acf3d 100644
@@ -24,6 +24,7 @@
 #ifndef _GVT_MPT_H_
 #define _GVT_MPT_H_
 
+struct guest_page;
 struct vgt_device;
 
 static inline unsigned long hypervisor_g2m_pfn(struct vgt_device *vgt,
@@ -63,6 +64,16 @@ static inline void *hypervisor_mfn_to_virt(int mfn)
 	return NULL;
 }
 
+static inline int hypervisor_set_wp_pages(struct vgt_device *vgt, struct guest_page *p)
+{
+        return 0;
+}
+
+static inline int hypervisor_unset_wp_pages(struct vgt_device *vgt, struct guest_page *p)
+{
+        return 0;
+}
+
 static inline void hypervisor_inject_msi(struct vgt_device *vgt)
 {
 	return;
drivers/gpu/drm/i915/gvt/params.c +12 -0
index fca49b0..b6eab30 100644
@@ -30,6 +30,9 @@ struct gvt_kernel_params gvt = {
 	.dom0_low_gm_sz = 96,
 	.dom0_high_gm_sz = 384,
 	.dom0_fence_sz = 4,
+	.preallocated_shadow_pages = -1,
+	.preallocated_oos_pages = -1,
+	.spt_out_of_sync = false,
 };
 
 module_param_named(dom0_low_gm_sz, gvt.dom0_low_gm_sz, int, 0600);
@@ -40,3 +43,12 @@ MODULE_PARM_DESC(dom0_high_gm_sz, "Amount of high memory size of DOM0");
 
 module_param_named(dom0_fence_sz, gvt.dom0_fence_sz, int, 0600);
 MODULE_PARM_DESC(dom0_fence_sz, "Amount of fence size of DOM0");
+
+module_param_named(preallocated_shadow_pages, gvt.preallocated_shadow_pages, int, 0600);
+MODULE_PARM_DESC(preallocated_shadow_pages, "Amount of pre-allocated shadow pages");
+
+module_param_named(preallocated_oos_pages, gvt.preallocated_oos_pages, int, 0600);
+MODULE_PARM_DESC(preallocated_oos_pages, "Amount of pre-allocated oos pages");
+
+module_param_named(spt_out_of_sync, gvt.spt_out_of_sync, bool, 0600);
+MODULE_PARM_DESC(spt_out_of_sync, "Enable SPT out of sync");
drivers/gpu/drm/i915/gvt/params.h +3 -0
index 80255c3..4e7f568 100644
@@ -31,6 +31,9 @@ struct gvt_kernel_params {
 	int dom0_low_gm_sz;
 	int dom0_high_gm_sz;
 	int dom0_fence_sz;
+	int preallocated_shadow_pages;
+	int preallocated_oos_pages;
+	bool spt_out_of_sync;
 };
 
 extern struct gvt_kernel_params gvt;
drivers/gpu/drm/i915/gvt/perf.h +23 -0
index 8f0cd15..146a1cb 100644
@@ -28,12 +28,35 @@ struct gvt_statistics {
 	u64	irq_num;
 	u64	events[GVT_EVENT_MAX];
 	u64	last_injection;
+	u64	gtt_mmio_rcnt;
+	u64	gtt_mmio_wcnt;
+	u64	gtt_mmio_wcycles;
+	u64	gtt_mmio_rcycles;
+	u64	wp_cnt;
+	u64	wp_cycles;
+	u64	ppgtt_wp_cnt;
+	u64	ppgtt_wp_cycles;
+	u64	spt_find_hit_cnt;
+	u64	spt_find_hit_cycles;
+	u64	spt_find_miss_cnt;
+	u64	spt_find_miss_cycles;
+	u64	gpt_find_hit_cnt;
+	u64	gpt_find_hit_cycles;
+	u64	gpt_find_miss_cnt;
+	u64	gpt_find_miss_cycles;
+
 };
 
 struct pgt_statistics {
 	u64	irq_num;
 	u64	irq_delay_cycles;
 	u64	events[GVT_EVENT_MAX];
+	u64	oos_page_cur_avail_cnt;
+	u64	oos_page_min_avail_cnt;
+	u64	oos_page_steal_cnt;
+	u64	oos_page_attach_cnt;
+	u64	oos_page_detach_cnt;
+
 };
 
 #endif
drivers/gpu/drm/i915/gvt/trace.h +172 -0

178 lines not on this page · raw message

Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#32 lore raw

RFC 16/29 drm/i915: gvt: Generic MPT framework

In reply to Zhi Wang

GVT-g supports both Xen/KVM hypervisors and requires a couple of hypervisor services to work. The MPT framework is a kinds of abstraction which provides a unique hypervisor APIs to GVT-g core logics.

---
 drivers/gpu/drm/i915/gvt/gvt.c       |   6 ++
 drivers/gpu/drm/i915/gvt/gvt.h       |  11 +-
 drivers/gpu/drm/i915/gvt/hypercall.h |  26 +++++
 drivers/gpu/drm/i915/gvt/mmio.c      | 194 +++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/gvt/mmio.h      |   4 +
 drivers/gpu/drm/i915/gvt/mpt.h       | 103 +++++++++++++++----
 drivers/gpu/drm/i915/gvt/perf.h      |   4 +
 7 files changed, 326 insertions(+), 22 deletions(-)
drivers/gpu/drm/i915/gvt/gvt.c +6 -0
index 13fecdf..a71873c 100644
@@ -31,6 +31,11 @@ struct gvt_host gvt_host;
 extern struct gvt_kernel_dm xengt_kdm;
 extern struct gvt_kernel_dm kvmgt_kdm;
 
+static struct gvt_io_emulation_ops default_io_emulation_ops = {
+	.emulate_mmio_read = gvt_emulate_mmio_read,
+	.emulate_mmio_write = gvt_emulate_mmio_write,
+};
+
 static const char *supported_hypervisors[] = {
 	[GVT_HYPERVISOR_TYPE_XEN] = "Xen Hypervisor",
 	[GVT_HYPERVISOR_TYPE_KVM] = "KVM",
@@ -72,6 +77,7 @@ static bool gvt_init_host(void)
 	gvt_info("Running with hypervisor %s in host mode",
 			supported_hypervisors[host->hypervisor_type]);
 
+	host->emulate_ops = &default_io_emulation_ops;
 	idr_init(&host->device_idr);
 	mutex_init(&host->device_idr_lock);
 
drivers/gpu/drm/i915/gvt/gvt.h +10 -1
index 542f3e6..eb5fd47 100644
@@ -31,7 +31,6 @@
 #include "params.h"
 #include "reg.h"
 #include "hypercall.h"
-#include "mpt.h"
 #include "fb_decoder.h"
 #include "mmio.h"
 #include "interrupt.h"
@@ -52,12 +51,20 @@ enum {
 	GVT_HYPERVISOR_TYPE_KVM,
 };
 
+struct gvt_io_emulation_ops {
+	bool (*emulate_mmio_read)(struct vgt_device *, uint64_t, void *, int);
+	bool (*emulate_mmio_write)(struct vgt_device *, uint64_t, void *, int);
+	bool (*emulate_cfg_read)(struct vgt_device *, unsigned int, void *, int);
+	bool (*emulate_cfg_write)(struct vgt_device *, unsigned int, void *, int);
+};
+
 struct gvt_host {
 	bool initialized;
 	int hypervisor_type;
 	struct mutex device_idr_lock;
 	struct idr device_idr;
 	struct gvt_kernel_dm *kdm;
+	struct gvt_io_emulation_ops *emulate_ops;
 };
 
 extern struct gvt_host gvt_host;
@@ -579,4 +586,6 @@ static inline u32 h2g_gtt_index(struct vgt_device *vgt, uint32_t h_index)
 	return (u32)(h2g_gm(vgt, h_addr) >> GTT_PAGE_SHIFT);
 }
 
+#include "mpt.h"
+
 #endif
drivers/gpu/drm/i915/gvt/hypercall.h +26 -0
index 0a41874..d30f5a7 100644
@@ -24,7 +24,33 @@
 #ifndef _GVT_HYPERCALL_H_
 #define _GVT_HYPERCALL_H_
 
+struct vgt_device;
+struct guest_page;
+
+enum map_type {
+        GVT_MAP_APERTURE,
+        GVT_MAP_OPREGION,
+};
+
 struct gvt_kernel_dm {
+        const char *name;
+        unsigned long (*g2m_pfn)(int vm_id, unsigned long g_pfn);
+        int (*pause_domain)(int vm_id);
+        int (*shutdown_domain)(int vm_id);
+        int (*map_mfn_to_gpfn)(int vm_id, unsigned long gpfn,
+                unsigned long mfn, int nr, int map, enum map_type type);
+        int (*set_trap_area)(struct vgt_device *vgt, uint64_t start, uint64_t end, bool map);
+        bool (*set_wp_pages)(struct vgt_device *vgt, struct guest_page *p);
+        bool (*unset_wp_pages)(struct vgt_device *vgt, struct guest_page *p);
+        int (*detect_host)(void);
+        int (*from_virt_to_mfn)(void *addr);
+        void *(*from_mfn_to_virt)(int mfn);
+        int (*inject_msi)(int vm_id, u32 addr, u16 data);
+        int (*hvm_init)(struct vgt_device *vgt);
+        void (*hvm_exit)(struct vgt_device *vgt);
+        void *(*gpa_to_va)(struct vgt_device *vgt, unsigned long gap);
+        bool (*read_va)(struct vgt_device *vgt, void *va, void *val, int len, int atomic);
+        bool (*write_va)(struct vgt_device *vgt, void *va, void *val, int len, int atomic);
 };
 
 #endif /* _GVT_HYPERCALL_H_ */
drivers/gpu/drm/i915/gvt/mmio.c +194 -0

197 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/mmio.h +4 -0
index caca60f..4301655 100644
@@ -84,4 +84,8 @@ struct gvt_reg_info {
 extern struct gvt_reg_info gvt_general_reg_info[];
 extern struct gvt_reg_info gvt_broadwell_reg_info[];
 extern int gvt_get_reg_num(int type);
+
+bool gvt_emulate_mmio_read(struct vgt_device *vgt, uint64_t pa, void *p_data,int bytes);
+bool gvt_emulate_mmio_write(struct vgt_device *vgt, uint64_t pa, void *p_data,int bytes);
+
 #endif
drivers/gpu/drm/i915/gvt/mpt.h +82 -21

167 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/perf.h +4 -0
index 146a1cb..21b0637 100644
@@ -28,6 +28,10 @@ struct gvt_statistics {
 	u64	irq_num;
 	u64	events[GVT_EVENT_MAX];
 	u64	last_injection;
+	u64	mmio_rcnt;
+	u64	mmio_wcnt;
+	u64	mmio_wcycles;
+	u64	mmio_rcycles;
 	u64	gtt_mmio_rcnt;
 	u64	gtt_mmio_wcnt;
 	u64	gtt_mmio_wcycles;
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
author
#33 lore raw

RFC 17/29 gvt: Xen hypervisor GVT-g MPT module

In reply to Zhi Wang

This is the xen hypervisor MPT module which let GVT-g be able to run under Xen hypervisor.

---
 arch/x86/include/asm/xen/hypercall.h |    7 +
 arch/x86/include/asm/xen/interface.h |    1 +
 arch/x86/xen/mmu.c                   |   83 +++
 drivers/gpu/drm/i915/gvt/gvt.c       |   10 +
 drivers/gpu/drm/i915/gvt/gvt.h       |   14 +
 drivers/xen/Kconfig                  |    5 +
 drivers/xen/Makefile                 |    6 +
 drivers/xen/xengt.c                  | 1153 ++++++++++++++++++++++++++++++++++
 include/xen/interface/hvm/hvm_op.h   |  177 +++++-
 include/xen/interface/hvm/ioreq.h    |  132 ++++
 include/xen/interface/memory.h       |   28 +
 include/xen/interface/xen.h          |  106 ++++
 include/xen/xen-ops.h                |    5 +
 13 files changed, 1726 insertions(+), 1 deletion(-)
 create mode 100644 drivers/xen/xengt.c
 create mode 100644 include/xen/interface/hvm/ioreq.h
arch/x86/include/asm/xen/hypercall.h +7 -0
index 3bcdcc8..aea97e3 100644
@@ -459,6 +459,13 @@ HYPERVISOR_hvm_op(int op, void *arg)
 }
 
 static inline int
+HYPERVISOR_domctl(
+        struct xen_domctl *arg)
+{
+        return _hypercall1(int, domctl, arg);
+}
+
+static inline int
 HYPERVISOR_tmem_op(
 	struct tmem_op *op)
 {
arch/x86/include/asm/xen/interface.h +1 -0
index 6ff4986..a4ee3f4 100644
@@ -89,6 +89,7 @@ typedef long xen_long_t;
 /* Guest handles for primitive C types. */
 __DEFINE_GUEST_HANDLE(uchar, unsigned char);
 __DEFINE_GUEST_HANDLE(uint,  unsigned int);
+__DEFINE_GUEST_HANDLE(ulong,  unsigned long);
 DEFINE_GUEST_HANDLE(char);
 DEFINE_GUEST_HANDLE(int);
 DEFINE_GUEST_HANDLE(void);
arch/x86/xen/mmu.c +83 -0

86 lines not on this page · raw message

drivers/gpu/drm/i915/gvt/gvt.c +10 -0
index a71873c..28a51d9 100644
@@ -21,12 +21,14 @@
  * SOFTWARE.
  */
 
+#include <linux/types.h>
 #include <xen/xen.h>
 #include <linux/kthread.h>
 
 #include "gvt.h"
 
 struct gvt_host gvt_host;
+EXPORT_SYMBOL(gvt_host);
 
 extern struct gvt_kernel_dm xengt_kdm;
 extern struct gvt_kernel_dm kvmgt_kdm;
@@ -36,6 +38,13 @@ static struct gvt_io_emulation_ops default_io_emulation_ops = {
 	.emulate_mmio_write = gvt_emulate_mmio_write,
 };
 
+unsigned int pa_to_mmio_offset(struct vgt_device *vgt,
+               uint64_t pa);
+
+static struct gvt_mpt_ops default_export_mpt_ops = {
+	.pa_to_mmio_offset = pa_to_mmio_offset,
+};
+
 static const char *supported_hypervisors[] = {
 	[GVT_HYPERVISOR_TYPE_XEN] = "Xen Hypervisor",
 	[GVT_HYPERVISOR_TYPE_KVM] = "KVM",
@@ -78,6 +87,7 @@ static bool gvt_init_host(void)
 			supported_hypervisors[host->hypervisor_type]);
 
 	host->emulate_ops = &default_io_emulation_ops;
+	host->mpt_ops = &default_export_mpt_ops;
 	idr_init(&host->device_idr);
 	mutex_init(&host->device_idr_lock);
 
drivers/gpu/drm/i915/gvt/gvt.h +14 -0
index eb5fd47..83f90a2 100644
@@ -58,6 +58,10 @@ struct gvt_io_emulation_ops {
 	bool (*emulate_cfg_write)(struct vgt_device *, unsigned int, void *, int);
 };
 
+struct gvt_mpt_ops {
+	unsigned int (*pa_to_mmio_offset)(struct vgt_device *, u64);
+};
+
 struct gvt_host {
 	bool initialized;
 	int hypervisor_type;
@@ -65,6 +69,7 @@ struct gvt_host {
 	struct idr device_idr;
 	struct gvt_kernel_dm *kdm;
 	struct gvt_io_emulation_ops *emulate_ops;
+	struct gvt_mpt_ops *mpt_ops;
 };
 
 extern struct gvt_host gvt_host;
@@ -123,6 +128,9 @@ struct vgt_device {
 	struct gvt_virtual_device_state state;
 	struct gvt_statistics stat;
 	struct gvt_vgtt_info gtt;
+	void *hypervisor_data;
+	unsigned long low_mem_max_gpfn;
+	atomic_t crashing;
 };
 
 struct gvt_gm_allocator {
@@ -423,6 +431,12 @@ static inline int gvt_pci_mmio_is_enabled(struct vgt_device *vgt)
 		_REGBIT_CFG_COMMAND_MEMORY;
 }
 
+static inline uint64_t gvt_mmio_bar_base(struct vgt_device *vgt)
+{
+        char *cfg_space = &vgt->state.cfg.space[0];
+        return *(u64 *)(cfg_space + GVT_REG_CFG_SPACE_BAR0);
+}
+
 #define __vreg(vgt, off) (*(u32*)(vgt->state.mmio.vreg + off))
 #define __vreg8(vgt, off) (*(u8*)(vgt->state.mmio.vreg + off))
 #define __vreg16(vgt, off) (*(u16*)(vgt->state.mmio.vreg + off))
drivers/xen/Kconfig +5 -0
index 73708ac..9ee2033 100644
@@ -291,4 +291,9 @@ config XEN_SYMS
 config XEN_HAVE_VPMU
        bool
 
+config XENGT
+        tristate "Xen Dom0 support for i915 gvt device model"
+        depends on XEN_DOM0 && I915_GVT
+        default m
+
 endmenu
drivers/xen/Makefile +6 -0
index 9b7a35c..ff75c36 100644
@@ -9,6 +9,10 @@ CFLAGS_features.o			:= $(nostackp)
 
 CFLAGS_efi.o				+= -fshort-wchar
 
+
+I915                     := drivers/gpu/drm/i915
+CFLAGS_xengt.o          += -Wall -Werror -I$(I915) -I$(I915)/gvt
+
 dom0-$(CONFIG_PCI) += pci.o
 dom0-$(CONFIG_USB_SUPPORT) += dbgp.o
 dom0-$(CONFIG_XEN_ACPI) += acpi.o $(xen-pad-y)
@@ -36,6 +40,8 @@ obj-$(CONFIG_XEN_ACPI_PROCESSOR)	+= xen-acpi-processor.o
 obj-$(CONFIG_XEN_EFI)			+= efi.o
 obj-$(CONFIG_XEN_SCSI_BACKEND)		+= xen-scsiback.o
 obj-$(CONFIG_XEN_AUTO_XLATE)		+= xlate_mmu.o
+obj-$(CONFIG_XENGT)                     += xengt.o
+
 xen-evtchn-y				:= evtchn.o
 xen-gntdev-y				:= gntdev.o
 xen-gntalloc-y				:= gntalloc.o
drivers/xen/xengt.c new file +1153 -0

1,153 lines not on this page · raw message

include/xen/interface/hvm/hvm_op.h +176 -1

198 lines not on this page · raw message

include/xen/interface/hvm/ioreq.h new file +132 -0

132 lines not on this page · raw message

include/xen/interface/memory.h +28 -0
index 2ecfe4f..92f18c5 100644
@@ -9,6 +9,7 @@
 #ifndef __XEN_PUBLIC_MEMORY_H__
 #define __XEN_PUBLIC_MEMORY_H__
 
+#include <xen/interface/event_channel.h>
 #include <linux/spinlock.h>
 
 /*
@@ -141,6 +142,11 @@ struct xen_machphys_mfn_list {
 DEFINE_GUEST_HANDLE_STRUCT(xen_machphys_mfn_list);
 
 /*
+ * Returns the maximum GPFN in use by the guest, or -ve errcode on failure.
+ */
+#define XENMEM_maximum_gpfn         14
+
+/*
  * Returns the location in virtual address space of the machine_to_phys
  * mapping table. Architectures which do not have a m2p table, or which do not
  * map it by default into guest address space, do not implement this command.
@@ -263,4 +269,26 @@ struct xen_remove_from_physmap {
 };
 DEFINE_GUEST_HANDLE_STRUCT(xen_remove_from_physmap);
 
+/*
+ * Translate the given guest PFNs to MFNs
+ */
+#define XENMEM_get_mfn_from_pfn    25
+struct xen_get_mfn_from_pfn {
+    /*
+     * Pointer to buffer to fill with list of pfn.
+     * for IN, it contains the guest PFN that need to translated
+     * for OUT, it contains the translated MFN. or INVALID_MFN if no valid translation
+     */
+    GUEST_HANDLE(ulong) pfn_list;
+
+    /*
+     * IN: Size of the pfn_array.
+     */
+    unsigned int nr_pfns;
+
+    /* IN: which domain */
+    domid_t domid;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_get_mfn_from_pfn);
+
 #endif /* __XEN_PUBLIC_MEMORY_H__ */
include/xen/interface/xen.h +106 -0

112 lines not on this page · raw message

include/xen/xen-ops.h +5 -0
index 86abe07..dde9eb0 100644
@@ -123,4 +123,9 @@ static inline void xen_preemptible_hcall_end(void)
 
 #endif /* CONFIG_PREEMPT */
 
+struct vm_struct * xen_remap_domain_mfn_range_in_kernel(unsigned long mfn,
+        int nr, unsigned domid);
+void xen_unmap_domain_mfn_range_in_kernel(struct vm_struct *area, int nr,
+                unsigned domid);
+
 #endif /* INCLUDE_XEN_OPS_H */
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx