Skip to contentSkip to conversation

xen: dma-buf support for grant device

PATCH Superseded Oleksandr Andrushchenko 3 participants 71 messages Jun 12, 2018 – Jun 14, 2018

Subsystems (2)

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v3 cover xen: dma-buf support for grant device

From: Oleksandr Andrushchenko <[email protected]>

This work is in response to my previous attempt to introduce Xen/DRM zero-copy driver [1] to enable Linux dma-buf API [2] for Xen based frontends/backends. There is also an existing hyper_dmabuf approach available [3] which, if reworked to utilize the proposed solution, can greatly benefit as well.

RFC for this series was published and discussed [9], comments addressed.

The original rationale behind this work was to enable zero-copying use-cases while working with Xen para-virtual display driver [4]:
when using Xen PV DRM frontend driver then on backend side one will need to do copying of display buffers' contents (filled by the frontend's user-space) into buffers allocated at the backend side. Taking into account the size of display buffers and frames per second it may result in unneeded huge data bus occupation and performance loss.

The helper driver [4] allows implementing zero-copying use-cases when using Xen para-virtualized frontend display driver by implementing a DRM/KMS helper driver running on backend's side.
It utilizes PRIME buffers API (implemented on top of Linux dma-buf) to share frontend's buffers with physical device drivers on backend's side:

- a dumb buffer created on backend's side can be shared with the Xen PV frontend driver, so it directly writes into backend's domain memory (into the buffer exported from DRM/KMS driver of a physical display device)
- a dumb buffer allocated by the frontend can be imported into physical device DRM/KMS driver, thus allowing to achieve no copying as well

Finally, it was discussed and decided ([1], [5]) that it is worth implementing such use-cases via extension of the existing Xen gntdev driver instead of introducing new DRM specific driver. Please note, that the support of dma-buf is Linux only, as dma-buf is a Linux only thing.

Now to the proposed solution. The changes to the existing Xen drivers in the Linux kernel fall into 2 categories:
1. DMA-able memory buffer allocation and increasing/decreasing memory reservation of the pages of such a buffer.
This is required if we are about to share dma-buf with the hardware that does require those to be allocated with dma_alloc_xxx API. (It is still possible to allocate a dma-buf from any system memory, e.g. system pages).
2. Extension of the gntdev driver to enable it to import/export dma-buf’s.

The first six patches are in preparation for Xen dma-buf support, but I consider those usable regardless of the dma-buf use-case, e.g. other frontend/backend kernel modules may also benefit from these for better code reuse:

    0001-xen-grant-table-Export-gnttab_-alloc-free-_pages-as-.patch
    0002-xen-grant-table-Make-set-clear-page-private-code-sha.patch
    0003-xen-balloon-Share-common-memory-reservation-routines.patch
    0004-xen-grant-table-Allow-allocating-buffers-suitable-fo.patch
    0005-xen-gntdev-Allow-mappings-for-DMA-buffers.patch
    0006-xen-gntdev-Make-private-routines-structures-accessib.patch

The next three patches are Xen implementation of dma-buf as part of the grant device:
0007-xen-gntdev-Add-initial-support-for-dma-buf-UAPI.patch 0008-xen-gntdev-Implement-dma-buf-export-functionality.patch 0009-xen-gntdev-Implement-dma-buf-import-functionality.patch

The corresponding libxengnttab changes are available at [6].

All the above was tested with display backend [7] and its accompanying helper library [8] on Renesas ARM64 based board.
Basic balloon tests on x86.

*To all the communities*: I would like to ask you to review the proposed solution and give feedback on it, so I can improve and send final patches for review (this is still work in progress, but enough to start discussing the implementation).

Thank you in advance,
Oleksandr Andrushchenko

[1] https://lists.freedesktop.org/archives/dri-devel/2018-April/173163.html
[2] https://elixir.bootlin.com/linux/v4.17-rc5/source/Documentation/driver-api/dma-buf.rst
[3] https://lists.xenproject.org/archives/html/xen-devel/2018-02/msg01202.html
[4] https://cgit.freedesktop.org/drm/drm-misc/tree/drivers/gpu/drm/xen
[5] https://patchwork.kernel.org/patch/10279681/
[6] https://github.com/andr2000/xen/tree/xen_dma_buf_v1
[7] https://github.com/andr2000/displ_be/tree/xen_dma_buf_v1
[8] https://github.com/andr2000/libxenbe/tree/xen_dma_buf_v1
[9] https://lkml.org/lkml/2018/5/17/215

Changes since v2:
*****************
- fixed missed break in dmabuf_exp_wait_obj_signal
- re-worked debug and error messages, be less verbose
- removed patch for making gntdev functions available to other drivers
- removed WARN_ON's in dma-buf code
- moved all dma-buf related code into gntdev-dmabuf
- introduced gntdev-common.h with common structures and function prototypes
- added additional checks for number of grants in IOCTLs
- gnttab patch cleanup
- made xenmem_reservation_scrub_page defined in the header as inline
- fixed __pfn_to_mfn use to pfn_to_bfn
- no changes to patches 1-2

Changes since v1:
*****************
- Define GNTDEV_DMA_FLAG_XXX starting from bit 0
- Rename mem_reservation.h to mem-reservation.h
- Remove usless comments
- Change licenses from GPLv2 OR MIT to GPLv2 only
- Make xenmem_reservation_va_mapping_{update|clear} inline
- Change EXPORT_SYMBOL to EXPORT_SYMBOL_GPL for new functions
- Make gnttab_dma_{alloc|free}_pages to request frames array
be allocated outside
- Fixe gnttab_dma_alloc_pages fail path (added xenmem_reservation_increase)
- Move most of dma-buf from gntdev.c to gntdev-dmabuf.c
- Add required dependencies to Kconfig
- Rework "#ifdef CONFIG_XEN_XXX" for if/else
- Export gnttab_{alloc|free}_pages as GPL symbols (patch 1)

Oleksandr Andrushchenko (9):
xen/grant-table: Export gnttab_{alloc|free}_pages as GPL xen/grant-table: Make set/clear page private code shared xen/balloon: Share common memory reservation routines xen/grant-table: Allow allocating buffers suitable for DMA xen/gntdev: Allow mappings for DMA buffers xen/gntdev: Make private routines/structures accessible xen/gntdev: Add initial support for dma-buf UAPI xen/gntdev: Implement dma-buf export functionality xen/gntdev: Implement dma-buf import functionality

 drivers/xen/Kconfig           |  23 +
 drivers/xen/Makefile          |   2 +
 drivers/xen/balloon.c         |  71 +---
 drivers/xen/gntdev-common.h   |  96 +++++
 drivers/xen/gntdev-dmabuf.c   | 774 ++++++++++++++++++++++++++++++++++
 drivers/xen/gntdev-dmabuf.h   |  39 ++
 drivers/xen/gntdev.c          | 329 ++++++++++++---
 drivers/xen/grant-table.c     | 153 ++++++-
 drivers/xen/mem-reservation.c | 112 +++++
 include/uapi/xen/gntdev.h     | 106 +++++
 include/xen/grant_table.h     |  21 +
 include/xen/mem-reservation.h |  64 +++
 12 files changed, 1635 insertions(+), 155 deletions(-)
 create mode 100644 drivers/xen/gntdev-common.h
 create mode 100644 drivers/xen/gntdev-dmabuf.c
 create mode 100644 drivers/xen/gntdev-dmabuf.h
 create mode 100644 drivers/xen/mem-reservation.c
 create mode 100644 include/xen/mem-reservation.h
Signature · 6 lines
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#1 lore raw

PATCH v3 1/9 xen/grant-table: Export gnttab_{alloc|free}_pages as GPL

From: Oleksandr Andrushchenko <[email protected]>

Only gnttab_{alloc|free}_pages are exported as EXPORT_SYMBOL while all the rest are exported as EXPORT_SYMBOL_GPL, thus effectively making it not possible for non-GPL driver modules to use grant table module. Export gnttab_{alloc|free}_pages as EXPORT_SYMBOL_GPL so all the exports are aligned.

---
 drivers/xen/grant-table.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/xen/grant-table.c +2 -2
index 27be107d6480..ba36ff3e4903 100644
@@ -799,7 +799,7 @@ int gnttab_alloc_pages(int nr_pages, struct page **pages)
 
 	return 0;
 }
-EXPORT_SYMBOL(gnttab_alloc_pages);
+EXPORT_SYMBOL_GPL(gnttab_alloc_pages);
 
 /**
  * gnttab_free_pages - free pages allocated by gnttab_alloc_pages()
@@ -820,7 +820,7 @@ void gnttab_free_pages(int nr_pages, struct page **pages)
 	}
 	free_xenballooned_pages(nr_pages, pages);
 }
-EXPORT_SYMBOL(gnttab_free_pages);
+EXPORT_SYMBOL_GPL(gnttab_free_pages);
 
 /* Handling of paged out grant targets (GNTST_eagain) */
 #define MAX_DELAY 256
Signature · 6 lines
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#2 lore raw

PATCH v3 1/9 xen/grant-table: Export gnttab_{alloc|free}_pages as GPL

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

Only gnttab_{alloc|free}_pages are exported as EXPORT_SYMBOL while all the rest are exported as EXPORT_SYMBOL_GPL, thus effectively making it not possible for non-GPL driver modules to use grant table module. Export gnttab_{alloc|free}_pages as EXPORT_SYMBOL_GPL so all the exports are aligned.

---
 drivers/xen/grant-table.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/xen/grant-table.c +2 -2
index 27be107d6480..ba36ff3e4903 100644
@@ -799,7 +799,7 @@ int gnttab_alloc_pages(int nr_pages, struct page **pages)
 
 	return 0;
 }
-EXPORT_SYMBOL(gnttab_alloc_pages);
+EXPORT_SYMBOL_GPL(gnttab_alloc_pages);
 
 /**
  * gnttab_free_pages - free pages allocated by gnttab_alloc_pages()
@@ -820,7 +820,7 @@ void gnttab_free_pages(int nr_pages, struct page **pages)
 	}
 	free_xenballooned_pages(nr_pages, pages);
 }
-EXPORT_SYMBOL(gnttab_free_pages);
+EXPORT_SYMBOL_GPL(gnttab_free_pages);
 
 /* Handling of paged out grant targets (GNTST_eagain) */
 #define MAX_DELAY 256
Signature · 7 lines
2.17.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#3 lore raw

PATCH v3 2/9 xen/grant-table: Make set/clear page private code shared

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

Make set/clear page private code shared and accessible to other kernel modules which can re-use these instead of open-coding.

---
 drivers/xen/grant-table.c | 54 +++++++++++++++++++++++++--------------
 include/xen/grant_table.h |  3 +++
 2 files changed, 38 insertions(+), 19 deletions(-)
drivers/xen/grant-table.c +35 -19
index ba36ff3e4903..dbb48a89e987 100644
@@ -769,29 +769,18 @@ void gnttab_free_auto_xlat_frames(void)
 }
 EXPORT_SYMBOL_GPL(gnttab_free_auto_xlat_frames);
 
-/**
- * gnttab_alloc_pages - alloc pages suitable for grant mapping into
- * @nr_pages: number of pages to alloc
- * @pages: returns the pages
- */
-int gnttab_alloc_pages(int nr_pages, struct page **pages)
+int gnttab_pages_set_private(int nr_pages, struct page **pages)
 {
 	int i;
-	int ret;
-
-	ret = alloc_xenballooned_pages(nr_pages, pages);
-	if (ret < 0)
-		return ret;
 
 	for (i = 0; i < nr_pages; i++) {
 #if BITS_PER_LONG < 64
 		struct xen_page_foreign *foreign;
 
 		foreign = kzalloc(sizeof(*foreign), GFP_KERNEL);
-		if (!foreign) {
-			gnttab_free_pages(nr_pages, pages);
+		if (!foreign)
 			return -ENOMEM;
-		}
+
 		set_page_private(pages[i], (unsigned long)foreign);
 #endif
 		SetPagePrivate(pages[i]);
@@ -799,14 +788,30 @@ int gnttab_alloc_pages(int nr_pages, struct page **pages)
 
 	return 0;
 }
-EXPORT_SYMBOL_GPL(gnttab_alloc_pages);
+EXPORT_SYMBOL_GPL(gnttab_pages_set_private);
 
 /**
- * gnttab_free_pages - free pages allocated by gnttab_alloc_pages()
- * @nr_pages; number of pages to free
- * @pages: the pages
+ * gnttab_alloc_pages - alloc pages suitable for grant mapping into
+ * @nr_pages: number of pages to alloc
+ * @pages: returns the pages
  */
-void gnttab_free_pages(int nr_pages, struct page **pages)
+int gnttab_alloc_pages(int nr_pages, struct page **pages)
+{
+	int ret;
+
+	ret = alloc_xenballooned_pages(nr_pages, pages);
+	if (ret < 0)
+		return ret;
+
+	ret = gnttab_pages_set_private(nr_pages, pages);
+	if (ret < 0)
+		gnttab_free_pages(nr_pages, pages);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(gnttab_alloc_pages);
+
+void gnttab_pages_clear_private(int nr_pages, struct page **pages)
 {
 	int i;
 
@@ -818,6 +823,17 @@ void gnttab_free_pages(int nr_pages, struct page **pages)
 			ClearPagePrivate(pages[i]);
 		}
 	}
+}
+EXPORT_SYMBOL_GPL(gnttab_pages_clear_private);
+
+/**
+ * gnttab_free_pages - free pages allocated by gnttab_alloc_pages()
+ * @nr_pages; number of pages to free
+ * @pages: the pages
+ */
+void gnttab_free_pages(int nr_pages, struct page **pages)
+{
+	gnttab_pages_clear_private(nr_pages, pages);
 	free_xenballooned_pages(nr_pages, pages);
 }
 EXPORT_SYMBOL_GPL(gnttab_free_pages);
include/xen/grant_table.h +3 -0
index 2e37741f6b8d..de03f2542bb7 100644
@@ -198,6 +198,9 @@ void gnttab_free_auto_xlat_frames(void);
 int gnttab_alloc_pages(int nr_pages, struct page **pages);
 void gnttab_free_pages(int nr_pages, struct page **pages);
 
+int gnttab_pages_set_private(int nr_pages, struct page **pages);
+void gnttab_pages_clear_private(int nr_pages, struct page **pages);
+
 int gnttab_map_refs(struct gnttab_map_grant_ref *map_ops,
 		    struct gnttab_map_grant_ref *kmap_ops,
 		    struct page **pages, unsigned int count);
Signature · 6 lines
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#4 lore raw

PATCH v3 2/9 xen/grant-table: Make set/clear page private code shared

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

Make set/clear page private code shared and accessible to other kernel modules which can re-use these instead of open-coding.

---
 drivers/xen/grant-table.c | 54 +++++++++++++++++++++++++--------------
 include/xen/grant_table.h |  3 +++
 2 files changed, 38 insertions(+), 19 deletions(-)
drivers/xen/grant-table.c +35 -19
index ba36ff3e4903..dbb48a89e987 100644
@@ -769,29 +769,18 @@ void gnttab_free_auto_xlat_frames(void)
 }
 EXPORT_SYMBOL_GPL(gnttab_free_auto_xlat_frames);
 
-/**
- * gnttab_alloc_pages - alloc pages suitable for grant mapping into
- * @nr_pages: number of pages to alloc
- * @pages: returns the pages
- */
-int gnttab_alloc_pages(int nr_pages, struct page **pages)
+int gnttab_pages_set_private(int nr_pages, struct page **pages)
 {
 	int i;
-	int ret;
-
-	ret = alloc_xenballooned_pages(nr_pages, pages);
-	if (ret < 0)
-		return ret;
 
 	for (i = 0; i < nr_pages; i++) {
 #if BITS_PER_LONG < 64
 		struct xen_page_foreign *foreign;
 
 		foreign = kzalloc(sizeof(*foreign), GFP_KERNEL);
-		if (!foreign) {
-			gnttab_free_pages(nr_pages, pages);
+		if (!foreign)
 			return -ENOMEM;
-		}
+
 		set_page_private(pages[i], (unsigned long)foreign);
 #endif
 		SetPagePrivate(pages[i]);
@@ -799,14 +788,30 @@ int gnttab_alloc_pages(int nr_pages, struct page **pages)
 
 	return 0;
 }
-EXPORT_SYMBOL_GPL(gnttab_alloc_pages);
+EXPORT_SYMBOL_GPL(gnttab_pages_set_private);
 
 /**
- * gnttab_free_pages - free pages allocated by gnttab_alloc_pages()
- * @nr_pages; number of pages to free
- * @pages: the pages
+ * gnttab_alloc_pages - alloc pages suitable for grant mapping into
+ * @nr_pages: number of pages to alloc
+ * @pages: returns the pages
  */
-void gnttab_free_pages(int nr_pages, struct page **pages)
+int gnttab_alloc_pages(int nr_pages, struct page **pages)
+{
+	int ret;
+
+	ret = alloc_xenballooned_pages(nr_pages, pages);
+	if (ret < 0)
+		return ret;
+
+	ret = gnttab_pages_set_private(nr_pages, pages);
+	if (ret < 0)
+		gnttab_free_pages(nr_pages, pages);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(gnttab_alloc_pages);
+
+void gnttab_pages_clear_private(int nr_pages, struct page **pages)
 {
 	int i;
 
@@ -818,6 +823,17 @@ void gnttab_free_pages(int nr_pages, struct page **pages)
 			ClearPagePrivate(pages[i]);
 		}
 	}
+}
+EXPORT_SYMBOL_GPL(gnttab_pages_clear_private);
+
+/**
+ * gnttab_free_pages - free pages allocated by gnttab_alloc_pages()
+ * @nr_pages; number of pages to free
+ * @pages: the pages
+ */
+void gnttab_free_pages(int nr_pages, struct page **pages)
+{
+	gnttab_pages_clear_private(nr_pages, pages);
 	free_xenballooned_pages(nr_pages, pages);
 }
 EXPORT_SYMBOL_GPL(gnttab_free_pages);
include/xen/grant_table.h +3 -0
index 2e37741f6b8d..de03f2542bb7 100644
@@ -198,6 +198,9 @@ void gnttab_free_auto_xlat_frames(void);
 int gnttab_alloc_pages(int nr_pages, struct page **pages);
 void gnttab_free_pages(int nr_pages, struct page **pages);
 
+int gnttab_pages_set_private(int nr_pages, struct page **pages);
+void gnttab_pages_clear_private(int nr_pages, struct page **pages);
+
 int gnttab_map_refs(struct gnttab_map_grant_ref *map_ops,
 		    struct gnttab_map_grant_ref *kmap_ops,
 		    struct page **pages, unsigned int count);
Signature · 7 lines
2.17.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#5 lore raw

PATCH v3 3/9 xen/balloon: Share common memory reservation routines

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

Memory {increase|decrease}_reservation and VA mappings update/reset code used in balloon driver can be made common, so other drivers can also re-use the same functionality without open-coding. Create a dedicated file for the shared code and export corresponding symbols for other kernel modules.

---
 drivers/xen/Makefile          |   1 +
 drivers/xen/balloon.c         |  71 ++-------------------
 drivers/xen/mem-reservation.c | 112 ++++++++++++++++++++++++++++++++++
 include/xen/mem-reservation.h |  64 +++++++++++++++++++
 4 files changed, 183 insertions(+), 65 deletions(-)
 create mode 100644 drivers/xen/mem-reservation.c
 create mode 100644 include/xen/mem-reservation.h
drivers/xen/Makefile +1 -0
index 451e833f5931..3c87b0c3aca6 100644
@@ -2,6 +2,7 @@
 obj-$(CONFIG_HOTPLUG_CPU)		+= cpu_hotplug.o
 obj-$(CONFIG_X86)			+= fallback.o
 obj-y	+= grant-table.o features.o balloon.o manage.o preempt.o time.o
+obj-y	+= mem-reservation.o
 obj-y	+= events/
 obj-y	+= xenbus/
 
drivers/xen/balloon.c +6 -65

126 lines not on this page · raw message

drivers/xen/mem-reservation.c new file +112 -0

112 lines not on this page · raw message

include/xen/mem-reservation.h new file +64 -0

64 lines not on this page · raw message

Signature · 6 lines
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#6 lore raw

Re: [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

Quoted 21 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

include/xen/mem-reservation.h new file +13 -0
new file mode 100644
index 000000000000..e0939387278d
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Xen memory reservation utilities.
+ *
+ * Copyright (c) 2003, B Dragovic
+ * Copyright (c) 2003-2004, M Williamson, K Fraser
+ * Copyright (c) 2005 Dan M. Smith, IBM Corporation
+ * Copyright (c) 2010 Daniel Kiper
+ * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _XENMEM_RESERVATION_H

include/xen/mem-reservation.h line 23

Quoted 10 lines
include/xen/mem-reservation.h new file +10 -0
+#define _XENMEM_RESERVATION_H
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include <asm/xen/hypercall.h>
+#include <asm/tlb.h>
+
+#include <xen/interface/memory.h>
+#include <xen/page.h>

I should have noticed this in the previous post but I suspect most of these includes belong in the C file. For example, there is no reason for hypercall.h here.

-boris

Quoted 42 lines
include/xen/mem-reservation.h new file +41 -0
+
+static inline void xenmem_reservation_scrub_page(struct page *page)
+{
+#ifdef CONFIG_XEN_SCRUB_PAGES
+	clear_highpage(page);
+#endif
+}
+
+#ifdef CONFIG_XEN_HAVE_PVMMU
+void __xenmem_reservation_va_mapping_update(unsigned long count,
+					    struct page **pages,
+					    xen_pfn_t *frames);
+
+void __xenmem_reservation_va_mapping_reset(unsigned long count,
+					   struct page **pages);
+#endif
+
+static inline void xenmem_reservation_va_mapping_update(unsigned long count,
+							struct page **pages,
+							xen_pfn_t *frames)
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+	if (!xen_feature(XENFEAT_auto_translated_physmap))
+		__xenmem_reservation_va_mapping_update(count, pages, frames);
+#endif
+}
+
+static inline void xenmem_reservation_va_mapping_reset(unsigned long count,
+						       struct page **pages)
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+	if (!xen_feature(XENFEAT_auto_translated_physmap))
+		__xenmem_reservation_va_mapping_reset(count, pages);
+#endif
+}
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames);
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames);
+
+#endif
#7 lore raw

Re: [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

Quoted 29 lines · Boris Ostrovsky
Quoted 26 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

include/xen/mem-reservation.h new file +18 -0
new file mode 100644
index 000000000000..e0939387278d
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Xen memory reservation utilities.
+ *
+ * Copyright (c) 2003, B Dragovic
+ * Copyright (c) 2003-2004, M Williamson, K Fraser
+ * Copyright (c) 2005 Dan M. Smith, IBM Corporation
+ * Copyright (c) 2010 Daniel Kiper
+ * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _XENMEM_RESERVATION_H
+#define _XENMEM_RESERVATION_H
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
Quoted 9 lines

include/xen/mem-reservation.h line 23

Quoted 5 lines
include/xen/mem-reservation.h new file +5 -0
+#include <asm/xen/hypercall.h>
+#include <asm/tlb.h>
+
+#include <xen/interface/memory.h>
+#include <xen/page.h>

I should have noticed this in the previous post but I suspect most of these includes belong in the C file. For example, there is no reason for hypercall.h here.

Yes, it seems that the header can only have
#include <xen/page.h>
Will move the rest into the .c file

Quoted 47 lines

-boris

Quoted 44 lines
include/xen/mem-reservation.h new file +18 -0
+
+static inline void xenmem_reservation_scrub_page(struct page *page)
+{
+#ifdef CONFIG_XEN_SCRUB_PAGES
+    clear_highpage(page);
+#endif
+}
+
+#ifdef CONFIG_XEN_HAVE_PVMMU
+void __xenmem_reservation_va_mapping_update(unsigned long count,
+                        struct page **pages,
+                        xen_pfn_t *frames);
+
+void __xenmem_reservation_va_mapping_reset(unsigned long count,
+                       struct page **pages);
+#endif
+
+static inline void xenmem_reservation_va_mapping_update(unsigned 

long count,
+                            struct page **pages,
+                            xen_pfn_t *frames)

include/xen/mem-reservation.h new file +8 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_update(count, pages, frames);
+#endif
+}
+
+static inline void xenmem_reservation_va_mapping_reset(unsigned long 

count,
+                               struct page **pages)

include/xen/mem-reservation.h new file +12 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_reset(count, pages);
+#endif
+}
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames);
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames);
+
+#endif
#8 lore raw

Re: [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

Quoted 39 lines · Oleksandr Andrushchenko
Quoted 29 lines · Boris Ostrovsky
Quoted 26 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

include/xen/mem-reservation.h new file +18 -0
new file mode 100644
index 000000000000..e0939387278d
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Xen memory reservation utilities.
+ *
+ * Copyright (c) 2003, B Dragovic
+ * Copyright (c) 2003-2004, M Williamson, K Fraser
+ * Copyright (c) 2005 Dan M. Smith, IBM Corporation
+ * Copyright (c) 2010 Daniel Kiper
+ * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _XENMEM_RESERVATION_H
+#define _XENMEM_RESERVATION_H
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
Quoted 9 lines

include/xen/mem-reservation.h line 23

Quoted 5 lines
include/xen/mem-reservation.h new file +5 -0
+#include <asm/xen/hypercall.h>
+#include <asm/tlb.h>
+
+#include <xen/interface/memory.h>
+#include <xen/page.h>

I should have noticed this in the previous post but I suspect most of these includes belong in the C file. For example, there is no reason for hypercall.h here.

Quoted 3 lines

Yes, it seems that the header can only have
#include <xen/page.h>
Will move the rest into the .c file

You may need something for clear_highpage() and maybe for Xen feature flags. But you'll find out for sure when you try to build. ;-)

-boris

Quoted 48 lines
Quoted 47 lines

-boris

Quoted 44 lines
include/xen/mem-reservation.h new file +18 -0
+
+static inline void xenmem_reservation_scrub_page(struct page *page)
+{
+#ifdef CONFIG_XEN_SCRUB_PAGES
+    clear_highpage(page);
+#endif
+}
+
+#ifdef CONFIG_XEN_HAVE_PVMMU
+void __xenmem_reservation_va_mapping_update(unsigned long count,
+                        struct page **pages,
+                        xen_pfn_t *frames);
+
+void __xenmem_reservation_va_mapping_reset(unsigned long count,
+                       struct page **pages);
+#endif
+
+static inline void xenmem_reservation_va_mapping_update(unsigned 

long count,
+                            struct page **pages,
+                            xen_pfn_t *frames)

include/xen/mem-reservation.h new file +8 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_update(count, pages, frames);
+#endif
+}
+
+static inline void xenmem_reservation_va_mapping_reset(unsigned long 

count,
+                               struct page **pages)

include/xen/mem-reservation.h new file +12 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_reset(count, pages);
+#endif
+}
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames);
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames);
+
+#endif
#9 lore raw

Re: [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

Quoted 42 lines · Boris Ostrovsky
Quoted 39 lines · Oleksandr Andrushchenko
Quoted 38 lines · Boris Ostrovsky
Quoted 21 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

include/xen/mem-reservation.h new file +13 -0
new file mode 100644
index 000000000000..e0939387278d
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Xen memory reservation utilities.
+ *
+ * Copyright (c) 2003, B Dragovic
+ * Copyright (c) 2003-2004, M Williamson, K Fraser
+ * Copyright (c) 2005 Dan M. Smith, IBM Corporation
+ * Copyright (c) 2010 Daniel Kiper
+ * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _XENMEM_RESERVATION_H

include/xen/mem-reservation.h line 23

Quoted 10 lines
include/xen/mem-reservation.h new file +10 -0
+#define _XENMEM_RESERVATION_H
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include <asm/xen/hypercall.h>
+#include <asm/tlb.h>
+
+#include <xen/interface/memory.h>
+#include <xen/page.h>

I should have noticed this in the previous post but I suspect most of these includes belong in the C file. For example, there is no reason for hypercall.h here.

Quoted 7 lines
Quoted 3 lines

Yes, it seems that the header can only have
#include <xen/page.h>
Will move the rest into the .c file

You may need something for clear_highpage() and maybe for Xen feature flags. But you'll find out for sure when you try to build. ;-)

#include <asm/tlb.h>

;)

Quoted 52 lines

-boris

Quoted 48 lines
Quoted 47 lines

-boris

Quoted 44 lines
include/xen/mem-reservation.h new file +18 -0
+
+static inline void xenmem_reservation_scrub_page(struct page *page)
+{
+#ifdef CONFIG_XEN_SCRUB_PAGES
+    clear_highpage(page);
+#endif
+}
+
+#ifdef CONFIG_XEN_HAVE_PVMMU
+void __xenmem_reservation_va_mapping_update(unsigned long count,
+                        struct page **pages,
+                        xen_pfn_t *frames);
+
+void __xenmem_reservation_va_mapping_reset(unsigned long count,
+                       struct page **pages);
+#endif
+
+static inline void xenmem_reservation_va_mapping_update(unsigned 

long count,
+                            struct page **pages,
+                            xen_pfn_t *frames)

include/xen/mem-reservation.h new file +8 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_update(count, pages, frames);
+#endif
+}
+
+static inline void xenmem_reservation_va_mapping_reset(unsigned 

long count,
+                               struct page **pages)

include/xen/mem-reservation.h new file +12 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_reset(count, pages);
+#endif
+}
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames);
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames);
+
+#endif
#10 lore raw

Re: [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

Quoted 43 lines · Oleksandr Andrushchenko
Quoted 42 lines · Boris Ostrovsky
Quoted 39 lines · Oleksandr Andrushchenko
Quoted 38 lines · Boris Ostrovsky
Quoted 21 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

include/xen/mem-reservation.h new file +13 -0
new file mode 100644
index 000000000000..e0939387278d
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Xen memory reservation utilities.
+ *
+ * Copyright (c) 2003, B Dragovic
+ * Copyright (c) 2003-2004, M Williamson, K Fraser
+ * Copyright (c) 2005 Dan M. Smith, IBM Corporation
+ * Copyright (c) 2010 Daniel Kiper
+ * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _XENMEM_RESERVATION_H

include/xen/mem-reservation.h line 23

Quoted 10 lines
include/xen/mem-reservation.h new file +10 -0
+#define _XENMEM_RESERVATION_H
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include <asm/xen/hypercall.h>
+#include <asm/tlb.h>
+
+#include <xen/interface/memory.h>
+#include <xen/page.h>

I should have noticed this in the previous post but I suspect most of these includes belong in the C file. For example, there is no reason for hypercall.h here.

Quoted 9 lines
Quoted 7 lines
Quoted 3 lines

Yes, it seems that the header can only have
#include <xen/page.h>
Will move the rest into the .c file

You may need something for clear_highpage() and maybe for Xen feature flags. But you'll find out for sure when you try to build. ;-)

#include <asm/tlb.h>

Or even
#include <linux/highmem.h>
according to [1]

Quoted 55 lines

;)

Quoted 53 lines

-boris

Quoted 49 lines
Quoted 48 lines

-boris

Quoted 45 lines
include/xen/mem-reservation.h new file +18 -0
+
+static inline void xenmem_reservation_scrub_page(struct page *page)
+{
+#ifdef CONFIG_XEN_SCRUB_PAGES
+    clear_highpage(page);
+#endif
+}
+
+#ifdef CONFIG_XEN_HAVE_PVMMU
+void __xenmem_reservation_va_mapping_update(unsigned long count,
+                        struct page **pages,
+                        xen_pfn_t *frames);
+
+void __xenmem_reservation_va_mapping_reset(unsigned long count,
+                       struct page **pages);
+#endif
+
+static inline void xenmem_reservation_va_mapping_update(unsigned 

long count,
+                            struct page **pages,
+                            xen_pfn_t *frames)

include/xen/mem-reservation.h new file +4 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_update(count, pages, 

frames);

include/xen/mem-reservation.h new file +4 -0
+#endif
+}
+
+static inline void xenmem_reservation_va_mapping_reset(unsigned 

long count,
+                               struct page **pages)

include/xen/mem-reservation.h new file +12 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_reset(count, pages);
+#endif
+}
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames);
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames);
+
+#endif

[1] https://elixir.bootlin.com/linux/v4.17.1/ident/clear_highpage

#11 lore raw

Re: [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

In reply to Oleksandr Andrushchenko
Quoted 43 lines · Oleksandr Andrushchenko
Quoted 42 lines · Boris Ostrovsky
Quoted 39 lines · Oleksandr Andrushchenko
Quoted 38 lines · Boris Ostrovsky
Quoted 21 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

include/xen/mem-reservation.h new file +13 -0
new file mode 100644
index 000000000000..e0939387278d
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Xen memory reservation utilities.
+ *
+ * Copyright (c) 2003, B Dragovic
+ * Copyright (c) 2003-2004, M Williamson, K Fraser
+ * Copyright (c) 2005 Dan M. Smith, IBM Corporation
+ * Copyright (c) 2010 Daniel Kiper
+ * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _XENMEM_RESERVATION_H

include/xen/mem-reservation.h line 23

Quoted 10 lines
include/xen/mem-reservation.h new file +10 -0
+#define _XENMEM_RESERVATION_H
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include <asm/xen/hypercall.h>
+#include <asm/tlb.h>
+
+#include <xen/interface/memory.h>
+#include <xen/page.h>

I should have noticed this in the previous post but I suspect most of these includes belong in the C file. For example, there is no reason for hypercall.h here.

Quoted 9 lines
Quoted 7 lines
Quoted 3 lines

Yes, it seems that the header can only have
#include <xen/page.h>
Will move the rest into the .c file

You may need something for clear_highpage() and maybe for Xen feature flags. But you'll find out for sure when you try to build. ;-)

#include <asm/tlb.h>

Or even
#include <linux/highmem.h>
according to [1]

Quoted 55 lines

;)

Quoted 53 lines

-boris

Quoted 49 lines
Quoted 48 lines

-boris

Quoted 45 lines
include/xen/mem-reservation.h new file +18 -0
+
+static inline void xenmem_reservation_scrub_page(struct page *page)
+{
+#ifdef CONFIG_XEN_SCRUB_PAGES
+    clear_highpage(page);
+#endif
+}
+
+#ifdef CONFIG_XEN_HAVE_PVMMU
+void __xenmem_reservation_va_mapping_update(unsigned long count,
+                        struct page **pages,
+                        xen_pfn_t *frames);
+
+void __xenmem_reservation_va_mapping_reset(unsigned long count,
+                       struct page **pages);
+#endif
+
+static inline void xenmem_reservation_va_mapping_update(unsigned 

long count,
+                            struct page **pages,
+                            xen_pfn_t *frames)

include/xen/mem-reservation.h new file +4 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_update(count, pages, 

frames);

include/xen/mem-reservation.h new file +4 -0
+#endif
+}
+
+static inline void xenmem_reservation_va_mapping_reset(unsigned 

long count,
+                               struct page **pages)

include/xen/mem-reservation.h new file +12 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_reset(count, pages);
+#endif
+}
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames);
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames);
+
+#endif

[1] https://elixir.bootlin.com/linux/v4.17.1/ident/clear_highpage

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#12 lore raw

Re: [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

In reply to Boris Ostrovsky
Quoted 42 lines · Boris Ostrovsky
Quoted 39 lines · Oleksandr Andrushchenko
Quoted 38 lines · Boris Ostrovsky
Quoted 21 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

include/xen/mem-reservation.h new file +13 -0
new file mode 100644
index 000000000000..e0939387278d
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Xen memory reservation utilities.
+ *
+ * Copyright (c) 2003, B Dragovic
+ * Copyright (c) 2003-2004, M Williamson, K Fraser
+ * Copyright (c) 2005 Dan M. Smith, IBM Corporation
+ * Copyright (c) 2010 Daniel Kiper
+ * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _XENMEM_RESERVATION_H

include/xen/mem-reservation.h line 23

Quoted 10 lines
include/xen/mem-reservation.h new file +10 -0
+#define _XENMEM_RESERVATION_H
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include <asm/xen/hypercall.h>
+#include <asm/tlb.h>
+
+#include <xen/interface/memory.h>
+#include <xen/page.h>

I should have noticed this in the previous post but I suspect most of these includes belong in the C file. For example, there is no reason for hypercall.h here.

Quoted 7 lines
Quoted 3 lines

Yes, it seems that the header can only have
#include <xen/page.h>
Will move the rest into the .c file

You may need something for clear_highpage() and maybe for Xen feature flags. But you'll find out for sure when you try to build. ;-)

#include <asm/tlb.h>

;)

Quoted 52 lines

-boris

Quoted 48 lines
Quoted 47 lines

-boris

Quoted 44 lines
include/xen/mem-reservation.h new file +18 -0
+
+static inline void xenmem_reservation_scrub_page(struct page *page)
+{
+#ifdef CONFIG_XEN_SCRUB_PAGES
+    clear_highpage(page);
+#endif
+}
+
+#ifdef CONFIG_XEN_HAVE_PVMMU
+void __xenmem_reservation_va_mapping_update(unsigned long count,
+                        struct page **pages,
+                        xen_pfn_t *frames);
+
+void __xenmem_reservation_va_mapping_reset(unsigned long count,
+                       struct page **pages);
+#endif
+
+static inline void xenmem_reservation_va_mapping_update(unsigned 

long count,
+                            struct page **pages,
+                            xen_pfn_t *frames)

include/xen/mem-reservation.h new file +8 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_update(count, pages, frames);
+#endif
+}
+
+static inline void xenmem_reservation_va_mapping_reset(unsigned 

long count,
+                               struct page **pages)

include/xen/mem-reservation.h new file +12 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_reset(count, pages);
+#endif
+}
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames);
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames);
+
+#endif

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#13 lore raw

Re: [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

In reply to Oleksandr Andrushchenko
Quoted 39 lines · Oleksandr Andrushchenko
Quoted 29 lines · Boris Ostrovsky
Quoted 26 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

include/xen/mem-reservation.h new file +18 -0
new file mode 100644
index 000000000000..e0939387278d
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Xen memory reservation utilities.
+ *
+ * Copyright (c) 2003, B Dragovic
+ * Copyright (c) 2003-2004, M Williamson, K Fraser
+ * Copyright (c) 2005 Dan M. Smith, IBM Corporation
+ * Copyright (c) 2010 Daniel Kiper
+ * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _XENMEM_RESERVATION_H
+#define _XENMEM_RESERVATION_H
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
Quoted 9 lines

include/xen/mem-reservation.h line 23

Quoted 5 lines
include/xen/mem-reservation.h new file +5 -0
+#include <asm/xen/hypercall.h>
+#include <asm/tlb.h>
+
+#include <xen/interface/memory.h>
+#include <xen/page.h>

I should have noticed this in the previous post but I suspect most of these includes belong in the C file. For example, there is no reason for hypercall.h here.

Quoted 3 lines

Yes, it seems that the header can only have
#include <xen/page.h>
Will move the rest into the .c file

You may need something for clear_highpage() and maybe for Xen feature flags. But you'll find out for sure when you try to build. ;-)

-boris

Quoted 39 lines
Quoted 39 lines

-boris

Quoted 36 lines
include/xen/mem-reservation.h new file +18 -0
+
+static inline void xenmem_reservation_scrub_page(struct page *page)
+{
+#ifdef CONFIG_XEN_SCRUB_PAGES
+    clear_highpage(page);
+#endif
+}
+
+#ifdef CONFIG_XEN_HAVE_PVMMU
+void __xenmem_reservation_va_mapping_update(unsigned long count,
+                        struct page **pages,
+                        xen_pfn_t *frames);
+
+void __xenmem_reservation_va_mapping_reset(unsigned long count,
+                       struct page **pages);
+#endif
+
+static inline void xenmem_reservation_va_mapping_update(unsigned 

long count,
+                            struct page **pages,
+                            xen_pfn_t *frames)

include/xen/mem-reservation.h new file +8 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_update(count, pages, frames);
+#endif
+}
+
+static inline void xenmem_reservation_va_mapping_reset(unsigned long 

count,
+                               struct page **pages)

include/xen/mem-reservation.h new file +5 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_reset(count, pages);
+#endif
Quoted 9 lines
Quoted 8 lines

include/xen/mem-reservation.h line 64

Quoted 8 lines
include/xen/mem-reservation.h new file +7 -0
+}
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames);
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames);
+
+#endif

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#14 lore raw

Re: [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

In reply to Boris Ostrovsky
Quoted 29 lines · Boris Ostrovsky
Quoted 26 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

include/xen/mem-reservation.h new file +18 -0
new file mode 100644
index 000000000000..e0939387278d
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Xen memory reservation utilities.
+ *
+ * Copyright (c) 2003, B Dragovic
+ * Copyright (c) 2003-2004, M Williamson, K Fraser
+ * Copyright (c) 2005 Dan M. Smith, IBM Corporation
+ * Copyright (c) 2010 Daniel Kiper
+ * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _XENMEM_RESERVATION_H
+#define _XENMEM_RESERVATION_H
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
Quoted 9 lines

include/xen/mem-reservation.h line 23

Quoted 5 lines
include/xen/mem-reservation.h new file +5 -0
+#include <asm/xen/hypercall.h>
+#include <asm/tlb.h>
+
+#include <xen/interface/memory.h>
+#include <xen/page.h>

I should have noticed this in the previous post but I suspect most of these includes belong in the C file. For example, there is no reason for hypercall.h here.

Yes, it seems that the header can only have
#include <xen/page.h>
Will move the rest into the .c file

Quoted 39 lines

-boris

Quoted 36 lines
include/xen/mem-reservation.h new file +18 -0
+
+static inline void xenmem_reservation_scrub_page(struct page *page)
+{
+#ifdef CONFIG_XEN_SCRUB_PAGES
+    clear_highpage(page);
+#endif
+}
+
+#ifdef CONFIG_XEN_HAVE_PVMMU
+void __xenmem_reservation_va_mapping_update(unsigned long count,
+                        struct page **pages,
+                        xen_pfn_t *frames);
+
+void __xenmem_reservation_va_mapping_reset(unsigned long count,
+                       struct page **pages);
+#endif
+
+static inline void xenmem_reservation_va_mapping_update(unsigned 

long count,
+                            struct page **pages,
+                            xen_pfn_t *frames)

include/xen/mem-reservation.h new file +8 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_update(count, pages, frames);
+#endif
+}
+
+static inline void xenmem_reservation_va_mapping_reset(unsigned long 

count,
+                               struct page **pages)

include/xen/mem-reservation.h new file +5 -0
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+        __xenmem_reservation_va_mapping_reset(count, pages);
+#endif
Quoted 8 lines

include/xen/mem-reservation.h line 64

Quoted 8 lines
include/xen/mem-reservation.h new file +7 -0
+}
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames);
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames);
+
+#endif

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#15 lore raw

Re: [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

In reply to Oleksandr Andrushchenko
Quoted 21 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

include/xen/mem-reservation.h new file +13 -0
new file mode 100644
index 000000000000..e0939387278d
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Xen memory reservation utilities.
+ *
+ * Copyright (c) 2003, B Dragovic
+ * Copyright (c) 2003-2004, M Williamson, K Fraser
+ * Copyright (c) 2005 Dan M. Smith, IBM Corporation
+ * Copyright (c) 2010 Daniel Kiper
+ * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _XENMEM_RESERVATION_H

include/xen/mem-reservation.h line 23

Quoted 10 lines
include/xen/mem-reservation.h new file +10 -0
+#define _XENMEM_RESERVATION_H
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include <asm/xen/hypercall.h>
+#include <asm/tlb.h>
+
+#include <xen/interface/memory.h>
+#include <xen/page.h>

I should have noticed this in the previous post but I suspect most of these includes belong in the C file. For example, there is no reason for hypercall.h here.

-boris

Quoted 32 lines
include/xen/mem-reservation.h new file +31 -0
+
+static inline void xenmem_reservation_scrub_page(struct page *page)
+{
+#ifdef CONFIG_XEN_SCRUB_PAGES
+	clear_highpage(page);
+#endif
+}
+
+#ifdef CONFIG_XEN_HAVE_PVMMU
+void __xenmem_reservation_va_mapping_update(unsigned long count,
+					    struct page **pages,
+					    xen_pfn_t *frames);
+
+void __xenmem_reservation_va_mapping_reset(unsigned long count,
+					   struct page **pages);
+#endif
+
+static inline void xenmem_reservation_va_mapping_update(unsigned long count,
+							struct page **pages,
+							xen_pfn_t *frames)
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+	if (!xen_feature(XENFEAT_auto_translated_physmap))
+		__xenmem_reservation_va_mapping_update(count, pages, frames);
+#endif
+}
+
+static inline void xenmem_reservation_va_mapping_reset(unsigned long count,
+						       struct page **pages)
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU

include/xen/mem-reservation.h line 64

Quoted 10 lines
include/xen/mem-reservation.h new file +10 -0
+	if (!xen_feature(XENFEAT_auto_translated_physmap))
+		__xenmem_reservation_va_mapping_reset(count, pages);
+#endif
+}
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames);
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames);
+
+#endif

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#16 lore raw

Re: [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

In reply to Oleksandr Andrushchenko

On 06/12/2018 09:41 AM, Oleksandr Andrushchenko wrote:

One more thing: please add a comment here saying that frames array is array of PFNs (in Xen granularity), which is what XENMEM_populate_physmap requires. And remove (or update to name the actual call you are making) the corresponding comment in increase_reservation().

Quoted 4 lines
drivers/xen/mem-reservation.c new file +4 -0
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames)
+{
+	struct xen_memory_reservation reservation = {

drivers/xen/mem-reservation.c line 98

Quoted 10 lines
drivers/xen/mem-reservation.c new file +10 -0
+		.address_bits = 0,
+		.extent_order = EXTENT_ORDER,
+		.domid        = DOMID_SELF
+	};
+
+	set_xen_guest_handle(reservation.extent_start, frames);
+	reservation.nr_extents = count;
+	return HYPERVISOR_memory_op(XENMEM_populate_physmap, &reservation);
+}
+EXPORT_SYMBOL_GPL(xenmem_reservation_increase);

And similarly, here we are requesting GFNs, and update decrease_reservation().

-boris

Quoted 12 lines
drivers/xen/mem-reservation.c new file +12 -0
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames)
+{
+	struct xen_memory_reservation reservation = {
+		.address_bits = 0,
+		.extent_order = EXTENT_ORDER,
+		.domid        = DOMID_SELF
+	};
+
+	set_xen_guest_handle(reservation.extent_start, frames);
+	reservation.nr_extents = count;
+	return HYPERVISOR_memory_op(XENMEM_decrease_reservation, &reservation);
#17 lore raw

Re: [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

Quoted 8 lines · Boris Ostrovsky

On 06/12/2018 09:41 AM, Oleksandr Andrushchenko wrote:

One more thing: please add a comment here saying that frames array is array of PFNs (in Xen granularity), which is what XENMEM_populate_physmap requires. And remove (or update to name the actual call you are making) the corresponding comment in increase_reservation().

I will remove corresponding comments from the balloon's {increase|decrease}_reservation and move those into xenmem_reservation{increase|decrease} where they belong now. I will also put a comment close to xenmem_reservation{increase|decrease}:

/* @frames is an array of PFNs */
int xenmem_reservation_increase(int count, xen_pfn_t *frames) {
    [...]
}

/* @frames is an array of GFNs */
int xenmem_reservation_decrease(int count, xen_pfn_t *frames) {
    [...]
}

Quoted 10 lines
Quoted 10 lines
drivers/xen/mem-reservation.c new file +9 -0
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames)
+{
+    struct xen_memory_reservation reservation = {
+        .address_bits = 0,
+        .extent_order = EXTENT_ORDER,
+        .domid        = DOMID_SELF
+    };
+
Quoted 9 lines

Review

Quoted 5 lines

+    set_xen_guest_handle(reservation.extent_start, frames); +    reservation.nr_extents = count;
+    return HYPERVISOR_memory_op(XENMEM_populate_physmap, &reservation);

(quoted diff) +2 -0
+}
+EXPORT_SYMBOL_GPL(xenmem_reservation_increase);

And similarly, here we are requesting GFNs, and update decrease_reservation().

Please see above

Quoted 3 lines

-boris

Thank you,
Oleksandr

Quoted 13 lines
Quoted 13 lines
drivers/xen/mem-reservation.c new file +12 -0
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames)
+{
+    struct xen_memory_reservation reservation = {
+        .address_bits = 0,
+        .extent_order = EXTENT_ORDER,
+        .domid        = DOMID_SELF
+    };
+
+    set_xen_guest_handle(reservation.extent_start, frames);
+    reservation.nr_extents = count;
+    return HYPERVISOR_memory_op(XENMEM_decrease_reservation, 

&reservation);

#18 lore raw

Re: [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

In reply to Boris Ostrovsky
Quoted 8 lines · Boris Ostrovsky

On 06/12/2018 09:41 AM, Oleksandr Andrushchenko wrote:

One more thing: please add a comment here saying that frames array is array of PFNs (in Xen granularity), which is what XENMEM_populate_physmap requires. And remove (or update to name the actual call you are making) the corresponding comment in increase_reservation().

I will remove corresponding comments from the balloon's {increase|decrease}_reservation and move those into xenmem_reservation{increase|decrease} where they belong now. I will also put a comment close to xenmem_reservation{increase|decrease}:

/* @frames is an array of PFNs */
int xenmem_reservation_increase(int count, xen_pfn_t *frames) {
    [...]
}

/* @frames is an array of GFNs */
int xenmem_reservation_decrease(int count, xen_pfn_t *frames) {
    [...]
}

Quoted 10 lines
Quoted 10 lines
drivers/xen/mem-reservation.c new file +9 -0
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames)
+{
+    struct xen_memory_reservation reservation = {
+        .address_bits = 0,
+        .extent_order = EXTENT_ORDER,
+        .domid        = DOMID_SELF
+    };
+
Quoted 9 lines

Review

Quoted 5 lines

+    set_xen_guest_handle(reservation.extent_start, frames); +    reservation.nr_extents = count;
+    return HYPERVISOR_memory_op(XENMEM_populate_physmap, &reservation);

(quoted diff) +2 -0
+}
+EXPORT_SYMBOL_GPL(xenmem_reservation_increase);

And similarly, here we are requesting GFNs, and update decrease_reservation().

Please see above

Quoted 3 lines

-boris

Thank you,
Oleksandr

Quoted 5 lines
Quoted 5 lines
drivers/xen/mem-reservation.c new file +5 -0
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames)
+{
+    struct xen_memory_reservation reservation = {
+        .address_bits = 0,
Quoted 8 lines
Quoted 8 lines

+        .extent_order = EXTENT_ORDER,
+        .domid        = DOMID_SELF
+    };
+
+    set_xen_guest_handle(reservation.extent_start, frames); +    reservation.nr_extents = count;
+    return HYPERVISOR_memory_op(XENMEM_decrease_reservation, &reservation);

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#19 lore raw

Re: [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

In reply to Oleksandr Andrushchenko

On 06/12/2018 09:41 AM, Oleksandr Andrushchenko wrote:

One more thing: please add a comment here saying that frames array is array of PFNs (in Xen granularity), which is what XENMEM_populate_physmap requires. And remove (or update to name the actual call you are making) the corresponding comment in increase_reservation().

Quoted 4 lines
drivers/xen/mem-reservation.c new file +4 -0
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames)
+{
+	struct xen_memory_reservation reservation = {

drivers/xen/mem-reservation.c line 98

Quoted 10 lines
drivers/xen/mem-reservation.c new file +10 -0
+		.address_bits = 0,
+		.extent_order = EXTENT_ORDER,
+		.domid        = DOMID_SELF
+	};
+
+	set_xen_guest_handle(reservation.extent_start, frames);
+	reservation.nr_extents = count;
+	return HYPERVISOR_memory_op(XENMEM_populate_physmap, &reservation);
+}
+EXPORT_SYMBOL_GPL(xenmem_reservation_increase);

And similarly, here we are requesting GFNs, and update decrease_reservation().

-boris

drivers/xen/mem-reservation.c line 110

Quoted 12 lines
drivers/xen/mem-reservation.c new file +12 -0
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames)
+{
+	struct xen_memory_reservation reservation = {
+		.address_bits = 0,
+		.extent_order = EXTENT_ORDER,
+		.domid        = DOMID_SELF
+	};
+
+	set_xen_guest_handle(reservation.extent_start, frames);
+	reservation.nr_extents = count;
+	return HYPERVISOR_memory_op(XENMEM_decrease_reservation, &reservation);

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#20 lore raw

PATCH v3 3/9 xen/balloon: Share common memory reservation routines

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

Memory {increase|decrease}_reservation and VA mappings update/reset code used in balloon driver can be made common, so other drivers can also re-use the same functionality without open-coding. Create a dedicated file for the shared code and export corresponding symbols for other kernel modules.

---
 drivers/xen/Makefile          |   1 +
 drivers/xen/balloon.c         |  71 ++-------------------
 drivers/xen/mem-reservation.c | 112 ++++++++++++++++++++++++++++++++++
 include/xen/mem-reservation.h |  64 +++++++++++++++++++
 4 files changed, 183 insertions(+), 65 deletions(-)
 create mode 100644 drivers/xen/mem-reservation.c
 create mode 100644 include/xen/mem-reservation.h
drivers/xen/Makefile +1 -0
index 451e833f5931..3c87b0c3aca6 100644
@@ -2,6 +2,7 @@
 obj-$(CONFIG_HOTPLUG_CPU)		+= cpu_hotplug.o
 obj-$(CONFIG_X86)			+= fallback.o
 obj-y	+= grant-table.o features.o balloon.o manage.o preempt.o time.o
+obj-y	+= mem-reservation.o
 obj-y	+= events/
 obj-y	+= xenbus/
 
drivers/xen/balloon.c +6 -65

126 lines not on this page · raw message

drivers/xen/mem-reservation.c new file +112 -0

112 lines not on this page · raw message

include/xen/mem-reservation.h new file +64 -0

64 lines not on this page · raw message

Signature · 7 lines
2.17.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#21 lore raw

PATCH v3 4/9 xen/grant-table: Allow allocating buffers suitable for DMA

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

Extend grant table module API to allow allocating buffers that can be used for DMA operations and mapping foreign grant references on top of those.
The resulting buffer is similar to the one allocated by the balloon driver in terms that proper memory reservation is made ({increase|decrease}_reservation and VA mappings updated if needed). This is useful for sharing foreign buffers with HW drivers which cannot work with scattered buffers provided by the balloon driver, but require DMAable memory instead.

---
 drivers/xen/Kconfig       | 13 ++++++
 drivers/xen/grant-table.c | 97 +++++++++++++++++++++++++++++++++++++++
 include/xen/grant_table.h | 18 ++++++++
 3 files changed, 128 insertions(+)
drivers/xen/Kconfig +13 -0
index e5d0c28372ea..39536ddfbce4 100644
@@ -161,6 +161,19 @@ config XEN_GRANT_DEV_ALLOC
 	  to other domains. This can be used to implement frontend drivers
 	  or as part of an inter-domain shared memory channel.
 
+config XEN_GRANT_DMA_ALLOC
+	bool "Allow allocating DMA capable buffers with grant reference module"
+	depends on XEN && HAS_DMA
+	help
+	  Extends grant table module API to allow allocating DMA capable
+	  buffers and mapping foreign grant references on top of it.
+	  The resulting buffer is similar to one allocated by the balloon
+	  driver in terms that proper memory reservation is made
+	  ({increase|decrease}_reservation and VA mappings updated if needed).
+	  This is useful for sharing foreign buffers with HW drivers which
+	  cannot work with scattered buffers provided by the balloon driver,
+	  but require DMAable memory instead.
+
 config SWIOTLB_XEN
 	def_bool y
 	select SWIOTLB
drivers/xen/grant-table.c +97 -0

115 lines not on this page · raw message

include/xen/grant_table.h +18 -0
index de03f2542bb7..9bc5bc07d4d3 100644
@@ -198,6 +198,24 @@ void gnttab_free_auto_xlat_frames(void);
 int gnttab_alloc_pages(int nr_pages, struct page **pages);
 void gnttab_free_pages(int nr_pages, struct page **pages);
 
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+struct gnttab_dma_alloc_args {
+	/* Device for which DMA memory will be/was allocated. */
+	struct device *dev;
+	/* If set then DMA buffer is coherent and write-combine otherwise. */
+	bool coherent;
+
+	int nr_pages;
+	struct page **pages;
+	xen_pfn_t *frames;
+	void *vaddr;
+	dma_addr_t dev_bus_addr;
+};
+
+int gnttab_dma_alloc_pages(struct gnttab_dma_alloc_args *args);
+int gnttab_dma_free_pages(struct gnttab_dma_alloc_args *args);
+#endif
+
 int gnttab_pages_set_private(int nr_pages, struct page **pages);
 void gnttab_pages_clear_private(int nr_pages, struct page **pages);
 
Signature · 6 lines
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Reviewed-by on v3 4/9
#22 lore raw

Re: [PATCH v3 4/9] xen/grant-table: Allow allocating buffers suitable for DMA

Quoted 12 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

Extend grant table module API to allow allocating buffers that can be used for DMA operations and mapping foreign grant references on top of those.
The resulting buffer is similar to the one allocated by the balloon driver in terms that proper memory reservation is made ({increase|decrease}_reservation and VA mappings updated if needed). This is useful for sharing foreign buffers with HW drivers which cannot work with scattered buffers provided by the balloon driver, but require DMAable memory instead.

Quoted 1 line

with a small nit below

Quoted 13 lines
---
  drivers/xen/Kconfig       | 13 ++++++
  drivers/xen/grant-table.c | 97 +++++++++++++++++++++++++++++++++++++++
  include/xen/grant_table.h | 18 ++++++++
  3 files changed, 128 insertions(+)
drivers/xen/Kconfig +0 -0
index e5d0c28372ea..39536ddfbce4 100644
@@ -161,6 +161,19 @@ config XEN_GRANT_DEV_ALLOC
  	  to other domains. This can be used to implement frontend drivers
  	  or as part of an inter-domain shared memory channel.

drivers/xen/Kconfig config XEN_GRANT_DEV_ALLOC · line 172

Quoted 10 lines
drivers/xen/Kconfig +9 -0
 
+config XEN_GRANT_DMA_ALLOC
+	bool "Allow allocating DMA capable buffers with grant reference module"
+	depends on XEN && HAS_DMA
+	help
+	  Extends grant table module API to allow allocating DMA capable
+	  buffers and mapping foreign grant references on top of it.
+	  The resulting buffer is similar to one allocated by the balloon
+	  driver in terms that proper memory reservation is made
+	  ({increase|decrease}_reservation and VA mappings updated if needed).

I think you should drop the word "terms" and say "is made *by*" and "VA mappings *are* updated"

And similar change in the commit message.

-boris

#23 lore raw

Re: [PATCH v3 4/9] xen/grant-table: Allow allocating buffers suitable for DMA

Quoted 49 lines · Boris Ostrovsky
Quoted 12 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

Extend grant table module API to allow allocating buffers that can be used for DMA operations and mapping foreign grant references on top of those.
The resulting buffer is similar to the one allocated by the balloon driver in terms that proper memory reservation is made ({increase|decrease}_reservation and VA mappings updated if needed). This is useful for sharing foreign buffers with HW drivers which cannot work with scattered buffers provided by the balloon driver, but require DMAable memory instead.

Quoted 1 line

with a small nit below

Quoted 16 lines

---
  drivers/xen/Kconfig       | 13 ++++++ drivers/xen/grant-table.c | 97 +++++++++++++++++++++++++++++++++++++++ include/xen/grant_table.h | 18 ++++++++

  3 files changed, 128 insertions(+)
drivers/xen/Kconfig +0 -0
index e5d0c28372ea..39536ddfbce4 100644
@@ -161,6 +161,19 @@ config XEN_GRANT_DEV_ALLOC

        to other domains. This can be used to implement frontend drivers or as part of an inter-domain shared memory channel. +config XEN_GRANT_DMA_ALLOC
+    bool "Allow allocating DMA capable buffers with grant reference module"

Quoted 8 lines

+    depends on XEN && HAS_DMA
+    help
+      Extends grant table module API to allow allocating DMA capable +      buffers and mapping foreign grant references on top of it. +      The resulting buffer is similar to one allocated by the balloon +      driver in terms that proper memory reservation is made +      ({increase|decrease}_reservation and VA mappings updated if needed).

I think you should drop the word "terms" and say "is made *by*" and "VA mappings *are* updated"

Quoted 2 lines

And similar change in the commit message.

Will, change, thank you

Quoted 2 lines

-boris

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#24 lore raw

Re: [PATCH v3 4/9] xen/grant-table: Allow allocating buffers suitable for DMA

In reply to Boris Ostrovsky
Quoted 49 lines · Boris Ostrovsky
Quoted 12 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

Extend grant table module API to allow allocating buffers that can be used for DMA operations and mapping foreign grant references on top of those.
The resulting buffer is similar to the one allocated by the balloon driver in terms that proper memory reservation is made ({increase|decrease}_reservation and VA mappings updated if needed). This is useful for sharing foreign buffers with HW drivers which cannot work with scattered buffers provided by the balloon driver, but require DMAable memory instead.

Quoted 1 line

with a small nit below

Quoted 16 lines

---
  drivers/xen/Kconfig       | 13 ++++++ drivers/xen/grant-table.c | 97 +++++++++++++++++++++++++++++++++++++++ include/xen/grant_table.h | 18 ++++++++

  3 files changed, 128 insertions(+)
drivers/xen/Kconfig +0 -0
index e5d0c28372ea..39536ddfbce4 100644
@@ -161,6 +161,19 @@ config XEN_GRANT_DEV_ALLOC

        to other domains. This can be used to implement frontend drivers or as part of an inter-domain shared memory channel. +config XEN_GRANT_DMA_ALLOC
+    bool "Allow allocating DMA capable buffers with grant reference module"

Quoted 8 lines

+    depends on XEN && HAS_DMA
+    help
+      Extends grant table module API to allow allocating DMA capable +      buffers and mapping foreign grant references on top of it. +      The resulting buffer is similar to one allocated by the balloon +      driver in terms that proper memory reservation is made +      ({increase|decrease}_reservation and VA mappings updated if needed).

I think you should drop the word "terms" and say "is made *by*" and "VA mappings *are* updated"

Quoted 2 lines

And similar change in the commit message.

Will, change, thank you

Quoted 2 lines

-boris

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Reviewed-by on v3 4/9
#25 lore raw

Re: [PATCH v3 4/9] xen/grant-table: Allow allocating buffers suitable for DMA

In reply to Oleksandr Andrushchenko
Quoted 12 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

Extend grant table module API to allow allocating buffers that can be used for DMA operations and mapping foreign grant references on top of those.
The resulting buffer is similar to the one allocated by the balloon driver in terms that proper memory reservation is made ({increase|decrease}_reservation and VA mappings updated if needed). This is useful for sharing foreign buffers with HW drivers which cannot work with scattered buffers provided by the balloon driver, but require DMAable memory instead.

Quoted 1 line

with a small nit below

Quoted 13 lines
---
  drivers/xen/Kconfig       | 13 ++++++
  drivers/xen/grant-table.c | 97 +++++++++++++++++++++++++++++++++++++++
  include/xen/grant_table.h | 18 ++++++++
  3 files changed, 128 insertions(+)
drivers/xen/Kconfig +0 -0
index e5d0c28372ea..39536ddfbce4 100644
@@ -161,6 +161,19 @@ config XEN_GRANT_DEV_ALLOC
  	  to other domains. This can be used to implement frontend drivers
  	  or as part of an inter-domain shared memory channel.

drivers/xen/Kconfig config XEN_GRANT_DEV_ALLOC · line 172

Quoted 10 lines
drivers/xen/Kconfig +9 -0
 
+config XEN_GRANT_DMA_ALLOC
+	bool "Allow allocating DMA capable buffers with grant reference module"
+	depends on XEN && HAS_DMA
+	help
+	  Extends grant table module API to allow allocating DMA capable
+	  buffers and mapping foreign grant references on top of it.
+	  The resulting buffer is similar to one allocated by the balloon
+	  driver in terms that proper memory reservation is made
+	  ({increase|decrease}_reservation and VA mappings updated if needed).

I think you should drop the word "terms" and say "is made *by*" and "VA mappings *are* updated"

And similar change in the commit message.

-boris

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#26 lore raw

PATCH v3 4/9 xen/grant-table: Allow allocating buffers suitable for DMA

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

Extend grant table module API to allow allocating buffers that can be used for DMA operations and mapping foreign grant references on top of those.
The resulting buffer is similar to the one allocated by the balloon driver in terms that proper memory reservation is made ({increase|decrease}_reservation and VA mappings updated if needed). This is useful for sharing foreign buffers with HW drivers which cannot work with scattered buffers provided by the balloon driver, but require DMAable memory instead.

---
 drivers/xen/Kconfig       | 13 ++++++
 drivers/xen/grant-table.c | 97 +++++++++++++++++++++++++++++++++++++++
 include/xen/grant_table.h | 18 ++++++++
 3 files changed, 128 insertions(+)
drivers/xen/Kconfig +13 -0
index e5d0c28372ea..39536ddfbce4 100644
@@ -161,6 +161,19 @@ config XEN_GRANT_DEV_ALLOC
 	  to other domains. This can be used to implement frontend drivers
 	  or as part of an inter-domain shared memory channel.
 
+config XEN_GRANT_DMA_ALLOC
+	bool "Allow allocating DMA capable buffers with grant reference module"
+	depends on XEN && HAS_DMA
+	help
+	  Extends grant table module API to allow allocating DMA capable
+	  buffers and mapping foreign grant references on top of it.
+	  The resulting buffer is similar to one allocated by the balloon
+	  driver in terms that proper memory reservation is made
+	  ({increase|decrease}_reservation and VA mappings updated if needed).
+	  This is useful for sharing foreign buffers with HW drivers which
+	  cannot work with scattered buffers provided by the balloon driver,
+	  but require DMAable memory instead.
+
 config SWIOTLB_XEN
 	def_bool y
 	select SWIOTLB
drivers/xen/grant-table.c +97 -0

115 lines not on this page · raw message

include/xen/grant_table.h +18 -0
index de03f2542bb7..9bc5bc07d4d3 100644
@@ -198,6 +198,24 @@ void gnttab_free_auto_xlat_frames(void);
 int gnttab_alloc_pages(int nr_pages, struct page **pages);
 void gnttab_free_pages(int nr_pages, struct page **pages);
 
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+struct gnttab_dma_alloc_args {
+	/* Device for which DMA memory will be/was allocated. */
+	struct device *dev;
+	/* If set then DMA buffer is coherent and write-combine otherwise. */
+	bool coherent;
+
+	int nr_pages;
+	struct page **pages;
+	xen_pfn_t *frames;
+	void *vaddr;
+	dma_addr_t dev_bus_addr;
+};
+
+int gnttab_dma_alloc_pages(struct gnttab_dma_alloc_args *args);
+int gnttab_dma_free_pages(struct gnttab_dma_alloc_args *args);
+#endif
+
 int gnttab_pages_set_private(int nr_pages, struct page **pages);
 void gnttab_pages_clear_private(int nr_pages, struct page **pages);
 
Signature · 7 lines
2.17.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#27 lore raw

PATCH v3 5/9 xen/gntdev: Allow mappings for DMA buffers

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

Allow mappings for DMA backed buffers if grant table module supports such: this extends grant device to not only map buffers made of balloon pages, but also from buffers allocated with dma_alloc_xxx.

---
 drivers/xen/gntdev.c      | 99 ++++++++++++++++++++++++++++++++++++++-
 include/uapi/xen/gntdev.h | 15 ++++++
 2 files changed, 112 insertions(+), 2 deletions(-)
drivers/xen/gntdev.c +97 -2
index bd56653b9bbc..d6b79ad1cd6f 100644
@@ -37,6 +37,9 @@
 #include <linux/slab.h>
 #include <linux/highmem.h>
 #include <linux/refcount.h>
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+#include <linux/of_device.h>
+#endif
 
 #include <xen/xen.h>
 #include <xen/grant_table.h>
@@ -72,6 +75,11 @@ struct gntdev_priv {
 	struct mutex lock;
 	struct mm_struct *mm;
 	struct mmu_notifier mn;
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	/* Device for which DMA memory is allocated. */
+	struct device *dma_dev;
+#endif
 };
 
 struct unmap_notify {
@@ -96,10 +104,27 @@ struct grant_map {
 	struct gnttab_unmap_grant_ref *kunmap_ops;
 	struct page **pages;
 	unsigned long pages_vm_start;
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	/*
+	 * If dmabuf_vaddr is not NULL then this mapping is backed by DMA
+	 * capable memory.
+	 */
+
+	struct device *dma_dev;
+	/* Flags used to create this DMA buffer: GNTDEV_DMA_FLAG_XXX. */
+	int dma_flags;
+	void *dma_vaddr;
+	dma_addr_t dma_bus_addr;
+	/* Needed to avoid allocation in gnttab_dma_free_pages(). */
+	xen_pfn_t *frames;
+#endif
 };
 
 static int unmap_grant_pages(struct grant_map *map, int offset, int pages);
 
+static struct miscdevice gntdev_miscdev;
+
 /* ------------------------------------------------------------------ */
 
 static void gntdev_print_maps(struct gntdev_priv *priv,
@@ -121,8 +146,27 @@ static void gntdev_free_map(struct grant_map *map)
 	if (map == NULL)
 		return;
 
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	if (map->dma_vaddr) {
+		struct gnttab_dma_alloc_args args;
+
+		args.dev = map->dma_dev;
+		args.coherent = map->dma_flags & GNTDEV_DMA_FLAG_COHERENT;
+		args.nr_pages = map->count;
+		args.pages = map->pages;
+		args.frames = map->frames;
+		args.vaddr = map->dma_vaddr;
+		args.dev_bus_addr = map->dma_bus_addr;
+
+		gnttab_dma_free_pages(&args);
+	} else
+#endif
 	if (map->pages)
 		gnttab_free_pages(map->count, map->pages);
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	kfree(map->frames);
+#endif
 	kfree(map->pages);
 	kfree(map->grants);
 	kfree(map->map_ops);
@@ -132,7 +176,8 @@ static void gntdev_free_map(struct grant_map *map)
 	kfree(map);
 }
 
-static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count)
+static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count,
+					  int dma_flags)
 {
 	struct grant_map *add;
 	int i;
@@ -155,6 +200,37 @@ static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count)
 	    NULL == add->pages)
 		goto err;
 
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	add->dma_flags = dma_flags;
+
+	/*
+	 * Check if this mapping is requested to be backed
+	 * by a DMA buffer.
+	 */
+	if (dma_flags & (GNTDEV_DMA_FLAG_WC | GNTDEV_DMA_FLAG_COHERENT)) {
+		struct gnttab_dma_alloc_args args;
+
+		add->frames = kcalloc(count, sizeof(add->frames[0]),
+				      GFP_KERNEL);
+		if (!add->frames)
+			goto err;
+
+		/* Remember the device, so we can free DMA memory. */
+		add->dma_dev = priv->dma_dev;
+
+		args.dev = priv->dma_dev;
+		args.coherent = dma_flags & GNTDEV_DMA_FLAG_COHERENT;
+		args.nr_pages = count;
+		args.pages = add->pages;
+		args.frames = add->frames;
+
+		if (gnttab_dma_alloc_pages(&args))
+			goto err;
+
+		add->dma_vaddr = args.vaddr;
+		add->dma_bus_addr = args.dev_bus_addr;
+	} else
+#endif
 	if (gnttab_alloc_pages(count, add->pages))
 		goto err;
 
@@ -325,6 +401,14 @@ static int map_grant_pages(struct grant_map *map)
 		map->unmap_ops[i].handle = map->map_ops[i].handle;
 		if (use_ptemod)
 			map->kunmap_ops[i].handle = map->kmap_ops[i].handle;
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+		else if (map->dma_vaddr) {
+			unsigned long mfn;
+
+			mfn = pfn_to_bfn(page_to_pfn(map->pages[i]));
+			map->unmap_ops[i].dev_bus_addr = __pfn_to_phys(mfn);
+		}
+#endif
 	}
 	return err;
 }
@@ -548,6 +632,17 @@ static int gntdev_open(struct inode *inode, struct file *flip)
 	}
 
 	flip->private_data = priv;
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	priv->dma_dev = gntdev_miscdev.this_device;
+
+	/*
+	 * The device is not spawn from a device tree, so arch_setup_dma_ops
+	 * is not called, thus leaving the device with dummy DMA ops.
+	 * Fix this call of_dma_configure() with a NULL node to set
+	 * default DMA ops.
+	 */
+	of_dma_configure(priv->dma_dev, NULL);
+#endif
 	pr_debug("priv %p\n", priv);
 
 	return 0;
@@ -589,7 +684,7 @@ static long gntdev_ioctl_map_grant_ref(struct gntdev_priv *priv,
 		return -EINVAL;
 
 	err = -ENOMEM;
-	map = gntdev_alloc_map(priv, op.count);
+	map = gntdev_alloc_map(priv, op.count, 0 /* This is not a dma-buf. */);
 	if (!map)
 		return err;
 
include/uapi/xen/gntdev.h +15 -0
index 6d1163456c03..4b9d498a31d4 100644
@@ -200,4 +200,19 @@ struct ioctl_gntdev_grant_copy {
 /* Send an interrupt on the indicated event channel */
 #define UNMAP_NOTIFY_SEND_EVENT 0x2
 
+/*
+ * Flags to be used while requesting memory mapping's backing storage
+ * to be allocated with DMA API.
+ */
+
+/*
+ * The buffer is backed with memory allocated with dma_alloc_wc.
+ */
+#define GNTDEV_DMA_FLAG_WC		(1 << 0)
+
+/*
+ * The buffer is backed with memory allocated with dma_alloc_coherent.
+ */
+#define GNTDEV_DMA_FLAG_COHERENT	(1 << 1)
+
 #endif /* __LINUX_PUBLIC_GNTDEV_H__ */
Signature · 6 lines
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Reviewed-by on v3 5/9
#28 lore raw

Re: [PATCH v3 5/9] xen/gntdev: Allow mappings for DMA buffers

drivers/xen/gntdev.c static void gntdev_free_map(struct grant_map *map) · line 154

Quoted 12 lines · Oleksandr Andrushchenko

static void gntdev_print_maps(struct gntdev_priv *priv,

drivers/xen/gntdev.c +6 -0
@@ -121,8 +146,27 @@ static void gntdev_free_map(struct grant_map *map)
  	if (map == NULL)
  		return;
 
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	if (map->dma_vaddr) {
+		struct gnttab_dma_alloc_args args;
+
+		args.dev = map->dma_dev;
+		args.coherent = map->dma_flags & GNTDEV_DMA_FLAG_COHERENT;

args.coherent = !!(map->dma_flags & GNTDEV_DMA_FLAG_COHERENT);

Quoted 42 lines
drivers/xen/gntdev.c +25 -1
+		args.nr_pages = map->count;
+		args.pages = map->pages;
+		args.frames = map->frames;
+		args.vaddr = map->dma_vaddr;
+		args.dev_bus_addr = map->dma_bus_addr;
+
+		gnttab_dma_free_pages(&args);
+	} else
+#endif
  	if (map->pages)
  		gnttab_free_pages(map->count, map->pages);
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	kfree(map->frames);
+#endif
  	kfree(map->pages);
  	kfree(map->grants);
  	kfree(map->map_ops);
@@ -132,7 +176,8 @@ static void gntdev_free_map(struct grant_map *map)
  	kfree(map);
  }
 
-static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count)
+static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count,
+					  int dma_flags)
  {
  	struct grant_map *add;
  	int i;
@@ -155,6 +200,37 @@ static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count)
  	    NULL == add->pages)
  		goto err;
 
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	add->dma_flags = dma_flags;
+
+	/*
+	 * Check if this mapping is requested to be backed
+	 * by a DMA buffer.
+	 */
+	if (dma_flags & (GNTDEV_DMA_FLAG_WC | GNTDEV_DMA_FLAG_COHERENT)) {
+		struct gnttab_dma_alloc_args args;
+

drivers/xen/gntdev.c static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count) · line 222

Quoted 10 lines
drivers/xen/gntdev.c +10 -0
+		add->frames = kcalloc(count, sizeof(add->frames[0]),
+				      GFP_KERNEL);
+		if (!add->frames)
+			goto err;
+
+		/* Remember the device, so we can free DMA memory. */
+		add->dma_dev = priv->dma_dev;
+
+		args.dev = priv->dma_dev;
+		args.coherent = dma_flags & GNTDEV_DMA_FLAG_COHERENT;

And again here.

Quoted 15 lines
drivers/xen/gntdev.c +11 -0
+		args.nr_pages = count;
+		args.pages = add->pages;
+		args.frames = add->frames;
+
+		if (gnttab_dma_alloc_pages(&args))
+			goto err;
+
+		add->dma_vaddr = args.vaddr;
+		add->dma_bus_addr = args.dev_bus_addr;
+	} else
+#endif
  	if (gnttab_alloc_pages(count, add->pages))
  		goto err;
 

drivers/xen/gntdev.c static int map_grant_pages(struct grant_map *map) · line 406

Quoted 6 lines
drivers/xen/gntdev.c +3 -0
@@ -325,6 +401,14 @@ static int map_grant_pages(struct grant_map *map)
  		map->unmap_ops[i].handle = map->map_ops[i].handle;
  		if (use_ptemod)
  			map->kunmap_ops[i].handle = map->kmap_ops[i].handle;
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+		else if (map->dma_vaddr) {
+			unsigned long mfn;

This should be called bfn now.

Quoted 9 lines
drivers/xen/gntdev.c +5 -0
+
+			mfn = pfn_to_bfn(page_to_pfn(map->pages[i]));
+			map->unmap_ops[i].dev_bus_addr = __pfn_to_phys(mfn);
+		}
+#endif
  	}
  	return err;
  }

drivers/xen/gntdev.c static int gntdev_open(struct inode *inode, struct file *flip) · line 641

Quoted 10 lines
drivers/xen/gntdev.c +7 -0
@@ -548,6 +632,17 @@ static int gntdev_open(struct inode *inode, struct file *flip)
  	}
 
  	flip->private_data = priv;
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	priv->dma_dev = gntdev_miscdev.this_device;
+
+	/*
+	 * The device is not spawn from a device tree, so arch_setup_dma_ops
+	 * is not called, thus leaving the device with dummy DMA ops.
+	 * Fix this call of_dma_configure() with a NULL node to set

"Fix this by calling ..." I think.

drivers/xen/gntdev.c static int gntdev_open(struct inode *inode, struct file *flip) · line 648

Quoted 7 lines
drivers/xen/gntdev.c +4 -0
+	 * default DMA ops.
+	 */
+	of_dma_configure(priv->dma_dev, NULL);
+#endif
  	pr_debug("priv %p\n", priv);
 
  	return 0;

With those fixed,

#29 lore raw

Re: [PATCH v3 5/9] xen/gntdev: Allow mappings for DMA buffers

Quoted 4 lines · Boris Ostrovsky

On 06/12/2018 09:41 AM, Oleksandr Andrushchenko wrote:

Quoted 14 lines
Quoted 10 lines

    static void gntdev_print_maps(struct gntdev_priv *priv,

drivers/xen/gntdev.c +0 -0
@@ -121,8 +146,27 @@ static void gntdev_free_map(struct grant_map *map)
       if (map == NULL)
           return;

  +#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+    if (map->dma_vaddr) {
+        struct gnttab_dma_alloc_args args;
+
+        args.dev = map->dma_dev;
+        args.coherent = map->dma_flags & GNTDEV_DMA_FLAG_COHERENT;

args.coherent = !!(map->dma_flags & GNTDEV_DMA_FLAG_COHERENT);

Will fix

Quoted 48 lines
Quoted 48 lines

+        args.nr_pages = map->count;
+        args.pages = map->pages;
+        args.frames = map->frames;
+        args.vaddr = map->dma_vaddr;
+        args.dev_bus_addr = map->dma_bus_addr;

drivers/xen/gntdev.c +4 -0
+
+        gnttab_dma_free_pages(&args);
+    } else
+#endif

      if (map->pages) gnttab_free_pages(map->count, map->pages);

drivers/xen/gntdev.c +4 -0
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+    kfree(map->frames);
+#endif

      kfree(map->pages);
      kfree(map->grants);
      kfree(map->map_ops);

drivers/xen/gntdev.c +2 -0
@@ -132,7 +176,8 @@ static void gntdev_free_map(struct grant_map *map)
       kfree(map);
   }
   -static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count)
+static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, 
 int count,
+                      int dma_flags)
   {
       struct grant_map *add;

      int i;

drivers/xen/gntdev.c +0 -0
@@ -155,6 +200,37 @@ static struct grant_map *gntdev_alloc_map(struct 
 gntdev_priv *priv, int count)
           NULL == add->pages)
           goto err;

  +#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+    add->dma_flags = dma_flags;

drivers/xen/gntdev.c +13 -0
+
+    /*
+     * Check if this mapping is requested to be backed
+     * by a DMA buffer.
+     */
+    if (dma_flags & (GNTDEV_DMA_FLAG_WC | GNTDEV_DMA_FLAG_COHERENT)) {
+        struct gnttab_dma_alloc_args args;
+
+        add->frames = kcalloc(count, sizeof(add->frames[0]),
+                      GFP_KERNEL);
+        if (!add->frames)
+            goto err;
+
Quoted 9 lines
Quoted 5 lines

+        /* Remember the device, so we can free DMA memory. */ +        add->dma_dev = priv->dma_dev;
+
+        args.dev = priv->dma_dev;
+        args.coherent = dma_flags & GNTDEV_DMA_FLAG_COHERENT;

And again here.

Will fix

Quoted 16 lines
Quoted 16 lines

+        args.nr_pages = count;
+        args.pages = add->pages;
+        args.frames = add->frames;

drivers/xen/gntdev.c +8 -0
+
+        if (gnttab_dma_alloc_pages(&args))
+            goto err;
+
+        add->dma_vaddr = args.vaddr;
+        add->dma_bus_addr = args.dev_bus_addr;
+    } else
+#endif

      if (gnttab_alloc_pages(count, add->pages))
          goto err;
  @@ -325,6 +401,14 @@ static int map_grant_pages(struct grant_map *map) map->unmap_ops[i].handle = map->map_ops[i].handle;

Quoted 9 lines
Quoted 5 lines

          if (use_ptemod)
              map->kunmap_ops[i].handle = map->kmap_ops[i].handle; +#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+        else if (map->dma_vaddr) {
+            unsigned long mfn;

This should be called bfn now.

Of course

Quoted 16 lines
Quoted 16 lines
drivers/xen/gntdev.c +5 -0
+
+            mfn = pfn_to_bfn(page_to_pfn(map->pages[i]));
+            map->unmap_ops[i].dev_bus_addr = __pfn_to_phys(mfn);
+        }
+#endif

      }
      return err;
  }

drivers/xen/gntdev.c +3 -0
@@ -548,6 +632,17 @@ static int gntdev_open(struct inode *inode, 
 struct file *flip)
       }
         flip->private_data = priv;
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+    priv->dma_dev = gntdev_miscdev.this_device;
+
Quoted 9 lines
Quoted 5 lines
(quoted diff) +2 -0
+    /*
+     * The device is not spawn from a device tree, so 

arch_setup_dma_ops
+     * is not called, thus leaving the device with dummy DMA ops. +     * Fix this call of_dma_configure() with a NULL node to set

"Fix this by calling ..." I think.

Will fix

Quoted 9 lines
Quoted 7 lines

+     * default DMA ops.
+     */
+    of_dma_configure(priv->dma_dev, NULL);
+#endif
      pr_debug("priv %p\n", priv);
        return 0;

Quoted 3 lines

With those fixed,

Thank you,
Oleksandr

#30 lore raw

Re: [PATCH v3 5/9] xen/gntdev: Allow mappings for DMA buffers

In reply to Boris Ostrovsky
Quoted 4 lines · Boris Ostrovsky

On 06/12/2018 09:41 AM, Oleksandr Andrushchenko wrote:

Quoted 14 lines
Quoted 10 lines

    static void gntdev_print_maps(struct gntdev_priv *priv,

drivers/xen/gntdev.c +0 -0
@@ -121,8 +146,27 @@ static void gntdev_free_map(struct grant_map *map)
       if (map == NULL)
           return;

  +#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+    if (map->dma_vaddr) {
+        struct gnttab_dma_alloc_args args;
+
+        args.dev = map->dma_dev;
+        args.coherent = map->dma_flags & GNTDEV_DMA_FLAG_COHERENT;

args.coherent = !!(map->dma_flags & GNTDEV_DMA_FLAG_COHERENT);

Will fix

Quoted 48 lines
Quoted 48 lines

+        args.nr_pages = map->count;
+        args.pages = map->pages;
+        args.frames = map->frames;
+        args.vaddr = map->dma_vaddr;
+        args.dev_bus_addr = map->dma_bus_addr;

drivers/xen/gntdev.c +4 -0
+
+        gnttab_dma_free_pages(&args);
+    } else
+#endif

      if (map->pages) gnttab_free_pages(map->count, map->pages);

drivers/xen/gntdev.c +4 -0
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+    kfree(map->frames);
+#endif

      kfree(map->pages);
      kfree(map->grants);
      kfree(map->map_ops);

drivers/xen/gntdev.c +2 -0
@@ -132,7 +176,8 @@ static void gntdev_free_map(struct grant_map *map)
       kfree(map);
   }
   -static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count)
+static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, 
 int count,
+                      int dma_flags)
   {
       struct grant_map *add;

      int i;

drivers/xen/gntdev.c +0 -0
@@ -155,6 +200,37 @@ static struct grant_map *gntdev_alloc_map(struct 
 gntdev_priv *priv, int count)
           NULL == add->pages)
           goto err;

  +#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+    add->dma_flags = dma_flags;

drivers/xen/gntdev.c +13 -0
+
+    /*
+     * Check if this mapping is requested to be backed
+     * by a DMA buffer.
+     */
+    if (dma_flags & (GNTDEV_DMA_FLAG_WC | GNTDEV_DMA_FLAG_COHERENT)) {
+        struct gnttab_dma_alloc_args args;
+
+        add->frames = kcalloc(count, sizeof(add->frames[0]),
+                      GFP_KERNEL);
+        if (!add->frames)
+            goto err;
+
Quoted 9 lines
Quoted 5 lines

+        /* Remember the device, so we can free DMA memory. */ +        add->dma_dev = priv->dma_dev;
+
+        args.dev = priv->dma_dev;
+        args.coherent = dma_flags & GNTDEV_DMA_FLAG_COHERENT;

And again here.

Will fix

Quoted 16 lines
Quoted 16 lines

+        args.nr_pages = count;
+        args.pages = add->pages;
+        args.frames = add->frames;

drivers/xen/gntdev.c +8 -0
+
+        if (gnttab_dma_alloc_pages(&args))
+            goto err;
+
+        add->dma_vaddr = args.vaddr;
+        add->dma_bus_addr = args.dev_bus_addr;
+    } else
+#endif

      if (gnttab_alloc_pages(count, add->pages))
          goto err;
  @@ -325,6 +401,14 @@ static int map_grant_pages(struct grant_map *map) map->unmap_ops[i].handle = map->map_ops[i].handle;

Quoted 9 lines
Quoted 5 lines

          if (use_ptemod)
              map->kunmap_ops[i].handle = map->kmap_ops[i].handle; +#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+        else if (map->dma_vaddr) {
+            unsigned long mfn;

This should be called bfn now.

Of course

Quoted 16 lines
Quoted 16 lines
drivers/xen/gntdev.c +5 -0
+
+            mfn = pfn_to_bfn(page_to_pfn(map->pages[i]));
+            map->unmap_ops[i].dev_bus_addr = __pfn_to_phys(mfn);
+        }
+#endif

      }
      return err;
  }

drivers/xen/gntdev.c +3 -0
@@ -548,6 +632,17 @@ static int gntdev_open(struct inode *inode, 
 struct file *flip)
       }
         flip->private_data = priv;
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+    priv->dma_dev = gntdev_miscdev.this_device;
+
Quoted 9 lines
Quoted 5 lines
(quoted diff) +2 -0
+    /*
+     * The device is not spawn from a device tree, so 

arch_setup_dma_ops
+     * is not called, thus leaving the device with dummy DMA ops. +     * Fix this call of_dma_configure() with a NULL node to set

"Fix this by calling ..." I think.

Will fix

Quoted 9 lines
Quoted 7 lines

+     * default DMA ops.
+     */
+    of_dma_configure(priv->dma_dev, NULL);
+#endif
      pr_debug("priv %p\n", priv);
        return 0;

Quoted 3 lines

With those fixed,

Thank you,
Oleksandr

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Reviewed-by on v3 5/9
#31 lore raw

Re: [PATCH v3 5/9] xen/gntdev: Allow mappings for DMA buffers

In reply to Oleksandr Andrushchenko

drivers/xen/gntdev.c static void gntdev_free_map(struct grant_map *map) · line 154

Quoted 12 lines · Oleksandr Andrushchenko

static void gntdev_print_maps(struct gntdev_priv *priv,

drivers/xen/gntdev.c +6 -0
@@ -121,8 +146,27 @@ static void gntdev_free_map(struct grant_map *map)
  	if (map == NULL)
  		return;
 
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	if (map->dma_vaddr) {
+		struct gnttab_dma_alloc_args args;
+
+		args.dev = map->dma_dev;
+		args.coherent = map->dma_flags & GNTDEV_DMA_FLAG_COHERENT;

args.coherent = !!(map->dma_flags & GNTDEV_DMA_FLAG_COHERENT);

Quoted 42 lines
drivers/xen/gntdev.c +25 -1
+		args.nr_pages = map->count;
+		args.pages = map->pages;
+		args.frames = map->frames;
+		args.vaddr = map->dma_vaddr;
+		args.dev_bus_addr = map->dma_bus_addr;
+
+		gnttab_dma_free_pages(&args);
+	} else
+#endif
  	if (map->pages)
  		gnttab_free_pages(map->count, map->pages);
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	kfree(map->frames);
+#endif
  	kfree(map->pages);
  	kfree(map->grants);
  	kfree(map->map_ops);
@@ -132,7 +176,8 @@ static void gntdev_free_map(struct grant_map *map)
  	kfree(map);
  }
 
-static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count)
+static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count,
+					  int dma_flags)
  {
  	struct grant_map *add;
  	int i;
@@ -155,6 +200,37 @@ static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count)
  	    NULL == add->pages)
  		goto err;
 
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	add->dma_flags = dma_flags;
+
+	/*
+	 * Check if this mapping is requested to be backed
+	 * by a DMA buffer.
+	 */
+	if (dma_flags & (GNTDEV_DMA_FLAG_WC | GNTDEV_DMA_FLAG_COHERENT)) {
+		struct gnttab_dma_alloc_args args;
+

drivers/xen/gntdev.c static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count) · line 222

Quoted 10 lines
drivers/xen/gntdev.c +10 -0
+		add->frames = kcalloc(count, sizeof(add->frames[0]),
+				      GFP_KERNEL);
+		if (!add->frames)
+			goto err;
+
+		/* Remember the device, so we can free DMA memory. */
+		add->dma_dev = priv->dma_dev;
+
+		args.dev = priv->dma_dev;
+		args.coherent = dma_flags & GNTDEV_DMA_FLAG_COHERENT;

And again here.

Quoted 15 lines
drivers/xen/gntdev.c +11 -0
+		args.nr_pages = count;
+		args.pages = add->pages;
+		args.frames = add->frames;
+
+		if (gnttab_dma_alloc_pages(&args))
+			goto err;
+
+		add->dma_vaddr = args.vaddr;
+		add->dma_bus_addr = args.dev_bus_addr;
+	} else
+#endif
  	if (gnttab_alloc_pages(count, add->pages))
  		goto err;
 

drivers/xen/gntdev.c static int map_grant_pages(struct grant_map *map) · line 406

Quoted 6 lines
drivers/xen/gntdev.c +3 -0
@@ -325,6 +401,14 @@ static int map_grant_pages(struct grant_map *map)
  		map->unmap_ops[i].handle = map->map_ops[i].handle;
  		if (use_ptemod)
  			map->kunmap_ops[i].handle = map->kmap_ops[i].handle;
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+		else if (map->dma_vaddr) {
+			unsigned long mfn;

This should be called bfn now.

Quoted 9 lines
drivers/xen/gntdev.c +5 -0
+
+			mfn = pfn_to_bfn(page_to_pfn(map->pages[i]));
+			map->unmap_ops[i].dev_bus_addr = __pfn_to_phys(mfn);
+		}
+#endif
  	}
  	return err;
  }

drivers/xen/gntdev.c static int gntdev_open(struct inode *inode, struct file *flip) · line 641

Quoted 10 lines
drivers/xen/gntdev.c +7 -0
@@ -548,6 +632,17 @@ static int gntdev_open(struct inode *inode, struct file *flip)
  	}
 
  	flip->private_data = priv;
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	priv->dma_dev = gntdev_miscdev.this_device;
+
+	/*
+	 * The device is not spawn from a device tree, so arch_setup_dma_ops
+	 * is not called, thus leaving the device with dummy DMA ops.
+	 * Fix this call of_dma_configure() with a NULL node to set

"Fix this by calling ..." I think.

drivers/xen/gntdev.c static int gntdev_open(struct inode *inode, struct file *flip) · line 648

Quoted 7 lines
drivers/xen/gntdev.c +4 -0
+	 * default DMA ops.
+	 */
+	of_dma_configure(priv->dma_dev, NULL);
+#endif
  	pr_debug("priv %p\n", priv);
 
  	return 0;

With those fixed,

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#32 lore raw

Re: [PATCH v3 5/9] xen/gntdev: Allow mappings for DMA buffers

In reply to Oleksandr Andrushchenko

@@ -548,6 +632,17 @@ static int gntdev_open(struct inode *inode, struct file *flip)

drivers/xen/gntdev.c static int gntdev_open(struct inode *inode, struct file *flip) · line 644

Quoted 13 lines
  	}
  
  	flip->private_data = priv;
drivers/xen/gntdev.c +10 -0
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	priv->dma_dev = gntdev_miscdev.this_device;
+
+	/*
+	 * The device is not spawn from a device tree, so arch_setup_dma_ops
+	 * is not called, thus leaving the device with dummy DMA ops.
+	 * Fix this call of_dma_configure() with a NULL node to set
+	 * default DMA ops.
+	 */
+	of_dma_configure(priv->dma_dev, NULL);

Please note, that the code above will need a change while applying to the mainline kernel because of API changes [1]. Unfortunately, current Xen tip kernel tree is v4.17-rc5 based, so I cannot make the change in this patch now.

The change is trivial and requires:
-of_dma_configure(priv->dma_dev, NULL);
+of_dma_configure(priv->dma_dev, NULL, true);

Quoted 5 lines

+#endif
pr_debug("priv %p\n", priv);

return 0;

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3d6ce86ee79465e1b1b6e287f8ea26b553fc768e

#33 lore raw

Re: [PATCH v3 5/9] xen/gntdev: Allow mappings for DMA buffers

In reply to Oleksandr Andrushchenko

@@ -548,6 +632,17 @@ static int gntdev_open(struct inode *inode, struct file *flip)

drivers/xen/gntdev.c static int gntdev_open(struct inode *inode, struct file *flip) · line 644

Quoted 13 lines
  	}
  
  	flip->private_data = priv;
drivers/xen/gntdev.c +10 -0
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	priv->dma_dev = gntdev_miscdev.this_device;
+
+	/*
+	 * The device is not spawn from a device tree, so arch_setup_dma_ops
+	 * is not called, thus leaving the device with dummy DMA ops.
+	 * Fix this call of_dma_configure() with a NULL node to set
+	 * default DMA ops.
+	 */
+	of_dma_configure(priv->dma_dev, NULL);

Please note, that the code above will need a change while applying to the mainline kernel because of API changes [1]. Unfortunately, current Xen tip kernel tree is v4.17-rc5 based, so I cannot make the change in this patch now.

The change is trivial and requires:
-of_dma_configure(priv->dma_dev, NULL);
+of_dma_configure(priv->dma_dev, NULL, true);

Quoted 5 lines

+#endif
pr_debug("priv %p\n", priv);

return 0;

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3d6ce86ee79465e1b1b6e287f8ea26b553fc768e

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#34 lore raw

PATCH v3 5/9 xen/gntdev: Allow mappings for DMA buffers

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

Allow mappings for DMA backed buffers if grant table module supports such: this extends grant device to not only map buffers made of balloon pages, but also from buffers allocated with dma_alloc_xxx.

---
 drivers/xen/gntdev.c      | 99 ++++++++++++++++++++++++++++++++++++++-
 include/uapi/xen/gntdev.h | 15 ++++++
 2 files changed, 112 insertions(+), 2 deletions(-)
drivers/xen/gntdev.c +97 -2
index bd56653b9bbc..d6b79ad1cd6f 100644
@@ -37,6 +37,9 @@
 #include <linux/slab.h>
 #include <linux/highmem.h>
 #include <linux/refcount.h>
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+#include <linux/of_device.h>
+#endif
 
 #include <xen/xen.h>
 #include <xen/grant_table.h>
@@ -72,6 +75,11 @@ struct gntdev_priv {
 	struct mutex lock;
 	struct mm_struct *mm;
 	struct mmu_notifier mn;
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	/* Device for which DMA memory is allocated. */
+	struct device *dma_dev;
+#endif
 };
 
 struct unmap_notify {
@@ -96,10 +104,27 @@ struct grant_map {
 	struct gnttab_unmap_grant_ref *kunmap_ops;
 	struct page **pages;
 	unsigned long pages_vm_start;
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	/*
+	 * If dmabuf_vaddr is not NULL then this mapping is backed by DMA
+	 * capable memory.
+	 */
+
+	struct device *dma_dev;
+	/* Flags used to create this DMA buffer: GNTDEV_DMA_FLAG_XXX. */
+	int dma_flags;
+	void *dma_vaddr;
+	dma_addr_t dma_bus_addr;
+	/* Needed to avoid allocation in gnttab_dma_free_pages(). */
+	xen_pfn_t *frames;
+#endif
 };
 
 static int unmap_grant_pages(struct grant_map *map, int offset, int pages);
 
+static struct miscdevice gntdev_miscdev;
+
 /* ------------------------------------------------------------------ */
 
 static void gntdev_print_maps(struct gntdev_priv *priv,
@@ -121,8 +146,27 @@ static void gntdev_free_map(struct grant_map *map)
 	if (map == NULL)
 		return;
 
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	if (map->dma_vaddr) {
+		struct gnttab_dma_alloc_args args;
+
+		args.dev = map->dma_dev;
+		args.coherent = map->dma_flags & GNTDEV_DMA_FLAG_COHERENT;
+		args.nr_pages = map->count;
+		args.pages = map->pages;
+		args.frames = map->frames;
+		args.vaddr = map->dma_vaddr;
+		args.dev_bus_addr = map->dma_bus_addr;
+
+		gnttab_dma_free_pages(&args);
+	} else
+#endif
 	if (map->pages)
 		gnttab_free_pages(map->count, map->pages);
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	kfree(map->frames);
+#endif
 	kfree(map->pages);
 	kfree(map->grants);
 	kfree(map->map_ops);
@@ -132,7 +176,8 @@ static void gntdev_free_map(struct grant_map *map)
 	kfree(map);
 }
 
-static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count)
+static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count,
+					  int dma_flags)
 {
 	struct grant_map *add;
 	int i;
@@ -155,6 +200,37 @@ static struct grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count)
 	    NULL == add->pages)
 		goto err;
 
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	add->dma_flags = dma_flags;
+
+	/*
+	 * Check if this mapping is requested to be backed
+	 * by a DMA buffer.
+	 */
+	if (dma_flags & (GNTDEV_DMA_FLAG_WC | GNTDEV_DMA_FLAG_COHERENT)) {
+		struct gnttab_dma_alloc_args args;
+
+		add->frames = kcalloc(count, sizeof(add->frames[0]),
+				      GFP_KERNEL);
+		if (!add->frames)
+			goto err;
+
+		/* Remember the device, so we can free DMA memory. */
+		add->dma_dev = priv->dma_dev;
+
+		args.dev = priv->dma_dev;
+		args.coherent = dma_flags & GNTDEV_DMA_FLAG_COHERENT;
+		args.nr_pages = count;
+		args.pages = add->pages;
+		args.frames = add->frames;
+
+		if (gnttab_dma_alloc_pages(&args))
+			goto err;
+
+		add->dma_vaddr = args.vaddr;
+		add->dma_bus_addr = args.dev_bus_addr;
+	} else
+#endif
 	if (gnttab_alloc_pages(count, add->pages))
 		goto err;
 
@@ -325,6 +401,14 @@ static int map_grant_pages(struct grant_map *map)
 		map->unmap_ops[i].handle = map->map_ops[i].handle;
 		if (use_ptemod)
 			map->kunmap_ops[i].handle = map->kmap_ops[i].handle;
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+		else if (map->dma_vaddr) {
+			unsigned long mfn;
+
+			mfn = pfn_to_bfn(page_to_pfn(map->pages[i]));
+			map->unmap_ops[i].dev_bus_addr = __pfn_to_phys(mfn);
+		}
+#endif
 	}
 	return err;
 }
@@ -548,6 +632,17 @@ static int gntdev_open(struct inode *inode, struct file *flip)
 	}
 
 	flip->private_data = priv;
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	priv->dma_dev = gntdev_miscdev.this_device;
+
+	/*
+	 * The device is not spawn from a device tree, so arch_setup_dma_ops
+	 * is not called, thus leaving the device with dummy DMA ops.
+	 * Fix this call of_dma_configure() with a NULL node to set
+	 * default DMA ops.
+	 */
+	of_dma_configure(priv->dma_dev, NULL);
+#endif
 	pr_debug("priv %p\n", priv);
 
 	return 0;
@@ -589,7 +684,7 @@ static long gntdev_ioctl_map_grant_ref(struct gntdev_priv *priv,
 		return -EINVAL;
 
 	err = -ENOMEM;
-	map = gntdev_alloc_map(priv, op.count);
+	map = gntdev_alloc_map(priv, op.count, 0 /* This is not a dma-buf. */);
 	if (!map)
 		return err;
 
include/uapi/xen/gntdev.h +15 -0
index 6d1163456c03..4b9d498a31d4 100644
@@ -200,4 +200,19 @@ struct ioctl_gntdev_grant_copy {
 /* Send an interrupt on the indicated event channel */
 #define UNMAP_NOTIFY_SEND_EVENT 0x2
 
+/*
+ * Flags to be used while requesting memory mapping's backing storage
+ * to be allocated with DMA API.
+ */
+
+/*
+ * The buffer is backed with memory allocated with dma_alloc_wc.
+ */
+#define GNTDEV_DMA_FLAG_WC		(1 << 0)
+
+/*
+ * The buffer is backed with memory allocated with dma_alloc_coherent.
+ */
+#define GNTDEV_DMA_FLAG_COHERENT	(1 << 1)
+
 #endif /* __LINUX_PUBLIC_GNTDEV_H__ */
Signature · 7 lines
2.17.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#35 lore raw

PATCH v3 6/9 xen/gntdev: Make private routines/structures accessible

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

This is in preparation for adding support of DMA buffer
functionality: make map/unmap related code and structures, used privately by gntdev, ready for dma-buf extension, which will re-use these. Rename corresponding structures as those become non-private to gntdev now.

---
 drivers/xen/gntdev-common.h |  86 +++++++++++++++++++++++
 drivers/xen/gntdev.c        | 132 ++++++++++++------------------------
 2 files changed, 128 insertions(+), 90 deletions(-)
 create mode 100644 drivers/xen/gntdev-common.h
drivers/xen/gntdev-common.h new file +86 -0
new file mode 100644
index 000000000000..7a9845a6bee9
@@ -0,0 +1,86 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Common functionality of grant device.
+ *
+ * Copyright (c) 2006-2007, D G Murray.
+ *           (c) 2009 Gerd Hoffmann <[email protected]>
+ *           (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _GNTDEV_COMMON_H
+#define _GNTDEV_COMMON_H
+
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/mmu_notifier.h>
+#include <linux/types.h>
+
+struct gntdev_priv {
+	/* maps with visible offsets in the file descriptor */
+	struct list_head maps;
+	/* maps that are not visible; will be freed on munmap.
+	 * Only populated if populate_freeable_maps == 1 */
+	struct list_head freeable_maps;
+	/* lock protects maps and freeable_maps */
+	struct mutex lock;
+	struct mm_struct *mm;
+	struct mmu_notifier mn;
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	/* Device for which DMA memory is allocated. */
+	struct device *dma_dev;
+#endif
+};
+
+struct gntdev_unmap_notify {
+	int flags;
+	/* Address relative to the start of the gntdev_grant_map */
+	int addr;
+	int event;
+};
+
+struct gntdev_grant_map {
+	struct list_head next;
+	struct vm_area_struct *vma;
+	int index;
+	int count;
+	int flags;
+	refcount_t users;
+	struct gntdev_unmap_notify notify;
+	struct ioctl_gntdev_grant_ref *grants;
+	struct gnttab_map_grant_ref   *map_ops;
+	struct gnttab_unmap_grant_ref *unmap_ops;
+	struct gnttab_map_grant_ref   *kmap_ops;
+	struct gnttab_unmap_grant_ref *kunmap_ops;
+	struct page **pages;
+	unsigned long pages_vm_start;
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	/*
+	 * If dmabuf_vaddr is not NULL then this mapping is backed by DMA
+	 * capable memory.
+	 */
+
+	struct device *dma_dev;
+	/* Flags used to create this DMA buffer: GNTDEV_DMA_FLAG_XXX. */
+	int dma_flags;
+	void *dma_vaddr;
+	dma_addr_t dma_bus_addr;
+	/* Needed to avoid allocation in gnttab_dma_free_pages(). */
+	xen_pfn_t *frames;
+#endif
+};
+
+struct gntdev_grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count,
+					  int dma_flags);
+
+void gntdev_add_map(struct gntdev_priv *priv, struct gntdev_grant_map *add);
+
+void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map);
+
+bool gntdev_account_mapped_pages(int count);
+
+int gntdev_map_grant_pages(struct gntdev_grant_map *map);
+
+#endif
drivers/xen/gntdev.c +42 -90

315 lines not on this page · raw message

Signature · 6 lines
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Reviewed-by on v3 6/9
#36 lore raw

Re: [PATCH v3 6/9] xen/gntdev: Make private routines/structures accessible

Quoted 34 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

This is in preparation for adding support of DMA buffer
functionality: make map/unmap related code and structures, used privately by gntdev, ready for dma-buf extension, which will re-use these. Rename corresponding structures as those become non-private to gntdev now.

---
  drivers/xen/gntdev-common.h |  86 +++++++++++++++++++++++
  drivers/xen/gntdev.c        | 132 ++++++++++++------------------------
  2 files changed, 128 insertions(+), 90 deletions(-)

create mode 100644 drivers/xen/gntdev-common.h

drivers/xen/gntdev-common.h new file +13 -0
new file mode 100644
index 000000000000..7a9845a6bee9
@@ -0,0 +1,86 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Common functionality of grant device.
+ *
+ * Copyright (c) 2006-2007, D G Murray.
+ *           (c) 2009 Gerd Hoffmann <[email protected]>
+ *           (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _GNTDEV_COMMON_H
+#define _GNTDEV_COMMON_H
+

drivers/xen/gntdev-common.h line 23

Quoted 10 lines
drivers/xen/gntdev-common.h new file +10 -0
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/mmu_notifier.h>
+#include <linux/types.h>
+
+struct gntdev_priv {
+	/* maps with visible offsets in the file descriptor */
+	struct list_head maps;
+	/* maps that are not visible; will be freed on munmap.
+	 * Only populated if populate_freeable_maps == 1 */

Since you are touching this code please fix comment style.

drivers/xen/gntdev-common.h line 34

Quoted 11 lines
drivers/xen/gntdev-common.h new file +11 -0
+	struct list_head freeable_maps;
+	/* lock protects maps and freeable_maps */
+	struct mutex lock;
+	struct mm_struct *mm;
+	struct mmu_notifier mn;
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	/* Device for which DMA memory is allocated. */
+	struct device *dma_dev;
+#endif
+};

With that fixed,

#37 lore raw

Re: [PATCH v3 6/9] xen/gntdev: Make private routines/structures accessible

Quoted 42 lines · Boris Ostrovsky
Quoted 39 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

This is in preparation for adding support of DMA buffer
functionality: make map/unmap related code and structures, used privately by gntdev, ready for dma-buf extension, which will re-use these. Rename corresponding structures as those become non-private to gntdev now.

---
  drivers/xen/gntdev-common.h |  86 +++++++++++++++++++++++ drivers/xen/gntdev.c        | 132 ++++++++++++------------------------

  2 files changed, 128 insertions(+), 90 deletions(-)

  create mode 100644 drivers/xen/gntdev-common.h

drivers/xen/gntdev-common.h new file +18 -0
new file mode 100644
index 000000000000..7a9845a6bee9
@@ -0,0 +1,86 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Common functionality of grant device.
+ *
+ * Copyright (c) 2006-2007, D G Murray.
+ *           (c) 2009 Gerd Hoffmann <[email protected]>
+ *           (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _GNTDEV_COMMON_H
+#define _GNTDEV_COMMON_H
+
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/mmu_notifier.h>
+#include <linux/types.h>
+
Quoted 9 lines

drivers/xen/gntdev-common.h line 23

Quoted 5 lines
drivers/xen/gntdev-common.h new file +5 -0
+struct gntdev_priv {
+    /* maps with visible offsets in the file descriptor */
+    struct list_head maps;
+    /* maps that are not visible; will be freed on munmap.
+     * Only populated if populate_freeable_maps == 1 */

Since you are touching this code please fix comment style.

I saw that while running checkpatch, but was not sure if I have to touch those as they seemed to be not related to the change itself. But I'll make sure all the comments are consistent.

Quoted 16 lines

drivers/xen/gntdev-common.h line 34

Quoted 12 lines
drivers/xen/gntdev-common.h new file +11 -0
+    struct list_head freeable_maps;
+    /* lock protects maps and freeable_maps */
+    struct mutex lock;
+    struct mm_struct *mm;
+    struct mmu_notifier mn;
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+    /* Device for which DMA memory is allocated. */
+    struct device *dma_dev;
+#endif
+};

With that fixed,

Quoted 2 lines

Thank you,
Oleksandr

#38 lore raw

Re: [PATCH v3 6/9] xen/gntdev: Make private routines/structures accessible

In reply to Boris Ostrovsky
Quoted 42 lines · Boris Ostrovsky
Quoted 39 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

This is in preparation for adding support of DMA buffer
functionality: make map/unmap related code and structures, used privately by gntdev, ready for dma-buf extension, which will re-use these. Rename corresponding structures as those become non-private to gntdev now.

---
  drivers/xen/gntdev-common.h |  86 +++++++++++++++++++++++ drivers/xen/gntdev.c        | 132 ++++++++++++------------------------

  2 files changed, 128 insertions(+), 90 deletions(-)

  create mode 100644 drivers/xen/gntdev-common.h

drivers/xen/gntdev-common.h new file +18 -0
new file mode 100644
index 000000000000..7a9845a6bee9
@@ -0,0 +1,86 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Common functionality of grant device.
+ *
+ * Copyright (c) 2006-2007, D G Murray.
+ *           (c) 2009 Gerd Hoffmann <[email protected]>
+ *           (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _GNTDEV_COMMON_H
+#define _GNTDEV_COMMON_H
+
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/mmu_notifier.h>
+#include <linux/types.h>
+
Quoted 9 lines

drivers/xen/gntdev-common.h line 23

Quoted 5 lines
drivers/xen/gntdev-common.h new file +5 -0
+struct gntdev_priv {
+    /* maps with visible offsets in the file descriptor */
+    struct list_head maps;
+    /* maps that are not visible; will be freed on munmap.
+     * Only populated if populate_freeable_maps == 1 */

Since you are touching this code please fix comment style.

I saw that while running checkpatch, but was not sure if I have to touch those as they seemed to be not related to the change itself. But I'll make sure all the comments are consistent.

Quoted 16 lines

drivers/xen/gntdev-common.h line 34

Quoted 12 lines
drivers/xen/gntdev-common.h new file +11 -0
+    struct list_head freeable_maps;
+    /* lock protects maps and freeable_maps */
+    struct mutex lock;
+    struct mm_struct *mm;
+    struct mmu_notifier mn;
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+    /* Device for which DMA memory is allocated. */
+    struct device *dma_dev;
+#endif
+};

With that fixed,

Quoted 2 lines

Thank you,
Oleksandr

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Reviewed-by on v3 6/9
#39 lore raw

Re: [PATCH v3 6/9] xen/gntdev: Make private routines/structures accessible

In reply to Oleksandr Andrushchenko
Quoted 34 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

This is in preparation for adding support of DMA buffer
functionality: make map/unmap related code and structures, used privately by gntdev, ready for dma-buf extension, which will re-use these. Rename corresponding structures as those become non-private to gntdev now.

---
  drivers/xen/gntdev-common.h |  86 +++++++++++++++++++++++
  drivers/xen/gntdev.c        | 132 ++++++++++++------------------------
  2 files changed, 128 insertions(+), 90 deletions(-)

create mode 100644 drivers/xen/gntdev-common.h

drivers/xen/gntdev-common.h new file +13 -0
new file mode 100644
index 000000000000..7a9845a6bee9
@@ -0,0 +1,86 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Common functionality of grant device.
+ *
+ * Copyright (c) 2006-2007, D G Murray.
+ *           (c) 2009 Gerd Hoffmann <[email protected]>
+ *           (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _GNTDEV_COMMON_H
+#define _GNTDEV_COMMON_H
+

drivers/xen/gntdev-common.h line 23

Quoted 10 lines
drivers/xen/gntdev-common.h new file +10 -0
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/mmu_notifier.h>
+#include <linux/types.h>
+
+struct gntdev_priv {
+	/* maps with visible offsets in the file descriptor */
+	struct list_head maps;
+	/* maps that are not visible; will be freed on munmap.
+	 * Only populated if populate_freeable_maps == 1 */

Since you are touching this code please fix comment style.

drivers/xen/gntdev-common.h line 34

Quoted 11 lines
drivers/xen/gntdev-common.h new file +11 -0
+	struct list_head freeable_maps;
+	/* lock protects maps and freeable_maps */
+	struct mutex lock;
+	struct mm_struct *mm;
+	struct mmu_notifier mn;
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	/* Device for which DMA memory is allocated. */
+	struct device *dma_dev;
+#endif
+};

With that fixed,

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#40 lore raw

PATCH v3 6/9 xen/gntdev: Make private routines/structures accessible

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

This is in preparation for adding support of DMA buffer
functionality: make map/unmap related code and structures, used privately by gntdev, ready for dma-buf extension, which will re-use these. Rename corresponding structures as those become non-private to gntdev now.

---
 drivers/xen/gntdev-common.h |  86 +++++++++++++++++++++++
 drivers/xen/gntdev.c        | 132 ++++++++++++------------------------
 2 files changed, 128 insertions(+), 90 deletions(-)
 create mode 100644 drivers/xen/gntdev-common.h
drivers/xen/gntdev-common.h new file +86 -0
new file mode 100644
index 000000000000..7a9845a6bee9
@@ -0,0 +1,86 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Common functionality of grant device.
+ *
+ * Copyright (c) 2006-2007, D G Murray.
+ *           (c) 2009 Gerd Hoffmann <[email protected]>
+ *           (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _GNTDEV_COMMON_H
+#define _GNTDEV_COMMON_H
+
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/mmu_notifier.h>
+#include <linux/types.h>
+
+struct gntdev_priv {
+	/* maps with visible offsets in the file descriptor */
+	struct list_head maps;
+	/* maps that are not visible; will be freed on munmap.
+	 * Only populated if populate_freeable_maps == 1 */
+	struct list_head freeable_maps;
+	/* lock protects maps and freeable_maps */
+	struct mutex lock;
+	struct mm_struct *mm;
+	struct mmu_notifier mn;
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	/* Device for which DMA memory is allocated. */
+	struct device *dma_dev;
+#endif
+};
+
+struct gntdev_unmap_notify {
+	int flags;
+	/* Address relative to the start of the gntdev_grant_map */
+	int addr;
+	int event;
+};
+
+struct gntdev_grant_map {
+	struct list_head next;
+	struct vm_area_struct *vma;
+	int index;
+	int count;
+	int flags;
+	refcount_t users;
+	struct gntdev_unmap_notify notify;
+	struct ioctl_gntdev_grant_ref *grants;
+	struct gnttab_map_grant_ref   *map_ops;
+	struct gnttab_unmap_grant_ref *unmap_ops;
+	struct gnttab_map_grant_ref   *kmap_ops;
+	struct gnttab_unmap_grant_ref *kunmap_ops;
+	struct page **pages;
+	unsigned long pages_vm_start;
+
+#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
+	/*
+	 * If dmabuf_vaddr is not NULL then this mapping is backed by DMA
+	 * capable memory.
+	 */
+
+	struct device *dma_dev;
+	/* Flags used to create this DMA buffer: GNTDEV_DMA_FLAG_XXX. */
+	int dma_flags;
+	void *dma_vaddr;
+	dma_addr_t dma_bus_addr;
+	/* Needed to avoid allocation in gnttab_dma_free_pages(). */
+	xen_pfn_t *frames;
+#endif
+};
+
+struct gntdev_grant_map *gntdev_alloc_map(struct gntdev_priv *priv, int count,
+					  int dma_flags);
+
+void gntdev_add_map(struct gntdev_priv *priv, struct gntdev_grant_map *add);
+
+void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map);
+
+bool gntdev_account_mapped_pages(int count);
+
+int gntdev_map_grant_pages(struct gntdev_grant_map *map);
+
+#endif
drivers/xen/gntdev.c +42 -90

315 lines not on this page · raw message

Signature · 7 lines
2.17.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#41 lore raw

PATCH v3 7/9 xen/gntdev: Add initial support for dma-buf UAPI

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

Add UAPI and IOCTLs for dma-buf grant device driver extension:
the extension allows userspace processes and kernel modules to use Xen backed dma-buf implementation. With this extension grant references to the pages of an imported dma-buf can be exported for other domain use and grant references coming from a foreign domain can be converted into a local dma-buf for local export. Implement basic initialization and stubs for Xen DMA buffers' support.

---
 drivers/xen/Kconfig         |  10 +++
 drivers/xen/Makefile        |   1 +
 drivers/xen/gntdev-common.h |   6 ++
 drivers/xen/gntdev-dmabuf.c |  72 ++++++++++++++++++++
 drivers/xen/gntdev-dmabuf.h |  39 +++++++++++
 drivers/xen/gntdev.c        | 132 ++++++++++++++++++++++++++++++++++++
 include/uapi/xen/gntdev.h   |  91 +++++++++++++++++++++++++
 7 files changed, 351 insertions(+)
 create mode 100644 drivers/xen/gntdev-dmabuf.c
 create mode 100644 drivers/xen/gntdev-dmabuf.h
drivers/xen/Kconfig +10 -0
index 39536ddfbce4..52d64e4b6b81 100644
@@ -152,6 +152,16 @@ config XEN_GNTDEV
 	help
 	  Allows userspace processes to use grants.
 
+config XEN_GNTDEV_DMABUF
+	bool "Add support for dma-buf grant access device driver extension"
+	depends on XEN_GNTDEV && XEN_GRANT_DMA_ALLOC && DMA_SHARED_BUFFER
+	help
+	  Allows userspace processes and kernel modules to use Xen backed
+	  dma-buf implementation. With this extension grant references to
+	  the pages of an imported dma-buf can be exported for other domain
+	  use and grant references coming from a foreign domain can be
+	  converted into a local dma-buf for local export.
+
 config XEN_GRANT_DEV_ALLOC
 	tristate "User-space grant reference allocator driver"
 	depends on XEN
drivers/xen/Makefile +1 -0
index 3c87b0c3aca6..33afb7b2b227 100644
@@ -41,5 +41,6 @@ obj-$(CONFIG_XEN_PVCALLS_BACKEND)	+= pvcalls-back.o
 obj-$(CONFIG_XEN_PVCALLS_FRONTEND)	+= pvcalls-front.o
 xen-evtchn-y				:= evtchn.o
 xen-gntdev-y				:= gntdev.o
+xen-gntdev-$(CONFIG_XEN_GNTDEV_DMABUF)	+= gntdev-dmabuf.o
 xen-gntalloc-y				:= gntalloc.o
 xen-privcmd-y				:= privcmd.o
drivers/xen/gntdev-common.h +6 -0
index 7a9845a6bee9..a3408fd39b07 100644
@@ -16,6 +16,8 @@
 #include <linux/mmu_notifier.h>
 #include <linux/types.h>
 
+struct gntdev_dmabuf_priv;
+
 struct gntdev_priv {
 	/* maps with visible offsets in the file descriptor */
 	struct list_head maps;
@@ -31,6 +33,10 @@ struct gntdev_priv {
 	/* Device for which DMA memory is allocated. */
 	struct device *dma_dev;
 #endif
+
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+	struct gntdev_dmabuf_priv *dmabuf_priv;
+#endif
 };
 
 struct gntdev_unmap_notify {
drivers/xen/gntdev-dmabuf.c new file +72 -0

72 lines not on this page · raw message

drivers/xen/gntdev-dmabuf.h new file +39 -0
new file mode 100644
index 000000000000..75324ecd3e62
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Xen dma-buf functionality for gntdev.
+ *
+ * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _GNTDEV_DMABUF_H
+#define _GNTDEV_DMABUF_H
+
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/types.h>
+
+struct gntdev_priv;
+struct gntdev_dmabuf_priv;
+struct gntdev_dmabuf;
+struct device;
+
+struct gntdev_dmabuf_priv *gntdev_dmabuf_init(void);
+
+void gntdev_dmabuf_fini(struct gntdev_dmabuf_priv *priv);
+
+int gntdev_dmabuf_exp_from_refs(struct gntdev_priv *priv, int flags,
+				int count, u32 domid, u32 *refs, u32 *fd);
+
+int gntdev_dmabuf_exp_wait_released(struct gntdev_dmabuf_priv *priv, int fd,
+				    int wait_to_ms);
+
+struct gntdev_dmabuf *
+gntdev_dmabuf_imp_to_refs(struct gntdev_dmabuf_priv *priv, struct device *dev,
+			  int fd, int count, int domid);
+
+u32 *gntdev_dmabuf_imp_get_refs(struct gntdev_dmabuf *gntdev_dmabuf);
+
+int gntdev_dmabuf_imp_release(struct gntdev_dmabuf_priv *priv, u32 fd);
+
+#endif
drivers/xen/gntdev.c +132 -0

164 lines not on this page · raw message

include/uapi/xen/gntdev.h +91 -0

101 lines not on this page · raw message

Signature · 1 line
2.17.1
#42 lore raw

Re: [PATCH v3 7/9] xen/gntdev: Add initial support for dma-buf UAPI

Quoted 45 lines · Oleksandr Andrushchenko
drivers/xen/gntdev.c +17 -0
index a09db23e9663..e82660d81d7e 100644
@@ -48,6 +48,9 @@
  #include <asm/xen/hypercall.h>
 
  #include "gntdev-common.h"
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+#include "gntdev-dmabuf.h"
+#endif
 
  MODULE_LICENSE("GPL");
  MODULE_AUTHOR("Derek G. Murray <[email protected]>, "
@@ -566,6 +569,15 @@ static int gntdev_open(struct inode *inode, struct file *flip)
  	INIT_LIST_HEAD(&priv->freeable_maps);
  	mutex_init(&priv->lock);
 
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+	priv->dmabuf_priv = gntdev_dmabuf_init();
+	if (IS_ERR(priv->dmabuf_priv)) {
+		ret = PTR_ERR(priv->dmabuf_priv);
+		kfree(priv);
+		return ret;
+	}
+#endif
+
  	if (use_ptemod) {
  		priv->mm = get_task_mm(current);
  		if (!priv->mm) {
@@ -616,8 +628,13 @@ static int gntdev_release(struct inode *inode, struct file *flip)
  	WARN_ON(!list_empty(&priv->freeable_maps));
  	mutex_unlock(&priv->lock);
 
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+	gntdev_dmabuf_fini(priv->dmabuf_priv);
+#endif
+
  	if (use_ptemod)
  		mmu_notifier_unregister(&priv->mn, priv->mm);
+
  	kfree(priv);
  	return 0;
  }

drivers/xen/gntdev.c static long gntdev_ioctl_grant_copy(struct gntdev_priv *priv, void __user *u) · line 1010

Quoted 7 lines
drivers/xen/gntdev.c +4 -0
@@ -987,6 +1004,107 @@ static long gntdev_ioctl_grant_copy(struct gntdev_priv *priv, void __user *u)
  	return ret;
  }
 
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+static long
+gntdev_ioctl_dmabuf_exp_from_refs(struct gntdev_priv *priv,
+				  struct ioctl_gntdev_dmabuf_exp_from_refs __user *u)

Didn't we agree that this code moves to gntdev-dmabuf.c ?

-boris

#43 lore raw

Re: [PATCH v3 7/9] xen/gntdev: Add initial support for dma-buf UAPI

Quoted 49 lines · Boris Ostrovsky
Quoted 45 lines · Oleksandr Andrushchenko
drivers/xen/gntdev.c +3 -0
index a09db23e9663..e82660d81d7e 100644
@@ -48,6 +48,9 @@
   #include <asm/xen/hypercall.h>
     #include "gntdev-common.h"
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+#include "gntdev-dmabuf.h"
+#endif
     MODULE_LICENSE("GPL");

  MODULE_AUTHOR("Derek G. Murray <[email protected]>, "

drivers/xen/gntdev.c +0 -0
@@ -566,6 +569,15 @@ static int gntdev_open(struct inode *inode, 
 struct file *flip)
       INIT_LIST_HEAD(&priv->freeable_maps);
       mutex_init(&priv->lock);

  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+    priv->dmabuf_priv = gntdev_dmabuf_init();
+    if (IS_ERR(priv->dmabuf_priv)) {
+        ret = PTR_ERR(priv->dmabuf_priv);
+        kfree(priv);
+        return ret;
+    }

drivers/xen/gntdev.c +2 -0
+#endif
+

      if (use_ptemod) {
          priv->mm = get_task_mm(current);
          if (!priv->mm) {

drivers/xen/gntdev.c +0 -0
@@ -616,8 +628,13 @@ static int gntdev_release(struct inode *inode, 
 struct file *flip)
       WARN_ON(!list_empty(&priv->freeable_maps));
       mutex_unlock(&priv->lock);

  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+    gntdev_dmabuf_fini(priv->dmabuf_priv);

drivers/xen/gntdev.c +2 -0
+#endif
+

      if (use_ptemod) mmu_notifier_unregister(&priv->mn, priv->mm); +
      kfree(priv);
      return 0;
  }

drivers/xen/gntdev.c +0 -0
@@ -987,6 +1004,107 @@ static long gntdev_ioctl_grant_copy(struct 
 gntdev_priv *priv, void __user *u)
       return ret;
Quoted 9 lines

drivers/xen/gntdev.c static long gntdev_ioctl_grant_copy(struct gntdev_priv *priv, void __user *u) · line 1010

Quoted 5 lines
drivers/xen/gntdev.c +0 -0
   }

  +#ifdef CONFIG_XEN_GNTDEV_DMABUF

drivers/xen/gntdev.c +3 -0
+static long
+gntdev_ioctl_dmabuf_exp_from_refs(struct gntdev_priv *priv,
+                  struct ioctl_gntdev_dmabuf_exp_from_refs __user *u)

Didn't we agree that this code moves to gntdev-dmabuf.c ?

Sure, didn't think we want IOCTL's code to be moved as well, but that does make sense - will move all

Quoted 2 lines

-boris

Thank you,
Oleksandr

#44 lore raw

Re: [PATCH v3 7/9] xen/gntdev: Add initial support for dma-buf UAPI

In reply to Boris Ostrovsky
Quoted 49 lines · Boris Ostrovsky
Quoted 45 lines · Oleksandr Andrushchenko
drivers/xen/gntdev.c +3 -0
index a09db23e9663..e82660d81d7e 100644
@@ -48,6 +48,9 @@
   #include <asm/xen/hypercall.h>
     #include "gntdev-common.h"
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+#include "gntdev-dmabuf.h"
+#endif
     MODULE_LICENSE("GPL");

  MODULE_AUTHOR("Derek G. Murray <[email protected]>, "

drivers/xen/gntdev.c +0 -0
@@ -566,6 +569,15 @@ static int gntdev_open(struct inode *inode, 
 struct file *flip)
       INIT_LIST_HEAD(&priv->freeable_maps);
       mutex_init(&priv->lock);

  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+    priv->dmabuf_priv = gntdev_dmabuf_init();
+    if (IS_ERR(priv->dmabuf_priv)) {
+        ret = PTR_ERR(priv->dmabuf_priv);
+        kfree(priv);
+        return ret;
+    }

drivers/xen/gntdev.c +2 -0
+#endif
+

      if (use_ptemod) {
          priv->mm = get_task_mm(current);
          if (!priv->mm) {

drivers/xen/gntdev.c +0 -0
@@ -616,8 +628,13 @@ static int gntdev_release(struct inode *inode, 
 struct file *flip)
       WARN_ON(!list_empty(&priv->freeable_maps));
       mutex_unlock(&priv->lock);

  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+    gntdev_dmabuf_fini(priv->dmabuf_priv);

drivers/xen/gntdev.c +2 -0
+#endif
+

      if (use_ptemod) mmu_notifier_unregister(&priv->mn, priv->mm); +
      kfree(priv);
      return 0;
  }

drivers/xen/gntdev.c +0 -0
@@ -987,6 +1004,107 @@ static long gntdev_ioctl_grant_copy(struct 
 gntdev_priv *priv, void __user *u)
       return ret;
Quoted 9 lines

drivers/xen/gntdev.c static long gntdev_ioctl_grant_copy(struct gntdev_priv *priv, void __user *u) · line 1010

Quoted 5 lines
drivers/xen/gntdev.c +0 -0
   }

  +#ifdef CONFIG_XEN_GNTDEV_DMABUF

drivers/xen/gntdev.c +3 -0
+static long
+gntdev_ioctl_dmabuf_exp_from_refs(struct gntdev_priv *priv,
+                  struct ioctl_gntdev_dmabuf_exp_from_refs __user *u)

Didn't we agree that this code moves to gntdev-dmabuf.c ?

Sure, didn't think we want IOCTL's code to be moved as well, but that does make sense - will move all

Quoted 2 lines

-boris

Thank you,
Oleksandr

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#45 lore raw

Re: [PATCH v3 7/9] xen/gntdev: Add initial support for dma-buf UAPI

In reply to Oleksandr Andrushchenko
Quoted 45 lines · Oleksandr Andrushchenko
drivers/xen/gntdev.c +17 -0
index a09db23e9663..e82660d81d7e 100644
@@ -48,6 +48,9 @@
  #include <asm/xen/hypercall.h>
 
  #include "gntdev-common.h"
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+#include "gntdev-dmabuf.h"
+#endif
 
  MODULE_LICENSE("GPL");
  MODULE_AUTHOR("Derek G. Murray <[email protected]>, "
@@ -566,6 +569,15 @@ static int gntdev_open(struct inode *inode, struct file *flip)
  	INIT_LIST_HEAD(&priv->freeable_maps);
  	mutex_init(&priv->lock);
 
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+	priv->dmabuf_priv = gntdev_dmabuf_init();
+	if (IS_ERR(priv->dmabuf_priv)) {
+		ret = PTR_ERR(priv->dmabuf_priv);
+		kfree(priv);
+		return ret;
+	}
+#endif
+
  	if (use_ptemod) {
  		priv->mm = get_task_mm(current);
  		if (!priv->mm) {
@@ -616,8 +628,13 @@ static int gntdev_release(struct inode *inode, struct file *flip)
  	WARN_ON(!list_empty(&priv->freeable_maps));
  	mutex_unlock(&priv->lock);
 
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+	gntdev_dmabuf_fini(priv->dmabuf_priv);
+#endif
+
  	if (use_ptemod)
  		mmu_notifier_unregister(&priv->mn, priv->mm);
+
  	kfree(priv);
  	return 0;
  }

drivers/xen/gntdev.c static long gntdev_ioctl_grant_copy(struct gntdev_priv *priv, void __user *u) · line 1010

Quoted 7 lines
drivers/xen/gntdev.c +4 -0
@@ -987,6 +1004,107 @@ static long gntdev_ioctl_grant_copy(struct gntdev_priv *priv, void __user *u)
  	return ret;
  }
 
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+static long
+gntdev_ioctl_dmabuf_exp_from_refs(struct gntdev_priv *priv,
+				  struct ioctl_gntdev_dmabuf_exp_from_refs __user *u)

Didn't we agree that this code moves to gntdev-dmabuf.c ?

-boris

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#46 lore raw

PATCH v3 7/9 xen/gntdev: Add initial support for dma-buf UAPI

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

Add UAPI and IOCTLs for dma-buf grant device driver extension:
the extension allows userspace processes and kernel modules to use Xen backed dma-buf implementation. With this extension grant references to the pages of an imported dma-buf can be exported for other domain use and grant references coming from a foreign domain can be converted into a local dma-buf for local export. Implement basic initialization and stubs for Xen DMA buffers' support.

---
 drivers/xen/Kconfig         |  10 +++
 drivers/xen/Makefile        |   1 +
 drivers/xen/gntdev-common.h |   6 ++
 drivers/xen/gntdev-dmabuf.c |  72 ++++++++++++++++++++
 drivers/xen/gntdev-dmabuf.h |  39 +++++++++++
 drivers/xen/gntdev.c        | 132 ++++++++++++++++++++++++++++++++++++
 include/uapi/xen/gntdev.h   |  91 +++++++++++++++++++++++++
 7 files changed, 351 insertions(+)
 create mode 100644 drivers/xen/gntdev-dmabuf.c
 create mode 100644 drivers/xen/gntdev-dmabuf.h
drivers/xen/Kconfig +10 -0
index 39536ddfbce4..52d64e4b6b81 100644
@@ -152,6 +152,16 @@ config XEN_GNTDEV
 	help
 	  Allows userspace processes to use grants.
 
+config XEN_GNTDEV_DMABUF
+	bool "Add support for dma-buf grant access device driver extension"
+	depends on XEN_GNTDEV && XEN_GRANT_DMA_ALLOC && DMA_SHARED_BUFFER
+	help
+	  Allows userspace processes and kernel modules to use Xen backed
+	  dma-buf implementation. With this extension grant references to
+	  the pages of an imported dma-buf can be exported for other domain
+	  use and grant references coming from a foreign domain can be
+	  converted into a local dma-buf for local export.
+
 config XEN_GRANT_DEV_ALLOC
 	tristate "User-space grant reference allocator driver"
 	depends on XEN
drivers/xen/Makefile +1 -0
index 3c87b0c3aca6..33afb7b2b227 100644
@@ -41,5 +41,6 @@ obj-$(CONFIG_XEN_PVCALLS_BACKEND)	+= pvcalls-back.o
 obj-$(CONFIG_XEN_PVCALLS_FRONTEND)	+= pvcalls-front.o
 xen-evtchn-y				:= evtchn.o
 xen-gntdev-y				:= gntdev.o
+xen-gntdev-$(CONFIG_XEN_GNTDEV_DMABUF)	+= gntdev-dmabuf.o
 xen-gntalloc-y				:= gntalloc.o
 xen-privcmd-y				:= privcmd.o
drivers/xen/gntdev-common.h +6 -0
index 7a9845a6bee9..a3408fd39b07 100644
@@ -16,6 +16,8 @@
 #include <linux/mmu_notifier.h>
 #include <linux/types.h>
 
+struct gntdev_dmabuf_priv;
+
 struct gntdev_priv {
 	/* maps with visible offsets in the file descriptor */
 	struct list_head maps;
@@ -31,6 +33,10 @@ struct gntdev_priv {
 	/* Device for which DMA memory is allocated. */
 	struct device *dma_dev;
 #endif
+
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+	struct gntdev_dmabuf_priv *dmabuf_priv;
+#endif
 };
 
 struct gntdev_unmap_notify {
drivers/xen/gntdev-dmabuf.c new file +72 -0

72 lines not on this page · raw message

drivers/xen/gntdev-dmabuf.h new file +39 -0
new file mode 100644
index 000000000000..75324ecd3e62
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Xen dma-buf functionality for gntdev.
+ *
+ * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _GNTDEV_DMABUF_H
+#define _GNTDEV_DMABUF_H
+
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/types.h>
+
+struct gntdev_priv;
+struct gntdev_dmabuf_priv;
+struct gntdev_dmabuf;
+struct device;
+
+struct gntdev_dmabuf_priv *gntdev_dmabuf_init(void);
+
+void gntdev_dmabuf_fini(struct gntdev_dmabuf_priv *priv);
+
+int gntdev_dmabuf_exp_from_refs(struct gntdev_priv *priv, int flags,
+				int count, u32 domid, u32 *refs, u32 *fd);
+
+int gntdev_dmabuf_exp_wait_released(struct gntdev_dmabuf_priv *priv, int fd,
+				    int wait_to_ms);
+
+struct gntdev_dmabuf *
+gntdev_dmabuf_imp_to_refs(struct gntdev_dmabuf_priv *priv, struct device *dev,
+			  int fd, int count, int domid);
+
+u32 *gntdev_dmabuf_imp_get_refs(struct gntdev_dmabuf *gntdev_dmabuf);
+
+int gntdev_dmabuf_imp_release(struct gntdev_dmabuf_priv *priv, u32 fd);
+
+#endif
drivers/xen/gntdev.c +132 -0

164 lines not on this page · raw message

include/uapi/xen/gntdev.h +91 -0

101 lines not on this page · raw message

Signature · 7 lines
2.17.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#47 lore raw

PATCH v3 8/9 xen/gntdev: Implement dma-buf export functionality

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

1. Create a dma-buf from grant references provided by the foreign domain. By default dma-buf is backed by system memory pages, but by providing GNTDEV_DMA_FLAG_XXX flags it can also be created as a DMA write-combine/coherent buffer, e.g. allocated with corresponding dma_alloc_xxx API.
Export the resulting buffer as a new dma-buf.

2. Implement waiting for the dma-buf to be released: block until the dma-buf with the file descriptor provided is released.
If within the time-out provided the buffer is not released then -ETIMEDOUT error is returned. If the buffer with the file descriptor does not exist or has already been released, then -ENOENT is returned. For valid file descriptors this must not be treated as error.

3. Make gntdev's common code and structures available to dma-buf.

---
 drivers/xen/gntdev-common.h |   4 +
 drivers/xen/gntdev-dmabuf.c | 470 +++++++++++++++++++++++++++++++++++-
 drivers/xen/gntdev.c        |  10 +
 3 files changed, 482 insertions(+), 2 deletions(-)
drivers/xen/gntdev-common.h +4 -0
index a3408fd39b07..72f80dbce861 100644
@@ -89,4 +89,8 @@ bool gntdev_account_mapped_pages(int count);
 
 int gntdev_map_grant_pages(struct gntdev_grant_map *map);
 
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct gntdev_grant_map *map);
+#endif
+
 #endif
drivers/xen/gntdev-dmabuf.c +468 -2

504 lines not on this page · raw message

drivers/xen/gntdev.c +10 -0
index e82660d81d7e..5f93cd534840 100644
@@ -262,6 +262,16 @@ void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
 	gntdev_free_map(map);
 }
 
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
+{
+	mutex_lock(&priv->lock);
+	list_del(&map->next);
+	gntdev_put_map(NULL /* already removed */, map);
+	mutex_unlock(&priv->lock);
+}
+#endif
+
 /* ------------------------------------------------------------------ */
 
 static int find_grant_ptes(pte_t *pte, pgtable_t token,
Signature · 6 lines
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#48 lore raw

Re: [PATCH v3 8/9] xen/gntdev: Implement dma-buf export functionality

Quoted 127 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

1. Create a dma-buf from grant references provided by the foreign domain. By default dma-buf is backed by system memory pages, but by providing GNTDEV_DMA_FLAG_XXX flags it can also be created as a DMA write-combine/coherent buffer, e.g. allocated with corresponding dma_alloc_xxx API.
Export the resulting buffer as a new dma-buf.

2. Implement waiting for the dma-buf to be released: block until the dma-buf with the file descriptor provided is released. If within the time-out provided the buffer is not released then -ETIMEDOUT error is returned. If the buffer with the file descriptor does not exist or has already been released, then -ENOENT is returned. For valid file descriptors this must not be treated as error.

3. Make gntdev's common code and structures available to dma-buf.

---
  drivers/xen/gntdev-common.h |   4 +
  drivers/xen/gntdev-dmabuf.c | 470 +++++++++++++++++++++++++++++++++++-
  drivers/xen/gntdev.c        |  10 +
  3 files changed, 482 insertions(+), 2 deletions(-)
drivers/xen/gntdev-common.h +4 -0
index a3408fd39b07..72f80dbce861 100644
@@ -89,4 +89,8 @@ bool gntdev_account_mapped_pages(int count);
 
  int gntdev_map_grant_pages(struct gntdev_grant_map *map);
 
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct gntdev_grant_map *map);
+#endif
+
  #endif
drivers/xen/gntdev-dmabuf.c +66 -0
index dc57c6a25525..84cba67c6ad7 100644
@@ -3,13 +3,53 @@
  /*
   * Xen dma-buf functionality for gntdev.
   *
+ * DMA buffer implementation is based on drivers/gpu/drm/drm_prime.c.
+ *
   * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
   */
 
+#include <linux/dma-buf.h>
  #include <linux/slab.h>
 
+#include <xen/grant_table.h>
+#include <xen/gntdev.h>
+
+#include "gntdev-common.h"
  #include "gntdev-dmabuf.h"
 
+struct gntdev_dmabuf {
+	struct gntdev_dmabuf_priv *priv;
+	struct dma_buf *dmabuf;
+	struct list_head next;
+	int fd;
+
+	union {
+		struct {
+			/* Exported buffers are reference counted. */
+			struct kref refcount;
+
+			struct gntdev_priv *priv;
+			struct gntdev_grant_map *map;
+		} exp;
+	} u;
+
+	/* Number of pages this buffer has. */
+	int nr_pages;
+	/* Pages of this buffer. */
+	struct page **pages;
+};
+
+struct gntdev_dmabuf_wait_obj {
+	struct list_head next;
+	struct gntdev_dmabuf *gntdev_dmabuf;
+	struct completion completion;
+};
+
+struct gntdev_dmabuf_attachment {
+	struct sg_table *sgt;
+	enum dma_data_direction dir;
+};
+
  struct gntdev_dmabuf_priv {
  	/* List of exported DMA buffers. */
  	struct list_head exp_list;
@@ -23,17 +63,439 @@ struct gntdev_dmabuf_priv {
 
  /* Implementation of wait for exported DMA buffer to be released. */
 
+static void dmabuf_exp_release(struct kref *kref);
+
+static struct gntdev_dmabuf_wait_obj *
+dmabuf_exp_wait_obj_new(struct gntdev_dmabuf_priv *priv,
+			struct gntdev_dmabuf *gntdev_dmabuf)
+{
+	struct gntdev_dmabuf_wait_obj *obj;
+
+	obj = kzalloc(sizeof(*obj), GFP_KERNEL);
+	if (!obj)
+		return ERR_PTR(-ENOMEM);
+
+	init_completion(&obj->completion);
+	obj->gntdev_dmabuf = gntdev_dmabuf;
+
+	mutex_lock(&priv->lock);
+	list_add(&obj->next, &priv->exp_wait_list);
+	/* Put our reference and wait for gntdev_dmabuf's release to fire. */
+	kref_put(&gntdev_dmabuf->u.exp.refcount, dmabuf_exp_release);
+	mutex_unlock(&priv->lock);
+	return obj;
+}
+
+static void dmabuf_exp_wait_obj_free(struct gntdev_dmabuf_priv *priv,
+				     struct gntdev_dmabuf_wait_obj *obj)
+{

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 101

Quoted 10 lines
drivers/xen/gntdev-dmabuf.c +10 -0
+	struct gntdev_dmabuf_wait_obj *cur_obj, *q;
+
+	mutex_lock(&priv->lock);
+	list_for_each_entry_safe(cur_obj, q, &priv->exp_wait_list, next)
+		if (cur_obj == obj) {
+			list_del(&obj->next);
+			kfree(obj);
+			break;
+		}
+	mutex_unlock(&priv->lock);

Do we really need to walk the list?

And if we do, do we need the safe variant of the walk? We are holding the lock. Here and elsewhere.

Quoted 17 lines
drivers/xen/gntdev-dmabuf.c +17 -0
+}
+
+static int dmabuf_exp_wait_obj_wait(struct gntdev_dmabuf_wait_obj *obj,
+				    u32 wait_to_ms)
+{
+	if (wait_for_completion_timeout(&obj->completion,
+			msecs_to_jiffies(wait_to_ms)) <= 0)
+		return -ETIMEDOUT;
+
+	return 0;
+}
+
+static void dmabuf_exp_wait_obj_signal(struct gntdev_dmabuf_priv *priv,
+				       struct gntdev_dmabuf *gntdev_dmabuf)
+{
+	struct gntdev_dmabuf_wait_obj *obj, *q;
+

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 128

Quoted 10 lines
drivers/xen/gntdev-dmabuf.c +10 -0
+	list_for_each_entry_safe(obj, q, &priv->exp_wait_list, next)
+		if (obj->gntdev_dmabuf == gntdev_dmabuf) {
+			pr_debug("Found gntdev_dmabuf in the wait list, wake\n");
+			complete_all(&obj->completion);
+			break;
+		}
+}
+
+static struct gntdev_dmabuf *
+dmabuf_exp_wait_obj_get_by_fd(struct gntdev_dmabuf_priv *priv, int fd)

The name of this routine implies (to me) that we are getting a wait object but IIUIC we are getting a gntdev_dmabuf that we are going to later associate with a wait object.

Quoted 235 lines
drivers/xen/gntdev-dmabuf.c +231 -1
+{
+	struct gntdev_dmabuf *q, *gntdev_dmabuf, *ret = ERR_PTR(-ENOENT);
+
+	mutex_lock(&priv->lock);
+	list_for_each_entry_safe(gntdev_dmabuf, q, &priv->exp_list, next)
+		if (gntdev_dmabuf->fd == fd) {
+			pr_debug("Found gntdev_dmabuf in the wait list\n");
+			kref_get(&gntdev_dmabuf->u.exp.refcount);
+			ret = gntdev_dmabuf;
+			break;
+		}
+	mutex_unlock(&priv->lock);
+	return ret;
+}
+
  int gntdev_dmabuf_exp_wait_released(struct gntdev_dmabuf_priv *priv, int fd,
  				    int wait_to_ms)
  {
-	return -EINVAL;
+	struct gntdev_dmabuf *gntdev_dmabuf;
+	struct gntdev_dmabuf_wait_obj *obj;
+	int ret;
+
+	pr_debug("Will wait for dma-buf with fd %d\n", fd);
+	/*
+	 * Try to find the DMA buffer: if not found means that
+	 * either the buffer has already been released or file descriptor
+	 * provided is wrong.
+	 */
+	gntdev_dmabuf = dmabuf_exp_wait_obj_get_by_fd(priv, fd);
+	if (IS_ERR(gntdev_dmabuf))
+		return PTR_ERR(gntdev_dmabuf);
+
+	/*
+	 * gntdev_dmabuf still exists and is reference count locked by us now,
+	 * so prepare to wait: allocate wait object and add it to the wait list,
+	 * so we can find it on release.
+	 */
+	obj = dmabuf_exp_wait_obj_new(priv, gntdev_dmabuf);
+	if (IS_ERR(obj))
+		return PTR_ERR(obj);
+
+	ret = dmabuf_exp_wait_obj_wait(obj, wait_to_ms);
+	dmabuf_exp_wait_obj_free(priv, obj);
+	return ret;
+}
+
+/* DMA buffer export support. */
+
+static struct sg_table *
+dmabuf_pages_to_sgt(struct page **pages, unsigned int nr_pages)
+{
+	struct sg_table *sgt;
+	int ret;
+
+	sgt = kmalloc(sizeof(*sgt), GFP_KERNEL);
+	if (!sgt) {
+		ret = -ENOMEM;
+		goto out;
+	}
+
+	ret = sg_alloc_table_from_pages(sgt, pages, nr_pages, 0,
+					nr_pages << PAGE_SHIFT,
+					GFP_KERNEL);
+	if (ret)
+		goto out;
+
+	return sgt;
+
+out:
+	kfree(sgt);
+	return ERR_PTR(ret);
+}
+
+static int dmabuf_exp_ops_attach(struct dma_buf *dma_buf,
+				 struct device *target_dev,
+				 struct dma_buf_attachment *attach)
+{
+	struct gntdev_dmabuf_attachment *gntdev_dmabuf_attach;
+
+	gntdev_dmabuf_attach = kzalloc(sizeof(*gntdev_dmabuf_attach),
+				       GFP_KERNEL);
+	if (!gntdev_dmabuf_attach)
+		return -ENOMEM;
+
+	gntdev_dmabuf_attach->dir = DMA_NONE;
+	attach->priv = gntdev_dmabuf_attach;
+	return 0;
+}
+
+static void dmabuf_exp_ops_detach(struct dma_buf *dma_buf,
+				  struct dma_buf_attachment *attach)
+{
+	struct gntdev_dmabuf_attachment *gntdev_dmabuf_attach = attach->priv;
+
+	if (gntdev_dmabuf_attach) {
+		struct sg_table *sgt = gntdev_dmabuf_attach->sgt;
+
+		if (sgt) {
+			if (gntdev_dmabuf_attach->dir != DMA_NONE)
+				dma_unmap_sg_attrs(attach->dev, sgt->sgl,
+						   sgt->nents,
+						   gntdev_dmabuf_attach->dir,
+						   DMA_ATTR_SKIP_CPU_SYNC);
+			sg_free_table(sgt);
+		}
+
+		kfree(sgt);
+		kfree(gntdev_dmabuf_attach);
+		attach->priv = NULL;
+	}
+}
+
+static struct sg_table *
+dmabuf_exp_ops_map_dma_buf(struct dma_buf_attachment *attach,
+			   enum dma_data_direction dir)
+{
+	struct gntdev_dmabuf_attachment *gntdev_dmabuf_attach = attach->priv;
+	struct gntdev_dmabuf *gntdev_dmabuf = attach->dmabuf->priv;
+	struct sg_table *sgt;
+
+	pr_debug("Mapping %d pages for dev %p\n", gntdev_dmabuf->nr_pages,
+		 attach->dev);
+
+	if (dir == DMA_NONE || !gntdev_dmabuf_attach)
+		return ERR_PTR(-EINVAL);
+
+	/* Return the cached mapping when possible. */
+	if (gntdev_dmabuf_attach->dir == dir)
+		return gntdev_dmabuf_attach->sgt;
+
+	/*
+	 * Two mappings with different directions for the same attachment are
+	 * not allowed.
+	 */
+	if (gntdev_dmabuf_attach->dir != DMA_NONE)
+		return ERR_PTR(-EBUSY);
+
+	sgt = dmabuf_pages_to_sgt(gntdev_dmabuf->pages,
+				  gntdev_dmabuf->nr_pages);
+	if (!IS_ERR(sgt)) {
+		if (!dma_map_sg_attrs(attach->dev, sgt->sgl, sgt->nents, dir,
+				      DMA_ATTR_SKIP_CPU_SYNC)) {
+			sg_free_table(sgt);
+			kfree(sgt);
+			sgt = ERR_PTR(-ENOMEM);
+		} else {
+			gntdev_dmabuf_attach->sgt = sgt;
+			gntdev_dmabuf_attach->dir = dir;
+		}
+	}
+	if (IS_ERR(sgt))
+		pr_debug("Failed to map sg table for dev %p\n", attach->dev);
+	return sgt;
+}
+
+static void dmabuf_exp_ops_unmap_dma_buf(struct dma_buf_attachment *attach,
+					 struct sg_table *sgt,
+					 enum dma_data_direction dir)
+{
+	/* Not implemented. The unmap is done at dmabuf_exp_ops_detach(). */
+}
+
+static void dmabuf_exp_release(struct kref *kref)
+{
+	struct gntdev_dmabuf *gntdev_dmabuf =
+		container_of(kref, struct gntdev_dmabuf, u.exp.refcount);
+
+	dmabuf_exp_wait_obj_signal(gntdev_dmabuf->priv, gntdev_dmabuf);
+	list_del(&gntdev_dmabuf->next);
+	kfree(gntdev_dmabuf);
+}
+
+static void dmabuf_exp_ops_release(struct dma_buf *dma_buf)
+{
+	struct gntdev_dmabuf *gntdev_dmabuf = dma_buf->priv;
+	struct gntdev_dmabuf_priv *priv = gntdev_dmabuf->priv;
+
+	gntdev_remove_map(gntdev_dmabuf->u.exp.priv, gntdev_dmabuf->u.exp.map);
+	mutex_lock(&priv->lock);
+	kref_put(&gntdev_dmabuf->u.exp.refcount, dmabuf_exp_release);
+	mutex_unlock(&priv->lock);
+}
+
+static void *dmabuf_exp_ops_kmap_atomic(struct dma_buf *dma_buf,
+					unsigned long page_num)
+{
+	/* Not implemented. */
+	return NULL;
+}
+
+static void dmabuf_exp_ops_kunmap_atomic(struct dma_buf *dma_buf,
+					 unsigned long page_num, void *addr)
+{
+	/* Not implemented. */
+}
+
+static void *dmabuf_exp_ops_kmap(struct dma_buf *dma_buf,
+				 unsigned long page_num)
+{
+	/* Not implemented. */
+	return NULL;
+}
+
+static void dmabuf_exp_ops_kunmap(struct dma_buf *dma_buf,
+				  unsigned long page_num, void *addr)
+{
+	/* Not implemented. */
+}
+
+static int dmabuf_exp_ops_mmap(struct dma_buf *dma_buf,
+			       struct vm_area_struct *vma)
+{
+	/* Not implemented. */
+	return 0;
+}
+
+static const struct dma_buf_ops dmabuf_exp_ops =  {
+	.attach = dmabuf_exp_ops_attach,
+	.detach = dmabuf_exp_ops_detach,
+	.map_dma_buf = dmabuf_exp_ops_map_dma_buf,
+	.unmap_dma_buf = dmabuf_exp_ops_unmap_dma_buf,
+	.release = dmabuf_exp_ops_release,
+	.map = dmabuf_exp_ops_kmap,
+	.map_atomic = dmabuf_exp_ops_kmap_atomic,
+	.unmap = dmabuf_exp_ops_kunmap,
+	.unmap_atomic = dmabuf_exp_ops_kunmap_atomic,
+	.mmap = dmabuf_exp_ops_mmap,
+};
+
+struct gntdev_dmabuf_export_args {
+	struct gntdev_priv *priv;
+	struct gntdev_grant_map *map;
+	struct gntdev_dmabuf_priv *dmabuf_priv;
+	struct device *dev;

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 372

Quoted 10 lines
drivers/xen/gntdev-dmabuf.c +10 -0
+	int count;
+	struct page **pages;
+	u32 fd;
+};
+
+static int dmabuf_exp_from_pages(struct gntdev_dmabuf_export_args *args)
+{
+	DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
+	struct gntdev_dmabuf *gntdev_dmabuf;
+	int ret = 0;

Not necessary.

Quoted 53 lines
drivers/xen/gntdev-dmabuf.c +53 -0
+
+	gntdev_dmabuf = kzalloc(sizeof(*gntdev_dmabuf), GFP_KERNEL);
+	if (!gntdev_dmabuf)
+		return -ENOMEM;
+
+	kref_init(&gntdev_dmabuf->u.exp.refcount);
+
+	gntdev_dmabuf->priv = args->dmabuf_priv;
+	gntdev_dmabuf->nr_pages = args->count;
+	gntdev_dmabuf->pages = args->pages;
+	gntdev_dmabuf->u.exp.priv = args->priv;
+	gntdev_dmabuf->u.exp.map = args->map;
+
+	exp_info.exp_name = KBUILD_MODNAME;
+	if (args->dev->driver && args->dev->driver->owner)
+		exp_info.owner = args->dev->driver->owner;
+	else
+		exp_info.owner = THIS_MODULE;
+	exp_info.ops = &dmabuf_exp_ops;
+	exp_info.size = args->count << PAGE_SHIFT;
+	exp_info.flags = O_RDWR;
+	exp_info.priv = gntdev_dmabuf;
+
+	gntdev_dmabuf->dmabuf = dma_buf_export(&exp_info);
+	if (IS_ERR(gntdev_dmabuf->dmabuf)) {
+		ret = PTR_ERR(gntdev_dmabuf->dmabuf);
+		gntdev_dmabuf->dmabuf = NULL;
+		goto fail;
+	}
+
+	ret = dma_buf_fd(gntdev_dmabuf->dmabuf, O_CLOEXEC);
+	if (ret < 0)
+		goto fail;
+
+	gntdev_dmabuf->fd = ret;
+	args->fd = ret;
+
+	pr_debug("Exporting DMA buffer with fd %d\n", ret);
+
+	mutex_lock(&args->dmabuf_priv->lock);
+	list_add(&gntdev_dmabuf->next, &args->dmabuf_priv->exp_list);
+	mutex_unlock(&args->dmabuf_priv->lock);
+	return 0;
+
+fail:
+	if (gntdev_dmabuf->dmabuf)
+		dma_buf_put(gntdev_dmabuf->dmabuf);
+	kfree(gntdev_dmabuf);
+	return ret;
+}
+
+static struct gntdev_grant_map *
+dmabuf_exp_alloc_backing_storage(struct gntdev_priv *priv, int dmabuf_flags,

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 435

Quoted 10 lines
drivers/xen/gntdev-dmabuf.c +10 -0
+				 int count)
+{
+	struct gntdev_grant_map *map;
+
+	if (unlikely(count <= 0))
+		return ERR_PTR(-EINVAL);
+
+	if ((dmabuf_flags & GNTDEV_DMA_FLAG_WC) &&
+	    (dmabuf_flags & GNTDEV_DMA_FLAG_COHERENT)) {
+		pr_debug("Wrong dma-buf flags: either WC or coherent, not both\n");

Why not just print the value of the flags?

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 444

Quoted 9 lines
drivers/xen/gntdev-dmabuf.c +9 -0
+		return ERR_PTR(-EINVAL);
+	}
+
+	map = gntdev_alloc_map(priv, count, dmabuf_flags);
+	if (!map)
+		return ERR_PTR(-ENOMEM);
+
+	if (unlikely(gntdev_account_mapped_pages(count))) {
+		pr_debug("can't map: over limit\n");

I think printing @count value here would be useful.

Quoted 4 lines
drivers/xen/gntdev-dmabuf.c +4 -0
+		gntdev_put_map(NULL, map);
+		return ERR_PTR(-ENOMEM);
+	}
+	return map;

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 458

Quoted 10 lines
drivers/xen/gntdev-dmabuf.c +4 -0
  }
 
  int gntdev_dmabuf_exp_from_refs(struct gntdev_priv *priv, int flags,
  				int count, u32 domid, u32 *refs, u32 *fd)
  {
+	struct gntdev_grant_map *map;
+	struct gntdev_dmabuf_export_args args;
+	int i, ret;
+
  	*fd = -1;

Is this still needed?

Quoted 60 lines
drivers/xen/gntdev-dmabuf.c +44 -1
-	return -EINVAL;
+
+	map = dmabuf_exp_alloc_backing_storage(priv, flags, count);
+	if (IS_ERR(map))
+		return PTR_ERR(map);
+
+	for (i = 0; i < count; i++) {
+		map->grants[i].domid = domid;
+		map->grants[i].ref = refs[i];
+	}
+
+	mutex_lock(&priv->lock);
+	gntdev_add_map(priv, map);
+	mutex_unlock(&priv->lock);
+
+	map->flags |= GNTMAP_host_map;
+#if defined(CONFIG_X86)
+	map->flags |= GNTMAP_device_map;
+#endif
+
+	ret = gntdev_map_grant_pages(map);
+	if (ret < 0)
+		goto out;
+
+	args.priv = priv;
+	args.map = map;
+	args.dev = priv->dma_dev;
+	args.dmabuf_priv = priv->dmabuf_priv;
+	args.count = map->count;
+	args.pages = map->pages;
+
+	ret = dmabuf_exp_from_pages(&args);
+	if (ret < 0)
+		goto out;
+
+	*fd = args.fd;
+	return 0;
+
+out:
+	gntdev_remove_map(priv, map);
+	return ret;
  }
 
  /* DMA buffer import support. */
@@ -63,6 +525,10 @@ struct gntdev_dmabuf_priv *gntdev_dmabuf_init(void)
  	if (!priv)
  		return ERR_PTR(-ENOMEM);
 
+	mutex_init(&priv->lock);
+	INIT_LIST_HEAD(&priv->exp_list);
+	INIT_LIST_HEAD(&priv->exp_wait_list);
+
  	return priv;
  }
 
drivers/xen/gntdev.c +0 -0
index e82660d81d7e..5f93cd534840 100644

drivers/xen/gntdev.c void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map) · line 270

Quoted 9 lines
drivers/xen/gntdev.c +6 -0
@@ -262,6 +262,16 @@ void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
  	gntdev_free_map(map);
  }
 
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
+{
+	mutex_lock(&priv->lock);
+	list_del(&map->next);
+	gntdev_put_map(NULL /* already removed */, map);

Why not pass call gntdev_put_map(priv, map) and then not have this routine at all?

I really dislike the fact that we are taking a lock here that gntdev_put_map() takes as well, although not with NULL argument. (And yes, I see that gntdev_release() does it too.)

-boris

Quoted 8 lines
drivers/xen/gntdev.c +4 -0
+	mutex_unlock(&priv->lock);
+}
+#endif
+
  /* ------------------------------------------------------------------ */
 
  static int find_grant_ptes(pte_t *pte, pgtable_t token,
#49 lore raw

Re: [PATCH v3 8/9] xen/gntdev: Implement dma-buf export functionality

Quoted 130 lines · Boris Ostrovsky
Quoted 127 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

1. Create a dma-buf from grant references provided by the foreign domain. By default dma-buf is backed by system memory pages, but by providing GNTDEV_DMA_FLAG_XXX flags it can also be created as a DMA write-combine/coherent buffer, e.g. allocated with corresponding dma_alloc_xxx API.
    Export the resulting buffer as a new dma-buf.

2. Implement waiting for the dma-buf to be released: block until the dma-buf with the file descriptor provided is released. If within the time-out provided the buffer is not released then -ETIMEDOUT error is returned. If the buffer with the file descriptor does not exist or has already been released, then -ENOENT is returned. For valid file descriptors this must not be treated as error.

3. Make gntdev's common code and structures available to dma-buf.

---
  drivers/xen/gntdev-common.h |   4 + drivers/xen/gntdev-dmabuf.c | 470 +++++++++++++++++++++++++++++++++++- drivers/xen/gntdev.c        |  10 +

  3 files changed, 482 insertions(+), 2 deletions(-)
drivers/xen/gntdev-common.h +0 -0
index a3408fd39b07..72f80dbce861 100644
@@ -89,4 +89,8 @@ bool gntdev_account_mapped_pages(int count);
     int gntdev_map_grant_pages(struct gntdev_grant_map *map);

  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct gntdev_grant_map *map);

drivers/xen/gntdev-common.h +2 -0
+#endif
+

  #endif

drivers/xen/gntdev-dmabuf.c +2 -0
index dc57c6a25525..84cba67c6ad7 100644
@@ -3,13 +3,53 @@
   /*
    * Xen dma-buf functionality for gntdev.
    *
+ * DMA buffer implementation is based on drivers/gpu/drm/drm_prime.c.
+ *
    * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
    */

  +#include <linux/dma-buf.h>
  #include <linux/slab.h>
  +#include <xen/grant_table.h>

drivers/xen/gntdev-dmabuf.c +3 -0
+#include <xen/gntdev.h>
+
+#include "gntdev-common.h"

  #include "gntdev-dmabuf.h"
  +struct gntdev_dmabuf {
+    struct gntdev_dmabuf_priv *priv;
+    struct dma_buf *dmabuf;
+    struct list_head next;
+    int fd;

drivers/xen/gntdev-dmabuf.c +28 -0
+
+    union {
+        struct {
+            /* Exported buffers are reference counted. */
+            struct kref refcount;
+
+            struct gntdev_priv *priv;
+            struct gntdev_grant_map *map;
+        } exp;
+    } u;
+
+    /* Number of pages this buffer has. */
+    int nr_pages;
+    /* Pages of this buffer. */
+    struct page **pages;
+};
+
+struct gntdev_dmabuf_wait_obj {
+    struct list_head next;
+    struct gntdev_dmabuf *gntdev_dmabuf;
+    struct completion completion;
+};
+
+struct gntdev_dmabuf_attachment {
+    struct sg_table *sgt;
+    enum dma_data_direction dir;
+};
+

  struct gntdev_dmabuf_priv {
      /* List of exported DMA buffers. */
      struct list_head exp_list;

drivers/xen/gntdev-dmabuf.c +17 -0
@@ -23,17 +63,439 @@ struct gntdev_dmabuf_priv {
     /* Implementation of wait for exported DMA buffer to be released. */
   +static void dmabuf_exp_release(struct kref *kref);
+
+static struct gntdev_dmabuf_wait_obj *
+dmabuf_exp_wait_obj_new(struct gntdev_dmabuf_priv *priv,
+            struct gntdev_dmabuf *gntdev_dmabuf)
+{
+    struct gntdev_dmabuf_wait_obj *obj;
+
+    obj = kzalloc(sizeof(*obj), GFP_KERNEL);
+    if (!obj)
+        return ERR_PTR(-ENOMEM);
+
+    init_completion(&obj->completion);
+    obj->gntdev_dmabuf = gntdev_dmabuf;
+
+    mutex_lock(&priv->lock);
+    list_add(&obj->next, &priv->exp_wait_list);
+    /* Put our reference and wait for gntdev_dmabuf's release to 

fire. */
+    kref_put(&gntdev_dmabuf->u.exp.refcount, dmabuf_exp_release); +    mutex_unlock(&priv->lock);
+    return obj;

drivers/xen/gntdev-dmabuf.c +10 -0
+}
+
+static void dmabuf_exp_wait_obj_free(struct gntdev_dmabuf_priv *priv,
+                     struct gntdev_dmabuf_wait_obj *obj)
+{
+    struct gntdev_dmabuf_wait_obj *cur_obj, *q;
+
+    mutex_lock(&priv->lock);
+    list_for_each_entry_safe(cur_obj, q, &priv->exp_wait_list, next)
+        if (cur_obj == obj) {
Quoted 9 lines
Quoted 5 lines

+            list_del(&obj->next);
+            kfree(obj);
+            break;
+        }
+    mutex_unlock(&priv->lock);

Do we really need to walk the list?

It can be deleted without walking the list and no reason to do that walk. Just an example of over-engineering here, thank you for spotting this.

Quoted 2 lines

And if we do, do we need the safe variant of the walk? We are holding the lock. Here and elsewhere.

You are perfectly right. I will not use safe variant of the walk, no need for that

Quoted 23 lines
Quoted 23 lines
drivers/xen/gntdev-dmabuf.c +22 -0
+}
+
+static int dmabuf_exp_wait_obj_wait(struct gntdev_dmabuf_wait_obj *obj,
+                    u32 wait_to_ms)
+{
+    if (wait_for_completion_timeout(&obj->completion,
+            msecs_to_jiffies(wait_to_ms)) <= 0)
+        return -ETIMEDOUT;
+
+    return 0;
+}
+
+static void dmabuf_exp_wait_obj_signal(struct gntdev_dmabuf_priv *priv,
+                       struct gntdev_dmabuf *gntdev_dmabuf)
+{
+    struct gntdev_dmabuf_wait_obj *obj, *q;
+
+    list_for_each_entry_safe(obj, q, &priv->exp_wait_list, next)
+        if (obj->gntdev_dmabuf == gntdev_dmabuf) {
+            pr_debug("Found gntdev_dmabuf in the wait list, wake\n");
+            complete_all(&obj->completion);
+            break;
Quoted 9 lines

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 128

Quoted 5 lines

+        }

drivers/xen/gntdev-dmabuf.c +4 -0
+}
+
+static struct gntdev_dmabuf *
+dmabuf_exp_wait_obj_get_by_fd(struct gntdev_dmabuf_priv *priv, int fd)

The name of this routine implies (to me) that we are getting a wait object but IIUIC we are getting a gntdev_dmabuf that we are going to later associate with a wait object.

How about dmabuf_exp_wait_obj_get_dmabuf_by_fd?
I would like to keep function prefixes, e.g. dmabuf_exp_wait_obj_ just to show to which functionality a routine belongs.

Quoted 250 lines
Quoted 250 lines
drivers/xen/gntdev-dmabuf.c +15 -0
+{
+    struct gntdev_dmabuf *q, *gntdev_dmabuf, *ret = ERR_PTR(-ENOENT);
+
+    mutex_lock(&priv->lock);
+    list_for_each_entry_safe(gntdev_dmabuf, q, &priv->exp_list, next)
+        if (gntdev_dmabuf->fd == fd) {
+            pr_debug("Found gntdev_dmabuf in the wait list\n");
+            kref_get(&gntdev_dmabuf->u.exp.refcount);
+            ret = gntdev_dmabuf;
+            break;
+        }
+    mutex_unlock(&priv->lock);
+    return ret;
+}
+

  int gntdev_dmabuf_exp_wait_released(struct gntdev_dmabuf_priv *priv, int fd, int wait_to_ms)
  {
-    return -EINVAL;
+    struct gntdev_dmabuf *gntdev_dmabuf;
+    struct gntdev_dmabuf_wait_obj *obj;
+    int ret;

drivers/xen/gntdev-dmabuf.c +13 -0
+
+    pr_debug("Will wait for dma-buf with fd %d\n", fd);
+    /*
+     * Try to find the DMA buffer: if not found means that
+     * either the buffer has already been released or file descriptor
+     * provided is wrong.
+     */
+    gntdev_dmabuf = dmabuf_exp_wait_obj_get_by_fd(priv, fd);
+    if (IS_ERR(gntdev_dmabuf))
+        return PTR_ERR(gntdev_dmabuf);
+
+    /*
+     * gntdev_dmabuf still exists and is reference count locked by 

us now,
+     * so prepare to wait: allocate wait object and add it to the wait list,
+     * so we can find it on release.
+     */
+    obj = dmabuf_exp_wait_obj_new(priv, gntdev_dmabuf); +    if (IS_ERR(obj))
+        return PTR_ERR(obj);

drivers/xen/gntdev-dmabuf.c +53 -0
+
+    ret = dmabuf_exp_wait_obj_wait(obj, wait_to_ms);
+    dmabuf_exp_wait_obj_free(priv, obj);
+    return ret;
+}
+
+/* DMA buffer export support. */
+
+static struct sg_table *
+dmabuf_pages_to_sgt(struct page **pages, unsigned int nr_pages)
+{
+    struct sg_table *sgt;
+    int ret;
+
+    sgt = kmalloc(sizeof(*sgt), GFP_KERNEL);
+    if (!sgt) {
+        ret = -ENOMEM;
+        goto out;
+    }
+
+    ret = sg_alloc_table_from_pages(sgt, pages, nr_pages, 0,
+                    nr_pages << PAGE_SHIFT,
+                    GFP_KERNEL);
+    if (ret)
+        goto out;
+
+    return sgt;
+
+out:
+    kfree(sgt);
+    return ERR_PTR(ret);
+}
+
+static int dmabuf_exp_ops_attach(struct dma_buf *dma_buf,
+                 struct device *target_dev,
+                 struct dma_buf_attachment *attach)
+{
+    struct gntdev_dmabuf_attachment *gntdev_dmabuf_attach;
+
+    gntdev_dmabuf_attach = kzalloc(sizeof(*gntdev_dmabuf_attach),
+                       GFP_KERNEL);
+    if (!gntdev_dmabuf_attach)
+        return -ENOMEM;
+
+    gntdev_dmabuf_attach->dir = DMA_NONE;
+    attach->priv = gntdev_dmabuf_attach;
+    return 0;
+}
+
+static void dmabuf_exp_ops_detach(struct dma_buf *dma_buf,
+                  struct dma_buf_attachment *attach)
+{
+    struct gntdev_dmabuf_attachment *gntdev_dmabuf_attach = 

attach->priv;

drivers/xen/gntdev-dmabuf.c +24 -0
+
+    if (gntdev_dmabuf_attach) {
+        struct sg_table *sgt = gntdev_dmabuf_attach->sgt;
+
+        if (sgt) {
+            if (gntdev_dmabuf_attach->dir != DMA_NONE)
+                dma_unmap_sg_attrs(attach->dev, sgt->sgl,
+                           sgt->nents,
+                           gntdev_dmabuf_attach->dir,
+                           DMA_ATTR_SKIP_CPU_SYNC);
+            sg_free_table(sgt);
+        }
+
+        kfree(sgt);
+        kfree(gntdev_dmabuf_attach);
+        attach->priv = NULL;
+    }
+}
+
+static struct sg_table *
+dmabuf_exp_ops_map_dma_buf(struct dma_buf_attachment *attach,
+               enum dma_data_direction dir)
+{
+    struct gntdev_dmabuf_attachment *gntdev_dmabuf_attach = 

attach->priv;
+    struct gntdev_dmabuf *gntdev_dmabuf = attach->dmabuf->priv; +    struct sg_table *sgt;

drivers/xen/gntdev-dmabuf.c +13 -0
+
+    pr_debug("Mapping %d pages for dev %p\n", gntdev_dmabuf->nr_pages,
+         attach->dev);
+
+    if (dir == DMA_NONE || !gntdev_dmabuf_attach)
+        return ERR_PTR(-EINVAL);
+
+    /* Return the cached mapping when possible. */
+    if (gntdev_dmabuf_attach->dir == dir)
+        return gntdev_dmabuf_attach->sgt;
+
+    /*
+     * Two mappings with different directions for the same 

attachment are
+     * not allowed.
+     */
+    if (gntdev_dmabuf_attach->dir != DMA_NONE)
+        return ERR_PTR(-EBUSY);

drivers/xen/gntdev-dmabuf.c +20 -0
+
+    sgt = dmabuf_pages_to_sgt(gntdev_dmabuf->pages,
+                  gntdev_dmabuf->nr_pages);
+    if (!IS_ERR(sgt)) {
+        if (!dma_map_sg_attrs(attach->dev, sgt->sgl, sgt->nents, dir,
+                      DMA_ATTR_SKIP_CPU_SYNC)) {
+            sg_free_table(sgt);
+            kfree(sgt);
+            sgt = ERR_PTR(-ENOMEM);
+        } else {
+            gntdev_dmabuf_attach->sgt = sgt;
+            gntdev_dmabuf_attach->dir = dir;
+        }
+    }
+    if (IS_ERR(sgt))
+        pr_debug("Failed to map sg table for dev %p\n", attach->dev);
+    return sgt;
+}
+
+static void dmabuf_exp_ops_unmap_dma_buf(struct dma_buf_attachment 

*attach,
+                     struct sg_table *sgt, +                     enum dma_data_direction dir) +{
+    /* Not implemented. The unmap is done at dmabuf_exp_ops_detach(). */

drivers/xen/gntdev-dmabuf.c +18 -0
+}
+
+static void dmabuf_exp_release(struct kref *kref)
+{
+    struct gntdev_dmabuf *gntdev_dmabuf =
+        container_of(kref, struct gntdev_dmabuf, u.exp.refcount);
+
+    dmabuf_exp_wait_obj_signal(gntdev_dmabuf->priv, gntdev_dmabuf);
+    list_del(&gntdev_dmabuf->next);
+    kfree(gntdev_dmabuf);
+}
+
+static void dmabuf_exp_ops_release(struct dma_buf *dma_buf)
+{
+    struct gntdev_dmabuf *gntdev_dmabuf = dma_buf->priv;
+    struct gntdev_dmabuf_priv *priv = gntdev_dmabuf->priv;
+
+    gntdev_remove_map(gntdev_dmabuf->u.exp.priv, 

gntdev_dmabuf->u.exp.map);
+    mutex_lock(&priv->lock);
+    kref_put(&gntdev_dmabuf->u.exp.refcount, dmabuf_exp_release); +    mutex_unlock(&priv->lock);

drivers/xen/gntdev-dmabuf.c +59 -0
+}
+
+static void *dmabuf_exp_ops_kmap_atomic(struct dma_buf *dma_buf,
+                    unsigned long page_num)
+{
+    /* Not implemented. */
+    return NULL;
+}
+
+static void dmabuf_exp_ops_kunmap_atomic(struct dma_buf *dma_buf,
+                     unsigned long page_num, void *addr)
+{
+    /* Not implemented. */
+}
+
+static void *dmabuf_exp_ops_kmap(struct dma_buf *dma_buf,
+                 unsigned long page_num)
+{
+    /* Not implemented. */
+    return NULL;
+}
+
+static void dmabuf_exp_ops_kunmap(struct dma_buf *dma_buf,
+                  unsigned long page_num, void *addr)
+{
+    /* Not implemented. */
+}
+
+static int dmabuf_exp_ops_mmap(struct dma_buf *dma_buf,
+                   struct vm_area_struct *vma)
+{
+    /* Not implemented. */
+    return 0;
+}
+
+static const struct dma_buf_ops dmabuf_exp_ops =  {
+    .attach = dmabuf_exp_ops_attach,
+    .detach = dmabuf_exp_ops_detach,
+    .map_dma_buf = dmabuf_exp_ops_map_dma_buf,
+    .unmap_dma_buf = dmabuf_exp_ops_unmap_dma_buf,
+    .release = dmabuf_exp_ops_release,
+    .map = dmabuf_exp_ops_kmap,
+    .map_atomic = dmabuf_exp_ops_kmap_atomic,
+    .unmap = dmabuf_exp_ops_kunmap,
+    .unmap_atomic = dmabuf_exp_ops_kunmap_atomic,
+    .mmap = dmabuf_exp_ops_mmap,
+};
+
+struct gntdev_dmabuf_export_args {
+    struct gntdev_priv *priv;
+    struct gntdev_grant_map *map;
+    struct gntdev_dmabuf_priv *dmabuf_priv;
+    struct device *dev;
+    int count;
+    struct page **pages;
+    u32 fd;
+};
+
+static int dmabuf_exp_from_pages(struct gntdev_dmabuf_export_args 
Quoted 9 lines
Quoted 5 lines

*args)
+{
+    DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
+    struct gntdev_dmabuf *gntdev_dmabuf;
+    int ret = 0;

Not necessary.

Will remove =0;

Quoted 59 lines
Quoted 59 lines
drivers/xen/gntdev-dmabuf.c +53 -0
+
+    gntdev_dmabuf = kzalloc(sizeof(*gntdev_dmabuf), GFP_KERNEL);
+    if (!gntdev_dmabuf)
+        return -ENOMEM;
+
+    kref_init(&gntdev_dmabuf->u.exp.refcount);
+
+    gntdev_dmabuf->priv = args->dmabuf_priv;
+    gntdev_dmabuf->nr_pages = args->count;
+    gntdev_dmabuf->pages = args->pages;
+    gntdev_dmabuf->u.exp.priv = args->priv;
+    gntdev_dmabuf->u.exp.map = args->map;
+
+    exp_info.exp_name = KBUILD_MODNAME;
+    if (args->dev->driver && args->dev->driver->owner)
+        exp_info.owner = args->dev->driver->owner;
+    else
+        exp_info.owner = THIS_MODULE;
+    exp_info.ops = &dmabuf_exp_ops;
+    exp_info.size = args->count << PAGE_SHIFT;
+    exp_info.flags = O_RDWR;
+    exp_info.priv = gntdev_dmabuf;
+
+    gntdev_dmabuf->dmabuf = dma_buf_export(&exp_info);
+    if (IS_ERR(gntdev_dmabuf->dmabuf)) {
+        ret = PTR_ERR(gntdev_dmabuf->dmabuf);
+        gntdev_dmabuf->dmabuf = NULL;
+        goto fail;
+    }
+
+    ret = dma_buf_fd(gntdev_dmabuf->dmabuf, O_CLOEXEC);
+    if (ret < 0)
+        goto fail;
+
+    gntdev_dmabuf->fd = ret;
+    args->fd = ret;
+
+    pr_debug("Exporting DMA buffer with fd %d\n", ret);
+
+    mutex_lock(&args->dmabuf_priv->lock);
+    list_add(&gntdev_dmabuf->next, &args->dmabuf_priv->exp_list);
+    mutex_unlock(&args->dmabuf_priv->lock);
+    return 0;
+
+fail:
+    if (gntdev_dmabuf->dmabuf)
+        dma_buf_put(gntdev_dmabuf->dmabuf);
+    kfree(gntdev_dmabuf);
+    return ret;
+}
+
+static struct gntdev_grant_map *
+dmabuf_exp_alloc_backing_storage(struct gntdev_priv *priv, int 

dmabuf_flags,
+                 int count)

drivers/xen/gntdev-dmabuf.c +4 -0
+{
+    struct gntdev_grant_map *map;
+
+    if (unlikely(count <= 0))
Quoted 8 lines
Quoted 6 lines

+        return ERR_PTR(-EINVAL);
+
+    if ((dmabuf_flags & GNTDEV_DMA_FLAG_WC) &&
+        (dmabuf_flags & GNTDEV_DMA_FLAG_COHERENT)) {
+        pr_debug("Wrong dma-buf flags: either WC or coherent, not both\n");

Why not just print the value of the flags?

Will print hex value of the flags

Quoted 14 lines

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 444

Quoted 10 lines

+        return ERR_PTR(-EINVAL);
+    }

drivers/xen/gntdev-dmabuf.c +7 -0
+
+    map = gntdev_alloc_map(priv, count, dmabuf_flags);
+    if (!map)
+        return ERR_PTR(-ENOMEM);
+
+    if (unlikely(gntdev_account_mapped_pages(count))) {
+        pr_debug("can't map: over limit\n");

I think printing @count value here would be useful.

Will add

Quoted 16 lines
Quoted 14 lines

+        gntdev_put_map(NULL, map);
+        return ERR_PTR(-ENOMEM);
+    }
+    return map;
  }
    int gntdev_dmabuf_exp_from_refs(struct gntdev_priv *priv, int flags, int count, u32 domid, u32 *refs, u32 *fd) {
+    struct gntdev_grant_map *map;
+    struct gntdev_dmabuf_export_args args;
+    int i, ret;
+
      *fd = -1;

Quoted 1 line

Is this still needed?

No, will remove. I was thinking here about if user-space ignores IOCTL return value and tries to use the fd so it fails on -1. But, ok, no reason to fix user-space bugs in the kernel

Quoted 63 lines
Quoted 63 lines

-    return -EINVAL;

drivers/xen/gntdev-dmabuf.c +40 -0
+
+    map = dmabuf_exp_alloc_backing_storage(priv, flags, count);
+    if (IS_ERR(map))
+        return PTR_ERR(map);
+
+    for (i = 0; i < count; i++) {
+        map->grants[i].domid = domid;
+        map->grants[i].ref = refs[i];
+    }
+
+    mutex_lock(&priv->lock);
+    gntdev_add_map(priv, map);
+    mutex_unlock(&priv->lock);
+
+    map->flags |= GNTMAP_host_map;
+#if defined(CONFIG_X86)
+    map->flags |= GNTMAP_device_map;
+#endif
+
+    ret = gntdev_map_grant_pages(map);
+    if (ret < 0)
+        goto out;
+
+    args.priv = priv;
+    args.map = map;
+    args.dev = priv->dma_dev;
+    args.dmabuf_priv = priv->dmabuf_priv;
+    args.count = map->count;
+    args.pages = map->pages;
+
+    ret = dmabuf_exp_from_pages(&args);
+    if (ret < 0)
+        goto out;
+
+    *fd = args.fd;
+    return 0;
+
+out:
+    gntdev_remove_map(priv, map);
+    return ret;

  }
    /* DMA buffer import support. */

drivers/xen/gntdev-dmabuf.c +3 -0
@@ -63,6 +525,10 @@ struct gntdev_dmabuf_priv *gntdev_dmabuf_init(void)
       if (!priv)
           return ERR_PTR(-ENOMEM);
   +    mutex_init(&priv->lock);
+    INIT_LIST_HEAD(&priv->exp_list);
+    INIT_LIST_HEAD(&priv->exp_wait_list);
+
       return priv;
   }

  diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c index e82660d81d7e..5f93cd534840 100644

drivers/xen/gntdev.c +0 -0
@@ -262,6 +262,16 @@ void gntdev_put_map(struct gntdev_priv *priv, 
 struct gntdev_grant_map *map)
       gntdev_free_map(map);
   }

  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct

Quoted 9 lines
Quoted 5 lines

gntdev_grant_map *map)
+{
+    mutex_lock(&priv->lock);
+    list_del(&map->next);
+    gntdev_put_map(NULL /* already removed */, map);

Why not pass call gntdev_put_map(priv, map) and then not have this routine at all?

Well, I wish I could, but the main difference when calling gntdev_put_map(priv, map) with priv != NULL and my code is that:

void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map) {
    [...]
    if (populate_freeable_maps && priv) { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ mutex_lock(&priv->lock);
        list_del(&map->next);
        mutex_unlock(&priv->lock);
    }
    [...]
}

and

#define populate_freeable_maps use_ptemod
                               ^^^^^^^^^^
which means the map will never be removed from the list in my case because use_ptemod == false for dma-buf.
This is why I do that by hand, e.g. remove the item from the list and then pass NULL for priv.

Also, I will remove gntdev_remove_map as I can now access priv->lock and gntdev_put_map directly form gntdev-dmabuf.c

Quoted 2 lines

I really dislike the fact that we are taking a lock here that gntdev_put_map() takes as well, although not with NULL argument. (And yes, I see that gntdev_release() does it too.)

This can be re-factored later I guess?

Quoted 3 lines

-boris

Thank you,
Oleksandr

Quoted 9 lines
Quoted 9 lines

+    mutex_unlock(&priv->lock);

drivers/xen/gntdev.c +3 -0
+}
+#endif
+

  /* ------------------------------------------------------------------ */ static int find_grant_ptes(pte_t *pte, pgtable_t token,

#50 lore raw

Re: [PATCH v3 8/9] xen/gntdev: Implement dma-buf export functionality

Quoted 14 lines · Oleksandr Andrushchenko
Quoted 4 lines · Boris Ostrovsky
Quoted 1 line · Oleksandr Andrushchenko
Quoted 9 lines

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 128

Quoted 3 lines
drivers/xen/gntdev-dmabuf.c +3 -0
+
+static struct gntdev_dmabuf *
+dmabuf_exp_wait_obj_get_by_fd(struct gntdev_dmabuf_priv *priv, int fd)

The name of this routine implies (to me) that we are getting a wait object but IIUIC we are getting a gntdev_dmabuf that we are going to later associate with a wait object.

Quoted 3 lines

How about dmabuf_exp_wait_obj_get_dmabuf_by_fd?
I would like to keep function prefixes, e.g. dmabuf_exp_wait_obj_ just to show to which functionality a routine belongs.

That's too long IMO. If you really want to keep the prefix then let's keep this the way it is. Maybe it's just me who read it that way.

Quoted 40 lines
Quoted 14 lines
Quoted 9 lines

  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
+{
+    mutex_lock(&priv->lock);
+    list_del(&map->next);
+    gntdev_put_map(NULL /* already removed */, map);

Why not pass call gntdev_put_map(priv, map) and then not have this routine at all?

Well, I wish I could, but the main difference when calling gntdev_put_map(priv, map)
with priv != NULL and my code is that:

void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
{
    [...]
    if (populate_freeable_maps && priv) { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ mutex_lock(&priv->lock);
        list_del(&map->next);
        mutex_unlock(&priv->lock);
    }
    [...]
}

and

#define populate_freeable_maps use_ptemod
                               ^^^^^^^^^^
which means the map will never be removed from the list in my case because use_ptemod == false for dma-buf.
This is why I do that by hand, e.g. remove the item from the list and then pass NULL for priv.

Quoted 2 lines

Also, I will remove gntdev_remove_map as I can now access priv->lock and gntdev_put_map directly form gntdev-dmabuf.c

Yes, that's a good idea.

Quoted 6 lines
Quoted 5 lines

I really dislike the fact that we are taking a lock here that gntdev_put_map() takes as well, although not with NULL argument. (And yes, I see that gntdev_release() does it too.)

This can be re-factored later I guess?

OK.

-boris

#51 lore raw

Re: [PATCH v3 8/9] xen/gntdev: Implement dma-buf export functionality

Quoted 12 lines · Boris Ostrovsky
Quoted 11 lines · Oleksandr Andrushchenko
Quoted 10 lines · Boris Ostrovsky

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 128

Quoted 3 lines · Oleksandr Andrushchenko
drivers/xen/gntdev-dmabuf.c +3 -0
+
+static struct gntdev_dmabuf *
+dmabuf_exp_wait_obj_get_by_fd(struct gntdev_dmabuf_priv *priv, int fd)

The name of this routine implies (to me) that we are getting a wait object but IIUIC we are getting a gntdev_dmabuf that we are going to later associate with a wait object.

Quoted 6 lines
Quoted 3 lines

How about dmabuf_exp_wait_obj_get_dmabuf_by_fd?
I would like to keep function prefixes, e.g. dmabuf_exp_wait_obj_ just to show to which functionality a routine belongs.

That's too long IMO. If you really want to keep the prefix then let's keep this the way it is. Maybe it's just me who read it that way.

I'll rename it to dmabuf_exp_wait_obj_get_dmabuf.
As it already wants fd it seems to be clear that
the lookup will be based on it.

Quoted 52 lines
Quoted 38 lines
Quoted 12 lines
Quoted 8 lines

  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
+{
+    mutex_lock(&priv->lock);
+    list_del(&map->next);
+    gntdev_put_map(NULL /* already removed */, map);

Why not pass call gntdev_put_map(priv, map) and then not have this routine at all?

Well, I wish I could, but the main difference when calling gntdev_put_map(priv, map)
with priv != NULL and my code is that:

void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
{
    [...]
    if (populate_freeable_maps && priv) { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ mutex_lock(&priv->lock);
        list_del(&map->next);
        mutex_unlock(&priv->lock);
    }
    [...]
}

and

#define populate_freeable_maps use_ptemod
                               ^^^^^^^^^^
which means the map will never be removed from the list in my case because use_ptemod == false for dma-buf.
This is why I do that by hand, e.g. remove the item from the list and then pass NULL for priv.

Quoted 2 lines

Also, I will remove gntdev_remove_map as I can now access priv->lock and gntdev_put_map directly form gntdev-dmabuf.c

Yes, that's a good idea.

Quoted 5 lines
Quoted 4 lines

I really dislike the fact that we are taking a lock here that gntdev_put_map() takes as well, although not with NULL argument. (And yes, I see that gntdev_release() does it too.)

This can be re-factored later I guess?

OK.

-boris

#52 lore raw

Re: [PATCH v3 8/9] xen/gntdev: Implement dma-buf export functionality

In reply to Boris Ostrovsky
Quoted 12 lines · Boris Ostrovsky
Quoted 11 lines · Oleksandr Andrushchenko
Quoted 10 lines · Boris Ostrovsky

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 128

Quoted 3 lines · Oleksandr Andrushchenko
drivers/xen/gntdev-dmabuf.c +3 -0
+
+static struct gntdev_dmabuf *
+dmabuf_exp_wait_obj_get_by_fd(struct gntdev_dmabuf_priv *priv, int fd)

The name of this routine implies (to me) that we are getting a wait object but IIUIC we are getting a gntdev_dmabuf that we are going to later associate with a wait object.

Quoted 6 lines
Quoted 3 lines

How about dmabuf_exp_wait_obj_get_dmabuf_by_fd?
I would like to keep function prefixes, e.g. dmabuf_exp_wait_obj_ just to show to which functionality a routine belongs.

That's too long IMO. If you really want to keep the prefix then let's keep this the way it is. Maybe it's just me who read it that way.

I'll rename it to dmabuf_exp_wait_obj_get_dmabuf.
As it already wants fd it seems to be clear that
the lookup will be based on it.

Quoted 43 lines
Quoted 38 lines
Quoted 12 lines
Quoted 8 lines

  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
+{
+    mutex_lock(&priv->lock);
+    list_del(&map->next);
+    gntdev_put_map(NULL /* already removed */, map);

Why not pass call gntdev_put_map(priv, map) and then not have this routine at all?

Well, I wish I could, but the main difference when calling gntdev_put_map(priv, map)
with priv != NULL and my code is that:

void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
{
    [...]
    if (populate_freeable_maps && priv) { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ mutex_lock(&priv->lock);
        list_del(&map->next);
        mutex_unlock(&priv->lock);
    }
    [...]
}

and

#define populate_freeable_maps use_ptemod
                               ^^^^^^^^^^
which means the map will never be removed from the list in my case because use_ptemod == false for dma-buf.
This is why I do that by hand, e.g. remove the item from the list and then pass NULL for priv.

Quoted 2 lines

Also, I will remove gntdev_remove_map as I can now access priv->lock and gntdev_put_map directly form gntdev-dmabuf.c

Yes, that's a good idea.

Quoted 9 lines
Quoted 5 lines
Quoted 4 lines

I really dislike the fact that we are taking a lock here that gntdev_put_map() takes as well, although not with NULL argument. (And yes, I see that gntdev_release() does it too.)

This can be re-factored later I guess?

OK.

-boris

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#53 lore raw

Re: [PATCH v3 8/9] xen/gntdev: Implement dma-buf export functionality

In reply to Oleksandr Andrushchenko
Quoted 14 lines · Oleksandr Andrushchenko
Quoted 4 lines · Boris Ostrovsky
Quoted 1 line · Oleksandr Andrushchenko
Quoted 9 lines

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 128

Quoted 3 lines
drivers/xen/gntdev-dmabuf.c +3 -0
+
+static struct gntdev_dmabuf *
+dmabuf_exp_wait_obj_get_by_fd(struct gntdev_dmabuf_priv *priv, int fd)

The name of this routine implies (to me) that we are getting a wait object but IIUIC we are getting a gntdev_dmabuf that we are going to later associate with a wait object.

Quoted 3 lines

How about dmabuf_exp_wait_obj_get_dmabuf_by_fd?
I would like to keep function prefixes, e.g. dmabuf_exp_wait_obj_ just to show to which functionality a routine belongs.

That's too long IMO. If you really want to keep the prefix then let's keep this the way it is. Maybe it's just me who read it that way.

Quoted 40 lines
Quoted 14 lines
Quoted 9 lines

  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
+{
+    mutex_lock(&priv->lock);
+    list_del(&map->next);
+    gntdev_put_map(NULL /* already removed */, map);

Why not pass call gntdev_put_map(priv, map) and then not have this routine at all?

Well, I wish I could, but the main difference when calling gntdev_put_map(priv, map)
with priv != NULL and my code is that:

void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
{
    [...]
    if (populate_freeable_maps && priv) { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ mutex_lock(&priv->lock);
        list_del(&map->next);
        mutex_unlock(&priv->lock);
    }
    [...]
}

and

#define populate_freeable_maps use_ptemod
                               ^^^^^^^^^^
which means the map will never be removed from the list in my case because use_ptemod == false for dma-buf.
This is why I do that by hand, e.g. remove the item from the list and then pass NULL for priv.

Quoted 2 lines

Also, I will remove gntdev_remove_map as I can now access priv->lock and gntdev_put_map directly form gntdev-dmabuf.c

Yes, that's a good idea.

Quoted 6 lines
Quoted 5 lines

I really dislike the fact that we are taking a lock here that gntdev_put_map() takes as well, although not with NULL argument. (And yes, I see that gntdev_release() does it too.)

This can be re-factored later I guess?

OK.

-boris

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#54 lore raw

Re: [PATCH v3 8/9] xen/gntdev: Implement dma-buf export functionality

In reply to Boris Ostrovsky
Quoted 130 lines · Boris Ostrovsky
Quoted 127 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

1. Create a dma-buf from grant references provided by the foreign domain. By default dma-buf is backed by system memory pages, but by providing GNTDEV_DMA_FLAG_XXX flags it can also be created as a DMA write-combine/coherent buffer, e.g. allocated with corresponding dma_alloc_xxx API.
    Export the resulting buffer as a new dma-buf.

2. Implement waiting for the dma-buf to be released: block until the dma-buf with the file descriptor provided is released. If within the time-out provided the buffer is not released then -ETIMEDOUT error is returned. If the buffer with the file descriptor does not exist or has already been released, then -ENOENT is returned. For valid file descriptors this must not be treated as error.

3. Make gntdev's common code and structures available to dma-buf.

---
  drivers/xen/gntdev-common.h |   4 + drivers/xen/gntdev-dmabuf.c | 470 +++++++++++++++++++++++++++++++++++- drivers/xen/gntdev.c        |  10 +

  3 files changed, 482 insertions(+), 2 deletions(-)
drivers/xen/gntdev-common.h +0 -0
index a3408fd39b07..72f80dbce861 100644
@@ -89,4 +89,8 @@ bool gntdev_account_mapped_pages(int count);
     int gntdev_map_grant_pages(struct gntdev_grant_map *map);

  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct gntdev_grant_map *map);

drivers/xen/gntdev-common.h +2 -0
+#endif
+

  #endif

drivers/xen/gntdev-dmabuf.c +2 -0
index dc57c6a25525..84cba67c6ad7 100644
@@ -3,13 +3,53 @@
   /*
    * Xen dma-buf functionality for gntdev.
    *
+ * DMA buffer implementation is based on drivers/gpu/drm/drm_prime.c.
+ *
    * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
    */

  +#include <linux/dma-buf.h>
  #include <linux/slab.h>
  +#include <xen/grant_table.h>

drivers/xen/gntdev-dmabuf.c +3 -0
+#include <xen/gntdev.h>
+
+#include "gntdev-common.h"

  #include "gntdev-dmabuf.h"
  +struct gntdev_dmabuf {
+    struct gntdev_dmabuf_priv *priv;
+    struct dma_buf *dmabuf;
+    struct list_head next;
+    int fd;

drivers/xen/gntdev-dmabuf.c +28 -0
+
+    union {
+        struct {
+            /* Exported buffers are reference counted. */
+            struct kref refcount;
+
+            struct gntdev_priv *priv;
+            struct gntdev_grant_map *map;
+        } exp;
+    } u;
+
+    /* Number of pages this buffer has. */
+    int nr_pages;
+    /* Pages of this buffer. */
+    struct page **pages;
+};
+
+struct gntdev_dmabuf_wait_obj {
+    struct list_head next;
+    struct gntdev_dmabuf *gntdev_dmabuf;
+    struct completion completion;
+};
+
+struct gntdev_dmabuf_attachment {
+    struct sg_table *sgt;
+    enum dma_data_direction dir;
+};
+

  struct gntdev_dmabuf_priv {
      /* List of exported DMA buffers. */
      struct list_head exp_list;

drivers/xen/gntdev-dmabuf.c +17 -0
@@ -23,17 +63,439 @@ struct gntdev_dmabuf_priv {
     /* Implementation of wait for exported DMA buffer to be released. */
   +static void dmabuf_exp_release(struct kref *kref);
+
+static struct gntdev_dmabuf_wait_obj *
+dmabuf_exp_wait_obj_new(struct gntdev_dmabuf_priv *priv,
+            struct gntdev_dmabuf *gntdev_dmabuf)
+{
+    struct gntdev_dmabuf_wait_obj *obj;
+
+    obj = kzalloc(sizeof(*obj), GFP_KERNEL);
+    if (!obj)
+        return ERR_PTR(-ENOMEM);
+
+    init_completion(&obj->completion);
+    obj->gntdev_dmabuf = gntdev_dmabuf;
+
+    mutex_lock(&priv->lock);
+    list_add(&obj->next, &priv->exp_wait_list);
+    /* Put our reference and wait for gntdev_dmabuf's release to 

fire. */
+    kref_put(&gntdev_dmabuf->u.exp.refcount, dmabuf_exp_release); +    mutex_unlock(&priv->lock);
+    return obj;

drivers/xen/gntdev-dmabuf.c +10 -0
+}
+
+static void dmabuf_exp_wait_obj_free(struct gntdev_dmabuf_priv *priv,
+                     struct gntdev_dmabuf_wait_obj *obj)
+{
+    struct gntdev_dmabuf_wait_obj *cur_obj, *q;
+
+    mutex_lock(&priv->lock);
+    list_for_each_entry_safe(cur_obj, q, &priv->exp_wait_list, next)
+        if (cur_obj == obj) {
Quoted 9 lines
Quoted 5 lines

+            list_del(&obj->next);
+            kfree(obj);
+            break;
+        }
+    mutex_unlock(&priv->lock);

Do we really need to walk the list?

It can be deleted without walking the list and no reason to do that walk. Just an example of over-engineering here, thank you for spotting this.

Quoted 2 lines

And if we do, do we need the safe variant of the walk? We are holding the lock. Here and elsewhere.

You are perfectly right. I will not use safe variant of the walk, no need for that

Quoted 23 lines
Quoted 23 lines
drivers/xen/gntdev-dmabuf.c +22 -0
+}
+
+static int dmabuf_exp_wait_obj_wait(struct gntdev_dmabuf_wait_obj *obj,
+                    u32 wait_to_ms)
+{
+    if (wait_for_completion_timeout(&obj->completion,
+            msecs_to_jiffies(wait_to_ms)) <= 0)
+        return -ETIMEDOUT;
+
+    return 0;
+}
+
+static void dmabuf_exp_wait_obj_signal(struct gntdev_dmabuf_priv *priv,
+                       struct gntdev_dmabuf *gntdev_dmabuf)
+{
+    struct gntdev_dmabuf_wait_obj *obj, *q;
+
+    list_for_each_entry_safe(obj, q, &priv->exp_wait_list, next)
+        if (obj->gntdev_dmabuf == gntdev_dmabuf) {
+            pr_debug("Found gntdev_dmabuf in the wait list, wake\n");
+            complete_all(&obj->completion);
+            break;
Quoted 9 lines

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 128

Quoted 5 lines

+        }

drivers/xen/gntdev-dmabuf.c +4 -0
+}
+
+static struct gntdev_dmabuf *
+dmabuf_exp_wait_obj_get_by_fd(struct gntdev_dmabuf_priv *priv, int fd)

The name of this routine implies (to me) that we are getting a wait object but IIUIC we are getting a gntdev_dmabuf that we are going to later associate with a wait object.

How about dmabuf_exp_wait_obj_get_dmabuf_by_fd?
I would like to keep function prefixes, e.g. dmabuf_exp_wait_obj_ just to show to which functionality a routine belongs.

Quoted 250 lines
Quoted 250 lines
drivers/xen/gntdev-dmabuf.c +15 -0
+{
+    struct gntdev_dmabuf *q, *gntdev_dmabuf, *ret = ERR_PTR(-ENOENT);
+
+    mutex_lock(&priv->lock);
+    list_for_each_entry_safe(gntdev_dmabuf, q, &priv->exp_list, next)
+        if (gntdev_dmabuf->fd == fd) {
+            pr_debug("Found gntdev_dmabuf in the wait list\n");
+            kref_get(&gntdev_dmabuf->u.exp.refcount);
+            ret = gntdev_dmabuf;
+            break;
+        }
+    mutex_unlock(&priv->lock);
+    return ret;
+}
+

  int gntdev_dmabuf_exp_wait_released(struct gntdev_dmabuf_priv *priv, int fd, int wait_to_ms)
  {
-    return -EINVAL;
+    struct gntdev_dmabuf *gntdev_dmabuf;
+    struct gntdev_dmabuf_wait_obj *obj;
+    int ret;

drivers/xen/gntdev-dmabuf.c +13 -0
+
+    pr_debug("Will wait for dma-buf with fd %d\n", fd);
+    /*
+     * Try to find the DMA buffer: if not found means that
+     * either the buffer has already been released or file descriptor
+     * provided is wrong.
+     */
+    gntdev_dmabuf = dmabuf_exp_wait_obj_get_by_fd(priv, fd);
+    if (IS_ERR(gntdev_dmabuf))
+        return PTR_ERR(gntdev_dmabuf);
+
+    /*
+     * gntdev_dmabuf still exists and is reference count locked by 

us now,
+     * so prepare to wait: allocate wait object and add it to the wait list,
+     * so we can find it on release.
+     */
+    obj = dmabuf_exp_wait_obj_new(priv, gntdev_dmabuf); +    if (IS_ERR(obj))
+        return PTR_ERR(obj);

drivers/xen/gntdev-dmabuf.c +53 -0
+
+    ret = dmabuf_exp_wait_obj_wait(obj, wait_to_ms);
+    dmabuf_exp_wait_obj_free(priv, obj);
+    return ret;
+}
+
+/* DMA buffer export support. */
+
+static struct sg_table *
+dmabuf_pages_to_sgt(struct page **pages, unsigned int nr_pages)
+{
+    struct sg_table *sgt;
+    int ret;
+
+    sgt = kmalloc(sizeof(*sgt), GFP_KERNEL);
+    if (!sgt) {
+        ret = -ENOMEM;
+        goto out;
+    }
+
+    ret = sg_alloc_table_from_pages(sgt, pages, nr_pages, 0,
+                    nr_pages << PAGE_SHIFT,
+                    GFP_KERNEL);
+    if (ret)
+        goto out;
+
+    return sgt;
+
+out:
+    kfree(sgt);
+    return ERR_PTR(ret);
+}
+
+static int dmabuf_exp_ops_attach(struct dma_buf *dma_buf,
+                 struct device *target_dev,
+                 struct dma_buf_attachment *attach)
+{
+    struct gntdev_dmabuf_attachment *gntdev_dmabuf_attach;
+
+    gntdev_dmabuf_attach = kzalloc(sizeof(*gntdev_dmabuf_attach),
+                       GFP_KERNEL);
+    if (!gntdev_dmabuf_attach)
+        return -ENOMEM;
+
+    gntdev_dmabuf_attach->dir = DMA_NONE;
+    attach->priv = gntdev_dmabuf_attach;
+    return 0;
+}
+
+static void dmabuf_exp_ops_detach(struct dma_buf *dma_buf,
+                  struct dma_buf_attachment *attach)
+{
+    struct gntdev_dmabuf_attachment *gntdev_dmabuf_attach = 

attach->priv;

drivers/xen/gntdev-dmabuf.c +24 -0
+
+    if (gntdev_dmabuf_attach) {
+        struct sg_table *sgt = gntdev_dmabuf_attach->sgt;
+
+        if (sgt) {
+            if (gntdev_dmabuf_attach->dir != DMA_NONE)
+                dma_unmap_sg_attrs(attach->dev, sgt->sgl,
+                           sgt->nents,
+                           gntdev_dmabuf_attach->dir,
+                           DMA_ATTR_SKIP_CPU_SYNC);
+            sg_free_table(sgt);
+        }
+
+        kfree(sgt);
+        kfree(gntdev_dmabuf_attach);
+        attach->priv = NULL;
+    }
+}
+
+static struct sg_table *
+dmabuf_exp_ops_map_dma_buf(struct dma_buf_attachment *attach,
+               enum dma_data_direction dir)
+{
+    struct gntdev_dmabuf_attachment *gntdev_dmabuf_attach = 

attach->priv;
+    struct gntdev_dmabuf *gntdev_dmabuf = attach->dmabuf->priv; +    struct sg_table *sgt;

drivers/xen/gntdev-dmabuf.c +13 -0
+
+    pr_debug("Mapping %d pages for dev %p\n", gntdev_dmabuf->nr_pages,
+         attach->dev);
+
+    if (dir == DMA_NONE || !gntdev_dmabuf_attach)
+        return ERR_PTR(-EINVAL);
+
+    /* Return the cached mapping when possible. */
+    if (gntdev_dmabuf_attach->dir == dir)
+        return gntdev_dmabuf_attach->sgt;
+
+    /*
+     * Two mappings with different directions for the same 

attachment are
+     * not allowed.
+     */
+    if (gntdev_dmabuf_attach->dir != DMA_NONE)
+        return ERR_PTR(-EBUSY);

drivers/xen/gntdev-dmabuf.c +20 -0
+
+    sgt = dmabuf_pages_to_sgt(gntdev_dmabuf->pages,
+                  gntdev_dmabuf->nr_pages);
+    if (!IS_ERR(sgt)) {
+        if (!dma_map_sg_attrs(attach->dev, sgt->sgl, sgt->nents, dir,
+                      DMA_ATTR_SKIP_CPU_SYNC)) {
+            sg_free_table(sgt);
+            kfree(sgt);
+            sgt = ERR_PTR(-ENOMEM);
+        } else {
+            gntdev_dmabuf_attach->sgt = sgt;
+            gntdev_dmabuf_attach->dir = dir;
+        }
+    }
+    if (IS_ERR(sgt))
+        pr_debug("Failed to map sg table for dev %p\n", attach->dev);
+    return sgt;
+}
+
+static void dmabuf_exp_ops_unmap_dma_buf(struct dma_buf_attachment 

*attach,
+                     struct sg_table *sgt, +                     enum dma_data_direction dir) +{
+    /* Not implemented. The unmap is done at dmabuf_exp_ops_detach(). */

drivers/xen/gntdev-dmabuf.c +18 -0
+}
+
+static void dmabuf_exp_release(struct kref *kref)
+{
+    struct gntdev_dmabuf *gntdev_dmabuf =
+        container_of(kref, struct gntdev_dmabuf, u.exp.refcount);
+
+    dmabuf_exp_wait_obj_signal(gntdev_dmabuf->priv, gntdev_dmabuf);
+    list_del(&gntdev_dmabuf->next);
+    kfree(gntdev_dmabuf);
+}
+
+static void dmabuf_exp_ops_release(struct dma_buf *dma_buf)
+{
+    struct gntdev_dmabuf *gntdev_dmabuf = dma_buf->priv;
+    struct gntdev_dmabuf_priv *priv = gntdev_dmabuf->priv;
+
+    gntdev_remove_map(gntdev_dmabuf->u.exp.priv, 

gntdev_dmabuf->u.exp.map);
+    mutex_lock(&priv->lock);
+    kref_put(&gntdev_dmabuf->u.exp.refcount, dmabuf_exp_release); +    mutex_unlock(&priv->lock);

drivers/xen/gntdev-dmabuf.c +59 -0
+}
+
+static void *dmabuf_exp_ops_kmap_atomic(struct dma_buf *dma_buf,
+                    unsigned long page_num)
+{
+    /* Not implemented. */
+    return NULL;
+}
+
+static void dmabuf_exp_ops_kunmap_atomic(struct dma_buf *dma_buf,
+                     unsigned long page_num, void *addr)
+{
+    /* Not implemented. */
+}
+
+static void *dmabuf_exp_ops_kmap(struct dma_buf *dma_buf,
+                 unsigned long page_num)
+{
+    /* Not implemented. */
+    return NULL;
+}
+
+static void dmabuf_exp_ops_kunmap(struct dma_buf *dma_buf,
+                  unsigned long page_num, void *addr)
+{
+    /* Not implemented. */
+}
+
+static int dmabuf_exp_ops_mmap(struct dma_buf *dma_buf,
+                   struct vm_area_struct *vma)
+{
+    /* Not implemented. */
+    return 0;
+}
+
+static const struct dma_buf_ops dmabuf_exp_ops =  {
+    .attach = dmabuf_exp_ops_attach,
+    .detach = dmabuf_exp_ops_detach,
+    .map_dma_buf = dmabuf_exp_ops_map_dma_buf,
+    .unmap_dma_buf = dmabuf_exp_ops_unmap_dma_buf,
+    .release = dmabuf_exp_ops_release,
+    .map = dmabuf_exp_ops_kmap,
+    .map_atomic = dmabuf_exp_ops_kmap_atomic,
+    .unmap = dmabuf_exp_ops_kunmap,
+    .unmap_atomic = dmabuf_exp_ops_kunmap_atomic,
+    .mmap = dmabuf_exp_ops_mmap,
+};
+
+struct gntdev_dmabuf_export_args {
+    struct gntdev_priv *priv;
+    struct gntdev_grant_map *map;
+    struct gntdev_dmabuf_priv *dmabuf_priv;
+    struct device *dev;
+    int count;
+    struct page **pages;
+    u32 fd;
+};
+
+static int dmabuf_exp_from_pages(struct gntdev_dmabuf_export_args 
Quoted 9 lines
Quoted 5 lines

*args)
+{
+    DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
+    struct gntdev_dmabuf *gntdev_dmabuf;
+    int ret = 0;

Not necessary.

Will remove =0;

Quoted 59 lines
Quoted 59 lines
drivers/xen/gntdev-dmabuf.c +53 -0
+
+    gntdev_dmabuf = kzalloc(sizeof(*gntdev_dmabuf), GFP_KERNEL);
+    if (!gntdev_dmabuf)
+        return -ENOMEM;
+
+    kref_init(&gntdev_dmabuf->u.exp.refcount);
+
+    gntdev_dmabuf->priv = args->dmabuf_priv;
+    gntdev_dmabuf->nr_pages = args->count;
+    gntdev_dmabuf->pages = args->pages;
+    gntdev_dmabuf->u.exp.priv = args->priv;
+    gntdev_dmabuf->u.exp.map = args->map;
+
+    exp_info.exp_name = KBUILD_MODNAME;
+    if (args->dev->driver && args->dev->driver->owner)
+        exp_info.owner = args->dev->driver->owner;
+    else
+        exp_info.owner = THIS_MODULE;
+    exp_info.ops = &dmabuf_exp_ops;
+    exp_info.size = args->count << PAGE_SHIFT;
+    exp_info.flags = O_RDWR;
+    exp_info.priv = gntdev_dmabuf;
+
+    gntdev_dmabuf->dmabuf = dma_buf_export(&exp_info);
+    if (IS_ERR(gntdev_dmabuf->dmabuf)) {
+        ret = PTR_ERR(gntdev_dmabuf->dmabuf);
+        gntdev_dmabuf->dmabuf = NULL;
+        goto fail;
+    }
+
+    ret = dma_buf_fd(gntdev_dmabuf->dmabuf, O_CLOEXEC);
+    if (ret < 0)
+        goto fail;
+
+    gntdev_dmabuf->fd = ret;
+    args->fd = ret;
+
+    pr_debug("Exporting DMA buffer with fd %d\n", ret);
+
+    mutex_lock(&args->dmabuf_priv->lock);
+    list_add(&gntdev_dmabuf->next, &args->dmabuf_priv->exp_list);
+    mutex_unlock(&args->dmabuf_priv->lock);
+    return 0;
+
+fail:
+    if (gntdev_dmabuf->dmabuf)
+        dma_buf_put(gntdev_dmabuf->dmabuf);
+    kfree(gntdev_dmabuf);
+    return ret;
+}
+
+static struct gntdev_grant_map *
+dmabuf_exp_alloc_backing_storage(struct gntdev_priv *priv, int 

dmabuf_flags,
+                 int count)

drivers/xen/gntdev-dmabuf.c +4 -0
+{
+    struct gntdev_grant_map *map;
+
+    if (unlikely(count <= 0))
Quoted 8 lines
Quoted 6 lines

+        return ERR_PTR(-EINVAL);
+
+    if ((dmabuf_flags & GNTDEV_DMA_FLAG_WC) &&
+        (dmabuf_flags & GNTDEV_DMA_FLAG_COHERENT)) {
+        pr_debug("Wrong dma-buf flags: either WC or coherent, not both\n");

Why not just print the value of the flags?

Will print hex value of the flags

Quoted 14 lines

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 444

Quoted 10 lines

+        return ERR_PTR(-EINVAL);
+    }

drivers/xen/gntdev-dmabuf.c +7 -0
+
+    map = gntdev_alloc_map(priv, count, dmabuf_flags);
+    if (!map)
+        return ERR_PTR(-ENOMEM);
+
+    if (unlikely(gntdev_account_mapped_pages(count))) {
+        pr_debug("can't map: over limit\n");

I think printing @count value here would be useful.

Will add

Quoted 16 lines
Quoted 14 lines

+        gntdev_put_map(NULL, map);
+        return ERR_PTR(-ENOMEM);
+    }
+    return map;
  }
    int gntdev_dmabuf_exp_from_refs(struct gntdev_priv *priv, int flags, int count, u32 domid, u32 *refs, u32 *fd) {
+    struct gntdev_grant_map *map;
+    struct gntdev_dmabuf_export_args args;
+    int i, ret;
+
      *fd = -1;

Quoted 1 line

Is this still needed?

No, will remove. I was thinking here about if user-space ignores IOCTL return value and tries to use the fd so it fails on -1. But, ok, no reason to fix user-space bugs in the kernel

Quoted 63 lines
Quoted 63 lines

-    return -EINVAL;

drivers/xen/gntdev-dmabuf.c +40 -0
+
+    map = dmabuf_exp_alloc_backing_storage(priv, flags, count);
+    if (IS_ERR(map))
+        return PTR_ERR(map);
+
+    for (i = 0; i < count; i++) {
+        map->grants[i].domid = domid;
+        map->grants[i].ref = refs[i];
+    }
+
+    mutex_lock(&priv->lock);
+    gntdev_add_map(priv, map);
+    mutex_unlock(&priv->lock);
+
+    map->flags |= GNTMAP_host_map;
+#if defined(CONFIG_X86)
+    map->flags |= GNTMAP_device_map;
+#endif
+
+    ret = gntdev_map_grant_pages(map);
+    if (ret < 0)
+        goto out;
+
+    args.priv = priv;
+    args.map = map;
+    args.dev = priv->dma_dev;
+    args.dmabuf_priv = priv->dmabuf_priv;
+    args.count = map->count;
+    args.pages = map->pages;
+
+    ret = dmabuf_exp_from_pages(&args);
+    if (ret < 0)
+        goto out;
+
+    *fd = args.fd;
+    return 0;
+
+out:
+    gntdev_remove_map(priv, map);
+    return ret;

  }
    /* DMA buffer import support. */

drivers/xen/gntdev-dmabuf.c +3 -0
@@ -63,6 +525,10 @@ struct gntdev_dmabuf_priv *gntdev_dmabuf_init(void)
       if (!priv)
           return ERR_PTR(-ENOMEM);
   +    mutex_init(&priv->lock);
+    INIT_LIST_HEAD(&priv->exp_list);
+    INIT_LIST_HEAD(&priv->exp_wait_list);
+
       return priv;
   }

  diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c index e82660d81d7e..5f93cd534840 100644

drivers/xen/gntdev.c +0 -0
@@ -262,6 +262,16 @@ void gntdev_put_map(struct gntdev_priv *priv, 
 struct gntdev_grant_map *map)
       gntdev_free_map(map);
   }

  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct

Quoted 9 lines
Quoted 5 lines

gntdev_grant_map *map)
+{
+    mutex_lock(&priv->lock);
+    list_del(&map->next);
+    gntdev_put_map(NULL /* already removed */, map);

Why not pass call gntdev_put_map(priv, map) and then not have this routine at all?

Well, I wish I could, but the main difference when calling gntdev_put_map(priv, map) with priv != NULL and my code is that:

void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map) {
    [...]
    if (populate_freeable_maps && priv) { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ mutex_lock(&priv->lock);
        list_del(&map->next);
        mutex_unlock(&priv->lock);
    }
    [...]
}

and

#define populate_freeable_maps use_ptemod
                               ^^^^^^^^^^
which means the map will never be removed from the list in my case because use_ptemod == false for dma-buf.
This is why I do that by hand, e.g. remove the item from the list and then pass NULL for priv.

Also, I will remove gntdev_remove_map as I can now access priv->lock and gntdev_put_map directly form gntdev-dmabuf.c

Quoted 2 lines

I really dislike the fact that we are taking a lock here that gntdev_put_map() takes as well, although not with NULL argument. (And yes, I see that gntdev_release() does it too.)

This can be re-factored later I guess?

Quoted 3 lines

-boris

Thank you,
Oleksandr

Quoted 9 lines
Quoted 9 lines

+    mutex_unlock(&priv->lock);

drivers/xen/gntdev.c +3 -0
+}
+#endif
+

  /* ------------------------------------------------------------------ */ static int find_grant_ptes(pte_t *pte, pgtable_t token,

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#55 lore raw

Re: [PATCH v3 8/9] xen/gntdev: Implement dma-buf export functionality

In reply to Oleksandr Andrushchenko
Quoted 127 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

1. Create a dma-buf from grant references provided by the foreign domain. By default dma-buf is backed by system memory pages, but by providing GNTDEV_DMA_FLAG_XXX flags it can also be created as a DMA write-combine/coherent buffer, e.g. allocated with corresponding dma_alloc_xxx API.
Export the resulting buffer as a new dma-buf.

2. Implement waiting for the dma-buf to be released: block until the dma-buf with the file descriptor provided is released. If within the time-out provided the buffer is not released then -ETIMEDOUT error is returned. If the buffer with the file descriptor does not exist or has already been released, then -ENOENT is returned. For valid file descriptors this must not be treated as error.

3. Make gntdev's common code and structures available to dma-buf.

---
  drivers/xen/gntdev-common.h |   4 +
  drivers/xen/gntdev-dmabuf.c | 470 +++++++++++++++++++++++++++++++++++-
  drivers/xen/gntdev.c        |  10 +
  3 files changed, 482 insertions(+), 2 deletions(-)
drivers/xen/gntdev-common.h +4 -0
index a3408fd39b07..72f80dbce861 100644
@@ -89,4 +89,8 @@ bool gntdev_account_mapped_pages(int count);
 
  int gntdev_map_grant_pages(struct gntdev_grant_map *map);
 
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct gntdev_grant_map *map);
+#endif
+
  #endif
drivers/xen/gntdev-dmabuf.c +66 -0
index dc57c6a25525..84cba67c6ad7 100644
@@ -3,13 +3,53 @@
  /*
   * Xen dma-buf functionality for gntdev.
   *
+ * DMA buffer implementation is based on drivers/gpu/drm/drm_prime.c.
+ *
   * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
   */
 
+#include <linux/dma-buf.h>
  #include <linux/slab.h>
 
+#include <xen/grant_table.h>
+#include <xen/gntdev.h>
+
+#include "gntdev-common.h"
  #include "gntdev-dmabuf.h"
 
+struct gntdev_dmabuf {
+	struct gntdev_dmabuf_priv *priv;
+	struct dma_buf *dmabuf;
+	struct list_head next;
+	int fd;
+
+	union {
+		struct {
+			/* Exported buffers are reference counted. */
+			struct kref refcount;
+
+			struct gntdev_priv *priv;
+			struct gntdev_grant_map *map;
+		} exp;
+	} u;
+
+	/* Number of pages this buffer has. */
+	int nr_pages;
+	/* Pages of this buffer. */
+	struct page **pages;
+};
+
+struct gntdev_dmabuf_wait_obj {
+	struct list_head next;
+	struct gntdev_dmabuf *gntdev_dmabuf;
+	struct completion completion;
+};
+
+struct gntdev_dmabuf_attachment {
+	struct sg_table *sgt;
+	enum dma_data_direction dir;
+};
+
  struct gntdev_dmabuf_priv {
  	/* List of exported DMA buffers. */
  	struct list_head exp_list;
@@ -23,17 +63,439 @@ struct gntdev_dmabuf_priv {
 
  /* Implementation of wait for exported DMA buffer to be released. */
 
+static void dmabuf_exp_release(struct kref *kref);
+
+static struct gntdev_dmabuf_wait_obj *
+dmabuf_exp_wait_obj_new(struct gntdev_dmabuf_priv *priv,
+			struct gntdev_dmabuf *gntdev_dmabuf)
+{
+	struct gntdev_dmabuf_wait_obj *obj;
+
+	obj = kzalloc(sizeof(*obj), GFP_KERNEL);
+	if (!obj)
+		return ERR_PTR(-ENOMEM);
+
+	init_completion(&obj->completion);
+	obj->gntdev_dmabuf = gntdev_dmabuf;
+
+	mutex_lock(&priv->lock);
+	list_add(&obj->next, &priv->exp_wait_list);
+	/* Put our reference and wait for gntdev_dmabuf's release to fire. */
+	kref_put(&gntdev_dmabuf->u.exp.refcount, dmabuf_exp_release);
+	mutex_unlock(&priv->lock);
+	return obj;
+}
+
+static void dmabuf_exp_wait_obj_free(struct gntdev_dmabuf_priv *priv,
+				     struct gntdev_dmabuf_wait_obj *obj)
+{

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 101

Quoted 10 lines
drivers/xen/gntdev-dmabuf.c +10 -0
+	struct gntdev_dmabuf_wait_obj *cur_obj, *q;
+
+	mutex_lock(&priv->lock);
+	list_for_each_entry_safe(cur_obj, q, &priv->exp_wait_list, next)
+		if (cur_obj == obj) {
+			list_del(&obj->next);
+			kfree(obj);
+			break;
+		}
+	mutex_unlock(&priv->lock);

Do we really need to walk the list?

And if we do, do we need the safe variant of the walk? We are holding the lock. Here and elsewhere.

Quoted 17 lines
drivers/xen/gntdev-dmabuf.c +17 -0
+}
+
+static int dmabuf_exp_wait_obj_wait(struct gntdev_dmabuf_wait_obj *obj,
+				    u32 wait_to_ms)
+{
+	if (wait_for_completion_timeout(&obj->completion,
+			msecs_to_jiffies(wait_to_ms)) <= 0)
+		return -ETIMEDOUT;
+
+	return 0;
+}
+
+static void dmabuf_exp_wait_obj_signal(struct gntdev_dmabuf_priv *priv,
+				       struct gntdev_dmabuf *gntdev_dmabuf)
+{
+	struct gntdev_dmabuf_wait_obj *obj, *q;
+

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 128

Quoted 10 lines
drivers/xen/gntdev-dmabuf.c +10 -0
+	list_for_each_entry_safe(obj, q, &priv->exp_wait_list, next)
+		if (obj->gntdev_dmabuf == gntdev_dmabuf) {
+			pr_debug("Found gntdev_dmabuf in the wait list, wake\n");
+			complete_all(&obj->completion);
+			break;
+		}
+}
+
+static struct gntdev_dmabuf *
+dmabuf_exp_wait_obj_get_by_fd(struct gntdev_dmabuf_priv *priv, int fd)

The name of this routine implies (to me) that we are getting a wait object but IIUIC we are getting a gntdev_dmabuf that we are going to later associate with a wait object.

Quoted 235 lines
drivers/xen/gntdev-dmabuf.c +231 -1
+{
+	struct gntdev_dmabuf *q, *gntdev_dmabuf, *ret = ERR_PTR(-ENOENT);
+
+	mutex_lock(&priv->lock);
+	list_for_each_entry_safe(gntdev_dmabuf, q, &priv->exp_list, next)
+		if (gntdev_dmabuf->fd == fd) {
+			pr_debug("Found gntdev_dmabuf in the wait list\n");
+			kref_get(&gntdev_dmabuf->u.exp.refcount);
+			ret = gntdev_dmabuf;
+			break;
+		}
+	mutex_unlock(&priv->lock);
+	return ret;
+}
+
  int gntdev_dmabuf_exp_wait_released(struct gntdev_dmabuf_priv *priv, int fd,
  				    int wait_to_ms)
  {
-	return -EINVAL;
+	struct gntdev_dmabuf *gntdev_dmabuf;
+	struct gntdev_dmabuf_wait_obj *obj;
+	int ret;
+
+	pr_debug("Will wait for dma-buf with fd %d\n", fd);
+	/*
+	 * Try to find the DMA buffer: if not found means that
+	 * either the buffer has already been released or file descriptor
+	 * provided is wrong.
+	 */
+	gntdev_dmabuf = dmabuf_exp_wait_obj_get_by_fd(priv, fd);
+	if (IS_ERR(gntdev_dmabuf))
+		return PTR_ERR(gntdev_dmabuf);
+
+	/*
+	 * gntdev_dmabuf still exists and is reference count locked by us now,
+	 * so prepare to wait: allocate wait object and add it to the wait list,
+	 * so we can find it on release.
+	 */
+	obj = dmabuf_exp_wait_obj_new(priv, gntdev_dmabuf);
+	if (IS_ERR(obj))
+		return PTR_ERR(obj);
+
+	ret = dmabuf_exp_wait_obj_wait(obj, wait_to_ms);
+	dmabuf_exp_wait_obj_free(priv, obj);
+	return ret;
+}
+
+/* DMA buffer export support. */
+
+static struct sg_table *
+dmabuf_pages_to_sgt(struct page **pages, unsigned int nr_pages)
+{
+	struct sg_table *sgt;
+	int ret;
+
+	sgt = kmalloc(sizeof(*sgt), GFP_KERNEL);
+	if (!sgt) {
+		ret = -ENOMEM;
+		goto out;
+	}
+
+	ret = sg_alloc_table_from_pages(sgt, pages, nr_pages, 0,
+					nr_pages << PAGE_SHIFT,
+					GFP_KERNEL);
+	if (ret)
+		goto out;
+
+	return sgt;
+
+out:
+	kfree(sgt);
+	return ERR_PTR(ret);
+}
+
+static int dmabuf_exp_ops_attach(struct dma_buf *dma_buf,
+				 struct device *target_dev,
+				 struct dma_buf_attachment *attach)
+{
+	struct gntdev_dmabuf_attachment *gntdev_dmabuf_attach;
+
+	gntdev_dmabuf_attach = kzalloc(sizeof(*gntdev_dmabuf_attach),
+				       GFP_KERNEL);
+	if (!gntdev_dmabuf_attach)
+		return -ENOMEM;
+
+	gntdev_dmabuf_attach->dir = DMA_NONE;
+	attach->priv = gntdev_dmabuf_attach;
+	return 0;
+}
+
+static void dmabuf_exp_ops_detach(struct dma_buf *dma_buf,
+				  struct dma_buf_attachment *attach)
+{
+	struct gntdev_dmabuf_attachment *gntdev_dmabuf_attach = attach->priv;
+
+	if (gntdev_dmabuf_attach) {
+		struct sg_table *sgt = gntdev_dmabuf_attach->sgt;
+
+		if (sgt) {
+			if (gntdev_dmabuf_attach->dir != DMA_NONE)
+				dma_unmap_sg_attrs(attach->dev, sgt->sgl,
+						   sgt->nents,
+						   gntdev_dmabuf_attach->dir,
+						   DMA_ATTR_SKIP_CPU_SYNC);
+			sg_free_table(sgt);
+		}
+
+		kfree(sgt);
+		kfree(gntdev_dmabuf_attach);
+		attach->priv = NULL;
+	}
+}
+
+static struct sg_table *
+dmabuf_exp_ops_map_dma_buf(struct dma_buf_attachment *attach,
+			   enum dma_data_direction dir)
+{
+	struct gntdev_dmabuf_attachment *gntdev_dmabuf_attach = attach->priv;
+	struct gntdev_dmabuf *gntdev_dmabuf = attach->dmabuf->priv;
+	struct sg_table *sgt;
+
+	pr_debug("Mapping %d pages for dev %p\n", gntdev_dmabuf->nr_pages,
+		 attach->dev);
+
+	if (dir == DMA_NONE || !gntdev_dmabuf_attach)
+		return ERR_PTR(-EINVAL);
+
+	/* Return the cached mapping when possible. */
+	if (gntdev_dmabuf_attach->dir == dir)
+		return gntdev_dmabuf_attach->sgt;
+
+	/*
+	 * Two mappings with different directions for the same attachment are
+	 * not allowed.
+	 */
+	if (gntdev_dmabuf_attach->dir != DMA_NONE)
+		return ERR_PTR(-EBUSY);
+
+	sgt = dmabuf_pages_to_sgt(gntdev_dmabuf->pages,
+				  gntdev_dmabuf->nr_pages);
+	if (!IS_ERR(sgt)) {
+		if (!dma_map_sg_attrs(attach->dev, sgt->sgl, sgt->nents, dir,
+				      DMA_ATTR_SKIP_CPU_SYNC)) {
+			sg_free_table(sgt);
+			kfree(sgt);
+			sgt = ERR_PTR(-ENOMEM);
+		} else {
+			gntdev_dmabuf_attach->sgt = sgt;
+			gntdev_dmabuf_attach->dir = dir;
+		}
+	}
+	if (IS_ERR(sgt))
+		pr_debug("Failed to map sg table for dev %p\n", attach->dev);
+	return sgt;
+}
+
+static void dmabuf_exp_ops_unmap_dma_buf(struct dma_buf_attachment *attach,
+					 struct sg_table *sgt,
+					 enum dma_data_direction dir)
+{
+	/* Not implemented. The unmap is done at dmabuf_exp_ops_detach(). */
+}
+
+static void dmabuf_exp_release(struct kref *kref)
+{
+	struct gntdev_dmabuf *gntdev_dmabuf =
+		container_of(kref, struct gntdev_dmabuf, u.exp.refcount);
+
+	dmabuf_exp_wait_obj_signal(gntdev_dmabuf->priv, gntdev_dmabuf);
+	list_del(&gntdev_dmabuf->next);
+	kfree(gntdev_dmabuf);
+}
+
+static void dmabuf_exp_ops_release(struct dma_buf *dma_buf)
+{
+	struct gntdev_dmabuf *gntdev_dmabuf = dma_buf->priv;
+	struct gntdev_dmabuf_priv *priv = gntdev_dmabuf->priv;
+
+	gntdev_remove_map(gntdev_dmabuf->u.exp.priv, gntdev_dmabuf->u.exp.map);
+	mutex_lock(&priv->lock);
+	kref_put(&gntdev_dmabuf->u.exp.refcount, dmabuf_exp_release);
+	mutex_unlock(&priv->lock);
+}
+
+static void *dmabuf_exp_ops_kmap_atomic(struct dma_buf *dma_buf,
+					unsigned long page_num)
+{
+	/* Not implemented. */
+	return NULL;
+}
+
+static void dmabuf_exp_ops_kunmap_atomic(struct dma_buf *dma_buf,
+					 unsigned long page_num, void *addr)
+{
+	/* Not implemented. */
+}
+
+static void *dmabuf_exp_ops_kmap(struct dma_buf *dma_buf,
+				 unsigned long page_num)
+{
+	/* Not implemented. */
+	return NULL;
+}
+
+static void dmabuf_exp_ops_kunmap(struct dma_buf *dma_buf,
+				  unsigned long page_num, void *addr)
+{
+	/* Not implemented. */
+}
+
+static int dmabuf_exp_ops_mmap(struct dma_buf *dma_buf,
+			       struct vm_area_struct *vma)
+{
+	/* Not implemented. */
+	return 0;
+}
+
+static const struct dma_buf_ops dmabuf_exp_ops =  {
+	.attach = dmabuf_exp_ops_attach,
+	.detach = dmabuf_exp_ops_detach,
+	.map_dma_buf = dmabuf_exp_ops_map_dma_buf,
+	.unmap_dma_buf = dmabuf_exp_ops_unmap_dma_buf,
+	.release = dmabuf_exp_ops_release,
+	.map = dmabuf_exp_ops_kmap,
+	.map_atomic = dmabuf_exp_ops_kmap_atomic,
+	.unmap = dmabuf_exp_ops_kunmap,
+	.unmap_atomic = dmabuf_exp_ops_kunmap_atomic,
+	.mmap = dmabuf_exp_ops_mmap,
+};
+
+struct gntdev_dmabuf_export_args {
+	struct gntdev_priv *priv;
+	struct gntdev_grant_map *map;
+	struct gntdev_dmabuf_priv *dmabuf_priv;
+	struct device *dev;

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 372

Quoted 10 lines
drivers/xen/gntdev-dmabuf.c +10 -0
+	int count;
+	struct page **pages;
+	u32 fd;
+};
+
+static int dmabuf_exp_from_pages(struct gntdev_dmabuf_export_args *args)
+{
+	DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
+	struct gntdev_dmabuf *gntdev_dmabuf;
+	int ret = 0;

Not necessary.

Quoted 53 lines
drivers/xen/gntdev-dmabuf.c +53 -0
+
+	gntdev_dmabuf = kzalloc(sizeof(*gntdev_dmabuf), GFP_KERNEL);
+	if (!gntdev_dmabuf)
+		return -ENOMEM;
+
+	kref_init(&gntdev_dmabuf->u.exp.refcount);
+
+	gntdev_dmabuf->priv = args->dmabuf_priv;
+	gntdev_dmabuf->nr_pages = args->count;
+	gntdev_dmabuf->pages = args->pages;
+	gntdev_dmabuf->u.exp.priv = args->priv;
+	gntdev_dmabuf->u.exp.map = args->map;
+
+	exp_info.exp_name = KBUILD_MODNAME;
+	if (args->dev->driver && args->dev->driver->owner)
+		exp_info.owner = args->dev->driver->owner;
+	else
+		exp_info.owner = THIS_MODULE;
+	exp_info.ops = &dmabuf_exp_ops;
+	exp_info.size = args->count << PAGE_SHIFT;
+	exp_info.flags = O_RDWR;
+	exp_info.priv = gntdev_dmabuf;
+
+	gntdev_dmabuf->dmabuf = dma_buf_export(&exp_info);
+	if (IS_ERR(gntdev_dmabuf->dmabuf)) {
+		ret = PTR_ERR(gntdev_dmabuf->dmabuf);
+		gntdev_dmabuf->dmabuf = NULL;
+		goto fail;
+	}
+
+	ret = dma_buf_fd(gntdev_dmabuf->dmabuf, O_CLOEXEC);
+	if (ret < 0)
+		goto fail;
+
+	gntdev_dmabuf->fd = ret;
+	args->fd = ret;
+
+	pr_debug("Exporting DMA buffer with fd %d\n", ret);
+
+	mutex_lock(&args->dmabuf_priv->lock);
+	list_add(&gntdev_dmabuf->next, &args->dmabuf_priv->exp_list);
+	mutex_unlock(&args->dmabuf_priv->lock);
+	return 0;
+
+fail:
+	if (gntdev_dmabuf->dmabuf)
+		dma_buf_put(gntdev_dmabuf->dmabuf);
+	kfree(gntdev_dmabuf);
+	return ret;
+}
+
+static struct gntdev_grant_map *
+dmabuf_exp_alloc_backing_storage(struct gntdev_priv *priv, int dmabuf_flags,

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 435

Quoted 10 lines
drivers/xen/gntdev-dmabuf.c +10 -0
+				 int count)
+{
+	struct gntdev_grant_map *map;
+
+	if (unlikely(count <= 0))
+		return ERR_PTR(-EINVAL);
+
+	if ((dmabuf_flags & GNTDEV_DMA_FLAG_WC) &&
+	    (dmabuf_flags & GNTDEV_DMA_FLAG_COHERENT)) {
+		pr_debug("Wrong dma-buf flags: either WC or coherent, not both\n");

Why not just print the value of the flags?

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 444

Quoted 9 lines
drivers/xen/gntdev-dmabuf.c +9 -0
+		return ERR_PTR(-EINVAL);
+	}
+
+	map = gntdev_alloc_map(priv, count, dmabuf_flags);
+	if (!map)
+		return ERR_PTR(-ENOMEM);
+
+	if (unlikely(gntdev_account_mapped_pages(count))) {
+		pr_debug("can't map: over limit\n");

I think printing @count value here would be useful.

Quoted 4 lines
drivers/xen/gntdev-dmabuf.c +4 -0
+		gntdev_put_map(NULL, map);
+		return ERR_PTR(-ENOMEM);
+	}
+	return map;

drivers/xen/gntdev-dmabuf.c struct gntdev_dmabuf_priv { · line 458

Quoted 10 lines
drivers/xen/gntdev-dmabuf.c +4 -0
  }
 
  int gntdev_dmabuf_exp_from_refs(struct gntdev_priv *priv, int flags,
  				int count, u32 domid, u32 *refs, u32 *fd)
  {
+	struct gntdev_grant_map *map;
+	struct gntdev_dmabuf_export_args args;
+	int i, ret;
+
  	*fd = -1;

Is this still needed?

Quoted 60 lines
drivers/xen/gntdev-dmabuf.c +44 -1
-	return -EINVAL;
+
+	map = dmabuf_exp_alloc_backing_storage(priv, flags, count);
+	if (IS_ERR(map))
+		return PTR_ERR(map);
+
+	for (i = 0; i < count; i++) {
+		map->grants[i].domid = domid;
+		map->grants[i].ref = refs[i];
+	}
+
+	mutex_lock(&priv->lock);
+	gntdev_add_map(priv, map);
+	mutex_unlock(&priv->lock);
+
+	map->flags |= GNTMAP_host_map;
+#if defined(CONFIG_X86)
+	map->flags |= GNTMAP_device_map;
+#endif
+
+	ret = gntdev_map_grant_pages(map);
+	if (ret < 0)
+		goto out;
+
+	args.priv = priv;
+	args.map = map;
+	args.dev = priv->dma_dev;
+	args.dmabuf_priv = priv->dmabuf_priv;
+	args.count = map->count;
+	args.pages = map->pages;
+
+	ret = dmabuf_exp_from_pages(&args);
+	if (ret < 0)
+		goto out;
+
+	*fd = args.fd;
+	return 0;
+
+out:
+	gntdev_remove_map(priv, map);
+	return ret;
  }
 
  /* DMA buffer import support. */
@@ -63,6 +525,10 @@ struct gntdev_dmabuf_priv *gntdev_dmabuf_init(void)
  	if (!priv)
  		return ERR_PTR(-ENOMEM);
 
+	mutex_init(&priv->lock);
+	INIT_LIST_HEAD(&priv->exp_list);
+	INIT_LIST_HEAD(&priv->exp_wait_list);
+
  	return priv;
  }
 
drivers/xen/gntdev.c +0 -0
index e82660d81d7e..5f93cd534840 100644

drivers/xen/gntdev.c void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map) · line 270

Quoted 9 lines
drivers/xen/gntdev.c +6 -0
@@ -262,6 +262,16 @@ void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
  	gntdev_free_map(map);
  }
 
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
+{
+	mutex_lock(&priv->lock);
+	list_del(&map->next);
+	gntdev_put_map(NULL /* already removed */, map);

Why not pass call gntdev_put_map(priv, map) and then not have this routine at all?

I really dislike the fact that we are taking a lock here that gntdev_put_map() takes as well, although not with NULL argument. (And yes, I see that gntdev_release() does it too.)

-boris

drivers/xen/gntdev.c void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map) · line 277

Quoted 8 lines
drivers/xen/gntdev.c +4 -0
+	mutex_unlock(&priv->lock);
+}
+#endif
+
  /* ------------------------------------------------------------------ */
 
  static int find_grant_ptes(pte_t *pte, pgtable_t token,

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#56 lore raw

PATCH v3 8/9 xen/gntdev: Implement dma-buf export functionality

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

1. Create a dma-buf from grant references provided by the foreign domain. By default dma-buf is backed by system memory pages, but by providing GNTDEV_DMA_FLAG_XXX flags it can also be created as a DMA write-combine/coherent buffer, e.g. allocated with corresponding dma_alloc_xxx API.
Export the resulting buffer as a new dma-buf.

2. Implement waiting for the dma-buf to be released: block until the dma-buf with the file descriptor provided is released.
If within the time-out provided the buffer is not released then -ETIMEDOUT error is returned. If the buffer with the file descriptor does not exist or has already been released, then -ENOENT is returned. For valid file descriptors this must not be treated as error.

3. Make gntdev's common code and structures available to dma-buf.

---
 drivers/xen/gntdev-common.h |   4 +
 drivers/xen/gntdev-dmabuf.c | 470 +++++++++++++++++++++++++++++++++++-
 drivers/xen/gntdev.c        |  10 +
 3 files changed, 482 insertions(+), 2 deletions(-)
drivers/xen/gntdev-common.h +4 -0
index a3408fd39b07..72f80dbce861 100644
@@ -89,4 +89,8 @@ bool gntdev_account_mapped_pages(int count);
 
 int gntdev_map_grant_pages(struct gntdev_grant_map *map);
 
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct gntdev_grant_map *map);
+#endif
+
 #endif
drivers/xen/gntdev-dmabuf.c +468 -2

504 lines not on this page · raw message

drivers/xen/gntdev.c +10 -0
index e82660d81d7e..5f93cd534840 100644
@@ -262,6 +262,16 @@ void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
 	gntdev_free_map(map);
 }
 
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
+{
+	mutex_lock(&priv->lock);
+	list_del(&map->next);
+	gntdev_put_map(NULL /* already removed */, map);
+	mutex_unlock(&priv->lock);
+}
+#endif
+
 /* ------------------------------------------------------------------ */
 
 static int find_grant_ptes(pte_t *pte, pgtable_t token,
Signature · 7 lines
2.17.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#57 lore raw

PATCH v3 9/9 xen/gntdev: Implement dma-buf import functionality

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

1. Import a dma-buf with the file descriptor provided and export granted references to the pages of that dma-buf into the array of grant references.

2. Add API to close all references to an imported buffer, so it can be released by the owner. This is only valid for buffers created with IOCTL_GNTDEV_DMABUF_IMP_TO_REFS.

---
 drivers/xen/gntdev-dmabuf.c | 240 +++++++++++++++++++++++++++++++++++-
 1 file changed, 238 insertions(+), 2 deletions(-)
drivers/xen/gntdev-dmabuf.c +238 -2
index 84cba67c6ad7..4d250eb8babc 100644
@@ -17,6 +17,15 @@
 #include "gntdev-common.h"
 #include "gntdev-dmabuf.h"
 
+#ifndef GRANT_INVALID_REF
+/*
+ * Note on usage of grant reference 0 as invalid grant reference:
+ * grant reference 0 is valid, but never exposed to a driver,
+ * because of the fact it is already in use/reserved by the PV console.
+ */
+#define GRANT_INVALID_REF	0
+#endif
+
 struct gntdev_dmabuf {
 	struct gntdev_dmabuf_priv *priv;
 	struct dma_buf *dmabuf;
@@ -31,6 +40,14 @@ struct gntdev_dmabuf {
 			struct gntdev_priv *priv;
 			struct gntdev_grant_map *map;
 		} exp;
+		struct {
+			/* Granted references of the imported buffer. */
+			grant_ref_t *refs;
+			/* Scatter-gather table of the imported buffer. */
+			struct sg_table *sgt;
+			/* dma-buf attachment of the imported buffer. */
+			struct dma_buf_attachment *attach;
+		} imp;
 	} u;
 
 	/* Number of pages this buffer has. */
@@ -55,6 +72,8 @@ struct gntdev_dmabuf_priv {
 	struct list_head exp_list;
 	/* List of wait objects. */
 	struct list_head exp_wait_list;
+	/* List of imported DMA buffers. */
+	struct list_head imp_list;
 	/* This is the lock which protects dma_buf_xxx lists. */
 	struct mutex lock;
 };
@@ -500,21 +519,237 @@ int gntdev_dmabuf_exp_from_refs(struct gntdev_priv *priv, int flags,
 
 /* DMA buffer import support. */
 
+static int
+dmabuf_imp_grant_foreign_access(struct page **pages, u32 *refs,
+				int count, int domid)
+{
+	grant_ref_t priv_gref_head;
+	int i, ret;
+
+	ret = gnttab_alloc_grant_references(count, &priv_gref_head);
+	if (ret < 0) {
+		pr_debug("Cannot allocate grant references, ret %d\n", ret);
+		return ret;
+	}
+
+	for (i = 0; i < count; i++) {
+		int cur_ref;
+
+		cur_ref = gnttab_claim_grant_reference(&priv_gref_head);
+		if (cur_ref < 0) {
+			ret = cur_ref;
+			pr_debug("Cannot claim grant reference, ret %d\n", ret);
+			goto out;
+		}
+
+		gnttab_grant_foreign_access_ref(cur_ref, domid,
+						xen_page_to_gfn(pages[i]), 0);
+		refs[i] = cur_ref;
+	}
+
+	return 0;
+
+out:
+	gnttab_free_grant_references(priv_gref_head);
+	return ret;
+}
+
+static void dmabuf_imp_end_foreign_access(u32 *refs, int count)
+{
+	int i;
+
+	for (i = 0; i < count; i++)
+		if (refs[i] != GRANT_INVALID_REF)
+			gnttab_end_foreign_access(refs[i], 0, 0UL);
+}
+
+static void dmabuf_imp_free_storage(struct gntdev_dmabuf *gntdev_dmabuf)
+{
+	kfree(gntdev_dmabuf->pages);
+	kfree(gntdev_dmabuf->u.imp.refs);
+	kfree(gntdev_dmabuf);
+}
+
+static struct gntdev_dmabuf *dmabuf_imp_alloc_storage(int count)
+{
+	struct gntdev_dmabuf *gntdev_dmabuf;
+	int i;
+
+	gntdev_dmabuf = kzalloc(sizeof(*gntdev_dmabuf), GFP_KERNEL);
+	if (!gntdev_dmabuf)
+		goto fail;
+
+	gntdev_dmabuf->u.imp.refs = kcalloc(count,
+					    sizeof(gntdev_dmabuf->u.imp.refs[0]),
+					    GFP_KERNEL);
+	if (!gntdev_dmabuf->u.imp.refs)
+		goto fail;
+
+	gntdev_dmabuf->pages = kcalloc(count,
+				       sizeof(gntdev_dmabuf->pages[0]),
+				       GFP_KERNEL);
+	if (!gntdev_dmabuf->pages)
+		goto fail;
+
+	gntdev_dmabuf->nr_pages = count;
+
+	for (i = 0; i < count; i++)
+		gntdev_dmabuf->u.imp.refs[i] = GRANT_INVALID_REF;
+
+	return gntdev_dmabuf;
+
+fail:
+	dmabuf_imp_free_storage(gntdev_dmabuf);
+	return ERR_PTR(-ENOMEM);
+}
+
 struct gntdev_dmabuf *
 gntdev_dmabuf_imp_to_refs(struct gntdev_dmabuf_priv *priv, struct device *dev,
 			  int fd, int count, int domid)
 {
-	return ERR_PTR(-ENOMEM);
+	struct gntdev_dmabuf *gntdev_dmabuf, *ret;
+	struct dma_buf *dma_buf;
+	struct dma_buf_attachment *attach;
+	struct sg_table *sgt;
+	struct sg_page_iter sg_iter;
+	int i;
+
+	dma_buf = dma_buf_get(fd);
+	if (IS_ERR(dma_buf))
+		return ERR_CAST(dma_buf);
+
+	gntdev_dmabuf = dmabuf_imp_alloc_storage(count);
+	if (IS_ERR(gntdev_dmabuf)) {
+		ret = gntdev_dmabuf;
+		goto fail_put;
+}
+
+	gntdev_dmabuf->priv = priv;
+	gntdev_dmabuf->fd = fd;
+
+	attach = dma_buf_attach(dma_buf, dev);
+	if (IS_ERR(attach)) {
+		ret = ERR_CAST(attach);
+		goto fail_free_obj;
+	}
+
+	gntdev_dmabuf->u.imp.attach = attach;
+
+	sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
+	if (IS_ERR(sgt)) {
+		ret = ERR_CAST(sgt);
+		goto fail_detach;
+	}
+
+	/* Check number of pages that imported buffer has. */
+	if (attach->dmabuf->size != gntdev_dmabuf->nr_pages << PAGE_SHIFT) {
+		ret = ERR_PTR(-EINVAL);
+		pr_debug("DMA buffer has %zu pages, user-space expects %d\n",
+			 attach->dmabuf->size, gntdev_dmabuf->nr_pages);
+		goto fail_unmap;
+	}
+
+	gntdev_dmabuf->u.imp.sgt = sgt;
+
+	/* Now convert sgt to array of pages and check for page validity. */
+	i = 0;
+	for_each_sg_page(sgt->sgl, &sg_iter, sgt->nents, 0) {
+		struct page *page = sg_page_iter_page(&sg_iter);
+		/*
+		 * Check if page is valid: this can happen if we are given
+		 * a page from VRAM or other resources which are not backed
+		 * by a struct page.
+		 */
+		if (!pfn_valid(page_to_pfn(page))) {
+			ret = ERR_PTR(-EINVAL);
+			goto fail_unmap;
+		}
+
+		gntdev_dmabuf->pages[i++] = page;
+	}
+
+	ret = ERR_PTR(dmabuf_imp_grant_foreign_access(gntdev_dmabuf->pages,
+						      gntdev_dmabuf->u.imp.refs,
+						      count, domid));
+	if (IS_ERR(ret))
+		goto fail_end_access;
+
+	pr_debug("Imported DMA buffer with fd %d\n", fd);
+
+	mutex_lock(&priv->lock);
+	list_add(&gntdev_dmabuf->next, &priv->imp_list);
+	mutex_unlock(&priv->lock);
+
+	return gntdev_dmabuf;
+
+fail_end_access:
+	dmabuf_imp_end_foreign_access(gntdev_dmabuf->u.imp.refs, count);
+fail_unmap:
+	dma_buf_unmap_attachment(attach, sgt, DMA_BIDIRECTIONAL);
+fail_detach:
+	dma_buf_detach(dma_buf, attach);
+fail_free_obj:
+	dmabuf_imp_free_storage(gntdev_dmabuf);
+fail_put:
+	dma_buf_put(dma_buf);
+	return ret;
 }
 
 u32 *gntdev_dmabuf_imp_get_refs(struct gntdev_dmabuf *gntdev_dmabuf)
 {
+	if (gntdev_dmabuf)
+		return gntdev_dmabuf->u.imp.refs;
+
 	return NULL;
 }
 
+/*
+ * Find the hyper dma-buf by its file descriptor and remove
+ * it from the buffer's list.
+ */
+static struct gntdev_dmabuf *
+dmabuf_imp_find_unlink(struct gntdev_dmabuf_priv *priv, int fd)
+{
+	struct gntdev_dmabuf *q, *gntdev_dmabuf, *ret = ERR_PTR(-ENOENT);
+
+	mutex_lock(&priv->lock);
+	list_for_each_entry_safe(gntdev_dmabuf, q, &priv->imp_list, next) {
+		if (gntdev_dmabuf->fd == fd) {
+			pr_debug("Found gntdev_dmabuf in the import list\n");
+			ret = gntdev_dmabuf;
+			list_del(&gntdev_dmabuf->next);
+			break;
+		}
+	}
+	mutex_unlock(&priv->lock);
+	return ret;
+}
+
 int gntdev_dmabuf_imp_release(struct gntdev_dmabuf_priv *priv, u32 fd)
 {
-	return -EINVAL;
+	struct gntdev_dmabuf *gntdev_dmabuf;
+	struct dma_buf_attachment *attach;
+	struct dma_buf *dma_buf;
+
+	gntdev_dmabuf = dmabuf_imp_find_unlink(priv, fd);
+	if (IS_ERR(gntdev_dmabuf))
+		return PTR_ERR(gntdev_dmabuf);
+
+	pr_debug("Releasing DMA buffer with fd %d\n", fd);
+
+	attach = gntdev_dmabuf->u.imp.attach;
+
+	if (gntdev_dmabuf->u.imp.sgt)
+		dma_buf_unmap_attachment(attach, gntdev_dmabuf->u.imp.sgt,
+					 DMA_BIDIRECTIONAL);
+	dma_buf = attach->dmabuf;
+	dma_buf_detach(attach->dmabuf, attach);
+	dma_buf_put(dma_buf);
+
+	dmabuf_imp_end_foreign_access(gntdev_dmabuf->u.imp.refs,
+				      gntdev_dmabuf->nr_pages);
+	dmabuf_imp_free_storage(gntdev_dmabuf);
+	return 0;
 }
 
 struct gntdev_dmabuf_priv *gntdev_dmabuf_init(void)
@@ -528,6 +763,7 @@ struct gntdev_dmabuf_priv *gntdev_dmabuf_init(void)
 	mutex_init(&priv->lock);
 	INIT_LIST_HEAD(&priv->exp_list);
 	INIT_LIST_HEAD(&priv->exp_wait_list);
+	INIT_LIST_HEAD(&priv->imp_list);
 
 	return priv;
 }
Signature · 1 line
2.17.1
#58 lore raw

Re: [PATCH v3 9/9] xen/gntdev: Implement dma-buf import functionality

Quoted 14 lines · Oleksandr Andrushchenko
  int gntdev_dmabuf_imp_release(struct gntdev_dmabuf_priv *priv, u32 fd)
  {
drivers/xen/gntdev-dmabuf.c +11 -1
-	return -EINVAL;
+	struct gntdev_dmabuf *gntdev_dmabuf;
+	struct dma_buf_attachment *attach;
+	struct dma_buf *dma_buf;
+
+	gntdev_dmabuf = dmabuf_imp_find_unlink(priv, fd);
+	if (IS_ERR(gntdev_dmabuf))
+		return PTR_ERR(gntdev_dmabuf);
+
+	pr_debug("Releasing DMA buffer with fd %d\n", fd);
+
+	attach = gntdev_dmabuf->u.imp.attach;

drivers/xen/gntdev-dmabuf.c int gntdev_dmabuf_exp_from_refs(struct gntdev_priv *priv, int flags, · line 750

Quoted 10 lines
drivers/xen/gntdev-dmabuf.c +10 -0
+
+	if (gntdev_dmabuf->u.imp.sgt)
+		dma_buf_unmap_attachment(attach, gntdev_dmabuf->u.imp.sgt,
+					 DMA_BIDIRECTIONAL);
+	dma_buf = attach->dmabuf;
+	dma_buf_detach(attach->dmabuf, attach);
+	dma_buf_put(dma_buf);
+
+	dmabuf_imp_end_foreign_access(gntdev_dmabuf->u.imp.refs,
+				      gntdev_dmabuf->nr_pages);

Should you first end foreign access, before doing anything?

-boris

Quoted 4 lines
drivers/xen/gntdev-dmabuf.c +2 -0
+	dmabuf_imp_free_storage(gntdev_dmabuf);
+	return 0;
  }
#59 lore raw

Re: [PATCH v3 9/9] xen/gntdev: Implement dma-buf import functionality

Quoted 24 lines · Boris Ostrovsky
Quoted 20 lines · Oleksandr Andrushchenko

  int gntdev_dmabuf_imp_release(struct gntdev_dmabuf_priv *priv, u32 fd) {
-    return -EINVAL;
+    struct gntdev_dmabuf *gntdev_dmabuf;
+    struct dma_buf_attachment *attach;
+    struct dma_buf *dma_buf;

drivers/xen/gntdev-dmabuf.c +14 -0
+
+    gntdev_dmabuf = dmabuf_imp_find_unlink(priv, fd);
+    if (IS_ERR(gntdev_dmabuf))
+        return PTR_ERR(gntdev_dmabuf);
+
+    pr_debug("Releasing DMA buffer with fd %d\n", fd);
+
+    attach = gntdev_dmabuf->u.imp.attach;
+
+    if (gntdev_dmabuf->u.imp.sgt)
+        dma_buf_unmap_attachment(attach, gntdev_dmabuf->u.imp.sgt,
+                     DMA_BIDIRECTIONAL);
+    dma_buf = attach->dmabuf;
+    dma_buf_detach(attach->dmabuf, attach);
Quoted 9 lines
Quoted 4 lines

+    dma_buf_put(dma_buf);
+
+    dmabuf_imp_end_foreign_access(gntdev_dmabuf->u.imp.refs, +                      gntdev_dmabuf->nr_pages);

Should you first end foreign access, before doing anything?

I am rolling back in reverse order here, so I think we first need to finish local activities with the buffer and then end foreign access.

Quoted 6 lines

-boris

Quoted 3 lines

+ dmabuf_imp_free_storage(gntdev_dmabuf);
+    return 0;
  }

#60 lore raw

Re: [PATCH v3 9/9] xen/gntdev: Implement dma-buf import functionality

Quoted 35 lines · Oleksandr Andrushchenko
Quoted 25 lines · Boris Ostrovsky
Quoted 21 lines · Oleksandr Andrushchenko

  int gntdev_dmabuf_imp_release(struct gntdev_dmabuf_priv *priv, u32 fd)
  {
-    return -EINVAL;
+    struct gntdev_dmabuf *gntdev_dmabuf;
+    struct dma_buf_attachment *attach;
+    struct dma_buf *dma_buf;

drivers/xen/gntdev-dmabuf.c +14 -0
+
+    gntdev_dmabuf = dmabuf_imp_find_unlink(priv, fd);
+    if (IS_ERR(gntdev_dmabuf))
+        return PTR_ERR(gntdev_dmabuf);
+
+    pr_debug("Releasing DMA buffer with fd %d\n", fd);
+
+    attach = gntdev_dmabuf->u.imp.attach;
+
+    if (gntdev_dmabuf->u.imp.sgt)
+        dma_buf_unmap_attachment(attach, gntdev_dmabuf->u.imp.sgt,
+                     DMA_BIDIRECTIONAL);
+    dma_buf = attach->dmabuf;
+    dma_buf_detach(attach->dmabuf, attach);
Quoted 9 lines
Quoted 4 lines

+    dma_buf_put(dma_buf);
+
+    dmabuf_imp_end_foreign_access(gntdev_dmabuf->u.imp.refs, +                      gntdev_dmabuf->nr_pages);

Should you first end foreign access, before doing anything?

Quoted 3 lines

I am rolling back in reverse order here, so I think we first need to finish local activities with the buffer and then end foreign access.

Looking at gntdev_dmabuf_imp_to_refs(), the order is dmabuf_imp_alloc_storage() dma_buf_attach() dma_buf_map_attachment() dmabuf_imp_grant_foreign_access()

Or was I looking at wrong place?

-boris

#61 lore raw

Re: [PATCH v3 9/9] xen/gntdev: Implement dma-buf import functionality

Quoted 37 lines · Boris Ostrovsky
Quoted 33 lines · Oleksandr Andrushchenko
Quoted 23 lines · Boris Ostrovsky
Quoted 20 lines · Oleksandr Andrushchenko

  int gntdev_dmabuf_imp_release(struct gntdev_dmabuf_priv *priv, u32
fd)
  {
-    return -EINVAL;
+    struct gntdev_dmabuf *gntdev_dmabuf;
+    struct dma_buf_attachment *attach;
+    struct dma_buf *dma_buf;

drivers/xen/gntdev-dmabuf.c +13 -0
+
+    gntdev_dmabuf = dmabuf_imp_find_unlink(priv, fd);
+    if (IS_ERR(gntdev_dmabuf))
+        return PTR_ERR(gntdev_dmabuf);
+
+    pr_debug("Releasing DMA buffer with fd %d\n", fd);
+
+    attach = gntdev_dmabuf->u.imp.attach;
+
+    if (gntdev_dmabuf->u.imp.sgt)
+        dma_buf_unmap_attachment(attach, gntdev_dmabuf->u.imp.sgt,
+                     DMA_BIDIRECTIONAL);
+    dma_buf = attach->dmabuf;
Quoted 9 lines
Quoted 5 lines

+    dma_buf_detach(attach->dmabuf, attach);
+    dma_buf_put(dma_buf);
+
+    dmabuf_imp_end_foreign_access(gntdev_dmabuf->u.imp.refs, +                      gntdev_dmabuf->nr_pages);

Should you first end foreign access, before doing anything?

Quoted 3 lines

I am rolling back in reverse order here, so I think we first need to finish local activities with the buffer and then end foreign access.

Quoted 7 lines

Looking at gntdev_dmabuf_imp_to_refs(), the order is
    dmabuf_imp_alloc_storage() dma_buf_attach() dma_buf_map_attachment() dmabuf_imp_grant_foreign_access()

Or was I looking at wrong place?

Agree, will move as you suggest

Quoted 1 line

-boris

#62 lore raw

Re: [PATCH v3 9/9] xen/gntdev: Implement dma-buf import functionality

In reply to Boris Ostrovsky
Quoted 37 lines · Boris Ostrovsky
Quoted 33 lines · Oleksandr Andrushchenko
Quoted 23 lines · Boris Ostrovsky
Quoted 20 lines · Oleksandr Andrushchenko

  int gntdev_dmabuf_imp_release(struct gntdev_dmabuf_priv *priv, u32
fd)
  {
-    return -EINVAL;
+    struct gntdev_dmabuf *gntdev_dmabuf;
+    struct dma_buf_attachment *attach;
+    struct dma_buf *dma_buf;

drivers/xen/gntdev-dmabuf.c +13 -0
+
+    gntdev_dmabuf = dmabuf_imp_find_unlink(priv, fd);
+    if (IS_ERR(gntdev_dmabuf))
+        return PTR_ERR(gntdev_dmabuf);
+
+    pr_debug("Releasing DMA buffer with fd %d\n", fd);
+
+    attach = gntdev_dmabuf->u.imp.attach;
+
+    if (gntdev_dmabuf->u.imp.sgt)
+        dma_buf_unmap_attachment(attach, gntdev_dmabuf->u.imp.sgt,
+                     DMA_BIDIRECTIONAL);
+    dma_buf = attach->dmabuf;
Quoted 9 lines
Quoted 5 lines

+    dma_buf_detach(attach->dmabuf, attach);
+    dma_buf_put(dma_buf);
+
+    dmabuf_imp_end_foreign_access(gntdev_dmabuf->u.imp.refs, +                      gntdev_dmabuf->nr_pages);

Should you first end foreign access, before doing anything?

Quoted 3 lines

I am rolling back in reverse order here, so I think we first need to finish local activities with the buffer and then end foreign access.

Quoted 7 lines

Looking at gntdev_dmabuf_imp_to_refs(), the order is
    dmabuf_imp_alloc_storage() dma_buf_attach() dma_buf_map_attachment() dmabuf_imp_grant_foreign_access()

Or was I looking at wrong place?

Agree, will move as you suggest

Quoted 1 line

-boris

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#63 lore raw

Re: [PATCH v3 9/9] xen/gntdev: Implement dma-buf import functionality

In reply to Oleksandr Andrushchenko
Quoted 35 lines · Oleksandr Andrushchenko
Quoted 25 lines · Boris Ostrovsky
Quoted 21 lines · Oleksandr Andrushchenko

  int gntdev_dmabuf_imp_release(struct gntdev_dmabuf_priv *priv, u32 fd)
  {
-    return -EINVAL;
+    struct gntdev_dmabuf *gntdev_dmabuf;
+    struct dma_buf_attachment *attach;
+    struct dma_buf *dma_buf;

drivers/xen/gntdev-dmabuf.c +14 -0
+
+    gntdev_dmabuf = dmabuf_imp_find_unlink(priv, fd);
+    if (IS_ERR(gntdev_dmabuf))
+        return PTR_ERR(gntdev_dmabuf);
+
+    pr_debug("Releasing DMA buffer with fd %d\n", fd);
+
+    attach = gntdev_dmabuf->u.imp.attach;
+
+    if (gntdev_dmabuf->u.imp.sgt)
+        dma_buf_unmap_attachment(attach, gntdev_dmabuf->u.imp.sgt,
+                     DMA_BIDIRECTIONAL);
+    dma_buf = attach->dmabuf;
+    dma_buf_detach(attach->dmabuf, attach);
Quoted 9 lines
Quoted 4 lines

+    dma_buf_put(dma_buf);
+
+    dmabuf_imp_end_foreign_access(gntdev_dmabuf->u.imp.refs, +                      gntdev_dmabuf->nr_pages);

Should you first end foreign access, before doing anything?

Quoted 3 lines

I am rolling back in reverse order here, so I think we first need to finish local activities with the buffer and then end foreign access.

Looking at gntdev_dmabuf_imp_to_refs(), the order is dmabuf_imp_alloc_storage() dma_buf_attach() dma_buf_map_attachment() dmabuf_imp_grant_foreign_access()

Or was I looking at wrong place?

-boris

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#64 lore raw

Re: [PATCH v3 9/9] xen/gntdev: Implement dma-buf import functionality

In reply to Boris Ostrovsky
Quoted 24 lines · Boris Ostrovsky
Quoted 20 lines · Oleksandr Andrushchenko

  int gntdev_dmabuf_imp_release(struct gntdev_dmabuf_priv *priv, u32 fd) {
-    return -EINVAL;
+    struct gntdev_dmabuf *gntdev_dmabuf;
+    struct dma_buf_attachment *attach;
+    struct dma_buf *dma_buf;

drivers/xen/gntdev-dmabuf.c +14 -0
+
+    gntdev_dmabuf = dmabuf_imp_find_unlink(priv, fd);
+    if (IS_ERR(gntdev_dmabuf))
+        return PTR_ERR(gntdev_dmabuf);
+
+    pr_debug("Releasing DMA buffer with fd %d\n", fd);
+
+    attach = gntdev_dmabuf->u.imp.attach;
+
+    if (gntdev_dmabuf->u.imp.sgt)
+        dma_buf_unmap_attachment(attach, gntdev_dmabuf->u.imp.sgt,
+                     DMA_BIDIRECTIONAL);
+    dma_buf = attach->dmabuf;
+    dma_buf_detach(attach->dmabuf, attach);
Quoted 9 lines
Quoted 4 lines

+    dma_buf_put(dma_buf);
+
+    dmabuf_imp_end_foreign_access(gntdev_dmabuf->u.imp.refs, +                      gntdev_dmabuf->nr_pages);

Should you first end foreign access, before doing anything?

I am rolling back in reverse order here, so I think we first need to finish local activities with the buffer and then end foreign access.

Quoted 6 lines

-boris

Quoted 3 lines

+ dmabuf_imp_free_storage(gntdev_dmabuf);
+    return 0;
  }

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#65 lore raw

Re: [PATCH v3 9/9] xen/gntdev: Implement dma-buf import functionality

In reply to Oleksandr Andrushchenko
Quoted 14 lines · Oleksandr Andrushchenko
  int gntdev_dmabuf_imp_release(struct gntdev_dmabuf_priv *priv, u32 fd)
  {
drivers/xen/gntdev-dmabuf.c +11 -1
-	return -EINVAL;
+	struct gntdev_dmabuf *gntdev_dmabuf;
+	struct dma_buf_attachment *attach;
+	struct dma_buf *dma_buf;
+
+	gntdev_dmabuf = dmabuf_imp_find_unlink(priv, fd);
+	if (IS_ERR(gntdev_dmabuf))
+		return PTR_ERR(gntdev_dmabuf);
+
+	pr_debug("Releasing DMA buffer with fd %d\n", fd);
+
+	attach = gntdev_dmabuf->u.imp.attach;

drivers/xen/gntdev-dmabuf.c int gntdev_dmabuf_exp_from_refs(struct gntdev_priv *priv, int flags, · line 750

Quoted 10 lines
drivers/xen/gntdev-dmabuf.c +10 -0
+
+	if (gntdev_dmabuf->u.imp.sgt)
+		dma_buf_unmap_attachment(attach, gntdev_dmabuf->u.imp.sgt,
+					 DMA_BIDIRECTIONAL);
+	dma_buf = attach->dmabuf;
+	dma_buf_detach(attach->dmabuf, attach);
+	dma_buf_put(dma_buf);
+
+	dmabuf_imp_end_foreign_access(gntdev_dmabuf->u.imp.refs,
+				      gntdev_dmabuf->nr_pages);

Should you first end foreign access, before doing anything?

-boris

drivers/xen/gntdev-dmabuf.c int gntdev_dmabuf_exp_from_refs(struct gntdev_priv *priv, int flags, · line 753

Quoted 4 lines
drivers/xen/gntdev-dmabuf.c +2 -0
+	dmabuf_imp_free_storage(gntdev_dmabuf);
+	return 0;
  }

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#66 lore raw

PATCH v3 9/9 xen/gntdev: Implement dma-buf import functionality

In reply to Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

1. Import a dma-buf with the file descriptor provided and export granted references to the pages of that dma-buf into the array of grant references.

2. Add API to close all references to an imported buffer, so it can be released by the owner. This is only valid for buffers created with IOCTL_GNTDEV_DMABUF_IMP_TO_REFS.

---
 drivers/xen/gntdev-dmabuf.c | 240 +++++++++++++++++++++++++++++++++++-
 1 file changed, 238 insertions(+), 2 deletions(-)
drivers/xen/gntdev-dmabuf.c +238 -2
index 84cba67c6ad7..4d250eb8babc 100644
@@ -17,6 +17,15 @@
 #include "gntdev-common.h"
 #include "gntdev-dmabuf.h"
 
+#ifndef GRANT_INVALID_REF
+/*
+ * Note on usage of grant reference 0 as invalid grant reference:
+ * grant reference 0 is valid, but never exposed to a driver,
+ * because of the fact it is already in use/reserved by the PV console.
+ */
+#define GRANT_INVALID_REF	0
+#endif
+
 struct gntdev_dmabuf {
 	struct gntdev_dmabuf_priv *priv;
 	struct dma_buf *dmabuf;
@@ -31,6 +40,14 @@ struct gntdev_dmabuf {
 			struct gntdev_priv *priv;
 			struct gntdev_grant_map *map;
 		} exp;
+		struct {
+			/* Granted references of the imported buffer. */
+			grant_ref_t *refs;
+			/* Scatter-gather table of the imported buffer. */
+			struct sg_table *sgt;
+			/* dma-buf attachment of the imported buffer. */
+			struct dma_buf_attachment *attach;
+		} imp;
 	} u;
 
 	/* Number of pages this buffer has. */
@@ -55,6 +72,8 @@ struct gntdev_dmabuf_priv {
 	struct list_head exp_list;
 	/* List of wait objects. */
 	struct list_head exp_wait_list;
+	/* List of imported DMA buffers. */
+	struct list_head imp_list;
 	/* This is the lock which protects dma_buf_xxx lists. */
 	struct mutex lock;
 };
@@ -500,21 +519,237 @@ int gntdev_dmabuf_exp_from_refs(struct gntdev_priv *priv, int flags,
 
 /* DMA buffer import support. */
 
+static int
+dmabuf_imp_grant_foreign_access(struct page **pages, u32 *refs,
+				int count, int domid)
+{
+	grant_ref_t priv_gref_head;
+	int i, ret;
+
+	ret = gnttab_alloc_grant_references(count, &priv_gref_head);
+	if (ret < 0) {
+		pr_debug("Cannot allocate grant references, ret %d\n", ret);
+		return ret;
+	}
+
+	for (i = 0; i < count; i++) {
+		int cur_ref;
+
+		cur_ref = gnttab_claim_grant_reference(&priv_gref_head);
+		if (cur_ref < 0) {
+			ret = cur_ref;
+			pr_debug("Cannot claim grant reference, ret %d\n", ret);
+			goto out;
+		}
+
+		gnttab_grant_foreign_access_ref(cur_ref, domid,
+						xen_page_to_gfn(pages[i]), 0);
+		refs[i] = cur_ref;
+	}
+
+	return 0;
+
+out:
+	gnttab_free_grant_references(priv_gref_head);
+	return ret;
+}
+
+static void dmabuf_imp_end_foreign_access(u32 *refs, int count)
+{
+	int i;
+
+	for (i = 0; i < count; i++)
+		if (refs[i] != GRANT_INVALID_REF)
+			gnttab_end_foreign_access(refs[i], 0, 0UL);
+}
+
+static void dmabuf_imp_free_storage(struct gntdev_dmabuf *gntdev_dmabuf)
+{
+	kfree(gntdev_dmabuf->pages);
+	kfree(gntdev_dmabuf->u.imp.refs);
+	kfree(gntdev_dmabuf);
+}
+
+static struct gntdev_dmabuf *dmabuf_imp_alloc_storage(int count)
+{
+	struct gntdev_dmabuf *gntdev_dmabuf;
+	int i;
+
+	gntdev_dmabuf = kzalloc(sizeof(*gntdev_dmabuf), GFP_KERNEL);
+	if (!gntdev_dmabuf)
+		goto fail;
+
+	gntdev_dmabuf->u.imp.refs = kcalloc(count,
+					    sizeof(gntdev_dmabuf->u.imp.refs[0]),
+					    GFP_KERNEL);
+	if (!gntdev_dmabuf->u.imp.refs)
+		goto fail;
+
+	gntdev_dmabuf->pages = kcalloc(count,
+				       sizeof(gntdev_dmabuf->pages[0]),
+				       GFP_KERNEL);
+	if (!gntdev_dmabuf->pages)
+		goto fail;
+
+	gntdev_dmabuf->nr_pages = count;
+
+	for (i = 0; i < count; i++)
+		gntdev_dmabuf->u.imp.refs[i] = GRANT_INVALID_REF;
+
+	return gntdev_dmabuf;
+
+fail:
+	dmabuf_imp_free_storage(gntdev_dmabuf);
+	return ERR_PTR(-ENOMEM);
+}
+
 struct gntdev_dmabuf *
 gntdev_dmabuf_imp_to_refs(struct gntdev_dmabuf_priv *priv, struct device *dev,
 			  int fd, int count, int domid)
 {
-	return ERR_PTR(-ENOMEM);
+	struct gntdev_dmabuf *gntdev_dmabuf, *ret;
+	struct dma_buf *dma_buf;
+	struct dma_buf_attachment *attach;
+	struct sg_table *sgt;
+	struct sg_page_iter sg_iter;
+	int i;
+
+	dma_buf = dma_buf_get(fd);
+	if (IS_ERR(dma_buf))
+		return ERR_CAST(dma_buf);
+
+	gntdev_dmabuf = dmabuf_imp_alloc_storage(count);
+	if (IS_ERR(gntdev_dmabuf)) {
+		ret = gntdev_dmabuf;
+		goto fail_put;
+}
+
+	gntdev_dmabuf->priv = priv;
+	gntdev_dmabuf->fd = fd;
+
+	attach = dma_buf_attach(dma_buf, dev);
+	if (IS_ERR(attach)) {
+		ret = ERR_CAST(attach);
+		goto fail_free_obj;
+	}
+
+	gntdev_dmabuf->u.imp.attach = attach;
+
+	sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
+	if (IS_ERR(sgt)) {
+		ret = ERR_CAST(sgt);
+		goto fail_detach;
+	}
+
+	/* Check number of pages that imported buffer has. */
+	if (attach->dmabuf->size != gntdev_dmabuf->nr_pages << PAGE_SHIFT) {
+		ret = ERR_PTR(-EINVAL);
+		pr_debug("DMA buffer has %zu pages, user-space expects %d\n",
+			 attach->dmabuf->size, gntdev_dmabuf->nr_pages);
+		goto fail_unmap;
+	}
+
+	gntdev_dmabuf->u.imp.sgt = sgt;
+
+	/* Now convert sgt to array of pages and check for page validity. */
+	i = 0;
+	for_each_sg_page(sgt->sgl, &sg_iter, sgt->nents, 0) {
+		struct page *page = sg_page_iter_page(&sg_iter);
+		/*
+		 * Check if page is valid: this can happen if we are given
+		 * a page from VRAM or other resources which are not backed
+		 * by a struct page.
+		 */
+		if (!pfn_valid(page_to_pfn(page))) {
+			ret = ERR_PTR(-EINVAL);
+			goto fail_unmap;
+		}
+
+		gntdev_dmabuf->pages[i++] = page;
+	}
+
+	ret = ERR_PTR(dmabuf_imp_grant_foreign_access(gntdev_dmabuf->pages,
+						      gntdev_dmabuf->u.imp.refs,
+						      count, domid));
+	if (IS_ERR(ret))
+		goto fail_end_access;
+
+	pr_debug("Imported DMA buffer with fd %d\n", fd);
+
+	mutex_lock(&priv->lock);
+	list_add(&gntdev_dmabuf->next, &priv->imp_list);
+	mutex_unlock(&priv->lock);
+
+	return gntdev_dmabuf;
+
+fail_end_access:
+	dmabuf_imp_end_foreign_access(gntdev_dmabuf->u.imp.refs, count);
+fail_unmap:
+	dma_buf_unmap_attachment(attach, sgt, DMA_BIDIRECTIONAL);
+fail_detach:
+	dma_buf_detach(dma_buf, attach);
+fail_free_obj:
+	dmabuf_imp_free_storage(gntdev_dmabuf);
+fail_put:
+	dma_buf_put(dma_buf);
+	return ret;
 }
 
 u32 *gntdev_dmabuf_imp_get_refs(struct gntdev_dmabuf *gntdev_dmabuf)
 {
+	if (gntdev_dmabuf)
+		return gntdev_dmabuf->u.imp.refs;
+
 	return NULL;
 }
 
+/*
+ * Find the hyper dma-buf by its file descriptor and remove
+ * it from the buffer's list.
+ */
+static struct gntdev_dmabuf *
+dmabuf_imp_find_unlink(struct gntdev_dmabuf_priv *priv, int fd)
+{
+	struct gntdev_dmabuf *q, *gntdev_dmabuf, *ret = ERR_PTR(-ENOENT);
+
+	mutex_lock(&priv->lock);
+	list_for_each_entry_safe(gntdev_dmabuf, q, &priv->imp_list, next) {
+		if (gntdev_dmabuf->fd == fd) {
+			pr_debug("Found gntdev_dmabuf in the import list\n");
+			ret = gntdev_dmabuf;
+			list_del(&gntdev_dmabuf->next);
+			break;
+		}
+	}
+	mutex_unlock(&priv->lock);
+	return ret;
+}
+
 int gntdev_dmabuf_imp_release(struct gntdev_dmabuf_priv *priv, u32 fd)
 {
-	return -EINVAL;
+	struct gntdev_dmabuf *gntdev_dmabuf;
+	struct dma_buf_attachment *attach;
+	struct dma_buf *dma_buf;
+
+	gntdev_dmabuf = dmabuf_imp_find_unlink(priv, fd);
+	if (IS_ERR(gntdev_dmabuf))
+		return PTR_ERR(gntdev_dmabuf);
+
+	pr_debug("Releasing DMA buffer with fd %d\n", fd);
+
+	attach = gntdev_dmabuf->u.imp.attach;
+
+	if (gntdev_dmabuf->u.imp.sgt)
+		dma_buf_unmap_attachment(attach, gntdev_dmabuf->u.imp.sgt,
+					 DMA_BIDIRECTIONAL);
+	dma_buf = attach->dmabuf;
+	dma_buf_detach(attach->dmabuf, attach);
+	dma_buf_put(dma_buf);
+
+	dmabuf_imp_end_foreign_access(gntdev_dmabuf->u.imp.refs,
+				      gntdev_dmabuf->nr_pages);
+	dmabuf_imp_free_storage(gntdev_dmabuf);
+	return 0;
 }
 
 struct gntdev_dmabuf_priv *gntdev_dmabuf_init(void)
@@ -528,6 +763,7 @@ struct gntdev_dmabuf_priv *gntdev_dmabuf_init(void)
 	mutex_init(&priv->lock);
 	INIT_LIST_HEAD(&priv->exp_list);
 	INIT_LIST_HEAD(&priv->exp_wait_list);
+	INIT_LIST_HEAD(&priv->imp_list);
 
 	return priv;
 }
Signature · 7 lines
2.17.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#68 lore raw
Quoted 8 lines · Oleksandr Andrushchenko

Hi, Boris!

It seems that I have resolved all the issues now which were mainly cleanup and code movement and 5 of 9 patches already have r-b's. Do you, as the primary reviewer of the series, think I can push (hopefully) the final version of the patches? Just in case you want to look at v4 it is at [1].

Quoted 5 lines

Thank you,
Oleksandr

[1]
https://github.com/andr2000/linux/commits/xen_tip_linux_next_xen_dma_buf_v4

Looks good to me.

-boris

#69 lore raw
In reply to Oleksandr Andrushchenko
Quoted 8 lines · Oleksandr Andrushchenko

Hi, Boris!

It seems that I have resolved all the issues now which were mainly cleanup and code movement and 5 of 9 patches already have r-b's. Do you, as the primary reviewer of the series, think I can push (hopefully) the final version of the patches? Just in case you want to look at v4 it is at [1].

Quoted 5 lines

Thank you,
Oleksandr

[1]
https://github.com/andr2000/linux/commits/xen_tip_linux_next_xen_dma_buf_v4

Looks good to me.

-boris

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#70 lore raw
In reply to Oleksandr Andrushchenko

Hi, Boris!

It seems that I have resolved all the issues now which were mainly cleanup and code movement and 5 of 9 patches already have r-b's. Do you, as the primary reviewer of the series, think I can push (hopefully) the final version of the patches? Just in case you want to look at v4 it is at [1].

Thank you,
Oleksandr

[1] https://github.com/andr2000/linux/commits/xen_tip_linux_next_xen_dma_buf_v4

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel