Skip to contentSkip to conversation

Android ION for drivers/staging

PATCH In review John Stultz 4 participants 134 messages Dec 13, 2013 – Dec 14, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Android ION for drivers/staging

This patchset is to submit the Android ION code to drivers/staging.

ION is a constraint aware memory allocator used by Android in order to allocate memory that can be properly shared between a number of devices that possibly have different memory access constraints.

You can read some background on ION as well as some problematic aspects of ION in my lwn summary here:

	https://lwn.net/Articles/565469/

Recently, Colin Cross has put in quite a bit of effort to resolve a number of the issues identified in that article as well as other.

While a number of issues remain, including negative sentiments from upstream maintainers about the interface, ION provides functionality (constraints based allocation), which is not currently available upstream.

As discussed at Linux Plumbers this year (https://lwn.net/Articles/569704/)
it was proposed to merge the ION code into staging. This will allow for better visibility to the upstream community, allowing further review and understanding of the code, but it will also aid in the design of dma-buf constraint-aware memory allocation helpers, allowing us to share the backing heap allocation implementations.

In order to preserve authorship history, I've taken all the ION changes in AOSP's common.git android-3.10 branch up to commit 8f1c42284159ebd6 (Improve ION config descriptions). The only modifications to the patches that I've made is pushing down Colin's patch to migrate ION to staging (ION was originally implemented in drivers/gpu/ion) so all the patches apply to staging from the start.

I've also had to add a single patch (the last one) which moves ION to use the new scan/count shrinker api introduced in 3.12.

Other then this one change, the only diff from AOSP's 3.10 branch are corrected path names in the header comment. ie:
- * drivers/gpu/ion/ion.c
+ * drivers/staging/android/ion/ion.c

This is a fairly large patchset, and is also available here:
http://git.linaro.org/git/people/john.stultz/android-dev.git dev/ion-staging

Please let me know if you have any thoughts or objections.

thanks
-john

Arve Hjønnevåg (2):
gpu: ion: __dma_page_cpu_to_dev -> arm_dma_ops.sync_single_for_device
hack
gpu: ion: Remove __GFP_NO_KSWAPD

Benjamin Gaignard (4):
gpu: ion: fix carveout ops
gpu: ion: fix compilation warning
gpu: ion: fix ion_platform_data definition
gpu: ion: add CMA heap

Colin Cross (36):
gpu: ion: ion_carveout_heap: fix for 3.4
ion: chunk_heap: fix leak in allocated counter
ion: add free list size to heap debug files
ion: convert map_kernel to return ERR_PTR
ion: remove IS_ERR_OR_NULL
ion: replace userspace handle cookies with idr
ion: index client->handles rbtree by buffer
ion: don't use id 0 for handle cookie
gpu: ion: delete ion_system_mapper.c
ion: move userspace api into uapi/ion.h
ion: hold reference to handle after ion_uhandle_get
ion: fix crash when alloc len is -1
ion: fix dma APIs
ion: convert sg_dma_len(sg) to sg->length
ion: check invalid values in ion_system_heap
ion: add test device for unit tests to interact with dma_bufs
ion: update idr to avoid deprecated apis
ion: don't use __arm_ioremap to map pages
ion: don't use phys_to_page or __phys_to_pfn
ion: fix printk warnings
gpu: ion: remove unnecessary function from system heap
ion: clean up ioctls
ion: drop dependency on ARM
ion: add alignment check to carveout heap
ion: optimize ion_heap_buffer_zero
ion: free low memory from page pools first
ion: check return value from remap_pfn_range
ion: use vm_insert_pfn for faulted pages
ion: remove ion_heap_alloc_pages
ion: allow cached mappings of chunk and system heap buffers
ion: use alloc_pages in system contig heap
ion: fix sparse warnings
ion: carveout heap: zero buffers on free, fix memory leak
ion: add helper to zero contiguous region of pages
ion: add alignment check to chunk heap
ion: fix bugs in cma heap

Dima Zavin (1):
gpu: ion: do not ask for compound pages in system heap

Greg Hackmann (2):
gpu: ion: fill in buffer->{dev,size} before mapping new buffers
gpu: ion: fix page offset in dma_buf_kmap()

JP Abgrall (1):
gpu: ion: fix kfree/list_del order

Johan Mossberg (2):
gpu: ion: Export ion_client_create
gpu: ion: Add support for sharing buffers with dma buf kernel handles

John Stultz (6):
ion: Add Kconfig dependency to ARM
ion: Fix compat support to use proper compat ioctl numbers
ion: Fix two small issues in system_heap allocation
ion: Cleanup whitespace issues and other checkpatch problems
ion: Improve ION config description
ion: Update system heap shrinker to use the new count/scan interface

KyongHo Cho (1):
gpu: ion: several bugfixes and enhancements of ION

Laura Abbott (3):
gpu: ion: Loop on the handle count when destroying
gpu: ion: Map only the vma size given
gpu: ion: Don't call ion_buffer_put on error path

Mitchel Humpherys (1):
gpu: ion: fix use-after-free in ion_heap_freelist_drain

Olav Haugan (3):
gpu: ion: Add missing argument to WARN call
gpu: ion: Add EXPORT_SYMBOL to functions
gpu: ion: IOCTL return success when error occurs

Rebecca Schultz Zavin (50):
gpu: ion: Add ION Memory Manager
ion: Switch map/unmap dma api to sg_tables
ion: Add reserve function to ion
ion: Switch ion to use dma-buf
gpu: ion: Use alloc_pages instead of vmalloc from the system heap
gpu: ion: support begin/end and kmap/kunmap dma_buf ops
gpu: ion: Allocate the sg_table at creation time rather than
dynamically
gpu: ion: Get an sg_table from an ion handle
gpu: ion: Set the dma_address of the sg list at alloc time
gpu: ion: ion_system_heap: Change allocations to GFP_HIGHUSER
gpu: ion: Add cache maintenance to ion.
gpu: ion: Modify the system heap to try to allocate large/huge pages
gpu: ion: Add explicit sync ioctl
gpu: ion: Only map as much of the vma as the user requested
gpu: ion: Switch to using kmalloc rather than kmap during allocation
gpu: ion: Fix race between ion_import and ion_free
gpu: ion: Fix bug in ion_free
gpu: ion: Add debug information for orphaned handles
gpu: ion: Fix memory leak of dirty bits
gpu: ion: Add support for cached mappings that don't fault
gpu: ion: optimize system heap for non fault buffers
gpu: ion: Stop trying to allocate from an order on first failure
gpu: ion: ion_system_heap: Fix bug preventing compilation
gpu: ion: use vmalloc to allocate page array to map kernel
gpu: ion: Add ion_page_pool.
gpu: ion: Use the ion_page_pool from the system heap
gpu: ion: Modify gfp flags in ion_system_heap
gpu: ion: Fix several issues with page pool
gpu: ion: Fix lockdep issue in ion_page_pool
gpu: ion: Switch to using a single shrink function
gpu: ion: Refactor locking
gpu: ion: Clear GFP_WAIT flag on high order allocations
gpu: ion: Don't flush allocatoins that come from the page pools
gpu: ion: Fix bug in ion_system_heap map_user
gpu: ion: Fix bug in zeroing pages in system heap
gpu: ion: Modify reserve function for carveouts with no start address
gpu: ion: Fix bug where MAP ioctl was no longer supported
gpu: ion: Switch heap rbtree to a prio list
gpu: ion: Refactor common mapping functions out of system heap
gpu: ion: Add chunk heap
gpu: ion: Clarify variable names and comments around heap ids v types
gpu: ion: Remove heapmask from client
gpu: ion: Modify zeroing code so it only allocates address space once
gpu: ion: Refactor the code to zero buffers
gpu: ion: Only flush buffers in the chunk heap if they were used
cached
gpu: ion: Make ion_free asynchronous
gpu: ion: ion_chunk_heap: Zero chunk heap memory at creation time
gpu: ion: Fix bug in ion shrinker
gpu: ion: Also shrink memory cached in the deferred free list
gpu: ion: Fix performance issue in faulting code

Rom Lemarchand (3):
ion: add new ion_user_handle_t type for the user-space token
ion: change ion_user_handle_t definition to int
ion: add compat_ioctl

 drivers/staging/android/Kconfig                 |    2 +
 drivers/staging/android/Makefile                |    2 +
 drivers/staging/android/ion/Kconfig             |   24 +
 drivers/staging/android/ion/Makefile            |    7 +
 drivers/staging/android/ion/compat_ion.c        |  177 +++
 drivers/staging/android/ion/compat_ion.h        |   30 +
 drivers/staging/android/ion/ion.c               | 1549 +++++++++++++++++++++++
 drivers/staging/android/ion/ion.h               |  204 +++
 drivers/staging/android/ion/ion_carveout_heap.c |  194 +++
 drivers/staging/android/ion/ion_chunk_heap.c    |  195 +++
 drivers/staging/android/ion/ion_cma_heap.c      |  218 ++++
 drivers/staging/android/ion/ion_heap.c          |  314 +++++
 drivers/staging/android/ion/ion_page_pool.c     |  195 +++
 drivers/staging/android/ion/ion_priv.h          |  360 ++++++
 drivers/staging/android/ion/ion_system_heap.c   |  483 +++++++
 drivers/staging/android/ion/ion_test.c          |  281 ++++
 drivers/staging/android/ion/tegra/Makefile      |    1 +
 drivers/staging/android/ion/tegra/tegra_ion.c   |   96 ++
 drivers/staging/android/uapi/ion.h              |  196 +++
 drivers/staging/android/uapi/ion_test.h         |   71 ++
 20 files changed, 4599 insertions(+)
 create mode 100644 drivers/staging/android/ion/Kconfig
 create mode 100644 drivers/staging/android/ion/Makefile
 create mode 100644 drivers/staging/android/ion/compat_ion.c
 create mode 100644 drivers/staging/android/ion/compat_ion.h
 create mode 100644 drivers/staging/android/ion/ion.c
 create mode 100644 drivers/staging/android/ion/ion.h
 create mode 100644 drivers/staging/android/ion/ion_carveout_heap.c
 create mode 100644 drivers/staging/android/ion/ion_chunk_heap.c
 create mode 100644 drivers/staging/android/ion/ion_cma_heap.c
 create mode 100644 drivers/staging/android/ion/ion_heap.c
 create mode 100644 drivers/staging/android/ion/ion_page_pool.c
 create mode 100644 drivers/staging/android/ion/ion_priv.h
 create mode 100644 drivers/staging/android/ion/ion_system_heap.c
 create mode 100644 drivers/staging/android/ion/ion_test.c
 create mode 100644 drivers/staging/android/ion/tegra/Makefile
 create mode 100644 drivers/staging/android/ion/tegra/tegra_ion.c
 create mode 100644 drivers/staging/android/uapi/ion.h
 create mode 100644 drivers/staging/android/uapi/ion_test.h
Signature · 1 line
1.8.3.2
#1 lore raw

PATCH 1/115 gpu: ion: Add ION Memory Manager

From: Rebecca Schultz Zavin <[email protected]>

[jstultz: Squished in Colin Cross' move to staging change]
---
 drivers/staging/android/Kconfig                 |    2 +
 drivers/staging/android/Makefile                |    2 +
 drivers/staging/android/ion/Kconfig             |   12 +
 drivers/staging/android/ion/Makefile            |    2 +
 drivers/staging/android/ion/ion.c               | 1187 +++++++++++++++++++++++
 drivers/staging/android/ion/ion.h               |  344 +++++++
 drivers/staging/android/ion/ion_carveout_heap.c |  162 ++++
 drivers/staging/android/ion/ion_heap.c          |   72 ++
 drivers/staging/android/ion/ion_priv.h          |  185 ++++
 drivers/staging/android/ion/ion_system_heap.c   |  198 ++++
 drivers/staging/android/ion/ion_system_mapper.c |  114 +++
 drivers/staging/android/ion/tegra/Makefile      |    1 +
 drivers/staging/android/ion/tegra/tegra_ion.c   |   96 ++
 13 files changed, 2377 insertions(+)
 create mode 100644 drivers/staging/android/ion/Kconfig
 create mode 100644 drivers/staging/android/ion/Makefile
 create mode 100644 drivers/staging/android/ion/ion.c
 create mode 100644 drivers/staging/android/ion/ion.h
 create mode 100644 drivers/staging/android/ion/ion_carveout_heap.c
 create mode 100644 drivers/staging/android/ion/ion_heap.c
 create mode 100644 drivers/staging/android/ion/ion_priv.h
 create mode 100644 drivers/staging/android/ion/ion_system_heap.c
 create mode 100644 drivers/staging/android/ion/ion_system_mapper.c
 create mode 100644 drivers/staging/android/ion/tegra/Makefile
 create mode 100644 drivers/staging/android/ion/tegra/tegra_ion.c
drivers/staging/android/Kconfig +2 -0
index 1e9ab6d..b91c758 100644
@@ -100,6 +100,8 @@ config SW_SYNC_USER
 	  *WARNING* improper use of this can result in deadlocking kernel
 	  drivers from userspace.
 
+source "drivers/staging/android/ion/Kconfig"
+
 endif # if ANDROID
 
 endmenu
drivers/staging/android/Makefile +2 -0
index c136299..0a01e191 100644
@@ -1,5 +1,7 @@
 ccflags-y += -I$(src)			# needed for trace events
 
+obj-y					+= ion/
+
 obj-$(CONFIG_ANDROID_BINDER_IPC)	+= binder.o
 obj-$(CONFIG_ASHMEM)			+= ashmem.o
 obj-$(CONFIG_ANDROID_LOGGER)		+= logger.o
drivers/staging/android/ion/Kconfig new file +12 -0
new file mode 100644
index 0000000..5b48b4e
@@ -0,0 +1,12 @@
+menuconfig ION
+	tristate "Ion Memory Manager"
+	select GENERIC_ALLOCATOR
+	help
+	  Chose this option to enable the ION Memory Manager.
+
+config ION_TEGRA
+	tristate "Ion for Tegra"
+	depends on ARCH_TEGRA && ION
+	help
+	  Choose this option if you wish to use ion on an nVidia Tegra.
+
drivers/staging/android/ion/Makefile new file +2 -0
new file mode 100644
index 0000000..73fe3fa
@@ -0,0 +1,2 @@
+obj-$(CONFIG_ION) +=	ion.o ion_heap.o ion_system_heap.o ion_carveout_heap.o
+obj-$(CONFIG_ION_TEGRA) += tegra/
drivers/staging/android/ion/ion.c new file +1187 -0

1,187 lines not on this page · raw message

drivers/staging/android/ion/ion.h new file +344 -0

344 lines not on this page · raw message

drivers/staging/android/ion/ion_carveout_heap.c new file +162 -0

162 lines not on this page · raw message

drivers/staging/android/ion/ion_heap.c new file +72 -0

72 lines not on this page · raw message

drivers/staging/android/ion/ion_priv.h new file +185 -0

185 lines not on this page · raw message

drivers/staging/android/ion/ion_system_heap.c new file +198 -0

198 lines not on this page · raw message

drivers/staging/android/ion/ion_system_mapper.c new file +114 -0

114 lines not on this page · raw message

drivers/staging/android/ion/tegra/Makefile new file +1 -0
new file mode 100644
index 0000000..11cd003
@@ -0,0 +1 @@
+obj-y += tegra_ion.o
drivers/staging/android/ion/tegra/tegra_ion.c new file +96 -0

96 lines not on this page · raw message

Signature · 1 line
1.8.3.2
maintainer
#2 lore raw

Re: [PATCH 001/115] gpu: ion: Add ION Memory Manager

Quoted 8 lines · John Stultz

From: Rebecca Schultz Zavin <[email protected]>

[jstultz: Squished in Colin Cross' move to staging change]
---
 drivers/staging/android/Kconfig                 |    2 +
 drivers/staging/android/Makefile                |    2 +

This patch breaks the build if this option is enabled.

I suggest only adding the driver to the build _after_ it can be built, for obvious reasons :)

Other than this "minor" thing, I have no objection to these patches at all. Can you redo this patch to fix this up, and maybe send another patch adding it to the build and let me know at what point in the series it should be applied?

Also, I only received 92 of these patches, not all 115, are they stuck somewhere in a mail queue?

thanks,

greg k-h

#3 lore raw

Re: [PATCH 001/115] gpu: ion: Add ION Memory Manager

Quoted 9 lines · Greg KH
Quoted 8 lines · John Stultz

From: Rebecca Schultz Zavin <[email protected]>

[jstultz: Squished in Colin Cross' move to staging change]
---
 drivers/staging/android/Kconfig                 |    2 +
 drivers/staging/android/Makefile                |    2 +
Quoted 4 lines

This patch breaks the build if this option is enabled.

I suggest only adding the driver to the build _after_ it can be built, for obvious reasons :)

Right. Until recently it really only builds on ARM. There's a patch which adds that dependency (and then later removes it once other fixes land), but its later in the patch series.

Apologies.

Quoted 4 lines

Other than this "minor" thing, I have no objection to these patches at all. Can you redo this patch to fix this up, and maybe send another patch adding it to the build and let me know at what point in the series it should be applied?

I'll squish that config dependency down to the first patch and go through verifying it builds.

Quoted 2 lines

Also, I only received 92 of these patches, not all 115, are they stuck somewhere in a mail queue?

Maybe? I hope! I've never played GregKH and sent such a large patchset before!

thanks
-john

#4 lore raw

Re: [PATCH 001/115] gpu: ion: Add ION Memory Manager

Quoted 10 lines · John Stultz
Quoted 9 lines · Greg KH
Quoted 8 lines · John Stultz

From: Rebecca Schultz Zavin <[email protected]>

[jstultz: Squished in Colin Cross' move to staging change]
---
 drivers/staging/android/Kconfig                 |    2 +
 drivers/staging/android/Makefile                |    2 +
Quoted 7 lines
Quoted 4 lines

This patch breaks the build if this option is enabled.

I suggest only adding the driver to the build _after_ it can be built, for obvious reasons :)

Right. Until recently it really only builds on ARM. There's a patch which adds that dependency (and then later removes it once other fixes land), but its later in the patch series.

Ah. Forgot about the shrinker interface changes (which I fix at the very end).

So yea, I'll disable the config in the first patch ("depends on BROKEN" good enough? or would you prefer "depends on BROKEN && !BROKEN" to be really sure? ;), and then yank that line in the last.

thanks
-john

#5 lore raw

Re: [PATCH 001/115] gpu: ion: Add ION Memory Manager

In reply to Greg KH
Quoted 11 lines · Greg KH
Quoted 8 lines · John Stultz

From: Rebecca Schultz Zavin <[email protected]>

[jstultz: Squished in Colin Cross' move to staging change]
---
 drivers/staging/android/Kconfig                 |    2 +
 drivers/staging/android/Makefile                |    2 +

This patch breaks the build if this option is enabled.

Quoted 7 lines

I suggest only adding the driver to the build _after_ it can be built, for obvious reasons :)

Other than this "minor" thing, I have no objection to these patches at all. Can you redo this patch to fix this up, and maybe send another patch adding it to the build and let me know at what point in the series it should be applied?

Ok. Two patches attached. I ended up just yanking the build directory since disabling the config option caused trivial collisions in the series.

0001-ion-Disable-ION-as-it-doesn-t-build.patch - This can be folded in after the first patch.

0117-ion-Reenable-the-build.patch - This can be applied to the end of the series, or folded into the last patch "ion: Update system heap shrinker to use the new count/scan interface"

Let me know if you'd rather this be done some other way. Also let me know if you've still not seen the other patches in the series.

thanks
-john

Quoted 1 line

From eba11fdcb50d53906f1b159969448803178c173f Mon Sep 17 00:00:00 2001

From: John Stultz <[email protected]>
Date: Fri, 13 Dec 2013 16:56:53 -0800
Subject: [PATCH] ion: Disable ION, as it doesn't build

Since the ion patches are forward ported from 3.10, they won't build. So disable it from the build for now and we'll re-enable it after things are building again.

---
 drivers/staging/android/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/staging/android/Makefile +2 -1
index 0a01e191..29c5ff0 100644
@@ -1,6 +1,7 @@
 ccflags-y += -I$(src)			# needed for trace events
 
-obj-y					+= ion/
+# ION doesn't build just yet, so disable it from the build
+#obj-y					+= ion/
 
 obj-$(CONFIG_ANDROID_BINDER_IPC)	+= binder.o
 obj-$(CONFIG_ASHMEM)			+= ashmem.o
Signature · 1 line
1.8.3.2
Quoted 1 line

From 60a715a53406a68c399dcf706bdb33c83e9c42ff Mon Sep 17 00:00:00 2001

From: John Stultz <[email protected]>
Date: Fri, 13 Dec 2013 17:12:18 -0800
Subject: [PATCH] ion: Reenable the build

Now that ION builds, reenable it in the build.

---
 drivers/staging/android/Makefile | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/staging/android/Makefile +1 -2
index 29c5ff0..0a01e191 100644
@@ -1,7 +1,6 @@
 ccflags-y += -I$(src)			# needed for trace events
 
-# ION doesn't build just yet, so disable it from the build
-#obj-y					+= ion/
+obj-y					+= ion/
 
 obj-$(CONFIG_ANDROID_BINDER_IPC)	+= binder.o
 obj-$(CONFIG_ASHMEM)			+= ashmem.o
Signature · 1 line
1.8.3.2
maintainer
#6 lore raw

Re: [PATCH 001/115] gpu: ion: Add ION Memory Manager

Quoted 3 lines · John Stultz

0117-ion-Reenable-the-build.patch - This can be applied to the end of the series, or folded into the last patch "ion: Update system heap shrinker to use the new count/scan interface"

Was there a 0116 patch somewhere?

Everything works fine until this one (i.e. the driver doesn't build), but after applying this patch, the build breaks with a ton of undefined symbols at link time, here's a sample of some of them:

ERROR: "ion_carveout_heap_destroy" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_chunk_heap_destroy" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_system_heap_create" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_carveout_heap_create" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_system_contig_heap_create" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_cma_heap_create" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_chunk_heap_create" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_buffer_destroy" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_heap_map_user" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_heap_unmap_kernel" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_heap_map_kernel" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_heap_pages_zero" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_pages_sync_for_device" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_buffer_cached" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_heap_buffer_zero" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!

So I'm not going to apply this patch, but I have the 115 previous patches (I took your modified 001 patch that you sent off-list to me.)

Care to fix up the symbol issues in a new series of patches and then enabling the driver to build?

thanks,

greg k-h

#7 lore raw

Re: [PATCH 001/115] gpu: ion: Add ION Memory Manager

Quoted 5 lines · Greg KH
Quoted 3 lines · John Stultz

0117-ion-Reenable-the-build.patch - This can be applied to the end of the series, or folded into the last patch "ion: Update system heap shrinker to use the new count/scan interface"

Was there a 0116 patch somewhere?

No.. the disable build patch was inserted between patch 1 and 2, so after the renumbering the re-enable patch was 117.

When I selected the disable patch to send I renamed it 0001 just to make it more clear which one it would apply to.

Sorry for the confusion.

Quoted 20 lines

Everything works fine until this one (i.e. the driver doesn't build), but after applying this patch, the build breaks with a ton of undefined symbols at link time, here's a sample of some of them:

ERROR: "ion_carveout_heap_destroy" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_chunk_heap_destroy" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_system_heap_create" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_carveout_heap_create" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_system_contig_heap_create" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_cma_heap_create" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_chunk_heap_create" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_buffer_destroy" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_heap_map_user" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_heap_unmap_kernel" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_heap_map_kernel" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_heap_pages_zero" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_pages_sync_for_device" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_buffer_cached" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_heap_buffer_zero" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!

Quoted 2 lines

So I'm not going to apply this patch, but I have the 115 previous patches (I took your modified 001 patch that you sent off-list to me.)

? Hrmm. I'm not seeing this (I wouldn't have sent the patches if I did!).

If you have the chance, could you send me your config?

Quoted 2 lines

Care to fix up the symbol issues in a new series of patches and then enabling the driver to build?

Will do.

thanks
-john

#8 lore raw

Re: [PATCH 001/115] gpu: ion: Add ION Memory Manager

Quoted 21 lines · John Stultz
Quoted 20 lines · Greg KH

Everything works fine until this one (i.e. the driver doesn't build), but after applying this patch, the build breaks with a ton of undefined symbols at link time, here's a sample of some of them:

ERROR: "ion_carveout_heap_destroy" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_chunk_heap_destroy" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_system_heap_create" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_carveout_heap_create" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_system_contig_heap_create" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_cma_heap_create" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_chunk_heap_create" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_buffer_destroy" [drivers/staging/android/ion/ion_heap.ko] undefined!
ERROR: "ion_heap_map_user" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_heap_unmap_kernel" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_heap_map_kernel" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_heap_pages_zero" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_pages_sync_for_device" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_buffer_cached" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!
ERROR: "ion_heap_buffer_zero" [drivers/staging/android/ion/ion_chunk_heap.ko] undefined!

Quoted 3 lines
Quoted 2 lines

So I'm not going to apply this patch, but I have the 115 previous patches (I took your modified 001 patch that you sent off-list to me.)

? Hrmm. I'm not seeing this (I wouldn't have sent the patches if I did!).

Ah.. its a problem with make modules. I see it now. Will have a fix for you shortly.

thanks
-john

#9 lore raw

PATCH 1/2 ion: Don't allow building ION as a module.

In reply to Greg KH

ION doesn't export the proper symbols for it to be a module. This causes build issues when ION is configured as a module.

Since Andorid kernels rarely use modules (I think recent policy requires no modules?), go ahead and set the ION config to a bool from the tristate option.

If folks decide ION as a module is important, we will have to go through and export the various needed symbols.

---
 drivers/staging/android/ion/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/android/ion/Kconfig +1 -1
index a342d96..b95281e 100644
@@ -1,5 +1,5 @@
 menuconfig ION
-	tristate "Ion Memory Manager"
+	bool "Ion Memory Manager"
 	select GENERIC_ALLOCATOR
 	select DMA_SHARED_BUFFER
 	---help---
Signature · 1 line
1.8.3.2
#10 lore raw

PATCH 2/2 ion: Reenable the build

Now that ION builds, reenable it in the build.

---
 drivers/staging/android/Makefile | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/staging/android/Makefile +1 -2
index 29c5ff0..0a01e191 100644
@@ -1,7 +1,6 @@
 ccflags-y += -I$(src)			# needed for trace events
 
-# ION doesn't build just yet, so disable it from the build
-#obj-y					+= ion/
+obj-y					+= ion/
 
 obj-$(CONFIG_ANDROID_BINDER_IPC)	+= binder.o
 obj-$(CONFIG_ASHMEM)			+= ashmem.o
Signature · 1 line
1.8.3.2
maintainer
#11 lore raw

Re: [PATCH 1/2] ion: Don't allow building ION as a module.

In reply to John Stultz
Quoted 20 lines · John Stultz

ION doesn't export the proper symbols for it to be a module. This causes build issues when ION is configured as a module.

Since Andorid kernels rarely use modules (I think recent policy requires no modules?), go ahead and set the ION config to a bool from the tristate option.

If folks decide ION as a module is important, we will have to go through and export the various needed symbols.

---
 drivers/staging/android/ion/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/android/ion/Kconfig +0 -0
index a342d96..b95281e 100644

drivers/staging/android/ion/Kconfig line 2

Quoted 3 lines
drivers/staging/android/ion/Kconfig +1 -1
@@ -1,5 +1,5 @@
 menuconfig ION
-	tristate "Ion Memory Manager"
+	bool "Ion Memory Manager"

Or you can do:
depends on m
and leave it as a "tristate", then when switching it off, I think you can then be prompted as 'm' again, but I could be wrong about that.

So should ION ever be able to be built as a module in the future?

thanks,

greg k-h

#12 lore raw

Re: [PATCH 1/2] ion: Don't allow building ION as a module.

Quoted 29 lines · Greg KH
Quoted 20 lines · John Stultz

ION doesn't export the proper symbols for it to be a module. This causes build issues when ION is configured as a module.

Since Andorid kernels rarely use modules (I think recent policy requires no modules?), go ahead and set the ION config to a bool from the tristate option.

If folks decide ION as a module is important, we will have to go through and export the various needed symbols.

---
 drivers/staging/android/ion/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/android/ion/Kconfig +0 -0
index a342d96..b95281e 100644

drivers/staging/android/ion/Kconfig line 2

Quoted 3 lines
drivers/staging/android/ion/Kconfig +1 -1
@@ -1,5 +1,5 @@
 menuconfig ION
-	tristate "Ion Memory Manager"
+	bool "Ion Memory Manager"

Or you can do:
depends on m
and leave it as a "tristate", then when switching it off, I think you can then be prompted as 'm' again, but I could be wrong about that.

Quoted 1 line

So should ION ever be able to be built as a module in the future?

My understanding is that Android is moving away from using modules, but others could comment more authoritatively. So I don't think this is an issue. But as I said in the patch, if folks do want to have ION as a module, we'll have to export all the various symbols that ION users need.

thanks
-john

#13 lore raw

Re: [PATCH 1/2] ion: Don't allow building ION as a module.

Quoted 30 lines · John Stultz
Quoted 29 lines · Greg KH
Quoted 20 lines · John Stultz

ION doesn't export the proper symbols for it to be a module. This causes build issues when ION is configured as a module.

Since Andorid kernels rarely use modules (I think recent policy requires no modules?), go ahead and set the ION config to a bool from the tristate option.

If folks decide ION as a module is important, we will have to go through and export the various needed symbols.

---
 drivers/staging/android/ion/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/android/ion/Kconfig +0 -0
index a342d96..b95281e 100644

drivers/staging/android/ion/Kconfig line 2

Quoted 3 lines
drivers/staging/android/ion/Kconfig +1 -1
@@ -1,5 +1,5 @@
 menuconfig ION
-    tristate "Ion Memory Manager"
+    bool "Ion Memory Manager"

Or you can do:
depends on m
and leave it as a "tristate", then when switching it off, I think you can then be prompted as 'm' again, but I could be wrong about that.

Quoted 5 lines
Quoted 1 line

So should ION ever be able to be built as a module in the future?

My understanding is that Android is moving away from using modules, but others could comment more authoritatively. So I don't think this is an issue. But as I said in the patch, if folks do want to have ION as a module, we'll have to export all the various symbols that ION users need.

There's no reason ION couldn't be built as a module, it just doesn't make much sense given how central it is to the Android devices it is used on. It will need some TODOs fixed in the removal paths, and probably some refcounting to prevent removing the module while buffers are allocated.

As for modules in general on Android, they are rarely used on ARM Android devices and were considered a security risk. I don't think anybody would have a problem re-enabling them if CONFIG_MODULE_SIG_FORCE was also enabled.

maintainer
#14 lore raw

Re: [PATCH 001/115] gpu: ion: Add ION Memory Manager

In reply to John Stultz

drivers/staging/android/ion/ion.c line 1171

Quoted 3 lines · John Stultz
drivers/staging/android/ion/ion.c new file +3 -0
+	idev->debug_root = debugfs_create_dir("ion", NULL);
+	if (IS_ERR_OR_NULL(idev->debug_root))
+		pr_err("ion: failed to create debug files.\n");

There's no need to check for this at all, you just printed out an error message if CONFIG_DEBUGFS is not enabled, which isn't all that nice.

Something to clean up later, no big deal now.

thanks,

greg k-h

#15 lore raw

Re: [PATCH 001/115] gpu: ion: Add ION Memory Manager

Quoted 9 lines · Greg KH

drivers/staging/android/ion/ion.c line 1171

Quoted 3 lines · John Stultz
drivers/staging/android/ion/ion.c new file +3 -0
+     idev->debug_root = debugfs_create_dir("ion", NULL);
+     if (IS_ERR_OR_NULL(idev->debug_root))
+             pr_err("ion: failed to create debug files.\n");

There's no need to check for this at all, you just printed out an error message if CONFIG_DEBUGFS is not enabled, which isn't all that nice.

Something to clean up later, no big deal now.

Patch 75/115 (ion: remove IS_ERR_OR_NULL) replaces this chunk with:

        idev->debug_root = debugfs_create_dir("ion", NULL);
        if (!idev->debug_root)
                pr_err("ion: failed to create debug files.\n");

which won't print an error if debugfs is disabled.

maintainer
#16 lore raw

Re: [PATCH 001/115] gpu: ion: Add ION Memory Manager

Quoted 9 lines · Colin Cross
Quoted 8 lines · Greg KH

drivers/staging/android/ion/ion.c line 1171

Quoted 3 lines · John Stultz
drivers/staging/android/ion/ion.c new file +3 -0
+     idev->debug_root = debugfs_create_dir("ion", NULL);
+     if (IS_ERR_OR_NULL(idev->debug_root))
+             pr_err("ion: failed to create debug files.\n");

There's no need to check for this at all, you just printed out an error message if CONFIG_DEBUGFS is not enabled, which isn't all that nice.

Quoted 7 lines
Quoted 1 line

Something to clean up later, no big deal now.

Patch 75/115 (ion: remove IS_ERR_OR_NULL) replaces this chunk with:

        idev->debug_root = debugfs_create_dir("ion", NULL);
        if (!idev->debug_root)
                pr_err("ion: failed to create debug files.\n");

which won't print an error if debugfs is disabled.

If debugfs is disabled, debugfs_create_dir() returns ERR_PTR(-ENODEV).

So are you sure you tested this? :)

#17 lore raw

Re: [PATCH 001/115] gpu: ion: Add ION Memory Manager

Quoted 12 lines · Greg KH
Quoted 11 lines · Colin Cross
Quoted 9 lines · Greg KH

drivers/staging/android/ion/ion.c line 1171

Quoted 3 lines · John Stultz
drivers/staging/android/ion/ion.c new file +3 -0
+     idev->debug_root = debugfs_create_dir("ion", NULL);
+     if (IS_ERR_OR_NULL(idev->debug_root))
+             pr_err("ion: failed to create debug files.\n");

There's no need to check for this at all, you just printed out an error message if CONFIG_DEBUGFS is not enabled, which isn't all that nice.

Something to clean up later, no big deal now.

Quoted 7 lines
Quoted 5 lines

Patch 75/115 (ion: remove IS_ERR_OR_NULL) replaces this chunk with:

        idev->debug_root = debugfs_create_dir("ion", NULL);
        if (!idev->debug_root)
                pr_err("ion: failed to create debug files.\n");

which won't print an error if debugfs is disabled.

If debugfs is disabled, debugfs_create_dir() returns ERR_PTR(-ENODEV).

ERR_PTR(-ENODEV) is not NULL, so this won't print an error. I thought that was the point of returning ERR_PTR when disabled and NULL when enabled.

Quoted 1 line

So are you sure you tested this? :)

I have not tested this, but it seems correct to me.

maintainer
#18 lore raw

Re: [PATCH 001/115] gpu: ion: Add ION Memory Manager

Quoted 19 lines · Colin Cross
Quoted 18 lines · Greg KH
Quoted 16 lines · Colin Cross
Quoted 9 lines · Greg KH

drivers/staging/android/ion/ion.c line 1171

Quoted 3 lines · John Stultz
drivers/staging/android/ion/ion.c new file +3 -0
+     idev->debug_root = debugfs_create_dir("ion", NULL);
+     if (IS_ERR_OR_NULL(idev->debug_root))
+             pr_err("ion: failed to create debug files.\n");

There's no need to check for this at all, you just printed out an error message if CONFIG_DEBUGFS is not enabled, which isn't all that nice.

Something to clean up later, no big deal now.

Patch 75/115 (ion: remove IS_ERR_OR_NULL) replaces this chunk with:

        idev->debug_root = debugfs_create_dir("ion", NULL);
        if (!idev->debug_root)
                pr_err("ion: failed to create debug files.\n");

which won't print an error if debugfs is disabled.

Quoted 5 lines
Quoted 1 line

If debugfs is disabled, debugfs_create_dir() returns ERR_PTR(-ENODEV).

ERR_PTR(-ENODEV) is not NULL, so this won't print an error. I thought that was the point of returning ERR_PTR when disabled and NULL when enabled.

{sigh}

Yes, you are right, I blame the jetlag, sorry.

greg k-h

#19 lore raw

PATCH 2/115 gpu: ion: ion_carveout_heap: fix for 3.4

In reply to John Stultz

From: Colin Cross <[email protected]>

__arch_ioremap is no longer available, use __arm_ioremap instead.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_carveout_heap.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/staging/android/ion/ion_carveout_heap.c +2 -2
index 6d02cd3..e8c4e36 100644
@@ -99,14 +99,14 @@ void ion_carveout_heap_unmap_dma(struct ion_heap *heap,
 void *ion_carveout_heap_map_kernel(struct ion_heap *heap,
 				   struct ion_buffer *buffer)
 {
-	return __arch_ioremap(buffer->priv_phys, buffer->size,
+	return __arm_ioremap(buffer->priv_phys, buffer->size,
 			      MT_MEMORY_NONCACHED);
 }
 
 void ion_carveout_heap_unmap_kernel(struct ion_heap *heap,
 				    struct ion_buffer *buffer)
 {
-	__arch_iounmap(buffer->vaddr);
+	__arm_iounmap(buffer->vaddr);
 	buffer->vaddr = NULL;
 	return;
 }
Signature · 1 line
1.8.3.2
#20 lore raw

PATCH 3/115 ion: Switch map/unmap dma api to sg_tables

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

Switch these api's from scatterlists to sg_tables

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c             | 16 +++----
 drivers/staging/android/ion/ion.h             |  6 +--
 drivers/staging/android/ion/ion_priv.h        | 18 ++------
 drivers/staging/android/ion/ion_system_heap.c | 66 ++++++++++++++++-----------
 4 files changed, 53 insertions(+), 53 deletions(-)
drivers/staging/android/ion/ion.c +8 -8
index d93fb33..bca8d9e 100644
@@ -445,11 +445,11 @@ void *ion_map_kernel(struct ion_client *client, struct ion_handle *handle)
 	return vaddr;
 }
 
-struct scatterlist *ion_map_dma(struct ion_client *client,
+struct sg_table *ion_map_dma(struct ion_client *client,
 				struct ion_handle *handle)
 {
 	struct ion_buffer *buffer;
-	struct scatterlist *sglist;
+	struct sg_table *table;
 
 	mutex_lock(&client->lock);
 	if (!ion_handle_validate(client, handle)) {
@@ -469,16 +469,16 @@ struct scatterlist *ion_map_dma(struct ion_client *client,
 		return ERR_PTR(-ENODEV);
 	}
 	if (_ion_map(&buffer->dmap_cnt, &handle->dmap_cnt)) {
-		sglist = buffer->heap->ops->map_dma(buffer->heap, buffer);
-		if (IS_ERR_OR_NULL(sglist))
+		table = buffer->heap->ops->map_dma(buffer->heap, buffer);
+		if (IS_ERR_OR_NULL(table))
 			_ion_unmap(&buffer->dmap_cnt, &handle->dmap_cnt);
-		buffer->sglist = sglist;
+		buffer->sg_table = table;
 	} else {
-		sglist = buffer->sglist;
+		table = buffer->sg_table;
 	}
 	mutex_unlock(&buffer->lock);
 	mutex_unlock(&client->lock);
-	return sglist;
+	return table;
 }
 
 void ion_unmap_kernel(struct ion_client *client, struct ion_handle *handle)
@@ -505,7 +505,7 @@ void ion_unmap_dma(struct ion_client *client, struct ion_handle *handle)
 	mutex_lock(&buffer->lock);
 	if (_ion_unmap(&buffer->dmap_cnt, &handle->dmap_cnt)) {
 		buffer->heap->ops->unmap_dma(buffer->heap, buffer);
-		buffer->sglist = NULL;
+		buffer->sg_table = NULL;
 	}
 	mutex_unlock(&buffer->lock);
 	mutex_unlock(&client->lock);
drivers/staging/android/ion/ion.h +3 -3
index 60b2543..87f6a45 100644
@@ -169,10 +169,10 @@ void ion_unmap_kernel(struct ion_client *client, struct ion_handle *handle);
  * @client:	the client
  * @handle:	handle to map
  *
- * Return an sglist describing the given handle
+ * Return an sg_table describing the given handle
  */
-struct scatterlist *ion_map_dma(struct ion_client *client,
-				struct ion_handle *handle);
+struct sg_table *ion_map_dma(struct ion_client *client,
+			     struct ion_handle *handle);
 
 /**
  * ion_unmap_dma() - destroy a dma mapping for a handle
drivers/staging/android/ion/ion_priv.h +3 -15
index cbd9f03..7a77f6f 100644
@@ -24,18 +24,6 @@
 
 #include "ion.h"
 
-struct ion_mapping;
-
-struct ion_dma_mapping {
-	struct kref ref;
-	struct scatterlist *sglist;
-};
-
-struct ion_kernel_mapping {
-	struct kref ref;
-	void *vaddr;
-};
-
 struct ion_buffer *ion_handle_buffer(struct ion_handle *handle);
 
 /**
@@ -54,7 +42,7 @@ struct ion_buffer *ion_handle_buffer(struct ion_handle *handle);
  * @kmap_cnt:		number of times the buffer is mapped to the kernel
  * @vaddr:		the kenrel mapping if kmap_cnt is not zero
  * @dmap_cnt:		number of times the buffer is mapped for dma
- * @sglist:		the scatterlist for the buffer is dmap_cnt is not zero
+ * @sg_table:		the sg table for the buffer if dmap_cnt is not zero
 */
 struct ion_buffer {
 	struct kref ref;
@@ -71,7 +59,7 @@ struct ion_buffer {
 	int kmap_cnt;
 	void *vaddr;
 	int dmap_cnt;
-	struct scatterlist *sglist;
+	struct sg_table *sg_table;
 };
 
 /**
@@ -93,7 +81,7 @@ struct ion_heap_ops {
 	void (*free) (struct ion_buffer *buffer);
 	int (*phys) (struct ion_heap *heap, struct ion_buffer *buffer,
 		     ion_phys_addr_t *addr, size_t *len);
-	struct scatterlist *(*map_dma) (struct ion_heap *heap,
+	struct sg_table *(*map_dma) (struct ion_heap *heap,
 					struct ion_buffer *buffer);
 	void (*unmap_dma) (struct ion_heap *heap, struct ion_buffer *buffer);
 	void * (*map_kernel) (struct ion_heap *heap, struct ion_buffer *buffer);
drivers/staging/android/ion/ion_system_heap.c +39 -27

96 lines not on this page · raw message

Signature · 1 line
1.8.3.2
#21 lore raw

PATCH 4/115 ion: Add reserve function to ion

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

Rather than requiring each platform call memblock_remove or reserve from the board file, add this to ion

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 17 +++++++++++++++++
 drivers/staging/android/ion/ion.h | 11 +++++++++++
 2 files changed, 28 insertions(+)
drivers/staging/android/ion/ion.c +17 -0
index bca8d9e..004c0d4 100644
@@ -19,6 +19,7 @@
 #include <linux/fs.h>
 #include <linux/anon_inodes.h>
 #include <linux/list.h>
+#include <linux/memblock.h>
 #include <linux/miscdevice.h>
 #include <linux/export.h>
 #include <linux/mm.h>
@@ -1185,3 +1186,19 @@ void ion_device_destroy(struct ion_device *dev)
 	/* XXX need to free the heaps and clients ? */
 	kfree(dev);
 }
+
+void __init ion_reserve(struct ion_platform_data *data)
+{
+	int i, ret;
+
+	for (i = 0; i < data->nr; i++) {
+		if (data->heaps[i].size == 0)
+			continue;
+		ret = memblock_reserve(data->heaps[i].base,
+				       data->heaps[i].size);
+		if (ret)
+			pr_err("memblock reserve of %x@%lx failed\n",
+			       data->heaps[i].size,
+			       data->heaps[i].base);
+	}
+}
drivers/staging/android/ion/ion.h +11 -0
index 87f6a45..4e012b8 100644
@@ -87,6 +87,17 @@ struct ion_platform_data {
 };
 
 /**
+ * ion_reserve() - reserve memory for ion heaps if applicable
+ * @data:	platform data specifying starting physical address and
+ *		size
+ *
+ * Calls memblock reserve to set aside memory for heaps that are
+ * located at specific memory addresses or of specfic sizes not
+ * managed by the kernel
+ */
+void ion_reserve(struct ion_platform_data *data);
+
+/**
  * ion_client_create() -  allocate a client and returns it
  * @dev:	the global ion device
  * @heap_mask:	mask of heaps this client can allocate from
Signature · 1 line
1.8.3.2
#22 lore raw

PATCH 5/115 gpu: ion: several bugfixes and enhancements of ION

In reply to John Stultz

From: KyongHo Cho <[email protected]>

1. Verifying if the size of memory allocation in ion_alloc() is aligned by PAGE_SIZE at least. If it is not, this change makes the size to be aligned by PAGE_SIZE.

2. Unmaps all mappings to the kernel and DMA address spaces when destroying ion_buffer in ion_buffer_destroy(). This prevents leaks in those virtual address spaces.

3. Makes the return value of ion_alloc() to be explicit Linux error code when it fails to allocate a buffer.

4. Makes ion_alloc() implementation simpler. Removes 'goto' statement and relavant call to ion_buffer_put().

5. Checks if the task is valid before calling put_task_struct() due to failure on creating a ion client in ion_client_create().

6. Returns error when buffer allocation requested by userspace is failed.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 52 +++++++++++++++++++++++++++++----------
 1 file changed, 39 insertions(+), 13 deletions(-)
drivers/staging/android/ion/ion.c +39 -13
index 004c0d4..3c9ca31 100644
@@ -166,6 +166,12 @@ static void ion_buffer_destroy(struct kref *kref)
 	struct ion_buffer *buffer = container_of(kref, struct ion_buffer, ref);
 	struct ion_device *dev = buffer->dev;
 
+	if (WARN_ON(buffer->kmap_cnt > 0))
+		buffer->heap->ops->unmap_kernel(buffer->heap, buffer);
+
+	if (WARN_ON(buffer->dmap_cnt > 0))
+		buffer->heap->ops->unmap_dma(buffer->heap, buffer);
+
 	buffer->heap->ops->free(buffer);
 	mutex_lock(&dev->lock);
 	rb_erase(&buffer->node, &dev->buffers);
@@ -296,6 +302,11 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 	 * request of the caller allocate from it.  Repeat until allocate has
 	 * succeeded or all heaps have been tried
 	 */
+	if (WARN_ON(!len))
+		return ERR_PTR(-EINVAL);
+
+	len = PAGE_ALIGN(len);
+
 	mutex_lock(&dev->lock);
 	for (n = rb_first(&dev->heaps); n != NULL; n = rb_next(n)) {
 		struct ion_heap *heap = rb_entry(n, struct ion_heap, node);
@@ -311,27 +322,26 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 	}
 	mutex_unlock(&dev->lock);
 
-	if (IS_ERR_OR_NULL(buffer))
+	if (buffer == NULL)
+		return ERR_PTR(-ENODEV);
+
+	if (IS_ERR(buffer))
 		return ERR_PTR(PTR_ERR(buffer));
 
 	handle = ion_handle_create(client, buffer);
 
-	if (IS_ERR_OR_NULL(handle))
-		goto end;
-
 	/*
 	 * ion_buffer_create will create a buffer with a ref_cnt of 1,
 	 * and ion_handle_create will take a second reference, drop one here
 	 */
 	ion_buffer_put(buffer);
 
-	mutex_lock(&client->lock);
-	ion_handle_add(client, handle);
-	mutex_unlock(&client->lock);
-	return handle;
+	if (!IS_ERR(handle)) {
+		mutex_lock(&client->lock);
+		ion_handle_add(client, handle);
+		mutex_unlock(&client->lock);
+	}
 
-end:
-	ion_buffer_put(buffer);
 	return handle;
 }
 
@@ -680,7 +690,8 @@ struct ion_client *ion_client_create(struct ion_device *dev,
 
 	client = kzalloc(sizeof(struct ion_client), GFP_KERNEL);
 	if (!client) {
-		put_task_struct(current->group_leader);
+		if (task)
+			put_task_struct(current->group_leader);
 		return ERR_PTR(-ENOMEM);
 	}
 
@@ -798,6 +809,15 @@ static void ion_vma_open(struct vm_area_struct *vma)
 		vma->vm_private_data = NULL;
 		return;
 	}
+
+	if (!ion_handle_validate(client, handle)) {
+		ion_client_put(client);
+		vma->vm_private_data = NULL;
+		return;
+	}
+
+	ion_handle_get(handle);
+
 	pr_debug("%s: %d client_cnt %d handle_cnt %d alloc_cnt %d\n",
 		 __func__, __LINE__,
 		 atomic_read(&client->ref.refcount),
@@ -919,7 +939,7 @@ static int ion_ioctl_share(struct file *parent, struct ion_client *client,
 	struct file *file;
 
 	if (fd < 0)
-		return -ENFILE;
+		return fd;
 
 	file = anon_inode_getfile("ion_share_fd", &ion_share_fops,
 				  handle->buffer, O_RDWR);
@@ -948,8 +968,14 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 			return -EFAULT;
 		data.handle = ion_alloc(client, data.len, data.align,
 					     data.flags);
-		if (copy_to_user((void __user *)arg, &data, sizeof(data)))
+
+		if (IS_ERR(data.handle))
+			return PTR_ERR(data.handle);
+
+		if (copy_to_user((void __user *)arg, &data, sizeof(data))) {
+			ion_free(client, data.handle);
 			return -EFAULT;
+		}
 		break;
 	}
 	case ION_IOC_FREE:
Signature · 1 line
1.8.3.2
#23 lore raw

PATCH 6/115 ion: Switch ion to use dma-buf

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

Ion now uses dma-buf file descriptors to share buffers with userspace. Ion becomes a dma-buf exporter and any driver that can import dma-bufs can now import ion file descriptors.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/Kconfig |   1 +
 drivers/staging/android/ion/ion.c   | 648 +++++++++++++-----------------------
 drivers/staging/android/ion/ion.h   |  58 +---
 3 files changed, 236 insertions(+), 471 deletions(-)
drivers/staging/android/ion/Kconfig +1 -0
index 5b48b4e..b5bfdb4 100644
@@ -1,6 +1,7 @@
 menuconfig ION
 	tristate "Ion Memory Manager"
 	select GENERIC_ALLOCATOR
+	select DMA_SHARED_BUFFER
 	help
 	  Chose this option to enable the ION Memory Manager.
 
drivers/staging/android/ion/ion.c +226 -422

872 lines not on this page · raw message

drivers/staging/android/ion/ion.h +9 -49

72 lines not on this page · raw message

Signature · 1 line
1.8.3.2
#24 lore raw

PATCH 7/115 gpu: ion: Use alloc_pages instead of vmalloc from the system heap

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

With this change the ion_system_heap will only use kernel address space when the memory is mapped into the kernel (rare case).

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 94 +++++++++++++++++----------
 1 file changed, 61 insertions(+), 33 deletions(-)
drivers/staging/android/ion/ion_system_heap.c +61 -33
index d7e0fa0..3383a88 100644
@@ -27,74 +27,102 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 				     unsigned long size, unsigned long align,
 				     unsigned long flags)
 {
-	buffer->priv_virt = vmalloc_user(size);
-	if (!buffer->priv_virt)
-		return -ENOMEM;
-	return 0;
-}
-
-void ion_system_heap_free(struct ion_buffer *buffer)
-{
-	vfree(buffer->priv_virt);
-}
-
-struct sg_table *ion_system_heap_map_dma(struct ion_heap *heap,
-					 struct ion_buffer *buffer)
-{
 	struct sg_table *table;
 	struct scatterlist *sg;
-	int i;
-	int npages = PAGE_ALIGN(buffer->size) / PAGE_SIZE;
-	void *vaddr = buffer->priv_virt;
-	int ret;
+	int i, j;
+	int npages = PAGE_ALIGN(size) / PAGE_SIZE;
 
-	table = kzalloc(sizeof(struct sg_table), GFP_KERNEL);
+	table = kmalloc(sizeof(struct sg_table), GFP_KERNEL);
 	if (!table)
-		return ERR_PTR(-ENOMEM);
-	ret = sg_alloc_table(table, npages, GFP_KERNEL);
-	if (ret)
+		return -ENOMEM;
+	i = sg_alloc_table(table, npages, GFP_KERNEL);
+	if (i)
 		goto err0;
 	for_each_sg(table->sgl, sg, table->nents, i) {
 		struct page *page;
-		page = vmalloc_to_page(vaddr);
-		if (!page) {
-			ret = -ENOMEM;
+		page = alloc_page(GFP_KERNEL);
+		if (!page)
 			goto err1;
-		}
 		sg_set_page(sg, page, PAGE_SIZE, 0);
-		vaddr += PAGE_SIZE;
 	}
-	return table;
+	buffer->priv_virt = table;
+	return 0;
 err1:
+	for_each_sg(table->sgl, sg, i, j)
+		__free_page(sg_page(sg));
 	sg_free_table(table);
 err0:
 	kfree(table);
-	return ERR_PTR(ret);
+	return -ENOMEM;
 }
 
-void ion_system_heap_unmap_dma(struct ion_heap *heap,
-			       struct ion_buffer *buffer)
+void ion_system_heap_free(struct ion_buffer *buffer)
 {
+	int i;
+	struct scatterlist *sg;
+	struct sg_table *table = buffer->priv_virt;
+
+	for_each_sg(table->sgl, sg, table->nents, i)
+		__free_page(sg_page(sg));
 	if (buffer->sg_table)
 		sg_free_table(buffer->sg_table);
 	kfree(buffer->sg_table);
 }
 
+struct sg_table *ion_system_heap_map_dma(struct ion_heap *heap,
+					 struct ion_buffer *buffer)
+{
+	return buffer->priv_virt;
+}
+
+void ion_system_heap_unmap_dma(struct ion_heap *heap,
+			       struct ion_buffer *buffer)
+{
+	return;
+}
+
 void *ion_system_heap_map_kernel(struct ion_heap *heap,
 				 struct ion_buffer *buffer)
 {
-	return buffer->priv_virt;
+	struct scatterlist *sg;
+	int i;
+	void *vaddr;
+	struct sg_table *table = buffer->priv_virt;
+	struct page **pages = kmalloc(sizeof(struct page *) * table->nents,
+				      GFP_KERNEL);
+
+	for_each_sg(table->sgl, sg, table->nents, i)
+		pages[i] = sg_page(sg);
+	vaddr = vmap(pages, table->nents, VM_MAP, PAGE_KERNEL);
+	kfree(pages);
+
+	return vaddr;
 }
 
 void ion_system_heap_unmap_kernel(struct ion_heap *heap,
 				  struct ion_buffer *buffer)
 {
+	vunmap(buffer->vaddr);
 }
 
 int ion_system_heap_map_user(struct ion_heap *heap, struct ion_buffer *buffer,
 			     struct vm_area_struct *vma)
 {
-	return remap_vmalloc_range(vma, buffer->priv_virt, vma->vm_pgoff);
+	struct sg_table *table = buffer->priv_virt;
+	unsigned long addr = vma->vm_start;
+	unsigned long offset = vma->vm_pgoff;
+	struct scatterlist *sg;
+	int i;
+
+	for_each_sg(table->sgl, sg, table->nents, i) {
+		if (offset) {
+			offset--;
+			continue;
+		}
+		vm_insert_page(vma, addr, sg_page(sg));
+		addr += PAGE_SIZE;
+	}
+	return 0;
 }
 
 static struct ion_heap_ops vmalloc_ops = {
Signature · 1 line
1.8.3.2
#25 lore raw

PATCH 8/115 gpu: ion: support begin/end and kmap/kunmap dma_buf ops

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

These ops were added in the 3.4 kernel. This patch adds support for them to ion. Previous ion_map/unmap_kernel api is also retained in addition to this new api.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 88 +++++++++++++++++++++++++++------------
 1 file changed, 62 insertions(+), 26 deletions(-)
drivers/staging/android/ion/ion.c +62 -26
index 75bf8f2..8937618 100644
@@ -391,6 +391,22 @@ int ion_phys(struct ion_client *client, struct ion_handle *handle,
 	return ret;
 }
 
+static void *ion_buffer_kmap_get(struct ion_buffer *buffer)
+{
+	void *vaddr;
+
+	if (buffer->kmap_cnt) {
+		buffer->kmap_cnt++;
+		return buffer->vaddr;
+	}
+	vaddr = buffer->heap->ops->map_kernel(buffer->heap, buffer);
+	if (IS_ERR_OR_NULL(vaddr))
+		return vaddr;
+	buffer->vaddr = vaddr;
+	buffer->kmap_cnt++;
+	return vaddr;
+}
+
 static void *ion_handle_kmap_get(struct ion_handle *handle)
 {
 	struct ion_buffer *buffer = handle->buffer;
@@ -399,34 +415,30 @@ static void *ion_handle_kmap_get(struct ion_handle *handle)
 	if (handle->kmap_cnt) {
 		handle->kmap_cnt++;
 		return buffer->vaddr;
-	} else if (buffer->kmap_cnt) {
-		handle->kmap_cnt++;
-		buffer->kmap_cnt++;
-		return buffer->vaddr;
 	}
-
-	vaddr = buffer->heap->ops->map_kernel(buffer->heap, buffer);
-	buffer->vaddr = vaddr;
-	if (IS_ERR_OR_NULL(vaddr)) {
-		buffer->vaddr = NULL;
+	vaddr = ion_buffer_kmap_get(buffer);
+	if (IS_ERR_OR_NULL(vaddr))
 		return vaddr;
-	}
 	handle->kmap_cnt++;
-	buffer->kmap_cnt++;
 	return vaddr;
 }
 
+static void ion_buffer_kmap_put(struct ion_buffer *buffer)
+{
+	buffer->kmap_cnt--;
+	if (!buffer->kmap_cnt) {
+		buffer->heap->ops->unmap_kernel(buffer->heap, buffer);
+		buffer->vaddr = NULL;
+	}
+}
+
 static void ion_handle_kmap_put(struct ion_handle *handle)
 {
 	struct ion_buffer *buffer = handle->buffer;
 
 	handle->kmap_cnt--;
 	if (!handle->kmap_cnt)
-		buffer->kmap_cnt--;
-	if (!buffer->kmap_cnt) {
-		buffer->heap->ops->unmap_kernel(buffer->heap, buffer);
-		buffer->vaddr = NULL;
-	}
+		ion_buffer_kmap_put(buffer);
 }
 
 void *ion_map_kernel(struct ion_client *client, struct ion_handle *handle)
@@ -675,7 +687,8 @@ static void ion_dma_buf_release(struct dma_buf *dmabuf)
 
 static void *ion_dma_buf_kmap(struct dma_buf *dmabuf, unsigned long offset)
 {
-	return NULL;
+	struct ion_buffer *buffer = dmabuf->priv;
+	return buffer->vaddr + offset;
 }
 
 static void ion_dma_buf_kunmap(struct dma_buf *dmabuf, unsigned long offset,
@@ -684,26 +697,49 @@ static void ion_dma_buf_kunmap(struct dma_buf *dmabuf, unsigned long offset,
 	return;
 }
 
-static void *ion_dma_buf_kmap_atomic(struct dma_buf *dmabuf,
-				     unsigned long offset)
+static int ion_dma_buf_begin_cpu_access(struct dma_buf *dmabuf, size_t start,
+					size_t len,
+					enum dma_data_direction direction)
 {
-	return NULL;
+	struct ion_buffer *buffer = dmabuf->priv;
+	void *vaddr;
+
+	if (!buffer->heap->ops->map_kernel) {
+		pr_err("%s: map kernel is not implemented by this heap.\n",
+		       __func__);
+		return -ENODEV;
+	}
+
+	mutex_lock(&buffer->lock);
+	vaddr = ion_buffer_kmap_get(buffer);
+	mutex_unlock(&buffer->lock);
+	if (IS_ERR(vaddr))
+		return PTR_ERR(vaddr);
+	if (!vaddr)
+		return -ENOMEM;
+	return 0;
 }
 
-static void ion_dma_buf_kunmap_atomic(struct dma_buf *dmabuf,
-				      unsigned long offset, void *ptr)
+static void ion_dma_buf_end_cpu_access(struct dma_buf *dmabuf, size_t start,
+				       size_t len,
+				       enum dma_data_direction direction)
 {
-	return;
-}
+	struct ion_buffer *buffer = dmabuf->priv;
 
+	mutex_lock(&buffer->lock);
+	ion_buffer_kmap_put(buffer);
+	mutex_unlock(&buffer->lock);
+}
 
 struct dma_buf_ops dma_buf_ops = {
 	.map_dma_buf = ion_map_dma_buf,
 	.unmap_dma_buf = ion_unmap_dma_buf,
 	.mmap = ion_mmap,
 	.release = ion_dma_buf_release,
-	.kmap_atomic = ion_dma_buf_kmap_atomic,
-	.kunmap_atomic = ion_dma_buf_kunmap_atomic,
+	.begin_cpu_access = ion_dma_buf_begin_cpu_access,
+	.end_cpu_access = ion_dma_buf_end_cpu_access,
+	.kmap_atomic = ion_dma_buf_kmap,
+	.kunmap_atomic = ion_dma_buf_kunmap,
 	.kmap = ion_dma_buf_kmap,
 	.kunmap = ion_dma_buf_kunmap,
 };
Signature · 1 line
1.8.3.2
#26 lore raw

PATCH 9/115 gpu: ion: Allocate the sg_table at creation time rather than dynamically

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

Rather than calling map_dma on the allocations dynamically, this patch switches to creating the sg_table at the time the buffer is created. This is necessary because in future updates the sg_table will be used for cache maintenance.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 79 ++++++++++++++++++++-------------------
 1 file changed, 41 insertions(+), 38 deletions(-)
drivers/staging/android/ion/ion.c +41 -38
index 8937618..b95202b 100644
@@ -135,6 +135,7 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 				     unsigned long flags)
 {
 	struct ion_buffer *buffer;
+	struct sg_table *table;
 	int ret;
 
 	buffer = kzalloc(sizeof(struct ion_buffer), GFP_KERNEL);
@@ -149,6 +150,15 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 		kfree(buffer);
 		return ERR_PTR(ret);
 	}
+
+	table = buffer->heap->ops->map_dma(buffer->heap, buffer);
+	if (IS_ERR_OR_NULL(table)) {
+		heap->ops->free(buffer);
+		kfree(buffer);
+		return ERR_PTR(PTR_ERR(table));
+	}
+	buffer->sg_table = table;
+
 	buffer->dev = dev;
 	buffer->size = len;
 	mutex_init(&buffer->lock);
@@ -164,9 +174,7 @@ static void ion_buffer_destroy(struct kref *kref)
 	if (WARN_ON(buffer->kmap_cnt > 0))
 		buffer->heap->ops->unmap_kernel(buffer->heap, buffer);
 
-	if (WARN_ON(buffer->dmap_cnt > 0))
-		buffer->heap->ops->unmap_dma(buffer->heap, buffer);
-
+	buffer->heap->ops->unmap_dma(buffer->heap, buffer);
 	buffer->heap->ops->free(buffer);
 	mutex_lock(&dev->lock);
 	rb_erase(&buffer->node, &dev->buffers);
@@ -346,6 +354,7 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 		mutex_unlock(&client->lock);
 	}
 
+
 	return handle;
 }
 
@@ -607,53 +616,42 @@ void ion_client_destroy(struct ion_client *client)
 	kfree(client);
 }
 
-static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment,
-					enum dma_data_direction direction)
+struct sg_table *ion_map_dma(struct ion_client *client,
+			     struct ion_handle *handle)
 {
-	struct dma_buf *dmabuf = attachment->dmabuf;
-	struct ion_buffer *buffer = dmabuf->priv;
+	struct ion_buffer *buffer;
 	struct sg_table *table;
 
-	mutex_lock(&buffer->lock);
-
-	if (!buffer->heap->ops->map_dma) {
-		pr_err("%s: map_dma is not implemented by this heap.\n",
+	mutex_lock(&client->lock);
+	if (!ion_handle_validate(client, handle)) {
+		pr_err("%s: invalid handle passed to map_dma.\n",
 		       __func__);
-		mutex_unlock(&buffer->lock);
-		return ERR_PTR(-ENODEV);
-	}
-	/* if an sg list already exists for this buffer just return it */
-	if (buffer->dmap_cnt) {
-		table = buffer->sg_table;
-		goto end;
+		mutex_unlock(&client->lock);
+		return ERR_PTR(-EINVAL);
 	}
-
-	/* otherwise call into the heap to create one */
-	table = buffer->heap->ops->map_dma(buffer->heap, buffer);
-	if (IS_ERR_OR_NULL(table))
-		goto err;
-	buffer->sg_table = table;
-end:
-	buffer->dmap_cnt++;
-err:
-	mutex_unlock(&buffer->lock);
+	buffer = handle->buffer;
+	table = buffer->sg_table;
+	mutex_unlock(&client->lock);
 	return table;
 }
 
-static void ion_unmap_dma_buf(struct dma_buf_attachment *attachment,
-			      struct sg_table *table,
-			      enum dma_data_direction direction)
+void ion_unmap_dma(struct ion_client *client, struct ion_handle *handle)
+{
+}
+
+static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment,
+					enum dma_data_direction direction)
 {
 	struct dma_buf *dmabuf = attachment->dmabuf;
 	struct ion_buffer *buffer = dmabuf->priv;
 
-	mutex_lock(&buffer->lock);
-	buffer->dmap_cnt--;
-	if (!buffer->dmap_cnt) {
-		buffer->heap->ops->unmap_dma(buffer->heap, buffer);
-		buffer->sg_table = NULL;
-	}
-	mutex_unlock(&buffer->lock);
+	return buffer->sg_table;
+}
+
+static void ion_unmap_dma_buf(struct dma_buf_attachment *attachment,
+			      struct sg_table *table,
+			      enum dma_data_direction direction)
+{
 }
 
 static int ion_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
@@ -987,6 +985,11 @@ void ion_device_add_heap(struct ion_device *dev, struct ion_heap *heap)
 	struct rb_node *parent = NULL;
 	struct ion_heap *entry;
 
+	if (!heap->ops->allocate || !heap->ops->free || !heap->ops->map_dma ||
+	    !heap->ops->unmap_dma)
+		pr_err("%s: can not add heap with invalid ops struct.\n",
+		       __func__);
+
 	heap->dev = dev;
 	mutex_lock(&dev->lock);
 	while (*p) {
Signature · 1 line
1.8.3.2
#27 lore raw

PATCH 10/115 gpu: ion: Get an sg_table from an ion handle

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

This patch adds an interface to return and sg_table given a valid ion handle.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c |  8 ++------
 drivers/staging/android/ion/ion.h | 13 ++++++++++++-
 2 files changed, 14 insertions(+), 7 deletions(-)
drivers/staging/android/ion/ion.c +2 -6
index b95202b..7ff9983 100644
@@ -616,8 +616,8 @@ void ion_client_destroy(struct ion_client *client)
 	kfree(client);
 }
 
-struct sg_table *ion_map_dma(struct ion_client *client,
-			     struct ion_handle *handle)
+struct sg_table *ion_sg_table(struct ion_client *client,
+			      struct ion_handle *handle)
 {
 	struct ion_buffer *buffer;
 	struct sg_table *table;
@@ -635,10 +635,6 @@ struct sg_table *ion_map_dma(struct ion_client *client,
 	return table;
 }
 
-void ion_unmap_dma(struct ion_client *client, struct ion_handle *handle)
-{
-}
-
 static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment,
 					enum dma_data_direction direction)
 {
drivers/staging/android/ion/ion.h +12 -1
index 84ca2a9..28810c8 100644
@@ -149,7 +149,7 @@ void ion_free(struct ion_client *client, struct ion_handle *handle);
  * This function queries the heap for a particular handle to get the
  * handle's physical address.  It't output is only correct if
  * a heap returns physically contiguous memory -- in other cases
- * this api should not be implemented -- ion_map_dma should be used
+ * this api should not be implemented -- ion_sg_table should be used
  * instead.  Returns -EINVAL if the handle is invalid.  This has
  * no implications on the reference counting of the handle --
  * the returned value may not be valid if the caller is not
@@ -159,6 +159,17 @@ int ion_phys(struct ion_client *client, struct ion_handle *handle,
 	     ion_phys_addr_t *addr, size_t *len);
 
 /**
+ * ion_map_dma - return an sg_table describing a handle
+ * @client:	the client
+ * @handle:	the handle
+ *
+ * This function returns the sg_table describing
+ * a particular ion handle.
+ */
+struct sg_table *ion_sg_table(struct ion_client *client,
+			      struct ion_handle *handle);
+
+/**
  * ion_map_kernel - create mapping for the given handle
  * @client:	the client
  * @handle:	handle to map
Signature · 1 line
1.8.3.2
#28 lore raw

PATCH 11/115 gpu: ion: fill in buffer->{dev,size} before mapping new buffers

In reply to John Stultz

From: Greg Hackmann <[email protected]>

At least one map_dma() implementation (EXYNOS_CONTIG) assumes the fields are filled in

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/staging/android/ion/ion.c +3 -2
index 7ff9983..552a86a 100644
@@ -151,6 +151,9 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 		return ERR_PTR(ret);
 	}
 
+	buffer->dev = dev;
+	buffer->size = len;
+
 	table = buffer->heap->ops->map_dma(buffer->heap, buffer);
 	if (IS_ERR_OR_NULL(table)) {
 		heap->ops->free(buffer);
@@ -159,8 +162,6 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 	}
 	buffer->sg_table = table;
 
-	buffer->dev = dev;
-	buffer->size = len;
 	mutex_init(&buffer->lock);
 	ion_buffer_add(dev, buffer);
 	return buffer;
Signature · 1 line
1.8.3.2
#29 lore raw

PATCH 12/115 gpu: ion: Set the dma_address of the sg list at alloc time

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

This patch sets the dma_address field of the sglist representing an allocation at allocation time. This technically breaks the dma api which states that these addresses should be set when a particular device takes ownership of a buffer via the dma_map apis. In the case of our systems the only dma address space is physical addresses. Additionally, we can not afford the overhead of calling dma_map_sg from this location as it implies a cache invalidate that is not necessary if the memory was previously mapped cached. Instead, the expectation is that memory being returned from the heaps is ready for dma in that if any cached mappings of that memory exist they have been invalidated.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
drivers/staging/android/ion/ion.c +12 -1
index 552a86a..ddfe49a 100644
@@ -136,7 +136,8 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 {
 	struct ion_buffer *buffer;
 	struct sg_table *table;
-	int ret;
+	struct scatterlist *sg;
+	int i, ret;
 
 	buffer = kzalloc(sizeof(struct ion_buffer), GFP_KERNEL);
 	if (!buffer)
@@ -163,6 +164,16 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 	buffer->sg_table = table;
 
 	mutex_init(&buffer->lock);
+	/* this will set up dma addresses for the sglist -- it is not
+	   technically correct as per the dma api -- a specific
+	   device isn't really taking ownership here.  However, in practice on
+	   our systems the only dma_address space is physical addresses.
+	   Additionally, we can't afford the overhead of invalidating every
+	   allocation via dma_map_sg. The implicit contract here is that
+	   memory comming from the heaps is ready for dma, ie if it has a
+	   cached mapping that mapping has been invalidated */
+	for_each_sg(buffer->sg_table->sgl, sg, buffer->sg_table->nents, i)
+		sg_dma_address(sg) = sg_phys(sg);
 	ion_buffer_add(dev, buffer);
 	return buffer;
 }
Signature · 1 line
1.8.3.2
#30 lore raw

PATCH 13/115 gpu: ion: ion_system_heap: Change allocations to GFP_HIGHUSER

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

Previously the ion_system_heap was using GFP_KERNEL, forcing all allocations to be in lowmem. This quickly causes us to run out of lowmem.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/android/ion/ion_system_heap.c +1 -1
index 3383a88..35b3726 100644
@@ -40,7 +40,7 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 		goto err0;
 	for_each_sg(table->sgl, sg, table->nents, i) {
 		struct page *page;
-		page = alloc_page(GFP_KERNEL);
+		page = alloc_page(GFP_HIGHUSER | __GFP_ZERO);
 		if (!page)
 			goto err1;
 		sg_set_page(sg, page, PAGE_SIZE, 0);
Signature · 1 line
1.8.3.2
#31 lore raw

PATCH 14/115 gpu: ion: Loop on the handle count when destroying

In reply to John Stultz

From: Laura Abbott <[email protected]>

When destroying a handle, all kernel mappings to that handle should be destroyed. Other handles may still have references and valid mappings to the buffer underneath which should not be destroyed. Loop on the handle reference count, not the buffer reference count to get rid of all kernel mappings for the handle.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/android/ion/ion.c +1 -1
index ddfe49a..dc7174d 100644
@@ -232,7 +232,7 @@ static void ion_handle_destroy(struct kref *kref)
 	mutex_lock(&client->lock);
 
 	mutex_lock(&buffer->lock);
-	while (buffer->kmap_cnt)
+	while (handle->kmap_cnt)
 		ion_handle_kmap_put(handle);
 	mutex_unlock(&buffer->lock);
 
Signature · 1 line
1.8.3.2
#32 lore raw

PATCH 15/115 gpu: ion: Map only the vma size given

In reply to John Stultz

From: Laura Abbott <[email protected]>

When mapping carveout buffers into userspace, only map the size of the vma given, not the full size of the buffer since clients may map less than the buffer size.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_carveout_heap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/android/ion/ion_carveout_heap.c +1 -1
index e8c4e36..16f4fc7 100644
@@ -116,7 +116,7 @@ int ion_carveout_heap_map_user(struct ion_heap *heap, struct ion_buffer *buffer,
 {
 	return remap_pfn_range(vma, vma->vm_start,
 			       __phys_to_pfn(buffer->priv_phys) + vma->vm_pgoff,
-			       buffer->size,
+			       vma->vm_end - vma->vm_start,
 			       pgprot_noncached(vma->vm_page_prot));
 }
 
Signature · 1 line
1.8.3.2
#33 lore raw

PATCH 16/115 gpu: ion: Add cache maintenance to ion.

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

This patch adds cache maintenance operations to ion. As per mailing list discussions regarding dma_buf, cache operations are done implicitly. At buffer allocaiton time the user can select whether he'd like mappings (both kernel and user) to be cached. When cached mappings are selected, no mappings will be created for a buffer at mmap time. Instead pages will be faulted in one at a time so we can track which pages require flushing before dma. When the buffers are mapped for dma (via the dma_buf apis) any pages which were touched will be synced for device.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c               | 162 ++++++++++++++++++++++--
 drivers/staging/android/ion/ion.h               |  19 ++-
 drivers/staging/android/ion/ion_carveout_heap.c |  26 +++-
 drivers/staging/android/ion/ion_priv.h          |   2 +
 drivers/staging/android/ion/ion_system_heap.c   |  20 ++-
 5 files changed, 210 insertions(+), 19 deletions(-)
drivers/staging/android/ion/ion.c +152 -10
index dc7174d..bc9e922 100644
@@ -34,7 +34,6 @@
 
 #include "ion.h"
 #include "ion_priv.h"
-#define DEBUG
 
 /**
  * struct ion_device - the metadata of the ion device node
@@ -127,6 +126,8 @@ static void ion_buffer_add(struct ion_device *dev,
 	rb_insert_color(&buffer->node, &dev->buffers);
 }
 
+static int ion_buffer_alloc_dirty(struct ion_buffer *buffer);
+
 /* this function should only be called while dev->lock is held */
 static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 				     struct ion_device *dev,
@@ -154,15 +155,38 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 
 	buffer->dev = dev;
 	buffer->size = len;
+	buffer->flags = flags;
 
-	table = buffer->heap->ops->map_dma(buffer->heap, buffer);
+	table = heap->ops->map_dma(heap, buffer);
 	if (IS_ERR_OR_NULL(table)) {
 		heap->ops->free(buffer);
 		kfree(buffer);
 		return ERR_PTR(PTR_ERR(table));
 	}
 	buffer->sg_table = table;
+	if (buffer->flags & ION_FLAG_CACHED)
+		for_each_sg(buffer->sg_table->sgl, sg, buffer->sg_table->nents,
+			    i) {
+			if (sg_dma_len(sg) == PAGE_SIZE)
+				continue;
+			pr_err("%s: cached mappings must have pagewise "
+			       "sg_lists\n", __func__);
+			heap->ops->unmap_dma(heap, buffer);
+			kfree(buffer);
+			return ERR_PTR(-EINVAL);
+		}
 
+	ret = ion_buffer_alloc_dirty(buffer);
+	if (ret) {
+		heap->ops->unmap_dma(heap, buffer);
+		heap->ops->free(buffer);
+		kfree(buffer);
+		return ERR_PTR(ret);
+	}
+
+	buffer->dev = dev;
+	buffer->size = len;
+	INIT_LIST_HEAD(&buffer->vmas);
 	mutex_init(&buffer->lock);
 	/* this will set up dma addresses for the sglist -- it is not
 	   technically correct as per the dma api -- a specific
@@ -313,13 +337,16 @@ static void ion_handle_add(struct ion_client *client, struct ion_handle *handle)
 }
 
 struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
-			     size_t align, unsigned int flags)
+			     size_t align, unsigned int heap_mask,
+			     unsigned int flags)
 {
 	struct rb_node *n;
 	struct ion_handle *handle;
 	struct ion_device *dev = client->dev;
 	struct ion_buffer *buffer = NULL;
 
+	pr_debug("%s: len %d align %d heap_mask %u flags %x\n", __func__, len,
+		 align, heap_mask, flags);
 	/*
 	 * traverse the list of heaps available in this system in priority
 	 * order.  If the heap type is supported by the client, and matches the
@@ -338,7 +365,7 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 		if (!((1 << heap->type) & client->heap_mask))
 			continue;
 		/* if the caller didn't specify this heap type */
-		if (!((1 << heap->id) & flags))
+		if (!((1 << heap->id) & heap_mask))
 			continue;
 		buffer = ion_buffer_create(heap, dev, len, align, flags);
 		if (!IS_ERR_OR_NULL(buffer))
@@ -647,12 +674,18 @@ struct sg_table *ion_sg_table(struct ion_client *client,
 	return table;
 }
 
+static void ion_buffer_sync_for_device(struct ion_buffer *buffer,
+				       struct device *dev,
+				       enum dma_data_direction direction);
+
 static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment,
 					enum dma_data_direction direction)
 {
 	struct dma_buf *dmabuf = attachment->dmabuf;
 	struct ion_buffer *buffer = dmabuf->priv;
 
+	if (buffer->flags & ION_FLAG_CACHED)
+		ion_buffer_sync_for_device(buffer, attachment->dev, direction);
 	return buffer->sg_table;
 }
 
@@ -662,10 +695,112 @@ static void ion_unmap_dma_buf(struct dma_buf_attachment *attachment,
 {
 }
 
+static int ion_buffer_alloc_dirty(struct ion_buffer *buffer)
+{
+	unsigned long pages = buffer->sg_table->nents;
+	unsigned long length = (pages + BITS_PER_LONG - 1)/BITS_PER_LONG;
+
+	buffer->dirty = kzalloc(length * sizeof(unsigned long), GFP_KERNEL);
+	if (!buffer->dirty)
+		return -ENOMEM;
+	return 0;
+}
+
+struct ion_vma_list {
+	struct list_head list;
+	struct vm_area_struct *vma;
+};
+
+static void ion_buffer_sync_for_device(struct ion_buffer *buffer,
+				       struct device *dev,
+				       enum dma_data_direction dir)
+{
+	struct scatterlist *sg;
+	int i;
+	struct ion_vma_list *vma_list;
+
+	pr_debug("%s: syncing for device %s\n", __func__,
+		 dev ? dev_name(dev) : "null");
+	mutex_lock(&buffer->lock);
+	for_each_sg(buffer->sg_table->sgl, sg, buffer->sg_table->nents, i) {
+		if (!test_bit(i, buffer->dirty))
+			continue;
+		dma_sync_sg_for_device(dev, sg, 1, dir);
+		clear_bit(i, buffer->dirty);
+	}
+	list_for_each_entry(vma_list, &buffer->vmas, list) {
+		struct vm_area_struct *vma = vma_list->vma;
+
+		zap_page_range(vma, vma->vm_start, vma->vm_end - vma->vm_start,
+			       NULL);
+	}
+	mutex_unlock(&buffer->lock);
+}
+
+int ion_vm_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
+{
+	struct ion_buffer *buffer = vma->vm_private_data;
+	struct scatterlist *sg;
+	int i;
+
+	mutex_lock(&buffer->lock);
+	set_bit(vmf->pgoff, buffer->dirty);
+
+	for_each_sg(buffer->sg_table->sgl, sg, buffer->sg_table->nents, i) {
+		if (i != vmf->pgoff)
+			continue;
+		dma_sync_sg_for_cpu(NULL, sg, 1, DMA_BIDIRECTIONAL);
+		vm_insert_page(vma, (unsigned long)vmf->virtual_address,
+			       sg_page(sg));
+		break;
+	}
+	mutex_unlock(&buffer->lock);
+	return VM_FAULT_NOPAGE;
+}
+
+static void ion_vm_open(struct vm_area_struct *vma)
+{
+	struct ion_buffer *buffer = vma->vm_private_data;
+	struct ion_vma_list *vma_list;
+
+	vma_list = kmalloc(sizeof(struct ion_vma_list), GFP_KERNEL);
+	if (!vma_list)
+		return;
+	vma_list->vma = vma;
+	mutex_lock(&buffer->lock);
+	list_add(&vma_list->list, &buffer->vmas);
+	mutex_unlock(&buffer->lock);
+	pr_debug("%s: adding %p\n", __func__, vma);
+}
+
+static void ion_vm_close(struct vm_area_struct *vma)
+{
+	struct ion_buffer *buffer = vma->vm_private_data;
+	struct ion_vma_list *vma_list, *tmp;
+
+	pr_debug("%s\n", __func__);
+	mutex_lock(&buffer->lock);
+	list_for_each_entry_safe(vma_list, tmp, &buffer->vmas, list) {
+		if (vma_list->vma != vma)
+			continue;
+		list_del(&vma_list->list);
+		kfree(vma_list);
+		pr_debug("%s: deleting %p\n", __func__, vma);
+		break;
+	}
+	mutex_unlock(&buffer->lock);
+}
+
+struct vm_operations_struct ion_vma_ops = {
+	.open = ion_vm_open,
+	.close = ion_vm_close,
+	.fault = ion_vm_fault,
+};
+
 static int ion_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
 {
 	struct ion_buffer *buffer = dmabuf->priv;
-	int ret;
+	int ret = 0;
 
 	if (!buffer->heap->ops->map_user) {
 		pr_err("%s: this heap does not define a method for mapping "
@@ -673,10 +808,17 @@ static int ion_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
 		return -EINVAL;
 	}
 
-	mutex_lock(&buffer->lock);
-	/* now map it to userspace */
-	ret = buffer->heap->ops->map_user(buffer->heap, buffer, vma);
-	mutex_unlock(&buffer->lock);
+	if (buffer->flags & ION_FLAG_CACHED) {
+		vma->vm_private_data = buffer;
+		vma->vm_ops = &ion_vma_ops;
+		ion_vm_open(vma);
+	} else {
+		vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
+		mutex_lock(&buffer->lock);
+		/* now map it to userspace */
+		ret = buffer->heap->ops->map_user(buffer->heap, buffer, vma);
+		mutex_unlock(&buffer->lock);
+	}
 
 	if (ret)
 		pr_err("%s: failure mapping buffer to userspace\n",
@@ -828,7 +970,7 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 		if (copy_from_user(&data, (void __user *)arg, sizeof(data)))
 			return -EFAULT;
 		data.handle = ion_alloc(client, data.len, data.align,
-					     data.flags);
+					     data.heap_mask, data.flags);
 
 		if (IS_ERR(data.handle))
 			return PTR_ERR(data.handle);
drivers/staging/android/ion/ion.h +17 -2
index 28810c8..88b9a2f 100644
@@ -42,6 +42,15 @@ enum ion_heap_type {
 #define ION_HEAP_SYSTEM_CONTIG_MASK	(1 << ION_HEAP_TYPE_SYSTEM_CONTIG)
 #define ION_HEAP_CARVEOUT_MASK		(1 << ION_HEAP_TYPE_CARVEOUT)
 
+/**
+ * heap flags - the lower 16 bits are used by core ion, the upper 16
+ * bits are reserved for use by the heaps themselves.
+ */
+#define ION_FLAG_CACHED 1		/* mappings of this buffer should be
+					   cached, ion will do cache
+					   maintenance when the buffer is
+					   mapped for dma */
+
 #ifdef __KERNEL__
 struct ion_device;
 struct ion_heap;
@@ -121,14 +130,18 @@ void ion_client_destroy(struct ion_client *client);
  * @len:	size of the allocation
  * @align:	requested allocation alignment, lots of hardware blocks have
  *		alignment requirements of some kind
- * @flags:	mask of heaps to allocate from, if multiple bits are set
+ * @heap_mask:	mask of heaps to allocate from, if multiple bits are set
  *		heaps will be tried in order from lowest to highest order bit
+ * @flags:	heap flags, the low 16 bits are consumed by ion, the high 16
+ *		bits are passed on to the respective heap and can be heap
+ * 		custom
  *
  * Allocate memory in one of the heaps provided in heap mask and return
  * an opaque handle to it.
  */
 struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
-			     size_t align, unsigned int flags);
+			     size_t align, unsigned int heap_mask,
+			     unsigned int flags);
 
 /**
  * ion_free - free a handle
@@ -218,6 +231,7 @@ struct ion_handle *ion_import_dma_buf(struct ion_client *client, int fd);
  * struct ion_allocation_data - metadata passed from userspace for allocations
  * @len:	size of the allocation
  * @align:	required alignment of the allocation
+ * @heap_mask:	mask of heaps to allocate from
  * @flags:	flags passed to heap
  * @handle:	pointer that will be populated with a cookie to use to refer
  *		to this allocation
@@ -227,6 +241,7 @@ struct ion_handle *ion_import_dma_buf(struct ion_client *client, int fd);
 struct ion_allocation_data {
 	size_t len;
 	size_t align;
+	unsigned int heap_mask;
 	unsigned int flags;
 	struct ion_handle *handle;
 };
drivers/staging/android/ion/ion_carveout_heap.c +22 -4
index 16f4fc7..daa348e 100644
@@ -84,23 +84,41 @@ static void ion_carveout_heap_free(struct ion_buffer *buffer)
 	buffer->priv_phys = ION_CARVEOUT_ALLOCATE_FAIL;
 }
 
-struct scatterlist *ion_carveout_heap_map_dma(struct ion_heap *heap,
+struct sg_table *ion_carveout_heap_map_dma(struct ion_heap *heap,
 					      struct ion_buffer *buffer)
 {
-	return ERR_PTR(-EINVAL);
+	struct sg_table *table;
+	int ret;
+
+	table = kzalloc(sizeof(struct sg_table), GFP_KERNEL);
+	if (!table)
+		return ERR_PTR(-ENOMEM);
+	ret = sg_alloc_table(table, 1, GFP_KERNEL);
+	if (ret) {
+		kfree(table);
+		return ERR_PTR(ret);
+	}
+	sg_set_page(table->sgl, phys_to_page(buffer->priv_phys), buffer->size,
+		    0);
+	return table;
 }
 
 void ion_carveout_heap_unmap_dma(struct ion_heap *heap,
 				 struct ion_buffer *buffer)
 {
-	return;
+	sg_free_table(buffer->sg_table);
 }
 
 void *ion_carveout_heap_map_kernel(struct ion_heap *heap,
 				   struct ion_buffer *buffer)
 {
+	int mtype = MT_MEMORY_NONCACHED;
+
+	if (buffer->flags & ION_FLAG_CACHED)
+		mtype = MT_MEMORY;
+
 	return __arm_ioremap(buffer->priv_phys, buffer->size,
-			      MT_MEMORY_NONCACHED);
+			      mtype);
 }
 
 void ion_carveout_heap_unmap_kernel(struct ion_heap *heap,
drivers/staging/android/ion/ion_priv.h +2 -0
index 7a77f6f..406d1f4 100644
@@ -60,6 +60,8 @@ struct ion_buffer {
 	void *vaddr;
 	int dmap_cnt;
 	struct sg_table *sg_table;
+	unsigned long *dirty;
+	struct list_head vmas;
 };
 
 /**
drivers/staging/android/ion/ion_system_heap.c +17 -3
index 35b3726..dceed5b 100644
@@ -87,13 +87,20 @@ void *ion_system_heap_map_kernel(struct ion_heap *heap,
 	struct scatterlist *sg;
 	int i;
 	void *vaddr;
+	pgprot_t pgprot;
 	struct sg_table *table = buffer->priv_virt;
 	struct page **pages = kmalloc(sizeof(struct page *) * table->nents,
 				      GFP_KERNEL);
 
 	for_each_sg(table->sgl, sg, table->nents, i)
 		pages[i] = sg_page(sg);
-	vaddr = vmap(pages, table->nents, VM_MAP, PAGE_KERNEL);
+
+	if (buffer->flags & ION_FLAG_CACHED)
+		pgprot = PAGE_KERNEL;
+	else
+		pgprot = pgprot_writecombine(PAGE_KERNEL);
+
+	vaddr = vmap(pages, table->nents, VM_MAP, pgprot);
 	kfree(pages);
 
 	return vaddr;
@@ -179,7 +186,7 @@ static int ion_system_contig_heap_phys(struct ion_heap *heap,
 }
 
 struct sg_table *ion_system_contig_heap_map_dma(struct ion_heap *heap,
-						   struct ion_buffer *buffer)
+						struct ion_buffer *buffer)
 {
 	struct sg_table *table;
 	int ret;
@@ -197,6 +204,13 @@ struct sg_table *ion_system_contig_heap_map_dma(struct ion_heap *heap,
 	return table;
 }
 
+void ion_system_contig_heap_unmap_dma(struct ion_heap *heap,
+				      struct ion_buffer *buffer)
+{
+	sg_free_table(buffer->sg_table);
+	kfree(buffer->sg_table);
+}
+
 int ion_system_contig_heap_map_user(struct ion_heap *heap,
 				    struct ion_buffer *buffer,
 				    struct vm_area_struct *vma)
@@ -213,7 +227,7 @@ static struct ion_heap_ops kmalloc_ops = {
 	.free = ion_system_contig_heap_free,
 	.phys = ion_system_contig_heap_phys,
 	.map_dma = ion_system_contig_heap_map_dma,
-	.unmap_dma = ion_system_heap_unmap_dma,
+	.unmap_dma = ion_system_contig_heap_unmap_dma,
 	.map_kernel = ion_system_heap_map_kernel,
 	.unmap_kernel = ion_system_heap_unmap_kernel,
 	.map_user = ion_system_contig_heap_map_user,
Signature · 1 line
1.8.3.2
#34 lore raw

PATCH 17/115 gpu: ion: Modify the system heap to try to allocate large/huge pages

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

On some systems there is a performance benefit to reducing tlb pressure by minimizing the number of chunks in an allocation.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 116 ++++++++++++++++++++------
 1 file changed, 90 insertions(+), 26 deletions(-)
drivers/staging/android/ion/ion_system_heap.c +90 -26
index dceed5b..98711ce 100644
@@ -14,7 +14,10 @@
  *
  */
 
+#include <asm/page.h>
+#include <linux/dma-mapping.h>
 #include <linux/err.h>
+#include <linux/highmem.h>
 #include <linux/mm.h>
 #include <linux/scatterlist.h>
 #include <linux/slab.h>
@@ -22,6 +25,35 @@
 #include "ion.h"
 #include "ion_priv.h"
 
+struct page_info {
+	struct page *page;
+	unsigned long order;
+	struct list_head list;
+};
+
+static struct page_info *alloc_largest_available(unsigned long size)
+{
+	static unsigned int orders[] = {8, 4, 0};
+	struct page *page;
+	struct page_info *info;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(orders); i++) {
+		if (size < (1 << orders[i]) * PAGE_SIZE)
+			continue;
+		page = alloc_pages(GFP_HIGHUSER | __GFP_ZERO | __GFP_COMP |
+				   __GFP_NOWARN | __GFP_NORETRY, orders[i]);
+		if (!page)
+			continue;
+		split_page(page, orders[i]);
+		info = kmap(page);
+		info->page = page;
+		info->order = orders[i];
+		return info;
+	}
+	return NULL;
+}
+
 static int ion_system_heap_allocate(struct ion_heap *heap,
 				     struct ion_buffer *buffer,
 				     unsigned long size, unsigned long align,
@@ -29,30 +61,54 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 {
 	struct sg_table *table;
 	struct scatterlist *sg;
-	int i, j;
-	int npages = PAGE_ALIGN(size) / PAGE_SIZE;
+	int ret;
+	struct list_head pages;
+	struct page_info *info, *tmp_info;
+	int i;
+	long size_remaining = PAGE_ALIGN(size);
+
+	INIT_LIST_HEAD(&pages);
+	while (size_remaining > 0) {
+		info = alloc_largest_available(size_remaining);
+		if (!info)
+			goto err;
+		list_add_tail(&info->list, &pages);
+		size_remaining -= (1 << info->order) * PAGE_SIZE;
+	}
 
 	table = kmalloc(sizeof(struct sg_table), GFP_KERNEL);
 	if (!table)
-		return -ENOMEM;
-	i = sg_alloc_table(table, npages, GFP_KERNEL);
-	if (i)
-		goto err0;
-	for_each_sg(table->sgl, sg, table->nents, i) {
-		struct page *page;
-		page = alloc_page(GFP_HIGHUSER | __GFP_ZERO);
-		if (!page)
-			goto err1;
-		sg_set_page(sg, page, PAGE_SIZE, 0);
+		goto err;
+
+	ret = sg_alloc_table(table, PAGE_ALIGN(size) / PAGE_SIZE, GFP_KERNEL);
+	if (ret)
+		goto err1;
+
+	sg = table->sgl;
+	list_for_each_entry_safe(info, tmp_info, &pages, list) {
+		struct page *page = info->page;
+		for (i = 0; i < (1 << info->order); i++) {
+			sg_set_page(sg, page + i, PAGE_SIZE, 0);
+			sg = sg_next(sg);
+		}
+		list_del(&info->list);
+		memset(info, 0, sizeof(struct page_info));
+		kunmap(page);
 	}
+
+	dma_sync_sg_for_device(NULL, table->sgl, table->nents,
+			       DMA_BIDIRECTIONAL);
+
 	buffer->priv_virt = table;
 	return 0;
 err1:
-	for_each_sg(table->sgl, sg, i, j)
-		__free_page(sg_page(sg));
-	sg_free_table(table);
-err0:
 	kfree(table);
+err:
+	list_for_each_entry(info, &pages, list) {
+		for (i = 0; i < (1 << info->order); i++)
+			__free_page(info->page + i);
+		kunmap(info->page);
+	}
 	return -ENOMEM;
 }
 
@@ -63,7 +119,7 @@ void ion_system_heap_free(struct ion_buffer *buffer)
 	struct sg_table *table = buffer->priv_virt;
 
 	for_each_sg(table->sgl, sg, table->nents, i)
-		__free_page(sg_page(sg));
+		__free_pages(sg_page(sg), get_order(sg_dma_len(sg)));
 	if (buffer->sg_table)
 		sg_free_table(buffer->sg_table);
 	kfree(buffer->sg_table);
@@ -85,22 +141,29 @@ void *ion_system_heap_map_kernel(struct ion_heap *heap,
 				 struct ion_buffer *buffer)
 {
 	struct scatterlist *sg;
-	int i;
+	int i, j;
 	void *vaddr;
 	pgprot_t pgprot;
 	struct sg_table *table = buffer->priv_virt;
-	struct page **pages = kmalloc(sizeof(struct page *) * table->nents,
-				      GFP_KERNEL);
-
-	for_each_sg(table->sgl, sg, table->nents, i)
-		pages[i] = sg_page(sg);
+	int npages = PAGE_ALIGN(buffer->size) / PAGE_SIZE;
+	struct page **pages = kzalloc(sizeof(struct page *) * npages,
+				     GFP_KERNEL);
+	struct page **tmp = pages;
 
 	if (buffer->flags & ION_FLAG_CACHED)
 		pgprot = PAGE_KERNEL;
 	else
 		pgprot = pgprot_writecombine(PAGE_KERNEL);
 
-	vaddr = vmap(pages, table->nents, VM_MAP, pgprot);
+	for_each_sg(table->sgl, sg, table->nents, i) {
+		int npages_this_entry = PAGE_ALIGN(sg_dma_len(sg)) / PAGE_SIZE;
+		struct page *page = sg_page(sg);
+		BUG_ON(i >= npages);
+		for (j = 0; j < npages_this_entry; j++) {
+			*(tmp++) = page++;
+		}
+	}
+	vaddr = vmap(pages, npages, VM_MAP, pgprot);
 	kfree(pages);
 
 	return vaddr;
@@ -126,8 +189,9 @@ int ion_system_heap_map_user(struct ion_heap *heap, struct ion_buffer *buffer,
 			offset--;
 			continue;
 		}
-		vm_insert_page(vma, addr, sg_page(sg));
-		addr += PAGE_SIZE;
+		remap_pfn_range(vma, addr, page_to_pfn(sg_page(sg)),
+				sg_dma_len(sg), vma->vm_page_prot);
+		addr += sg_dma_len(sg);
 	}
 	return 0;
 }
Signature · 1 line
1.8.3.2
#35 lore raw

PATCH 18/115 gpu: ion: Add explicit sync ioctl

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

This is deprecated in favor of using the dma_buf api which will automatically sync a buffer to memory when it is mapped to a device. However, that functionality is not ready, so this patch adds the ability to sync a buffer explicitly.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 38 ++++++++++++++++++++++++++++++++++++--
 drivers/staging/android/ion/ion.h | 12 +++++++++++-
 2 files changed, 47 insertions(+), 3 deletions(-)
drivers/staging/android/ion/ion.c +36 -2
index bc9e922..d4e4c68 100644
@@ -684,8 +684,7 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment,
 	struct dma_buf *dmabuf = attachment->dmabuf;
 	struct ion_buffer *buffer = dmabuf->priv;
 
-	if (buffer->flags & ION_FLAG_CACHED)
-		ion_buffer_sync_for_device(buffer, attachment->dev, direction);
+	ion_buffer_sync_for_device(buffer, attachment->dev, direction);
 	return buffer->sg_table;
 }
 
@@ -721,6 +720,10 @@ static void ion_buffer_sync_for_device(struct ion_buffer *buffer,
 
 	pr_debug("%s: syncing for device %s\n", __func__,
 		 dev ? dev_name(dev) : "null");
+
+	if (!(buffer->flags & ION_FLAG_CACHED))
+		return;
+
 	mutex_lock(&buffer->lock);
 	for_each_sg(buffer->sg_table->sgl, sg, buffer->sg_table->nents, i) {
 		if (!test_bit(i, buffer->dirty))
@@ -958,6 +961,28 @@ end:
 	return handle;
 }
 
+static int ion_sync_for_device(struct ion_client *client, int fd)
+{
+	struct dma_buf *dmabuf;
+	struct ion_buffer *buffer;
+
+	dmabuf = dma_buf_get(fd);
+	if (IS_ERR_OR_NULL(dmabuf))
+		return PTR_ERR(dmabuf);
+
+	/* if this memory came from ion */
+	if (dmabuf->ops != &dma_buf_ops) {
+		pr_err("%s: can not sync dmabuf from another exporter\n",
+		       __func__);
+		dma_buf_put(dmabuf);
+		return -EINVAL;
+	}
+	buffer = dmabuf->priv;
+	ion_buffer_sync_for_device(buffer, NULL, DMA_BIDIRECTIONAL);
+	dma_buf_put(dmabuf);
+	return 0;
+}
+
 static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 {
 	struct ion_client *client = filp->private_data;
@@ -1022,6 +1047,15 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 			return -EFAULT;
 		break;
 	}
+	case ION_IOC_SYNC:
+	{
+		struct ion_fd_data data;
+		if (copy_from_user(&data, (void __user *)arg,
+				   sizeof(struct ion_fd_data)))
+			return -EFAULT;
+		ion_sync_for_device(client, data.fd);
+		break;
+	}
 	case ION_IOC_CUSTOM:
 	{
 		struct ion_device *dev = client->dev;
drivers/staging/android/ion/ion.h +11 -1
index 88b9a2f..83baa38 100644
@@ -328,7 +328,17 @@ struct ion_custom_data {
  * descriptor obtained from ION_IOC_SHARE and returns the struct with the handle
  * filed set to the corresponding opaque handle.
  */
-#define ION_IOC_IMPORT		_IOWR(ION_IOC_MAGIC, 5, int)
+#define ION_IOC_IMPORT		_IOWR(ION_IOC_MAGIC, 5, struct ion_fd_data)
+
+/**
+ * DOC: ION_IOC_SYNC - syncs a shared file descriptors to memory
+ *
+ * Deprecated in favor of using the dma_buf api's correctly (syncing
+ * will happend automatically when the buffer is mapped to a device).
+ * If necessary should be used after touching a cached buffer from the cpu,
+ * this will make the buffer in memory coherent.
+ */
+#define ION_IOC_SYNC		_IOWR(ION_IOC_MAGIC, 7, struct ion_fd_data)
 
 /**
  * DOC: ION_IOC_CUSTOM - call architecture specific ion ioctl
Signature · 1 line
1.8.3.2
#36 lore raw

PATCH 19/115 gpu: ion: do not ask for compound pages in system heap

In reply to John Stultz

From: Dima Zavin <[email protected]>

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/android/ion/ion_system_heap.c +1 -1
index 98711ce..3f607df 100644
@@ -41,7 +41,7 @@ static struct page_info *alloc_largest_available(unsigned long size)
 	for (i = 0; i < ARRAY_SIZE(orders); i++) {
 		if (size < (1 << orders[i]) * PAGE_SIZE)
 			continue;
-		page = alloc_pages(GFP_HIGHUSER | __GFP_ZERO | __GFP_COMP |
+		page = alloc_pages(GFP_HIGHUSER | __GFP_ZERO |
 				   __GFP_NOWARN | __GFP_NORETRY, orders[i]);
 		if (!page)
 			continue;
Signature · 1 line
1.8.3.2
#37 lore raw

PATCH 20/115 gpu: ion: Add missing argument to WARN call

In reply to John Stultz

From: Olav Haugan <[email protected]>

The condition argument to the WARN call in ion_free and ion_share_dma_buf are missing. Add the argument to allow correct printing of warning message.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/staging/android/ion/ion.c +2 -2
index d4e4c68..5872dd3 100644
@@ -408,7 +408,7 @@ void ion_free(struct ion_client *client, struct ion_handle *handle)
 	mutex_unlock(&client->lock);
 
 	if (!valid_handle) {
-		WARN("%s: invalid handle passed to free.\n", __func__);
+		WARN(1, "%s: invalid handle passed to free.\n", __func__);
 		return;
 	}
 	ion_handle_put(handle);
@@ -906,7 +906,7 @@ int ion_share_dma_buf(struct ion_client *client, struct ion_handle *handle)
 	valid_handle = ion_handle_validate(client, handle);
 	mutex_unlock(&client->lock);
 	if (!valid_handle) {
-		WARN("%s: invalid handle passed to share.\n", __func__);
+		WARN(1, "%s: invalid handle passed to share.\n", __func__);
 		return -EINVAL;
 	}
 
Signature · 1 line
1.8.3.2
#38 lore raw

PATCH 21/115 gpu: ion: Add EXPORT_SYMBOL to functions

In reply to John Stultz

From: Olav Haugan <[email protected]>

Several functions in the ion interface is missing EXPORT_SYMBOL. This is needed to allow clients to use these functions from kernel modules.

Add EXPORT_SYMBOL to functions that are supposed to be exposed.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/staging/android/ion/ion.c +9 -0
index 5872dd3..f536ce5 100644
@@ -396,6 +396,7 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 
 	return handle;
 }
+EXPORT_SYMBOL(ion_alloc);
 
 void ion_free(struct ion_client *client, struct ion_handle *handle)
 {
@@ -413,6 +414,7 @@ void ion_free(struct ion_client *client, struct ion_handle *handle)
 	}
 	ion_handle_put(handle);
 }
+EXPORT_SYMBOL(ion_free);
 
 int ion_phys(struct ion_client *client, struct ion_handle *handle,
 	     ion_phys_addr_t *addr, size_t *len)
@@ -438,6 +440,7 @@ int ion_phys(struct ion_client *client, struct ion_handle *handle,
 	ret = buffer->heap->ops->phys(buffer->heap, buffer, addr, len);
 	return ret;
 }
+EXPORT_SYMBOL(ion_phys);
 
 static void *ion_buffer_kmap_get(struct ion_buffer *buffer)
 {
@@ -517,6 +520,7 @@ void *ion_map_kernel(struct ion_client *client, struct ion_handle *handle)
 	mutex_unlock(&client->lock);
 	return vaddr;
 }
+EXPORT_SYMBOL(ion_map_kernel);
 
 void ion_unmap_kernel(struct ion_client *client, struct ion_handle *handle)
 {
@@ -529,6 +533,7 @@ void ion_unmap_kernel(struct ion_client *client, struct ion_handle *handle)
 	mutex_unlock(&buffer->lock);
 	mutex_unlock(&client->lock);
 }
+EXPORT_SYMBOL(ion_unmap_kernel);
 
 static int ion_debug_client_show(struct seq_file *s, void *unused)
 {
@@ -654,6 +659,7 @@ void ion_client_destroy(struct ion_client *client)
 
 	kfree(client);
 }
+EXPORT_SYMBOL(ion_client_destroy);
 
 struct sg_table *ion_sg_table(struct ion_client *client,
 			      struct ion_handle *handle)
@@ -673,6 +679,7 @@ struct sg_table *ion_sg_table(struct ion_client *client,
 	mutex_unlock(&client->lock);
 	return table;
 }
+EXPORT_SYMBOL(ion_sg_table);
 
 static void ion_buffer_sync_for_device(struct ion_buffer *buffer,
 				       struct device *dev,
@@ -924,6 +931,7 @@ int ion_share_dma_buf(struct ion_client *client, struct ion_handle *handle)
 	}
 	return fd;
 }
+EXPORT_SYMBOL(ion_share_dma_buf);
 
 struct ion_handle *ion_import_dma_buf(struct ion_client *client, int fd)
 {
@@ -960,6 +968,7 @@ end:
 	dma_buf_put(dmabuf);
 	return handle;
 }
+EXPORT_SYMBOL(ion_import_dma_buf);
 
 static int ion_sync_for_device(struct ion_client *client, int fd)
 {
Signature · 1 line
1.8.3.2
#39 lore raw

PATCH 22/115 gpu: ion: IOCTL return success when error occurs

In reply to John Stultz

From: Olav Haugan <[email protected]>

ION_IOC_MAP, ION_IOC_SHARE, and ION_IOC_IMPORT may return success when an error occurs.

Add correct error handling to ION_IOC_MAP, ION_IOC_SHARE, and ION_IOC_IMPORT.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/staging/android/ion/ion.c +8 -1
index f536ce5..a054edd 100644
@@ -1040,20 +1040,27 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 		data.fd = ion_share_dma_buf(client, data.handle);
 		if (copy_to_user((void __user *)arg, &data, sizeof(data)))
 			return -EFAULT;
+		if (data.fd < 0)
+			return data.fd;
 		break;
 	}
 	case ION_IOC_IMPORT:
 	{
 		struct ion_fd_data data;
+		int ret = 0;
 		if (copy_from_user(&data, (void __user *)arg,
 				   sizeof(struct ion_fd_data)))
 			return -EFAULT;
 		data.handle = ion_import_dma_buf(client, data.fd);
-		if (IS_ERR(data.handle))
+		if (IS_ERR(data.handle)) {
+			ret = PTR_ERR(data.handle);
 			data.handle = NULL;
+		}
 		if (copy_to_user((void __user *)arg, &data,
 				 sizeof(struct ion_fd_data)))
 			return -EFAULT;
+		if (ret < 0)
+			return ret;
 		break;
 	}
 	case ION_IOC_SYNC:
Signature · 1 line
1.8.3.2
#40 lore raw

PATCH 23/115 gpu: ion: Don't call ion_buffer_put on error path

In reply to John Stultz

From: Laura Abbott <[email protected]>

If dma_buf_fd fails, the dma_buf needs to be cleaned up by calling dma_buf_put. dma_buf_put will call ion_dma_buf_release which in turn calls ion_buffer_put to clean up the buffer reference. Calling ion_buffer_put after dma_buf_put drops the reference count by one more which is incorrect. Fix this by getting rid of the extra ion_buffer_put call.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/staging/android/ion/ion.c +2 -3
index a054edd..4d91d20 100644
@@ -925,10 +925,9 @@ int ion_share_dma_buf(struct ion_client *client, struct ion_handle *handle)
 		return PTR_ERR(dmabuf);
 	}
 	fd = dma_buf_fd(dmabuf, O_CLOEXEC);
-	if (fd < 0) {
+	if (fd < 0)
 		dma_buf_put(dmabuf);
-		ion_buffer_put(buffer);
-	}
+
 	return fd;
 }
 EXPORT_SYMBOL(ion_share_dma_buf);
Signature · 1 line
1.8.3.2
#41 lore raw

PATCH 24/115 gpu: ion: Only map as much of the vma as the user requested

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/staging/android/ion/ion_system_heap.c +2 -0
index 3f607df..7397bc4 100644
@@ -192,6 +192,8 @@ int ion_system_heap_map_user(struct ion_heap *heap, struct ion_buffer *buffer,
 		remap_pfn_range(vma, addr, page_to_pfn(sg_page(sg)),
 				sg_dma_len(sg), vma->vm_page_prot);
 		addr += sg_dma_len(sg);
+		if (addr >= vma->vm_end)
+			return 0;
 	}
 	return 0;
 }
Signature · 1 line
1.8.3.2
#42 lore raw

PATCH 25/115 gpu: ion: Switch to using kmalloc rather than kmap during allocation

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

Previously, metadata was stored in the allocated pages themselves during allocation. However the system can only have a limited number of kmapped pages. A very large allocation might exceed this limit.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/staging/android/ion/ion_system_heap.c +3 -3
index 7397bc4..2fb9a64 100644
@@ -46,7 +46,7 @@ static struct page_info *alloc_largest_available(unsigned long size)
 		if (!page)
 			continue;
 		split_page(page, orders[i]);
-		info = kmap(page);
+		info = kmalloc(sizeof(struct page_info *), GFP_KERNEL);
 		info->page = page;
 		info->order = orders[i];
 		return info;
@@ -93,7 +93,7 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 		}
 		list_del(&info->list);
 		memset(info, 0, sizeof(struct page_info));
-		kunmap(page);
+		kfree(info);
 	}
 
 	dma_sync_sg_for_device(NULL, table->sgl, table->nents,
@@ -107,7 +107,7 @@ err:
 	list_for_each_entry(info, &pages, list) {
 		for (i = 0; i < (1 << info->order); i++)
 			__free_page(info->page + i);
-		kunmap(info->page);
+		kfree(info);
 	}
 	return -ENOMEM;
 }
Signature · 1 line
1.8.3.2
#43 lore raw

PATCH 26/115 gpu: ion: fix page offset in dma_buf_kmap()

In reply to John Stultz

From: Greg Hackmann <[email protected]>

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/android/ion/ion.c +1 -1
index 4d91d20..84229bd 100644
@@ -846,7 +846,7 @@ static void ion_dma_buf_release(struct dma_buf *dmabuf)
 static void *ion_dma_buf_kmap(struct dma_buf *dmabuf, unsigned long offset)
 {
 	struct ion_buffer *buffer = dmabuf->priv;
-	return buffer->vaddr + offset;
+	return buffer->vaddr + offset * PAGE_SIZE;
 }
 
 static void ion_dma_buf_kunmap(struct dma_buf *dmabuf, unsigned long offset,
Signature · 1 line
1.8.3.2
#44 lore raw

PATCH 27/115 gpu: ion: Fix race between ion_import and ion_free

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

If preemted during ion_free after the refcount is updated but before the handle can be removed from the rb_tree, import might find that handle in the tree and try to reuse it when execution returns to free, the handle will be cleaned up leaving the caller of import with a corrupt handle. This patch modifies the locking to protect agains this race.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
drivers/staging/android/ion/ion.c +1 -4
index 84229bd..f7dc821 100644
@@ -253,8 +253,6 @@ static void ion_handle_destroy(struct kref *kref)
 	struct ion_client *client = handle->client;
 	struct ion_buffer *buffer = handle->buffer;
 
-	mutex_lock(&client->lock);
-
 	mutex_lock(&buffer->lock);
 	while (handle->kmap_cnt)
 		ion_handle_kmap_put(handle);
@@ -262,7 +260,6 @@ static void ion_handle_destroy(struct kref *kref)
 
 	if (!RB_EMPTY_NODE(&handle->node))
 		rb_erase(&handle->node, &client->handles);
-	mutex_unlock(&client->lock);
 
 	ion_buffer_put(buffer);
 	kfree(handle);
@@ -406,13 +403,13 @@ void ion_free(struct ion_client *client, struct ion_handle *handle)
 
 	mutex_lock(&client->lock);
 	valid_handle = ion_handle_validate(client, handle);
-	mutex_unlock(&client->lock);
 
 	if (!valid_handle) {
 		WARN(1, "%s: invalid handle passed to free.\n", __func__);
 		return;
 	}
 	ion_handle_put(handle);
+	mutex_unlock(&client->lock);
 }
 EXPORT_SYMBOL(ion_free);
 
Signature · 1 line
1.8.3.2
#45 lore raw

PATCH 28/115 gpu: ion: Fix bug in ion_free

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 1 +
 1 file changed, 1 insertion(+)
drivers/staging/android/ion/ion.c +1 -0
index f7dc821..e52522c 100644
@@ -406,6 +406,7 @@ void ion_free(struct ion_client *client, struct ion_handle *handle)
 
 	if (!valid_handle) {
 		WARN(1, "%s: invalid handle passed to free.\n", __func__);
+		mutex_unlock(&client->lock);
 		return;
 	}
 	ion_handle_put(handle);
Signature · 1 line
1.8.3.2
#46 lore raw

PATCH 29/115 gpu: ion: Add debug information for orphaned handles

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

It is possible for a buffer to exist only as a dma_buf file descriptor without it being held in any handles. When this occurs it is impossible to track where the buffer is in the system (without traversing every process in the system and inspecting its file table). When buffers are orphaned like this, copy the task comm and pid of the last client to hold them into the buffer so we have a debugging hint as to where this buffer came from. In practice this will probalby be the process that allocated the buffer.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c      | 58 ++++++++++++++++++++++++++++++++++
 drivers/staging/android/ion/ion_priv.h | 14 ++++++++
 2 files changed, 72 insertions(+)
drivers/staging/android/ion/ion.c +58 -0
index e52522c..6ba2c39 100644
@@ -228,6 +228,37 @@ static int ion_buffer_put(struct ion_buffer *buffer)
 	return kref_put(&buffer->ref, ion_buffer_destroy);
 }
 
+static void ion_buffer_add_to_handle(struct ion_buffer *buffer)
+{
+	mutex_lock(&buffer->dev->lock);
+	buffer->handle_count++;
+	mutex_unlock(&buffer->dev->lock);
+}
+
+static void ion_buffer_remove_from_handle(struct ion_buffer *buffer)
+{
+	/*
+	 * when a buffer is removed from a handle, if it is not in
+	 * any other handles, copy the taskcomm and the pid of the
+	 * process it's being removed from into the buffer.  At this
+	 * point there will be no way to track what processes this buffer is
+	 * being used by, it only exists as a dma_buf file descriptor.
+	 * The taskcomm and pid can provide a debug hint as to where this fd
+	 * is in the system
+	 */
+	mutex_lock(&buffer->dev->lock);
+	buffer->handle_count--;
+	BUG_ON(buffer->handle_count < 0);
+	if (!buffer->handle_count) {
+		struct task_struct *task;
+
+		task = current->group_leader;
+		get_task_comm(buffer->task_comm, task);
+		buffer->pid = task_pid_nr(task);
+	}
+	mutex_unlock(&buffer->dev->lock);
+}
+
 static struct ion_handle *ion_handle_create(struct ion_client *client,
 				     struct ion_buffer *buffer)
 {
@@ -240,6 +271,7 @@ static struct ion_handle *ion_handle_create(struct ion_client *client,
 	RB_CLEAR_NODE(&handle->node);
 	handle->client = client;
 	ion_buffer_get(buffer);
+	ion_buffer_add_to_handle(buffer);
 	handle->buffer = buffer;
 
 	return handle;
@@ -261,7 +293,9 @@ static void ion_handle_destroy(struct kref *kref)
 	if (!RB_EMPTY_NODE(&handle->node))
 		rb_erase(&handle->node, &client->handles);
 
+	ion_buffer_remove_from_handle(buffer);
 	ion_buffer_put(buffer);
+
 	kfree(handle);
 }
 
@@ -1141,8 +1175,11 @@ static int ion_debug_heap_show(struct seq_file *s, void *unused)
 	struct ion_heap *heap = s->private;
 	struct ion_device *dev = heap->dev;
 	struct rb_node *n;
+	size_t total_size = 0;
+	size_t total_orphaned_size = 0;
 
 	seq_printf(s, "%16.s %16.s %16.s\n", "client", "pid", "size");
+	seq_printf(s, "----------------------------------------------------\n");
 
 	for (n = rb_first(&dev->clients); n; n = rb_next(n)) {
 		struct ion_client *client = rb_entry(n, struct ion_client,
@@ -1161,6 +1198,27 @@ static int ion_debug_heap_show(struct seq_file *s, void *unused)
 				   client->pid, size);
 		}
 	}
+	seq_printf(s, "----------------------------------------------------\n");
+	seq_printf(s, "orphaned allocations (info is from last known client):"
+		   "\n");
+	mutex_lock(&dev->lock);
+	for (n = rb_first(&dev->buffers); n; n = rb_next(n)) {
+		struct ion_buffer *buffer = rb_entry(n, struct ion_buffer,
+						     node);
+		if (buffer->heap->type == heap->type)
+			total_size += buffer->size;
+		if (!buffer->handle_count) {
+			seq_printf(s, "%16.s %16u %16u\n", buffer->task_comm,
+				   buffer->pid, buffer->size);
+			total_orphaned_size += buffer->size;
+		}
+	}
+	mutex_unlock(&dev->lock);
+	seq_printf(s, "----------------------------------------------------\n");
+	seq_printf(s, "%16.s %16u\n", "total orphaned",
+		   total_orphaned_size);
+	seq_printf(s, "%16.s %16u\n", "total ", total_size);
+
 	return 0;
 }
 
drivers/staging/android/ion/ion_priv.h +14 -0
index 406d1f4..dabe1e8 100644
@@ -21,6 +21,7 @@
 #include <linux/mm_types.h>
 #include <linux/mutex.h>
 #include <linux/rbtree.h>
+#include <linux/sched.h>
 
 #include "ion.h"
 
@@ -43,6 +44,15 @@ struct ion_buffer *ion_handle_buffer(struct ion_handle *handle);
  * @vaddr:		the kenrel mapping if kmap_cnt is not zero
  * @dmap_cnt:		number of times the buffer is mapped for dma
  * @sg_table:		the sg table for the buffer if dmap_cnt is not zero
+ * @dirty:		bitmask representing which pages of this buffer have
+ *			been dirtied by the cpu and need cache maintenance
+ *			before dma
+ * @vmas:		list of vma's mapping this buffer
+ * @handle_count:	count of handles referencing this buffer
+ * @task_comm:		taskcomm of last client to reference this buffer in a
+ *			handle, used for debugging
+ * @pid:		pid of last client to reference this buffer in a
+ *			handle, used for debugging
 */
 struct ion_buffer {
 	struct kref ref;
@@ -62,6 +72,10 @@ struct ion_buffer {
 	struct sg_table *sg_table;
 	unsigned long *dirty;
 	struct list_head vmas;
+	/* used to track orphaned buffers */
+	int handle_count;
+	char task_comm[TASK_COMM_LEN];
+	pid_t pid;
 };
 
 /**
Signature · 1 line
1.8.3.2
#47 lore raw

PATCH 30/115 gpu: ion: Fix memory leak of dirty bits

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)
drivers/staging/android/ion/ion.c +14 -11
index 6ba2c39..3aeb41c 100644
@@ -164,24 +164,20 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 		return ERR_PTR(PTR_ERR(table));
 	}
 	buffer->sg_table = table;
-	if (buffer->flags & ION_FLAG_CACHED)
+	if (buffer->flags & ION_FLAG_CACHED) {
 		for_each_sg(buffer->sg_table->sgl, sg, buffer->sg_table->nents,
 			    i) {
 			if (sg_dma_len(sg) == PAGE_SIZE)
 				continue;
 			pr_err("%s: cached mappings must have pagewise "
 			       "sg_lists\n", __func__);
-			heap->ops->unmap_dma(heap, buffer);
-			kfree(buffer);
-			return ERR_PTR(-EINVAL);
+			ret = -EINVAL;
+			goto err;
 		}
 
-	ret = ion_buffer_alloc_dirty(buffer);
-	if (ret) {
-		heap->ops->unmap_dma(heap, buffer);
-		heap->ops->free(buffer);
-		kfree(buffer);
-		return ERR_PTR(ret);
+		ret = ion_buffer_alloc_dirty(buffer);
+		if (ret)
+			goto err;
 	}
 
 	buffer->dev = dev;
@@ -200,6 +196,12 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 		sg_dma_address(sg) = sg_phys(sg);
 	ion_buffer_add(dev, buffer);
 	return buffer;
+
+err:
+	heap->ops->unmap_dma(heap, buffer);
+	heap->ops->free(buffer);
+	kfree(buffer);
+	return ERR_PTR(ret);
 }
 
 static void ion_buffer_destroy(struct kref *kref)
@@ -209,12 +211,13 @@ static void ion_buffer_destroy(struct kref *kref)
 
 	if (WARN_ON(buffer->kmap_cnt > 0))
 		buffer->heap->ops->unmap_kernel(buffer->heap, buffer);
-
 	buffer->heap->ops->unmap_dma(buffer->heap, buffer);
 	buffer->heap->ops->free(buffer);
 	mutex_lock(&dev->lock);
 	rb_erase(&buffer->node, &dev->buffers);
 	mutex_unlock(&dev->lock);
+	if (buffer->flags & ION_FLAG_CACHED)
+		kfree(buffer->dirty);
 	kfree(buffer);
 }
 
Signature · 1 line
1.8.3.2
#48 lore raw

PATCH 31/115 gpu: ion: Add support for cached mappings that don't fault

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

We have found that faulting in the mappings for cached allocations has a significant performance impact and is only a benefit if only a small part of the buffer is touched by the cpu (an uncommon case for software rendering). This patch introduces a ION_FLAG_CACHED_NEEDS_SYNC which determines whether a mapping should be created by faulting or at mmap time. If this flag is set, userspace must manage the caches explictly using the SYNC ioctl.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 28 ++++++++++++++++++----------
 drivers/staging/android/ion/ion.h |  3 +++
 2 files changed, 21 insertions(+), 10 deletions(-)
drivers/staging/android/ion/ion.c +18 -10
index 3aeb41c..d1c7b84 100644
@@ -164,7 +164,8 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 		return ERR_PTR(PTR_ERR(table));
 	}
 	buffer->sg_table = table;
-	if (buffer->flags & ION_FLAG_CACHED) {
+	if (buffer->flags & ION_FLAG_CACHED &&
+	    !(buffer->flags & ION_FLAG_CACHED_NEEDS_SYNC)) {
 		for_each_sg(buffer->sg_table->sgl, sg, buffer->sg_table->nents,
 			    i) {
 			if (sg_dma_len(sg) == PAGE_SIZE)
@@ -763,7 +764,8 @@ static void ion_buffer_sync_for_device(struct ion_buffer *buffer,
 	pr_debug("%s: syncing for device %s\n", __func__,
 		 dev ? dev_name(dev) : "null");
 
-	if (!(buffer->flags & ION_FLAG_CACHED))
+	if (!(buffer->flags & ION_FLAG_CACHED) ||
+	    (buffer->flags & ION_FLAG_CACHED_NEEDS_SYNC))
 		return;
 
 	mutex_lock(&buffer->lock);
@@ -853,18 +855,22 @@ static int ion_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
 		return -EINVAL;
 	}
 
-	if (buffer->flags & ION_FLAG_CACHED) {
+	if (buffer->flags & ION_FLAG_CACHED &&
+	    !(buffer->flags & ION_FLAG_CACHED_NEEDS_SYNC)) {
 		vma->vm_private_data = buffer;
 		vma->vm_ops = &ion_vma_ops;
 		ion_vm_open(vma);
-	} else {
-		vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
-		mutex_lock(&buffer->lock);
-		/* now map it to userspace */
-		ret = buffer->heap->ops->map_user(buffer->heap, buffer, vma);
-		mutex_unlock(&buffer->lock);
+		return 0;
 	}
 
+	if (!(buffer->flags & ION_FLAG_CACHED))
+		vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
+
+	mutex_lock(&buffer->lock);
+	/* now map it to userspace */
+	ret = buffer->heap->ops->map_user(buffer->heap, buffer, vma);
+	mutex_unlock(&buffer->lock);
+
 	if (ret)
 		pr_err("%s: failure mapping buffer to userspace\n",
 		       __func__);
@@ -1021,7 +1027,9 @@ static int ion_sync_for_device(struct ion_client *client, int fd)
 		return -EINVAL;
 	}
 	buffer = dmabuf->priv;
-	ion_buffer_sync_for_device(buffer, NULL, DMA_BIDIRECTIONAL);
+
+	dma_sync_sg_for_device(NULL, buffer->sg_table->sgl,
+			       buffer->sg_table->nents, DMA_BIDIRECTIONAL);
 	dma_buf_put(dmabuf);
 	return 0;
 }
drivers/staging/android/ion/ion.h +3 -0
index 83baa38..072d64d 100644
@@ -50,6 +50,9 @@ enum ion_heap_type {
 					   cached, ion will do cache
 					   maintenance when the buffer is
 					   mapped for dma */
+#define ION_FLAG_CACHED_NEEDS_SYNC 2	/* mappings of this buffer will created
+					   at mmap time, if this is set
+					   caches must be managed manually */
 
 #ifdef __KERNEL__
 struct ion_device;
Signature · 1 line
1.8.3.2
#49 lore raw

PATCH 32/115 gpu: ion: optimize system heap for non fault buffers

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

If a buffer's user mappings are not going to be faulted in it need not be allocated page wise. We can optimize this common case by allocating an sglist of larger chunks rather than creating an entry for each page in the allocation.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c             | 21 ++++++++------
 drivers/staging/android/ion/ion_priv.h        |  9 ++++++
 drivers/staging/android/ion/ion_system_heap.c | 40 ++++++++++++++++++++-------
 3 files changed, 51 insertions(+), 19 deletions(-)
drivers/staging/android/ion/ion.c +12 -9
index d1c7b84..6c589cb 100644
@@ -100,6 +100,12 @@ struct ion_handle {
 	unsigned int kmap_cnt;
 };
 
+bool ion_buffer_fault_user_mappings(struct ion_buffer *buffer)
+{
+        return ((buffer->flags & ION_FLAG_CACHED) &&
+                !(buffer->flags & ION_FLAG_CACHED_NEEDS_SYNC));
+}
+
 /* this function should only be called while dev->lock is held */
 static void ion_buffer_add(struct ion_device *dev,
 			   struct ion_buffer *buffer)
@@ -145,6 +151,7 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 		return ERR_PTR(-ENOMEM);
 
 	buffer->heap = heap;
+	buffer->flags = flags;
 	kref_init(&buffer->ref);
 
 	ret = heap->ops->allocate(heap, buffer, len, align, flags);
@@ -155,7 +162,6 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 
 	buffer->dev = dev;
 	buffer->size = len;
-	buffer->flags = flags;
 
 	table = heap->ops->map_dma(heap, buffer);
 	if (IS_ERR_OR_NULL(table)) {
@@ -164,14 +170,13 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 		return ERR_PTR(PTR_ERR(table));
 	}
 	buffer->sg_table = table;
-	if (buffer->flags & ION_FLAG_CACHED &&
-	    !(buffer->flags & ION_FLAG_CACHED_NEEDS_SYNC)) {
+	if (ion_buffer_fault_user_mappings(buffer)) {
 		for_each_sg(buffer->sg_table->sgl, sg, buffer->sg_table->nents,
 			    i) {
 			if (sg_dma_len(sg) == PAGE_SIZE)
 				continue;
-			pr_err("%s: cached mappings must have pagewise "
-			       "sg_lists\n", __func__);
+			pr_err("%s: cached mappings that will be faulted in "
+			       "must have pagewise sg_lists\n", __func__);
 			ret = -EINVAL;
 			goto err;
 		}
@@ -764,8 +769,7 @@ static void ion_buffer_sync_for_device(struct ion_buffer *buffer,
 	pr_debug("%s: syncing for device %s\n", __func__,
 		 dev ? dev_name(dev) : "null");
 
-	if (!(buffer->flags & ION_FLAG_CACHED) ||
-	    (buffer->flags & ION_FLAG_CACHED_NEEDS_SYNC))
+	if (!ion_buffer_fault_user_mappings(buffer))
 		return;
 
 	mutex_lock(&buffer->lock);
@@ -855,8 +859,7 @@ static int ion_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
 		return -EINVAL;
 	}
 
-	if (buffer->flags & ION_FLAG_CACHED &&
-	    !(buffer->flags & ION_FLAG_CACHED_NEEDS_SYNC)) {
+	if (ion_buffer_fault_user_mappings(buffer)) {
 		vma->vm_private_data = buffer;
 		vma->vm_ops = &ion_vma_ops;
 		ion_vm_open(vma);
drivers/staging/android/ion/ion_priv.h +9 -0
index dabe1e8..1027ef4 100644
@@ -132,6 +132,15 @@ struct ion_heap {
 };
 
 /**
+ * ion_buffer_fault_user_mappings - fault in user mappings of this buffer
+ * @buffer:		buffer
+ *
+ * indicates whether userspace mappings of this buffer will be faulted
+ * in, this can affect how buffers are allocated from the heap.
+ */
+bool ion_buffer_fault_user_mappings(struct ion_buffer *buffer);
+
+/**
  * ion_device_create - allocates and returns an ion device
  * @custom_ioctl:	arch specific ioctl function if applicable
  *
drivers/staging/android/ion/ion_system_heap.c +30 -10
index 2fb9a64..ef8afc7 100644
@@ -31,7 +31,8 @@ struct page_info {
 	struct list_head list;
 };
 
-static struct page_info *alloc_largest_available(unsigned long size)
+static struct page_info *alloc_largest_available(unsigned long size,
+						 bool split_pages)
 {
 	static unsigned int orders[] = {8, 4, 0};
 	struct page *page;
@@ -45,7 +46,8 @@ static struct page_info *alloc_largest_available(unsigned long size)
 				   __GFP_NOWARN | __GFP_NORETRY, orders[i]);
 		if (!page)
 			continue;
-		split_page(page, orders[i]);
+		if (split_pages)
+			split_page(page, orders[i]);
 		info = kmalloc(sizeof(struct page_info *), GFP_KERNEL);
 		info->page = page;
 		info->order = orders[i];
@@ -64,35 +66,49 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 	int ret;
 	struct list_head pages;
 	struct page_info *info, *tmp_info;
-	int i;
+	int i = 0;
 	long size_remaining = PAGE_ALIGN(size);
+	bool split_pages = ion_buffer_fault_user_mappings(buffer);
+
 
 	INIT_LIST_HEAD(&pages);
 	while (size_remaining > 0) {
-		info = alloc_largest_available(size_remaining);
+		info = alloc_largest_available(size_remaining, split_pages);
 		if (!info)
 			goto err;
 		list_add_tail(&info->list, &pages);
 		size_remaining -= (1 << info->order) * PAGE_SIZE;
+		i++;
 	}
 
 	table = kmalloc(sizeof(struct sg_table), GFP_KERNEL);
 	if (!table)
 		goto err;
 
-	ret = sg_alloc_table(table, PAGE_ALIGN(size) / PAGE_SIZE, GFP_KERNEL);
+	if (split_pages)
+		ret = sg_alloc_table(table, PAGE_ALIGN(size) / PAGE_SIZE,
+				     GFP_KERNEL);
+	else
+		ret = sg_alloc_table(table, i, GFP_KERNEL);
+
 	if (ret)
 		goto err1;
 
 	sg = table->sgl;
 	list_for_each_entry_safe(info, tmp_info, &pages, list) {
 		struct page *page = info->page;
-		for (i = 0; i < (1 << info->order); i++) {
-			sg_set_page(sg, page + i, PAGE_SIZE, 0);
+
+		if (split_pages) {
+			for (i = 0; i < (1 << info->order); i++) {
+				sg_set_page(sg, page + i, PAGE_SIZE, 0);
+				sg = sg_next(sg);
+			}
+		} else {
+			sg_set_page(sg, page, (1 << info->order) * PAGE_SIZE,
+				    0);
 			sg = sg_next(sg);
 		}
 		list_del(&info->list);
-		memset(info, 0, sizeof(struct page_info));
 		kfree(info);
 	}
 
@@ -105,8 +121,12 @@ err1:
 	kfree(table);
 err:
 	list_for_each_entry(info, &pages, list) {
-		for (i = 0; i < (1 << info->order); i++)
-			__free_page(info->page + i);
+		if (split_pages)
+			for (i = 0; i < (1 << info->order); i++)
+				__free_page(info->page + i);
+		else
+			__free_pages(info->page, info->order);
+
 		kfree(info);
 	}
 	return -ENOMEM;
Signature · 1 line
1.8.3.2
#50 lore raw

PATCH 33/115 gpu: ion: Stop trying to allocate from an order on first failure

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

With this patch the system heap will only try to allocate from each order as long as allocations succeed. If it failes to obtain a higher order allocation, it doesn't retry that order.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
drivers/staging/android/ion/ion_system_heap.c +9 -2
index ef8afc7..30a225a 100644
@@ -32,7 +32,8 @@ struct page_info {
 };
 
 static struct page_info *alloc_largest_available(unsigned long size,
-						 bool split_pages)
+						 bool split_pages,
+						 unsigned int max_order)
 {
 	static unsigned int orders[] = {8, 4, 0};
 	struct page *page;
@@ -42,6 +43,8 @@ static struct page_info *alloc_largest_available(unsigned long size,
 	for (i = 0; i < ARRAY_SIZE(orders); i++) {
 		if (size < (1 << orders[i]) * PAGE_SIZE)
 			continue;
+		if (max_order < orders[i])
+			continue;
 		page = alloc_pages(GFP_HIGHUSER | __GFP_ZERO |
 				   __GFP_NOWARN | __GFP_NORETRY, orders[i]);
 		if (!page)
@@ -71,13 +74,17 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 	bool split_pages = ion_buffer_fault_user_mappings(buffer);
 
 
+	unsigned int max_order = orders[0];
+
 	INIT_LIST_HEAD(&pages);
 	while (size_remaining > 0) {
-		info = alloc_largest_available(size_remaining, split_pages);
+		info = alloc_largest_available(size_remaining, split_pages,
+					       max_order);
 		if (!info)
 			goto err;
 		list_add_tail(&info->list, &pages);
 		size_remaining -= (1 << info->order) * PAGE_SIZE;
+		max_order = info->order;
 		i++;
 	}
 
Signature · 1 line
1.8.3.2
#51 lore raw

PATCH 34/115 gpu: ion: ion_system_heap: Fix bug preventing compilation

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/staging/android/ion/ion_system_heap.c +2 -1
index 30a225a..25b0255 100644
@@ -31,11 +31,12 @@ struct page_info {
 	struct list_head list;
 };
 
+static unsigned int orders[] = {8, 4, 0};
+
 static struct page_info *alloc_largest_available(unsigned long size,
 						 bool split_pages,
 						 unsigned int max_order)
 {
-	static unsigned int orders[] = {8, 4, 0};
 	struct page *page;
 	struct page_info *info;
 	int i;
Signature · 1 line
1.8.3.2
#52 lore raw

PATCH 35/115 gpu: ion: use vmalloc to allocate page array to map kernel

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

When ion_map_kernel is execute the system must allocate an array large enough to hold a pointer to each page in the buffer. If the buffer is very large and the system memory has become very fragmented, there may not be sufficient high order allocations available from kmalloc. Use vmalloc instead.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/staging/android/ion/ion_system_heap.c +5 -3
index 25b0255..fcfbb66 100644
@@ -174,10 +174,12 @@ void *ion_system_heap_map_kernel(struct ion_heap *heap,
 	pgprot_t pgprot;
 	struct sg_table *table = buffer->priv_virt;
 	int npages = PAGE_ALIGN(buffer->size) / PAGE_SIZE;
-	struct page **pages = kzalloc(sizeof(struct page *) * npages,
-				     GFP_KERNEL);
+	struct page **pages = vmalloc(sizeof(struct page *) * npages);
 	struct page **tmp = pages;
 
+	if (!pages)
+		return 0;
+
 	if (buffer->flags & ION_FLAG_CACHED)
 		pgprot = PAGE_KERNEL;
 	else
@@ -192,7 +194,7 @@ void *ion_system_heap_map_kernel(struct ion_heap *heap,
 		}
 	}
 	vaddr = vmap(pages, npages, VM_MAP, pgprot);
-	kfree(pages);
+	vfree(pages);
 
 	return vaddr;
 }
Signature · 1 line
1.8.3.2
#53 lore raw

PATCH 36/115 gpu: ion: Add ion_page_pool.

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

This patch adds a new utility heaps can use to manage memory. In the past we have found it can be very expensive to manage the caches when allocating memory, but it is imposible to know whether a previous user of a given memory allocation had a cached mapping. This patch adds the ability to store a pool of pages that were previously used uncached so that cache maintenance only need be done when growing this pool. The pool also contains a shrinker so memory from the pool can be recovered in low memory conditions.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/Makefile        |   3 +-
 drivers/staging/android/ion/ion_page_pool.c | 163 ++++++++++++++++++++++++++++
 drivers/staging/android/ion/ion_priv.h      |  44 ++++++++
 3 files changed, 209 insertions(+), 1 deletion(-)
 create mode 100644 drivers/staging/android/ion/ion_page_pool.c
drivers/staging/android/ion/Makefile +2 -1
index 73fe3fa..d1ddebb 100644
@@ -1,2 +1,3 @@
-obj-$(CONFIG_ION) +=	ion.o ion_heap.o ion_system_heap.o ion_carveout_heap.o
+obj-$(CONFIG_ION) +=	ion.o ion_heap.o ion_page_pool.o ion_system_heap.o \
+			ion_carveout_heap.o
 obj-$(CONFIG_ION_TEGRA) += tegra/
drivers/staging/android/ion/ion_page_pool.c new file +163 -0

163 lines not on this page · raw message

drivers/staging/android/ion/ion_priv.h +44 -0

54 lines not on this page · raw message

Signature · 1 line
1.8.3.2
#54 lore raw

PATCH 37/115 gpu: ion: Use the ion_page_pool from the system heap

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

With this change the system heap will use pagepools to avoid having to invalidate memory when it is allocated, a significant performance improvement on some systems.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c             |  18 ++-
 drivers/staging/android/ion/ion_priv.h        |  11 ++
 drivers/staging/android/ion/ion_system_heap.c | 191 +++++++++++++++++++++-----
 3 files changed, 180 insertions(+), 40 deletions(-)
drivers/staging/android/ion/ion.c +14 -4
index 6c589cb..0070c9b 100644
@@ -106,6 +106,11 @@ bool ion_buffer_fault_user_mappings(struct ion_buffer *buffer)
                 !(buffer->flags & ION_FLAG_CACHED_NEEDS_SYNC));
 }
 
+bool ion_buffer_cached(struct ion_buffer *buffer)
+{
+        return !!(buffer->flags & ION_FLAG_CACHED);
+}
+
 /* this function should only be called while dev->lock is held */
 static void ion_buffer_add(struct ion_device *dev,
 			   struct ion_buffer *buffer)
@@ -1219,11 +1224,12 @@ static int ion_debug_heap_show(struct seq_file *s, void *unused)
 	for (n = rb_first(&dev->buffers); n; n = rb_next(n)) {
 		struct ion_buffer *buffer = rb_entry(n, struct ion_buffer,
 						     node);
-		if (buffer->heap->type == heap->type)
-			total_size += buffer->size;
+		if (buffer->heap->type != heap->type)
+			continue;
+		total_size += buffer->size;
 		if (!buffer->handle_count) {
-			seq_printf(s, "%16.s %16u %16u\n", buffer->task_comm,
-				   buffer->pid, buffer->size);
+			seq_printf(s, "%16.s %16u %16u %d %d\n", buffer->task_comm,
+				   buffer->pid, buffer->size, buffer->kmap_cnt, buffer->ref);
 			total_orphaned_size += buffer->size;
 		}
 	}
@@ -1232,6 +1238,10 @@ static int ion_debug_heap_show(struct seq_file *s, void *unused)
 	seq_printf(s, "%16.s %16u\n", "total orphaned",
 		   total_orphaned_size);
 	seq_printf(s, "%16.s %16u\n", "total ", total_size);
+	seq_printf(s, "----------------------------------------------------\n");
+
+	if (heap->debug_show)
+		heap->debug_show(heap, s, unused);
 
 	return 0;
 }
drivers/staging/android/ion/ion_priv.h +11 -0
index 0707733..9f32634 100644
@@ -118,6 +118,8 @@ struct ion_heap_ops {
  *			allocating.  These are specified by platform data and
  *			MUST be unique
  * @name:		used for debugging
+ * @debug_show:		called when heap debug file is read to add any
+ *			heap specific debug info to output
  *
  * Represents a pool of memory from which buffers can be made.  In some
  * systems the only heap is regular system memory allocated via vmalloc.
@@ -131,9 +133,18 @@ struct ion_heap {
 	struct ion_heap_ops *ops;
 	int id;
 	const char *name;
+	int (*debug_show)(struct ion_heap *heap, struct seq_file *, void *);
 };
 
 /**
+ * ion_buffer_cached - this ion buffer is cached
+ * @buffer:		buffer
+ *
+ * indicates whether this ion buffer is cached
+ */
+bool ion_buffer_cached(struct ion_buffer *buffer);
+
+/**
  * ion_buffer_fault_user_mappings - fault in user mappings of this buffer
  * @buffer:		buffer
  *
drivers/staging/android/ion/ion_system_heap.c +155 -36

277 lines not on this page · raw message

Signature · 1 line
1.8.3.2
#55 lore raw

PATCH 38/115 gpu: ion: Modify gfp flags in ion_system_heap

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

When allocations larger than order 4 are made, use _GFP_NORETRY and __GFP_NO_KSWAPD so kswapd doesn't get kicked off to reclaim these larger chunks. For smaller allocaitons, these are unnecessary, as the system should be able to reclaim these.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 25 ++++++++++++++++++-------
 1 file changed, 18 insertions(+), 7 deletions(-)
drivers/staging/android/ion/ion_system_heap.c +18 -7
index 35fbfff..b99fa0e 100644
@@ -26,6 +26,11 @@
 #include "ion.h"
 #include "ion_priv.h"
 
+static unsigned int high_order_gfp_flags = (GFP_HIGHUSER | __GFP_ZERO |
+					    __GFP_NOWARN | __GFP_NORETRY |
+					    __GFP_NO_KSWAPD);
+static unsigned int low_order_gfp_flags  = (GFP_HIGHUSER | __GFP_ZERO |
+					 __GFP_NOWARN);
 static const unsigned int orders[] = {8, 4, 0};
 static const int num_orders = ARRAY_SIZE(orders);
 static int order_to_index(unsigned int order)
@@ -63,11 +68,15 @@ static struct page *alloc_buffer_page(struct ion_system_heap *heap,
 	struct ion_page_pool *pool = heap->pools[order_to_index(order)];
 	struct page *page;
 
-	if (!cached)
+	if (!cached) {
 		page = ion_page_pool_alloc(pool);
-	else
-		page = alloc_pages(GFP_HIGHUSER | __GFP_ZERO |
-				   __GFP_NOWARN | __GFP_NORETRY, order);
+	} else {
+		gfp_t gfp_flags = low_order_gfp_flags;
+
+		if (order > 4)
+			gfp_flags = high_order_gfp_flags;
+		page = alloc_pages(gfp_flags, order);
+	}
 	if (!page)
 		return 0;
 	if (split_pages)
@@ -342,9 +351,11 @@ struct ion_heap *ion_system_heap_create(struct ion_platform_heap *unused)
 		goto err_alloc_pools;
 	for (i = 0; i < num_orders; i++) {
 		struct ion_page_pool *pool;
-		pool = ion_page_pool_create(GFP_HIGHUSER | __GFP_ZERO |
-					   __GFP_NOWARN | __GFP_NORETRY,
-					   orders[i]);
+		gfp_t gfp_flags = low_order_gfp_flags;
+
+		if (orders[i] > 4)
+			gfp_flags = high_order_gfp_flags;
+		pool = ion_page_pool_create(gfp_flags, orders[i]);
 		if (!pool)
 			goto err_create_pool;
 		heap->pools[i] = pool;
Signature · 1 line
1.8.3.2
#56 lore raw

PATCH 39/115 gpu: ion: Fix several issues with page pool

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

Split out low and high mem pages so they are correctly reported when the shrinker is called.
Fix potential deadlock caused by holding the page pool lock while allocationg and also needing that lock from the shrink function

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_page_pool.c   | 77 ++++++++++++++++++---------
 drivers/staging/android/ion/ion_priv.h        | 12 +++--
 drivers/staging/android/ion/ion_system_heap.c |  9 ++--
 3 files changed, 65 insertions(+), 33 deletions(-)
drivers/staging/android/ion/ion_page_pool.c +51 -26
index 78f39f7..ea45380 100644
@@ -54,24 +54,36 @@ static int ion_page_pool_add(struct ion_page_pool *pool, struct page *page)
 	if (!item)
 		return -ENOMEM;
 	item->page = page;
-	list_add_tail(&item->list, &pool->items);
-	pool->count++;
+	if (PageHighMem(page)) {
+		list_add_tail(&item->list, &pool->high_items);
+		pool->high_count++;
+	} else {
+		list_add_tail(&item->list, &pool->low_items);
+		pool->low_count++;
+	}
 	return 0;
 }
 
-static struct page *ion_page_pool_remove(struct ion_page_pool *pool)
+static struct page *ion_page_pool_remove(struct ion_page_pool *pool, bool high)
 {
 	struct ion_page_pool_item *item;
 	struct page *page;
 
-	BUG_ON(!pool->count);
-	BUG_ON(list_empty(&pool->items));
+	if (high) {
+		BUG_ON(!pool->high_count);
+		item = list_first_entry(&pool->high_items,
+					struct ion_page_pool_item, list);
+		pool->high_count--;
+	} else {
+		BUG_ON(!pool->low_count);
+		item = list_first_entry(&pool->low_items,
+					struct ion_page_pool_item, list);
+		pool->low_count--;
+	}
 
-	item = list_first_entry(&pool->items, struct ion_page_pool_item, list);
 	list_del(&item->list);
 	page = item->page;
 	kfree(item);
-	pool->count--;
 	return page;
 }
 
@@ -82,12 +94,15 @@ void *ion_page_pool_alloc(struct ion_page_pool *pool)
 	BUG_ON(!pool);
 
 	mutex_lock(&pool->mutex);
-	if (pool->count)
-		page = ion_page_pool_remove(pool);
-	else
-		page = ion_page_pool_alloc_pages(pool);
+	if (pool->high_count)
+		page = ion_page_pool_remove(pool, true);
+	else if (pool->low_count)
+		page = ion_page_pool_remove(pool, false);
 	mutex_unlock(&pool->mutex);
 
+	if (!page)
+		page = ion_page_pool_alloc_pages(pool);
+
 	return page;
 }
 
@@ -97,9 +112,9 @@ void ion_page_pool_free(struct ion_page_pool *pool, struct page* page)
 
 	mutex_lock(&pool->mutex);
 	ret = ion_page_pool_add(pool, page);
+	mutex_unlock(&pool->mutex);
 	if (ret)
 		ion_page_pool_free_pages(pool, page);
-	mutex_unlock(&pool->mutex);
 }
 
 static int ion_page_pool_shrink(struct shrinker *shrinker,
@@ -110,30 +125,38 @@ static int ion_page_pool_shrink(struct shrinker *shrinker,
 						 shrinker);
 	int nr_freed = 0;
 	int i;
+	bool high;
+
+	if (sc->gfp_mask & __GFP_HIGHMEM)
+		high = true;
 
 	if (sc->nr_to_scan == 0)
-		return pool->count * (1 << pool->order);
+		return high ? (pool->high_count + pool->low_count) *
+			(1 << pool->order) :
+			pool->low_count * (1 << pool->order);
 
-	mutex_lock(&pool->mutex);
-	for (i = 0; i < sc->nr_to_scan && pool->count; i++) {
-		struct ion_page_pool_item *item;
+	for (i = 0; i < sc->nr_to_scan; i++) {
 		struct page *page;
 
-		item = list_first_entry(&pool->items, struct ion_page_pool_item, list);
-		page = item->page;
-		if (PageHighMem(page) && !(sc->gfp_mask & __GFP_HIGHMEM)) {
-			list_move_tail(&item->list, &pool->items);
-			continue;
+		mutex_lock(&pool->mutex);
+		if (high && pool->high_count) {
+			page = ion_page_pool_remove(pool, true);
+		} else if (pool->low_count) {
+			page = ion_page_pool_remove(pool, false);
+		} else {
+			mutex_unlock(&pool->mutex);
+			break;
 		}
-		BUG_ON(page != ion_page_pool_remove(pool));
+		mutex_unlock(&pool->mutex);
 		ion_page_pool_free_pages(pool, page);
 		nr_freed += (1 << pool->order);
 	}
 	pr_info("%s: shrunk page_pool of order %d by %d pages\n", __func__,
 		pool->order, nr_freed);
-	mutex_unlock(&pool->mutex);
 
-	return pool->count * (1 << pool->order);
+	return high ? (pool->high_count + pool->low_count) *
+		(1 << pool->order) :
+		pool->low_count * (1 << pool->order);
 }
 
 struct ion_page_pool *ion_page_pool_create(gfp_t gfp_mask, unsigned int order)
@@ -142,8 +165,10 @@ struct ion_page_pool *ion_page_pool_create(gfp_t gfp_mask, unsigned int order)
 					     GFP_KERNEL);
 	if (!pool)
 		return NULL;
-	pool->count = 0;
-	INIT_LIST_HEAD(&pool->items);
+	pool->high_count = 0;
+	pool->low_count = 0;
+	INIT_LIST_HEAD(&pool->low_items);
+	INIT_LIST_HEAD(&pool->high_items);
 	pool->shrinker.shrink = ion_page_pool_shrink;
 	pool->shrinker.seeks = DEFAULT_SEEKS * 16;
 	pool->shrinker.batch = 0;
drivers/staging/android/ion/ion_priv.h +8 -4
index 9f32634..beb1595 100644
@@ -217,8 +217,10 @@ void ion_carveout_free(struct ion_heap *heap, ion_phys_addr_t addr,
 
 /**
  * struct ion_page_pool - pagepool struct
- * @count:		number of items in the pool
- * @items:		list of items
+ * @high_count:		number of highmem items in the pool
+ * @low_count:		number of lowmem items in the pool
+ * @high_items:		list of highmem items
+ * @low_items:		list of lowmem items
  * @shrinker:		a shrinker for the items
  * @mutex:		lock protecting this struct and especially the count
  *			item list
@@ -235,8 +237,10 @@ void ion_carveout_free(struct ion_heap *heap, ion_phys_addr_t addr,
  * on many systems
  */
 struct ion_page_pool {
-	int count;
-	struct list_head items;
+	int high_count;
+	int low_count;
+	struct list_head high_items;
+	struct list_head low_items;
 	struct shrinker shrinker;
 	struct mutex mutex;
 	void *(*alloc)(struct ion_page_pool *pool);
drivers/staging/android/ion/ion_system_heap.c +6 -3
index b99fa0e..2204206 100644
@@ -328,9 +328,12 @@ static int ion_system_heap_debug_show(struct ion_heap *heap, struct seq_file *s,
 	int i;
 	for (i = 0; i < num_orders; i++) {
 		struct ion_page_pool *pool = sys_heap->pools[i];
-		seq_printf(s, "%d order %u pages in pool = %lu total\n",
-			   pool->count, pool->order,
-			   (1 << pool->order) * PAGE_SIZE * pool->count);
+		seq_printf(s, "%d order %u highmem pages in pool = %lu total\n",
+			   pool->high_count, pool->order,
+			   (1 << pool->order) * PAGE_SIZE * pool->high_count);
+		seq_printf(s, "%d order %u lowmem pages in pool = %lu total\n",
+			   pool->low_count, pool->order,
+			   (1 << pool->order) * PAGE_SIZE * pool->low_count);
 	}
 	return 0;
 }
Signature · 1 line
1.8.3.2
#57 lore raw

PATCH 40/115 gpu: ion: Fix lockdep issue in ion_page_pool

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

Currently the mutex is held while kmalloc is called, under a low memory condition this might trigger the shrinker which also takes this mutex. Refactor so the mutex is not held during allocation.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_page_pool.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/staging/android/ion/ion_page_pool.c +3 -2
index ea45380..20790eb 100644
@@ -53,6 +53,8 @@ static int ion_page_pool_add(struct ion_page_pool *pool, struct page *page)
 	item = kmalloc(sizeof(struct ion_page_pool_item), GFP_KERNEL);
 	if (!item)
 		return -ENOMEM;
+
+	mutex_lock(&pool->mutex);
 	item->page = page;
 	if (PageHighMem(page)) {
 		list_add_tail(&item->list, &pool->high_items);
@@ -61,6 +63,7 @@ static int ion_page_pool_add(struct ion_page_pool *pool, struct page *page)
 		list_add_tail(&item->list, &pool->low_items);
 		pool->low_count++;
 	}
+	mutex_unlock(&pool->mutex);
 	return 0;
 }
 
@@ -110,9 +113,7 @@ void ion_page_pool_free(struct ion_page_pool *pool, struct page* page)
 {
 	int ret;
 
-	mutex_lock(&pool->mutex);
 	ret = ion_page_pool_add(pool, page);
-	mutex_unlock(&pool->mutex);
 	if (ret)
 		ion_page_pool_free_pages(pool, page);
 }
Signature · 1 line
1.8.3.2
#58 lore raw

PATCH 41/115 gpu: ion: Switch to using a single shrink function

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

The single shrink function will free lower order pages first. This enables compaction to work properly.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_page_pool.c | 150 ++++++++++++++++++++++------
 drivers/staging/android/ion/ion_priv.h      |   3 +-
 2 files changed, 123 insertions(+), 30 deletions(-)
drivers/staging/android/ion/ion_page_pool.c +121 -29
index 20790eb..e59da3a 100644
@@ -14,13 +14,21 @@
  *
  */
 
+#include <linux/debugfs.h>
 #include <linux/dma-mapping.h>
 #include <linux/err.h>
+#include <linux/fs.h>
 #include <linux/list.h>
+#include <linux/module.h>
 #include <linux/slab.h>
 #include <linux/shrinker.h>
 #include "ion_priv.h"
 
+/* #define DEBUG_PAGE_POOL_SHRINKER */
+
+static struct plist_head pools = PLIST_HEAD_INIT(pools);
+static struct shrinker shrinker;
+
 struct ion_page_pool_item {
 	struct page *page;
 	struct list_head list;
@@ -118,46 +126,110 @@ void ion_page_pool_free(struct ion_page_pool *pool, struct page* page)
 		ion_page_pool_free_pages(pool, page);
 }
 
+#ifdef DEBUG_PAGE_POOL_SHRINKER
+static int debug_drop_pools_set(void *data, u64 val)
+{
+	struct shrink_control sc;
+	int objs;
+
+	sc.gfp_mask = -1;
+	sc.nr_to_scan = 0;
+
+	if (!val)
+		return 0;
+
+	objs = shrinker.shrink(&shrinker, &sc);
+	sc.nr_to_scan = objs;
+
+	shrinker.shrink(&shrinker, &sc);
+	return 0;
+}
+
+static int debug_drop_pools_get(void *data, u64 *val)
+{
+	struct shrink_control sc;
+	int objs;
+
+	sc.gfp_mask = -1;
+	sc.nr_to_scan = 0;
+
+	objs = shrinker.shrink(&shrinker, &sc);
+	*val = objs;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(debug_drop_pools_fops, debug_drop_pools_get,
+                        debug_drop_pools_set, "%llu\n");
+
+static int debug_grow_pools_set(void *data, u64 val)
+{
+	struct ion_page_pool *pool;
+	struct page *page;
+
+	plist_for_each_entry(pool, &pools, list) {
+		if (val != pool->list.prio)
+			continue;
+		page = ion_page_pool_alloc_pages(pool);
+		if (page)
+			ion_page_pool_add(pool, page);
+	}
+
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(debug_grow_pools_fops, debug_drop_pools_get,
+			debug_grow_pools_set, "%llu\n");
+#endif
+
+static int ion_page_pool_total(bool high)
+{
+	struct ion_page_pool *pool;
+	int total = 0;
+
+	plist_for_each_entry(pool, &pools, list) {
+		total += high ? (pool->high_count + pool->low_count) *
+			(1 << pool->order) :
+			pool->low_count * (1 << pool->order);
+	}
+	return total;
+}
+
 static int ion_page_pool_shrink(struct shrinker *shrinker,
 				 struct shrink_control *sc)
 {
-	struct ion_page_pool *pool = container_of(shrinker,
-						 struct ion_page_pool,
-						 shrinker);
+	struct ion_page_pool *pool;
 	int nr_freed = 0;
 	int i;
 	bool high;
+	int nr_to_scan = sc->nr_to_scan;
 
 	if (sc->gfp_mask & __GFP_HIGHMEM)
 		high = true;
 
-	if (sc->nr_to_scan == 0)
-		return high ? (pool->high_count + pool->low_count) *
-			(1 << pool->order) :
-			pool->low_count * (1 << pool->order);
+	if (nr_to_scan == 0)
+		return ion_page_pool_total(high);
 
-	for (i = 0; i < sc->nr_to_scan; i++) {
-		struct page *page;
+	plist_for_each_entry(pool, &pools, list) {
+		for (i = 0; i < nr_to_scan; i++) {
+			struct page *page;
 
-		mutex_lock(&pool->mutex);
-		if (high && pool->high_count) {
-			page = ion_page_pool_remove(pool, true);
-		} else if (pool->low_count) {
-			page = ion_page_pool_remove(pool, false);
-		} else {
+			mutex_lock(&pool->mutex);
+			if (high && pool->high_count) {
+				page = ion_page_pool_remove(pool, true);
+			} else if (pool->low_count) {
+				page = ion_page_pool_remove(pool, false);
+			} else {
+				mutex_unlock(&pool->mutex);
+				break;
+			}
 			mutex_unlock(&pool->mutex);
-			break;
+			ion_page_pool_free_pages(pool, page);
+			nr_freed += (1 << pool->order);
 		}
-		mutex_unlock(&pool->mutex);
-		ion_page_pool_free_pages(pool, page);
-		nr_freed += (1 << pool->order);
+		nr_to_scan -= i;
 	}
-	pr_info("%s: shrunk page_pool of order %d by %d pages\n", __func__,
-		pool->order, nr_freed);
 
-	return high ? (pool->high_count + pool->low_count) *
-		(1 << pool->order) :
-		pool->low_count * (1 << pool->order);
+	return ion_page_pool_total(high);
 }
 
 struct ion_page_pool *ion_page_pool_create(gfp_t gfp_mask, unsigned int order)
@@ -170,20 +242,40 @@ struct ion_page_pool *ion_page_pool_create(gfp_t gfp_mask, unsigned int order)
 	pool->low_count = 0;
 	INIT_LIST_HEAD(&pool->low_items);
 	INIT_LIST_HEAD(&pool->high_items);
-	pool->shrinker.shrink = ion_page_pool_shrink;
-	pool->shrinker.seeks = DEFAULT_SEEKS * 16;
-	pool->shrinker.batch = 0;
-	register_shrinker(&pool->shrinker);
 	pool->gfp_mask = gfp_mask;
 	pool->order = order;
 	mutex_init(&pool->mutex);
+	plist_node_init(&pool->list, order);
+	plist_add(&pool->list, &pools);
 
 	return pool;
 }
 
 void ion_page_pool_destroy(struct ion_page_pool *pool)
 {
-	unregister_shrinker(&pool->shrinker);
+	plist_del(&pool->list, &pools);
 	kfree(pool);
 }
 
+static int __init ion_page_pool_init(void)
+{
+	shrinker.shrink = ion_page_pool_shrink;
+	shrinker.seeks = DEFAULT_SEEKS;
+	shrinker.batch = 0;
+	register_shrinker(&shrinker);
+#ifdef DEBUG_PAGE_POOL_SHRINKER
+	debugfs_create_file("ion_pools_shrink", 0644, NULL, NULL,
+			    &debug_drop_pools_fops);
+	debugfs_create_file("ion_pools_grow", 0644, NULL, NULL,
+			    &debug_grow_pools_fops);
+#endif
+	return 0;
+}
+
+static void __exit ion_page_pool_exit(void)
+{
+	unregister_shrinker(&shrinker);
+}
+
+module_init(ion_page_pool_init);
+module_exit(ion_page_pool_exit);
drivers/staging/android/ion/ion_priv.h +2 -1
index beb1595..77e2ac0 100644
@@ -230,6 +230,7 @@ void ion_carveout_free(struct ion_heap *heap, ion_phys_addr_t addr,
  *			when the shrinker fires
  * @gfp_mask:		gfp_mask to use from alloc
  * @order:		order of pages in the pool
+ * @list:		plist node for list of pools
  *
  * Allows you to keep a pool of pre allocated pages to use from your heap.
  * Keeping a pool of pages that is ready for dma, ie any cached mapping have
@@ -241,12 +242,12 @@ struct ion_page_pool {
 	int low_count;
 	struct list_head high_items;
 	struct list_head low_items;
-	struct shrinker shrinker;
 	struct mutex mutex;
 	void *(*alloc)(struct ion_page_pool *pool);
 	void (*free)(struct ion_page_pool *pool, struct page *page);
 	gfp_t gfp_mask;
 	unsigned int order;
+	struct plist_node list;
 };
 
 struct ion_page_pool *ion_page_pool_create(gfp_t gfp_mask, unsigned int order);
Signature · 1 line
1.8.3.2
#59 lore raw

PATCH 42/115 gpu: ion: Refactor locking

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

Removes contention for lock between allocate and free by reducing the length of time the lock is held for. Split out a seperate lock to protect the list of heaps and replace it with a rwsem since the list will most likely only be updated during initialization.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 45 ++++++++++++++++++++++-----------------
 1 file changed, 25 insertions(+), 20 deletions(-)
drivers/staging/android/ion/ion.c +25 -20
index 0070c9b..ef8f132 100644
@@ -38,15 +38,17 @@
 /**
  * struct ion_device - the metadata of the ion device node
  * @dev:		the actual misc device
- * @buffers:	an rb tree of all the existing buffers
- * @lock:		lock protecting the buffers & heaps trees
+ * @buffers:		an rb tree of all the existing buffers
+ * @buffer_lock:	lock protecting the tree of buffers
+ * @lock:		rwsem protecting the tree of heaps and clients
  * @heaps:		list of all the heaps in the system
  * @user_clients:	list of all the clients created from userspace
  */
 struct ion_device {
 	struct miscdevice dev;
 	struct rb_root buffers;
-	struct mutex lock;
+	struct mutex buffer_lock;
+	struct rw_semaphore lock;
 	struct rb_root heaps;
 	long (*custom_ioctl) (struct ion_client *client, unsigned int cmd,
 			      unsigned long arg);
@@ -205,7 +207,9 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 	   cached mapping that mapping has been invalidated */
 	for_each_sg(buffer->sg_table->sgl, sg, buffer->sg_table->nents, i)
 		sg_dma_address(sg) = sg_phys(sg);
+	mutex_lock(&dev->buffer_lock);
 	ion_buffer_add(dev, buffer);
+	mutex_unlock(&dev->buffer_lock);
 	return buffer;
 
 err:
@@ -224,9 +228,9 @@ static void ion_buffer_destroy(struct kref *kref)
 		buffer->heap->ops->unmap_kernel(buffer->heap, buffer);
 	buffer->heap->ops->unmap_dma(buffer->heap, buffer);
 	buffer->heap->ops->free(buffer);
-	mutex_lock(&dev->lock);
+	mutex_lock(&dev->buffer_lock);
 	rb_erase(&buffer->node, &dev->buffers);
-	mutex_unlock(&dev->lock);
+	mutex_unlock(&dev->buffer_lock);
 	if (buffer->flags & ION_FLAG_CACHED)
 		kfree(buffer->dirty);
 	kfree(buffer);
@@ -244,9 +248,9 @@ static int ion_buffer_put(struct ion_buffer *buffer)
 
 static void ion_buffer_add_to_handle(struct ion_buffer *buffer)
 {
-	mutex_lock(&buffer->dev->lock);
+	mutex_lock(&buffer->lock);
 	buffer->handle_count++;
-	mutex_unlock(&buffer->dev->lock);
+	mutex_unlock(&buffer->lock);
 }
 
 static void ion_buffer_remove_from_handle(struct ion_buffer *buffer)
@@ -260,7 +264,7 @@ static void ion_buffer_remove_from_handle(struct ion_buffer *buffer)
 	 * The taskcomm and pid can provide a debug hint as to where this fd
 	 * is in the system
 	 */
-	mutex_lock(&buffer->dev->lock);
+	mutex_lock(&buffer->lock);
 	buffer->handle_count--;
 	BUG_ON(buffer->handle_count < 0);
 	if (!buffer->handle_count) {
@@ -270,7 +274,7 @@ static void ion_buffer_remove_from_handle(struct ion_buffer *buffer)
 		get_task_comm(buffer->task_comm, task);
 		buffer->pid = task_pid_nr(task);
 	}
-	mutex_unlock(&buffer->dev->lock);
+	mutex_unlock(&buffer->lock);
 }
 
 static struct ion_handle *ion_handle_create(struct ion_client *client,
@@ -403,7 +407,7 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 
 	len = PAGE_ALIGN(len);
 
-	mutex_lock(&dev->lock);
+	down_read(&dev->lock);
 	for (n = rb_first(&dev->heaps); n != NULL; n = rb_next(n)) {
 		struct ion_heap *heap = rb_entry(n, struct ion_heap, node);
 		/* if the client doesn't support this heap type */
@@ -416,7 +420,7 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 		if (!IS_ERR_OR_NULL(buffer))
 			break;
 	}
-	mutex_unlock(&dev->lock);
+	up_read(&dev->lock);
 
 	if (buffer == NULL)
 		return ERR_PTR(-ENODEV);
@@ -662,7 +666,7 @@ struct ion_client *ion_client_create(struct ion_device *dev,
 	client->task = task;
 	client->pid = pid;
 
-	mutex_lock(&dev->lock);
+	down_write(&dev->lock);
 	p = &dev->clients.rb_node;
 	while (*p) {
 		parent = *p;
@@ -680,7 +684,7 @@ struct ion_client *ion_client_create(struct ion_device *dev,
 	client->debug_root = debugfs_create_file(debug_name, 0664,
 						 dev->debug_root, client,
 						 &debug_client_fops);
-	mutex_unlock(&dev->lock);
+	up_write(&dev->lock);
 
 	return client;
 }
@@ -696,12 +700,12 @@ void ion_client_destroy(struct ion_client *client)
 						     node);
 		ion_handle_destroy(&handle->ref);
 	}
-	mutex_lock(&dev->lock);
+	down_write(&dev->lock);
 	if (client->task)
 		put_task_struct(client->task);
 	rb_erase(&client->node, &dev->clients);
 	debugfs_remove_recursive(client->debug_root);
-	mutex_unlock(&dev->lock);
+	up_write(&dev->lock);
 
 	kfree(client);
 }
@@ -1220,7 +1224,7 @@ static int ion_debug_heap_show(struct seq_file *s, void *unused)
 	seq_printf(s, "----------------------------------------------------\n");
 	seq_printf(s, "orphaned allocations (info is from last known client):"
 		   "\n");
-	mutex_lock(&dev->lock);
+	mutex_lock(&dev->buffer_lock);
 	for (n = rb_first(&dev->buffers); n; n = rb_next(n)) {
 		struct ion_buffer *buffer = rb_entry(n, struct ion_buffer,
 						     node);
@@ -1233,7 +1237,7 @@ static int ion_debug_heap_show(struct seq_file *s, void *unused)
 			total_orphaned_size += buffer->size;
 		}
 	}
-	mutex_unlock(&dev->lock);
+	mutex_unlock(&dev->buffer_lock);
 	seq_printf(s, "----------------------------------------------------\n");
 	seq_printf(s, "%16.s %16u\n", "total orphaned",
 		   total_orphaned_size);
@@ -1270,7 +1274,7 @@ void ion_device_add_heap(struct ion_device *dev, struct ion_heap *heap)
 		       __func__);
 
 	heap->dev = dev;
-	mutex_lock(&dev->lock);
+	down_write(&dev->lock);
 	while (*p) {
 		parent = *p;
 		entry = rb_entry(parent, struct ion_heap, node);
@@ -1291,7 +1295,7 @@ void ion_device_add_heap(struct ion_device *dev, struct ion_heap *heap)
 	debugfs_create_file(heap->name, 0664, dev->debug_root, heap,
 			    &debug_heap_fops);
 end:
-	mutex_unlock(&dev->lock);
+	up_write(&dev->lock);
 }
 
 struct ion_device *ion_device_create(long (*custom_ioctl)
@@ -1322,7 +1326,8 @@ struct ion_device *ion_device_create(long (*custom_ioctl)
 
 	idev->custom_ioctl = custom_ioctl;
 	idev->buffers = RB_ROOT;
-	mutex_init(&idev->lock);
+	mutex_init(&idev->buffer_lock);
+	init_rwsem(&idev->lock);
 	idev->heaps = RB_ROOT;
 	idev->clients = RB_ROOT;
 	return idev;
Signature · 1 line
1.8.3.2
#60 lore raw

PATCH 43/115 gpu: ion: Clear GFP_WAIT flag on high order allocations

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

This will prevent the kernel from kicking off compaction when higher order allocations are made. Instead we will get these high order allocations only if they are readily available.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/android/ion/ion_system_heap.c +1 -1
index 2204206..ad4933e 100644
@@ -28,7 +28,7 @@
 
 static unsigned int high_order_gfp_flags = (GFP_HIGHUSER | __GFP_ZERO |
 					    __GFP_NOWARN | __GFP_NORETRY |
-					    __GFP_NO_KSWAPD);
+					    __GFP_NO_KSWAPD) & ~__GFP_WAIT;
 static unsigned int low_order_gfp_flags  = (GFP_HIGHUSER | __GFP_ZERO |
 					 __GFP_NOWARN);
 static const unsigned int orders[] = {8, 4, 0};
Signature · 1 line
1.8.3.2
#61 lore raw

PATCH 44/115 gpu: ion: Don't flush allocatoins that come from the page pools

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/staging/android/ion/ion_system_heap.c +5 -3
index ad4933e..02726a9 100644
@@ -76,9 +76,14 @@ static struct page *alloc_buffer_page(struct ion_system_heap *heap,
 		if (order > 4)
 			gfp_flags = high_order_gfp_flags;
 		page = alloc_pages(gfp_flags, order);
+		if (!page)
+			return 0;
+		__dma_page_cpu_to_dev(page, 0, PAGE_SIZE << order,
+				      DMA_BIDIRECTIONAL);
 	}
 	if (!page)
 		return 0;
+
 	if (split_pages)
 		split_page(page, order);
 	return page;
@@ -201,9 +206,6 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 		kfree(info);
 	}
 
-	dma_sync_sg_for_device(NULL, table->sgl, table->nents,
-			       DMA_BIDIRECTIONAL);
-
 	buffer->priv_virt = table;
 	return 0;
 err1:
Signature · 1 line
1.8.3.2
#62 lore raw

PATCH 45/115 gpu: ion: Fix bug in ion_system_heap map_user

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

When the requested mmap length was not an integer number of chunks or the buffer, or if an offset was provided, a bug would cause extra or incorrect pages of the buffer to be mapped.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)
drivers/staging/android/ion/ion_system_heap.c +15 -6
index 02726a9..df08469 100644
@@ -292,18 +292,27 @@ int ion_system_heap_map_user(struct ion_heap *heap, struct ion_buffer *buffer,
 {
 	struct sg_table *table = buffer->priv_virt;
 	unsigned long addr = vma->vm_start;
-	unsigned long offset = vma->vm_pgoff;
+	unsigned long offset = vma->vm_pgoff * PAGE_SIZE;
 	struct scatterlist *sg;
 	int i;
 
 	for_each_sg(table->sgl, sg, table->nents, i) {
-		if (offset) {
-			offset--;
+		struct page *page = sg_page(sg);
+		unsigned long remainder = vma->vm_end - addr;
+		unsigned long len = sg_dma_len(sg);
+
+		if (offset >= sg_dma_len(sg)) {
+			offset -= sg_dma_len(sg);
 			continue;
+		} else if (offset) {
+			page += offset / PAGE_SIZE;
+			len = sg_dma_len(sg) - offset;
+			offset = 0;
 		}
-		remap_pfn_range(vma, addr, page_to_pfn(sg_page(sg)),
-				sg_dma_len(sg), vma->vm_page_prot);
-		addr += sg_dma_len(sg);
+		len = min(len, remainder);
+		remap_pfn_range(vma, addr, page_to_pfn(page), len,
+				vma->vm_page_prot);
+		addr += len;
 		if (addr >= vma->vm_end)
 			return 0;
 	}
Signature · 1 line
1.8.3.2
#63 lore raw

PATCH 46/115 gpu: ion: Fix bug in zeroing pages in system heap

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

Pages are zeroed for security purposes when returned to the ion heap. There was a bug in this code preventing this from happening.

Bug 7573871
[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/android/ion/ion_system_heap.c +1 -1
index df08469..9327a32 100644
@@ -103,7 +103,7 @@ static void free_buffer_page(struct ion_system_heap *heap,
 		   security.  This uses vmap as we want to set the pgprot so
 		   the writes to occur to noncached mappings, as the pool's
 		   purpose is to keep the pages out of the cache */
-		for (i = 0; i < order / PAGE_SIZE; i++) {
+		for (i = 0; i < (1 << order); i++) {
 			struct page *sub_page = page + i;
 			void *addr = vmap(&sub_page, 1, VM_MAP,
 					  pgprot_writecombine(PAGE_KERNEL));
Signature · 1 line
1.8.3.2
#64 lore raw

PATCH 47/115 gpu: ion: fix carveout ops

In reply to John Stultz

From: Benjamin Gaignard <[email protected]>

when using carveout heap ion_buffer_create function failed because map_dma and unmap_dma operations aren't set by carveout heap.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_carveout_heap.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/staging/android/ion/ion_carveout_heap.c +2 -0
index daa348e..fd64168 100644
@@ -142,6 +142,8 @@ static struct ion_heap_ops carveout_heap_ops = {
 	.allocate = ion_carveout_heap_allocate,
 	.free = ion_carveout_heap_free,
 	.phys = ion_carveout_heap_phys,
+	.map_dma = ion_carveout_heap_map_dma,
+	.unmap_dma = ion_carveout_heap_unmap_dma,
 	.map_user = ion_carveout_heap_map_user,
 	.map_kernel = ion_carveout_heap_map_kernel,
 	.unmap_kernel = ion_carveout_heap_unmap_kernel,
Signature · 1 line
1.8.3.2
#65 lore raw

PATCH 48/115 gpu: ion: fix compilation warning

In reply to John Stultz

From: Benjamin Gaignard <[email protected]>

use atomic_read to get the refcount value to avoid compilation warning

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/staging/android/ion/ion.c +2 -1
index ef8f132..e0eb2de 100644
@@ -1233,7 +1233,8 @@ static int ion_debug_heap_show(struct seq_file *s, void *unused)
 		total_size += buffer->size;
 		if (!buffer->handle_count) {
 			seq_printf(s, "%16.s %16u %16u %d %d\n", buffer->task_comm,
-				   buffer->pid, buffer->size, buffer->kmap_cnt, buffer->ref);
+				   buffer->pid, buffer->size, buffer->kmap_cnt,
+				   atomic_read(&buffer->ref.refcount));
 			total_orphaned_size += buffer->size;
 		}
 	}
Signature · 1 line
1.8.3.2
#66 lore raw

PATCH 49/115 gpu: ion: Modify reserve function for carveouts with no start address

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

This patch allows you to specify a heap that requires carveout memory but that doesn't specify a start address. Memblock_alloc will be called to find a location for these heaps.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 33 ++++++++++++++++++++++++++-------
 1 file changed, 26 insertions(+), 7 deletions(-)
drivers/staging/android/ion/ion.c +26 -7
index e0eb2de..3167c1d 100644
@@ -1343,16 +1343,35 @@ void ion_device_destroy(struct ion_device *dev)
 
 void __init ion_reserve(struct ion_platform_data *data)
 {
-	int i, ret;
+	int i;
 
 	for (i = 0; i < data->nr; i++) {
 		if (data->heaps[i].size == 0)
 			continue;
-		ret = memblock_reserve(data->heaps[i].base,
-				       data->heaps[i].size);
-		if (ret)
-			pr_err("memblock reserve of %x@%lx failed\n",
-			       data->heaps[i].size,
-			       data->heaps[i].base);
+
+		if (data->heaps[i].base == 0) {
+			phys_addr_t paddr;
+			paddr = memblock_alloc_base(data->heaps[i].size,
+						    data->heaps[i].align,
+						    MEMBLOCK_ALLOC_ANYWHERE);
+			if (!paddr) {
+				pr_err("%s: error allocating memblock for "
+				       "heap %d\n",
+					__func__, i);
+				continue;
+			}
+			data->heaps[i].base = paddr;
+		} else {
+			int ret = memblock_reserve(data->heaps[i].base,
+					       data->heaps[i].size);
+			if (ret)
+				pr_err("memblock reserve of %x@%lx failed\n",
+				       data->heaps[i].size,
+				       data->heaps[i].base);
+		}
+		pr_info("%s: %s reserved base %lx size %d\n", __func__,
+			data->heaps[i].name,
+			data->heaps[i].base,
+			data->heaps[i].size);
 	}
 }
Signature · 1 line
1.8.3.2
#67 lore raw

PATCH 50/115 gpu: ion: Fix bug where MAP ioctl was no longer supported

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 1 +
 1 file changed, 1 insertion(+)
drivers/staging/android/ion/ion.c +1 -0
index 3167c1d..3f22f88 100644
@@ -1086,6 +1086,7 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 		break;
 	}
 	case ION_IOC_SHARE:
+	case ION_IOC_MAP:
 	{
 		struct ion_fd_data data;
 
Signature · 1 line
1.8.3.2
#68 lore raw

PATCH 51/115 gpu: ion: Switch heap rbtree to a prio list

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

Switches the rbtree tree of heaps for a plist. This significantly simplifies the code and the list is small and is modified only at first boot so the rbtree is unnecessary. This also switches the traversal of the heap list to traverse from highest to lowest id's. This allows allocations to pass a heap mask that falls back on the system heap -- typically id 0, which is the common case.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c      | 35 ++++++++--------------------------
 drivers/staging/android/ion/ion_priv.h |  4 ++--
 2 files changed, 10 insertions(+), 29 deletions(-)
drivers/staging/android/ion/ion.c +8 -27
index 3f22f88..8cc0489 100644
@@ -49,7 +49,7 @@ struct ion_device {
 	struct rb_root buffers;
 	struct mutex buffer_lock;
 	struct rw_semaphore lock;
-	struct rb_root heaps;
+	struct plist_head heaps;
 	long (*custom_ioctl) (struct ion_client *client, unsigned int cmd,
 			      unsigned long arg);
 	struct rb_root clients;
@@ -389,10 +389,10 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 			     size_t align, unsigned int heap_mask,
 			     unsigned int flags)
 {
-	struct rb_node *n;
 	struct ion_handle *handle;
 	struct ion_device *dev = client->dev;
 	struct ion_buffer *buffer = NULL;
+	struct ion_heap *heap;
 
 	pr_debug("%s: len %d align %d heap_mask %u flags %x\n", __func__, len,
 		 align, heap_mask, flags);
@@ -408,8 +408,7 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 	len = PAGE_ALIGN(len);
 
 	down_read(&dev->lock);
-	for (n = rb_first(&dev->heaps); n != NULL; n = rb_next(n)) {
-		struct ion_heap *heap = rb_entry(n, struct ion_heap, node);
+	plist_for_each_entry(heap, &dev->heaps, node) {
 		/* if the client doesn't support this heap type */
 		if (!((1 << heap->type) & client->heap_mask))
 			continue;
@@ -1266,10 +1265,6 @@ static const struct file_operations debug_heap_fops = {
 
 void ion_device_add_heap(struct ion_device *dev, struct ion_heap *heap)
 {
-	struct rb_node **p = &dev->heaps.rb_node;
-	struct rb_node *parent = NULL;
-	struct ion_heap *entry;
-
 	if (!heap->ops->allocate || !heap->ops->free || !heap->ops->map_dma ||
 	    !heap->ops->unmap_dma)
 		pr_err("%s: can not add heap with invalid ops struct.\n",
@@ -1277,26 +1272,12 @@ void ion_device_add_heap(struct ion_device *dev, struct ion_heap *heap)
 
 	heap->dev = dev;
 	down_write(&dev->lock);
-	while (*p) {
-		parent = *p;
-		entry = rb_entry(parent, struct ion_heap, node);
-
-		if (heap->id < entry->id) {
-			p = &(*p)->rb_left;
-		} else if (heap->id > entry->id ) {
-			p = &(*p)->rb_right;
-		} else {
-			pr_err("%s: can not insert multiple heaps with "
-				"id %d\n", __func__, heap->id);
-			goto end;
-		}
-	}
-
-	rb_link_node(&heap->node, parent, p);
-	rb_insert_color(&heap->node, &dev->heaps);
+	/* use negative heap->id to reverse the priority -- when traversing
+	   the list later attempt higher id numbers first */
+	plist_node_init(&heap->node, -heap->id);
+	plist_add(&heap->node, &dev->heaps);
 	debugfs_create_file(heap->name, 0664, dev->debug_root, heap,
 			    &debug_heap_fops);
-end:
 	up_write(&dev->lock);
 }
 
@@ -1330,7 +1311,7 @@ struct ion_device *ion_device_create(long (*custom_ioctl)
 	idev->buffers = RB_ROOT;
 	mutex_init(&idev->buffer_lock);
 	init_rwsem(&idev->lock);
-	idev->heaps = RB_ROOT;
+	plist_head_init(&idev->heaps);
 	idev->clients = RB_ROOT;
 	return idev;
 }
drivers/staging/android/ion/ion_priv.h +2 -2
index 77e2ac0..57dbf6a 100644
@@ -127,11 +127,11 @@ struct ion_heap_ops {
  * that are allocated from a specially reserved heap.
  */
 struct ion_heap {
-	struct rb_node node;
+	struct plist_node node;
 	struct ion_device *dev;
 	enum ion_heap_type type;
 	struct ion_heap_ops *ops;
-	int id;
+	unsigned int id;
 	const char *name;
 	int (*debug_show)(struct ion_heap *heap, struct seq_file *, void *);
 };
Signature · 1 line
1.8.3.2
#69 lore raw

PATCH 52/115 gpu: ion: Refactor common mapping functions out of system heap

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

The system heap contained several general purpose functions to map buffers to the kernel and userspace. This patch refactors those into ion_heap.c so they can be used by other heaps.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_heap.c        | 75 ++++++++++++++++++++++++
 drivers/staging/android/ion/ion_priv.h        | 10 ++++
 drivers/staging/android/ion/ion_system_heap.c | 84 ++-------------------------
 3 files changed, 91 insertions(+), 78 deletions(-)
drivers/staging/android/ion/ion_heap.c +75 -0
index 4f7caa8..441e9ef 100644
@@ -15,9 +15,84 @@
  */
 
 #include <linux/err.h>
+#include <linux/mm.h>
+#include <linux/scatterlist.h>
+#include <linux/vmalloc.h>
 #include "ion.h"
 #include "ion_priv.h"
 
+void *ion_heap_map_kernel(struct ion_heap *heap,
+			  struct ion_buffer *buffer)
+{
+	struct scatterlist *sg;
+	int i, j;
+	void *vaddr;
+	pgprot_t pgprot;
+	struct sg_table *table = buffer->sg_table;
+	int npages = PAGE_ALIGN(buffer->size) / PAGE_SIZE;
+	struct page **pages = vmalloc(sizeof(struct page *) * npages);
+	struct page **tmp = pages;
+
+	if (!pages)
+		return 0;
+
+	if (buffer->flags & ION_FLAG_CACHED)
+		pgprot = PAGE_KERNEL;
+	else
+		pgprot = pgprot_writecombine(PAGE_KERNEL);
+
+	for_each_sg(table->sgl, sg, table->nents, i) {
+		int npages_this_entry = PAGE_ALIGN(sg_dma_len(sg)) / PAGE_SIZE;
+		struct page *page = sg_page(sg);
+		BUG_ON(i >= npages);
+		for (j = 0; j < npages_this_entry; j++) {
+			*(tmp++) = page++;
+		}
+	}
+	vaddr = vmap(pages, npages, VM_MAP, pgprot);
+	vfree(pages);
+
+	return vaddr;
+}
+
+void ion_heap_unmap_kernel(struct ion_heap *heap,
+			   struct ion_buffer *buffer)
+{
+	vunmap(buffer->vaddr);
+}
+
+int ion_heap_map_user(struct ion_heap *heap, struct ion_buffer *buffer,
+		      struct vm_area_struct *vma)
+{
+	struct sg_table *table = buffer->sg_table;
+	unsigned long addr = vma->vm_start;
+	unsigned long offset = vma->vm_pgoff * PAGE_SIZE;
+	struct scatterlist *sg;
+	int i;
+
+	for_each_sg(table->sgl, sg, table->nents, i) {
+		struct page *page = sg_page(sg);
+		unsigned long remainder = vma->vm_end - addr;
+		unsigned long len = sg_dma_len(sg);
+
+		if (offset >= sg_dma_len(sg)) {
+			offset -= sg_dma_len(sg);
+			continue;
+		} else if (offset) {
+			page += offset / PAGE_SIZE;
+			len = sg_dma_len(sg) - offset;
+			offset = 0;
+		}
+		len = min(len, remainder);
+		remap_pfn_range(vma, addr, page_to_pfn(page), len,
+				vma->vm_page_prot);
+		addr += len;
+		if (addr >= vma->vm_end)
+			return 0;
+	}
+	return 0;
+}
+
 struct ion_heap *ion_heap_create(struct ion_platform_heap *heap_data)
 {
 	struct ion_heap *heap = NULL;
drivers/staging/android/ion/ion_priv.h +10 -0
index 57dbf6a..6b3616e 100644
@@ -178,6 +178,16 @@ void ion_device_destroy(struct ion_device *dev);
 void ion_device_add_heap(struct ion_device *dev, struct ion_heap *heap);
 
 /**
+ * some helpers for common operations on buffers using the sg_table
+ * and vaddr fields
+ */
+void *ion_heap_map_kernel(struct ion_heap *, struct ion_buffer *);
+void ion_heap_unmap_kernel(struct ion_heap *, struct ion_buffer *);
+int ion_heap_map_user(struct ion_heap *, struct ion_buffer *,
+			struct vm_area_struct *);
+
+
+/**
  * functions for creating and destroying the built in ion heaps.
  * architectures can add their own custom architecture specific
  * heaps as appropriate.
drivers/staging/android/ion/ion_system_heap.c +6 -78

107 lines not on this page · raw message

Signature · 1 line
1.8.3.2
#70 lore raw

PATCH 53/115 gpu: ion: Add chunk heap

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

This patch adds support for a chunk heap that allows for buffers that are made up of a list of fixed size chunks taken from a carveout. Chunk sizes are configured when the heaps are created by passing the chunk size in the priv field of the heap platform data.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/Makefile         |   2 +-
 drivers/staging/android/ion/ion.h            |   5 +
 drivers/staging/android/ion/ion_chunk_heap.c | 174 +++++++++++++++++++++++++++
 drivers/staging/android/ion/ion_heap.c       |   6 +
 drivers/staging/android/ion/ion_priv.h       |   4 +-
 5 files changed, 189 insertions(+), 2 deletions(-)
 create mode 100644 drivers/staging/android/ion/ion_chunk_heap.c
drivers/staging/android/ion/Makefile +1 -1
index d1ddebb..306fff9 100644
@@ -1,3 +1,3 @@
 obj-$(CONFIG_ION) +=	ion.o ion_heap.o ion_page_pool.o ion_system_heap.o \
-			ion_carveout_heap.o
+			ion_carveout_heap.o ion_chunk_heap.o
 obj-$(CONFIG_ION_TEGRA) += tegra/
drivers/staging/android/ion/ion.h +5 -0
index 072d64d..e5f679f 100644
@@ -33,6 +33,7 @@ enum ion_heap_type {
 	ION_HEAP_TYPE_SYSTEM,
 	ION_HEAP_TYPE_SYSTEM_CONTIG,
 	ION_HEAP_TYPE_CARVEOUT,
+	ION_HEAP_TYPE_CHUNK,
 	ION_HEAP_TYPE_CUSTOM, /* must be last so device specific heaps always
 				 are at the end of this enum */
 	ION_NUM_HEAPS,
@@ -75,6 +76,8 @@ struct ion_buffer;
  * @name:	used for debug purposes
  * @base:	base address of heap in physical memory if applicable
  * @size:	size of the heap in bytes if applicable
+ * @align:	required alignment in physical memory if applicable
+ * @priv:	private info passed from the board file
  *
  * Provided by the board file.
  */
@@ -84,6 +87,8 @@ struct ion_platform_heap {
 	const char *name;
 	ion_phys_addr_t base;
 	size_t size;
+	ion_phys_addr_t align;
+	void *priv;
 };
 
 /**
drivers/staging/android/ion/ion_chunk_heap.c new file +174 -0

174 lines not on this page · raw message

drivers/staging/android/ion/ion_heap.c +6 -0
index 441e9ef..a9bf52e 100644
@@ -107,6 +107,9 @@ struct ion_heap *ion_heap_create(struct ion_platform_heap *heap_data)
 	case ION_HEAP_TYPE_CARVEOUT:
 		heap = ion_carveout_heap_create(heap_data);
 		break;
+	case ION_HEAP_TYPE_CHUNK:
+		heap = ion_chunk_heap_create(heap_data);
+		break;
 	default:
 		pr_err("%s: Invalid heap type %d\n", __func__,
 		       heap_data->type);
@@ -140,6 +143,9 @@ void ion_heap_destroy(struct ion_heap *heap)
 	case ION_HEAP_TYPE_CARVEOUT:
 		ion_carveout_heap_destroy(heap);
 		break;
+	case ION_HEAP_TYPE_CHUNK:
+		ion_chunk_heap_destroy(heap);
+		break;
 	default:
 		pr_err("%s: Invalid heap type %d\n", __func__,
 		       heap->type);
drivers/staging/android/ion/ion_priv.h +3 -1
index 6b3616e..eef4de8 100644
@@ -195,7 +195,6 @@ int ion_heap_map_user(struct ion_heap *, struct ion_buffer *,
 
 struct ion_heap *ion_heap_create(struct ion_platform_heap *);
 void ion_heap_destroy(struct ion_heap *);
-
 struct ion_heap *ion_system_heap_create(struct ion_platform_heap *);
 void ion_system_heap_destroy(struct ion_heap *);
 
@@ -204,6 +203,9 @@ void ion_system_contig_heap_destroy(struct ion_heap *);
 
 struct ion_heap *ion_carveout_heap_create(struct ion_platform_heap *);
 void ion_carveout_heap_destroy(struct ion_heap *);
+
+struct ion_heap *ion_chunk_heap_create(struct ion_platform_heap *);
+void ion_chunk_heap_destroy(struct ion_heap *);
 /**
  * kernel api to allocate/free from carveout -- used when carveout is
  * used to back an architecture specific custom heap
Signature · 1 line
1.8.3.2
#71 lore raw

PATCH 54/115 gpu: ion: Clarify variable names and comments around heap ids v types

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

There is some confusion between when to use the heap type and when the id. This patch clarifies this by using clearer variable names and describing the intention in the comments. Also fixes the client debug code to print heaps by id instead of type.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 37 ++++++++++++++--------------
 drivers/staging/android/ion/ion.h | 51 +++++++++++++++++++++------------------
 2 files changed, 47 insertions(+), 41 deletions(-)
drivers/staging/android/ion/ion.c +19 -18
index 8cc0489..c775e4f 100644
@@ -1,4 +1,5 @@
 /*
+
  * drivers/staging/android/ion/ion.c
  *
  * Copyright (C) 2011 Google, Inc.
@@ -62,7 +63,7 @@ struct ion_device {
  * @dev:		backpointer to ion device
  * @handles:		an rb tree of all the handles in this client
  * @lock:		lock protecting the tree of handles
- * @heap_mask:		mask of all supported heaps
+ * @heap_type_mask:	mask of all supported heap types
  * @name:		used for debugging
  * @task:		used for debugging
  *
@@ -75,7 +76,7 @@ struct ion_client {
 	struct ion_device *dev;
 	struct rb_root handles;
 	struct mutex lock;
-	unsigned int heap_mask;
+	unsigned int heap_type_mask;
 	const char *name;
 	struct task_struct *task;
 	pid_t pid;
@@ -386,7 +387,7 @@ static void ion_handle_add(struct ion_client *client, struct ion_handle *handle)
 }
 
 struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
-			     size_t align, unsigned int heap_mask,
+			     size_t align, unsigned int heap_id_mask,
 			     unsigned int flags)
 {
 	struct ion_handle *handle;
@@ -394,8 +395,8 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 	struct ion_buffer *buffer = NULL;
 	struct ion_heap *heap;
 
-	pr_debug("%s: len %d align %d heap_mask %u flags %x\n", __func__, len,
-		 align, heap_mask, flags);
+	pr_debug("%s: len %d align %d heap_id_mask %u flags %x\n", __func__,
+		 len, align, heap_id_mask, flags);
 	/*
 	 * traverse the list of heaps available in this system in priority
 	 * order.  If the heap type is supported by the client, and matches the
@@ -410,10 +411,10 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 	down_read(&dev->lock);
 	plist_for_each_entry(heap, &dev->heaps, node) {
 		/* if the client doesn't support this heap type */
-		if (!((1 << heap->type) & client->heap_mask))
+		if (!((1 << heap->type) & client->heap_type_mask))
 			continue;
-		/* if the caller didn't specify this heap type */
-		if (!((1 << heap->id) & heap_mask))
+		/* if the caller didn't specify this heap id */
+		if (!((1 << heap->id) & heap_id_mask))
 			continue;
 		buffer = ion_buffer_create(heap, dev, len, align, flags);
 		if (!IS_ERR_OR_NULL(buffer))
@@ -588,24 +589,24 @@ static int ion_debug_client_show(struct seq_file *s, void *unused)
 {
 	struct ion_client *client = s->private;
 	struct rb_node *n;
-	size_t sizes[ION_NUM_HEAPS] = {0};
-	const char *names[ION_NUM_HEAPS] = {0};
+	size_t sizes[ION_NUM_HEAP_IDS] = {0};
+	const char *names[ION_NUM_HEAP_IDS] = {0};
 	int i;
 
 	mutex_lock(&client->lock);
 	for (n = rb_first(&client->handles); n; n = rb_next(n)) {
 		struct ion_handle *handle = rb_entry(n, struct ion_handle,
 						     node);
-		enum ion_heap_type type = handle->buffer->heap->type;
+		unsigned int id = handle->buffer->heap->id;
 
-		if (!names[type])
-			names[type] = handle->buffer->heap->name;
-		sizes[type] += handle->buffer->size;
+		if (!names[id])
+			names[id] = handle->buffer->heap->name;
+		sizes[id] += handle->buffer->size;
 	}
 	mutex_unlock(&client->lock);
 
 	seq_printf(s, "%16.16s: %16.16s\n", "heap_name", "size_in_bytes");
-	for (i = 0; i < ION_NUM_HEAPS; i++) {
+	for (i = 0; i < ION_NUM_HEAP_IDS; i++) {
 		if (!names[i])
 			continue;
 		seq_printf(s, "%16.16s: %16u\n", names[i], sizes[i]);
@@ -626,7 +627,7 @@ static const struct file_operations debug_client_fops = {
 };
 
 struct ion_client *ion_client_create(struct ion_device *dev,
-				     unsigned int heap_mask,
+				     unsigned int heap_type_mask,
 				     const char *name)
 {
 	struct ion_client *client;
@@ -661,7 +662,7 @@ struct ion_client *ion_client_create(struct ion_device *dev,
 	client->handles = RB_ROOT;
 	mutex_init(&client->lock);
 	client->name = name;
-	client->heap_mask = heap_mask;
+	client->heap_type_mask = heap_type_mask;
 	client->task = task;
 	client->pid = pid;
 
@@ -1057,7 +1058,7 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 		if (copy_from_user(&data, (void __user *)arg, sizeof(data)))
 			return -EFAULT;
 		data.handle = ion_alloc(client, data.len, data.align,
-					     data.heap_mask, data.flags);
+					     data.heap_id_mask, data.flags);
 
 		if (IS_ERR(data.handle))
 			return PTR_ERR(data.handle);
drivers/staging/android/ion/ion.h +28 -23

94 lines not on this page · raw message

Signature · 1 line
1.8.3.2
#72 lore raw

PATCH 55/115 gpu: ion: Export ion_client_create

In reply to John Stultz

From: Johan Mossberg <[email protected]>

Will enable modules to allocate memory with ion.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 1 +
 1 file changed, 1 insertion(+)
drivers/staging/android/ion/ion.c +1 -0
index c775e4f..7c25d9f 100644
@@ -688,6 +688,7 @@ struct ion_client *ion_client_create(struct ion_device *dev,
 
 	return client;
 }
+EXPORT_SYMBOL(ion_client_create);
 
 void ion_client_destroy(struct ion_client *client)
 {
Signature · 1 line
1.8.3.2
#73 lore raw

PATCH 56/115 gpu: ion: Remove heapmask from client

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

The heapmask in the client generally wasn't being used. This patch removes it.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 17 +++++------------
 drivers/staging/android/ion/ion.h |  1 -
 2 files changed, 5 insertions(+), 13 deletions(-)
drivers/staging/android/ion/ion.c +5 -12
index 7c25d9f..054e672 100644
@@ -63,7 +63,6 @@ struct ion_device {
  * @dev:		backpointer to ion device
  * @handles:		an rb tree of all the handles in this client
  * @lock:		lock protecting the tree of handles
- * @heap_type_mask:	mask of all supported heap types
  * @name:		used for debugging
  * @task:		used for debugging
  *
@@ -76,7 +75,6 @@ struct ion_client {
 	struct ion_device *dev;
 	struct rb_root handles;
 	struct mutex lock;
-	unsigned int heap_type_mask;
 	const char *name;
 	struct task_struct *task;
 	pid_t pid;
@@ -410,9 +408,6 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 
 	down_read(&dev->lock);
 	plist_for_each_entry(heap, &dev->heaps, node) {
-		/* if the client doesn't support this heap type */
-		if (!((1 << heap->type) & client->heap_type_mask))
-			continue;
 		/* if the caller didn't specify this heap id */
 		if (!((1 << heap->id) & heap_id_mask))
 			continue;
@@ -627,7 +622,6 @@ static const struct file_operations debug_client_fops = {
 };
 
 struct ion_client *ion_client_create(struct ion_device *dev,
-				     unsigned int heap_type_mask,
 				     const char *name)
 {
 	struct ion_client *client;
@@ -662,7 +656,6 @@ struct ion_client *ion_client_create(struct ion_device *dev,
 	client->handles = RB_ROOT;
 	mutex_init(&client->lock);
 	client->name = name;
-	client->heap_type_mask = heap_type_mask;
 	client->task = task;
 	client->pid = pid;
 
@@ -1162,7 +1155,7 @@ static int ion_open(struct inode *inode, struct file *file)
 	struct ion_client *client;
 
 	pr_debug("%s: %d\n", __func__, __LINE__);
-	client = ion_client_create(dev, -1, "user");
+	client = ion_client_create(dev, "user");
 	if (IS_ERR_OR_NULL(client))
 		return PTR_ERR(client);
 	file->private_data = client;
@@ -1178,7 +1171,7 @@ static const struct file_operations ion_fops = {
 };
 
 static size_t ion_debug_heap_total(struct ion_client *client,
-				   enum ion_heap_type type)
+				   unsigned int id)
 {
 	size_t size = 0;
 	struct rb_node *n;
@@ -1188,7 +1181,7 @@ static size_t ion_debug_heap_total(struct ion_client *client,
 		struct ion_handle *handle = rb_entry(n,
 						     struct ion_handle,
 						     node);
-		if (handle->buffer->heap->type == type)
+		if (handle->buffer->heap->id == id)
 			size += handle->buffer->size;
 	}
 	mutex_unlock(&client->lock);
@@ -1209,7 +1202,7 @@ static int ion_debug_heap_show(struct seq_file *s, void *unused)
 	for (n = rb_first(&dev->clients); n; n = rb_next(n)) {
 		struct ion_client *client = rb_entry(n, struct ion_client,
 						     node);
-		size_t size = ion_debug_heap_total(client, heap->type);
+		size_t size = ion_debug_heap_total(client, heap->id);
 		if (!size)
 			continue;
 		if (client->task) {
@@ -1230,7 +1223,7 @@ static int ion_debug_heap_show(struct seq_file *s, void *unused)
 	for (n = rb_first(&dev->buffers); n; n = rb_next(n)) {
 		struct ion_buffer *buffer = rb_entry(n, struct ion_buffer,
 						     node);
-		if (buffer->heap->type != heap->type)
+		if (buffer->heap->id != heap->id)
 			continue;
 		total_size += buffer->size;
 		if (!buffer->handle_count) {
drivers/staging/android/ion/ion.h +0 -1
index a315858..0f67014f 100644
@@ -124,7 +124,6 @@ void ion_reserve(struct ion_platform_data *data);
  * @name:		used for debugging
  */
 struct ion_client *ion_client_create(struct ion_device *dev,
-				     unsigned int heap_type_mask,
 				     const char *name);
 
 /**
Signature · 1 line
1.8.3.2
#74 lore raw

PATCH 57/115 gpu: ion: Modify zeroing code so it only allocates address space once

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

vmap/vunmap spend a significant amount of time allocating the address space to map into. Rather than allocating address space for each page, instead allocate once for the entire allocation and then just map and unmap each page into that address space.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 28 ++++++++++++++++++++-------
 1 file changed, 21 insertions(+), 7 deletions(-)
drivers/staging/android/ion/ion_system_heap.c +21 -7
index 89247cf..e54307f 100644
@@ -91,7 +91,7 @@ static struct page *alloc_buffer_page(struct ion_system_heap *heap,
 
 static void free_buffer_page(struct ion_system_heap *heap,
 			     struct ion_buffer *buffer, struct page *page,
-			     unsigned int order)
+			     unsigned int order, struct vm_struct *vm_struct)
 {
 	bool cached = ion_buffer_cached(buffer);
 	bool split_pages = ion_buffer_fault_user_mappings(buffer);
@@ -105,10 +105,13 @@ static void free_buffer_page(struct ion_system_heap *heap,
 		   purpose is to keep the pages out of the cache */
 		for (i = 0; i < (1 << order); i++) {
 			struct page *sub_page = page + i;
-			void *addr = vmap(&sub_page, 1, VM_MAP,
-					  pgprot_writecombine(PAGE_KERNEL));
-			memset(addr, 0, PAGE_SIZE);
-			vunmap(addr);
+			struct page **pages = &sub_page;
+			map_vm_area(vm_struct,
+					 pgprot_writecombine(PAGE_KERNEL),
+					 &pages);
+			memset(vm_struct->addr, 0, PAGE_SIZE);
+			unmap_kernel_range((unsigned long)vm_struct->addr,
+					PAGE_SIZE);
 		}
 		ion_page_pool_free(pool, page);
 	} else if (split_pages) {
@@ -164,6 +167,8 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 	long size_remaining = PAGE_ALIGN(size);
 	unsigned int max_order = orders[0];
 	bool split_pages = ion_buffer_fault_user_mappings(buffer);
+	struct vm_struct *vm_struct;
+	pte_t *ptes;
 
 	INIT_LIST_HEAD(&pages);
 	while (size_remaining > 0) {
@@ -211,10 +216,13 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 err1:
 	kfree(table);
 err:
+	vm_struct = get_vm_area(PAGE_SIZE, &ptes);
 	list_for_each_entry(info, &pages, list) {
-		free_buffer_page(sys_heap, buffer, info->page, info->order);
+		free_buffer_page(sys_heap, buffer, info->page, info->order,
+				vm_struct);
 		kfree(info);
 	}
+	free_vm_area(vm_struct);
 	return -ENOMEM;
 }
 
@@ -227,10 +235,16 @@ void ion_system_heap_free(struct ion_buffer *buffer)
 	struct sg_table *table = buffer->sg_table;
 	struct scatterlist *sg;
 	LIST_HEAD(pages);
+	struct vm_struct *vm_struct;
+	pte_t *ptes;
 	int i;
 
+	vm_struct = get_vm_area(PAGE_SIZE, &ptes);
+
 	for_each_sg(table->sgl, sg, table->nents, i)
-		free_buffer_page(sys_heap, buffer, sg_page(sg), get_order(sg_dma_len(sg)));
+		free_buffer_page(sys_heap, buffer, sg_page(sg),
+				get_order(sg_dma_len(sg)), vm_struct);
+	free_vm_area(vm_struct);
 	sg_free_table(table);
 	kfree(table);
 }
Signature · 1 line
1.8.3.2
#75 lore raw

PATCH 58/115 gpu: ion: Refactor the code to zero buffers

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

Refactor the code in the system heap used to map and zero the buffers into a seperate utility so it can be called from other heaps. Use it from the chunk heap.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_chunk_heap.c  |  2 ++
 drivers/staging/android/ion/ion_heap.c        | 37 +++++++++++++++++++++++++++
 drivers/staging/android/ion/ion_priv.h        |  1 +
 drivers/staging/android/ion/ion_system_heap.c | 34 ++++++------------------
 4 files changed, 48 insertions(+), 26 deletions(-)
drivers/staging/android/ion/ion_chunk_heap.c +2 -0
index 1d47409..5582909 100644
@@ -101,6 +101,8 @@ static void ion_chunk_heap_free(struct ion_buffer *buffer)
 	struct scatterlist *sg;
 	int i;
 
+	ion_heap_buffer_zero(buffer);
+
 	for_each_sg(table->sgl, sg, table->nents, i) {
 		__dma_page_cpu_to_dev(sg_page(sg), 0, sg_dma_len(sg),
 				      DMA_BIDIRECTIONAL);
drivers/staging/android/ion/ion_heap.c +37 -0
index a9bf52e..11066a2 100644
@@ -93,6 +93,43 @@ int ion_heap_map_user(struct ion_heap *heap, struct ion_buffer *buffer,
 	return 0;
 }
 
+int ion_heap_buffer_zero(struct ion_buffer *buffer)
+{
+	struct sg_table *table = buffer->sg_table;
+	pgprot_t pgprot;
+	struct scatterlist *sg;
+	struct vm_struct *vm_struct;
+	int i, j, ret = 0;
+
+	if (buffer->flags & ION_FLAG_CACHED)
+		pgprot = PAGE_KERNEL;
+	else
+		pgprot = pgprot_writecombine(PAGE_KERNEL);
+
+	vm_struct = get_vm_area(PAGE_SIZE, VM_ALLOC);
+	if (!vm_struct)
+		return -ENOMEM;
+
+	for_each_sg(table->sgl, sg, table->nents, i) {
+		struct page *page = sg_page(sg);
+		unsigned long len = sg_dma_len(sg);
+
+		for (j = 0; j < len / PAGE_SIZE; j++) {
+			struct page *sub_page = page + j;
+			struct page **pages = &sub_page;
+			ret = map_vm_area(vm_struct, pgprot, &pages);
+			if (ret)
+				goto end;
+			memset(vm_struct->addr, 0, PAGE_SIZE);
+			unmap_kernel_range((unsigned long)vm_struct->addr,
+					   PAGE_SIZE);
+		}
+	}
+end:
+	free_vm_area(vm_struct);
+	return ret;
+}
+
 struct ion_heap *ion_heap_create(struct ion_platform_heap *heap_data)
 {
 	struct ion_heap *heap = NULL;
drivers/staging/android/ion/ion_priv.h +1 -0
index eef4de8..cfb4264 100644
@@ -185,6 +185,7 @@ void *ion_heap_map_kernel(struct ion_heap *, struct ion_buffer *);
 void ion_heap_unmap_kernel(struct ion_heap *, struct ion_buffer *);
 int ion_heap_map_user(struct ion_heap *, struct ion_buffer *,
 			struct vm_area_struct *);
+int ion_heap_buffer_zero(struct ion_buffer *buffer);
 
 
 /**
drivers/staging/android/ion/ion_system_heap.c +8 -26
index e54307f..3ca704e 100644
@@ -91,7 +91,7 @@ static struct page *alloc_buffer_page(struct ion_system_heap *heap,
 
 static void free_buffer_page(struct ion_system_heap *heap,
 			     struct ion_buffer *buffer, struct page *page,
-			     unsigned int order, struct vm_struct *vm_struct)
+			     unsigned int order)
 {
 	bool cached = ion_buffer_cached(buffer);
 	bool split_pages = ion_buffer_fault_user_mappings(buffer);
@@ -99,20 +99,6 @@ static void free_buffer_page(struct ion_system_heap *heap,
 
 	if (!cached) {
 		struct ion_page_pool *pool = heap->pools[order_to_index(order)];
-		/* zero the pages before returning them to the pool for
-		   security.  This uses vmap as we want to set the pgprot so
-		   the writes to occur to noncached mappings, as the pool's
-		   purpose is to keep the pages out of the cache */
-		for (i = 0; i < (1 << order); i++) {
-			struct page *sub_page = page + i;
-			struct page **pages = &sub_page;
-			map_vm_area(vm_struct,
-					 pgprot_writecombine(PAGE_KERNEL),
-					 &pages);
-			memset(vm_struct->addr, 0, PAGE_SIZE);
-			unmap_kernel_range((unsigned long)vm_struct->addr,
-					PAGE_SIZE);
-		}
 		ion_page_pool_free(pool, page);
 	} else if (split_pages) {
 		for (i = 0; i < (1 << order); i++)
@@ -167,8 +153,6 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 	long size_remaining = PAGE_ALIGN(size);
 	unsigned int max_order = orders[0];
 	bool split_pages = ion_buffer_fault_user_mappings(buffer);
-	struct vm_struct *vm_struct;
-	pte_t *ptes;
 
 	INIT_LIST_HEAD(&pages);
 	while (size_remaining > 0) {
@@ -216,13 +200,10 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 err1:
 	kfree(table);
 err:
-	vm_struct = get_vm_area(PAGE_SIZE, &ptes);
 	list_for_each_entry(info, &pages, list) {
-		free_buffer_page(sys_heap, buffer, info->page, info->order,
-				vm_struct);
+		free_buffer_page(sys_heap, buffer, info->page, info->order);
 		kfree(info);
 	}
-	free_vm_area(vm_struct);
 	return -ENOMEM;
 }
 
@@ -233,18 +214,19 @@ void ion_system_heap_free(struct ion_buffer *buffer)
 							struct ion_system_heap,
 							heap);
 	struct sg_table *table = buffer->sg_table;
+	bool cached = ion_buffer_cached(buffer);
 	struct scatterlist *sg;
 	LIST_HEAD(pages);
-	struct vm_struct *vm_struct;
-	pte_t *ptes;
 	int i;
 
-	vm_struct = get_vm_area(PAGE_SIZE, &ptes);
+	/* uncached pages come from the page pools, zero them before returning
+	   for security purposes (other allocations are zerod at alloc time */
+	if (!cached)
+		ion_heap_buffer_zero(buffer);
 
 	for_each_sg(table->sgl, sg, table->nents, i)
 		free_buffer_page(sys_heap, buffer, sg_page(sg),
-				get_order(sg_dma_len(sg)), vm_struct);
-	free_vm_area(vm_struct);
+				get_order(sg_dma_len(sg)));
 	sg_free_table(table);
 	kfree(table);
 }
Signature · 1 line
1.8.3.2
#76 lore raw

PATCH 59/115 gpu: ion: Only flush buffers in the chunk heap if they were used cached

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_chunk_heap.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/staging/android/ion/ion_chunk_heap.c +3 -2
index 5582909..60cd91c 100644
@@ -104,8 +104,9 @@ static void ion_chunk_heap_free(struct ion_buffer *buffer)
 	ion_heap_buffer_zero(buffer);
 
 	for_each_sg(table->sgl, sg, table->nents, i) {
-		__dma_page_cpu_to_dev(sg_page(sg), 0, sg_dma_len(sg),
-				      DMA_BIDIRECTIONAL);
+		if (ion_buffer_cached(buffer))
+			__dma_page_cpu_to_dev(sg_page(sg), 0, sg_dma_len(sg),
+					      DMA_BIDIRECTIONAL);
 		gen_pool_free(chunk_heap->pool, page_to_phys(sg_page(sg)),
 			      sg_dma_len(sg));
 	}
Signature · 1 line
1.8.3.2
#77 lore raw

PATCH 60/115 gpu: ion: Add support for sharing buffers with dma buf kernel handles

In reply to John Stultz

From: Johan Mossberg <[email protected]>

Currently ion can only share buffers with dma buf fd's. Fd's can not be used inside the kernel as they are process specific so support for sharing buffers with dma buf kernel handles is needed to support kernel only use cases. An example use case could be a GPU driver using ion that wants to share its output buffers with a 3d party display controller driver supporting dma buf.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 26 ++++++++++++++++++++------
 drivers/staging/android/ion/ion.h | 12 ++++++++++--
 2 files changed, 30 insertions(+), 8 deletions(-)
drivers/staging/android/ion/ion.c +20 -6
index 054e672..ba65bef 100644
@@ -949,19 +949,19 @@ struct dma_buf_ops dma_buf_ops = {
 	.kunmap = ion_dma_buf_kunmap,
 };
 
-int ion_share_dma_buf(struct ion_client *client, struct ion_handle *handle)
+struct dma_buf *ion_share_dma_buf(struct ion_client *client,
+						struct ion_handle *handle)
 {
 	struct ion_buffer *buffer;
 	struct dma_buf *dmabuf;
 	bool valid_handle;
-	int fd;
 
 	mutex_lock(&client->lock);
 	valid_handle = ion_handle_validate(client, handle);
 	mutex_unlock(&client->lock);
 	if (!valid_handle) {
 		WARN(1, "%s: invalid handle passed to share.\n", __func__);
-		return -EINVAL;
+		return ERR_PTR(-EINVAL);
 	}
 
 	buffer = handle->buffer;
@@ -969,15 +969,29 @@ int ion_share_dma_buf(struct ion_client *client, struct ion_handle *handle)
 	dmabuf = dma_buf_export(buffer, &dma_buf_ops, buffer->size, O_RDWR);
 	if (IS_ERR(dmabuf)) {
 		ion_buffer_put(buffer);
-		return PTR_ERR(dmabuf);
+		return dmabuf;
 	}
+
+	return dmabuf;
+}
+EXPORT_SYMBOL(ion_share_dma_buf);
+
+int ion_share_dma_buf_fd(struct ion_client *client, struct ion_handle *handle)
+{
+	struct dma_buf *dmabuf;
+	int fd;
+
+	dmabuf = ion_share_dma_buf(client, handle);
+	if (IS_ERR(dmabuf))
+		return PTR_ERR(dmabuf);
+
 	fd = dma_buf_fd(dmabuf, O_CLOEXEC);
 	if (fd < 0)
 		dma_buf_put(dmabuf);
 
 	return fd;
 }
-EXPORT_SYMBOL(ion_share_dma_buf);
+EXPORT_SYMBOL(ion_share_dma_buf_fd);
 
 struct ion_handle *ion_import_dma_buf(struct ion_client *client, int fd)
 {
@@ -1086,7 +1100,7 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 
 		if (copy_from_user(&data, (void __user *)arg, sizeof(data)))
 			return -EFAULT;
-		data.fd = ion_share_dma_buf(client, data.handle);
+		data.fd = ion_share_dma_buf_fd(client, data.handle);
 		if (copy_to_user((void __user *)arg, &data, sizeof(data)))
 			return -EFAULT;
 		if (data.fd < 0)
drivers/staging/android/ion/ion.h +10 -2
index 0f67014f..976123b 100644
@@ -212,11 +212,19 @@ void *ion_map_kernel(struct ion_client *client, struct ion_handle *handle);
 void ion_unmap_kernel(struct ion_client *client, struct ion_handle *handle);
 
 /**
- * ion_share_dma_buf() - given an ion client, create a dma-buf fd
+ * ion_share_dma_buf() - share buffer as dma-buf
  * @client:	the client
  * @handle:	the handle
  */
-int ion_share_dma_buf(struct ion_client *client, struct ion_handle *handle);
+struct dma_buf *ion_share_dma_buf(struct ion_client *client,
+						struct ion_handle *handle);
+
+/**
+ * ion_share_dma_buf_fd() - given an ion client, create a dma-buf fd
+ * @client:	the client
+ * @handle:	the handle
+ */
+int ion_share_dma_buf_fd(struct ion_client *client, struct ion_handle *handle);
 
 /**
  * ion_import_dma_buf() - given an dma-buf fd from the ion exporter get handle
Signature · 1 line
1.8.3.2
#78 lore raw

PATCH 61/115 gpu: ion: Make ion_free asynchronous

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

Add the ability for a heap to free buffers asynchrounously. Freed buffers are placed on a free list and freed from a low priority background thread. If allocations from a particular heap fail, the free list is drained. This patch also enable asynchronous frees from the chunk heap.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c             | 112 +++++++++++++++++++++++---
 drivers/staging/android/ion/ion.h             |   2 +-
 drivers/staging/android/ion/ion_chunk_heap.c  |   3 +-
 drivers/staging/android/ion/ion_priv.h        |  20 ++++-
 drivers/staging/android/ion/ion_system_heap.c |   1 +
 5 files changed, 126 insertions(+), 12 deletions(-)
drivers/staging/android/ion/ion.c +103 -9
index ba65bef..b965f15 100644
@@ -17,8 +17,10 @@
 
 #include <linux/device.h>
 #include <linux/file.h>
+#include <linux/freezer.h>
 #include <linux/fs.h>
 #include <linux/anon_inodes.h>
+#include <linux/kthread.h>
 #include <linux/list.h>
 #include <linux/memblock.h>
 #include <linux/miscdevice.h>
@@ -26,6 +28,7 @@
 #include <linux/mm.h>
 #include <linux/mm_types.h>
 #include <linux/rbtree.h>
+#include <linux/rtmutex.h>
 #include <linux/sched.h>
 #include <linux/slab.h>
 #include <linux/seq_file.h>
@@ -140,6 +143,7 @@ static void ion_buffer_add(struct ion_device *dev,
 
 static int ion_buffer_alloc_dirty(struct ion_buffer *buffer);
 
+static bool ion_heap_drain_freelist(struct ion_heap *heap);
 /* this function should only be called while dev->lock is held */
 static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 				     struct ion_device *dev,
@@ -161,9 +165,16 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 	kref_init(&buffer->ref);
 
 	ret = heap->ops->allocate(heap, buffer, len, align, flags);
+
 	if (ret) {
-		kfree(buffer);
-		return ERR_PTR(ret);
+		if (!(heap->flags & ION_HEAP_FLAG_DEFER_FREE))
+			goto err2;
+
+		ion_heap_drain_freelist(heap);
+		ret = heap->ops->allocate(heap, buffer, len, align,
+					  flags);
+		if (ret)
+			goto err2;
 	}
 
 	buffer->dev = dev;
@@ -214,27 +225,42 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 err:
 	heap->ops->unmap_dma(heap, buffer);
 	heap->ops->free(buffer);
+err2:
 	kfree(buffer);
 	return ERR_PTR(ret);
 }
 
-static void ion_buffer_destroy(struct kref *kref)
+static void _ion_buffer_destroy(struct ion_buffer *buffer)
 {
-	struct ion_buffer *buffer = container_of(kref, struct ion_buffer, ref);
-	struct ion_device *dev = buffer->dev;
-
 	if (WARN_ON(buffer->kmap_cnt > 0))
 		buffer->heap->ops->unmap_kernel(buffer->heap, buffer);
 	buffer->heap->ops->unmap_dma(buffer->heap, buffer);
 	buffer->heap->ops->free(buffer);
-	mutex_lock(&dev->buffer_lock);
-	rb_erase(&buffer->node, &dev->buffers);
-	mutex_unlock(&dev->buffer_lock);
 	if (buffer->flags & ION_FLAG_CACHED)
 		kfree(buffer->dirty);
 	kfree(buffer);
 }
 
+static void ion_buffer_destroy(struct kref *kref)
+{
+	struct ion_buffer *buffer = container_of(kref, struct ion_buffer, ref);
+	struct ion_heap *heap = buffer->heap;
+	struct ion_device *dev = buffer->dev;
+
+	mutex_lock(&dev->buffer_lock);
+	rb_erase(&buffer->node, &dev->buffers);
+	mutex_unlock(&dev->buffer_lock);
+
+	if (heap->flags & ION_HEAP_FLAG_DEFER_FREE) {
+		rt_mutex_lock(&heap->lock);
+		list_add(&buffer->list, &heap->free_list);
+		rt_mutex_unlock(&heap->lock);
+		wake_up(&heap->waitqueue);
+		return;
+	}
+	_ion_buffer_destroy(buffer);
+}
+
 static void ion_buffer_get(struct ion_buffer *buffer)
 {
 	kref_get(&buffer->ref);
@@ -1272,13 +1298,81 @@ static const struct file_operations debug_heap_fops = {
 	.release = single_release,
 };
 
+static size_t ion_heap_free_list_is_empty(struct ion_heap *heap)
+{
+	bool is_empty;
+
+	rt_mutex_lock(&heap->lock);
+	is_empty = list_empty(&heap->free_list);
+	rt_mutex_unlock(&heap->lock);
+
+	return is_empty;
+}
+
+static int ion_heap_deferred_free(void *data)
+{
+	struct ion_heap *heap = data;
+
+	while (true) {
+		struct ion_buffer *buffer;
+
+		wait_event_freezable(heap->waitqueue,
+				     !ion_heap_free_list_is_empty(heap));
+
+		rt_mutex_lock(&heap->lock);
+		if (list_empty(&heap->free_list)) {
+			rt_mutex_unlock(&heap->lock);
+			continue;
+		}
+		buffer = list_first_entry(&heap->free_list, struct ion_buffer,
+					  list);
+		list_del(&buffer->list);
+		rt_mutex_unlock(&heap->lock);
+		_ion_buffer_destroy(buffer);
+	}
+
+	return 0;
+}
+
+static bool ion_heap_drain_freelist(struct ion_heap *heap)
+{
+	struct ion_buffer *buffer, *tmp;
+
+	if (ion_heap_free_list_is_empty(heap))
+		return false;
+	rt_mutex_lock(&heap->lock);
+	list_for_each_entry_safe(buffer, tmp, &heap->free_list, list) {
+		_ion_buffer_destroy(buffer);
+		list_del(&buffer->list);
+	}
+	BUG_ON(!list_empty(&heap->free_list));
+	rt_mutex_unlock(&heap->lock);
+
+
+	return true;
+}
+
 void ion_device_add_heap(struct ion_device *dev, struct ion_heap *heap)
 {
+	struct sched_param param = { .sched_priority = 0 };
+
 	if (!heap->ops->allocate || !heap->ops->free || !heap->ops->map_dma ||
 	    !heap->ops->unmap_dma)
 		pr_err("%s: can not add heap with invalid ops struct.\n",
 		       __func__);
 
+	if (heap->flags & ION_HEAP_FLAG_DEFER_FREE) {
+		INIT_LIST_HEAD(&heap->free_list);
+		rt_mutex_init(&heap->lock);
+		init_waitqueue_head(&heap->waitqueue);
+		heap->task = kthread_run(ion_heap_deferred_free, heap,
+					 "%s", heap->name);
+		sched_setscheduler(heap->task, SCHED_IDLE, &param);
+		if (IS_ERR(heap->task))
+			pr_err("%s: creating thread for deferred free failed\n",
+			       __func__);
+	}
+
 	heap->dev = dev;
 	down_write(&dev->lock);
 	/* use negative heap->id to reverse the priority -- when traversing
drivers/staging/android/ion/ion.h +1 -1
index 976123b..679031c 100644
@@ -46,7 +46,7 @@ enum ion_heap_type {
 #define ION_NUM_HEAP_IDS		sizeof(unsigned int) * 8
 
 /**
- * heap flags - the lower 16 bits are used by core ion, the upper 16
+ * allocation flags - the lower 16 bits are used by core ion, the upper 16
  * bits are reserved for use by the heaps themselves.
  */
 #define ION_FLAG_CACHED 1		/* mappings of this buffer should be
drivers/staging/android/ion/ion_chunk_heap.c +2 -1
index 60cd91c..ac7cf13 100644
@@ -160,7 +160,8 @@ struct ion_heap *ion_chunk_heap_create(struct ion_platform_heap *heap_data)
 	gen_pool_add(chunk_heap->pool, chunk_heap->base, heap_data->size, -1);
 	chunk_heap->heap.ops = &chunk_heap_ops;
 	chunk_heap->heap.type = ION_HEAP_TYPE_CHUNK;
-	pr_info("%s: base %lu size %ld align %ld\n", __func__, chunk_heap->base,
+	chunk_heap->heap.flags = ION_HEAP_FLAG_DEFER_FREE;
+	pr_info("%s: base %lu size %u align %ld\n", __func__, chunk_heap->base,
 		heap_data->size, heap_data->align);
 
 	return &chunk_heap->heap;
drivers/staging/android/ion/ion_priv.h +19 -1
index cfb4264..ab1a8d9 100644
@@ -58,7 +58,10 @@ struct ion_buffer *ion_handle_buffer(struct ion_handle *handle);
 */
 struct ion_buffer {
 	struct kref ref;
-	struct rb_node node;
+	union {
+		struct rb_node node;
+		struct list_head list;
+	};
 	struct ion_device *dev;
 	struct ion_heap *heap;
 	unsigned long flags;
@@ -109,15 +112,25 @@ struct ion_heap_ops {
 };
 
 /**
+ * heap flags - flags between the heaps and core ion code
+ */
+#define ION_HEAP_FLAG_DEFER_FREE (1 << 0)
+
+/**
  * struct ion_heap - represents a heap in the system
  * @node:		rb node to put the heap on the device's tree of heaps
  * @dev:		back pointer to the ion_device
  * @type:		type of heap
  * @ops:		ops struct as above
+ * @flags:		flags
  * @id:			id of heap, also indicates priority of this heap when
  *			allocating.  These are specified by platform data and
  *			MUST be unique
  * @name:		used for debugging
+ * @free_list:		free list head if deferred free is used
+ * @lock:		protects the free list
+ * @waitqueue:		queue to wait on from deferred free thread
+ * @task:		task struct of deferred free thread
  * @debug_show:		called when heap debug file is read to add any
  *			heap specific debug info to output
  *
@@ -131,8 +144,13 @@ struct ion_heap {
 	struct ion_device *dev;
 	enum ion_heap_type type;
 	struct ion_heap_ops *ops;
+	unsigned long flags;
 	unsigned int id;
 	const char *name;
+	struct list_head free_list;
+	struct rt_mutex lock;
+	wait_queue_head_t waitqueue;
+	struct task_struct *task;
 	int (*debug_show)(struct ion_heap *heap, struct seq_file *, void *);
 };
 
drivers/staging/android/ion/ion_system_heap.c +1 -0
index 3ca704e..6665797 100644
@@ -283,6 +283,7 @@ struct ion_heap *ion_system_heap_create(struct ion_platform_heap *unused)
 		return ERR_PTR(-ENOMEM);
 	heap->heap.ops = &system_heap_ops;
 	heap->heap.type = ION_HEAP_TYPE_SYSTEM;
+	heap->heap.flags = ION_HEAP_FLAG_DEFER_FREE;
 	heap->pools = kzalloc(sizeof(struct ion_page_pool *) * num_orders,
 			      GFP_KERNEL);
 	if (!heap->pools)
Signature · 1 line
1.8.3.2
#79 lore raw

PATCH 62/115 gpu: ion: fix kfree/list_del order

In reply to John Stultz

From: JP Abgrall <[email protected]>

With CONFIG_SLUB_DEBUG_ON it would panic during
ion_alloc()
ion_buffer_create()
io_heap_drain_freelist()

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/android/ion/ion.c +1 -1
index b965f15..b2dcee5 100644
@@ -1342,8 +1342,8 @@ static bool ion_heap_drain_freelist(struct ion_heap *heap)
 		return false;
 	rt_mutex_lock(&heap->lock);
 	list_for_each_entry_safe(buffer, tmp, &heap->free_list, list) {
-		_ion_buffer_destroy(buffer);
 		list_del(&buffer->list);
+		_ion_buffer_destroy(buffer);
 	}
 	BUG_ON(!list_empty(&heap->free_list));
 	rt_mutex_unlock(&heap->lock);
Signature · 1 line
1.8.3.2
#80 lore raw

PATCH 63/115 gpu: ion: ion_chunk_heap: Zero chunk heap memory at creation time

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

Allocations from the ion heap need to be zeroed to protect userspace from seeing memory belonging to other processes. First allocations from this heap were not zero'd allowing users to see memory from other processes on a warm reset.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_chunk_heap.c | 34 ++++++++++++++++++++++++++--
 1 file changed, 32 insertions(+), 2 deletions(-)
drivers/staging/android/ion/ion_chunk_heap.c +32 -2
index ac7cf13..4c2f223 100644
@@ -140,6 +140,10 @@ static struct ion_heap_ops chunk_heap_ops = {
 struct ion_heap *ion_chunk_heap_create(struct ion_platform_heap *heap_data)
 {
 	struct ion_chunk_heap *chunk_heap;
+	struct vm_struct *vm_struct;
+	pgprot_t pgprot = pgprot_writecombine(PAGE_KERNEL);
+	int i, ret;
+
 
 	chunk_heap = kzalloc(sizeof(struct ion_chunk_heap), GFP_KERNEL);
 	if (!chunk_heap)
@@ -149,12 +153,30 @@ struct ion_heap *ion_chunk_heap_create(struct ion_platform_heap *heap_data)
 	chunk_heap->pool = gen_pool_create(get_order(chunk_heap->chunk_size) +
 					   PAGE_SHIFT, -1);
 	if (!chunk_heap->pool) {
-		kfree(chunk_heap);
-		return ERR_PTR(-ENOMEM);
+		ret = -ENOMEM;
+		goto error_gen_pool_create;
 	}
 	chunk_heap->base = heap_data->base;
 	chunk_heap->size = heap_data->size;
 	chunk_heap->allocated = 0;
+
+	vm_struct = get_vm_area(PAGE_SIZE, VM_ALLOC);
+	if (!vm_struct) {
+		ret = -ENOMEM;
+		goto error;
+	}
+	for (i = 0; i < chunk_heap->size; i += PAGE_SIZE) {
+		struct page *page = phys_to_page(chunk_heap->base + i);
+		struct page **pages = &page;
+
+		ret = map_vm_area(vm_struct, pgprot, &pages);
+		if (ret)
+			goto error_map_vm_area;
+		memset(vm_struct->addr, 0, PAGE_SIZE);
+		unmap_kernel_range((unsigned long)vm_struct->addr, PAGE_SIZE);
+	}
+	free_vm_area(vm_struct);
+
 	__dma_page_cpu_to_dev(phys_to_page(heap_data->base), 0, heap_data->size,
 			      DMA_BIDIRECTIONAL);
 	gen_pool_add(chunk_heap->pool, chunk_heap->base, heap_data->size, -1);
@@ -165,6 +187,14 @@ struct ion_heap *ion_chunk_heap_create(struct ion_platform_heap *heap_data)
 		heap_data->size, heap_data->align);
 
 	return &chunk_heap->heap;
+
+error_map_vm_area:
+	free_vm_area(vm_struct);
+error:
+	gen_pool_destroy(chunk_heap->pool);
+error_gen_pool_create:
+	kfree(chunk_heap);
+	return ERR_PTR(ret);
 }
 
 void ion_chunk_heap_destroy(struct ion_heap *heap)
Signature · 1 line
1.8.3.2
#81 lore raw

PATCH 64/115 gpu: ion: Fix bug in ion shrinker

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

The high variable was sometimes used uninitialized

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_page_pool.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/staging/android/ion/ion_page_pool.c +1 -2
index e59da3a..df6e29c 100644
@@ -203,8 +203,7 @@ static int ion_page_pool_shrink(struct shrinker *shrinker,
 	bool high;
 	int nr_to_scan = sc->nr_to_scan;
 
-	if (sc->gfp_mask & __GFP_HIGHMEM)
-		high = true;
+	high = sc->gfp_mask & __GFP_HIGHMEM;
 
 	if (nr_to_scan == 0)
 		return ion_page_pool_total(high);
Signature · 1 line
1.8.3.2
#82 lore raw

PATCH 65/115 gpu: ion: Also shrink memory cached in the deferred free list

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

When the system is low on memory, we want to shrink any cached system memory ion is holding. Previously we were shrinking memory in the page pools, but not in the deferred free list. This patch makes it possible to shrink both. It also moves the shrinker code into the heaps so they can correctly manage any caches they might contain.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c             | 115 +++++++++--------------
 drivers/staging/android/ion/ion_heap.c        | 107 ++++++++++++++++++++++
 drivers/staging/android/ion/ion_page_pool.c   | 126 +++++---------------------
 drivers/staging/android/ion/ion_priv.h        |  57 +++++++++++-
 drivers/staging/android/ion/ion_system_heap.c |  49 ++++++++++
 5 files changed, 278 insertions(+), 176 deletions(-)
drivers/staging/android/ion/ion.c +45 -70
index b2dcee5..ddf8fde 100644
@@ -28,8 +28,6 @@
 #include <linux/mm.h>
 #include <linux/mm_types.h>
 #include <linux/rbtree.h>
-#include <linux/rtmutex.h>
-#include <linux/sched.h>
 #include <linux/slab.h>
 #include <linux/seq_file.h>
 #include <linux/uaccess.h>
@@ -143,7 +141,6 @@ static void ion_buffer_add(struct ion_device *dev,
 
 static int ion_buffer_alloc_dirty(struct ion_buffer *buffer);
 
-static bool ion_heap_drain_freelist(struct ion_heap *heap);
 /* this function should only be called while dev->lock is held */
 static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 				     struct ion_device *dev,
@@ -170,7 +167,7 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 		if (!(heap->flags & ION_HEAP_FLAG_DEFER_FREE))
 			goto err2;
 
-		ion_heap_drain_freelist(heap);
+		ion_heap_freelist_drain(heap, 0);
 		ret = heap->ops->allocate(heap, buffer, len, align,
 					  flags);
 		if (ret)
@@ -230,7 +227,7 @@ err2:
 	return ERR_PTR(ret);
 }
 
-static void _ion_buffer_destroy(struct ion_buffer *buffer)
+void ion_buffer_destroy(struct ion_buffer *buffer)
 {
 	if (WARN_ON(buffer->kmap_cnt > 0))
 		buffer->heap->ops->unmap_kernel(buffer->heap, buffer);
@@ -241,7 +238,7 @@ static void _ion_buffer_destroy(struct ion_buffer *buffer)
 	kfree(buffer);
 }
 
-static void ion_buffer_destroy(struct kref *kref)
+static void _ion_buffer_destroy(struct kref *kref)
 {
 	struct ion_buffer *buffer = container_of(kref, struct ion_buffer, ref);
 	struct ion_heap *heap = buffer->heap;
@@ -251,14 +248,10 @@ static void ion_buffer_destroy(struct kref *kref)
 	rb_erase(&buffer->node, &dev->buffers);
 	mutex_unlock(&dev->buffer_lock);
 
-	if (heap->flags & ION_HEAP_FLAG_DEFER_FREE) {
-		rt_mutex_lock(&heap->lock);
-		list_add(&buffer->list, &heap->free_list);
-		rt_mutex_unlock(&heap->lock);
-		wake_up(&heap->waitqueue);
-		return;
-	}
-	_ion_buffer_destroy(buffer);
+	if (heap->flags & ION_HEAP_FLAG_DEFER_FREE)
+		ion_heap_freelist_add(heap, buffer);
+	else
+		ion_buffer_destroy(buffer);
 }
 
 static void ion_buffer_get(struct ion_buffer *buffer)
@@ -268,7 +261,7 @@ static void ion_buffer_get(struct ion_buffer *buffer)
 
 static int ion_buffer_put(struct ion_buffer *buffer)
 {
-	return kref_put(&buffer->ref, ion_buffer_destroy);
+	return kref_put(&buffer->ref, _ion_buffer_destroy);
 }
 
 static void ion_buffer_add_to_handle(struct ion_buffer *buffer)
@@ -1298,80 +1291,53 @@ static const struct file_operations debug_heap_fops = {
 	.release = single_release,
 };
 
-static size_t ion_heap_free_list_is_empty(struct ion_heap *heap)
+#ifdef DEBUG_HEAP_SHRINKER
+static int debug_shrink_set(void *data, u64 val)
 {
-	bool is_empty;
+        struct ion_heap *heap = data;
+        struct shrink_control sc;
+        int objs;
 
-	rt_mutex_lock(&heap->lock);
-	is_empty = list_empty(&heap->free_list);
-	rt_mutex_unlock(&heap->lock);
+        sc.gfp_mask = -1;
+        sc.nr_to_scan = 0;
 
-	return is_empty;
-}
-
-static int ion_heap_deferred_free(void *data)
-{
-	struct ion_heap *heap = data;
+        if (!val)
+                return 0;
 
-	while (true) {
-		struct ion_buffer *buffer;
+        objs = heap->shrinker.shrink(&heap->shrinker, &sc);
+        sc.nr_to_scan = objs;
 
-		wait_event_freezable(heap->waitqueue,
-				     !ion_heap_free_list_is_empty(heap));
-
-		rt_mutex_lock(&heap->lock);
-		if (list_empty(&heap->free_list)) {
-			rt_mutex_unlock(&heap->lock);
-			continue;
-		}
-		buffer = list_first_entry(&heap->free_list, struct ion_buffer,
-					  list);
-		list_del(&buffer->list);
-		rt_mutex_unlock(&heap->lock);
-		_ion_buffer_destroy(buffer);
-	}
-
-	return 0;
+        heap->shrinker.shrink(&heap->shrinker, &sc);
+        return 0;
 }
 
-static bool ion_heap_drain_freelist(struct ion_heap *heap)
+static int debug_shrink_get(void *data, u64 *val)
 {
-	struct ion_buffer *buffer, *tmp;
-
-	if (ion_heap_free_list_is_empty(heap))
-		return false;
-	rt_mutex_lock(&heap->lock);
-	list_for_each_entry_safe(buffer, tmp, &heap->free_list, list) {
-		list_del(&buffer->list);
-		_ion_buffer_destroy(buffer);
-	}
-	BUG_ON(!list_empty(&heap->free_list));
-	rt_mutex_unlock(&heap->lock);
+        struct ion_heap *heap = data;
+        struct shrink_control sc;
+        int objs;
 
+        sc.gfp_mask = -1;
+        sc.nr_to_scan = 0;
 
-	return true;
+        objs = heap->shrinker.shrink(&heap->shrinker, &sc);
+        *val = objs;
+        return 0;
 }
 
+DEFINE_SIMPLE_ATTRIBUTE(debug_shrink_fops, debug_shrink_get,
+                        debug_shrink_set, "%llu\n");
+#endif
+
 void ion_device_add_heap(struct ion_device *dev, struct ion_heap *heap)
 {
-	struct sched_param param = { .sched_priority = 0 };
-
 	if (!heap->ops->allocate || !heap->ops->free || !heap->ops->map_dma ||
 	    !heap->ops->unmap_dma)
 		pr_err("%s: can not add heap with invalid ops struct.\n",
 		       __func__);
 
-	if (heap->flags & ION_HEAP_FLAG_DEFER_FREE) {
-		INIT_LIST_HEAD(&heap->free_list);
-		rt_mutex_init(&heap->lock);
-		init_waitqueue_head(&heap->waitqueue);
-		heap->task = kthread_run(ion_heap_deferred_free, heap,
-					 "%s", heap->name);
-		sched_setscheduler(heap->task, SCHED_IDLE, &param);
-		if (IS_ERR(heap->task))
-			pr_err("%s: creating thread for deferred free failed\n",
-			       __func__);
-	}
+	if (heap->flags & ION_HEAP_FLAG_DEFER_FREE)
+		ion_heap_init_deferred_free(heap);
 
 	heap->dev = dev;
 	down_write(&dev->lock);
@@ -1381,6 +1347,15 @@ void ion_device_add_heap(struct ion_device *dev, struct ion_heap *heap)
 	plist_add(&heap->node, &dev->heaps);
 	debugfs_create_file(heap->name, 0664, dev->debug_root, heap,
 			    &debug_heap_fops);
+#ifdef DEBUG_HEAP_SHRINKER
+	if (heap->shrinker.shrink) {
+		char debug_name[64];
+
+		snprintf(debug_name, 64, "%s_shrink", heap->name);
+		debugfs_create_file(debug_name, 0644, dev->debug_root, heap,
+				    &debug_shrink_fops);
+	}
+#endif
 	up_write(&dev->lock);
 }
 
drivers/staging/android/ion/ion_heap.c +107 -0

120 lines not on this page · raw message

drivers/staging/android/ion/ion_page_pool.c +22 -104

180 lines not on this page · raw message

drivers/staging/android/ion/ion_priv.h +55 -2

93 lines not on this page · raw message

drivers/staging/android/ion/ion_system_heap.c +49 -0

61 lines not on this page · raw message

Signature · 1 line
1.8.3.2
#83 lore raw

PATCH 66/115 gpu: ion: __dma_page_cpu_to_dev -> arm_dma_ops.sync_single_for_device hack

In reply to John Stultz

From: Arve Hjønnevåg <[email protected]>

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_chunk_heap.c  | 10 ++++++----
 drivers/staging/android/ion/ion_page_pool.c   |  6 ++++--
 drivers/staging/android/ion/ion_system_heap.c |  5 +++--
 3 files changed, 13 insertions(+), 8 deletions(-)
drivers/staging/android/ion/ion_chunk_heap.c +6 -4
index 4c2f223..410c08e 100644
@@ -105,8 +105,9 @@ static void ion_chunk_heap_free(struct ion_buffer *buffer)
 
 	for_each_sg(table->sgl, sg, table->nents, i) {
 		if (ion_buffer_cached(buffer))
-			__dma_page_cpu_to_dev(sg_page(sg), 0, sg_dma_len(sg),
-					      DMA_BIDIRECTIONAL);
+			arm_dma_ops.sync_single_for_device(NULL,
+				pfn_to_dma(NULL, page_to_pfn(sg_page(sg))),
+				sg_dma_len(sg), DMA_BIDIRECTIONAL);
 		gen_pool_free(chunk_heap->pool, page_to_phys(sg_page(sg)),
 			      sg_dma_len(sg));
 	}
@@ -177,8 +178,9 @@ struct ion_heap *ion_chunk_heap_create(struct ion_platform_heap *heap_data)
 	}
 	free_vm_area(vm_struct);
 
-	__dma_page_cpu_to_dev(phys_to_page(heap_data->base), 0, heap_data->size,
-			      DMA_BIDIRECTIONAL);
+	arm_dma_ops.sync_single_for_device(NULL,
+		pfn_to_dma(NULL, page_to_pfn(phys_to_page(heap_data->base))),
+		heap_data->size, DMA_BIDIRECTIONAL);
 	gen_pool_add(chunk_heap->pool, chunk_heap->base, heap_data->size, -1);
 	chunk_heap->heap.ops = &chunk_heap_ops;
 	chunk_heap->heap.type = ION_HEAP_TYPE_CHUNK;
drivers/staging/android/ion/ion_page_pool.c +4 -2
index d624dee..855f30c 100644
@@ -37,8 +37,10 @@ static void *ion_page_pool_alloc_pages(struct ion_page_pool *pool)
 	/* this is only being used to flush the page for dma,
 	   this api is not really suitable for calling from a driver
 	   but no better way to flush a page for dma exist at this time */
-	__dma_page_cpu_to_dev(page, 0, PAGE_SIZE << pool->order,
-			      DMA_BIDIRECTIONAL);
+	arm_dma_ops.sync_single_for_device(NULL,
+					   pfn_to_dma(NULL, page_to_pfn(page)),
+					   PAGE_SIZE << pool->order,
+					   DMA_BIDIRECTIONAL);
 	return page;
 }
 
drivers/staging/android/ion/ion_system_heap.c +3 -2
index fedbc0d..d4f0a56 100644
@@ -78,8 +78,9 @@ static struct page *alloc_buffer_page(struct ion_system_heap *heap,
 		page = alloc_pages(gfp_flags, order);
 		if (!page)
 			return 0;
-		__dma_page_cpu_to_dev(page, 0, PAGE_SIZE << order,
-				      DMA_BIDIRECTIONAL);
+		arm_dma_ops.sync_single_for_device(NULL,
+			pfn_to_dma(NULL, page_to_pfn(page)),
+			PAGE_SIZE << order, DMA_BIDIRECTIONAL);
 	}
 	if (!page)
 		return 0;
Signature · 1 line
1.8.3.2
#84 lore raw

PATCH 67/115 gpu: ion: Remove __GFP_NO_KSWAPD

In reply to John Stultz

From: Arve Hjønnevåg <[email protected]>

It no longer exists.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_system_heap.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/staging/android/ion/ion_system_heap.c +2 -2
index d4f0a56..344f753 100644
@@ -27,8 +27,8 @@
 #include "ion_priv.h"
 
 static unsigned int high_order_gfp_flags = (GFP_HIGHUSER | __GFP_ZERO |
-					    __GFP_NOWARN | __GFP_NORETRY |
-					    __GFP_NO_KSWAPD) & ~__GFP_WAIT;
+					    __GFP_NOWARN | __GFP_NORETRY) &
+					   ~__GFP_WAIT;
 static unsigned int low_order_gfp_flags  = (GFP_HIGHUSER | __GFP_ZERO |
 					 __GFP_NOWARN);
 static const unsigned int orders[] = {8, 4, 0};
Signature · 1 line
1.8.3.2
#85 lore raw

PATCH 68/115 ion: Add Kconfig dependency to ARM

In reply to John Stultz

The ion code has some very specific arm-isms which keeps it from building on other architectures. These should probably be resolved, but in the mean time, add a dependency on CONFIG_ARM to avoid build failures.

v2: Fix earlier flub, sending out an early untested version of the patch.

Cc Arve Hjønnevåg <[email protected]>
Cc Rebecca Schultz Zavin <[email protected]>
Cc Android Kernel Team <[email protected]>
[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/Kconfig | 1 +
 1 file changed, 1 insertion(+)
drivers/staging/android/ion/Kconfig +1 -0
index b5bfdb4..c62f2cb 100644
@@ -1,5 +1,6 @@
 menuconfig ION
 	tristate "Ion Memory Manager"
+	depends on ARM
 	select GENERIC_ALLOCATOR
 	select DMA_SHARED_BUFFER
 	help
Signature · 1 line
1.8.3.2
#86 lore raw

PATCH 69/115 gpu: ion: fix ion_platform_data definition

In reply to John Stultz

From: Benjamin Gaignard <[email protected]>

fix ion_platform_heap to make is use an usual way in board configuration file.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/android/ion/ion.h +1 -1
index 679031c..226bfd5 100644
@@ -103,7 +103,7 @@ struct ion_platform_heap {
  */
 struct ion_platform_data {
 	int nr;
-	struct ion_platform_heap heaps[];
+	struct ion_platform_heap *heaps;
 };
 
 /**
Signature · 1 line
1.8.3.2
#87 lore raw

PATCH 70/115 gpu: ion: add CMA heap

In reply to John Stultz

From: Benjamin Gaignard <[email protected]>

New heap type ION_HEAP_TYPE_DMA where allocation is done with dma_alloc_coherent API. device coherent_dma_mask must be set to DMA_BIT_MASK(32). ion_platform_heap private field is used to retrieve the device linked to CMA, if NULL the default CMA area is used.
ion_cma_get_sgtable is a copy of dma_common_get_sgtable function which should be in kernel 3.5

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/Makefile       |   2 +-
 drivers/staging/android/ion/ion.h          |   7 +-
 drivers/staging/android/ion/ion_cma_heap.c | 233 +++++++++++++++++++++++++++++
 drivers/staging/android/ion/ion_heap.c     |   6 +
 drivers/staging/android/ion/ion_priv.h     |   3 +
 5 files changed, 249 insertions(+), 2 deletions(-)
 create mode 100644 drivers/staging/android/ion/ion_cma_heap.c
drivers/staging/android/ion/Makefile +1 -1
index 306fff9..ce0f1ef 100644
@@ -1,3 +1,3 @@
 obj-$(CONFIG_ION) +=	ion.o ion_heap.o ion_page_pool.o ion_system_heap.o \
-			ion_carveout_heap.o ion_chunk_heap.o
+			ion_carveout_heap.o ion_chunk_heap.o ion_cma_heap.o
 obj-$(CONFIG_ION_TEGRA) += tegra/
drivers/staging/android/ion/ion.h +6 -1
index 226bfd5..3942b3e 100644
@@ -27,13 +27,17 @@ struct ion_handle;
  * @ION_HEAP_TYPE_CARVEOUT:	 memory allocated from a prereserved
  * 				 carveout heap, allocations are physically
  * 				 contiguous
- * @ION_HEAP_END:		 helper for iterating over heaps
+ * @ION_HEAP_TYPE_DMA:		 memory allocated via DMA API
+ * @ION_NUM_HEAPS:		 helper for iterating over heaps, a bit mask
+ * 				 is used to identify the heaps, so only 32
+ * 				 total heap types are supported
  */
 enum ion_heap_type {
 	ION_HEAP_TYPE_SYSTEM,
 	ION_HEAP_TYPE_SYSTEM_CONTIG,
 	ION_HEAP_TYPE_CARVEOUT,
 	ION_HEAP_TYPE_CHUNK,
+	ION_HEAP_TYPE_DMA,
 	ION_HEAP_TYPE_CUSTOM, /* must be last so device specific heaps always
 				 are at the end of this enum */
 	ION_NUM_HEAPS,
@@ -42,6 +46,7 @@ enum ion_heap_type {
 #define ION_HEAP_SYSTEM_MASK		(1 << ION_HEAP_TYPE_SYSTEM)
 #define ION_HEAP_SYSTEM_CONTIG_MASK	(1 << ION_HEAP_TYPE_SYSTEM_CONTIG)
 #define ION_HEAP_CARVEOUT_MASK		(1 << ION_HEAP_TYPE_CARVEOUT)
+#define ION_HEAP_TYPE_DMA_MASK		(1 << ION_HEAP_TYPE_DMA)
 
 #define ION_NUM_HEAP_IDS		sizeof(unsigned int) * 8
 
drivers/staging/android/ion/ion_cma_heap.c new file +233 -0

233 lines not on this page · raw message

drivers/staging/android/ion/ion_heap.c +6 -0
index f541bff..3fc1dcc 100644
@@ -254,6 +254,9 @@ struct ion_heap *ion_heap_create(struct ion_platform_heap *heap_data)
 	case ION_HEAP_TYPE_CHUNK:
 		heap = ion_chunk_heap_create(heap_data);
 		break;
+	case ION_HEAP_TYPE_DMA:
+		heap = ion_cma_heap_create(heap_data);
+		break;
 	default:
 		pr_err("%s: Invalid heap type %d\n", __func__,
 		       heap_data->type);
@@ -290,6 +293,9 @@ void ion_heap_destroy(struct ion_heap *heap)
 	case ION_HEAP_TYPE_CHUNK:
 		ion_chunk_heap_destroy(heap);
 		break;
+	case ION_HEAP_TYPE_DMA:
+		ion_cma_heap_destroy(heap);
+		break;
 	default:
 		pr_err("%s: Invalid heap type %d\n", __func__,
 		       heap->type);
drivers/staging/android/ion/ion_priv.h +3 -0
index bdc1e34..f5a09b6 100644
@@ -270,6 +270,9 @@ void ion_carveout_heap_destroy(struct ion_heap *);
 
 struct ion_heap *ion_chunk_heap_create(struct ion_platform_heap *);
 void ion_chunk_heap_destroy(struct ion_heap *);
+struct ion_heap *ion_cma_heap_create(struct ion_platform_heap *);
+void ion_cma_heap_destroy(struct ion_heap *);
+
 /**
  * kernel api to allocate/free from carveout -- used when carveout is
  * used to back an architecture specific custom heap
Signature · 1 line
1.8.3.2
#88 lore raw

PATCH 71/115 gpu: ion: Fix performance issue in faulting code

In reply to John Stultz

From: Rebecca Schultz Zavin <[email protected]>

Previously the code to fault ion buffers in one page at a time had a performance problem caused by the requirement to traverse the sg list looking for the right page to load in (a result of the fact that the items in the list may not be of uniform size). To fix the problem, for buffers that will be faulted in, also keep a flat array of all the pages in the buffer to use from the fault handler. To recover some of the additional memory footprint this creates per buffer, dirty bits used to indicate which pages have been faulted in to the cpu are now stored in the low bit of each page struct pointer in the page array.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c             | 105 +++++++++++++++-----------
 drivers/staging/android/ion/ion_cma_heap.c    |  35 +--------
 drivers/staging/android/ion/ion_heap.c        |  18 ++++-
 drivers/staging/android/ion/ion_priv.h        |  20 ++++-
 drivers/staging/android/ion/ion_system_heap.c |  25 +-----
 5 files changed, 99 insertions(+), 104 deletions(-)
drivers/staging/android/ion/ion.c +60 -45
index ddf8fde..38d65c2 100644
@@ -31,6 +31,7 @@
 #include <linux/slab.h>
 #include <linux/seq_file.h>
 #include <linux/uaccess.h>
+#include <linux/vmalloc.h>
 #include <linux/debugfs.h>
 #include <linux/dma-buf.h>
 
@@ -104,13 +105,33 @@ struct ion_handle {
 
 bool ion_buffer_fault_user_mappings(struct ion_buffer *buffer)
 {
-        return ((buffer->flags & ION_FLAG_CACHED) &&
-                !(buffer->flags & ION_FLAG_CACHED_NEEDS_SYNC));
+	return ((buffer->flags & ION_FLAG_CACHED) &&
+		!(buffer->flags & ION_FLAG_CACHED_NEEDS_SYNC));
 }
 
 bool ion_buffer_cached(struct ion_buffer *buffer)
 {
-        return !!(buffer->flags & ION_FLAG_CACHED);
+	return !!(buffer->flags & ION_FLAG_CACHED);
+}
+
+static inline struct page *ion_buffer_page(struct page *page)
+{
+	return (struct page *)((unsigned long)page & ~(1UL));
+}
+
+static inline bool ion_buffer_page_is_dirty(struct page *page)
+{
+	return !!((unsigned long)page & 1UL);
+}
+
+static inline void ion_buffer_page_dirty(struct page **page)
+{
+	*page = (struct page *)((unsigned long)(*page) | 1UL);
+}
+
+static inline void ion_buffer_page_clean(struct page **page)
+{
+	*page = (struct page *)((unsigned long)(*page) & ~(1UL));
 }
 
 /* this function should only be called while dev->lock is held */
@@ -139,8 +160,6 @@ static void ion_buffer_add(struct ion_device *dev,
 	rb_insert_color(&buffer->node, &dev->buffers);
 }
 
-static int ion_buffer_alloc_dirty(struct ion_buffer *buffer);
-
 /* this function should only be called while dev->lock is held */
 static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 				     struct ion_device *dev,
@@ -185,17 +204,23 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 	}
 	buffer->sg_table = table;
 	if (ion_buffer_fault_user_mappings(buffer)) {
-		for_each_sg(buffer->sg_table->sgl, sg, buffer->sg_table->nents,
-			    i) {
-			if (sg_dma_len(sg) == PAGE_SIZE)
-				continue;
-			pr_err("%s: cached mappings that will be faulted in "
-			       "must have pagewise sg_lists\n", __func__);
-			ret = -EINVAL;
-			goto err;
+		int num_pages = PAGE_ALIGN(buffer->size) / PAGE_SIZE;
+		struct scatterlist *sg;
+		int i, j, k = 0;
+
+		buffer->pages = vmalloc(sizeof(struct page *) * num_pages);
+		if (!buffer->pages) {
+			ret = -ENOMEM;
+			goto err1;
+		}
+
+		for_each_sg(table->sgl, sg, table->nents, i) {
+			struct page *page = sg_page(sg);
+
+			for (j = 0; j < sg_dma_len(sg) / PAGE_SIZE; j++)
+				buffer->pages[k++] = page++;
 		}
 
-		ret = ion_buffer_alloc_dirty(buffer);
 		if (ret)
 			goto err;
 	}
@@ -222,6 +247,9 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 err:
 	heap->ops->unmap_dma(heap, buffer);
 	heap->ops->free(buffer);
+err1:
+	if (buffer->pages)
+		vfree(buffer->pages);
 err2:
 	kfree(buffer);
 	return ERR_PTR(ret);
@@ -233,8 +261,8 @@ void ion_buffer_destroy(struct ion_buffer *buffer)
 		buffer->heap->ops->unmap_kernel(buffer->heap, buffer);
 	buffer->heap->ops->unmap_dma(buffer->heap, buffer);
 	buffer->heap->ops->free(buffer);
-	if (buffer->flags & ION_FLAG_CACHED)
-		kfree(buffer->dirty);
+	if (buffer->pages)
+		vfree(buffer->pages);
 	kfree(buffer);
 }
 
@@ -764,17 +792,6 @@ static void ion_unmap_dma_buf(struct dma_buf_attachment *attachment,
 {
 }
 
-static int ion_buffer_alloc_dirty(struct ion_buffer *buffer)
-{
-	unsigned long pages = buffer->sg_table->nents;
-	unsigned long length = (pages + BITS_PER_LONG - 1)/BITS_PER_LONG;
-
-	buffer->dirty = kzalloc(length * sizeof(unsigned long), GFP_KERNEL);
-	if (!buffer->dirty)
-		return -ENOMEM;
-	return 0;
-}
-
 struct ion_vma_list {
 	struct list_head list;
 	struct vm_area_struct *vma;
@@ -784,9 +801,9 @@ static void ion_buffer_sync_for_device(struct ion_buffer *buffer,
 				       struct device *dev,
 				       enum dma_data_direction dir)
 {
-	struct scatterlist *sg;
-	int i;
 	struct ion_vma_list *vma_list;
+	int pages = PAGE_ALIGN(buffer->size) / PAGE_SIZE;
+	int i;
 
 	pr_debug("%s: syncing for device %s\n", __func__,
 		 dev ? dev_name(dev) : "null");
@@ -795,11 +812,12 @@ static void ion_buffer_sync_for_device(struct ion_buffer *buffer,
 		return;
 
 	mutex_lock(&buffer->lock);
-	for_each_sg(buffer->sg_table->sgl, sg, buffer->sg_table->nents, i) {
-		if (!test_bit(i, buffer->dirty))
-			continue;
-		dma_sync_sg_for_device(dev, sg, 1, dir);
-		clear_bit(i, buffer->dirty);
+	for (i = 0; i < pages; i++) {
+		struct page *page = buffer->pages[i];
+
+		if (ion_buffer_page_is_dirty(page))
+			__dma_page_cpu_to_dev(page, 0, PAGE_SIZE, dir);
+		ion_buffer_page_clean(buffer->pages + i);
 	}
 	list_for_each_entry(vma_list, &buffer->vmas, list) {
 		struct vm_area_struct *vma = vma_list->vma;
@@ -813,21 +831,18 @@ static void ion_buffer_sync_for_device(struct ion_buffer *buffer,
 int ion_vm_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
 {
 	struct ion_buffer *buffer = vma->vm_private_data;
-	struct scatterlist *sg;
-	int i;
+	int ret;
 
 	mutex_lock(&buffer->lock);
-	set_bit(vmf->pgoff, buffer->dirty);
+	ion_buffer_page_dirty(buffer->pages + vmf->pgoff);
 
-	for_each_sg(buffer->sg_table->sgl, sg, buffer->sg_table->nents, i) {
-		if (i != vmf->pgoff)
-			continue;
-		dma_sync_sg_for_cpu(NULL, sg, 1, DMA_BIDIRECTIONAL);
-		vm_insert_page(vma, (unsigned long)vmf->virtual_address,
-			       sg_page(sg));
-		break;
-	}
+	BUG_ON(!buffer->pages || !buffer->pages[vmf->pgoff]);
+	ret = vm_insert_page(vma, (unsigned long)vmf->virtual_address,
+			     ion_buffer_page(buffer->pages[vmf->pgoff]));
 	mutex_unlock(&buffer->lock);
+	if (ret)
+		return VM_FAULT_ERROR;
+
 	return VM_FAULT_NOPAGE;
 }
 
drivers/staging/android/ion/ion_cma_heap.c +3 -32
index 74e3c77..86b6cf5 100644
@@ -58,29 +58,6 @@ int ion_cma_get_sgtable(struct device *dev, struct sg_table *sgt,
 	return 0;
 }
 
-/*
- * Create scatter-list for each page of the already allocated DMA buffer.
- */
-int ion_cma_get_sgtable_per_page(struct device *dev, struct sg_table *sgt,
-			void *cpu_addr, dma_addr_t handle, size_t size)
-{
-	struct page *page = virt_to_page(cpu_addr);
-	int ret, i;
-	struct scatterlist *sg;
-
-	ret = sg_alloc_table(sgt, PAGE_ALIGN(size) / PAGE_SIZE, GFP_KERNEL);
-	if (unlikely(ret))
-		return ret;
-
-	sg = sgt->sgl;
-	for (i = 0; i < (PAGE_ALIGN(size) / PAGE_SIZE); i++) {
-		page = virt_to_page(cpu_addr + (i * PAGE_SIZE));
-		sg_set_page(sg, page, PAGE_SIZE, 0);
-		sg = sg_next(sg);
-	}
-	return 0;
-}
-
 /* ION CMA heap operations functions */
 static int ion_cma_allocate(struct ion_heap *heap, struct ion_buffer *buffer,
 			    unsigned long len, unsigned long align,
@@ -111,15 +88,9 @@ static int ion_cma_allocate(struct ion_heap *heap, struct ion_buffer *buffer,
 		goto free_mem;
 	}
 
-	if (ion_buffer_fault_user_mappings(buffer)) {
-		if (ion_cma_get_sgtable_per_page
-			(dev, info->table, info->cpu_addr, info->handle, len))
-			goto free_table;
-	} else {
-		if (ion_cma_get_sgtable
-			(dev, info->table, info->cpu_addr, info->handle, len))
-			goto free_table;
-	}
+	if (ion_cma_get_sgtable
+	    (dev, info->table, info->cpu_addr, info->handle, len))
+		goto free_table;
 	/* keep this for memory release */
 	buffer->priv_virt = info;
 	dev_dbg(dev, "Allocate buffer %p\n", buffer);
drivers/staging/android/ion/ion_heap.c +16 -2
index 3fc1dcc..cc2a425 100644
@@ -134,8 +134,22 @@ end:
 	return ret;
 }
 
-void ion_heap_free_page(struct ion_buffer *buffer, struct page *page,
-		       unsigned int order)
+struct page *ion_heap_alloc_pages(struct ion_buffer *buffer, gfp_t gfp_flags,
+				  unsigned int order)
+{
+	struct page *page = alloc_pages(gfp_flags, order);
+
+	if (!page)
+		return page;
+
+	if (ion_buffer_fault_user_mappings(buffer))
+		split_page(page, order);
+
+	return page;
+}
+
+void ion_heap_free_pages(struct ion_buffer *buffer, struct page *page,
+			 unsigned int order)
 {
 	int i;
 
drivers/staging/android/ion/ion_priv.h +16 -4
index f5a09b6..965471a 100644
@@ -46,9 +46,8 @@ struct ion_buffer *ion_handle_buffer(struct ion_handle *handle);
  * @vaddr:		the kenrel mapping if kmap_cnt is not zero
  * @dmap_cnt:		number of times the buffer is mapped for dma
  * @sg_table:		the sg table for the buffer if dmap_cnt is not zero
- * @dirty:		bitmask representing which pages of this buffer have
- *			been dirtied by the cpu and need cache maintenance
- *			before dma
+ * @pages:		flat array of pages in the buffer -- used by fault
+ *			handler and only valid for buffers that are faulted in
  * @vmas:		list of vma's mapping this buffer
  * @handle_count:	count of handles referencing this buffer
  * @task_comm:		taskcomm of last client to reference this buffer in a
@@ -75,7 +74,7 @@ struct ion_buffer {
 	void *vaddr;
 	int dmap_cnt;
 	struct sg_table *sg_table;
-	unsigned long *dirty;
+	struct page **pages;
 	struct list_head vmas;
 	/* used to track orphaned buffers */
 	int handle_count;
@@ -214,6 +213,19 @@ int ion_heap_map_user(struct ion_heap *, struct ion_buffer *,
 int ion_heap_buffer_zero(struct ion_buffer *buffer);
 
 /**
+ * ion_heap_alloc_pages - allocate pages from alloc_pages
+ * @buffer:		the buffer to allocate for, used to extract the flags
+ * @gfp_flags:		the gfp_t for the allocation
+ * @order:		the order of the allocatoin
+ *
+ * This funciton allocations from alloc pages and also does any other
+ * necessary operations based on the buffer->flags.  For buffers which
+ * will be faulted in the pages are split using split_page
+ */
+struct page *ion_heap_alloc_pages(struct ion_buffer *buffer, gfp_t gfp_flags,
+				  unsigned int order);
+
+/**
  * ion_heap_init_deferred_free -- initialize deferred free functionality
  * @heap:		the heap
  *
drivers/staging/android/ion/ion_system_heap.c +4 -21
index 344f753..b9b1036 100644
@@ -64,7 +64,6 @@ static struct page *alloc_buffer_page(struct ion_system_heap *heap,
 				      unsigned long order)
 {
 	bool cached = ion_buffer_cached(buffer);
-	bool split_pages = ion_buffer_fault_user_mappings(buffer);
 	struct ion_page_pool *pool = heap->pools[order_to_index(order)];
 	struct page *page;
 
@@ -75,7 +74,7 @@ static struct page *alloc_buffer_page(struct ion_system_heap *heap,
 
 		if (order > 4)
 			gfp_flags = high_order_gfp_flags;
-		page = alloc_pages(gfp_flags, order);
+		page = ion_heap_alloc_pages(buffer, gfp_flags, order);
 		if (!page)
 			return 0;
 		arm_dma_ops.sync_single_for_device(NULL,
@@ -85,8 +84,6 @@ static struct page *alloc_buffer_page(struct ion_system_heap *heap,
 	if (!page)
 		return 0;
 
-	if (split_pages)
-		split_page(page, order);
 	return page;
 }
 
@@ -153,7 +150,6 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 	int i = 0;
 	long size_remaining = PAGE_ALIGN(size);
 	unsigned int max_order = orders[0];
-	bool split_pages = ion_buffer_fault_user_mappings(buffer);
 
 	INIT_LIST_HEAD(&pages);
 	while (size_remaining > 0) {
@@ -170,28 +166,15 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 	if (!table)
 		goto err;
 
-	if (split_pages)
-		ret = sg_alloc_table(table, PAGE_ALIGN(size) / PAGE_SIZE,
-				     GFP_KERNEL);
-	else
-		ret = sg_alloc_table(table, i, GFP_KERNEL);
-
+	ret = sg_alloc_table(table, i, GFP_KERNEL);
 	if (ret)
 		goto err1;
 
 	sg = table->sgl;
 	list_for_each_entry_safe(info, tmp_info, &pages, list) {
 		struct page *page = info->page;
-		if (split_pages) {
-			for (i = 0; i < (1 << info->order); i++) {
-				sg_set_page(sg, page + i, PAGE_SIZE, 0);
-				sg = sg_next(sg);
-			}
-		} else {
-			sg_set_page(sg, page, (1 << info->order) * PAGE_SIZE,
-				    0);
-			sg = sg_next(sg);
-		}
+		sg_set_page(sg, page, (1 << info->order) * PAGE_SIZE, 0);
+		sg = sg_next(sg);
 		list_del(&info->list);
 		kfree(info);
 	}
Signature · 1 line
1.8.3.2
#89 lore raw

PATCH 72/115 ion: chunk_heap: fix leak in allocated counter

In reply to John Stultz

From: Colin Cross <[email protected]>

buffer->size is controlled by the outer ion layer, don't modify it inside the heap. Instead, compute the rounded up allocated size on demand.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_chunk_heap.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
drivers/staging/android/ion/ion_chunk_heap.c +9 -6
index 410c08e..85fefe7 100644
@@ -47,15 +47,15 @@ static int ion_chunk_heap_allocate(struct ion_heap *heap,
 	struct scatterlist *sg;
 	int ret, i;
 	unsigned long num_chunks;
+	unsigned long allocated_size;
 
 	if (ion_buffer_fault_user_mappings(buffer))
 		return -ENOMEM;
 
-	num_chunks = ALIGN(size, chunk_heap->chunk_size) /
-		chunk_heap->chunk_size;
-	buffer->size = num_chunks * chunk_heap->chunk_size;
+	allocated_size = ALIGN(size, chunk_heap->chunk_size);
+	num_chunks = allocated_size / chunk_heap->chunk_size;
 
-	if (buffer->size > chunk_heap->size - chunk_heap->allocated)
+	if (allocated_size > chunk_heap->size - chunk_heap->allocated)
 		return -ENOMEM;
 
 	table = kzalloc(sizeof(struct sg_table), GFP_KERNEL);
@@ -78,7 +78,7 @@ static int ion_chunk_heap_allocate(struct ion_heap *heap,
 	}
 
 	buffer->priv_virt = table;
-	chunk_heap->allocated += buffer->size;
+	chunk_heap->allocated += allocated_size;
 	return 0;
 err:
 	sg = table->sgl;
@@ -100,6 +100,9 @@ static void ion_chunk_heap_free(struct ion_buffer *buffer)
 	struct sg_table *table = buffer->priv_virt;
 	struct scatterlist *sg;
 	int i;
+	unsigned long allocated_size;
+
+	allocated_size = ALIGN(buffer->size, chunk_heap->chunk_size);
 
 	ion_heap_buffer_zero(buffer);
 
@@ -111,7 +114,7 @@ static void ion_chunk_heap_free(struct ion_buffer *buffer)
 		gen_pool_free(chunk_heap->pool, page_to_phys(sg_page(sg)),
 			      sg_dma_len(sg));
 	}
-	chunk_heap->allocated -= buffer->size;
+	chunk_heap->allocated -= allocated_size;
 	sg_free_table(table);
 	kfree(table);
 }
Signature · 1 line
1.8.3.2
#90 lore raw

PATCH 73/115 ion: add free list size to heap debug files

In reply to John Stultz

From: Colin Cross <[email protected]>

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/staging/android/ion/ion.c +3 -0
index 38d65c2..93b84a9 100644
@@ -1286,6 +1286,9 @@ static int ion_debug_heap_show(struct seq_file *s, void *unused)
 	seq_printf(s, "%16.s %16u\n", "total orphaned",
 		   total_orphaned_size);
 	seq_printf(s, "%16.s %16u\n", "total ", total_size);
+	if (heap->flags & ION_HEAP_FLAG_DEFER_FREE)
+		seq_printf(s, "%16.s %16u\n", "deferred free",
+				heap->free_list_size);
 	seq_printf(s, "----------------------------------------------------\n");
 
 	if (heap->debug_show)
Signature · 1 line
1.8.3.2
#91 lore raw

PATCH 74/115 ion: convert map_kernel to return ERR_PTR

In reply to John Stultz

From: Colin Cross <[email protected]>

ion is going to stop accepting NULL as an error value, use ERR_PTR.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion_carveout_heap.c | 7 ++++++-
 drivers/staging/android/ion/ion_heap.c          | 3 +++
 2 files changed, 9 insertions(+), 1 deletion(-)
drivers/staging/android/ion/ion_carveout_heap.c +6 -1
index fd64168..36df34c 100644
@@ -112,13 +112,18 @@ void ion_carveout_heap_unmap_dma(struct ion_heap *heap,
 void *ion_carveout_heap_map_kernel(struct ion_heap *heap,
 				   struct ion_buffer *buffer)
 {
+	void *ret;
 	int mtype = MT_MEMORY_NONCACHED;
 
 	if (buffer->flags & ION_FLAG_CACHED)
 		mtype = MT_MEMORY;
 
-	return __arm_ioremap(buffer->priv_phys, buffer->size,
+	ret = __arm_ioremap(buffer->priv_phys, buffer->size,
 			      mtype);
+	if (ret == NULL)
+		return ERR_PTR(-ENOMEM);
+
+	return ret;
 }
 
 void ion_carveout_heap_unmap_kernel(struct ion_heap *heap,
drivers/staging/android/ion/ion_heap.c +3 -0
index cc2a425..a584ec0 100644
@@ -56,6 +56,9 @@ void *ion_heap_map_kernel(struct ion_heap *heap,
 	vaddr = vmap(pages, npages, VM_MAP, pgprot);
 	vfree(pages);
 
+	if (vaddr == NULL)
+		return ERR_PTR(-ENOMEM);
+
 	return vaddr;
 }
 
Signature · 1 line
1.8.3.2
#92 lore raw

PATCH 75/115 ion: remove IS_ERR_OR_NULL

In reply to John Stultz

From: Colin Cross <[email protected]>

IS_ERR_OR_NULL is often part of a bad pattern that can accidentally return 0 on error:
if (IS_ERR_OR_NULL(ptr))

    return PTR_ERR(ptr);

It also usually means that the errors of a function are not well defined. Replace all uses in ion.c by ensure that the return type of any function in ion is an ERR_PTR.

Specify that the expected return value from map_kernel or map_dma heap ops is ERR_PTR, and warn if a heap returns NULL.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c      | 28 +++++++++++++++-------------
 drivers/staging/android/ion/ion_priv.h |  3 +++
 2 files changed, 18 insertions(+), 13 deletions(-)
drivers/staging/android/ion/ion.c +15 -13
index 93b84a9..54a00fe 100644
@@ -197,7 +197,9 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 	buffer->size = len;
 
 	table = heap->ops->map_dma(heap, buffer);
-	if (IS_ERR_OR_NULL(table)) {
+	if (WARN_ONCE(table == NULL, "heap->ops->map_dma should return ERR_PTR on error"))
+		table = ERR_PTR(-EINVAL);
+	if (IS_ERR(table)) {
 		heap->ops->free(buffer);
 		kfree(buffer);
 		return ERR_PTR(PTR_ERR(table));
@@ -389,7 +391,7 @@ static struct ion_handle *ion_handle_lookup(struct ion_client *client,
 		if (handle->buffer == buffer)
 			return handle;
 	}
-	return NULL;
+	return ERR_PTR(-EINVAL);
 }
 
 static bool ion_handle_validate(struct ion_client *client, struct ion_handle *handle)
@@ -459,7 +461,7 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 		if (!((1 << heap->id) & heap_id_mask))
 			continue;
 		buffer = ion_buffer_create(heap, dev, len, align, flags);
-		if (!IS_ERR_OR_NULL(buffer))
+		if (!IS_ERR(buffer))
 			break;
 	}
 	up_read(&dev->lock);
@@ -543,7 +545,9 @@ static void *ion_buffer_kmap_get(struct ion_buffer *buffer)
 		return buffer->vaddr;
 	}
 	vaddr = buffer->heap->ops->map_kernel(buffer->heap, buffer);
-	if (IS_ERR_OR_NULL(vaddr))
+	if (WARN_ONCE(vaddr == NULL, "heap->ops->map_kernel should return ERR_PTR on error"))
+		return ERR_PTR(-EINVAL);
+	if (IS_ERR(vaddr))
 		return vaddr;
 	buffer->vaddr = vaddr;
 	buffer->kmap_cnt++;
@@ -560,7 +564,7 @@ static void *ion_handle_kmap_get(struct ion_handle *handle)
 		return buffer->vaddr;
 	}
 	vaddr = ion_buffer_kmap_get(buffer);
-	if (IS_ERR_OR_NULL(vaddr))
+	if (IS_ERR(vaddr))
 		return vaddr;
 	handle->kmap_cnt++;
 	return vaddr;
@@ -954,8 +958,6 @@ static int ion_dma_buf_begin_cpu_access(struct dma_buf *dmabuf, size_t start,
 	mutex_unlock(&buffer->lock);
 	if (IS_ERR(vaddr))
 		return PTR_ERR(vaddr);
-	if (!vaddr)
-		return -ENOMEM;
 	return 0;
 }
 
@@ -1034,7 +1036,7 @@ struct ion_handle *ion_import_dma_buf(struct ion_client *client, int fd)
 	struct ion_handle *handle;
 
 	dmabuf = dma_buf_get(fd);
-	if (IS_ERR_OR_NULL(dmabuf))
+	if (IS_ERR(dmabuf))
 		return ERR_PTR(PTR_ERR(dmabuf));
 	/* if this memory came from ion */
 
@@ -1049,12 +1051,12 @@ struct ion_handle *ion_import_dma_buf(struct ion_client *client, int fd)
 	mutex_lock(&client->lock);
 	/* if a handle exists for this buffer just take a reference to it */
 	handle = ion_handle_lookup(client, buffer);
-	if (!IS_ERR_OR_NULL(handle)) {
+	if (!IS_ERR(handle)) {
 		ion_handle_get(handle);
 		goto end;
 	}
 	handle = ion_handle_create(client, buffer);
-	if (IS_ERR_OR_NULL(handle))
+	if (IS_ERR(handle))
 		goto end;
 	ion_handle_add(client, handle);
 end:
@@ -1070,7 +1072,7 @@ static int ion_sync_for_device(struct ion_client *client, int fd)
 	struct ion_buffer *buffer;
 
 	dmabuf = dma_buf_get(fd);
-	if (IS_ERR_OR_NULL(dmabuf))
+	if (IS_ERR(dmabuf))
 		return PTR_ERR(dmabuf);
 
 	/* if this memory came from ion */
@@ -1204,7 +1206,7 @@ static int ion_open(struct inode *inode, struct file *file)
 
 	pr_debug("%s: %d\n", __func__, __LINE__);
 	client = ion_client_create(dev, "user");
-	if (IS_ERR_OR_NULL(client))
+	if (IS_ERR(client))
 		return PTR_ERR(client);
 	file->private_data = client;
 
@@ -1400,7 +1402,7 @@ struct ion_device *ion_device_create(long (*custom_ioctl)
 	}
 
 	idev->debug_root = debugfs_create_dir("ion", NULL);
-	if (IS_ERR_OR_NULL(idev->debug_root))
+	if (!idev->debug_root)
 		pr_err("ion: failed to create debug files.\n");
 
 	idev->custom_ioctl = custom_ioctl;
drivers/staging/android/ion/ion_priv.h +3 -0
index 965471a..f263563 100644
@@ -94,6 +94,9 @@ void ion_buffer_destroy(struct ion_buffer *buffer);
  * @map_kernel		map memory to the kernel
  * @unmap_kernel	unmap memory to the kernel
  * @map_user		map memory to userspace
+ *
+ * allocate, phys, and map_user return 0 on success, -errno on error.
+ * map_dma and map_kernel return pointer on success, ERR_PTR on error.
  */
 struct ion_heap_ops {
 	int (*allocate) (struct ion_heap *heap,
Signature · 1 line
1.8.3.2
#93 lore raw

PATCH 76/115 ion: replace userspace handle cookies with idr

In reply to John Stultz

From: Colin Cross <[email protected]>

Userspace handles should not leak kernel virtual addresses to userspace. They have to be validated by looking them up in an rbtree anyways, so replace them with an idr and validate them by using idr_find to convert the id number to the struct ion_handle pointer.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 107 +++++++++++++++++++++++++-------------
 1 file changed, 71 insertions(+), 36 deletions(-)
drivers/staging/android/ion/ion.c +71 -36
index 54a00fe..ece1a4c 100644
@@ -34,6 +34,7 @@
 #include <linux/vmalloc.h>
 #include <linux/debugfs.h>
 #include <linux/dma-buf.h>
+#include <linux/idr.h>
 
 #include "ion.h"
 #include "ion_priv.h"
@@ -64,6 +65,7 @@ struct ion_device {
  * @node:		node in the tree of all clients
  * @dev:		backpointer to ion device
  * @handles:		an rb tree of all the handles in this client
+ * @idr:		an idr space for allocating handle ids
  * @lock:		lock protecting the tree of handles
  * @name:		used for debugging
  * @task:		used for debugging
@@ -76,6 +78,7 @@ struct ion_client {
 	struct rb_node node;
 	struct ion_device *dev;
 	struct rb_root handles;
+	struct idr idr;
 	struct mutex lock;
 	const char *name;
 	struct task_struct *task;
@@ -90,7 +93,7 @@ struct ion_client {
  * @buffer:		pointer to the buffer
  * @node:		node in the client's handle rbtree
  * @kmap_cnt:		count of times this client has mapped to kernel
- * @dmap_cnt:		count of times this client has mapped for dma
+ * @id:			client-unique id allocated by client->idr
  *
  * Modifications to node, map_cnt or mapping should be protected by the
  * lock in the client.  Other fields are never changed after initialization.
@@ -101,6 +104,7 @@ struct ion_handle {
 	struct ion_buffer *buffer;
 	struct rb_node node;
 	unsigned int kmap_cnt;
+	int id;
 };
 
 bool ion_buffer_fault_user_mappings(struct ion_buffer *buffer)
@@ -356,6 +360,7 @@ static void ion_handle_destroy(struct kref *kref)
 		ion_handle_kmap_put(handle);
 	mutex_unlock(&buffer->lock);
 
+	idr_remove(&client->idr, handle->id);
 	if (!RB_EMPTY_NODE(&handle->node))
 		rb_erase(&handle->node, &client->handles);
 
@@ -387,36 +392,42 @@ static struct ion_handle *ion_handle_lookup(struct ion_client *client,
 
 	for (n = rb_first(&client->handles); n; n = rb_next(n)) {
 		struct ion_handle *handle = rb_entry(n, struct ion_handle,
-						     node);
+						   node);
 		if (handle->buffer == buffer)
 			return handle;
 	}
 	return ERR_PTR(-EINVAL);
 }
 
-static bool ion_handle_validate(struct ion_client *client, struct ion_handle *handle)
+static struct ion_handle *ion_uhandle_get(struct ion_client *client, int id)
 {
-	struct rb_node *n = client->handles.rb_node;
+	return idr_find(&client->idr, id);
+}
 
-	while (n) {
-		struct ion_handle *handle_node = rb_entry(n, struct ion_handle,
-							  node);
-		if (handle < handle_node)
-			n = n->rb_left;
-		else if (handle > handle_node)
-			n = n->rb_right;
-		else
-			return true;
-	}
-	return false;
+static bool ion_handle_validate(struct ion_client *client, struct ion_handle *handle)
+{
+	return (ion_uhandle_get(client, handle->id) == handle);
 }
 
-static void ion_handle_add(struct ion_client *client, struct ion_handle *handle)
+static int ion_handle_add(struct ion_client *client, struct ion_handle *handle)
 {
+	int rc;
 	struct rb_node **p = &client->handles.rb_node;
 	struct rb_node *parent = NULL;
 	struct ion_handle *entry;
 
+	do {
+		int id;
+		rc = idr_pre_get(&client->idr, GFP_KERNEL);
+		if (!rc)
+			return -ENOMEM;
+		rc = idr_get_new(&client->idr, handle, &id);
+		handle->id = id;
+	} while (rc == -EAGAIN);
+
+	if (rc < 0)
+		return rc;
+
 	while (*p) {
 		parent = *p;
 		entry = rb_entry(parent, struct ion_handle, node);
@@ -431,6 +442,8 @@ static void ion_handle_add(struct ion_client *client, struct ion_handle *handle)
 
 	rb_link_node(&handle->node, parent, p);
 	rb_insert_color(&handle->node, &client->handles);
+
+	return 0;
 }
 
 struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
@@ -441,6 +454,7 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 	struct ion_device *dev = client->dev;
 	struct ion_buffer *buffer = NULL;
 	struct ion_heap *heap;
+	int ret;
 
 	pr_debug("%s: len %d align %d heap_id_mask %u flags %x\n", __func__,
 		 len, align, heap_id_mask, flags);
@@ -480,12 +494,16 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 	 */
 	ion_buffer_put(buffer);
 
-	if (!IS_ERR(handle)) {
-		mutex_lock(&client->lock);
-		ion_handle_add(client, handle);
-		mutex_unlock(&client->lock);
-	}
+	if (IS_ERR(handle))
+		return handle;
 
+	mutex_lock(&client->lock);
+	ret = ion_handle_add(client, handle);
+	if (ret) {
+		ion_handle_put(handle);
+		handle = ERR_PTR(ret);
+	}
+	mutex_unlock(&client->lock);
 
 	return handle;
 }
@@ -705,6 +723,7 @@ struct ion_client *ion_client_create(struct ion_device *dev,
 
 	client->dev = dev;
 	client->handles = RB_ROOT;
+	idr_init(&client->idr);
 	mutex_init(&client->lock);
 	client->name = name;
 	client->task = task;
@@ -745,6 +764,10 @@ void ion_client_destroy(struct ion_client *client)
 						     node);
 		ion_handle_destroy(&handle->ref);
 	}
+
+	idr_remove_all(&client->idr);
+	idr_destroy(&client->idr);
+
 	down_write(&dev->lock);
 	if (client->task)
 		put_task_struct(client->task);
@@ -1034,6 +1057,7 @@ struct ion_handle *ion_import_dma_buf(struct ion_client *client, int fd)
 	struct dma_buf *dmabuf;
 	struct ion_buffer *buffer;
 	struct ion_handle *handle;
+	int ret;
 
 	dmabuf = dma_buf_get(fd);
 	if (IS_ERR(dmabuf))
@@ -1058,7 +1082,11 @@ struct ion_handle *ion_import_dma_buf(struct ion_client *client, int fd)
 	handle = ion_handle_create(client, buffer);
 	if (IS_ERR(handle))
 		goto end;
-	ion_handle_add(client, handle);
+	ret = ion_handle_add(client, handle);
+	if (ret) {
+		ion_handle_put(handle);
+		handle = ERR_PTR(ret);
+	}
 end:
 	mutex_unlock(&client->lock);
 	dma_buf_put(dmabuf);
@@ -1098,17 +1126,20 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 	case ION_IOC_ALLOC:
 	{
 		struct ion_allocation_data data;
+		struct ion_handle *handle;
 
 		if (copy_from_user(&data, (void __user *)arg, sizeof(data)))
 			return -EFAULT;
-		data.handle = ion_alloc(client, data.len, data.align,
+		handle = ion_alloc(client, data.len, data.align,
 					     data.heap_id_mask, data.flags);
 
-		if (IS_ERR(data.handle))
-			return PTR_ERR(data.handle);
+		if (IS_ERR(handle))
+			return PTR_ERR(handle);
+
+		data.handle = (struct ion_handle *)handle->id;
 
 		if (copy_to_user((void __user *)arg, &data, sizeof(data))) {
-			ion_free(client, data.handle);
+			ion_free(client, handle);
 			return -EFAULT;
 		}
 		break;
@@ -1116,27 +1147,29 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 	case ION_IOC_FREE:
 	{
 		struct ion_handle_data data;
-		bool valid;
+		struct ion_handle *handle;
 
 		if (copy_from_user(&data, (void __user *)arg,
 				   sizeof(struct ion_handle_data)))
 			return -EFAULT;
 		mutex_lock(&client->lock);
-		valid = ion_handle_validate(client, data.handle);
+		handle = ion_uhandle_get(client, (int)data.handle);
 		mutex_unlock(&client->lock);
-		if (!valid)
+		if (!handle)
 			return -EINVAL;
-		ion_free(client, data.handle);
+		ion_free(client, handle);
 		break;
 	}
 	case ION_IOC_SHARE:
 	case ION_IOC_MAP:
 	{
 		struct ion_fd_data data;
+		struct ion_handle *handle;
 
 		if (copy_from_user(&data, (void __user *)arg, sizeof(data)))
 			return -EFAULT;
-		data.fd = ion_share_dma_buf_fd(client, data.handle);
+		handle = ion_uhandle_get(client, (int)data.handle);
+		data.fd = ion_share_dma_buf_fd(client, handle);
 		if (copy_to_user((void __user *)arg, &data, sizeof(data)))
 			return -EFAULT;
 		if (data.fd < 0)
@@ -1146,15 +1179,17 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 	case ION_IOC_IMPORT:
 	{
 		struct ion_fd_data data;
+		struct ion_handle *handle;
 		int ret = 0;
 		if (copy_from_user(&data, (void __user *)arg,
 				   sizeof(struct ion_fd_data)))
 			return -EFAULT;
-		data.handle = ion_import_dma_buf(client, data.fd);
-		if (IS_ERR(data.handle)) {
-			ret = PTR_ERR(data.handle);
-			data.handle = NULL;
-		}
+		handle = ion_import_dma_buf(client, data.fd);
+		if (IS_ERR(handle))
+			ret = PTR_ERR(handle);
+		else
+			data.handle = (struct ion_handle *)handle->id;
+
 		if (copy_to_user((void __user *)arg, &data,
 				 sizeof(struct ion_fd_data)))
 			return -EFAULT;
Signature · 1 line
1.8.3.2
#94 lore raw

PATCH 77/115 ion: index client->handles rbtree by buffer

In reply to John Stultz

From: Colin Cross <[email protected]>

The only remaining users of the client->handles rbtree are iterating through it like a list. Keep the rbtree, but change its index to be the buffer address instead of the handle address, which makes ion_handle_lookup a fast rbtree search.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 21 ++++++++++++---------
 1 file changed, 12 insertions(+), 9 deletions(-)
drivers/staging/android/ion/ion.c +12 -9
index ece1a4c..ef0b2d7 100644
@@ -388,13 +388,16 @@ static int ion_handle_put(struct ion_handle *handle)
 static struct ion_handle *ion_handle_lookup(struct ion_client *client,
 					    struct ion_buffer *buffer)
 {
-	struct rb_node *n;
-
-	for (n = rb_first(&client->handles); n; n = rb_next(n)) {
-		struct ion_handle *handle = rb_entry(n, struct ion_handle,
-						   node);
-		if (handle->buffer == buffer)
-			return handle;
+	struct rb_node *n = client->handles.rb_node;
+
+	while (n) {
+		struct ion_handle *entry = rb_entry(n, struct ion_handle, node);
+		if (buffer < entry->buffer)
+			n = n->rb_left;
+		else if (buffer > entry->buffer)
+			n = n->rb_right;
+		else
+			return entry;
 	}
 	return ERR_PTR(-EINVAL);
 }
@@ -432,9 +435,9 @@ static int ion_handle_add(struct ion_client *client, struct ion_handle *handle)
 		parent = *p;
 		entry = rb_entry(parent, struct ion_handle, node);
 
-		if (handle < entry)
+		if (handle->buffer < entry->buffer)
 			p = &(*p)->rb_left;
-		else if (handle > entry)
+		else if (handle->buffer > entry->buffer)
 			p = &(*p)->rb_right;
 		else
 			WARN(1, "%s: buffer already found.", __func__);
Signature · 1 line
1.8.3.2
#95 lore raw

PATCH 78/115 ion: don't use id 0 for handle cookie

In reply to John Stultz

From: Colin Cross <[email protected]>

ion userspace clients think that the cookie is a pointer, so they use NULL to check if the handle has been initialized. Set the first id number to 1.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/android/ion/ion.c +1 -1
index ef0b2d7..273ec3a 100644
@@ -424,7 +424,7 @@ static int ion_handle_add(struct ion_client *client, struct ion_handle *handle)
 		rc = idr_pre_get(&client->idr, GFP_KERNEL);
 		if (!rc)
 			return -ENOMEM;
-		rc = idr_get_new(&client->idr, handle, &id);
+		rc = idr_get_new_above(&client->idr, handle, 1, &id);
 		handle->id = id;
 	} while (rc == -EAGAIN);
 
Signature · 1 line
1.8.3.2
#96 lore raw

PATCH 79/115 ion: add new ion_user_handle_t type for the user-space token

In reply to John Stultz

From: Rom Lemarchand <[email protected]>

Declare new ion_user_handle_t type to contain the token returned to user-space. This allows a 2-step migration of the user-space code to a new kernel header first, then will allow us to change the definition of the ion_user_handle_type_t to int without breaking the API.

(cherry picked from commit ebb8269bbb05b06ecedca3e21b3e65f23d48eadd)
[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 4 ++--
 drivers/staging/android/ion/ion.h | 8 +++++---
 2 files changed, 7 insertions(+), 5 deletions(-)
drivers/staging/android/ion/ion.c +2 -2
index 273ec3a..e2a3033 100644
@@ -1139,7 +1139,7 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 		if (IS_ERR(handle))
 			return PTR_ERR(handle);
 
-		data.handle = (struct ion_handle *)handle->id;
+		data.handle = (ion_user_handle_t)handle->id;
 
 		if (copy_to_user((void __user *)arg, &data, sizeof(data))) {
 			ion_free(client, handle);
@@ -1191,7 +1191,7 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 		if (IS_ERR(handle))
 			ret = PTR_ERR(handle);
 		else
-			data.handle = (struct ion_handle *)handle->id;
+			data.handle = (ion_user_handle_t)handle->id;
 
 		if (copy_to_user((void __user *)arg, &data,
 				 sizeof(struct ion_fd_data)))
drivers/staging/android/ion/ion.h +5 -3
index 3942b3e..244f085 100644
@@ -20,6 +20,8 @@
 #include <linux/types.h>
 
 struct ion_handle;
+typedef struct ion_handle *ion_user_handle_t;
+
 /**
  * enum ion_heap_types - list of all possible types of heaps
  * @ION_HEAP_TYPE_SYSTEM:	 memory allocated via vmalloc
@@ -268,7 +270,7 @@ struct ion_allocation_data {
 	size_t align;
 	unsigned int heap_id_mask;
 	unsigned int flags;
-	struct ion_handle *handle;
+	ion_user_handle_t handle;
 };
 
 /**
@@ -282,7 +284,7 @@ struct ion_allocation_data {
  * provides the file descriptor and the kernel returns the handle.
  */
 struct ion_fd_data {
-	struct ion_handle *handle;
+	ion_user_handle_t handle;
 	int fd;
 };
 
@@ -291,7 +293,7 @@ struct ion_fd_data {
  * @handle:	a handle
  */
 struct ion_handle_data {
-	struct ion_handle *handle;
+	ion_user_handle_t handle;
 };
 
 /**
Signature · 1 line
1.8.3.2
#97 lore raw

PATCH 80/115 ion: change ion_user_handle_t definition to int

In reply to John Stultz

From: Rom Lemarchand <[email protected]>

Turn ion_user_handle_t to int. This change reflects the underlying type returned by the ion driver.

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/ion.c | 8 ++++----
 drivers/staging/android/ion/ion.h | 4 ++--
 2 files changed, 6 insertions(+), 6 deletions(-)
drivers/staging/android/ion/ion.c +4 -4
index e2a3033..1c40b74 100644
@@ -1139,7 +1139,7 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 		if (IS_ERR(handle))
 			return PTR_ERR(handle);
 
-		data.handle = (ion_user_handle_t)handle->id;
+		data.handle = handle->id;
 
 		if (copy_to_user((void __user *)arg, &data, sizeof(data))) {
 			ion_free(client, handle);
@@ -1156,7 +1156,7 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 				   sizeof(struct ion_handle_data)))
 			return -EFAULT;
 		mutex_lock(&client->lock);
-		handle = ion_uhandle_get(client, (int)data.handle);
+		handle = ion_uhandle_get(client, data.handle);
 		mutex_unlock(&client->lock);
 		if (!handle)
 			return -EINVAL;
@@ -1171,7 +1171,7 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 
 		if (copy_from_user(&data, (void __user *)arg, sizeof(data)))
 			return -EFAULT;
-		handle = ion_uhandle_get(client, (int)data.handle);
+		handle = ion_uhandle_get(client, data.handle);
 		data.fd = ion_share_dma_buf_fd(client, handle);
 		if (copy_to_user((void __user *)arg, &data, sizeof(data)))
 			return -EFAULT;
@@ -1191,7 +1191,7 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 		if (IS_ERR(handle))
 			ret = PTR_ERR(handle);
 		else
-			data.handle = (ion_user_handle_t)handle->id;
+			data.handle = handle->id;
 
 		if (copy_to_user((void __user *)arg, &data,
 				 sizeof(struct ion_fd_data)))
drivers/staging/android/ion/ion.h +2 -2
index 244f085..d979890 100644
@@ -19,8 +19,7 @@
 
 #include <linux/types.h>
 
-struct ion_handle;
-typedef struct ion_handle *ion_user_handle_t;
+typedef int ion_user_handle_t;
 
 /**
  * enum ion_heap_types - list of all possible types of heaps
@@ -65,6 +64,7 @@ enum ion_heap_type {
 					   caches must be managed manually */
 
 #ifdef __KERNEL__
+struct ion_handle;
 struct ion_device;
 struct ion_heap;
 struct ion_mapper;
Signature · 1 line
1.8.3.2
#98 lore raw

PATCH 81/115 ion: add compat_ioctl

In reply to John Stultz

From: Rom Lemarchand <[email protected]>

Add a compat_ioctl to the ion driver

[jstultz: modified patch to apply to staging directory]
---
 drivers/staging/android/ion/Makefile     |   3 +
 drivers/staging/android/ion/compat_ion.c | 172 +++++++++++++++++++++++++++++++
 drivers/staging/android/ion/compat_ion.h |  30 ++++++
 drivers/staging/android/ion/ion.c        |   2 +
 4 files changed, 207 insertions(+)
 create mode 100644 drivers/staging/android/ion/compat_ion.c
 create mode 100644 drivers/staging/android/ion/compat_ion.h
drivers/staging/android/ion/Makefile +3 -0
index ce0f1ef..9c95665 100644
@@ -1,3 +1,6 @@
 obj-$(CONFIG_ION) +=	ion.o ion_heap.o ion_page_pool.o ion_system_heap.o \
 			ion_carveout_heap.o ion_chunk_heap.o ion_cma_heap.o
+ifdef CONFIG_COMPAT
+obj-$(CONFIG_ION) += compat_ion.o
+endif
 obj-$(CONFIG_ION_TEGRA) += tegra/
drivers/staging/android/ion/compat_ion.c new file +172 -0

172 lines not on this page · raw message

drivers/staging/android/ion/compat_ion.h new file +30 -0
new file mode 100644
index 0000000..c2ad589
@@ -0,0 +1,30 @@
+/*
+
+ * drivers/staging/android/ion/compat_ion.h
+ *
+ * Copyright (C) 2013 Google, Inc.
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#ifndef _LINUX_COMPAT_ION_H
+#define _LINUX_COMPAT_ION_H
+
+#if IS_ENABLED(CONFIG_COMPAT)
+
+long compat_ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg);
+
+#else
+
+#define compat_ion_ioctl  NULL
+
+#endif /* CONFIG_COMPAT */
+#endif /* _LINUX_COMPAT_ION_H */
drivers/staging/android/ion/ion.c +2 -0
index 1c40b74..d97117f 100644
@@ -38,6 +38,7 @@
 
 #include "ion.h"
 #include "ion_priv.h"
+#include "compat_ion.h"
 
 /**
  * struct ion_device - the metadata of the ion device node
@@ -1256,6 +1257,7 @@ static const struct file_operations ion_fops = {
 	.open           = ion_open,
 	.release        = ion_release,
 	.unlocked_ioctl = ion_ioctl,
+	.compat_ioctl   = compat_ion_ioctl,
 };
 
 static size_t ion_debug_heap_total(struct ion_client *client,
Signature · 1 line
1.8.3.2
#99 lore raw

PATCH 82/115 gpu: ion: delete ion_system_mapper.c

In reply to John Stultz

From: Colin Cross <[email protected]>

The mapper abstraction layer was removed before the initial ion commit, but a stray ion_system_mapper.c file was left in. Delete it.

---
 drivers/staging/android/ion/ion_system_mapper.c | 114 ------------------------
 1 file changed, 114 deletions(-)
 delete mode 100644 drivers/staging/android/ion/ion_system_mapper.c
drivers/staging/android/ion/ion_system_mapper.c deleted +0 -114
deleted file mode 100644
index 285430d..0000000
@@ -1,114 +0,0 @@
-/*
- * drivers/staging/android/ion/ion_system_mapper.c
- *
- * Copyright (C) 2011 Google, Inc.
- *
- * This software is licensed under the terms of the GNU General Public
- * License version 2, as published by the Free Software Foundation, and
- * may be copied, distributed, and modified under those terms.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- */
-
-#include <linux/err.h>
-#include <linux/memory.h>
-#include <linux/mm.h>
-#include <linux/slab.h>
-#include <linux/vmalloc.h>
-#include "ion.h"
-#include "ion_priv.h"
-/*
- * This mapper is valid for any heap that allocates memory that already has
- * a kernel mapping, this includes vmalloc'd memory, kmalloc'd memory,
- * pages obtained via io_remap, etc.
- */
-static void *ion_kernel_mapper_map(struct ion_mapper *mapper,
-				   struct ion_buffer *buffer,
-				   struct ion_mapping **mapping)
-{
-	if (!((1 << buffer->heap->type) & mapper->heap_mask)) {
-		pr_err("%s: attempting to map an unsupported heap\n", __func__);
-		return ERR_PTR(-EINVAL);
-	}
-	/* XXX REVISIT ME!!! */
-	*((unsigned long *)mapping) = (unsigned long)buffer->priv;
-	return buffer->priv;
-}
-
-static void ion_kernel_mapper_unmap(struct ion_mapper *mapper,
-				    struct ion_buffer *buffer,
-				    struct ion_mapping *mapping)
-{
-	if (!((1 << buffer->heap->type) & mapper->heap_mask))
-		pr_err("%s: attempting to unmap an unsupported heap\n",
-		       __func__);
-}
-
-static void *ion_kernel_mapper_map_kernel(struct ion_mapper *mapper,
-					struct ion_buffer *buffer,
-					struct ion_mapping *mapping)
-{
-	if (!((1 << buffer->heap->type) & mapper->heap_mask)) {
-		pr_err("%s: attempting to unmap an unsupported heap\n",
-		       __func__);
-		return ERR_PTR(-EINVAL);
-	}
-	return buffer->priv;
-}
-
-static int ion_kernel_mapper_map_user(struct ion_mapper *mapper,
-				      struct ion_buffer *buffer,
-				      struct vm_area_struct *vma,
-				      struct ion_mapping *mapping)
-{
-	int ret;
-
-	switch (buffer->heap->type) {
-	case ION_HEAP_KMALLOC:
-	{
-		unsigned long pfn = __phys_to_pfn(virt_to_phys(buffer->priv));
-		ret = remap_pfn_range(vma, vma->vm_start, pfn + vma->vm_pgoff,
-				      vma->vm_end - vma->vm_start,
-				      vma->vm_page_prot);
-		break;
-	}
-	case ION_HEAP_VMALLOC:
-		ret = remap_vmalloc_range(vma, buffer->priv, vma->vm_pgoff);
-		break;
-	default:
-		pr_err("%s: attempting to map unsupported heap to userspace\n",
-		       __func__);
-		return -EINVAL;
-	}
-
-	return ret;
-}
-
-static struct ion_mapper_ops ops = {
-	.map = ion_kernel_mapper_map,
-	.map_kernel = ion_kernel_mapper_map_kernel,
-	.map_user = ion_kernel_mapper_map_user,
-	.unmap = ion_kernel_mapper_unmap,
-};
-
-struct ion_mapper *ion_system_mapper_create(void)
-{
-	struct ion_mapper *mapper;
-	mapper = kzalloc(sizeof(struct ion_mapper), GFP_KERNEL);
-	if (!mapper)
-		return ERR_PTR(-ENOMEM);
-	mapper->type = ION_SYSTEM_MAPPER;
-	mapper->ops = &ops;
-	mapper->heap_mask = (1 << ION_HEAP_VMALLOC) | (1 << ION_HEAP_KMALLOC);
-	return mapper;
-}
-
-void ion_system_mapper_destroy(struct ion_mapper *mapper)
-{
-	kfree(mapper);
-}
-
Signature · 1 line
1.8.3.2
#100 lore raw

PATCH 83/115 ion: move userspace api into uapi/ion.h

In reply to John Stultz

From: Colin Cross <[email protected]>

Split the userspace api out of drivers/staging/android/ion/ion.h into drivers/staging/android/uapi/ion.h

---
 drivers/staging/android/ion/compat_ion.c |   2 +-
 drivers/staging/android/ion/ion.h        | 176 +--------------------------
 drivers/staging/android/uapi/ion.h       | 196 +++++++++++++++++++++++++++++++
 3 files changed, 198 insertions(+), 176 deletions(-)
 create mode 100644 drivers/staging/android/uapi/ion.h
drivers/staging/android/ion/compat_ion.c +1 -1
index 83842a6..d4a33c7 100644
@@ -21,7 +21,7 @@
 #include "ion.h"
 #include "compat_ion.h"
 
-/* See drivers/staging/android/ion/ion.h for the definition of these structs */
+/* See drivers/staging/android/uapi/ion.h for the definition of these structs */
 struct compat_ion_allocation_data {
 	compat_size_t len;
 	compat_size_t align;
drivers/staging/android/ion/ion.h +1 -175

187 lines not on this page · raw message

drivers/staging/android/uapi/ion.h new file +196 -0

196 lines not on this page · raw message

Signature · 1 line
1.8.3.2
#101 lore raw

PATCH 84/115 ion: Fix compat support to use proper compat ioctl numbers

In reply to John Stultz

The compat support added to ion didn't provide compat ioctl numbers (who's value depends on the compat structure size). So 32bit applications would get an error when trying to make ioctl calls.

This patch adds the needed COMPAT_ macros and uses them in the compat_ion_ioctl, translating them to their non-compat cmd when calling the normal ioctl call.

---
 drivers/staging/android/ion/compat_ion.c | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)
drivers/staging/android/ion/compat_ion.c +12 -7
index d4a33c7..079a5bb 100644
@@ -35,6 +35,12 @@ struct compat_ion_custom_data {
 	compat_ulong_t arg;
 };
 
+#define COMPAT_ION_IOC_ALLOC	_IOWR(ION_IOC_MAGIC, 0, \
+				      struct compat_ion_allocation_data)
+#define COMPAT_ION_IOC_FREE	_IOWR(ION_IOC_MAGIC, 1, struct ion_handle_data)
+#define COMPAT_ION_IOC_CUSTOM	_IOWR(ION_IOC_MAGIC, 6, \
+				      struct compat_ion_custom_data)
+
 static int compat_get_ion_allocation_data(
 			struct compat_ion_allocation_data __user *data32,
 			struct ion_allocation_data __user *data)
@@ -105,7 +111,7 @@ long compat_ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 		return -ENOTTY;
 
 	switch (cmd) {
-	case ION_IOC_ALLOC:
+	case COMPAT_ION_IOC_ALLOC:
 	{
 		struct compat_ion_allocation_data __user *data32;
 		struct ion_allocation_data __user *data;
@@ -119,13 +125,12 @@ long compat_ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 		err = compat_get_ion_allocation_data(data32, data);
 		if (err)
 			return err;
-
-		ret = filp->f_op->unlocked_ioctl(filp, cmd,
+		ret = filp->f_op->unlocked_ioctl(filp, ION_IOC_ALLOC,
 							(unsigned long)data);
 		err = compat_put_ion_allocation_data(data32, data);
 		return ret ? ret : err;
 	}
-	case ION_IOC_FREE:
+	case COMPAT_ION_IOC_FREE:
 	{
 		struct compat_ion_allocation_data __user *data32;
 		struct ion_allocation_data __user *data;
@@ -140,10 +145,10 @@ long compat_ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 		if (err)
 			return err;
 
-		return filp->f_op->unlocked_ioctl(filp, cmd,
+		return filp->f_op->unlocked_ioctl(filp, ION_IOC_FREE,
 							(unsigned long)data);
 	}
-	case ION_IOC_CUSTOM: {
+	case COMPAT_ION_IOC_CUSTOM: {
 		struct compat_ion_custom_data __user *data32;
 		struct ion_custom_data __user *data;
 		int err;
@@ -157,7 +162,7 @@ long compat_ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 		if (err)
 			return err;
 
-		return filp->f_op->unlocked_ioctl(filp, cmd,
+		return filp->f_op->unlocked_ioctl(filp, ION_IOC_CUSTOM,
 							(unsigned long)data);
 	}
 	case ION_IOC_SHARE:
Signature · 1 line
1.8.3.2
#102 lore raw

PATCH 85/115 ion: hold reference to handle after ion_uhandle_get

In reply to John Stultz

From: Colin Cross <[email protected]>

commit 1262ab1846cf76f7549c66ef709120dbfbe6d49f (ion: replace userspace handle cookies with idr) broke the locking in ion. The ION_IOC_FREE and ION_IOC_MAP ioctls were relying on ion_handle_validate to detect the case where a call raced with another ION_IOC_FREE which may have freed the struct ion_handle.

Rename ion_uhandle_get to ion_handle_get_by_id, and have it take the client lock and return with an extra reference to the handle. Make each caller put its reference once it is done with the handle.

Also modify users of ion_handle_validate to continue to hold the client lock after calling ion_handle_validate until they are done with the handle, and warn if ion_handle_validate is called without the client lock held.

---
 drivers/staging/android/ion/ion.c | 56 ++++++++++++++++++++++++++++-----------
 1 file changed, 41 insertions(+), 15 deletions(-)
drivers/staging/android/ion/ion.c +41 -15
index d97117f..cf9fc78 100644
@@ -383,7 +383,14 @@ static void ion_handle_get(struct ion_handle *handle)
 
 static int ion_handle_put(struct ion_handle *handle)
 {
-	return kref_put(&handle->ref, ion_handle_destroy);
+	struct ion_client *client = handle->client;
+	int ret;
+
+	mutex_lock(&client->lock);
+	ret = kref_put(&handle->ref, ion_handle_destroy);
+	mutex_unlock(&client->lock);
+
+	return ret;
 }
 
 static struct ion_handle *ion_handle_lookup(struct ion_client *client,
@@ -403,14 +410,24 @@ static struct ion_handle *ion_handle_lookup(struct ion_client *client,
 	return ERR_PTR(-EINVAL);
 }
 
-static struct ion_handle *ion_uhandle_get(struct ion_client *client, int id)
+static struct ion_handle *ion_handle_get_by_id(struct ion_client *client,
+						int id)
 {
-	return idr_find(&client->idr, id);
+	struct ion_handle *handle;
+
+	mutex_lock(&client->lock);
+	handle = idr_find(&client->idr, id);
+	if (handle)
+		ion_handle_get(handle);
+	mutex_unlock(&client->lock);
+
+	return handle ? handle : ERR_PTR(-EINVAL);
 }
 
 static bool ion_handle_validate(struct ion_client *client, struct ion_handle *handle)
 {
-	return (ion_uhandle_get(client, handle->id) == handle);
+	WARN_ON(!mutex_is_locked(&client->lock));
+	return (idr_find(&client->idr, handle->id) == handle);
 }
 
 static int ion_handle_add(struct ion_client *client, struct ion_handle *handle)
@@ -503,11 +520,11 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 
 	mutex_lock(&client->lock);
 	ret = ion_handle_add(client, handle);
+	mutex_unlock(&client->lock);
 	if (ret) {
 		ion_handle_put(handle);
 		handle = ERR_PTR(ret);
 	}
-	mutex_unlock(&client->lock);
 
 	return handle;
 }
@@ -527,8 +544,8 @@ void ion_free(struct ion_client *client, struct ion_handle *handle)
 		mutex_unlock(&client->lock);
 		return;
 	}
-	ion_handle_put(handle);
 	mutex_unlock(&client->lock);
+	ion_handle_put(handle);
 }
 EXPORT_SYMBOL(ion_free);
 
@@ -1021,14 +1038,15 @@ struct dma_buf *ion_share_dma_buf(struct ion_client *client,
 
 	mutex_lock(&client->lock);
 	valid_handle = ion_handle_validate(client, handle);
-	mutex_unlock(&client->lock);
 	if (!valid_handle) {
 		WARN(1, "%s: invalid handle passed to share.\n", __func__);
+		mutex_unlock(&client->lock);
 		return ERR_PTR(-EINVAL);
 	}
-
 	buffer = handle->buffer;
 	ion_buffer_get(buffer);
+	mutex_unlock(&client->lock);
+
 	dmabuf = dma_buf_export(buffer, &dma_buf_ops, buffer->size, O_RDWR);
 	if (IS_ERR(dmabuf)) {
 		ion_buffer_put(buffer);
@@ -1081,18 +1099,24 @@ struct ion_handle *ion_import_dma_buf(struct ion_client *client, int fd)
 	handle = ion_handle_lookup(client, buffer);
 	if (!IS_ERR(handle)) {
 		ion_handle_get(handle);
+		mutex_unlock(&client->lock);
 		goto end;
 	}
+	mutex_unlock(&client->lock);
+
 	handle = ion_handle_create(client, buffer);
 	if (IS_ERR(handle))
 		goto end;
+
+	mutex_lock(&client->lock);
 	ret = ion_handle_add(client, handle);
+	mutex_unlock(&client->lock);
 	if (ret) {
 		ion_handle_put(handle);
 		handle = ERR_PTR(ret);
 	}
+
 end:
-	mutex_unlock(&client->lock);
 	dma_buf_put(dmabuf);
 	return handle;
 }
@@ -1156,12 +1180,11 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 		if (copy_from_user(&data, (void __user *)arg,
 				   sizeof(struct ion_handle_data)))
 			return -EFAULT;
-		mutex_lock(&client->lock);
-		handle = ion_uhandle_get(client, data.handle);
-		mutex_unlock(&client->lock);
-		if (!handle)
-			return -EINVAL;
+		handle = ion_handle_get_by_id(client, data.handle);
+		if (IS_ERR(handle))
+			return PTR_ERR(handle);
 		ion_free(client, handle);
+		ion_handle_put(handle);
 		break;
 	}
 	case ION_IOC_SHARE:
@@ -1172,8 +1195,11 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 
 		if (copy_from_user(&data, (void __user *)arg, sizeof(data)))
 			return -EFAULT;
-		handle = ion_uhandle_get(client, data.handle);
+		handle = ion_handle_get_by_id(client, data.handle);
+		if (IS_ERR(handle))
+			return PTR_ERR(handle);
 		data.fd = ion_share_dma_buf_fd(client, handle);
+		ion_handle_put(handle);
 		if (copy_to_user((void __user *)arg, &data, sizeof(data)))
 			return -EFAULT;
 		if (data.fd < 0)
Signature · 1 line
1.8.3.2
#103 lore raw

PATCH 86/115 ion: fix crash when alloc len is -1

In reply to John Stultz

From: Colin Cross <[email protected]>

If userspace passes a length between -4095 and -1 to allocate it will pass the len != 0 check, but when len is page aligned it will be 0. Check len after page aligning.

Drop the warning as well, userspace shouldn't be able to trigger a warning in the kernel.

---
 drivers/staging/android/ion/ion.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/staging/android/ion/ion.c +3 -3
index cf9fc78..2e7be70 100644
@@ -485,11 +485,11 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 	 * request of the caller allocate from it.  Repeat until allocate has
 	 * succeeded or all heaps have been tried
 	 */
-	if (WARN_ON(!len))
-		return ERR_PTR(-EINVAL);
-
 	len = PAGE_ALIGN(len);
 
+	if (!len)
+		return ERR_PTR(-EINVAL);
+
 	down_read(&dev->lock);
 	plist_for_each_entry(heap, &dev->heaps, node) {
 		/* if the caller didn't specify this heap id */
Signature · 1 line
1.8.3.2
#104 lore raw

PATCH 87/115 ion: fix dma APIs

In reply to John Stultz

From: Colin Cross <[email protected]>

__dma_page_cpu_to_dev is a private ARM api that is not available on 3.10 and was never available on other architectures. We can get the same behavior by calling dma_sync_sg_for_device with a scatterlist containing a single page. It's still not quite a kosher use of the dma apis, we still conflate physical addresses with bus addresses, but it should at least compile on all platforms, and work on any platform that doesn't have a physical to bus address translation.

---
 drivers/staging/android/ion/ion.c             | 20 +++++++++++++++++++-
 drivers/staging/android/ion/ion_chunk_heap.c  | 15 +++++++--------
 drivers/staging/android/ion/ion_page_pool.c   |  9 ++-------
 drivers/staging/android/ion/ion_priv.h        | 12 ++++++++++++
 drivers/staging/android/ion/ion_system_heap.c |  5 ++---
 5 files changed, 42 insertions(+), 19 deletions(-)
drivers/staging/android/ion/ion.c +19 -1
index 2e7be70..4bdbdec 100644
@@ -840,6 +840,22 @@ static void ion_unmap_dma_buf(struct dma_buf_attachment *attachment,
 {
 }
 
+void ion_pages_sync_for_device(struct device *dev, struct page *page,
+		size_t size, enum dma_data_direction dir)
+{
+	struct scatterlist sg;
+
+	sg_init_table(&sg, 1);
+	sg_set_page(&sg, page, size, 0);
+	/*
+	 * This is not correct - sg_dma_address needs a dma_addr_t that is valid
+	 * for the the targeted device, but this works on the currently targeted
+	 * hardware.
+	 */
+	sg_dma_address(&sg) = page_to_phys(page);
+	dma_sync_sg_for_device(dev, &sg, 1, dir);
+}
+
 struct ion_vma_list {
 	struct list_head list;
 	struct vm_area_struct *vma;
@@ -864,7 +880,9 @@ static void ion_buffer_sync_for_device(struct ion_buffer *buffer,
 		struct page *page = buffer->pages[i];
 
 		if (ion_buffer_page_is_dirty(page))
-			__dma_page_cpu_to_dev(page, 0, PAGE_SIZE, dir);
+			ion_pages_sync_for_device(dev, ion_buffer_page(page),
+							PAGE_SIZE, dir);
+
 		ion_buffer_page_clean(buffer->pages + i);
 	}
 	list_for_each_entry(vma_list, &buffer->vmas, list) {
drivers/staging/android/ion/ion_chunk_heap.c +7 -8
index 85fefe7..2fddc04 100644
@@ -106,11 +106,11 @@ static void ion_chunk_heap_free(struct ion_buffer *buffer)
 
 	ion_heap_buffer_zero(buffer);
 
+	if (ion_buffer_cached(buffer))
+		dma_sync_sg_for_device(NULL, table->sgl, table->nents,
+								DMA_BIDIRECTIONAL);
+
 	for_each_sg(table->sgl, sg, table->nents, i) {
-		if (ion_buffer_cached(buffer))
-			arm_dma_ops.sync_single_for_device(NULL,
-				pfn_to_dma(NULL, page_to_pfn(sg_page(sg))),
-				sg_dma_len(sg), DMA_BIDIRECTIONAL);
 		gen_pool_free(chunk_heap->pool, page_to_phys(sg_page(sg)),
 			      sg_dma_len(sg));
 	}
@@ -148,7 +148,6 @@ struct ion_heap *ion_chunk_heap_create(struct ion_platform_heap *heap_data)
 	pgprot_t pgprot = pgprot_writecombine(PAGE_KERNEL);
 	int i, ret;
 
-
 	chunk_heap = kzalloc(sizeof(struct ion_chunk_heap), GFP_KERNEL);
 	if (!chunk_heap)
 		return ERR_PTR(-ENOMEM);
@@ -181,9 +180,9 @@ struct ion_heap *ion_chunk_heap_create(struct ion_platform_heap *heap_data)
 	}
 	free_vm_area(vm_struct);
 
-	arm_dma_ops.sync_single_for_device(NULL,
-		pfn_to_dma(NULL, page_to_pfn(phys_to_page(heap_data->base))),
-		heap_data->size, DMA_BIDIRECTIONAL);
+	ion_pages_sync_for_device(NULL, pfn_to_page(PFN_DOWN(heap_data->base)),
+			heap_data->size, DMA_BIDIRECTIONAL);
+
 	gen_pool_add(chunk_heap->pool, chunk_heap->base, heap_data->size, -1);
 	chunk_heap->heap.ops = &chunk_heap_ops;
 	chunk_heap->heap.type = ION_HEAP_TYPE_CHUNK;
drivers/staging/android/ion/ion_page_pool.c +2 -7
index 855f30c..979d2e4 100644
@@ -34,13 +34,8 @@ static void *ion_page_pool_alloc_pages(struct ion_page_pool *pool)
 
 	if (!page)
 		return NULL;
-	/* this is only being used to flush the page for dma,
-	   this api is not really suitable for calling from a driver
-	   but no better way to flush a page for dma exist at this time */
-	arm_dma_ops.sync_single_for_device(NULL,
-					   pfn_to_dma(NULL, page_to_pfn(page)),
-					   PAGE_SIZE << pool->order,
-					   DMA_BIDIRECTIONAL);
+	ion_pages_sync_for_device(NULL, page, PAGE_SIZE << pool->order,
+						DMA_BIDIRECTIONAL);
 	return page;
 }
 
drivers/staging/android/ion/ion_priv.h +12 -0
index f263563..d1fa771 100644
@@ -17,6 +17,7 @@
 #ifndef _ION_PRIV_H
 #define _ION_PRIV_H
 
+#include <linux/dma-direction.h>
 #include <linux/kref.h>
 #include <linux/mm_types.h>
 #include <linux/mutex.h>
@@ -357,4 +358,15 @@ void ion_page_pool_free(struct ion_page_pool *, struct page *);
 int ion_page_pool_shrink(struct ion_page_pool *pool, gfp_t gfp_mask,
 			  int nr_to_scan);
 
+/**
+ * ion_pages_sync_for_device - cache flush pages for use with the specified
+ *                             device
+ * @dev:		the device the pages will be used with
+ * @page:		the first page to be flushed
+ * @size:		size in bytes of region to be flushed
+ * @dir:		direction of dma transfer
+ */
+void ion_pages_sync_for_device(struct device *dev, struct page *page,
+		size_t size, enum dma_data_direction dir);
+
 #endif /* _ION_PRIV_H */
drivers/staging/android/ion/ion_system_heap.c +2 -3
index b9b1036..792cade 100644
@@ -77,9 +77,8 @@ static struct page *alloc_buffer_page(struct ion_system_heap *heap,
 		page = ion_heap_alloc_pages(buffer, gfp_flags, order);
 		if (!page)
 			return 0;
-		arm_dma_ops.sync_single_for_device(NULL,
-			pfn_to_dma(NULL, page_to_pfn(page)),
-			PAGE_SIZE << order, DMA_BIDIRECTIONAL);
+		ion_pages_sync_for_device(NULL, page, PAGE_SIZE << order,
+						DMA_BIDIRECTIONAL);
 	}
 	if (!page)
 		return 0;
Signature · 1 line
1.8.3.2
#105 lore raw

PATCH 88/115 ion: convert sg_dma_len(sg) to sg->length

In reply to John Stultz

From: Colin Cross <[email protected]>

ion is always dealing with the allocation and not the mapping, so it should always be using sg->length and not sg->dma_length.

---
 drivers/staging/android/ion/ion.c             |  2 +-
 drivers/staging/android/ion/ion_chunk_heap.c  |  4 ++--
 drivers/staging/android/ion/ion_heap.c        | 12 ++++++------
 drivers/staging/android/ion/ion_system_heap.c |  2 +-
 4 files changed, 10 insertions(+), 10 deletions(-)
drivers/staging/android/ion/ion.c +1 -1
index 4bdbdec..8d568ca 100644
@@ -224,7 +224,7 @@ static struct ion_buffer *ion_buffer_create(struct ion_heap *heap,
 		for_each_sg(table->sgl, sg, table->nents, i) {
 			struct page *page = sg_page(sg);
 
-			for (j = 0; j < sg_dma_len(sg) / PAGE_SIZE; j++)
+			for (j = 0; j < sg->length / PAGE_SIZE; j++)
 				buffer->pages[k++] = page++;
 		}
 
drivers/staging/android/ion/ion_chunk_heap.c +2 -2
index 2fddc04..62f7439 100644
@@ -84,7 +84,7 @@ err:
 	sg = table->sgl;
 	for (i -= 1; i >= 0; i--) {
 		gen_pool_free(chunk_heap->pool, page_to_phys(sg_page(sg)),
-			      sg_dma_len(sg));
+			      sg->length);
 		sg = sg_next(sg);
 	}
 	sg_free_table(table);
@@ -112,7 +112,7 @@ static void ion_chunk_heap_free(struct ion_buffer *buffer)
 
 	for_each_sg(table->sgl, sg, table->nents, i) {
 		gen_pool_free(chunk_heap->pool, page_to_phys(sg_page(sg)),
-			      sg_dma_len(sg));
+			      sg->length);
 	}
 	chunk_heap->allocated -= allocated_size;
 	sg_free_table(table);
drivers/staging/android/ion/ion_heap.c +6 -6
index a584ec0..a5a7c572 100644
@@ -46,7 +46,7 @@ void *ion_heap_map_kernel(struct ion_heap *heap,
 		pgprot = pgprot_writecombine(PAGE_KERNEL);
 
 	for_each_sg(table->sgl, sg, table->nents, i) {
-		int npages_this_entry = PAGE_ALIGN(sg_dma_len(sg)) / PAGE_SIZE;
+		int npages_this_entry = PAGE_ALIGN(sg->length) / PAGE_SIZE;
 		struct page *page = sg_page(sg);
 		BUG_ON(i >= npages);
 		for (j = 0; j < npages_this_entry; j++) {
@@ -80,14 +80,14 @@ int ion_heap_map_user(struct ion_heap *heap, struct ion_buffer *buffer,
 	for_each_sg(table->sgl, sg, table->nents, i) {
 		struct page *page = sg_page(sg);
 		unsigned long remainder = vma->vm_end - addr;
-		unsigned long len = sg_dma_len(sg);
+		unsigned long len = sg->length;
 
-		if (offset >= sg_dma_len(sg)) {
-			offset -= sg_dma_len(sg);
+		if (offset >= sg->length) {
+			offset -= sg->length;
 			continue;
 		} else if (offset) {
 			page += offset / PAGE_SIZE;
-			len = sg_dma_len(sg) - offset;
+			len = sg->length - offset;
 			offset = 0;
 		}
 		len = min(len, remainder);
@@ -119,7 +119,7 @@ int ion_heap_buffer_zero(struct ion_buffer *buffer)
 
 	for_each_sg(table->sgl, sg, table->nents, i) {
 		struct page *page = sg_page(sg);
-		unsigned long len = sg_dma_len(sg);
+		unsigned long len = sg->length;
 
 		for (j = 0; j < len / PAGE_SIZE; j++) {
 			struct page *sub_page = page + j;
drivers/staging/android/ion/ion_system_heap.c +1 -1
index 792cade..967eedc 100644
@@ -209,7 +209,7 @@ void ion_system_heap_free(struct ion_buffer *buffer)
 
 	for_each_sg(table->sgl, sg, table->nents, i)
 		free_buffer_page(sys_heap, buffer, sg_page(sg),
-				get_order(sg_dma_len(sg)));
+				get_order(sg->length));
 	sg_free_table(table);
 	kfree(table);
 }
Signature · 1 line
1.8.3.2
#106 lore raw

PATCH 89/115 ion: check invalid values in ion_system_heap

In reply to John Stultz

From: Colin Cross <[email protected]>

ion_system_heap can only satisfy page alignment, and ion_system_contig_heap can only satisify alignment to the allocation size. Neither can support faulting user mappings because they use slab pages.

---
 drivers/staging/android/ion/ion_system_heap.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
drivers/staging/android/ion/ion_system_heap.c +14 -0
index 967eedc..62a07ec 100644
@@ -150,6 +150,12 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 	long size_remaining = PAGE_ALIGN(size);
 	unsigned int max_order = orders[0];
 
+	if (align > PAGE_SIZE)
+		return -EINVAL;
+
+	if (ion_buffer_fault_user_mappings(buffer))
+		return -EINVAL;
+
 	INIT_LIST_HEAD(&pages);
 	while (size_remaining > 0) {
 		info = alloc_largest_available(sys_heap, buffer, size_remaining, max_order);
@@ -362,6 +368,14 @@ static int ion_system_contig_heap_allocate(struct ion_heap *heap,
 					   unsigned long align,
 					   unsigned long flags)
 {
+	int order = get_order(len);
+
+	if (align > (PAGE_SIZE << order))
+		return -EINVAL;
+
+	if (ion_buffer_fault_user_mappings(buffer))
+		return -EINVAL;
+
 	buffer->priv_virt = kzalloc(len, GFP_KERNEL);
 	if (!buffer->priv_virt)
 		return -ENOMEM;
Signature · 1 line
1.8.3.2
#107 lore raw

PATCH 90/115 ion: add test device for unit tests to interact with dma_bufs

In reply to John Stultz

From: Colin Cross <[email protected]>

Add a /dev/ion-test device that will be created if CONFIG_ION_TEST is set. The device accepts a dma_buf fd and allows reading and writing to the backing memory using DMA-like apis or kernel mapping apis. Can be used to test the dma_buf mapping ops, including the ion implementations, from userspace.

---
 drivers/staging/android/ion/Kconfig     |   7 +
 drivers/staging/android/ion/Makefile    |   1 +
 drivers/staging/android/ion/ion_test.c  | 281 ++++++++++++++++++++++++++++++++
 drivers/staging/android/uapi/ion_test.h |  71 ++++++++
 4 files changed, 360 insertions(+)
 create mode 100644 drivers/staging/android/ion/ion_test.c
 create mode 100644 drivers/staging/android/uapi/ion_test.h
drivers/staging/android/ion/Kconfig +7 -0
index c62f2cb..6a5d8cf 100644
@@ -6,6 +6,13 @@ menuconfig ION
 	help
 	  Chose this option to enable the ION Memory Manager.
 
+config ION_TEST
+	tristate "Ion Test Device"
+	depends on ION
+	help
+	  Choose this option to create a device that can be used to test the
+	  kernel and device side ION functions.
+
 config ION_TEGRA
 	tristate "Ion for Tegra"
 	depends on ARCH_TEGRA && ION
drivers/staging/android/ion/Makefile +1 -0
index 9c95665..75039b9 100644
@@ -1,5 +1,6 @@
 obj-$(CONFIG_ION) +=	ion.o ion_heap.o ion_page_pool.o ion_system_heap.o \
 			ion_carveout_heap.o ion_chunk_heap.o ion_cma_heap.o
+obj-$(CONFIG_ION_TEST) += ion_test.o
 ifdef CONFIG_COMPAT
 obj-$(CONFIG_ION) += compat_ion.o
 endif
drivers/staging/android/ion/ion_test.c new file +281 -0

281 lines not on this page · raw message

drivers/staging/android/uapi/ion_test.h new file +71 -0

71 lines not on this page · raw message

Signature · 1 line
1.8.3.2
#108 lore raw

PATCH 91/115 ion: update idr to avoid deprecated apis

In reply to John Stultz

From: Colin Cross <[email protected]>

Use idr_alloc instead if idr_pre_get/idr_get_new_above, and remove idr_remove_all.

---
 drivers/staging/android/ion/ion.c | 17 +++++------------
 1 file changed, 5 insertions(+), 12 deletions(-)
drivers/staging/android/ion/ion.c +5 -12
index 8d568ca..199ab9a 100644
@@ -432,22 +432,16 @@ static bool ion_handle_validate(struct ion_client *client, struct ion_handle *ha
 
 static int ion_handle_add(struct ion_client *client, struct ion_handle *handle)
 {
-	int rc;
+	int id;
 	struct rb_node **p = &client->handles.rb_node;
 	struct rb_node *parent = NULL;
 	struct ion_handle *entry;
 
-	do {
-		int id;
-		rc = idr_pre_get(&client->idr, GFP_KERNEL);
-		if (!rc)
-			return -ENOMEM;
-		rc = idr_get_new_above(&client->idr, handle, 1, &id);
-		handle->id = id;
-	} while (rc == -EAGAIN);
+	id = idr_alloc(&client->idr, handle, 1, 0, GFP_KERNEL);
+	if (id < 0)
+		return id;
 
-	if (rc < 0)
-		return rc;
+	handle->id = id;
 
 	while (*p) {
 		parent = *p;
@@ -786,7 +780,6 @@ void ion_client_destroy(struct ion_client *client)
 		ion_handle_destroy(&handle->ref);
 	}
 
-	idr_remove_all(&client->idr);
 	idr_destroy(&client->idr);
 
 	down_write(&dev->lock);
Signature · 1 line
1.8.3.2
#109 lore raw

PATCH 92/115 ion: don't use __arm_ioremap to map pages

In reply to John Stultz

From: Colin Cross <[email protected]>

ion_heap_map_kernel already implements mapping a scatterlist of pages into the kernel, and all heaps are required to have struct pages associated with them, so delete the functions that use __arm_ioremap and use ion_heap_map_kernel instead.

---
 drivers/staging/android/ion/ion_carveout_heap.c | 42 ++-----------------------
 drivers/staging/android/ion/ion_chunk_heap.c    |  2 --
 2 files changed, 3 insertions(+), 41 deletions(-)
drivers/staging/android/ion/ion_carveout_heap.c +3 -39
index 36df34c..051363a 100644
@@ -25,8 +25,6 @@
 #include "ion.h"
 #include "ion_priv.h"
 
-#include <asm/mach/map.h>
-
 struct ion_carveout_heap {
 	struct ion_heap heap;
 	struct gen_pool *pool;
@@ -109,49 +107,15 @@ void ion_carveout_heap_unmap_dma(struct ion_heap *heap,
 	sg_free_table(buffer->sg_table);
 }
 
-void *ion_carveout_heap_map_kernel(struct ion_heap *heap,
-				   struct ion_buffer *buffer)
-{
-	void *ret;
-	int mtype = MT_MEMORY_NONCACHED;
-
-	if (buffer->flags & ION_FLAG_CACHED)
-		mtype = MT_MEMORY;
-
-	ret = __arm_ioremap(buffer->priv_phys, buffer->size,
-			      mtype);
-	if (ret == NULL)
-		return ERR_PTR(-ENOMEM);
-
-	return ret;
-}
-
-void ion_carveout_heap_unmap_kernel(struct ion_heap *heap,
-				    struct ion_buffer *buffer)
-{
-	__arm_iounmap(buffer->vaddr);
-	buffer->vaddr = NULL;
-	return;
-}
-
-int ion_carveout_heap_map_user(struct ion_heap *heap, struct ion_buffer *buffer,
-			       struct vm_area_struct *vma)
-{
-	return remap_pfn_range(vma, vma->vm_start,
-			       __phys_to_pfn(buffer->priv_phys) + vma->vm_pgoff,
-			       vma->vm_end - vma->vm_start,
-			       pgprot_noncached(vma->vm_page_prot));
-}
-
 static struct ion_heap_ops carveout_heap_ops = {
 	.allocate = ion_carveout_heap_allocate,
 	.free = ion_carveout_heap_free,
 	.phys = ion_carveout_heap_phys,
 	.map_dma = ion_carveout_heap_map_dma,
 	.unmap_dma = ion_carveout_heap_unmap_dma,
-	.map_user = ion_carveout_heap_map_user,
-	.map_kernel = ion_carveout_heap_map_kernel,
-	.unmap_kernel = ion_carveout_heap_unmap_kernel,
+	.map_user = ion_heap_map_user,
+	.map_kernel = ion_heap_map_kernel,
+	.unmap_kernel = ion_heap_unmap_kernel,
 };
 
 struct ion_heap *ion_carveout_heap_create(struct ion_platform_heap *heap_data)
drivers/staging/android/ion/ion_chunk_heap.c +0 -2
index 62f7439..d3363d6 100644
@@ -25,8 +25,6 @@
 #include "ion.h"
 #include "ion_priv.h"
 
-#include <asm/mach/map.h>
-
 struct ion_chunk_heap {
 	struct ion_heap heap;
 	struct gen_pool *pool;
Signature · 1 line
1.8.3.2
#110 lore raw

PATCH 93/115 ion: don't use phys_to_page or __phys_to_pfn

In reply to John Stultz

From: Colin Cross <[email protected]>

phys_to_page and __phys_to_pfn don't exist on all platforms. Use a combination of pfn_to_page, PFN_DOWN, page_to_pfn, and virt_to_page to get the same results.

---
 drivers/staging/android/ion/ion_carveout_heap.c | 4 ++--
 drivers/staging/android/ion/ion_chunk_heap.c    | 5 +++--
 drivers/staging/android/ion/ion_system_heap.c   | 2 +-
 3 files changed, 6 insertions(+), 5 deletions(-)
drivers/staging/android/ion/ion_carveout_heap.c +2 -2
index 051363a..1d53e91 100644
@@ -96,8 +96,8 @@ struct sg_table *ion_carveout_heap_map_dma(struct ion_heap *heap,
 		kfree(table);
 		return ERR_PTR(ret);
 	}
-	sg_set_page(table->sgl, phys_to_page(buffer->priv_phys), buffer->size,
-		    0);
+	sg_set_page(table->sgl, pfn_to_page(PFN_DOWN(buffer->priv_phys)),
+			buffer->size, 0);
 	return table;
 }
 
drivers/staging/android/ion/ion_chunk_heap.c +3 -2
index d3363d6..fb1a7b9 100644
@@ -71,7 +71,8 @@ static int ion_chunk_heap_allocate(struct ion_heap *heap,
 						     chunk_heap->chunk_size);
 		if (!paddr)
 			goto err;
-		sg_set_page(sg, phys_to_page(paddr), chunk_heap->chunk_size, 0);
+		sg_set_page(sg, pfn_to_page(PFN_DOWN(paddr)),
+				chunk_heap->chunk_size, 0);
 		sg = sg_next(sg);
 	}
 
@@ -167,7 +168,7 @@ struct ion_heap *ion_chunk_heap_create(struct ion_platform_heap *heap_data)
 		goto error;
 	}
 	for (i = 0; i < chunk_heap->size; i += PAGE_SIZE) {
-		struct page *page = phys_to_page(chunk_heap->base + i);
+		struct page *page = pfn_to_page(PFN_DOWN(chunk_heap->base + i));
 		struct page **pages = &page;
 
 		ret = map_vm_area(vm_struct, pgprot, &pages);
drivers/staging/android/ion/ion_system_heap.c +1 -1
index 62a07ec..3b2f420 100644
@@ -426,7 +426,7 @@ int ion_system_contig_heap_map_user(struct ion_heap *heap,
 				    struct ion_buffer *buffer,
 				    struct vm_area_struct *vma)
 {
-	unsigned long pfn = __phys_to_pfn(virt_to_phys(buffer->priv_virt));
+	unsigned long pfn = page_to_pfn(virt_to_page(buffer->priv_virt));
 	return remap_pfn_range(vma, vma->vm_start, pfn + vma->vm_pgoff,
 			       vma->vm_end - vma->vm_start,
 			       vma->vm_page_prot);
Signature · 1 line
1.8.3.2
#111 lore raw

PATCH 94/115 ion: fix printk warnings

From: Colin Cross <[email protected]>

Use %z for size_t and %pa for dma_addr_t to avoid warnings in printks.

---
 drivers/staging/android/ion/ion.c            | 23 ++++++++++++-----------
 drivers/staging/android/ion/ion_chunk_heap.c |  2 +-
 drivers/staging/android/ion/ion_cma_heap.c   |  4 ++--
 drivers/staging/android/ion/ion_heap.c       |  2 +-
 4 files changed, 16 insertions(+), 15 deletions(-)
drivers/staging/android/ion/ion.c +12 -11
index 199ab9a..57698e6 100644
@@ -471,7 +471,7 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 	struct ion_heap *heap;
 	int ret;
 
-	pr_debug("%s: len %d align %d heap_id_mask %u flags %x\n", __func__,
+	pr_debug("%s: len %zu align %zu heap_id_mask %u flags %x\n", __func__,
 		 len, align, heap_id_mask, flags);
 	/*
 	 * traverse the list of heaps available in this system in priority
@@ -688,7 +688,7 @@ static int ion_debug_client_show(struct seq_file *s, void *unused)
 	for (i = 0; i < ION_NUM_HEAP_IDS; i++) {
 		if (!names[i])
 			continue;
-		seq_printf(s, "%16.16s: %16u\n", names[i], sizes[i]);
+		seq_printf(s, "%16.16s: %16zu\n", names[i], sizes[i]);
 	}
 	return 0;
 }
@@ -1336,10 +1336,10 @@ static int ion_debug_heap_show(struct seq_file *s, void *unused)
 			char task_comm[TASK_COMM_LEN];
 
 			get_task_comm(task_comm, client->task);
-			seq_printf(s, "%16.s %16u %16u\n", task_comm,
+			seq_printf(s, "%16.s %16u %16zu\n", task_comm,
 				   client->pid, size);
 		} else {
-			seq_printf(s, "%16.s %16u %16u\n", client->name,
+			seq_printf(s, "%16.s %16u %16zu\n", client->name,
 				   client->pid, size);
 		}
 	}
@@ -1354,19 +1354,20 @@ static int ion_debug_heap_show(struct seq_file *s, void *unused)
 			continue;
 		total_size += buffer->size;
 		if (!buffer->handle_count) {
-			seq_printf(s, "%16.s %16u %16u %d %d\n", buffer->task_comm,
-				   buffer->pid, buffer->size, buffer->kmap_cnt,
+			seq_printf(s, "%16.s %16u %16zu %d %d\n",
+				   buffer->task_comm, buffer->pid,
+				   buffer->size, buffer->kmap_cnt,
 				   atomic_read(&buffer->ref.refcount));
 			total_orphaned_size += buffer->size;
 		}
 	}
 	mutex_unlock(&dev->buffer_lock);
 	seq_printf(s, "----------------------------------------------------\n");
-	seq_printf(s, "%16.s %16u\n", "total orphaned",
+	seq_printf(s, "%16.s %16zu\n", "total orphaned",
 		   total_orphaned_size);
-	seq_printf(s, "%16.s %16u\n", "total ", total_size);
+	seq_printf(s, "%16.s %16zu\n", "total ", total_size);
 	if (heap->flags & ION_HEAP_FLAG_DEFER_FREE)
-		seq_printf(s, "%16.s %16u\n", "deferred free",
+		seq_printf(s, "%16.s %16zu\n", "deferred free",
 				heap->free_list_size);
 	seq_printf(s, "----------------------------------------------------\n");
 
@@ -1522,11 +1523,11 @@ void __init ion_reserve(struct ion_platform_data *data)
 			int ret = memblock_reserve(data->heaps[i].base,
 					       data->heaps[i].size);
 			if (ret)
-				pr_err("memblock reserve of %x@%lx failed\n",
+				pr_err("memblock reserve of %zx@%lx failed\n",
 				       data->heaps[i].size,
 				       data->heaps[i].base);
 		}
-		pr_info("%s: %s reserved base %lx size %d\n", __func__,
+		pr_info("%s: %s reserved base %lx size %zu\n", __func__,
 			data->heaps[i].name,
 			data->heaps[i].base,
 			data->heaps[i].size);
drivers/staging/android/ion/ion_chunk_heap.c +1 -1
index fb1a7b9..4eacf52 100644
@@ -186,7 +186,7 @@ struct ion_heap *ion_chunk_heap_create(struct ion_platform_heap *heap_data)
 	chunk_heap->heap.ops = &chunk_heap_ops;
 	chunk_heap->heap.type = ION_HEAP_TYPE_CHUNK;
 	chunk_heap->heap.flags = ION_HEAP_FLAG_DEFER_FREE;
-	pr_info("%s: base %lu size %u align %ld\n", __func__, chunk_heap->base,
+	pr_info("%s: base %lu size %zu align %ld\n", __func__, chunk_heap->base,
 		heap_data->size, heap_data->align);
 
 	return &chunk_heap->heap;
drivers/staging/android/ion/ion_cma_heap.c +2 -2
index 86b6cf5..1e308a0 100644
@@ -128,8 +128,8 @@ static int ion_cma_phys(struct ion_heap *heap, struct ion_buffer *buffer,
 	struct device *dev = cma_heap->dev;
 	struct ion_cma_buffer_info *info = buffer->priv_virt;
 
-	dev_dbg(dev, "Return buffer %p physical address 0x%x\n", buffer,
-		info->handle);
+	dev_dbg(dev, "Return buffer %p physical address 0x%pa\n", buffer,
+		&info->handle);
 
 	*addr = info->handle;
 	*len = buffer->size;
drivers/staging/android/ion/ion_heap.c +1 -1
index a5a7c572..deaab7c 100644
@@ -281,7 +281,7 @@ struct ion_heap *ion_heap_create(struct ion_platform_heap *heap_data)
 	}
 
 	if (IS_ERR_OR_NULL(heap)) {
-		pr_err("%s: error creating heap %s type %d base %lu size %u\n",
+		pr_err("%s: error creating heap %s type %d base %lu size %zu\n",
 		       __func__, heap_data->name, heap_data->type,
 		       heap_data->base, heap_data->size);
 		return ERR_PTR(-EINVAL);
Signature · 1 line
1.8.3.2
#112 lore raw

Re: [PATCH 094/115] ion: fix printk warnings

Quoted 3 lines · John Stultz

From: Colin Cross <[email protected]>

Use %z for size_t and %pa for dma_addr_t to avoid warnings in printks.

dma_addr_t is not a phys_addr_t and should _not_ be emitted with %pa.

Until %pad is in -next, use %lx, (unsigned long long)dma_addr_t.

Quoted 125 lines
---
 drivers/staging/android/ion/ion.c            | 23 ++++++++++++-----------
 drivers/staging/android/ion/ion_chunk_heap.c |  2 +-
 drivers/staging/android/ion/ion_cma_heap.c   |  4 ++--
 drivers/staging/android/ion/ion_heap.c       |  2 +-
 4 files changed, 16 insertions(+), 15 deletions(-)
drivers/staging/android/ion/ion.c +12 -11
index 199ab9a..57698e6 100644
@@ -471,7 +471,7 @@ struct ion_handle *ion_alloc(struct ion_client *client, size_t len,
 	struct ion_heap *heap;
 	int ret;
 
-	pr_debug("%s: len %d align %d heap_id_mask %u flags %x\n", __func__,
+	pr_debug("%s: len %zu align %zu heap_id_mask %u flags %x\n", __func__,
 		 len, align, heap_id_mask, flags);
 	/*
 	 * traverse the list of heaps available in this system in priority
@@ -688,7 +688,7 @@ static int ion_debug_client_show(struct seq_file *s, void *unused)
 	for (i = 0; i < ION_NUM_HEAP_IDS; i++) {
 		if (!names[i])
 			continue;
-		seq_printf(s, "%16.16s: %16u\n", names[i], sizes[i]);
+		seq_printf(s, "%16.16s: %16zu\n", names[i], sizes[i]);
 	}
 	return 0;
 }
@@ -1336,10 +1336,10 @@ static int ion_debug_heap_show(struct seq_file *s, void *unused)
 			char task_comm[TASK_COMM_LEN];
 
 			get_task_comm(task_comm, client->task);
-			seq_printf(s, "%16.s %16u %16u\n", task_comm,
+			seq_printf(s, "%16.s %16u %16zu\n", task_comm,
 				   client->pid, size);
 		} else {
-			seq_printf(s, "%16.s %16u %16u\n", client->name,
+			seq_printf(s, "%16.s %16u %16zu\n", client->name,
 				   client->pid, size);
 		}
 	}
@@ -1354,19 +1354,20 @@ static int ion_debug_heap_show(struct seq_file *s, void *unused)
 			continue;
 		total_size += buffer->size;
 		if (!buffer->handle_count) {
-			seq_printf(s, "%16.s %16u %16u %d %d\n", buffer->task_comm,
-				   buffer->pid, buffer->size, buffer->kmap_cnt,
+			seq_printf(s, "%16.s %16u %16zu %d %d\n",
+				   buffer->task_comm, buffer->pid,
+				   buffer->size, buffer->kmap_cnt,
 				   atomic_read(&buffer->ref.refcount));
 			total_orphaned_size += buffer->size;
 		}
 	}
 	mutex_unlock(&dev->buffer_lock);
 	seq_printf(s, "----------------------------------------------------\n");
-	seq_printf(s, "%16.s %16u\n", "total orphaned",
+	seq_printf(s, "%16.s %16zu\n", "total orphaned",
 		   total_orphaned_size);
-	seq_printf(s, "%16.s %16u\n", "total ", total_size);
+	seq_printf(s, "%16.s %16zu\n", "total ", total_size);
 	if (heap->flags & ION_HEAP_FLAG_DEFER_FREE)
-		seq_printf(s, "%16.s %16u\n", "deferred free",
+		seq_printf(s, "%16.s %16zu\n", "deferred free",
 				heap->free_list_size);
 	seq_printf(s, "----------------------------------------------------\n");
 
@@ -1522,11 +1523,11 @@ void __init ion_reserve(struct ion_platform_data *data)
 			int ret = memblock_reserve(data->heaps[i].base,
 					       data->heaps[i].size);
 			if (ret)
-				pr_err("memblock reserve of %x@%lx failed\n",
+				pr_err("memblock reserve of %zx@%lx failed\n",
 				       data->heaps[i].size,
 				       data->heaps[i].base);
 		}
-		pr_info("%s: %s reserved base %lx size %d\n", __func__,
+		pr_info("%s: %s reserved base %lx size %zu\n", __func__,
 			data->heaps[i].name,
 			data->heaps[i].base,
 			data->heaps[i].size);
drivers/staging/android/ion/ion_chunk_heap.c +1 -1
index fb1a7b9..4eacf52 100644
@@ -186,7 +186,7 @@ struct ion_heap *ion_chunk_heap_create(struct ion_platform_heap *heap_data)
 	chunk_heap->heap.ops = &chunk_heap_ops;
 	chunk_heap->heap.type = ION_HEAP_TYPE_CHUNK;
 	chunk_heap->heap.flags = ION_HEAP_FLAG_DEFER_FREE;
-	pr_info("%s: base %lu size %u align %ld\n", __func__, chunk_heap->base,
+	pr_info("%s: base %lu size %zu align %ld\n", __func__, chunk_heap->base,
 		heap_data->size, heap_data->align);
 
 	return &chunk_heap->heap;
drivers/staging/android/ion/ion_cma_heap.c +2 -2
index 86b6cf5..1e308a0 100644
@@ -128,8 +128,8 @@ static int ion_cma_phys(struct ion_heap *heap, struct ion_buffer *buffer,
 	struct device *dev = cma_heap->dev;
 	struct ion_cma_buffer_info *info = buffer->priv_virt;
 
-	dev_dbg(dev, "Return buffer %p physical address 0x%x\n", buffer,
-		info->handle);
+	dev_dbg(dev, "Return buffer %p physical address 0x%pa\n", buffer,
+		&info->handle);
 
 	*addr = info->handle;
 	*len = buffer->size;
drivers/staging/android/ion/ion_heap.c +1 -1
index a5a7c572..deaab7c 100644
@@ -281,7 +281,7 @@ struct ion_heap *ion_heap_create(struct ion_platform_heap *heap_data)
 	}
 
 	if (IS_ERR_OR_NULL(heap)) {
-		pr_err("%s: error creating heap %s type %d base %lu size %u\n",
+		pr_err("%s: error creating heap %s type %d base %lu size %zu\n",
 		       __func__, heap_data->name, heap_data->type,
 		       heap_data->base, heap_data->size);
 		return ERR_PTR(-EINVAL);
#113 lore raw

PATCH 95/115 gpu: ion: remove unnecessary function from system heap

In reply to John Stultz

From: Colin Cross <[email protected]>

ion_system_contig_heap buffers have an sglist, just call ion_heap_map_user to map it.

---
 drivers/staging/android/ion/ion_system_heap.c | 13 +------------
 1 file changed, 1 insertion(+), 12 deletions(-)
drivers/staging/android/ion/ion_system_heap.c +1 -12
index 3b2f420..9250ee5 100644
@@ -422,17 +422,6 @@ void ion_system_contig_heap_unmap_dma(struct ion_heap *heap,
 	kfree(buffer->sg_table);
 }
 
-int ion_system_contig_heap_map_user(struct ion_heap *heap,
-				    struct ion_buffer *buffer,
-				    struct vm_area_struct *vma)
-{
-	unsigned long pfn = page_to_pfn(virt_to_page(buffer->priv_virt));
-	return remap_pfn_range(vma, vma->vm_start, pfn + vma->vm_pgoff,
-			       vma->vm_end - vma->vm_start,
-			       vma->vm_page_prot);
-
-}
-
 static struct ion_heap_ops kmalloc_ops = {
 	.allocate = ion_system_contig_heap_allocate,
 	.free = ion_system_contig_heap_free,
@@ -441,7 +430,7 @@ static struct ion_heap_ops kmalloc_ops = {
 	.unmap_dma = ion_system_contig_heap_unmap_dma,
 	.map_kernel = ion_heap_map_kernel,
 	.unmap_kernel = ion_heap_unmap_kernel,
-	.map_user = ion_system_contig_heap_map_user,
+	.map_user = ion_heap_map_user,
 };
 
 struct ion_heap *ion_system_contig_heap_create(struct ion_platform_heap *unused)
Signature · 1 line
1.8.3.2
#114 lore raw

PATCH 96/115 ion: clean up ioctls

In reply to John Stultz

From: Colin Cross <[email protected]>

Convert the ion ioctls to use _IOW instead of _IOWR where appropriate, and factor out the copy_from_user and copy_to_user based on the _IOC_DIR bits. For the existing incorrect ioctls, add a function to wrap _IOC_DIR to return the corrected value.

---
 drivers/staging/android/ion/ion.c | 110 ++++++++++++++++++++------------------
 1 file changed, 59 insertions(+), 51 deletions(-)
drivers/staging/android/ion/ion.c +59 -51
index 57698e6..fc77aa6 100644
@@ -1157,41 +1157,65 @@ static int ion_sync_for_device(struct ion_client *client, int fd)
 	return 0;
 }
 
+/* fix up the cases where the ioctl direction bits are incorrect */
+static unsigned int ion_ioctl_dir(unsigned int cmd)
+{
+	switch (cmd) {
+	case ION_IOC_SYNC:
+	case ION_IOC_FREE:
+	case ION_IOC_CUSTOM:
+		return _IOC_WRITE;
+	default:
+		return _IOC_DIR(cmd);
+	}
+}
+
 static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 {
 	struct ion_client *client = filp->private_data;
+	struct ion_device *dev = client->dev;
+	struct ion_handle *cleanup_handle = NULL;
+	int ret = 0;
+	unsigned int dir;
+
+	union {
+		struct ion_fd_data fd;
+		struct ion_allocation_data allocation;
+		struct ion_handle_data handle;
+		struct ion_custom_data custom;
+	} data;
+
+	dir = ion_ioctl_dir(cmd);
+
+	if (_IOC_SIZE(cmd) > sizeof(data))
+		return -EINVAL;
+
+	if (dir & _IOC_WRITE)
+		if (copy_from_user(&data, (void __user *)arg, _IOC_SIZE(cmd)))
+			return -EFAULT;
 
 	switch (cmd) {
 	case ION_IOC_ALLOC:
 	{
-		struct ion_allocation_data data;
 		struct ion_handle *handle;
 
-		if (copy_from_user(&data, (void __user *)arg, sizeof(data)))
-			return -EFAULT;
-		handle = ion_alloc(client, data.len, data.align,
-					     data.heap_id_mask, data.flags);
-
+		handle = ion_alloc(client, data.allocation.len,
+						data.allocation.align,
+						data.allocation.heap_id_mask,
+						data.allocation.flags);
 		if (IS_ERR(handle))
 			return PTR_ERR(handle);
 
-		data.handle = handle->id;
+		data.allocation.handle = handle->id;
 
-		if (copy_to_user((void __user *)arg, &data, sizeof(data))) {
-			ion_free(client, handle);
-			return -EFAULT;
-		}
+		cleanup_handle = handle;
 		break;
 	}
 	case ION_IOC_FREE:
 	{
-		struct ion_handle_data data;
 		struct ion_handle *handle;
 
-		if (copy_from_user(&data, (void __user *)arg,
-				   sizeof(struct ion_handle_data)))
-			return -EFAULT;
-		handle = ion_handle_get_by_id(client, data.handle);
+		handle = ion_handle_get_by_id(client, data.handle.handle);
 		if (IS_ERR(handle))
 			return PTR_ERR(handle);
 		ion_free(client, handle);
@@ -1201,68 +1225,52 @@ static long ion_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 	case ION_IOC_SHARE:
 	case ION_IOC_MAP:
 	{
-		struct ion_fd_data data;
 		struct ion_handle *handle;
 
-		if (copy_from_user(&data, (void __user *)arg, sizeof(data)))
-			return -EFAULT;
-		handle = ion_handle_get_by_id(client, data.handle);
+		handle = ion_handle_get_by_id(client, data.handle.handle);
 		if (IS_ERR(handle))
 			return PTR_ERR(handle);
-		data.fd = ion_share_dma_buf_fd(client, handle);
+		data.fd.fd = ion_share_dma_buf_fd(client, handle);
 		ion_handle_put(handle);
-		if (copy_to_user((void __user *)arg, &data, sizeof(data)))
-			return -EFAULT;
-		if (data.fd < 0)
-			return data.fd;
+		if (data.fd.fd < 0)
+			ret = data.fd.fd;
 		break;
 	}
 	case ION_IOC_IMPORT:
 	{
-		struct ion_fd_data data;
 		struct ion_handle *handle;
-		int ret = 0;
-		if (copy_from_user(&data, (void __user *)arg,
-				   sizeof(struct ion_fd_data)))
-			return -EFAULT;
-		handle = ion_import_dma_buf(client, data.fd);
+		handle = ion_import_dma_buf(client, data.fd.fd);
 		if (IS_ERR(handle))
 			ret = PTR_ERR(handle);
 		else
-			data.handle = handle->id;
-
-		if (copy_to_user((void __user *)arg, &data,
-				 sizeof(struct ion_fd_data)))
-			return -EFAULT;
-		if (ret < 0)
-			return ret;
+			data.handle.handle = handle->id;
 		break;
 	}
 	case ION_IOC_SYNC:
 	{
-		struct ion_fd_data data;
-		if (copy_from_user(&data, (void __user *)arg,
-				   sizeof(struct ion_fd_data)))
-			return -EFAULT;
-		ion_sync_for_device(client, data.fd);
+		ret = ion_sync_for_device(client, data.fd.fd);
 		break;
 	}
 	case ION_IOC_CUSTOM:
 	{
-		struct ion_device *dev = client->dev;
-		struct ion_custom_data data;
-
 		if (!dev->custom_ioctl)
 			return -ENOTTY;
-		if (copy_from_user(&data, (void __user *)arg,
-				sizeof(struct ion_custom_data)))
-			return -EFAULT;
-		return dev->custom_ioctl(client, data.cmd, data.arg);
+		ret = dev->custom_ioctl(client, data.custom.cmd,
+						data.custom.arg);
+		break;
 	}
 	default:
 		return -ENOTTY;
 	}
-	return 0;
+
+	if (dir & _IOC_READ) {
+		if (copy_to_user((void __user *)arg, &data, _IOC_SIZE(cmd))) {
+			if (cleanup_handle)
+				ion_free(client, cleanup_handle);
+			return -EFAULT;
+		}
+	}
+	return ret;
 }
 
 static int ion_release(struct inode *inode, struct file *file)
Signature · 1 line
1.8.3.2
#115 lore raw

PATCH 97/115 gpu: ion: fix use-after-free in ion_heap_freelist_drain

In reply to John Stultz

From: Mitchel Humpherys <[email protected]>

The `buffer' variable is being used after being freed. Fix this.

---
 drivers/staging/android/ion/ion_heap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/android/ion/ion_heap.c +1 -1
index deaab7c..0a5cea0 100644
@@ -200,9 +200,9 @@ size_t ion_heap_freelist_drain(struct ion_heap *heap, size_t size)
 		if (total_drained >= size)
 			break;
 		list_del(&buffer->list);
-		ion_buffer_destroy(buffer);
 		heap->free_list_size -= buffer->size;
 		total_drained += buffer->size;
+		ion_buffer_destroy(buffer);
 	}
 	rt_mutex_unlock(&heap->lock);
 
Signature · 1 line
1.8.3.2
#116 lore raw

PATCH 98/115 ion: Fix two small issues in system_heap allocation

In reply to John Stultz

In testing ion system heap allocations, I ran across two issues:

1) Not k*z*allocing the sg table. This can cause trouble if we end up trying call sg_alloc_table() with too many entries, then sg_alloc_table() internally fails and tries to free what it thinks is internal table structure, which causes bad pointer traversals.

2) The second list_for_each_entry probably should be _safe, since I was seeing strange lock warnings and oopses on occasion. This seems to resolve it, but could use some extra checking.

---
 drivers/staging/android/ion/ion_system_heap.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/staging/android/ion/ion_system_heap.c +2 -3
index 9250ee5..bec9d45 100644
@@ -166,8 +166,7 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 		max_order = info->order;
 		i++;
 	}
-
-	table = kmalloc(sizeof(struct sg_table), GFP_KERNEL);
+	table = kzalloc(sizeof(struct sg_table), GFP_KERNEL);
 	if (!table)
 		goto err;
 
@@ -189,7 +188,7 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 err1:
 	kfree(table);
 err:
-	list_for_each_entry(info, &pages, list) {
+	list_for_each_entry_safe(info, tmp_info, &pages, list) {
 		free_buffer_page(sys_heap, buffer, info->page, info->order);
 		kfree(info);
 	}
Signature · 1 line
1.8.3.2
#117 lore raw

PATCH 99/115 ion: drop dependency on ARM

In reply to John Stultz

From: Colin Cross <[email protected]>

Ion will compile and run on other platforms now, remove the dependency on ARM.

---
 drivers/staging/android/ion/Kconfig | 1 -
 1 file changed, 1 deletion(-)
drivers/staging/android/ion/Kconfig +0 -1
index 6a5d8cf..3215bd8 100644
@@ -1,6 +1,5 @@
 menuconfig ION
 	tristate "Ion Memory Manager"
-	depends on ARM
 	select GENERIC_ALLOCATOR
 	select DMA_SHARED_BUFFER
 	help
Signature · 1 line
1.8.3.2
#118 lore raw

PATCH 100/115 ion: add alignment check to carveout heap

In reply to John Stultz

From: Colin Cross <[email protected]>

---
 drivers/staging/android/ion/ion_carveout_heap.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/staging/android/ion/ion_carveout_heap.c +3 -0
index 1d53e91..b0b08c3 100644
@@ -70,6 +70,9 @@ static int ion_carveout_heap_allocate(struct ion_heap *heap,
 				      unsigned long size, unsigned long align,
 				      unsigned long flags)
 {
+	if (align > PAGE_SIZE)
+		return -EINVAL;
+
 	buffer->priv_phys = ion_carveout_allocate(heap, size, align);
 	return buffer->priv_phys == ION_CARVEOUT_ALLOCATE_FAIL ? -ENOMEM : 0;
 }
Signature · 1 line
1.8.3.2
#119 lore raw

PATCH 101/115 ion: optimize ion_heap_buffer_zero

In reply to John Stultz

From: Colin Cross <[email protected]>

ion_heap_buffer_zero can spend a long time in unmap_kernel_range if it has to broadcast a tlb flush to every cpu for every page. Modify it to batch pages into a larger region to clear using a single mapping. This may cause the mapping size to change if the buffer size is not a multiple of the mapping size, so switch to allocating the address space for each chunk. This allows us to use vm_map_ram to handle the allocation and mapping together.

The number of pages to zero using a single mapping is set to 32 to hit the fastpath in vm_map_ram.

---
 drivers/staging/android/ion/ion_heap.c | 36 +++++++++++++++++++++-------------
 1 file changed, 22 insertions(+), 14 deletions(-)
drivers/staging/android/ion/ion_heap.c +22 -14
index 0a5cea0..ce31561 100644
@@ -100,40 +100,48 @@ int ion_heap_map_user(struct ion_heap *heap, struct ion_buffer *buffer,
 	return 0;
 }
 
+static int ion_heap_clear_pages(struct page **pages, int num, pgprot_t pgprot)
+{
+	void *addr = vm_map_ram(pages, num, -1, pgprot);
+	if (!addr)
+		return -ENOMEM;
+	memset(addr, 0, PAGE_SIZE * num);
+	vm_unmap_ram(addr, num);
+
+	return 0;
+}
+
 int ion_heap_buffer_zero(struct ion_buffer *buffer)
 {
 	struct sg_table *table = buffer->sg_table;
 	pgprot_t pgprot;
 	struct scatterlist *sg;
-	struct vm_struct *vm_struct;
 	int i, j, ret = 0;
+	struct page *pages[32];
+	int k = 0;
 
 	if (buffer->flags & ION_FLAG_CACHED)
 		pgprot = PAGE_KERNEL;
 	else
 		pgprot = pgprot_writecombine(PAGE_KERNEL);
 
-	vm_struct = get_vm_area(PAGE_SIZE, VM_ALLOC);
-	if (!vm_struct)
-		return -ENOMEM;
-
 	for_each_sg(table->sgl, sg, table->nents, i) {
 		struct page *page = sg_page(sg);
 		unsigned long len = sg->length;
 
 		for (j = 0; j < len / PAGE_SIZE; j++) {
-			struct page *sub_page = page + j;
-			struct page **pages = &sub_page;
-			ret = map_vm_area(vm_struct, pgprot, &pages);
-			if (ret)
-				goto end;
-			memset(vm_struct->addr, 0, PAGE_SIZE);
-			unmap_kernel_range((unsigned long)vm_struct->addr,
-					   PAGE_SIZE);
+			pages[k++] = page + j;
+			if (k == ARRAY_SIZE(pages)) {
+				ret = ion_heap_clear_pages(pages, k, pgprot);
+				if (ret)
+					goto end;
+				k = 0;
+			}
 		}
+		if (k)
+			ret = ion_heap_clear_pages(pages, k, pgprot);
 	}
 end:
-	free_vm_area(vm_struct);
 	return ret;
 }
 
Signature · 1 line
1.8.3.2
#120 lore raw

PATCH 102/115 ion: free low memory from page pools first

In reply to John Stultz

From: Colin Cross <[email protected]>

When the shrinkers are called with GFP_HIGH free low memory first, it is more important to have free than high memory.

---
 drivers/staging/android/ion/ion_page_pool.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/staging/android/ion/ion_page_pool.c +3 -3
index 979d2e4..0ad0263 100644
@@ -143,10 +143,10 @@ int ion_page_pool_shrink(struct ion_page_pool *pool, gfp_t gfp_mask,
 		struct page *page;
 
 		mutex_lock(&pool->mutex);
-		if (high && pool->high_count) {
-			page = ion_page_pool_remove(pool, true);
-		} else if (pool->low_count) {
+		if (pool->low_count) {
 			page = ion_page_pool_remove(pool, false);
+		} else if (high && pool->high_count) {
+			page = ion_page_pool_remove(pool, true);
 		} else {
 			mutex_unlock(&pool->mutex);
 			break;
Signature · 1 line
1.8.3.2
#121 lore raw

PATCH 103/115 ion: check return value from remap_pfn_range

In reply to John Stultz

From: Colin Cross <[email protected]>

Check the return value of remap_pfn_range and return an error if it fails.

---
 drivers/staging/android/ion/ion_heap.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/staging/android/ion/ion_heap.c +4 -1
index ce31561..ecc14fd 100644
@@ -76,6 +76,7 @@ int ion_heap_map_user(struct ion_heap *heap, struct ion_buffer *buffer,
 	unsigned long offset = vma->vm_pgoff * PAGE_SIZE;
 	struct scatterlist *sg;
 	int i;
+	int ret;
 
 	for_each_sg(table->sgl, sg, table->nents, i) {
 		struct page *page = sg_page(sg);
@@ -91,8 +92,10 @@ int ion_heap_map_user(struct ion_heap *heap, struct ion_buffer *buffer,
 			offset = 0;
 		}
 		len = min(len, remainder);
-		remap_pfn_range(vma, addr, page_to_pfn(page), len,
+		ret = remap_pfn_range(vma, addr, page_to_pfn(page), len,
 				vma->vm_page_prot);
+		if (ret)
+			return ret;
 		addr += len;
 		if (addr >= vma->vm_end)
 			return 0;
Signature · 1 line
1.8.3.2
#122 lore raw

PATCH 104/115 ion: use vm_insert_pfn for faulted pages

In reply to John Stultz

From: Colin Cross <[email protected]>

Most ion userspace mappings are created with remap_pfn_range. Use vm_insert_pfn instead of vm_insert_page to make faulted cached mappings look more like uncached mappings.

---
 drivers/staging/android/ion/ion.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
drivers/staging/android/ion/ion.c +6 -3
index fc77aa6..45127f5 100644
@@ -890,14 +890,15 @@ static void ion_buffer_sync_for_device(struct ion_buffer *buffer,
 int ion_vm_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
 {
 	struct ion_buffer *buffer = vma->vm_private_data;
+	unsigned long pfn;
 	int ret;
 
 	mutex_lock(&buffer->lock);
 	ion_buffer_page_dirty(buffer->pages + vmf->pgoff);
-
 	BUG_ON(!buffer->pages || !buffer->pages[vmf->pgoff]);
-	ret = vm_insert_page(vma, (unsigned long)vmf->virtual_address,
-			     ion_buffer_page(buffer->pages[vmf->pgoff]));
+
+	pfn = page_to_pfn(ion_buffer_page(buffer->pages[vmf->pgoff]));
+	ret = vm_insert_pfn(vma, (unsigned long)vmf->virtual_address, pfn);
 	mutex_unlock(&buffer->lock);
 	if (ret)
 		return VM_FAULT_ERROR;
@@ -956,6 +957,8 @@ static int ion_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
 	}
 
 	if (ion_buffer_fault_user_mappings(buffer)) {
+		vma->vm_flags |= VM_IO | VM_PFNMAP | VM_DONTEXPAND |
+							VM_DONTDUMP;
 		vma->vm_private_data = buffer;
 		vma->vm_ops = &ion_vma_ops;
 		ion_vm_open(vma);
Signature · 1 line
1.8.3.2
#123 lore raw

PATCH 105/115 ion: remove ion_heap_alloc_pages

In reply to John Stultz

From: Colin Cross <[email protected]>

Now that ion_vm_fault doesn't need a struct page with a nonzero refcount, there is no need allocate heap memory for cached pages using split_page. Remove the ion_heap_alloc_pages and ion_heap_free_pages helpers in favor of direct calls to alloc_pages and __free_pages, and remove the special handling in the system heap.

---
 drivers/staging/android/ion/ion_heap.c        | 27 ---------------------------
 drivers/staging/android/ion/ion_priv.h        | 13 -------------
 drivers/staging/android/ion/ion_system_heap.c |  7 +------
 3 files changed, 1 insertion(+), 46 deletions(-)
drivers/staging/android/ion/ion_heap.c +0 -27
index ecc14fd..8054611 100644
@@ -148,33 +148,6 @@ end:
 	return ret;
 }
 
-struct page *ion_heap_alloc_pages(struct ion_buffer *buffer, gfp_t gfp_flags,
-				  unsigned int order)
-{
-	struct page *page = alloc_pages(gfp_flags, order);
-
-	if (!page)
-		return page;
-
-	if (ion_buffer_fault_user_mappings(buffer))
-		split_page(page, order);
-
-	return page;
-}
-
-void ion_heap_free_pages(struct ion_buffer *buffer, struct page *page,
-			 unsigned int order)
-{
-	int i;
-
-	if (!ion_buffer_fault_user_mappings(buffer)) {
-		__free_pages(page, order);
-		return;
-	}
-	for (i = 0; i < (1 << order); i++)
-		__free_page(page + i);
-}
-
 void ion_heap_freelist_add(struct ion_heap *heap, struct ion_buffer * buffer)
 {
 	rt_mutex_lock(&heap->lock);
drivers/staging/android/ion/ion_priv.h +0 -13
index d1fa771..c9b507d 100644
@@ -217,19 +217,6 @@ int ion_heap_map_user(struct ion_heap *, struct ion_buffer *,
 int ion_heap_buffer_zero(struct ion_buffer *buffer);
 
 /**
- * ion_heap_alloc_pages - allocate pages from alloc_pages
- * @buffer:		the buffer to allocate for, used to extract the flags
- * @gfp_flags:		the gfp_t for the allocation
- * @order:		the order of the allocatoin
- *
- * This funciton allocations from alloc pages and also does any other
- * necessary operations based on the buffer->flags.  For buffers which
- * will be faulted in the pages are split using split_page
- */
-struct page *ion_heap_alloc_pages(struct ion_buffer *buffer, gfp_t gfp_flags,
-				  unsigned int order);
-
-/**
  * ion_heap_init_deferred_free -- initialize deferred free functionality
  * @heap:		the heap
  *
drivers/staging/android/ion/ion_system_heap.c +1 -6
index bec9d45..a4bcf09 100644
@@ -74,7 +74,7 @@ static struct page *alloc_buffer_page(struct ion_system_heap *heap,
 
 		if (order > 4)
 			gfp_flags = high_order_gfp_flags;
-		page = ion_heap_alloc_pages(buffer, gfp_flags, order);
+		page = alloc_pages(gfp_flags, order);
 		if (!page)
 			return 0;
 		ion_pages_sync_for_device(NULL, page, PAGE_SIZE << order,
@@ -91,15 +91,10 @@ static void free_buffer_page(struct ion_system_heap *heap,
 			     unsigned int order)
 {
 	bool cached = ion_buffer_cached(buffer);
-	bool split_pages = ion_buffer_fault_user_mappings(buffer);
-	int i;
 
 	if (!cached) {
 		struct ion_page_pool *pool = heap->pools[order_to_index(order)];
 		ion_page_pool_free(pool, page);
-	} else if (split_pages) {
-		for (i = 0; i < (1 << order); i++)
-			__free_page(page + i);
 	} else {
 		__free_pages(page, order);
 	}
Signature · 1 line
1.8.3.2
#124 lore raw

PATCH 106/115 ion: allow cached mappings of chunk and system heap buffers

In reply to John Stultz

From: Colin Cross <[email protected]>

Now that ion_vm_fault uses vm_insert_pfn instead of vm_insert_page cached buffers can be supported in any heap. Remove the checks in the chunk and system heaps.

---
 drivers/staging/android/ion/ion_chunk_heap.c  | 3 ---
 drivers/staging/android/ion/ion_system_heap.c | 3 ---
 2 files changed, 6 deletions(-)
drivers/staging/android/ion/ion_chunk_heap.c +0 -3
index 4eacf52..34d1d18 100644
@@ -47,9 +47,6 @@ static int ion_chunk_heap_allocate(struct ion_heap *heap,
 	unsigned long num_chunks;
 	unsigned long allocated_size;
 
-	if (ion_buffer_fault_user_mappings(buffer))
-		return -ENOMEM;
-
 	allocated_size = ALIGN(size, chunk_heap->chunk_size);
 	num_chunks = allocated_size / chunk_heap->chunk_size;
 
drivers/staging/android/ion/ion_system_heap.c +0 -3
index a4bcf09..53afa33 100644
@@ -148,9 +148,6 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 	if (align > PAGE_SIZE)
 		return -EINVAL;
 
-	if (ion_buffer_fault_user_mappings(buffer))
-		return -EINVAL;
-
 	INIT_LIST_HEAD(&pages);
 	while (size_remaining > 0) {
 		info = alloc_largest_available(sys_heap, buffer, size_remaining, max_order);
Signature · 1 line
1.8.3.2
#125 lore raw

PATCH 107/115 ion: use alloc_pages in system contig heap

In reply to John Stultz

From: Colin Cross <[email protected]>

There is no reason to use kzalloc, just call alloc_pages directly. Change the GFP from GFP_KERNEL to include __GFP_HIGH, to allow it to return contiguous pages from highmem. virt_to_* functions aren't valid on highmem pages, so store the struct page * in an sg_table in buffer->priv_virt like most other heaps, and replace virt_to_* with page_to_*.

---
 drivers/staging/android/ion/ion_system_heap.c | 71 ++++++++++++++++++---------
 1 file changed, 48 insertions(+), 23 deletions(-)
drivers/staging/android/ion/ion_system_heap.c +48 -23
index 53afa33..841511d 100644
@@ -360,29 +360,69 @@ static int ion_system_contig_heap_allocate(struct ion_heap *heap,
 					   unsigned long flags)
 {
 	int order = get_order(len);
+	struct page *page;
+	struct sg_table *table;
+	unsigned long i;
+	int ret;
 
 	if (align > (PAGE_SIZE << order))
 		return -EINVAL;
 
-	if (ion_buffer_fault_user_mappings(buffer))
-		return -EINVAL;
-
-	buffer->priv_virt = kzalloc(len, GFP_KERNEL);
-	if (!buffer->priv_virt)
+	page = alloc_pages(low_order_gfp_flags, order);
+	if (!page)
 		return -ENOMEM;
+
+	split_page(page, order);
+
+	len = PAGE_ALIGN(len);
+	for (i = len >> PAGE_SHIFT; i < (1 << order); i++)
+		__free_page(page + i);
+
+	table = kzalloc(sizeof(struct sg_table), GFP_KERNEL);
+	if (!table) {
+		ret = -ENOMEM;
+		goto out;
+	}
+
+	ret = sg_alloc_table(table, 1, GFP_KERNEL);
+	if (ret)
+		goto out;
+
+	sg_set_page(table->sgl, page, len, 0);
+
+	buffer->priv_virt = table;
+
+	ion_pages_sync_for_device(NULL, page, len, DMA_BIDIRECTIONAL);
+
 	return 0;
+
+out:
+	for (i = 0; i < len >> PAGE_SHIFT; i++)
+		__free_page(page + i);
+	kfree(table);
+	return ret;
 }
 
 void ion_system_contig_heap_free(struct ion_buffer *buffer)
 {
-	kfree(buffer->priv_virt);
+	struct sg_table *table = buffer->priv_virt;
+	struct page *page = sg_page(table->sgl);
+	unsigned long pages = PAGE_ALIGN(buffer->size) >> PAGE_SHIFT;
+	unsigned long i;
+
+	for (i = 0; i < pages; i++)
+		__free_page(page + i);
+	sg_free_table(table);
+	kfree(table);
 }
 
 static int ion_system_contig_heap_phys(struct ion_heap *heap,
 				       struct ion_buffer *buffer,
 				       ion_phys_addr_t *addr, size_t *len)
 {
-	*addr = virt_to_phys(buffer->priv_virt);
+	struct sg_table *table = buffer->priv_virt;
+	struct page *page = sg_page(table->sgl);
+	*addr = page_to_phys(page);
 	*len = buffer->size;
 	return 0;
 }
@@ -390,27 +430,12 @@ static int ion_system_contig_heap_phys(struct ion_heap *heap,
 struct sg_table *ion_system_contig_heap_map_dma(struct ion_heap *heap,
 						struct ion_buffer *buffer)
 {
-	struct sg_table *table;
-	int ret;
-
-	table = kzalloc(sizeof(struct sg_table), GFP_KERNEL);
-	if (!table)
-		return ERR_PTR(-ENOMEM);
-	ret = sg_alloc_table(table, 1, GFP_KERNEL);
-	if (ret) {
-		kfree(table);
-		return ERR_PTR(ret);
-	}
-	sg_set_page(table->sgl, virt_to_page(buffer->priv_virt), buffer->size,
-		    0);
-	return table;
+	return buffer->priv_virt;
 }
 
 void ion_system_contig_heap_unmap_dma(struct ion_heap *heap,
 				      struct ion_buffer *buffer)
 {
-	sg_free_table(buffer->sg_table);
-	kfree(buffer->sg_table);
 }
 
 static struct ion_heap_ops kmalloc_ops = {
Signature · 1 line
1.8.3.2
#126 lore raw

PATCH 108/115 ion: fix sparse warnings

In reply to John Stultz

From: Colin Cross <[email protected]>

Fix sparse warnings in ion.

---
 drivers/staging/android/ion/ion.c               |  8 +++----
 drivers/staging/android/ion/ion_carveout_heap.c |  8 +++----
 drivers/staging/android/ion/ion_chunk_heap.c    |  8 +++----
 drivers/staging/android/ion/ion_cma_heap.c      | 15 +++++++------
 drivers/staging/android/ion/ion_heap.c          |  4 ++--
 drivers/staging/android/ion/ion_page_pool.c     |  2 +-
 drivers/staging/android/ion/ion_system_heap.c   | 30 ++++++++++++-------------
 7 files changed, 37 insertions(+), 38 deletions(-)
drivers/staging/android/ion/ion.c +4 -4
index 45127f5..559e4ee 100644
@@ -669,7 +669,7 @@ static int ion_debug_client_show(struct seq_file *s, void *unused)
 	struct ion_client *client = s->private;
 	struct rb_node *n;
 	size_t sizes[ION_NUM_HEAP_IDS] = {0};
-	const char *names[ION_NUM_HEAP_IDS] = {0};
+	const char *names[ION_NUM_HEAP_IDS] = {NULL};
 	int i;
 
 	mutex_lock(&client->lock);
@@ -887,7 +887,7 @@ static void ion_buffer_sync_for_device(struct ion_buffer *buffer,
 	mutex_unlock(&buffer->lock);
 }
 
-int ion_vm_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
+static int ion_vm_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
 {
 	struct ion_buffer *buffer = vma->vm_private_data;
 	unsigned long pfn;
@@ -939,7 +939,7 @@ static void ion_vm_close(struct vm_area_struct *vma)
 	mutex_unlock(&buffer->lock);
 }
 
-struct vm_operations_struct ion_vma_ops = {
+static struct vm_operations_struct ion_vma_ops = {
 	.open = ion_vm_open,
 	.close = ion_vm_close,
 	.fault = ion_vm_fault,
@@ -1030,7 +1030,7 @@ static void ion_dma_buf_end_cpu_access(struct dma_buf *dmabuf, size_t start,
 	mutex_unlock(&buffer->lock);
 }
 
-struct dma_buf_ops dma_buf_ops = {
+static struct dma_buf_ops dma_buf_ops = {
 	.map_dma_buf = ion_map_dma_buf,
 	.unmap_dma_buf = ion_unmap_dma_buf,
 	.mmap = ion_mmap,
drivers/staging/android/ion/ion_carveout_heap.c +4 -4
index b0b08c3..9bf20a3 100644
@@ -85,8 +85,8 @@ static void ion_carveout_heap_free(struct ion_buffer *buffer)
 	buffer->priv_phys = ION_CARVEOUT_ALLOCATE_FAIL;
 }
 
-struct sg_table *ion_carveout_heap_map_dma(struct ion_heap *heap,
-					      struct ion_buffer *buffer)
+static struct sg_table *ion_carveout_heap_map_dma(struct ion_heap *heap,
+						  struct ion_buffer *buffer)
 {
 	struct sg_table *table;
 	int ret;
@@ -104,8 +104,8 @@ struct sg_table *ion_carveout_heap_map_dma(struct ion_heap *heap,
 	return table;
 }
 
-void ion_carveout_heap_unmap_dma(struct ion_heap *heap,
-				 struct ion_buffer *buffer)
+static void ion_carveout_heap_unmap_dma(struct ion_heap *heap,
+					struct ion_buffer *buffer)
 {
 	sg_free_table(buffer->sg_table);
 }
drivers/staging/android/ion/ion_chunk_heap.c +4 -4
index 34d1d18..46e8a3b 100644
@@ -115,14 +115,14 @@ static void ion_chunk_heap_free(struct ion_buffer *buffer)
 	kfree(table);
 }
 
-struct sg_table *ion_chunk_heap_map_dma(struct ion_heap *heap,
-					 struct ion_buffer *buffer)
+static struct sg_table *ion_chunk_heap_map_dma(struct ion_heap *heap,
+					       struct ion_buffer *buffer)
 {
 	return buffer->priv_virt;
 }
 
-void ion_chunk_heap_unmap_dma(struct ion_heap *heap,
-			       struct ion_buffer *buffer)
+static void ion_chunk_heap_unmap_dma(struct ion_heap *heap,
+				     struct ion_buffer *buffer)
 {
 	return;
 }
drivers/staging/android/ion/ion_cma_heap.c +8 -7
index 1e308a0..3dcf9b4 100644
@@ -44,8 +44,8 @@ struct ion_cma_buffer_info {
  * This function could be replaced by dma_common_get_sgtable
  * as soon as it will avalaible.
  */
-int ion_cma_get_sgtable(struct device *dev, struct sg_table *sgt,
-			void *cpu_addr, dma_addr_t handle, size_t size)
+static int ion_cma_get_sgtable(struct device *dev, struct sg_table *sgt,
+			       void *cpu_addr, dma_addr_t handle, size_t size)
 {
 	struct page *page = virt_to_page(cpu_addr);
 	int ret;
@@ -137,16 +137,16 @@ static int ion_cma_phys(struct ion_heap *heap, struct ion_buffer *buffer,
 	return 0;
 }
 
-struct sg_table *ion_cma_heap_map_dma(struct ion_heap *heap,
-					 struct ion_buffer *buffer)
+static struct sg_table *ion_cma_heap_map_dma(struct ion_heap *heap,
+					     struct ion_buffer *buffer)
 {
 	struct ion_cma_buffer_info *info = buffer->priv_virt;
 
 	return info->table;
 }
 
-void ion_cma_heap_unmap_dma(struct ion_heap *heap,
-			       struct ion_buffer *buffer)
+static void ion_cma_heap_unmap_dma(struct ion_heap *heap,
+				   struct ion_buffer *buffer)
 {
 	return;
 }
@@ -162,7 +162,8 @@ static int ion_cma_mmap(struct ion_heap *mapper, struct ion_buffer *buffer,
 				 buffer->size);
 }
 
-void *ion_cma_map_kernel(struct ion_heap *heap, struct ion_buffer *buffer)
+static void *ion_cma_map_kernel(struct ion_heap *heap,
+				struct ion_buffer *buffer)
 {
 	struct ion_cma_buffer_info *info = buffer->priv_virt;
 	/* kernel memory mapping has been done at allocation time */
drivers/staging/android/ion/ion_heap.c +2 -2
index 8054611..c3a213b 100644
@@ -38,7 +38,7 @@ void *ion_heap_map_kernel(struct ion_heap *heap,
 	struct page **tmp = pages;
 
 	if (!pages)
-		return 0;
+		return NULL;
 
 	if (buffer->flags & ION_FLAG_CACHED)
 		pgprot = PAGE_KERNEL;
@@ -193,7 +193,7 @@ size_t ion_heap_freelist_drain(struct ion_heap *heap, size_t size)
 	return total_drained;
 }
 
-int ion_heap_deferred_free(void *data)
+static int ion_heap_deferred_free(void *data)
 {
 	struct ion_heap *heap = data;
 
drivers/staging/android/ion/ion_page_pool.c +1 -1
index 0ad0263..a1c51a8 100644
@@ -134,7 +134,7 @@ int ion_page_pool_shrink(struct ion_page_pool *pool, gfp_t gfp_mask,
 	int i;
 	bool high;
 
-	high = gfp_mask & __GFP_HIGHMEM;
+	high = !!(gfp_mask & __GFP_HIGHMEM);
 
 	if (nr_to_scan == 0)
 		return ion_page_pool_total(pool, high);
drivers/staging/android/ion/ion_system_heap.c +14 -16
index 841511d..301d019 100644
@@ -26,11 +26,9 @@
 #include "ion.h"
 #include "ion_priv.h"
 
-static unsigned int high_order_gfp_flags = (GFP_HIGHUSER | __GFP_ZERO |
-					    __GFP_NOWARN | __GFP_NORETRY) &
-					   ~__GFP_WAIT;
-static unsigned int low_order_gfp_flags  = (GFP_HIGHUSER | __GFP_ZERO |
-					 __GFP_NOWARN);
+static gfp_t high_order_gfp_flags = (GFP_HIGHUSER | __GFP_ZERO | __GFP_NOWARN |
+				     __GFP_NORETRY) & ~__GFP_WAIT;
+static gfp_t low_order_gfp_flags  = (GFP_HIGHUSER | __GFP_ZERO | __GFP_NOWARN);
 static const unsigned int orders[] = {8, 4, 0};
 static const int num_orders = ARRAY_SIZE(orders);
 static int order_to_index(unsigned int order)
@@ -76,12 +74,12 @@ static struct page *alloc_buffer_page(struct ion_system_heap *heap,
 			gfp_flags = high_order_gfp_flags;
 		page = alloc_pages(gfp_flags, order);
 		if (!page)
-			return 0;
+			return NULL;
 		ion_pages_sync_for_device(NULL, page, PAGE_SIZE << order,
 						DMA_BIDIRECTIONAL);
 	}
 	if (!page)
-		return 0;
+		return NULL;
 
 	return page;
 }
@@ -187,7 +185,7 @@ err:
 	return -ENOMEM;
 }
 
-void ion_system_heap_free(struct ion_buffer *buffer)
+static void ion_system_heap_free(struct ion_buffer *buffer)
 {
 	struct ion_heap *heap = buffer->heap;
 	struct ion_system_heap *sys_heap = container_of(heap,
@@ -211,14 +209,14 @@ void ion_system_heap_free(struct ion_buffer *buffer)
 	kfree(table);
 }
 
-struct sg_table *ion_system_heap_map_dma(struct ion_heap *heap,
-					 struct ion_buffer *buffer)
+static struct sg_table *ion_system_heap_map_dma(struct ion_heap *heap,
+						struct ion_buffer *buffer)
 {
 	return buffer->priv_virt;
 }
 
-void ion_system_heap_unmap_dma(struct ion_heap *heap,
-			       struct ion_buffer *buffer)
+static void ion_system_heap_unmap_dma(struct ion_heap *heap,
+				      struct ion_buffer *buffer)
 {
 	return;
 }
@@ -403,7 +401,7 @@ out:
 	return ret;
 }
 
-void ion_system_contig_heap_free(struct ion_buffer *buffer)
+static void ion_system_contig_heap_free(struct ion_buffer *buffer)
 {
 	struct sg_table *table = buffer->priv_virt;
 	struct page *page = sg_page(table->sgl);
@@ -427,14 +425,14 @@ static int ion_system_contig_heap_phys(struct ion_heap *heap,
 	return 0;
 }
 
-struct sg_table *ion_system_contig_heap_map_dma(struct ion_heap *heap,
+static struct sg_table *ion_system_contig_heap_map_dma(struct ion_heap *heap,
 						struct ion_buffer *buffer)
 {
 	return buffer->priv_virt;
 }
 
-void ion_system_contig_heap_unmap_dma(struct ion_heap *heap,
-				      struct ion_buffer *buffer)
+static void ion_system_contig_heap_unmap_dma(struct ion_heap *heap,
+					     struct ion_buffer *buffer)
 {
 }
 
Signature · 1 line
1.8.3.2
#127 lore raw

PATCH 109/115 ion: carveout heap: zero buffers on free, fix memory leak

In reply to John Stultz

From: Colin Cross <[email protected]>

The carveout heap wasn't zeroing its buffers after use. Create the sg_table during allocate instead of map_dma, to allow using the sg_table during free, and call ion_heap_buffer_zero during free. Also fixes a missing kfree when destroying the table.

---
 drivers/staging/android/ion/ion_carveout_heap.c | 69 +++++++++++++++++--------
 1 file changed, 48 insertions(+), 21 deletions(-)
drivers/staging/android/ion/ion_carveout_heap.c +48 -21
index 9bf20a3..66dec4c 100644
@@ -14,7 +14,7 @@
  *
  */
 #include <linux/spinlock.h>
-
+#include <linux/dma-mapping.h>
 #include <linux/err.h>
 #include <linux/genalloc.h>
 #include <linux/io.h>
@@ -60,7 +60,11 @@ static int ion_carveout_heap_phys(struct ion_heap *heap,
 				  struct ion_buffer *buffer,
 				  ion_phys_addr_t *addr, size_t *len)
 {
-	*addr = buffer->priv_phys;
+	struct sg_table *table = buffer->priv_virt;
+	struct page *page = sg_page(table->sgl);
+	ion_phys_addr_t paddr = PFN_PHYS(page_to_pfn(page));
+
+	*addr = paddr;
 	*len = buffer->size;
 	return 0;
 }
@@ -70,44 +74,66 @@ static int ion_carveout_heap_allocate(struct ion_heap *heap,
 				      unsigned long size, unsigned long align,
 				      unsigned long flags)
 {
+	struct sg_table *table;
+	ion_phys_addr_t paddr;
+	int ret;
+
 	if (align > PAGE_SIZE)
 		return -EINVAL;
 
-	buffer->priv_phys = ion_carveout_allocate(heap, size, align);
-	return buffer->priv_phys == ION_CARVEOUT_ALLOCATE_FAIL ? -ENOMEM : 0;
+	table = kzalloc(sizeof(struct sg_table), GFP_KERNEL);
+	if (!table)
+		return -ENOMEM;
+	ret = sg_alloc_table(table, 1, GFP_KERNEL);
+	if (ret)
+		goto err_free;
+
+	paddr = ion_carveout_allocate(heap, size, align);
+	if (paddr == ION_CARVEOUT_ALLOCATE_FAIL) {
+		ret = -ENOMEM;
+		goto err_free_table;
+	}
+
+	sg_set_page(table->sgl, pfn_to_page(PFN_DOWN(paddr)), size, 0);
+	buffer->priv_virt = table;
+
+	return 0;
+
+err_free_table:
+	sg_free_table(table);
+err_free:
+	kfree(table);
+	return ret;
 }
 
 static void ion_carveout_heap_free(struct ion_buffer *buffer)
 {
 	struct ion_heap *heap = buffer->heap;
+	struct sg_table *table = buffer->priv_virt;
+	struct page *page = sg_page(table->sgl);
+	ion_phys_addr_t paddr = PFN_PHYS(page_to_pfn(page));
+
+	ion_heap_buffer_zero(buffer);
 
-	ion_carveout_free(heap, buffer->priv_phys, buffer->size);
-	buffer->priv_phys = ION_CARVEOUT_ALLOCATE_FAIL;
+	if (ion_buffer_cached(buffer))
+		dma_sync_sg_for_device(NULL, table->sgl, table->nents,
+							DMA_BIDIRECTIONAL);
+
+	ion_carveout_free(heap, paddr, buffer->size);
+	sg_free_table(table);
+	kfree(table);
 }
 
 static struct sg_table *ion_carveout_heap_map_dma(struct ion_heap *heap,
 						  struct ion_buffer *buffer)
 {
-	struct sg_table *table;
-	int ret;
-
-	table = kzalloc(sizeof(struct sg_table), GFP_KERNEL);
-	if (!table)
-		return ERR_PTR(-ENOMEM);
-	ret = sg_alloc_table(table, 1, GFP_KERNEL);
-	if (ret) {
-		kfree(table);
-		return ERR_PTR(ret);
-	}
-	sg_set_page(table->sgl, pfn_to_page(PFN_DOWN(buffer->priv_phys)),
-			buffer->size, 0);
-	return table;
+	return buffer->priv_virt;
 }
 
 static void ion_carveout_heap_unmap_dma(struct ion_heap *heap,
 					struct ion_buffer *buffer)
 {
-	sg_free_table(buffer->sg_table);
+	return;
 }
 
 static struct ion_heap_ops carveout_heap_ops = {
@@ -139,6 +165,7 @@ struct ion_heap *ion_carveout_heap_create(struct ion_platform_heap *heap_data)
 		     -1);
 	carveout_heap->heap.ops = &carveout_heap_ops;
 	carveout_heap->heap.type = ION_HEAP_TYPE_CARVEOUT;
+	carveout_heap->heap.flags = ION_HEAP_FLAG_DEFER_FREE;
 
 	return &carveout_heap->heap;
 }
Signature · 1 line
1.8.3.2
#128 lore raw

PATCH 110/115 ion: add helper to zero contiguous region of pages

In reply to John Stultz

From: Colin Cross <[email protected]>

Add ion_heap_pages_zero for ion heaps to use to zero pages during initialization or allocation, when a struct ion_buffer may not be available. Use it from the chunk heap and carveout heaps.

---
 drivers/staging/android/ion/ion_carveout_heap.c | 13 ++++++
 drivers/staging/android/ion/ion_chunk_heap.c    | 39 ++++++------------
 drivers/staging/android/ion/ion_heap.c          | 53 +++++++++++++++----------
 drivers/staging/android/ion/ion_priv.h          |  1 +
 4 files changed, 58 insertions(+), 48 deletions(-)
drivers/staging/android/ion/ion_carveout_heap.c +13 -0
index 66dec4c..3cb05b9 100644
@@ -150,6 +150,19 @@ static struct ion_heap_ops carveout_heap_ops = {
 struct ion_heap *ion_carveout_heap_create(struct ion_platform_heap *heap_data)
 {
 	struct ion_carveout_heap *carveout_heap;
+	int ret;
+
+	struct page *page;
+	size_t size;
+
+	page = pfn_to_page(PFN_DOWN(heap_data->base));
+	size = heap_data->size;
+
+	ion_pages_sync_for_device(NULL, page, size, DMA_BIDIRECTIONAL);
+
+	ret = ion_heap_pages_zero(page, size, pgprot_writecombine(PAGE_KERNEL));
+	if (ret)
+		return ERR_PTR(ret);
 
 	carveout_heap = kzalloc(sizeof(struct ion_carveout_heap), GFP_KERNEL);
 	if (!carveout_heap)
drivers/staging/android/ion/ion_chunk_heap.c +12 -27
index 46e8a3b..f21530f 100644
@@ -140,9 +140,18 @@ static struct ion_heap_ops chunk_heap_ops = {
 struct ion_heap *ion_chunk_heap_create(struct ion_platform_heap *heap_data)
 {
 	struct ion_chunk_heap *chunk_heap;
-	struct vm_struct *vm_struct;
-	pgprot_t pgprot = pgprot_writecombine(PAGE_KERNEL);
-	int i, ret;
+	int ret;
+	struct page *page;
+	size_t size;
+
+	page = pfn_to_page(PFN_DOWN(heap_data->base));
+	size = heap_data->size;
+
+	ion_pages_sync_for_device(NULL, page, size, DMA_BIDIRECTIONAL);
+
+	ret = ion_heap_pages_zero(page, size, pgprot_writecombine(PAGE_KERNEL));
+	if (ret)
+		return ERR_PTR(ret);
 
 	chunk_heap = kzalloc(sizeof(struct ion_chunk_heap), GFP_KERNEL);
 	if (!chunk_heap)
@@ -159,26 +168,6 @@ struct ion_heap *ion_chunk_heap_create(struct ion_platform_heap *heap_data)
 	chunk_heap->size = heap_data->size;
 	chunk_heap->allocated = 0;
 
-	vm_struct = get_vm_area(PAGE_SIZE, VM_ALLOC);
-	if (!vm_struct) {
-		ret = -ENOMEM;
-		goto error;
-	}
-	for (i = 0; i < chunk_heap->size; i += PAGE_SIZE) {
-		struct page *page = pfn_to_page(PFN_DOWN(chunk_heap->base + i));
-		struct page **pages = &page;
-
-		ret = map_vm_area(vm_struct, pgprot, &pages);
-		if (ret)
-			goto error_map_vm_area;
-		memset(vm_struct->addr, 0, PAGE_SIZE);
-		unmap_kernel_range((unsigned long)vm_struct->addr, PAGE_SIZE);
-	}
-	free_vm_area(vm_struct);
-
-	ion_pages_sync_for_device(NULL, pfn_to_page(PFN_DOWN(heap_data->base)),
-			heap_data->size, DMA_BIDIRECTIONAL);
-
 	gen_pool_add(chunk_heap->pool, chunk_heap->base, heap_data->size, -1);
 	chunk_heap->heap.ops = &chunk_heap_ops;
 	chunk_heap->heap.type = ION_HEAP_TYPE_CHUNK;
@@ -188,10 +177,6 @@ struct ion_heap *ion_chunk_heap_create(struct ion_platform_heap *heap_data)
 
 	return &chunk_heap->heap;
 
-error_map_vm_area:
-	free_vm_area(vm_struct);
-error:
-	gen_pool_destroy(chunk_heap->pool);
 error_gen_pool_create:
 	kfree(chunk_heap);
 	return ERR_PTR(ret);
drivers/staging/android/ion/ion_heap.c +32 -21

70 lines not on this page · raw message

drivers/staging/android/ion/ion_priv.h +1 -0
index c9b507d..fa96175 100644
@@ -215,6 +215,7 @@ void ion_heap_unmap_kernel(struct ion_heap *, struct ion_buffer *);
 int ion_heap_map_user(struct ion_heap *, struct ion_buffer *,
 			struct vm_area_struct *);
 int ion_heap_buffer_zero(struct ion_buffer *buffer);
+int ion_heap_pages_zero(struct page *page, size_t size, pgprot_t pgprot);
 
 /**
  * ion_heap_init_deferred_free -- initialize deferred free functionality
Signature · 1 line
1.8.3.2
#129 lore raw

PATCH 111/115 ion: add alignment check to chunk heap

In reply to John Stultz

From: Colin Cross <[email protected]>

---
 drivers/staging/android/ion/ion_chunk_heap.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/staging/android/ion/ion_chunk_heap.c +3 -0
index f21530f..a072089 100644
@@ -47,6 +47,9 @@ static int ion_chunk_heap_allocate(struct ion_heap *heap,
 	unsigned long num_chunks;
 	unsigned long allocated_size;
 
+	if (align > chunk_heap->chunk_size)
+		return -EINVAL;
+
 	allocated_size = ALIGN(size, chunk_heap->chunk_size);
 	num_chunks = allocated_size / chunk_heap->chunk_size;
 
Signature · 1 line
1.8.3.2
#130 lore raw

PATCH 112/115 ion: fix bugs in cma heap

In reply to John Stultz

From: Colin Cross <[email protected]>

Implement ion_cma_unmap_kernel, ion will call it unconditionally. Use correct gfp flags when calling dma_alloc_coherent so it doesn't try to use atomic DMA memory.
Check for invalid alignment when allocating.
Reject cached allocations - the cpu address returned by dma_alloc_coherent is always going to be an uncached mapping, so map_kernel will not see data written by a cached userspace mapping.

---
 drivers/staging/android/ion/ion_cma_heap.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
drivers/staging/android/ion/ion_cma_heap.c +14 -1
index 3dcf9b4..f0f9889 100644
@@ -69,13 +69,20 @@ static int ion_cma_allocate(struct ion_heap *heap, struct ion_buffer *buffer,
 
 	dev_dbg(dev, "Request buffer allocation len %ld\n", len);
 
+	if (buffer->flags & ION_FLAG_CACHED)
+		return -EINVAL;
+
+	if (align > PAGE_SIZE)
+		return -EINVAL;
+
 	info = kzalloc(sizeof(struct ion_cma_buffer_info), GFP_KERNEL);
 	if (!info) {
 		dev_err(dev, "Can't allocate buffer info\n");
 		return ION_CMA_ALLOCATE_FAILED;
 	}
 
-	info->cpu_addr = dma_alloc_coherent(dev, len, &(info->handle), 0);
+	info->cpu_addr = dma_alloc_coherent(dev, len, &(info->handle),
+						GFP_HIGHUSER | __GFP_ZERO);
 
 	if (!info->cpu_addr) {
 		dev_err(dev, "Fail to allocate buffer\n");
@@ -170,6 +177,11 @@ static void *ion_cma_map_kernel(struct ion_heap *heap,
 	return info->cpu_addr;
 }
 
+static void ion_cma_unmap_kernel(struct ion_heap *heap,
+					struct ion_buffer *buffer)
+{
+}
+
 static struct ion_heap_ops ion_cma_ops = {
 	.allocate = ion_cma_allocate,
 	.free = ion_cma_free,
@@ -178,6 +190,7 @@ static struct ion_heap_ops ion_cma_ops = {
 	.phys = ion_cma_phys,
 	.map_user = ion_cma_mmap,
 	.map_kernel = ion_cma_map_kernel,
+	.unmap_kernel = ion_cma_unmap_kernel,
 };
 
 struct ion_heap *ion_cma_heap_create(struct ion_platform_heap *data)
Signature · 1 line
1.8.3.2
#131 lore raw

PATCH 113/115 ion: Cleanup whitespace issues and other checkpatch problems

In reply to John Stultz

Just some simple cleanups to address whitespace issues and other issues found w/ checkpatch.

---
 drivers/staging/android/ion/compat_ion.c      |  4 +-
 drivers/staging/android/ion/ion.c             | 53 ++++++++++++++-------------
 drivers/staging/android/ion/ion.h             |  2 +-
 drivers/staging/android/ion/ion_chunk_heap.c  |  1 -
 drivers/staging/android/ion/ion_heap.c        |  5 +--
 drivers/staging/android/ion/ion_page_pool.c   |  2 +-
 drivers/staging/android/ion/ion_priv.h        |  2 +-
 drivers/staging/android/ion/ion_system_heap.c |  3 +-
 8 files changed, 37 insertions(+), 35 deletions(-)
drivers/staging/android/ion/compat_ion.c +2 -2
index 079a5bb..af6cd37 100644
@@ -91,8 +91,8 @@ static int compat_get_ion_custom_data(
 			struct compat_ion_custom_data __user *data32,
 			struct ion_custom_data __user *data)
 {
-        compat_uint_t cmd;
-        compat_ulong_t arg;
+	compat_uint_t cmd;
+	compat_ulong_t arg;
 	int err;
 
 	err = get_user(cmd, &data32->cmd);
drivers/staging/android/ion/ion.c +28 -25

96 lines not on this page · raw message

drivers/staging/android/ion/ion.h +1 -1
index 962e1c5..dcd2a0c 100644
@@ -38,7 +38,7 @@ struct ion_buffer;
  * struct ion_platform_heap - defines a heap in the given platform
  * @type:	type of the heap from ion_heap_type enum
  * @id:		unique identifier for heap.  When allocating higher numbers
- * 		will be allocated from first.  At allocation these are passed
+ *		will be allocated from first.  At allocation these are passed
  *		as a bit mask and therefore can not exceed ION_NUM_HEAP_IDS.
  * @name:	used for debug purposes
  * @base:	base address of heap in physical memory if applicable
drivers/staging/android/ion/ion_chunk_heap.c +0 -1
index a072089..d40f5f8 100644
@@ -13,7 +13,6 @@
  * GNU General Public License for more details.
  *
  */
-//#include <linux/spinlock.h>
 #include <linux/dma-mapping.h>
 #include <linux/err.h>
 #include <linux/genalloc.h>
drivers/staging/android/ion/ion_heap.c +2 -3
index 2d1d555..9cf5622 100644
@@ -49,9 +49,8 @@ void *ion_heap_map_kernel(struct ion_heap *heap,
 		int npages_this_entry = PAGE_ALIGN(sg->length) / PAGE_SIZE;
 		struct page *page = sg_page(sg);
 		BUG_ON(i >= npages);
-		for (j = 0; j < npages_this_entry; j++) {
+		for (j = 0; j < npages_this_entry; j++)
 			*(tmp++) = page++;
-		}
 	}
 	vaddr = vmap(pages, npages, VM_MAP, pgprot);
 	vfree(pages);
@@ -159,7 +158,7 @@ int ion_heap_pages_zero(struct page *page, size_t size, pgprot_t pgprot)
 	return ion_heap_sglist_zero(&sg, 1, pgprot);
 }
 
-void ion_heap_freelist_add(struct ion_heap *heap, struct ion_buffer * buffer)
+void ion_heap_freelist_add(struct ion_heap *heap, struct ion_buffer *buffer)
 {
 	rt_mutex_lock(&heap->lock);
 	list_add(&buffer->list, &heap->free_list);
drivers/staging/android/ion/ion_page_pool.c +1 -1
index a1c51a8..fa693c2 100644
@@ -108,7 +108,7 @@ void *ion_page_pool_alloc(struct ion_page_pool *pool)
 	return page;
 }
 
-void ion_page_pool_free(struct ion_page_pool *pool, struct page* page)
+void ion_page_pool_free(struct ion_page_pool *pool, struct page *page)
 {
 	int ret;
 
drivers/staging/android/ion/ion_priv.h +1 -1
index fa96175..fc8a4c3 100644
@@ -230,7 +230,7 @@ int ion_heap_init_deferred_free(struct ion_heap *heap);
 /**
  * ion_heap_freelist_add - add a buffer to the deferred free list
  * @heap:		the heap
- * @buffer: 		the buffer
+ * @buffer:		the buffer
  *
  * Adds an item to the deferred freelist.
  */
drivers/staging/android/ion/ion_system_heap.c +2 -1
index 301d019..b5acd6c 100644
@@ -148,7 +148,8 @@ static int ion_system_heap_allocate(struct ion_heap *heap,
 
 	INIT_LIST_HEAD(&pages);
 	while (size_remaining > 0) {
-		info = alloc_largest_available(sys_heap, buffer, size_remaining, max_order);
+		info = alloc_largest_available(sys_heap, buffer, size_remaining,
+						max_order);
 		if (!info)
 			goto err;
 		list_add_tail(&info->list, &pages);
Signature · 1 line
1.8.3.2
#132 lore raw

PATCH 114/115 ion: Improve ION config description

In reply to John Stultz

Mostly just to quiet checkpatch warnings, be more verbose in describing the ION config option.

---
 drivers/staging/android/ion/Kconfig | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/staging/android/ion/Kconfig +6 -2
index 3215bd8..a342d96 100644
@@ -2,8 +2,12 @@ menuconfig ION
 	tristate "Ion Memory Manager"
 	select GENERIC_ALLOCATOR
 	select DMA_SHARED_BUFFER
-	help
-	  Chose this option to enable the ION Memory Manager.
+	---help---
+	  Chose this option to enable the ION Memory Manager,
+	  used by Android to efficiently allocate buffers
+	  from userspace that can be shared between drivers.
+	  If you're not using Android its probably safe to
+	  say N here.
 
 config ION_TEST
 	tristate "Ion Test Device"
Signature · 1 line
1.8.3.2
#133 lore raw

PATCH 115/115 ion: Update system heap shrinker to use the new count/scan interface

In reply to John Stultz

Update the ION system heap shrinker to use the new count/scan interfaces that landed in 3.12

---
 drivers/staging/android/ion/ion_system_heap.c | 40 +++++++++++++++++++--------
 1 file changed, 28 insertions(+), 12 deletions(-)
drivers/staging/android/ion/ion_system_heap.c +28 -12
index b5acd6c..144b2272 100644
@@ -232,15 +232,37 @@ static struct ion_heap_ops system_heap_ops = {
 	.map_user = ion_heap_map_user,
 };
 
-static int ion_system_heap_shrink(struct shrinker *shrinker,
-				  struct shrink_control *sc) {
-
+static unsigned long ion_system_heap_shrink_count(struct shrinker *shrinker,
+				  struct shrink_control *sc)
+{
 	struct ion_heap *heap = container_of(shrinker, struct ion_heap,
 					     shrinker);
 	struct ion_system_heap *sys_heap = container_of(heap,
 							struct ion_system_heap,
 							heap);
 	int nr_total = 0;
+	int i;
+
+	/* total number of items is whatever the page pools are holding
+	   plus whatever's in the freelist */
+	for (i = 0; i < num_orders; i++) {
+		struct ion_page_pool *pool = sys_heap->pools[i];
+		nr_total += ion_page_pool_shrink(pool, sc->gfp_mask, 0);
+	}
+	nr_total += ion_heap_freelist_size(heap) / PAGE_SIZE;
+	return nr_total;
+
+}
+
+static unsigned long ion_system_heap_shrink_scan(struct shrinker *shrinker,
+				  struct shrink_control *sc)
+{
+
+	struct ion_heap *heap = container_of(shrinker, struct ion_heap,
+					     shrinker);
+	struct ion_system_heap *sys_heap = container_of(heap,
+							struct ion_system_heap,
+							heap);
 	int nr_freed = 0;
 	int i;
 
@@ -265,14 +287,7 @@ static int ion_system_heap_shrink(struct shrinker *shrinker,
 	}
 
 end:
-	/* total number of items is whatever the page pools are holding
-	   plus whatever's in the freelist */
-	for (i = 0; i < num_orders; i++) {
-		struct ion_page_pool *pool = sys_heap->pools[i];
-		nr_total += ion_page_pool_shrink(pool, sc->gfp_mask, 0);
-	}
-	nr_total += ion_heap_freelist_size(heap) / PAGE_SIZE;
-	return nr_total;
+	return nr_freed;
 
 }
 
@@ -323,7 +338,8 @@ struct ion_heap *ion_system_heap_create(struct ion_platform_heap *unused)
 		heap->pools[i] = pool;
 	}
 
-	heap->heap.shrinker.shrink = ion_system_heap_shrink;
+	heap->heap.shrinker.scan_objects = ion_system_heap_shrink_scan;
+	heap->heap.shrinker.count_objects = ion_system_heap_shrink_count;
 	heap->heap.shrinker.seeks = DEFAULT_SEEKS;
 	heap->heap.shrinker.batch = 0;
 	register_shrinker(&heap->heap.shrinker);
Signature · 1 line
1.8.3.2