TTM page pool allocator
Subsystems (3)
Hi,
Thomas i attach a reworked page pool allocator based on Dave works, this one should be ok with ttm cache status tracking. It definitely helps on AGP system, now the bottleneck is in mesa vertex's dma allocation.
Cheers,
Jerome
Quoted 1 line
From
d2581e2a97e9c694fd1238f1a5652e1f385b5636Mon Sep 17 00:00:00 2001
From: Dave Airlie <[email protected]>
Date: Wed, 24 Jun 2009 16:31:43 +1000
Subject: [PATCH] ttm: add pool wc/uc page allocator
On AGP system we might allocate/free routinely uncached or wc memory, changing page from cached (wb) to uc or wc is very expensive and involves a lot of flushing. To improve performance this allocator use a pool of uc,wc pages.
Currently each pool (wc, uc) is 256 pages big, improvement would be to tweak this according to memory pressure so we can give back memory to system.
--- drivers/gpu/drm/ttm/Makefile | 2 +- drivers/gpu/drm/ttm/ttm_memory.c | 3 + drivers/gpu/drm/ttm/ttm_page_alloc.c | 255 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/ttm/ttm_page_alloc.h | 43 ++++++ drivers/gpu/drm/ttm/ttm_tt.c | 42 +++++- include/drm/ttm/ttm_bo_driver.h | 6 + 6 files changed, 342 insertions(+), 9 deletions(-) create mode 100644 drivers/gpu/drm/ttm/ttm_page_alloc.c create mode 100644 drivers/gpu/drm/ttm/ttm_page_alloc.h
drivers/gpu/drm/ttm/Makefile +1 -1
@@ -3,6 +3,6 @@ ccflags-y := -Iinclude/drm ttm-y := ttm_agp_backend.o ttm_memory.o ttm_tt.o ttm_bo.o \ - ttm_bo_util.o ttm_bo_vm.o ttm_module.o ttm_global.o + ttm_bo_util.o ttm_bo_vm.o ttm_module.o ttm_global.o ttm_page_alloc.o obj-$(CONFIG_DRM_TTM) += ttm.o
drivers/gpu/drm/ttm/ttm_memory.c +3 -0
@@ -32,6 +32,7 @@#include <linux/mm.h> #include <linux/module.h> +#include "ttm_page_alloc.h" #define TTM_PFX "[TTM] "#define TTM_MEMORY_ALLOC_RETRIES 4
@@ -124,6 +125,7 @@ int ttm_mem_global_init(struct ttm_mem_global *glob) printk(KERN_INFO TTM_PFX "TTM available object memory: %llu MiB\n", glob->max_memory >> 20); +ttm_page_alloc_init ();return 0 ; }EXPORT_SYMBOL (ttm_mem_global_init);
@@ -135,6 +137,7 @@ void ttm_mem_global_release(struct ttm_mem_global *glob)flush_workqueue (glob->swap_queue);destroy_workqueue (glob->swap_queue); glob->swap_queue =NULL ; +ttm_page_alloc_fini (); }EXPORT_SYMBOL (ttm_mem_global_release);
drivers/gpu/drm/ttm/ttm_page_alloc.c new file +255 -0
255 lines not on this page · raw message
drivers/gpu/drm/ttm/ttm_page_alloc.h new file +43 -0
43 lines not on this page · raw message
drivers/gpu/drm/ttm/ttm_tt.c +34 -8
81 lines not on this page · raw message
include/drm/ttm/ttm_bo_driver.h +6 -0
@@ -121,6 +121,12 @@ struct ttm_backend {#define TTM_PAGE_FLAG_SWAPPED (1 << 4)#define TTM_PAGE_FLAG_PERSISTANT_SWAP (1 << 5)#define TTM_PAGE_FLAG_ZERO_ALLOC (1 << 6) +#define TTM_PAGE_FLAG_DMA32 (1 << 7) +#define TTM_PAGE_FLAG_CACHE_MASK (3 << 8) +#define TTM_PAGE_FLAG_CACHE_SHIFT 8 +#define TTM_PAGE_FLAG_CACHE_UC 0 +#define TTM_PAGE_FLAG_CACHE_WC 1 +#define TTM_PAGE_FLAG_CACHE_WB 2enum ttm_caching_state { tt_uncached,
Signature · 1 line
1.6.2.2
Jerome Glisse skrev:
Quoted 10 lines
Hi,
Thomas i attach a reworked page pool allocator based on Dave works, this one should be ok with ttm cache status tracking. It definitely helps on AGP system, now the bottleneck is in mesa vertex's dma allocation.
Cheers,
Jerome
Hi, Jerome!
In general it looks very good. Some things that need fixing:
1) We must have a way to hand back pages. I still not quite understand how the shrink callbacks work and whether they are applicable. Another scheme would be to free, say 1MB when we have at least 2MB available.
2) We should avoid including AGP headers if AGP is not configured. Either reimplement unmap_page_from_agp or map_page_into_agp or move them out from the AGP headers. We've hade complaints before from people with AGP free systems that the code doesn't compile.
3) Since we're allocating (and freeing) in batches we should use the set_pages_array() interface to avoid a global tlb flush per page.
4) We could now skip the ttm_tt_populate() in ttm_tt_set_caching, since it will always allocate cached pages and then transition them.
5) Use TTM_PFX in printouts.
/Thomas
Quoted 23 lines · Thomas Hellström
Jerome Glisse skrev:
Quoted 10 lines
Hi,
Thomas i attach a reworked page pool allocator based on Dave works, this one should be ok with ttm cache status tracking. It definitely helps on AGP system, now the bottleneck is in mesa vertex's dma allocation.
Cheers,
JeromeHi, Jerome!
In general it looks very good. Some things that need fixing:1) We must have a way to hand back pages. I still not quite understand how the shrink callbacks work and whether they are applicable. Another scheme would be to free, say 1MB when we have at least 2MB available.
2) We should avoid including AGP headers if AGP is not configured. Either reimplement
unmap_page_from_agpormap_page_into_agpor move them out from the AGP headers. We've hade complaints before from people with AGP free systems that the code doesn't compile.
Quoted 6 lines
3) Since we're allocating (and freeing) in batches we should use the
set_pages_array()interface to avoid a global tlb flush per page.4) We could now skip the
ttm_tt_populate()inttm_tt_set_caching, since it will always allocate cached pages and then transition them.
Okay 4) is bad, what happens (my brain is a bit meltdown so i might be wrong) :
1 - bo get allocated tt->state = unpopulated
2 - bo is mapped few page are faulted tt->state = unpopulated 3 - bo is cache transitioned but tt->state == unpopulated but
they are page which have been touch by the cpu so we need to clflush them and transition them, this never happen if we don't call ttm_tt_populate and proceed with the remaining of the cache transitioning functions
As a workaround i will try to go through the pages tables and transition existing pages. Do you have any idea for a better plan ?
Cheers,
Jerome
Quoted 16 lines · Jerome Glisse
Quoted 4 lines · Thomas Hellström
4) We could now skip the
ttm_tt_populate()inttm_tt_set_caching, since it will always allocate cached pages and then transition them.Okay 4) is bad, what happens (my brain is a bit meltdown so i might be wrong) :
1 - bo get allocatedtt->state= unpopulated
2 - bo is mapped few page are faultedtt->state= unpopulated 3 - bo is cache transitioned buttt->state== unpopulated but
they are page which have been touch by the cpu so we need to clflush them and transition them, this never happen if we don't callttm_tt_populateand proceed with the remaining of the cache transitioning functions
Quoted 6 lines
As a workaround i will try to go through the pages tables and transition existing pages. Do you have any idea for a better plan ?
Cheers,
Jerome
My workaround ruin the whole idea of pool allocation what happens is that most bo get cache transition page per page. My thinking is that we should do the following:
- is there is a least one page allocated then fully populate the object and do cache transition on all the pages.
- otherwise update caching_state and leaves object unpopulated
This needs that we some how reflect the fact that there is at least one page allocated, i am thinking to adding a new state for that :ttm_partialy_populated
Thomas what do you think about that ?
Cheers,
Jerome
Quoted 31 lines · Jerome Glisse
Quoted 16 lines · Jerome Glisse
Quoted 4 lines · Thomas Hellström
4) We could now skip the
ttm_tt_populate()inttm_tt_set_caching, since it will always allocate cached pages and then transition them.Okay 4) is bad, what happens (my brain is a bit meltdown so i might be wrong) :
1 - bo get allocatedtt->state= unpopulated
2 - bo is mapped few page are faultedtt->state= unpopulated 3 - bo is cache transitioned buttt->state== unpopulated but
they are page which have been touch by the cpu so we need to clflush them and transition them, this never happen if we don't callttm_tt_populateand proceed with the remaining of the cache transitioning functionsQuoted 6 lines
As a workaround i will try to go through the pages tables and transition existing pages. Do you have any idea for a better plan ?
Cheers,
JeromeMy workaround ruin the whole idea of pool allocation what happens is that most bo get cache transition page per page. My thinking is that we should do the following:
- is there is a least one page allocated then fully populate the object and do cache transition on all the pages.
- otherwise updatecaching_stateand leaves object unpopulated
Quoted 8 lines
This needs that we some how reflect the fact that there is at least one page allocated, i am thinking to adding a new state for that :
ttm_partialy_populatedThomas what do you think about that ?
Cheers,
Jerome
Attached updated patch it doesn't introduce ttm_partialy_populated but keep the populate call in cache transition. So far it seems to work properly on AGP platform and helps quite a lot with performances. I wonder if i should rather allocate some memory to store the pool structure in ttm_page_pool_init rather than having quite a lot of static variables ? Anyone has thought on that ?
Cheers,
Jerome
Quoted 1 line
From
35cbb15ca10e2ed8ba215298268f2ead8d13e759Mon Sep 17 00:00:00 2001
From: Jerome Glisse <[email protected]>
Date: Thu, 16 Jul 2009 18:12:22 +0200
Subject: [PATCH] ttm: add pool wc/uc page allocator
On AGP system we might allocate/free routinely uncached or wc memory, changing page from cached (wb) to uc or wc is very expensive and involves a lot of flushing. To improve performance this allocator use a pool of uc,wc pages.
Currently each pool (wc, uc) is 256 pages big, improvement would be to tweak this according to memory pressure so we can give back memory to system.
--- drivers/gpu/drm/ttm/Makefile | 2 +- drivers/gpu/drm/ttm/ttm_memory.c | 3 + drivers/gpu/drm/ttm/ttm_page_alloc.c | 354 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/ttm/ttm_page_alloc.h | 36 ++++ drivers/gpu/drm/ttm/ttm_tt.c | 37 ++--- 5 files changed, 407 insertions(+), 25 deletions(-) create mode 100644 drivers/gpu/drm/ttm/ttm_page_alloc.c create mode 100644 drivers/gpu/drm/ttm/ttm_page_alloc.h
drivers/gpu/drm/ttm/Makefile +1 -1
@@ -3,6 +3,6 @@ ccflags-y := -Iinclude/drm ttm-y := ttm_agp_backend.o ttm_memory.o ttm_tt.o ttm_bo.o \ - ttm_bo_util.o ttm_bo_vm.o ttm_module.o ttm_global.o + ttm_bo_util.o ttm_bo_vm.o ttm_module.o ttm_global.o ttm_page_alloc.o obj-$(CONFIG_DRM_TTM) += ttm.o
drivers/gpu/drm/ttm/ttm_memory.c +3 -0
@@ -32,6 +32,7 @@#include <linux/mm.h> #include <linux/module.h> +#include "ttm_page_alloc.h" #define TTM_PFX "[TTM] "#define TTM_MEMORY_ALLOC_RETRIES 4
@@ -124,6 +125,7 @@ int ttm_mem_global_init(struct ttm_mem_global *glob) printk(KERN_INFO TTM_PFX "TTM available object memory: %llu MiB\n", glob->max_memory >> 20); +ttm_page_alloc_init ();return 0 ; }EXPORT_SYMBOL (ttm_mem_global_init);
@@ -135,6 +137,7 @@ void ttm_mem_global_release(struct ttm_mem_global *glob)flush_workqueue (glob->swap_queue);destroy_workqueue (glob->swap_queue); glob->swap_queue =NULL ; +ttm_page_alloc_fini (); }EXPORT_SYMBOL (ttm_mem_global_release);
drivers/gpu/drm/ttm/ttm_page_alloc.c new file +354 -0
354 lines not on this page · raw message
drivers/gpu/drm/ttm/ttm_page_alloc.h new file +36 -0
@@ -0,0 +1,36 @@ +/* +* Copyright (c) Red Hat Inc. + +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sub license, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the +* next paragraph) shall be included in all copies or substantial portions +* of the Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +* DEALINGS IN THE SOFTWARE. +* +* Authors: Dave Airlie <[email protected]> +* Jerome Glisse <[email protected]> +*/ +#ifndef TTM_PAGE_ALLOC +#define TTM_PAGE_ALLOC + +#include "ttm/ttm_bo_driver.h" + +void ttm_put_page (struct page *page,int flags,enum ttm_caching_state cstate); +struct page *ttm_get_page (int flags,enum ttm_caching_state cstate); +int ttm_page_alloc_init (void ); +void ttm_page_alloc_fini (void ); + +#endif
drivers/gpu/drm/ttm/ttm_tt.c +13 -24
@@ -38,6 +38,7 @@#include "ttm/ttm_module.h" #include "ttm/ttm_bo_driver.h" #include "ttm/ttm_placement.h" +#include "ttm_page_alloc.h" static int ttm_tt_swapin (struct ttm_tt *ttm);
@@ -72,21 +73,6 @@ static void ttm_tt_free_page_directory(struct ttm_tt *ttm) ttm->pages =NULL ; } -static struct page *ttm_tt_alloc_page (unsigned page_flags) -{ -gfp_t gfp_flags =GFP_USER ; - -if (page_flags &TTM_PAGE_FLAG_ZERO_ALLOC ) - gfp_flags |= __GFP_ZERO; - -if (page_flags &TTM_PAGE_FLAG_DMA32 ) - gfp_flags |= __GFP_DMA32; -else - gfp_flags |= __GFP_HIGHMEM; - -return alloc_page (gfp_flags); -} -static void ttm_tt_free_user_pages (struct ttm_tt *ttm) {int write;
@@ -132,7 +118,7 @@ static struct page *__ttm_tt_get_page(struct ttm_tt *ttm, int index)int ret;while (NULL == (p = ttm->pages[index])) { - p =ttm_tt_alloc_page (ttm->page_flags); + p =ttm_get_page (ttm->page_flags, ttm->caching_state);if (!p)return NULL ;
@@ -155,7 +141,8 @@ static struct page *__ttm_tt_get_page(struct ttm_tt *ttm, int index) }return p; out_err: -put_page (p); +if (p) +ttm_put_page (p, ttm->page_flags, ttm->caching_state);return NULL ; }
@@ -229,7 +216,6 @@ static inline int ttm_tt_set_page_caching(struct page *p,* Change caching policy for the linear kernel map * for range of pages in a ttm. */ -static int ttm_tt_set_caching (struct ttm_tt *ttm,enum ttm_caching_state c_state) {
@@ -246,6 +232,12 @@ static int ttm_tt_set_caching(struct ttm_tt *ttm,return ret; } +if (ttm->state == tt_unpopulated) { +/* Change caching but don't populate */ + ttm->caching_state = c_state; +return 0 ; + } +if (ttm->caching_state == tt_cached)drm_clflush_pages (ttm->pages, ttm->num_pages);
@@ -257,11 +249,8 @@ static int ttm_tt_set_caching(struct ttm_tt *ttm, goto out_err; } } - ttm->caching_state = c_state; -return 0 ; - out_err:for (j =0 ; j < i; ++j) { cur_page = ttm->pages[j];
@@ -296,7 +285,6 @@ static void ttm_tt_free_alloced_pages(struct ttm_tt *ttm)if (be) be->func->clear (be); - (void )ttm_tt_set_caching (ttm, tt_cached);for (i =0 ; i < ttm->num_pages; ++i) { cur_page = ttm->pages[i]; ttm->pages[i] =NULL ;
@@ -304,10 +292,11 @@ static void ttm_tt_free_alloced_pages(struct ttm_tt *ttm)if (page_count (cur_page) !=1 ) printk(KERN_ERR TTM_PFX"Erroneous page count. " -"Leaking pages.\n" ); +"Leaking pages (%d).\n" , +page_count (cur_page));ttm_mem_global_free (ttm->bdev->mem_glob,PAGE_SIZE ,PageHighMem (cur_page)); -__free_page (cur_page); +ttm_put_page (cur_page, ttm->page_flags, ttm->caching_state); } } ttm->state = tt_unpopulated;
Signature · 1 line
1.6.2.2
Quoted 43 lines · Jerome Glisse
Quoted 31 lines · Jerome Glisse
Quoted 15 lines · Jerome Glisse
Quoted 3 lines · Thomas Hellström
4) We could now skip the
ttm_tt_populate()inttm_tt_set_caching, since it will always allocate cached pages and then transition them.Okay 4) is bad, what happens (my brain is a bit meltdown so i might be wrong) :
1 - bo get allocatedtt->state= unpopulated
2 - bo is mapped few page are faultedtt->state= unpopulated 3 - bo is cache transitioned buttt->state== unpopulated but
they are page which have been touch by the cpu so we need to clflush them and transition them, this never happen if we don't callttm_tt_populateand proceed with the remaining of the cache transitioning functionsQuoted 7 lines
As a workaround i will try to go through the pages tables and transition existing pages. Do you have any idea for a better plan ?
Cheers,
JeromeMy workaround ruin the whole idea of pool allocation what happens is that most bo get cache transition page per page. My thinking is that we should do the following:
- is there is a least one page allocated then fully populate the object and do cache transition on all the pages.
- otherwise updatecaching_stateand leaves object unpopulatedQuoted 9 lines
This needs that we some how reflect the fact that there is at least one page allocated, i am thinking to adding a new state for that :
ttm_partialy_populatedThomas what do you think about that ?
Cheers,
Jerome
Quoted 8 lines
Attached updated patch it doesn't introduce
ttm_partialy_populatedbut keep the populate call in cache transition. So far it seems to work properly on AGP platform and helps quite a lot with performances. I wonder if i should rather allocate some memory to store the pool structure inttm_page_pool_initrather than having quite a lot of static variables ? Anyone has thought on that ?
Jerome,
TTM has a device struct per device and an optional global struct that is common for all devices and intended to be per subsystem.
The only subsystem currently having a global structure is the memory accounting subsystem:
struct ttm_mem_global
You can either put the global stuff there or create and register a separate global struct for the page pool.
I'll probably also later add a global structure for the bo subsystem, since we need a common buffer object memory footprint shrinker for multiple cards.
/Thomas
Quoted 3 lines
Cheers,
Jerome
ttm_mem_global
Quoted 6 lines · Thomas Hellström
TTM has a device struct per device and an optional global struct that is common for all devices and intended to be per subsystem.
The only subsystem currently having a global structure is the memory accounting subsystem:
structttm_mem_global
Thomas i don't think the way we init ttm_mem_global today make it follow the 1 struct ttm_mem_global for everyone. I think it should be initialized and refcounted by device struct.
So on first device creation a ttm_mem_global is created and then anytime a new device is created the refcount of ttm_mem_global is increased. This would mean some static global refcount inside ttm_memory.c, maybe there is something similar to singleton in the linux toolbox.
Thought ? Idea ?
Cheers,
Jerome
Re: ttm_mem_global
Jerome Glisse skrev:
Quoted 8 lines · Thomas Hellström
Quoted 6 lines
TTM has a device struct per device and an optional global struct that is common for all devices and intended to be per subsystem.
The only subsystem currently having a global structure is the memory accounting subsystem:
structttm_mem_global
Quoted 7 lines
Thomas i don't think the way we init
ttm_mem_globaltoday make it follow the 1 structttm_mem_globalfor everyone. I think it should be initialized and refcounted by device struct.So on first device creation a
ttm_mem_globalis created and then anytime a new device is created the refcount ofttm_mem_globalis increased.
Jerome,
This is exactly what the current code intends to do.
Are you seeing something different?
/Thomas
Re: ttm_mem_global
Quoted 15 lines · Thomas Hellström
Jerome Glisse skrev:
Quoted 14 lines · Thomas Hellström
Quoted 8 lines
TTM has a device struct per device and an optional global struct that is common for all devices and intended to be per subsystem.
The only subsystem currently having a global structure is the memory accounting subsystem:
structttm_mem_globalThomas i don't think the way we init
ttm_mem_globaltoday make it follow the 1 structttm_mem_globalfor everyone. I think it should be initialized and refcounted by device struct.
Quoted 7 lines
Quoted 3 lines
So on first device creation a
ttm_mem_globalis created and then anytime a new device is created the refcount ofttm_mem_globalis increased.Jerome,
This is exactly what the current code intends to do.Are you seeing something different?
I definitly don't see that :) In radeon we do create a structure which hold the ttm_mem_global struct so it's not shared at all it got inited & destroyed along the driver. This is why i think it's better to remove the driver initialization and let bo_device init path take care of initializing one and only one object which can be shared by multiple driverttm_mem_global_inits.
So what i propose is remove mem_glob parameter from :ttm_bo_device_init, add a call to ttm_mem_global_init in ttm_bo_device_init and add some static refcount in ttm_memory.c if refcount = 0 then ttm_mem_global_init create a ttm_mem_global struct and initialize things, if refcount > 0 then it gives back the already initialized ttm_mem_global.
Of course we unref with ttm_mem_global_release and destroy once refcount reach 0.
Cheers,
Jerome
Re: ttm_mem_global
Quoted 27 lines · Jerome Glisse
Quoted 15 lines · Thomas Hellström
Jerome Glisse skrev:
Quoted 13 lines · Thomas Hellström
Quoted 8 lines
TTM has a device struct per device and an optional global struct that is common for all devices and intended to be per subsystem.
The only subsystem currently having a global structure is the memory accounting subsystem:
structttm_mem_globalThomas i don't think the way we init
ttm_mem_globaltoday make it follow the 1 structttm_mem_globalfor everyone. I think it should be initialized and refcounted by device struct.Quoted 9 lines
Quoted 4 lines
So on first device creation a
ttm_mem_globalis created and then anytime a new device is created the refcount ofttm_mem_globalis increased.Jerome,
This is exactly what the current code intends to do.Are you seeing something different?
Quoted 8 lines
I definitly don't see that :) In radeon we do create a structure which hold the
ttm_mem_globalstruct so it's not shared at all it got inited & destroyed along the driver. This is why i think it's better to remove the driver initialization and letbo_deviceinit path take care of initializing one and only one object which can be shared by multipledriverttm_mem_global_inits.
Which radeon struct is holding the ttm_mem_global struct?
The radeon code looks very similar to the openchrome code in which the struct ttm_mem_global is allocated at ttm_global.c, line 74 and freed at ttm_global.c, line 108 when its refcount has reached zero.
So the device holds a struct ttm_global_reference that *only points* to the global item, and which is destroyed on device takedown. If there are more than one device pointing to the mem_global object, it won't get destroyed.
So the code should be working perfectly fine unless there is a bug.
Quoted 3 lines
So what i propose is remove
mem_globparameter from :ttm_bo_device_init, add a call tottm_mem_global_initinttm_bo_device_init
Nope, The ttm_mem_global object is used by other ttm subsystems (fencing, user-space objects), so that can't be done.
Quoted 6 lines
and add some static refcount in
ttm_memory.c
if refcount = 0 thenttm_mem_global_initcreate attm_mem_globalstruct and initialize things, if refcount > 0 then it gives back the already initializedttm_mem_global.
This is exactly what ttm_global was created to do, and what it hopefully does. If you create two radeon devices the ttm_mem_global object should be the same, even though the global references pointing to it are of course different. Have you actually tried this?
/Thomas
Quoted 7 lines
Of course we unref with
ttm_mem_global_releaseand destroy once refcount reach 0.Cheers,
Jerome
Re: ttm_mem_global
Quoted 66 lines · Thomas Hellström
Quoted 29 lines · Jerome Glisse
Quoted 17 lines · Thomas Hellström
Jerome Glisse skrev:
Quoted 15 lines · Thomas Hellström
Quoted 10 lines
TTM has a device struct per device and an optional global struct that is common for all devices and intended to be per subsystem.
The only subsystem currently having a global structure is the memory accounting subsystem:
structttm_mem_globalThomas i don't think the way we init
ttm_mem_globaltoday make it follow the 1 structttm_mem_globalfor everyone. I think it should be initialized and refcounted by device struct.Quoted 9 lines
Quoted 4 lines
So on first device creation a
ttm_mem_globalis created and then anytime a new device is created the refcount ofttm_mem_globalis increased.Jerome,
This is exactly what the current code intends to do.Are you seeing something different?
Quoted 8 lines
I definitly don't see that :) In radeon we do create a structure which hold the
ttm_mem_globalstruct so it's not shared at all it got inited & destroyed along the driver. This is why i think it's better to remove the driver initialization and letbo_deviceinit path take care of initializing one and only one object which can be shared by multipledriverttm_mem_global_inits.Which radeon struct is holding the
ttm_mem_globalstruct?The radeon code looks very similar to the openchrome code in which the struct
ttm_mem_globalis allocated atttm_global.c, line 74 and freed atttm_global.c, line 108 when its refcount has reached zero.So the device holds a struct
ttm_global_referencethat *only points* to the global item, and which is destroyed on device takedown. If there are more than one device pointing to themem_globalobject, it won't get destroyed.So the code should be working perfectly fine unless there is a bug.
Quoted 3 lines
So what i propose is remove
mem_globparameter from :ttm_bo_device_init, add a call tottm_mem_global_initinttm_bo_device_initNope, The
ttm_mem_globalobject is used by other ttm subsystems (fencing, user-space objects),
so that can't be done.Quoted 6 lines
and add some static refcount in
ttm_memory.c
if refcount = 0 thenttm_mem_global_initcreate attm_mem_globalstruct and initialize things, if refcount > 0 then it gives back the already initializedttm_mem_global.
Quoted 7 lines
This is exactly what
ttm_globalwas created to do, and what it hopefully does. If you create two radeon devices thettm_mem_globalobject should be the same, even though the global references pointing to it are of course different. Have you actually tried this?/Thomas
Ok code wasn't clear for me until i read ttm_global.c
Cheers,
Jerome
Quoted 36 lines · Jerome Glisse
Quoted 35 lines · Jerome Glisse
Quoted 16 lines · Jerome Glisse
Quoted 4 lines · Thomas Hellström
4) We could now skip the
ttm_tt_populate()inttm_tt_set_caching, since it will always allocate cached pages and then transition them.Okay 4) is bad, what happens (my brain is a bit meltdown so i might be wrong) :
1 - bo get allocatedtt->state= unpopulated
2 - bo is mapped few page are faultedtt->state= unpopulated 3 - bo is cache transitioned buttt->state== unpopulated but
they are page which have been touch by the cpu so we need to clflush them and transition them, this never happen if we don't callttm_tt_populateand proceed with the remaining of the cache transitioning functionsQuoted 6 lines
As a workaround i will try to go through the pages tables and transition existing pages. Do you have any idea for a better plan ?
Cheers,
JeromeMy workaround ruin the whole idea of pool allocation what happens is that most bo get cache transition page per page. My thinking is that we should do the following:
- is there is a least one page allocated then fully populate the object and do cache transition on all the pages.
- otherwise updatecaching_stateand leaves object unpopulatedThis needs that we some how reflect the fact that there is at least one page allocated, i am thinking to adding a new state for that :
ttm_partialy_populated
Quoted 8 lines
Quoted 4 lines
Thomas what do you think about that ?
Cheers,
JeromeAttached updated patch it doesn't introduce
ttm_partialy_populatedbut keep the populate call in cache transition. So far it seems to work properly on AGP platform
Yeah, this one works for me as well.
Quoted 1 line
and helps quite a lot with performances.
Can't say I've noticed that however. How did you measure?
Signature · 2 lines
Earthling Michel Dänzer | http://www.vmware.com Libre software enthusiast | Debian, X and DRI developer
Quoted 46 lines · Michel Dänzer
Quoted 36 lines · Jerome Glisse
Quoted 35 lines · Jerome Glisse
Quoted 16 lines · Jerome Glisse
Quoted 4 lines · Thomas Hellström
4) We could now skip the
ttm_tt_populate()inttm_tt_set_caching, since it will always allocate cached pages and then transition them.Okay 4) is bad, what happens (my brain is a bit meltdown so i might be wrong) :
1 - bo get allocatedtt->state= unpopulated
2 - bo is mapped few page are faultedtt->state= unpopulated 3 - bo is cache transitioned buttt->state== unpopulated but
they are page which have been touch by the cpu so we need to clflush them and transition them, this never happen if we don't callttm_tt_populateand proceed with the remaining of the cache transitioning functionsQuoted 6 lines
As a workaround i will try to go through the pages tables and transition existing pages. Do you have any idea for a better plan ?
Cheers,
JeromeMy workaround ruin the whole idea of pool allocation what happens is that most bo get cache transition page per page. My thinking is that we should do the following:
- is there is a least one page allocated then fully populate the object and do cache transition on all the pages.
- otherwise updatecaching_stateand leaves object unpopulatedThis needs that we some how reflect the fact that there is at least one page allocated, i am thinking to adding a new state for that :
ttm_partialy_populatedQuoted 8 lines
Quoted 4 lines
Thomas what do you think about that ?
Cheers,
JeromeAttached updated patch it doesn't introduce
ttm_partialy_populatedbut keep the populate call in cache transition. So far it seems to work properly on AGP platform
Quoted 5 lines
Yeah, this one works for me as well.
Quoted 1 line
and helps quite a lot with performances.
Can't say I've noticed that however. How did you measure?
gears & quake3 but i haven't yet done any X 2d benchmark, 2d is dead slow with kms at leat this is the feeling i have.
Cheers,
Jerome
Quoted 53 lines · Jerome Glisse
Quoted 51 lines · Michel Dänzer
Quoted 37 lines · Jerome Glisse
Quoted 35 lines · Jerome Glisse
Quoted 15 lines · Jerome Glisse
Quoted 3 lines · Thomas Hellström
4) We could now skip the
ttm_tt_populate()inttm_tt_set_caching, since it will always allocate cached pages and then transition them.Okay 4) is bad, what happens (my brain is a bit meltdown so i might be wrong) :
1 - bo get allocatedtt->state= unpopulated
2 - bo is mapped few page are faultedtt->state= unpopulated 3 - bo is cache transitioned buttt->state== unpopulated but
they are page which have been touch by the cpu so we need to clflush them and transition them, this never happen if we don't callttm_tt_populateand proceed with the remaining of the cache transitioning functionsQuoted 7 lines
As a workaround i will try to go through the pages tables and transition existing pages. Do you have any idea for a better plan ?
Cheers,
JeromeMy workaround ruin the whole idea of pool allocation what happens is that most bo get cache transition page per page. My thinking is that we should do the following:
- is there is a least one page allocated then fully populate the object and do cache transition on all the pages.
- otherwise updatecaching_stateand leaves object unpopulatedThis needs that we some how reflect the fact that there is at least one page allocated, i am thinking to adding a new state for that :
ttm_partialy_populatedQuoted 9 lines
Quoted 5 lines
Thomas what do you think about that ?
Cheers,
JeromeAttached updated patch it doesn't introduce
ttm_partialy_populatedbut keep the populate call in cache transition. So far it seems to work properly on AGP platformYeah, this one works for me as well.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
and helps quite a lot with performances.
Can't say I've noticed that however. How did you measure?
gears
Hmm,
In gears there shouldn't really be any buffer allocation / freeing going on at all once the display lists are set up, and gears should really be gpu bound in most cases.
what's the source of the buffer allocations / frees when gears is run?
/Thomas
Quoted 61 lines · Thomas Hellström
Quoted 54 lines · Jerome Glisse
Quoted 52 lines · Michel Dänzer
Quoted 38 lines · Jerome Glisse
Quoted 36 lines · Jerome Glisse
Quoted 16 lines · Jerome Glisse
Quoted 4 lines · Thomas Hellström
4) We could now skip the
ttm_tt_populate()inttm_tt_set_caching, since it will always allocate cached pages and then transition them.Okay 4) is bad, what happens (my brain is a bit meltdown so i might be wrong) :
1 - bo get allocatedtt->state= unpopulated
2 - bo is mapped few page are faultedtt->state= unpopulated 3 - bo is cache transitioned buttt->state== unpopulated but
they are page which have been touch by the cpu so we need to clflush them and transition them, this never happen if we don't callttm_tt_populateand proceed with the remaining of the cache transitioning functionsQuoted 7 lines
As a workaround i will try to go through the pages tables and transition existing pages. Do you have any idea for a better plan ?
Cheers,
JeromeMy workaround ruin the whole idea of pool allocation what happens is that most bo get cache transition page per page. My thinking is that we should do the following:
- is there is a least one page allocated then fully populate the object and do cache transition on all the pages.
- otherwise updatecaching_stateand leaves object unpopulatedThis needs that we some how reflect the fact that there is at least one page allocated, i am thinking to adding a new state for that :
ttm_partialy_populatedQuoted 9 lines
Quoted 5 lines
Thomas what do you think about that ?
Cheers,
JeromeAttached updated patch it doesn't introduce
ttm_partialy_populatedbut keep the populate call in cache transition. So far it seems to work properly on AGP platformYeah, this one works for me as well.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
and helps quite a lot with performances.
Can't say I've noticed that however. How did you measure?
gears
Quoted 8 lines
Hmm,
In gears there shouldn't really be any buffer allocation / freeing going on at all once the display lists are set up, and gears should really be gpu bound in most cases.what's the source of the buffer allocations / frees when gears is run?
/Thomas
We free reallocate vertex buffer each frame iirc.
Cheers,
Jerome
Quoted 67 lines · Jerome Glisse
Quoted 66 lines · Thomas Hellström
Quoted 54 lines · Jerome Glisse
Quoted 52 lines · Michel Dänzer
Quoted 38 lines · Jerome Glisse
Quoted 36 lines · Jerome Glisse
Quoted 16 lines · Jerome Glisse
Quoted 4 lines · Thomas Hellström
4) We could now skip the
ttm_tt_populate()inttm_tt_set_caching, since it will always allocate cached pages and then transition them.Okay 4) is bad, what happens (my brain is a bit meltdown so i might be wrong) :
1 - bo get allocatedtt->state= unpopulated
2 - bo is mapped few page are faultedtt->state= unpopulated 3 - bo is cache transitioned buttt->state== unpopulated but
they are page which have been touch by the cpu so we need to clflush them and transition them, this never happen if we don't callttm_tt_populateand proceed with the remaining of the cache transitioning functionsQuoted 7 lines
As a workaround i will try to go through the pages tables and transition existing pages. Do you have any idea for a better plan ?
Cheers,
JeromeMy workaround ruin the whole idea of pool allocation what happens is that most bo get cache transition page per page. My thinking is that we should do the following:
- is there is a least one page allocated then fully populate the object and do cache transition on all the pages.
- otherwise updatecaching_stateand leaves object unpopulatedThis needs that we some how reflect the fact that there is at least one page allocated, i am thinking to adding a new state for that :
ttm_partialy_populatedQuoted 9 lines
Quoted 5 lines
Thomas what do you think about that ?
Cheers,
JeromeAttached updated patch it doesn't introduce
ttm_partialy_populatedbut keep the populate call in cache transition. So far it seems to work properly on AGP platformYeah, this one works for me as well.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
and helps quite a lot with performances.
Can't say I've noticed that however. How did you measure?
gears
Hmm,
In gears there shouldn't really be any buffer allocation / freeing going on at all once the display lists are set up, and gears should really be gpu bound in most cases.
Quoted 5 lines
Quoted 3 lines
what's the source of the buffer allocations / frees when gears is run?
/Thomas
We free reallocate vertex buffer each frame iirc.
Gears does everything in display lists which means geometry is held in VBOs retained for the life of the application. Once the first frame is rendered there shouldn't be any more uploads.
Keith
Dave Airlie
maintainerQuoted 70 lines · Keith Whitwell
Quoted 69 lines · Jerome Glisse
Quoted 68 lines · Thomas Hellström
Quoted 54 lines · Jerome Glisse
Quoted 52 lines · Michel Dänzer
Quoted 38 lines · Jerome Glisse
Quoted 36 lines · Jerome Glisse
Quoted 16 lines · Jerome Glisse
Quoted 4 lines · Thomas Hellström
4) We could now skip the
ttm_tt_populate()inttm_tt_set_caching, since it will always allocate cached pages and then transition them.Okay 4) is bad, what happens (my brain is a bit meltdown so i might be wrong) :
1 - bo get allocatedtt->state= unpopulated
2 - bo is mapped few page are faultedtt->state= unpopulated 3 - bo is cache transitioned buttt->state== unpopulated but they are page which have been touch by the cpu so we need to clflush them and transition them, this never happen if we don't callttm_tt_populateand proceed with the remaining of the cache transitioning functionsQuoted 7 lines
As a workaround i will try to go through the pages tables and transition existing pages. Do you have any idea for a better plan ?
Cheers,
JeromeMy workaround ruin the whole idea of pool allocation what happens is that most bo get cache transition page per page. My thinking is that we should do the following:
- is there is a least one page allocated then fully populate the object and do cache transition on all the pages.
- otherwise updatecaching_stateand leaves object unpopulatedThis needs that we some how reflect the fact that there is at least one page allocated, i am thinking to adding a new state for that :
ttm_partialy_populatedQuoted 9 lines
Quoted 5 lines
Thomas what do you think about that ?
Cheers,
JeromeAttached updated patch it doesn't introduce
ttm_partialy_populatedbut keep the populate call in cache transition. So far it seems to work properly on AGP platformYeah, this one works for me as well.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
and helps quite a lot with performances.
Can't say I've noticed that however. How did you measure?
gears
Hmm,
In gears there shouldn't really be any buffer allocation / freeing going on at all once the display lists are set up, and gears should really be gpu bound in most cases.what's the source of the buffer allocations / frees when gears is run?
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
/Thomas
We free reallocate vertex buffer each frame iirc.
Gears does everything in display lists which means geometry is held in VBOs retained for the life of the application. Once the first frame is rendered there shouldn't be any more uploads.
I don't think the r300 driver does proper vbo's yet.
Dave.
Quoted 28 lines · Jerome Glisse
Quoted 16 lines · Thomas Hellström
Jerome Glisse skrev:
Quoted 11 lines
Hi,
Thomas i attach a reworked page pool allocator based on Dave works, this one should be ok with ttm cache status tracking. It definitely helps on AGP system, now the bottleneck is in mesa vertex's dma allocation.
Cheers,
JeromeHi, Jerome!
In general it looks very good. Some things that need fixing:Quoted 9 lines
1) We must have a way to hand back pages. I still not quite understand how the shrink callbacks work and whether they are applicable. Another scheme would be to free, say 1MB when we have at least 2MB available.
2) We should avoid including AGP headers if AGP is not configured. Either reimplement
unmap_page_from_agpormap_page_into_agpor move them out from the AGP headers. We've hade complaints before from people with AGP free systems that the code doesn't compile.3) Since we're allocating (and freeing) in batches we should use the
set_pages_array()interface to avoid a global tlb flush per page.4) We could now skip the
ttm_tt_populate()inttm_tt_set_caching, since it will always allocate cached pages and then transition them.
Quoted 7 lines
Okay 4) is bad, what happens (my brain is a bit meltdown so i might be wrong) :
1 - bo get allocatedtt->state= unpopulated
2 - bo is mapped few page are faultedtt->state= unpopulated 3 - bo is cache transitioned buttt->state== unpopulated but
they are page which have been touch by the cpu so we need to clflush them and transition them,
Right.
Quoted 4 lines
this never happen if
we don't callttm_tt_populateand proceed with the remaining of the cache transitioning functions
Why can't we just skip ttm_tt_populate and proceed with the remaining of the cache transitioning functions? Then all pages currently allocated to the TTM will be transitioned.
/Thomas
Quoted 46 lines · Thomas Hellström
Quoted 35 lines · Jerome Glisse
Quoted 25 lines · Thomas Hellström
Jerome Glisse skrev:
Quoted 11 lines
Hi,
Thomas i attach a reworked page pool allocator based on Dave works, this one should be ok with ttm cache status tracking. It definitely helps on AGP system, now the bottleneck is in mesa vertex's dma allocation.
Cheers,
JeromeHi, Jerome!
In general it looks very good. Some things that need fixing:1) We must have a way to hand back pages. I still not quite understand how the shrink callbacks work and whether they are applicable. Another scheme would be to free, say 1MB when we have at least 2MB available.
2) We should avoid including AGP headers if AGP is not configured. Either reimplement
unmap_page_from_agpormap_page_into_agpor move them out from the AGP headers. We've hade complaints before from people with AGP free systems that the code doesn't compile.Quoted 7 lines
3) Since we're allocating (and freeing) in batches we should use the
set_pages_array()interface to avoid a global tlb flush per page.4) We could now skip the
ttm_tt_populate()inttm_tt_set_caching, since it will always allocate cached pages and then transition them.Quoted 7 lines
Okay 4) is bad, what happens (my brain is a bit meltdown so i might be wrong) :
1 - bo get allocatedtt->state= unpopulated
2 - bo is mapped few page are faultedtt->state= unpopulated 3 - bo is cache transitioned buttt->state== unpopulated but
they are page which have been touch by the cpu so we need to clflush them and transition them,Right.
Quoted 9 lines
Quoted 4 lines
this never happen if
we don't callttm_tt_populateand proceed with the remaining of the cache transitioning functionsWhy can't we just skip
ttm_tt_populateand proceed with the remaining of the cache transitioning functions? Then all pages currently allocated to the TTM will be transitioned.
Because not all tt->pages entry have valid page some are null and thus we have to go through the whole table and transition page per page. I did that and quick benchmark showed that it's faster to fully populate and fully transition.
Cheers,
Jerome
Quoted 51 lines · Jerome Glisse
Quoted 49 lines · Thomas Hellström
Quoted 31 lines · Jerome Glisse
Quoted 18 lines · Thomas Hellström
Jerome Glisse skrev:
Quoted 12 lines
Hi,
Thomas i attach a reworked page pool allocator based on Dave works, this one should be ok with ttm cache status tracking. It definitely helps on AGP system, now the bottleneck is in mesa vertex's dma allocation.
Cheers,
JeromeHi, Jerome!
In general it looks very good. Some things that need fixing:Quoted 10 lines
1) We must have a way to hand back pages. I still not quite understand how the shrink callbacks work and whether they are applicable. Another scheme would be to free, say 1MB when we have at least 2MB available.
2) We should avoid including AGP headers if AGP is not configured. Either reimplement
unmap_page_from_agpormap_page_into_agpor move them out from the AGP headers. We've hade complaints before from people with AGP free systems that the code doesn't compile.3) Since we're allocating (and freeing) in batches we should use the
set_pages_array()interface to avoid a global tlb flush per page.4) We could now skip the
ttm_tt_populate()inttm_tt_set_caching, since it will always allocate cached pages and then transition them.Quoted 8 lines
Okay 4) is bad, what happens (my brain is a bit meltdown so i might be wrong) :
1 - bo get allocatedtt->state= unpopulated
2 - bo is mapped few page are faultedtt->state= unpopulated 3 - bo is cache transitioned buttt->state== unpopulated but
they are page which have been touch by the cpu so we need to clflush them and transition them,Right.
Quoted 5 lines
this never happen if
we don't callttm_tt_populateand proceed with the remaining of the cache transitioning functions
Quoted 10 lines
Quoted 3 lines
Why can't we just skip
ttm_tt_populateand proceed with the remaining of the cache transitioning functions? Then all pages currently allocated to the TTM will be transitioned.Because not all
tt->pagesentry have valid page some are null and thus we have to go through the whole table and transition page per page. I did that and quick benchmark showed that it's faster to fully populate and fully transition.
OK.
That's probably sufficient for now.
/Thomas
Quoted 4 lines
Cheers,
Jerome
Dave Airlie
maintainerQuoted 7 lines · Jerome Glisse
Hi,
Thomas i attach a reworked page pool allocator based on Dave works, this one should be ok with ttm cache status tracking. It definitely helps on AGP system, now the bottleneck is in mesa vertex's dma allocation.
My original version kept a list of wb pages as well, this proved to be quite a useful
optimisation on my test systems when I implemented it, without it I was spending ~20%
of my CPU in getting free pages, granted I always used WB pages on PCIE/IGP systems.
Another optimisation I made at the time was around the populate call, (not sure if this
is what still happens):
Allocate a 64K local BO for DMA object.
Write into the first 5 pages from userspace - get WB pages. Bind to GART, swap those 5 pages to WC + flush. Then populate the rest with WC pages from the list.
Granted I think allocating WC in the first place from the pool might work just as well since most of the DMA buffers are write only.
Dave.
Dave Airlie skrev:
Quoted 22 lines · Jerome Glisse
Quoted 9 lines
Hi,
Thomas i attach a reworked page pool allocator based on Dave works, this one should be ok with ttm cache status tracking. It definitely helps on AGP system, now the bottleneck is in mesa vertex's dma allocation.
My original version kept a list of wb pages as well, this proved to be quite a useful
optimisation on my test systems when I implemented it, without it I was spending ~20%
of my CPU in getting free pages, granted I always used WB pages on PCIE/IGP systems.Another optimisation I made at the time was around the populate call, (not sure if this
is what still happens):
Quoted 7 lines
Allocate a 64K local BO for DMA object.
Write into the first 5 pages from userspace - get WB pages. Bind to GART, swap those 5 pages to WC + flush.
Then populate the rest with WC pages from the list. Granted I think allocating WC in the first place from the pool might work just as well since most of the DMA buffers are write only.
Yes, I think in the latter case the user-space driver should take care to specify WC from the beginning, when the BO is allocated.
BTW is there any DMA buffer verification taking place on WC buffers on Radeon?
Quoted 2 lines
Dave.
/Thomas
Quoted 31 lines · Thomas Hellström
Dave Airlie skrev:
Quoted 22 lines · Jerome Glisse
Quoted 9 lines
Hi,
Thomas i attach a reworked page pool allocator based on Dave works, this one should be ok with ttm cache status tracking. It definitely helps on AGP system, now the bottleneck is in mesa vertex's dma allocation.
My original version kept a list of wb pages as well, this proved to be quite a useful
optimisation on my test systems when I implemented it, without it I was spending ~20%
of my CPU in getting free pages, granted I always used WB pages on PCIE/IGP systems.Another optimisation I made at the time was around the populate call, (not sure if this
is what still happens):Quoted 7 lines
Allocate a 64K local BO for DMA object.
Write into the first 5 pages from userspace - get WB pages. Bind to GART, swap those 5 pages to WC + flush.
Then populate the rest with WC pages from the list. Granted I think allocating WC in the first place from the pool might work just as well since most of the DMA buffers are write only.
Quoted 5 lines
Yes, I think in the latter case the user-space driver should take care to specify WC from the beginning, when the BO is allocated.
BTW is there any DMA buffer verification taking place on WC buffers on Radeon?
Command buffer submitted from userspace are never bo object, they are normal memory so they are cached, kernel copy the content into bo buffer which have proper cache status.
Cheers,
Jerome
Quoted 25 lines · Dave Airlie
Quoted 7 lines · Jerome Glisse
Hi,
Thomas i attach a reworked page pool allocator based on Dave works, this one should be ok with ttm cache status tracking. It definitely helps on AGP system, now the bottleneck is in mesa vertex's dma allocation.
My original version kept a list of wb pages as well, this proved to be quite a useful
optimisation on my test systems when I implemented it, without it I was spending ~20%
of my CPU in getting free pages, granted I always used WB pages on PCIE/IGP systems.Another optimisation I made at the time was around the populate call, (not sure if this
is what still happens):Allocate a 64K local BO for DMA object.
Write into the first 5 pages from userspace - get WB pages. Bind to GART, swap those 5 pages to WC + flush. Then populate the rest with WC pages from the list.
Quoted 5 lines
Granted I think allocating WC in the first place from the pool might work just as well since most of the DMA buffers are write only.
Dave.
I think it's better to fix userspace to not allocate as much buffer per frame as it does now rather than having a pool of wb pages, i removed it because on my 64M box memory is getting tight, we need to compute the number of page we still based on memory. Also i think it's ok to assume that page allocation is fast enough.
I am reworking the patch with lastes Thomas comment, will post new one after a bit of testing.
Cheers,
Jerome
Quoted 6 lines
I think it's better to fix userspace to not allocate as much buffer per frame as it does now rather than having a pool of wb pages, i removed it because on my 64M box memory is getting tight, we need to compute the number of page we still based on memory. Also i think it's ok to assume that page allocation is fast enough.
Yup we could try decreasing the DMA buffer size, I think the DMA buffer would be a lot better suited to suballocation, doing individual bo allocs like the code used to was a major hit, assuming page alloc is fast enough isn't valid since I've already proved it was a major CPU user when I fixed it the first time :)
The problem with making the dma buffer smaller is the having a buffer mapped and sent to the kernel for relocs.
Dave.
Quoted 25 lines · Dave Airlie
Quoted 7 lines · Jerome Glisse
Hi,
Thomas i attach a reworked page pool allocator based on Dave works, this one should be ok with ttm cache status tracking. It definitely helps on AGP system, now the bottleneck is in mesa vertex's dma allocation.
My original version kept a list of wb pages as well, this proved to be quite a useful
optimisation on my test systems when I implemented it, without it I was spending ~20%
of my CPU in getting free pages, granted I always used WB pages on PCIE/IGP systems.Another optimisation I made at the time was around the populate call, (not sure if this
is what still happens):Allocate a 64K local BO for DMA object.
Write into the first 5 pages from userspace - get WB pages. Bind to GART, swap those 5 pages to WC + flush. Then populate the rest with WC pages from the list.
Quoted 5 lines
Granted I think allocating WC in the first place from the pool might work just as well since most of the DMA buffers are write only.
Dave.
--
Attached a new version of the patch, which integrate changes discussed.
Cheers,
Jerome
Quoted 1 line
From
9083e5466a7b81edcc989ac22d6ccdebef63116dMon Sep 17 00:00:00 2001
From: Dave Airlie <[email protected]>
Date: Wed, 24 Jun 2009 16:31:43 +1000
Subject: [PATCH] ttm: add pool wc/uc page allocator
On AGP system we might allocate/free routinely uncached or wc memory, changing page from cached (wb) to uc or wc is very expensive and involves a lot of flushing. To improve performance this allocator use a pool of uc,wc pages.
Currently each pool (wc, uc) is 256 pages big, improvement would be to tweak this according to memory pressure so we can give back memory to system.
--- drivers/gpu/drm/ttm/Makefile | 2 +- drivers/gpu/drm/ttm/ttm_memory.c | 3 + drivers/gpu/drm/ttm/ttm_page_alloc.c | 352 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/ttm/ttm_page_alloc.h | 34 ++++ drivers/gpu/drm/ttm/ttm_tt.c | 50 ++++-- include/drm/ttm/ttm_bo_driver.h | 6 + 6 files changed, 434 insertions(+), 13 deletions(-) create mode 100644 drivers/gpu/drm/ttm/ttm_page_alloc.c create mode 100644 drivers/gpu/drm/ttm/ttm_page_alloc.h
drivers/gpu/drm/ttm/Makefile +1 -1
@@ -3,6 +3,6 @@ ccflags-y := -Iinclude/drm ttm-y := ttm_agp_backend.o ttm_memory.o ttm_tt.o ttm_bo.o \ - ttm_bo_util.o ttm_bo_vm.o ttm_module.o ttm_global.o + ttm_bo_util.o ttm_bo_vm.o ttm_module.o ttm_global.o ttm_page_alloc.o obj-$(CONFIG_DRM_TTM) += ttm.o
drivers/gpu/drm/ttm/ttm_memory.c +3 -0
@@ -32,6 +32,7 @@#include <linux/mm.h> #include <linux/module.h> +#include "ttm_page_alloc.h" #define TTM_PFX "[TTM] "#define TTM_MEMORY_ALLOC_RETRIES 4
@@ -124,6 +125,7 @@ int ttm_mem_global_init(struct ttm_mem_global *glob) printk(KERN_INFO TTM_PFX "TTM available object memory: %llu MiB\n", glob->max_memory >> 20); +ttm_page_alloc_init ();return 0 ; }EXPORT_SYMBOL (ttm_mem_global_init);
@@ -135,6 +137,7 @@ void ttm_mem_global_release(struct ttm_mem_global *glob)flush_workqueue (glob->swap_queue);destroy_workqueue (glob->swap_queue); glob->swap_queue =NULL ; +ttm_page_alloc_fini (); }EXPORT_SYMBOL (ttm_mem_global_release);
drivers/gpu/drm/ttm/ttm_page_alloc.c new file +352 -0
352 lines not on this page · raw message
drivers/gpu/drm/ttm/ttm_page_alloc.h new file +34 -0
@@ -0,0 +1,34 @@ +/* +* Copyright (c) Red Hat Inc. + +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sub license, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the +* next paragraph) shall be included in all copies or substantial portions +* of the Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +* DEALINGS IN THE SOFTWARE. +* +* Authors: Dave Airlie <[email protected]> +*/ + +#ifndef TTM_PAGE_ALLOC +#define TTM_PAGE_ALLOC + +void ttm_put_page (struct page *page,int flags); +struct page *ttm_get_page (int flags); +int ttm_page_alloc_init (void ); +void ttm_page_alloc_fini (void ); + +#endif
drivers/gpu/drm/ttm/ttm_tt.c +38 -12
95 lines not on this page · raw message
include/drm/ttm/ttm_bo_driver.h +6 -0
@@ -121,6 +121,12 @@ struct ttm_backend {#define TTM_PAGE_FLAG_SWAPPED (1 << 4)#define TTM_PAGE_FLAG_PERSISTANT_SWAP (1 << 5)#define TTM_PAGE_FLAG_ZERO_ALLOC (1 << 6) +#define TTM_PAGE_FLAG_DMA32 (1 << 7) +#define TTM_PAGE_FLAG_CACHE_MASK (3 << 8) +#define TTM_PAGE_FLAG_CACHE_SHIFT 8 +#define TTM_PAGE_FLAG_CACHE_UC 0 +#define TTM_PAGE_FLAG_CACHE_WC 1 +#define TTM_PAGE_FLAG_CACHE_WB 2enum ttm_caching_state { tt_uncached,
Signature · 1 line
1.6.2.2
Jerome Glisse skrev:
Quoted 31 lines · Dave Airlie
Quoted 30 lines · Jerome Glisse
Quoted 9 lines
Hi,
Thomas i attach a reworked page pool allocator based on Dave works, this one should be ok with ttm cache status tracking. It definitely helps on AGP system, now the bottleneck is in mesa vertex's dma allocation.
My original version kept a list of wb pages as well, this proved to be quite a useful
optimisation on my test systems when I implemented it, without it I was spending ~20%
of my CPU in getting free pages, granted I always used WB pages on PCIE/IGP systems.Another optimisation I made at the time was around the populate call, (not sure if this
is what still happens):Allocate a 64K local BO for DMA object.
Write into the first 5 pages from userspace - get WB pages. Bind to GART, swap those 5 pages to WC + flush. Then populate the rest with WC pages from the list.Granted I think allocating WC in the first place from the pool might work just as well since most of the DMA buffers are write only.
Quoted 9 lines
Quoted 3 lines
Dave.
--Attached a new version of the patch, which integrate changes discussed.
Cheers,
Jerome
Hi, Jerome!
Still some outstanding things:
1) The AGP protection fixes compilation errors when AGP is not enabled, but what about architectures that need the map_page_into_agp() semantics for TTM even when AGP is not enabled? At the very least TTM should be disabled on those architectures. The best option would be to make those calls non-agp specific.
2) Why is the page refcount upped with get_page() after an alloc_page()?
3) It seems like pages are cache-transitioned one-by-one when freed. Again, this is a global TLB flush per page. Can't we free a large chunk of pages at once?
/Thanks,
Thomas
Dave Airlie
maintainer2009/6/30 Thomas Hellström <[email protected]>:
Quoted 56 lines
Jerome Glisse skrev:
Quoted 36 lines · Dave Airlie
Quoted 34 lines
Quoted 9 lines · Jerome Glisse
Hi,
Thomas i attach a reworked page pool allocator based on Dave works, this one should be ok with ttm cache status tracking. It definitely helps on AGP system, now the bottleneck is in mesa vertex's dma allocation.
My original version kept a list of wb pages as well, this proved to be quite a useful
optimisation on my test systems when I implemented it, without it I was spending ~20%
of my CPU in getting free pages, granted I always used WB pages on PCIE/IGP systems.Another optimisation I made at the time was around the populate call, (not sure if this
is what still happens):Allocate a 64K local BO for DMA object.
Write into the first 5 pages from userspace - get WB pages. Bind to GART, swap those 5 pages to WC + flush. Then populate the rest with WC pages from the list.Granted I think allocating WC in the first place from the pool might work just as well since most of the DMA buffers are write only.
Quoted 9 lines
Quoted 3 lines
Dave.
--Attached a new version of the patch, which integrate changes discussed.
Cheers,
JeromeHi, Jerome!
Still some outstanding things:1) The AGP protection fixes compilation errors when AGP is not enabled, but what about architectures that need the
map_page_into_agp()semantics for TTM even when AGP is not enabled? At the very least TTM should be disabled on those architectures. The best option would be to make those calls non-agp specific.
Quoted 6 lines
2) Why is the page refcount upped with
get_page()after analloc_page()?3) It seems like pages are cache-transitioned one-by-one when freed. Again, this is a global TLB flush per page. Can't we free a large chunk of pages at once?
Jerome,
have we addressed these?
I'd really like to push this soon, as I'd like to fix up the 32 vs 36 bit dma masks if possible
which relies on us being able to tell the allocator to use GFP_DMA32 on some hw (32-bit PAE mainly with a PCI card).
Dave.
Quoted 64 lines · Dave Airlie
2009/6/30 Thomas Hellström <[email protected]>:
Quoted 56 lines
Jerome Glisse skrev:
Quoted 36 lines · Dave Airlie
Quoted 34 lines
Quoted 9 lines · Jerome Glisse
Hi,
Thomas i attach a reworked page pool allocator based on Dave works, this one should be ok with ttm cache status tracking. It definitely helps on AGP system, now the bottleneck is in mesa vertex's dma allocation.
My original version kept a list of wb pages as well, this proved to be quite a useful
optimisation on my test systems when I implemented it, without it I was spending ~20%
of my CPU in getting free pages, granted I always used WB pages on PCIE/IGP systems.Another optimisation I made at the time was around the populate call, (not sure if this
is what still happens):Allocate a 64K local BO for DMA object.
Write into the first 5 pages from userspace - get WB pages. Bind to GART, swap those 5 pages to WC + flush. Then populate the rest with WC pages from the list.Granted I think allocating WC in the first place from the pool might work just as well since most of the DMA buffers are write only.
Quoted 9 lines
Quoted 3 lines
Dave.
--Attached a new version of the patch, which integrate changes discussed.
Cheers,
JeromeHi, Jerome!
Still some outstanding things:1) The AGP protection fixes compilation errors when AGP is not enabled, but what about architectures that need the
map_page_into_agp()semantics for TTM even when AGP is not enabled? At the very least TTM should be disabled on those architectures. The best option would be to make those calls non-agp specific.Quoted 6 lines
2) Why is the page refcount upped with
get_page()after analloc_page()?3) It seems like pages are cache-transitioned one-by-one when freed. Again, this is a global TLB flush per page. Can't we free a large chunk of pages at once?
Quoted 8 lines
Jerome,
have we addressed these?
I'd really like to push this soon, as I'd like to fix up the 32 vs 36 bit dma masks if possible
which relies on us being able to tell the allocator to useGFP_DMA32on some hw (32-bit PAE mainly with a PCI card).
FWIW, I tried this patch on my PowerBook, and it didn't go too well:
With AGP enabled, the kernel panics before there's any KMS display.
With AGP disabled, I get a KMS display, but it shows a failure to allocate the ring buffer, and then it stops updating.
Signature · 2 lines
Earthling Michel Dänzer | http://www.vmware.com Libre software enthusiast | Debian, X and DRI developer