Skip to contentSkip to conversation

L3 cache way-locking memory zone

DISCUSSION Marek Bykowski 2 participants 3 messages Apr 15, 2019 – Apr 16, 2019

lore
Share & tools
lore
#0 lore raw

Hi All,

Would the mainline be interested in definition/creation of the memory zone to L3 cache way-locking memory range/s?

We have this implemented around the system featuring Cortex-A53/57 and the CCN-512/504 with L3 cache (LLC) configured to way-locking. The benefits arising are a faster memory access.

If you reckon this is something of interest I can share more details, patch, use case, etc.

Thanks,
Marek

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#1 lore raw
Quoted 5 lines · Marek Bykowski

Hi All,

Would the mainline be interested in definition/creation of the memory zone to L3 cache way-locking memory range/s?

Quoted 7 lines

We have this implemented around the system featuring Cortex-A53/57 and the CCN-512/504 with L3 cache (LLC) configured to way-locking. The benefits arising are a faster memory access.

If you reckon this is something of interest I can share more details, patch, use case, etc.

I am sure many will be interested. But depends on how generic is the solution ? It's extremely difficult to come up with one.

ARM has Memory System Resource Partitioning(MPAM) to solve this in a generic way. I know it may not solve issue for current platforms, but just thought it's worth pointing out so that people are aware of it.

--
Regards,
Sudeep

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#2 lore raw

Hi Sudeep,

To me it is quite generic but I don't have an experience working on a broad range of ARM products.

It works more less along this: one needs to set up an SMC service allowing querying the L3 way-locking registers/configuration from within kernel. The L3 way-locking is configured itself statically in SPL Uboot running EL3, secure state. The static configuration is a result of the constraint on when and how the registers can be accessed, namely "normal"/memory access to HN-F cannot be in-flight while the write to these registers occur - to my best knowledge we cannot meet that with the system/Linux up and running. The secure state results from the L3 lock registers being only accessed from the secure.

After then when Linux boots it does the SMC call to query whether there is the L3 way-locking in place and what base addresses and the sizes and it sets up the zone upon it.

The following are some confs/printouts from the Linux having 2G of RAM, and 24M of LLC with 16M configured to locked:

zone_size[0] 131072 zhole_size[0] 0
zone_size[1] 4096 zhole_size[1] 0
zone_size[2] 389120 zhole_size[2] 0
zone_size[3] 0 zhole_size[3] 0

L3LOCKED zone: 64 pages used for memmap
L3LOCKED zone: 4096 pages, LIFO batch:0

mem_init() DMA->managed_pages 129024
mem_init() L3LOCK->managed_pages 4032
mem_init() Normal->managed_pages 383040
mem_init() Movable->managed_pages 0

And after releasing free pages to the buddy allocator

mem_init() DMA->managed_pages 123787
mem_init() L3LOCK->managed_pages 4096
mem_init() Normal->managed_pages 379305
mem_init() Movable->managed_pages 0

# cat /proc/buddyinfo (reformatted to squeeze)
Node 0, zone DMA    4  4  3  4  2  3  2  4  2  3  122 
Node 0, zone L3LOCK 0  0  0  0  0  0  0  0  0  0  4  
Node 0, zone Normal 21 3  7  2  2  1  1  1  1  2  306 
# cat /proc/slabinfo
[...]
Node 0, zone    L3LOCK
  pages free     4096
        min      135
        low      168
        high     201
   node_scanned  0
        spanned  4096
        present  4096
        managed  4096

Then we can allocate to the L3LOCK zone with 'kmalloc(size, GFP_L3LOCK)'.

We have used the L3LOCK to benchmark the DMA that presented the better results when copying from L3 locked to L3 locked than from "unlocked" to "unlocked".

Thanks,
Marek

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel