Skip to contentSkip to conversation

blk: Increase cache element size

RFC PATCH In review Marek Vasut 2 participants 10 messages Aug 8, 2018 – Jan 16, 2019 Block layer

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 blk: Increase cache element size

Cache up to 4 kiB entries. 4 kiB is the default block size on ext4, yet the underlying block layer devices usually report support for 512B . In most cases, the 512B support is emulated (ie. SD cards, SSDs, USB sticks etc.) and the real block size of those devices is much bigger.

To avoid performance degradation with such devices and FS setup, bump the maximum cache entry size to 4 kiB.

---
 drivers/block/blkcache.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/block/blkcache.c +1 -1
index 294511fcdb..1fa64989d3 100644
@@ -24,7 +24,7 @@ struct block_cache_node {
 static LIST_HEAD(block_cache);
 
 static struct block_cache_stats _stats = {
-	.max_blocks_per_entry = 2,
+	.max_blocks_per_entry = 8,
 	.max_entries = 32
 };
 
Signature · 1 line
2.16.2
#1 lore raw

RFC PATCH 1/1 blk: Increase cache element size

Quoted 11 lines · Marek Vasut

Cache up to 4 kiB entries. 4 kiB is the default block size on ext4, yet the underlying block layer devices usually report support for 512B . In most cases, the 512B support is emulated (ie. SD cards, SSDs, USB sticks etc.) and the real block size of those devices is much bigger.

To avoid performance degradation with such devices and FS setup, bump the maximum cache entry size to 4 kiB.

I'll pick this up post v2018.09 if no one objects, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180815/4a89d264/attachment.sig>
#2 lore raw

RFC PATCH 1/1 blk: Increase cache element size

Quoted 10 lines · Tom Rini
Quoted 8 lines · Marek Vasut

Cache up to 4 kiB entries. 4 kiB is the default block size on ext4, yet the underlying block layer devices usually report support for 512B . In most cases, the 512B support is emulated (ie. SD cards, SSDs, USB sticks etc.) and the real block size of those devices is much bigger.

To avoid performance degradation with such devices and FS setup, bump the maximum cache entry size to 4 kiB.

Quoted 7 lines
Quoted 3 lines

I'll pick this up post v2018.09 if no one objects, thanks!

I object. I was hoping there'd be some discussion on how to solve this in a future-proof manner ... it's only a matter of time until someone uses ext4 with 8k blocks on an SSD ...

Signature · 2 lines
Best regards,
Marek Vasut
#3 lore raw

RFC PATCH 1/1 blk: Increase cache element size

Quoted 15 lines · Marek Vasut
Quoted 14 lines · Tom Rini
Quoted 11 lines · Marek Vasut

Cache up to 4 kiB entries. 4 kiB is the default block size on ext4, yet the underlying block layer devices usually report support for 512B . In most cases, the 512B support is emulated (ie. SD cards, SSDs, USB sticks etc.) and the real block size of those devices is much bigger.

To avoid performance degradation with such devices and FS setup, bump the maximum cache entry size to 4 kiB.

Quoted 7 lines
Quoted 3 lines

I'll pick this up post v2018.09 if no one objects, thanks!

I object. I was hoping there'd be some discussion on how to solve this in a future-proof manner ... it's only a matter of time until someone uses ext4 with 8k blocks on an SSD ...

In general, sure? In specific, mkfs.ext4 1.42.13 man page says 1/2/4KiB are the only valid values of block size, and based on having to whack this for some other projects it's pretty common for OpenEmbedded at least to spit out 1KiB block size images. So unless you know of cases today (or tomorrow, but not next year) where 8KiB is common or likely, we should probably just bump this for now and maybe make it a tunable so it's easily changed?

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180815/68c58710/attachment.sig>
#4 lore raw

RFC PATCH 1/1 blk: Increase cache element size

Quoted 21 lines · Tom Rini
Quoted 19 lines · Marek Vasut
Quoted 17 lines · Tom Rini
Quoted 11 lines · Marek Vasut

Cache up to 4 kiB entries. 4 kiB is the default block size on ext4, yet the underlying block layer devices usually report support for 512B . In most cases, the 512B support is emulated (ie. SD cards, SSDs, USB sticks etc.) and the real block size of those devices is much bigger.

To avoid performance degradation with such devices and FS setup, bump the maximum cache entry size to 4 kiB.

I'll pick this up post v2018.09 if no one objects, thanks!

Quoted 8 lines
Quoted 3 lines

I object. I was hoping there'd be some discussion on how to solve this in a future-proof manner ... it's only a matter of time until someone uses ext4 with 8k blocks on an SSD ...

In general, sure? In specific, mkfs.ext4 1.42.13 man page says 1/2/4KiB are the only valid values of block size, and based on having to whack this for some other projects it's pretty common for OpenEmbedded at least to spit out 1KiB block size images.

OE spits 4k , that's how I triggered it, meta/classes/image_types.bbclass:EXTRA_IMAGECMD_ext2 ?= "-i 4096" meta/classes/image_types.bbclass:EXTRA_IMAGECMD_ext3 ?= "-i 4096" meta/classes/image_types.bbclass:EXTRA_IMAGECMD_ext4 ?= "-i 4096"

Quoted 4 lines

So unless you know of cases
today (or tomorrow, but not next year) where 8KiB is common or likely, we should probably just bump this for now and maybe make it a tunable so it's easily changed?

It is already tunable, see blkcache config in blkcache command.

But what I'd like to see is somehow the FS and the underlying storage negotiating the best settings. Can we get the FS block size from the block cache perspective ?

Signature · 2 lines
Best regards,
Marek Vasut
#5 lore raw

RFC PATCH 1/1 blk: Increase cache element size

Quoted 31 lines · Marek Vasut
Quoted 21 lines · Tom Rini
Quoted 19 lines · Marek Vasut
Quoted 17 lines · Tom Rini
Quoted 11 lines · Marek Vasut

Cache up to 4 kiB entries. 4 kiB is the default block size on ext4, yet the underlying block layer devices usually report support for 512B . In most cases, the 512B support is emulated (ie. SD cards, SSDs, USB sticks etc.) and the real block size of those devices is much bigger.

To avoid performance degradation with such devices and FS setup, bump the maximum cache entry size to 4 kiB.

I'll pick this up post v2018.09 if no one objects, thanks!

Quoted 8 lines
Quoted 3 lines

I object. I was hoping there'd be some discussion on how to solve this in a future-proof manner ... it's only a matter of time until someone uses ext4 with 8k blocks on an SSD ...

In general, sure? In specific, mkfs.ext4 1.42.13 man page says 1/2/4KiB are the only valid values of block size, and based on having to whack this for some other projects it's pretty common for OpenEmbedded at least to spit out 1KiB block size images.

Quoted 4 lines

OE spits 4k , that's how I triggered it, meta/classes/image_types.bbclass:EXTRA_IMAGECMD_ext2 ?= "-i 4096" meta/classes/image_types.bbclass:EXTRA_IMAGECMD_ext3 ?= "-i 4096" meta/classes/image_types.bbclass:EXTRA_IMAGECMD_ext4 ?= "-i 4096"

That's bytes-per-inode, I was referring to block size which is -b and dynamic unless specified.

Quoted 10 lines
Quoted 4 lines

So unless you know of cases
today (or tomorrow, but not next year) where 8KiB is common or likely, we should probably just bump this for now and maybe make it a tunable so it's easily changed?

It is already tunable, see blkcache config in blkcache command.

But what I'd like to see is somehow the FS and the underlying storage negotiating the best settings. Can we get the FS block size from the block cache perspective ?

Good questions that I don't have an answer to.

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180815/61c269cb/attachment.sig>
#6 lore raw

RFC PATCH 1/1 blk: Increase cache element size

Quoted 32 lines · Tom Rini
Quoted 31 lines · Marek Vasut
Quoted 29 lines · Tom Rini
Quoted 15 lines · Marek Vasut
Quoted 14 lines · Tom Rini
Quoted 11 lines · Marek Vasut

Cache up to 4 kiB entries. 4 kiB is the default block size on ext4, yet the underlying block layer devices usually report support for 512B . In most cases, the 512B support is emulated (ie. SD cards, SSDs, USB sticks etc.) and the real block size of those devices is much bigger.

To avoid performance degradation with such devices and FS setup, bump the maximum cache entry size to 4 kiB.

Quoted 7 lines
Quoted 3 lines

I'll pick this up post v2018.09 if no one objects, thanks!

I object. I was hoping there'd be some discussion on how to solve this in a future-proof manner ... it's only a matter of time until someone uses ext4 with 8k blocks on an SSD ...

In general, sure? In specific, mkfs.ext4 1.42.13 man page says 1/2/4KiB are the only valid values of block size, and based on having to whack this for some other projects it's pretty common for OpenEmbedded at least to spit out 1KiB block size images.

Quoted 7 lines
Quoted 4 lines

OE spits 4k , that's how I triggered it, meta/classes/image_types.bbclass:EXTRA_IMAGECMD_ext2 ?= "-i 4096" meta/classes/image_types.bbclass:EXTRA_IMAGECMD_ext3 ?= "-i 4096" meta/classes/image_types.bbclass:EXTRA_IMAGECMD_ext4 ?= "-i 4096"

That's bytes-per-inode, I was referring to block size which is -b and dynamic unless specified.

Right. Although, I think we mostly care about caching the inodes, not blocks, since the inodes are accessed repeatedly.

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

So unless you know of cases
today (or tomorrow, but not next year) where 8KiB is common or likely, we should probably just bump this for now and maybe make it a tunable so it's easily changed?

Quoted 7 lines
Quoted 5 lines

It is already tunable, see blkcache config in blkcache command.

But what I'd like to see is somehow the FS and the underlying storage negotiating the best settings. Can we get the FS block size from the block cache perspective ?

Good questions that I don't have an answer to.

What we can do is extend fs_devread() and blk_dread() with a new flag, bool cachable, to allow FS and other upper layers which read block devices to mark blocks that should specifically be cached as such. The block cache would then cache the block no matter what size it has.

The FS implementations should know which data should be cached because they will be accessed repeatedly and which data are not, so they can set the flag accordingly.

The other users of blk_dread() would need to be examined. Possibly the best solution right now to avoid problems would be to apply this patch first. Then, second, add the flag to blk_dread() and set it accordingly where applicable AND make the block cache cache blocks which are either <= 8 kiB OR with cachable flag. Then finally, probably in next release, drop the <= 8 kiB condition and cache only blocks which are flagged as cachable to avoid polluting the block cache with useless small crumbs of data.

The downside is, the blk_dread() would look a bit asymmetrical compared to the other functions with the cachable flag in it's parameters.

Thoughts ?

Signature · 2 lines
Best regards,
Marek Vasut
#7 lore raw

RFC PATCH 1/1 blk: Increase cache element size

Quoted 38 lines · Marek Vasut
Quoted 37 lines · Tom Rini
Quoted 35 lines · Marek Vasut
Quoted 21 lines · Tom Rini
Quoted 19 lines · Marek Vasut
Quoted 17 lines · Tom Rini
Quoted 11 lines · Marek Vasut

Cache up to 4 kiB entries. 4 kiB is the default block size on ext4, yet the underlying block layer devices usually report support for 512B . In most cases, the 512B support is emulated (ie. SD cards, SSDs, USB sticks etc.) and the real block size of those devices is much bigger.

To avoid performance degradation with such devices and FS setup, bump the maximum cache entry size to 4 kiB.

I'll pick this up post v2018.09 if no one objects, thanks!

Quoted 8 lines
Quoted 3 lines

I object. I was hoping there'd be some discussion on how to solve this in a future-proof manner ... it's only a matter of time until someone uses ext4 with 8k blocks on an SSD ...

In general, sure? In specific, mkfs.ext4 1.42.13 man page says 1/2/4KiB are the only valid values of block size, and based on having to whack this for some other projects it's pretty common for OpenEmbedded at least to spit out 1KiB block size images.

OE spits 4k , that's how I triggered it, meta/classes/image_types.bbclass:EXTRA_IMAGECMD_ext2 ?= "-i 4096" meta/classes/image_types.bbclass:EXTRA_IMAGECMD_ext3 ?= "-i 4096" meta/classes/image_types.bbclass:EXTRA_IMAGECMD_ext4 ?= "-i 4096"

Quoted 5 lines
Quoted 2 lines

That's bytes-per-inode, I was referring to block size which is -b and dynamic unless specified.

Right. Although, I think we mostly care about caching the inodes, not blocks, since the inodes are accessed repeatedly.

OK, so it's the inode block.

Quoted 31 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

So unless you know of cases
today (or tomorrow, but not next year) where 8KiB is common or likely, we should probably just bump this for now and maybe make it a tunable so it's easily changed?

Quoted 7 lines
Quoted 5 lines

It is already tunable, see blkcache config in blkcache command.

But what I'd like to see is somehow the FS and the underlying storage negotiating the best settings. Can we get the FS block size from the block cache perspective ?

Good questions that I don't have an answer to.

What we can do is extend fs_devread() and blk_dread() with a new flag, bool cachable, to allow FS and other upper layers which read block devices to mark blocks that should specifically be cached as such. The block cache would then cache the block no matter what size it has.

The FS implementations should know which data should be cached because they will be accessed repeatedly and which data are not, so they can set the flag accordingly.

The other users of blk_dread() would need to be examined. Possibly the best solution right now to avoid problems would be to apply this patch first. Then, second, add the flag to blk_dread() and set it accordingly where applicable AND make the block cache cache blocks which are either <= 8 kiB OR with cachable flag. Then finally, probably in next release, drop the <= 8 kiB condition and cache only blocks which are flagged as cachable to avoid polluting the block cache with useless small crumbs of data.

Quoted 4 lines

The downside is, the blk_dread() would look a bit asymmetrical compared to the other functions with the cachable flag in it's parameters.

Thoughts ?

Frankly, this sounds rather complicated when we might just need to increase the size of what we can cache a bit. We aren't a long running facility and so long as it's tunable, it's also most likely being used in machines where we have enough DDR that we can a few more kilobytes.

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180816/54283ee9/attachment.sig>
#8 lore raw

RFC PATCH 1/1 blk: Increase cache element size

Quoted 81 lines · Tom Rini
Quoted 38 lines · Marek Vasut
Quoted 37 lines · Tom Rini
Quoted 35 lines · Marek Vasut
Quoted 21 lines · Tom Rini
Quoted 19 lines · Marek Vasut
Quoted 17 lines · Tom Rini
Quoted 11 lines · Marek Vasut

Cache up to 4 kiB entries. 4 kiB is the default block size on ext4, yet the underlying block layer devices usually report support for 512B . In most cases, the 512B support is emulated (ie. SD cards, SSDs, USB sticks etc.) and the real block size of those devices is much bigger.

To avoid performance degradation with such devices and FS setup, bump the maximum cache entry size to 4 kiB.

I'll pick this up post v2018.09 if no one objects, thanks!

Quoted 8 lines
Quoted 3 lines

I object. I was hoping there'd be some discussion on how to solve this in a future-proof manner ... it's only a matter of time until someone uses ext4 with 8k blocks on an SSD ...

In general, sure? In specific, mkfs.ext4 1.42.13 man page says 1/2/4KiB are the only valid values of block size, and based on having to whack this for some other projects it's pretty common for OpenEmbedded at least to spit out 1KiB block size images.

OE spits 4k , that's how I triggered it, meta/classes/image_types.bbclass:EXTRA_IMAGECMD_ext2 ?= "-i 4096" meta/classes/image_types.bbclass:EXTRA_IMAGECMD_ext3 ?= "-i 4096" meta/classes/image_types.bbclass:EXTRA_IMAGECMD_ext4 ?= "-i 4096"

Quoted 5 lines
Quoted 2 lines

That's bytes-per-inode, I was referring to block size which is -b and dynamic unless specified.

Right. Although, I think we mostly care about caching the inodes, not blocks, since the inodes are accessed repeatedly.

OK, so it's the inode block.

Quoted 34 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

So unless you know of cases
today (or tomorrow, but not next year) where 8KiB is common or likely, we should probably just bump this for now and maybe make it a tunable so it's easily changed?

Quoted 7 lines
Quoted 5 lines

It is already tunable, see blkcache config in blkcache command.

But what I'd like to see is somehow the FS and the underlying storage negotiating the best settings. Can we get the FS block size from the block cache perspective ?

Good questions that I don't have an answer to.

What we can do is extend fs_devread() and blk_dread() with a new flag, bool cachable, to allow FS and other upper layers which read block devices to mark blocks that should specifically be cached as such. The block cache would then cache the block no matter what size it has.

The FS implementations should know which data should be cached because they will be accessed repeatedly and which data are not, so they can set the flag accordingly.

The other users of blk_dread() would need to be examined. Possibly the best solution right now to avoid problems would be to apply this patch first. Then, second, add the flag to blk_dread() and set it accordingly where applicable AND make the block cache cache blocks which are either <= 8 kiB OR with cachable flag. Then finally, probably in next release, drop the <= 8 kiB condition and cache only blocks which are flagged as cachable to avoid polluting the block cache with useless small crumbs of data.

The downside is, the blk_dread() would look a bit asymmetrical compared to the other functions with the cachable flag in it's parameters.

Quoted 6 lines
Quoted 1 line

Thoughts ?

Frankly, this sounds rather complicated when we might just need to increase the size of what we can cache a bit. We aren't a long running facility and so long as it's tunable, it's also most likely being used in machines where we have enough DDR that we can a few more kilobytes.

It maybe increase the performance slightly though by not polluting the cache with which shouldn't be there. Might even be useful in SPL.

Signature · 2 lines
Best regards,
Marek Vasut
#9 lore raw
In reply to Marek Vasut
Quoted 5 lines · Marek Vasut

Cache up to 4 kiB entries. 4 kiB is the default block size on ext4, yet the underlying block layer devices usually report support for 512B . In most cases, the 512B support is emulated (ie. SD cards, SSDs, USB sticks etc.) and the real block size of those devices is much bigger.

Quoted 7 lines

To avoid performance degradation with such devices and FS setup, bump the maximum cache entry size to 4 kiB.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190115/4476c498/attachment.sig>