Skip to contentSkip to conversation

Streaming ciphers

dm-crypt David Niklas 3 participants 5 messages Feb 16, 2020 – Feb 17, 2020

lore
Share & tools
lore
#0 lore raw

Hello,
Today I was helping a poor luser who's Crucial P1 QLC SSD died from too many writes. We still don't know what's doing the writing, but he is running luks using FDE so write amplification is definitely a factor.
I forget the answer to this so please enlighten me, is there a streaming cypher he can use instead of a block cipher? A better option that I didn't consider?
Not that the TCG Opal is to be fully trusted (nor is it available on that model), but how does it work with respect to write amplification vs. luks?

Thanks,
David

#1 lore raw

Hi David,

using LUKS should not result in write-amplification. LUKS uses 512B blocks and just provides an 1:1 mapping.

Regards,
Arno

Quoted 18 lines · David Niklas

Hello,
Today I was helping a poor luser who's Crucial P1 QLC SSD died from too many writes. We still don't know what's doing the writing, but he is running luks using FDE so write amplification is definitely a factor.
I forget the answer to this so please enlighten me, is there a streaming cypher he can use instead of a block cipher? A better option that I didn't consider?
Not that the TCG Opal is to be fully trusted (nor is it available on that model), but how does it work with respect to write amplification vs. luks?

Thanks,
David
_______________________________________________
dm-crypt mailing list
[email protected]
https://www.saout.de/mailman/listinfo/dm-crypt

Signature · 7 lines
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: [email protected]
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier
#2 lore raw
Quoted 4 lines · Arno Wagner

Hi David,

using LUKS should not result in write-amplification. LUKS uses 512B blocks and just provides an 1:1 mapping.

Using defaults it does cause a higher write-amplification. By default dm-crypt doesn't not enable discard-support, whereas it is normaly supported for an unencrypted filesystem.

And given that at least some distributions even install a fstrim-cronjob, the filesystems are trimmed, even if not mounted with the discard-option.

This results in an unencrypted filesystem having it's empty space "empty", as far as the SSD is concerned.
Wheras "empty" space (either "is" or "stays") "used" for a LUKS device.

The "is" or "stays" difference comes with having the encrypted device pre-inited or not. With pre-init the blocks are always "in use". Without pre-init the blocks are "unused" as long as they aren't written and switch to "in use" after first beeing written, after which they will stay "in use" forever.

Less "empty" space results in higher write-amplification because the SSD has to remap "empty" space as it is "in use", as far as the SSD is concerned.

A pre-inited encrypted device, in this scenario, never has any empty space.

I'm not saying that you can't make it equivalent, just that there are differences when using the defaults.

Quoted 32 lines · David Niklas
Quoted 18 lines

Hello,
Today I was helping a poor luser who's Crucial P1 QLC SSD died from too many writes. We still don't know what's doing the writing, but he is running luks using FDE so write amplification is definitely a factor.
I forget the answer to this so please enlighten me, is there a streaming cypher he can use instead of a block cipher? A better option that I didn't consider?
Not that the TCG Opal is to be fully trusted (nor is it available on that model), but how does it work with respect to write amplification vs. luks?

Thanks,
David
_______________________________________________
dm-crypt mailing list
[email protected]
https://www.saout.de/mailman/listinfo/dm-crypt

Signature · 11 lines
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: [email protected]
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier
_______________________________________________
dm-crypt mailing list
[email protected]
https://www.saout.de/mailman/listinfo/dm-crypt
Signature · 1 line
Matthias
#3 lore raw

Yuck, top post. Reformat and reply.

Quoted 43 lines · On Sun, 16 Feb 2020 14:58:10 +0100 Matthias Schniedermeyer <[email protected]> wrote:
Quoted 15 lines · David Niklas
Quoted 14 lines

Hello,
Today I was helping a poor luser who's Crucial P1 QLC SSD died from too many writes. We still don't know what's doing the writing, but he is running luks using FDE so write amplification is definitely a factor.
I forget the answer to this so please enlighten me, is there a streaming cypher he can use instead of a block cipher? A better option that I didn't consider?
Not that the TCG Opal is to be fully trusted (nor is it available on that model), but how does it work with respect to write amplification vs. luks?

Thanks,
David

Quoted 4 lines · Arno Wagner

Hi David,

using LUKS should not result in write-amplification. LUKS uses 512B blocks and just provides an 1:1 mapping.

Using defaults it does cause a higher write-amplification. By default dm-crypt doesn't not enable discard-support, whereas it is normaly supported for an unencrypted filesystem.

And given that at least some distributions even install a fstrim-cronjob, the filesystems are trimmed, even if not mounted with the discard-option.

This results in an unencrypted filesystem having it's empty space "empty", as far as the SSD is concerned.
Wheras "empty" space (either "is" or "stays") "used" for a LUKS device.

The "is" or "stays" difference comes with having the encrypted device pre-inited or not. With pre-init the blocks are always "in use". Without pre-init the blocks are "unused" as long as they aren't written and switch to "in use" after first beeing written, after which they will stay "in use" forever.

Less "empty" space results in higher write-amplification because the SSD has to remap "empty" space as it is "in use", as far as the SSD is concerned.

Quoted 6 lines

A pre-inited encrypted device, in this scenario, never has any empty space.

I'm not saying that you can't make it equivalent, just that there are differences when using the defaults.

Thanks! I understand.
Is there anything else I should be aware of (aside from a kernel option I need to look up again to delay writing to disk), that will lessen disk writes/amplification from luks?

Thanks,
David

#4 lore raw
Quoted 51 lines · David Niklas

Yuck, top post. Reformat and reply.

Quoted 47 lines · On Sun, 16 Feb 2020 14:58:10 +0100 Matthias Schniedermeyer <[email protected]> wrote:
Quoted 15 lines · David Niklas
Quoted 14 lines

Hello,
Today I was helping a poor luser who's Crucial P1 QLC SSD died from too many writes. We still don't know what's doing the writing, but he is running luks using FDE so write amplification is definitely a factor.
I forget the answer to this so please enlighten me, is there a streaming cypher he can use instead of a block cipher? A better option that I didn't consider?
Not that the TCG Opal is to be fully trusted (nor is it available on that model), but how does it work with respect to write amplification vs. luks?

Thanks,
David

Quoted 4 lines · Arno Wagner

Hi David,

using LUKS should not result in write-amplification. LUKS uses 512B blocks and just provides an 1:1 mapping.

Using defaults it does cause a higher write-amplification. By default dm-crypt doesn't not enable discard-support, whereas it is normaly supported for an unencrypted filesystem.

And given that at least some distributions even install a fstrim-cronjob, the filesystems are trimmed, even if not mounted with the discard-option.

This results in an unencrypted filesystem having it's empty space "empty", as far as the SSD is concerned.
Wheras "empty" space (either "is" or "stays") "used" for a LUKS device.

The "is" or "stays" difference comes with having the encrypted device pre-inited or not. With pre-init the blocks are always "in use". Without pre-init the blocks are "unused" as long as they aren't written and switch to "in use" after first beeing written, after which they will stay "in use" forever.

Less "empty" space results in higher write-amplification because the SSD has to remap "empty" space as it is "in use", as far as the SSD is concerned.

A pre-inited encrypted device, in this scenario, never has any empty space.

Quoted 7 lines
Quoted 2 lines

I'm not saying that you can't make it equivalent, just that there are differences when using the defaults.

Thanks! I understand.
Is there anything else I should be aware of (aside from a kernel option I need to look up again to delay writing to disk), that will lessen disk writes/amplification from luks?

If you don't have a "SSD killing write load" (at least hundreth's of GB/s per day) you normaly don't have to worry about killing any halfway recent SSD.

Of course you can always have bad luck. I personally had 2 enterprise SSDs die on me @work, both after only a few weeks in production. (IOW "Infant Death" category)

But i've no personal experience with "very recent" QLC drives. QLC drives are expected to have an even lower Write Endurance than TLCs.

Given that the Crucial P1 appears to have entered the market just over a years ago it is a quite recent drive.

The 1TB drive is listed with an endurance of 200TB and 5 year warranty. That makes about 110GB per day, or about 0.11 DWPD.

So i would guess that meantioned drive is less than 1 year old.

Unless you had written to the SSD in excess of 600GB per day continously since it was bought, it was likely more because of "bad luck" that the SSD died.

Signature · 1 line
Matthias