out of order encryption
Would dm-crypt execute correctly if sector encryption ended asynchronously? For example:
If sector 1, 2, 3 are sent to be done asynchronously to the same algorithm instance, and the jobs end in the following order: 2, 1, 3; does the dm-crypt module know to write the data in the proper place when the encryption "done" callback is called?
Quoted 8 lines · On 08/06/2015 09:00 AM, Vasile Catalin-B50542 wrote:
Would dm-crypt execute correctly if sector encryption ended asynchronously? For example:
If sector 1, 2, 3 are sent to be done asynchronously to the same algorithm instance,
and the jobs end in the following order: 2, 1, 3; does the dm-crypt module know to
write the data in the proper place when the encryption "done" callback is called?
Not sure if I understand your question - encryption in dmcrypt works on sector level, sectors are encrypted independetly, so order cannot influence result of encryption of individual sectors.
Or if the question is about order of submitted requests:
If you need to ensure order of processing, you have to issue "flush" operation before submitting next data content. Filesystems typically must do this when handling journal or so.
In general, requests order can be rearranged (anywhere in block layer, not only in dmcrypt). But this is correct behaviour.
Milan
I was referring to the submitted requests.
Does the underlying encryption layer (CryptoAPI) have to ensure the complete callbacks are called in the order the requests were submitted? Or does dm-crypt figure out where to read/write after request is done no matter in which order the crypto requests finished?
Quoted 22 lines · Milan Broz
Quoted 8 lines · On 08/06/2015 09:00 AM, Vasile Catalin-B50542 wrote:
Would dm-crypt execute correctly if sector encryption ended asynchronously? For example:
If sector 1, 2, 3 are sent to be done asynchronously to the same algorithm instance,
and the jobs end in the following order: 2, 1, 3; does the dm-crypt module know to
write the data in the proper place when the encryption "done" callback is called?Not sure if I understand your question - encryption in dmcrypt works on sector level, sectors are encrypted independetly, so order cannot influence result of encryption of individual sectors.
Or if the question is about order of submitted requests:
If you need to ensure order of processing, you have to issue "flush" operation before submitting next data content. Filesystems typically must do this when handling journal or so.In general, requests order can be rearranged (anywhere in block layer, not only in dmcrypt). But this is correct behaviour.
Milan
Signature · 21 lines
CatalinVasile Intern, DN-Software FreescaleSemiconductor, Inc. www.freescale.com phone: 073-021-1938 e-mail: [email protected] <mailto:[email protected]> *Freescale_Logo-nosemi_Lh_4c*** This e-mail, and any associated attachments have been classified as: [ ] Public [ ] Freescale Semiconductor Internal Use Only [ ] Freescale Semiconductor Confidential Proprietary
I was referring to the submitted requests.
Does the underlying encryption layer (CryptoAPI) have to ensure the complete callbacks are called in the order the requests were submitted? Or does dm-crypt figure out where to read/write after request is done no matter in which order the crypto requests finished?
Quoted 22 lines · Milan Broz
Quoted 8 lines · On 08/06/2015 09:00 AM, Vasile Catalin-B50542 wrote:
Would dm-crypt execute correctly if sector encryption ended asynchronously? For example:
If sector 1, 2, 3 are sent to be done asynchronously to the same algorithm instance,
and the jobs end in the following order: 2, 1, 3; does the dm-crypt module know to
write the data in the proper place when the encryption "done" callback is called?Not sure if I understand your question - encryption in dmcrypt works on sector level, sectors are encrypted independetly, so order cannot influence result of encryption of individual sectors.
Or if the question is about order of submitted requests:
If you need to ensure order of processing, you have to issue "flush" operation before submitting next data content. Filesystems typically must do this when handling journal or so.In general, requests order can be rearranged (anywhere in block layer, not only in dmcrypt). But this is correct behaviour.
Milan
Quoted 3 lines · On 08/06/2015 12:38 PM, Vasile Catalin-B50542 wrote:
I was referring to the submitted requests.
Does the underlying encryption layer (CryptoAPI) have to ensure the complete callbacks are called in the order the requests were submitted?
No. And it really does not matter, requests are independent.
Quoted 2 lines
Or does dm-crypt figure out where to read/write after request is done no matter in which order the crypto requests finished?
In recent kernels write request are in dmcrypt partially sorted for performance reasons
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/md/dm-crypt.c?id=b3c5fd3052492f1b8d060799d4f18be5a5438add
Otherwise there is no other logic related.
m.
Quoted 30 lines · Milan Broz
Quoted 18 lines · On 08/06/2015 09:00 AM, Vasile Catalin-B50542 wrote:
Quoted 8 lines
Would dm-crypt execute correctly if sector encryption ended asynchronously? For example:
If sector 1, 2, 3 are sent to be done asynchronously to the same algorithm instance,
and the jobs end in the following order: 2, 1, 3; does the dm-crypt module know to
write the data in the proper place when the encryption "done" callback is called?Not sure if I understand your question - encryption in dmcrypt works on sector level, sectors are encrypted independetly, so order cannot influence result of encryption of individual sectors.
Or if the question is about order of submitted requests:
If you need to ensure order of processing, you have to issue "flush" operation before submitting next data content. Filesystems typically must do this when handling journal or so.Quoted 4 lines
In general, requests order can be rearranged (anywhere in block layer, not only in dmcrypt). But this is correct behaviour.
Milan
_______________________________________________
dm-crypt mailing list
[email protected]
http://www.saout.de/mailman/listinfo/dm-crypt
Am 06.08.2015 um 12:38 schrieb Vasile Catalin-B50542:
Quoted 2 lines
Does the underlying encryption layer (CryptoAPI) have to ensure the complete callbacks are called in the order the requests were submitted?
Are you thinking about parallelization or asynchronous processing?
I am sending crypto requests to a hardware component which has an option to keeping them in order or not. I was trying to use the out of order option because this would let requests run in parallel. If I tried running them with the out of order option bun then I get:
No key available with this passphrase.
When trying to execute the following commands:
cryptsetup -c aes-xts-plain -y -v luksFormat /dev/sda1 cryptsetup open /dev/sda1 test_xts1
Quoted 5 lines · Lars Winterfeld
Am 06.08.2015 um 12:38 schrieb Vasile Catalin-B50542:
Quoted 2 lines
Does the underlying encryption layer (CryptoAPI) have to ensure the complete callbacks are called in the order the requests were submitted?
Are you thinking about parallelization or asynchronous processing?
As far as I understand, your commands are used to set up the crypto mapper. They should be executed previous to any read/write operations. I think Milan was talking about read/write operations that are executed after the crypto mapper has been set up successfully.
Btw... I guess that if two write operations result in two different changes in the same sector, there are no write conflicts, because for each write, the current sector contents is decrypted first, changes are applies, encrypted and written back. Is that correct?
Am 06.08.2015 um 13:32 schrieb Vasile Catalin-B50542:
Quoted 15 lines
I am sending crypto requests to a hardware component which has an option to keeping them in order or not. I was trying to use the out of order option because this would let requests run in parallel. If I tried running them with the out of order option bun then I get:
No key available with this passphrase.
When trying to execute the following commands:
cryptsetup -c aes-xts-plain -y -v luksFormat/dev/sda1cryptsetup open/dev/sda1test_xts1Quoted 5 lines · Lars Winterfeld
Am 06.08.2015 um 12:38 schrieb Vasile Catalin-B50542:
Quoted 2 lines
Does the underlying encryption layer (CryptoAPI) have to ensure the complete callbacks are called in the order the requests were submitted?
Are you thinking about parallelization or asynchronous processing?
Quoted 8 lines · On 08/06/2015 01:32 PM, Vasile Catalin-B50542 wrote:
I am sending crypto requests to a hardware component which has an option to keeping them in order or not. I was trying to use the out of order option because this would let requests run in parallel. If I tried running them with the out of order option bun then I get:
No key available with this passphrase.
When trying to execute the following commands:
cryptsetup -c aes-xts-plain -y -v luksFormat/dev/sda1cryptsetup open/dev/sda1test_xts1
Sigh. Please when you asking about some explicit problem, please always mention it from the beginning...
So this is probably unrelated to dmcrypt. Userspace cryptsetup could use kernel crypto API wrapper directly (add --debug and you will see, there should be log message that it uses kernel crypto userspace interface).
This seems like a bug in the kernel crypto API provider, what exactly it is? What architecture?
We have seen the same problem with Raspbery PI2 & NEON crypto driver (it was a bug in kernel code), I guess it is the same here.
Milan
Quoted 14 lines · Lars Winterfeld
Quoted 5 lines
Am 06.08.2015 um 12:38 schrieb Vasile Catalin-B50542:
Quoted 2 lines
Does the underlying encryption layer (CryptoAPI) have to ensure the complete callbacks are called in the order the requests were submitted?
Are you thinking about parallelization or asynchronous processing?
_______________________________________________
dm-crypt mailing list
[email protected]
http://www.saout.de/mailman/listinfo/dm-crypt
I am using a Freescale P3041DS platform with CAAM (drivers/crypto/CAAM/*) hardware accelerator. The platforms is a PowerPC processor.
Quoted 37 lines · Milan Broz
Quoted 8 lines · On 08/06/2015 01:32 PM, Vasile Catalin-B50542 wrote:
I am sending crypto requests to a hardware component which has an option to keeping them in order or not. I was trying to use the out of order option because this would let requests run in parallel. If I tried running them with the out of order option bun then I get:
No key available with this passphrase.
When trying to execute the following commands:
cryptsetup -c aes-xts-plain -y -v luksFormat/dev/sda1cryptsetup open/dev/sda1test_xts1Sigh. Please when you asking about some explicit problem, please always mention it from the beginning...
So this is probably unrelated to dmcrypt. Userspace cryptsetup could use kernel crypto API wrapper directly (add --debug and you will see, there should be log message that it uses kernel crypto userspace interface).
This seems like a bug in the kernel crypto API provider, what exactly it is? What architecture?
We have seen the same problem with Raspbery PI2 & NEON crypto driver (it was a bug in kernel code), I guess it is the same here.
Milan
Quoted 12 lines · Lars Winterfeld
Quoted 5 lines
Am 06.08.2015 um 12:38 schrieb Vasile Catalin-B50542:
Quoted 2 lines
Does the underlying encryption layer (CryptoAPI) have to ensure the complete callbacks are called in the order the requests were submitted?
Are you thinking about parallelization or asynchronous processing?
_______________________________________________
dm-crypt mailing list
[email protected]
http://www.saout.de/mailman/listinfo/dm-crypt
Signature · 21 lines
CatalinVasile Intern, DN-Software FreescaleSemiconductor, Inc. www.freescale.com phone: 073-021-1938 e-mail: [email protected] <mailto:[email protected]> *Freescale_Logo-nosemi_Lh_4c*** This e-mail, and any associated attachments have been classified as: [ ] Public [ ] Freescale Semiconductor Internal Use Only [ ] Freescale Semiconductor Confidential Proprietary
Quoted 2 lines · On 08/06/2015 02:10 PM, Vasile Catalin-B50542 wrote:
I am using a Freescale P3041DS platform with CAAM (
drivers/crypto/CAAM/*) hardware accelerator. The platforms is a PowerPC processor.
Well, I think there is some problem with the crypto driver in kernel.
And because I see that this driver is currently in active development, I would suggest to contact the developer and ask for an advice, see
http://thread.gmane.org/gmane.linux.kernel.cryptoapi/16176
Unfortunately we do not have this hw available so I cannot provide more help here.
You can try to compile cryptsetup with --disable-kernel_crypto option - maybe it is a workaround. (But there can be a data corruption later...)
Anyway, if it works, the I would guess there is a bug in userspace interface code.
Maybe it can help how we debugged a similar issue here
http://thread.gmane.org/gmane.linux.kernel.device-mapper.dm-crypt/7966/focus=13358
Milan
Quoted 71 lines · Milan Broz
Quoted 37 lines · On 08/06/2015 01:32 PM, Vasile Catalin-B50542 wrote:
Quoted 8 lines
I am sending crypto requests to a hardware component which has an option to keeping them in order or not. I was trying to use the out of order option because this would let requests run in parallel. If I tried running them with the out of order option bun then I get:
No key available with this passphrase.
When trying to execute the following commands:
cryptsetup -c aes-xts-plain -y -v luksFormat/dev/sda1cryptsetup open/dev/sda1test_xts1Sigh. Please when you asking about some explicit problem, please always mention it from the beginning...
So this is probably unrelated to dmcrypt. Userspace cryptsetup could use kernel crypto API wrapper directly (add --debug and you will see, there should be log message that it uses kernel crypto userspace interface).
This seems like a bug in the kernel crypto API provider, what exactly it is? What architecture?
We have seen the same problem with Raspbery PI2 & NEON crypto driver (it was a bug in kernel code), I guess it is the same here.
Milan
Quoted 12 lines · Lars Winterfeld
Quoted 5 lines
Am 06.08.2015 um 12:38 schrieb Vasile Catalin-B50542:
Quoted 2 lines
Does the underlying encryption layer (CryptoAPI) have to ensure the complete callbacks are called in the order the requests were submitted?
Are you thinking about parallelization or asynchronous processing?
_______________________________________________
dm-crypt mailing list
[email protected]
http://www.saout.de/mailman/listinfo/dm-cryptSignature · 28 lines
CatalinVasile Intern, DN-Software FreescaleSemiconductor, Inc. www.freescale.com phone: 073-021-1938 e-mail: [email protected] <mailto:[email protected]> *Freescale_Logo-nosemi_Lh_4c*** This e-mail, and any associated attachments have been classified as: [ ] Public [ ] Freescale Semiconductor Internal Use Only [ ] Freescale Semiconductor Confidential Proprietary _______________________________________________ dm-crypt mailing list [email protected] http://www.saout.de/mailman/listinfo/dm-crypt