kernel.org tarball/patch signature files
I noticed that patch-3.0.7.sign is a detached signature file for DECOMPRESSED patch-3.0.7.{bz2,gz,xz}. Maybe this is not the best possible way to sign compressed tarballs/patches. This is because it places hell of lot of trust on quality/security of decompressor implementation. Historically decompressor implementations have had bugs and security flaws. It is stupid to assume that there won't be any more of them.
Wrong order to verify compressed tarball/patch:
(1) Feed potentially maliciously formatted data to decompressor, and exploit any undiscovered/unpatched vulnerability in decompressor implementation.
(2) Verify decompressed output.
Much better order would be:
(1) Verify compressed data.
(2) Feed trusted data to decompressor.
So, would it be possible to have multiple signature files like this? Please.
patch-3.X.Y.bz2
patch-3.X.Y.bz2.sign
patch-3.X.Y.gz
patch-3.X.Y.gz.sign
patch-3.X.Y.xz
patch-3.X.Y.xz.sign
Signature · 1 line
Jari Ruusu 1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9 DB 1D EB E3 24 0E A9 DD
Quoted 2 lines · Jari Ruusu
I noticed that patch-3.0.7.sign is a detached signature file for DECOMPRESSED patch-3.0.7.{bz2,gz,xz}.
That's exactly what I said in my announcement:
https://lkml.org/lkml/2011/10/23/51
so I'm glad it's working properly :)
Quoted 17 lines
Maybe this is not the best possible
way to sign compressed tarballs/patches. This is because it places hell of lot of trust on quality/security of decompressor implementation. Historically decompressor implementations have had bugs and security flaws. It is stupid to assume that there won't be any more of them.Wrong order to verify compressed tarball/patch:
(1) Feed potentially maliciously formatted data to decompressor, and exploit any undiscovered/unpatched vulnerability in decompressor implementation.
(2) Verify decompressed output.Much better order would be:
(1) Verify compressed data.
(2) Feed trusted data to decompressor.
Quoted 8 lines
So, would it be possible to have multiple signature files like this? Please.
patch-3.X.Y.bz2
patch-3.X.Y.bz2.sign
patch-3.X.Y.gz
patch-3.X.Y.gz.sign
patch-3.X.Y.xz
patch-3.X.Y.xz.sign
Nope, sorry, let's try this way instead. That way we only have to generate one signature, not 3.
If you are really worried about decompressor bugs, then run them in a virtual machine/chroot :)
greg k-h
Quoted 21 lines · Greg KH
Quoted 11 lines · Jari Ruusu
Wrong order to verify compressed tarball/patch:
(1) Feed potentially maliciously formatted data to decompressor, and exploit any undiscovered/unpatched vulnerability in decompressor implementation.
(2) Verify decompressed output.Much better order would be:
(1) Verify compressed data.
(2) Feed trusted data to decompressor.Quoted 8 lines
So, would it be possible to have multiple signature files like this? Please.
patch-3.X.Y.bz2
patch-3.X.Y.bz2.sign
patch-3.X.Y.gz
patch-3.X.Y.gz.sign
patch-3.X.Y.xz
patch-3.X.Y.xz.sign
Quoted 2 lines
Nope, sorry, let's try this way instead. That way we only have to generate one signature, not 3.
How about one signed message that contains multiple SHA256 sums or whatever?
sha256sum patch-3.X.Y.{bz2,gz,xz} | gpg --clearsign -a >patch-3.X.Y.signThat allows verification before decompression.
Quoted 2 lines
If you are really worried about decompressor bugs, then run them in a virtual machine/chroot :)
I am not amused.
Greg, please put your security hat on, and look at this from security point of view. Decompression after verify removes/closes attacks utilizing yet unidentified decompressor bugs or security flaws.
If I remember correctly, newer versions of OpenSSH disable compression before authentication. They do that to pre-emptively close attacks resulting yet unidentified bugs in decompression code.
Signature · 1 line
Jari Ruusu 1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9 DB 1D EB E3 24 0E A9 DD
Quoted 31 lines · Jari Ruusu
Quoted 21 lines · Greg KH
Quoted 11 lines · Jari Ruusu
Wrong order to verify compressed tarball/patch:
(1) Feed potentially maliciously formatted data to decompressor, and exploit any undiscovered/unpatched vulnerability in decompressor implementation.
(2) Verify decompressed output.Much better order would be:
(1) Verify compressed data.
(2) Feed trusted data to decompressor.Quoted 8 lines
So, would it be possible to have multiple signature files like this? Please.
patch-3.X.Y.bz2
patch-3.X.Y.bz2.sign
patch-3.X.Y.gz
patch-3.X.Y.gz.sign
patch-3.X.Y.xz
patch-3.X.Y.xz.signQuoted 2 lines
Nope, sorry, let's try this way instead. That way we only have to generate one signature, not 3.
How about one signed message that contains multiple SHA256 sums or whatever?
sha256sum patch-3.X.Y.{bz2,gz,xz} | gpg --clearsign -a >patch-3.X.Y.signThat allows verification before decompression.
Quoted 8 lines
Quoted 2 lines
If you are really worried about decompressor bugs, then run them in a virtual machine/chroot :)
I am not amused.
Greg, please put your security hat on, and look at this from security point of view. Decompression after verify removes/closes attacks utilizing yet unidentified decompressor bugs or security flaws.
Please realize that we are building this whole thing back up from scratch here, and some services are not possible to do just yet.
Also realize our constraints. I don't want to, and in fact it's almost impossible to, upload the 3 compressed tarballs over a large number of internet connections that I have access to as I travel around the world. So, given that you don't want all kernel release maintainers to store their private keys on the server (remember what just happened), the ability to generate that signature would be a pretty difficult thing, right?
So, we are working on a proposal that has a "throw away" signature for the compressed files, that can be verified by people after they download the tarball to ensure that they didn't get something "odd". That will handle the sha256sum, and allow us to add future compression types with no need to regenerate the "real" signatures from the original release.
The real check, to verify that this tarball really came from "me" should be done on the uncompressed tarball, which is what I can sign, and it is something that you, or anyone else, can reliable duplicate on their own by just using git and not even downloading the tarball at all.
In other words, we just saved you a MASSIVE bandwidth transation for all of your future kernel downloads, and you can reliable know that the tarball you have in your system is what is on the kernel.org servers without you even having to download it yourself and run those decompression tools that you don't trus.
And still you complain :)
Hope this helps explain why this is the way it is. It's as if people think we are just making it up as we go along...
greg k-h
Quoted 28 lines · Greg KH
Please realize that we are building this whole thing back up from scratch here, and some services are not possible to do just yet.
Also realize our constraints. I don't want to, and in fact it's almost impossible to, upload the 3 compressed tarballs over a large number of internet connections that I have access to as I travel around the world. So, given that you don't want all kernel release maintainers to store their private keys on the server (remember what just happened), the ability to generate that signature would be a pretty difficult thing, right?
So, we are working on a proposal that has a "throw away" signature for the compressed files, that can be verified by people after they download the tarball to ensure that they didn't get something "odd". That will handle the sha256sum, and allow us to add future compression types with no need to regenerate the "real" signatures from the original release.
The real check, to verify that this tarball really came from "me" should be done on the uncompressed tarball, which is what I can sign, and it is something that you, or anyone else, can reliable duplicate on their own by just using git and not even downloading the tarball at all.
In other words, we just saved you a MASSIVE bandwidth transation for all of your future kernel downloads, and you can reliable know that the tarball you have in your system is what is on the kernel.org servers without you even having to download it yourself and run those decompression tools that you don't trus.
Quoted 4 lines
And still you complain :)
Hope this helps explain why this is the way it is. It's as if people think we are just making it up as we go along...
Well, in theory it could still be done (albeit not in practice right now).
There is a tool called "pristine-tar" designed to generate a byte-identical tar.gz given the original source tree. Obviously with git-archive you don't need the "tar" portion, but it also includes "pristine-gz" and "pristine-bz2" tools which tease out and store the original compression parameters in a small file. It's commonly used to be able to "import" a bunch of source files and a tarball into GIT without storing the entire original tarball as a binary file.
So the kernel developers would locally produce the uncompressed and compressed tarballs, then digitally sign them and create small "pristine-{gz,bz2,xz}" archive deltas; the signatures and deltas would be uploaded.
The kernel.org server would use the original input and compression parameters to reconstruct the byte-identical compressed archive from the output of "git archive --format=tar", again with the "pristine-*" tools.
Unfortunately, while there are "pristine-gz" and "pristine-bz2" tools, there has not yet been developed a "pristine-xz" tool:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=499489
So it's not yet reasonably possible for the kernel.org archive, but sometime in the future it might become so.
Cheers,
Kyle Moffett
Signature · 2 lines
Curious about my work on the Debian powerpcspe port? I'm keeping a blog here: http://pureperl.blogspot.com/
Quoted 8 lines · Kyle Moffett
Unfortunately, while there are "pristine-gz" and "pristine-bz2" tools, there has not yet been developed a "pristine-xz" tool:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=499489So it's not yet reasonably possible for the kernel.org archive, but sometime in the future it might become so.
There are a lot of strong reasons to NOT do so, however.
Any time we have something signed by a developer, we have something that is precious and cannot be replicated by kernel.org staff.
Any time we have something signed by a robot, we have something that people *will* misinterpret as having security properties that they don't -- this has unfortunately been amply shown.
Compression formats evolve over time; right now we're concerned with gz, bz2, and xz, but xz is brand new here and there may very well be new things in the future.
It would be a very good thing for people to develop tools to run compressors and decompressors in locked-down boxes. It should be possible to run these kinds of programs without access to either network or filesystem; only read from stdin and out on stdout (and presumably stderr for errors.) This would solve problems for much more than just kernel.org.
-hpa
Quoted 6 lines · H. Peter Anvin
It would be a very good thing for people to develop tools to run compressors and decompressors in locked-down boxes. It should be possible to run these kinds of programs without access to either network or filesystem; only read from stdin and out on stdout (and presumably stderr for errors.) This would solve problems for much more than just kernel.org.
Wasn't there once a kernel hack called seccomp, that only allowed read syscalls on stdin and writes on stdout and that was it? ;)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 4 lines · On 10/25/2011 11:13 AM, [email protected] wrote:
Wasn't there once a kernel hack called seccomp, that only allowed read syscalls on stdin and writes on stdout and that was it? ;)
It's still there (and there are a bunch of other ways to do this in Linux). The big thing is to wrapper this up and get people to use it.
Even better would be to get this into gzip, bzip2 and xz so it
happens automatically!
-hpa
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQIcBAEBAgAGBQJOpoIwAAoJEL2gYIVJO6zkzr4P/2NRkQuMXLtaVLLaxGo1CRx+ 7PrNNbUqOMCWcLaMJeNQZPabgWzpaUUshtxkUsmDcKuiHzZcXGK8Pp7QmCGVssmR zPCn/D2bN+JFkqKxAJyTy57OE35ccXGanGd3BtmNe5KQic06SsEZJ/EWFRuTZxDP t+Ub08/Fq35D8/dw6ZQ3UdFzwGDQt/8Oxs+gVymuYro08Cezz7aL6bYllS+v5Y1m 2EQ9Nk9sBe9wKLLQJqvwTTXl9abhzlugFCcCsK41PZ0fW38s9728NgKru4sn8CmC O7E3aClUJPEu/58h6WnAavaemEjHPhQn0KqShgn+amgyqRTMbX+uwq5RY49pLWZ2 eEVRHbGkLLHkfQYJuCjM09y5sRGp1HZC3CUPiZoEZTZXTeZ0Fl162v6utyo39dxT 1TJYsYpoYYrjtg5jFBNLh1uQ1ZeI46fOvhP84v9JbSmYBpo/Fb1yb5nj5YflIpUs XVXVv9mu0Cuz60tuf0ceqAsvoiYaliM56bZjr65pscOPDzTP/qPtY4Q9UfEjeiAO 1N/l3gBJpOcHD3K5E1v3wcfD0PMeAu1W3WfW9Mj4KswPIz3EFQ6h1ja/urGC8GD3 pw19Uv33mDVHAlKFmZ5N2qIAmbmNaXejDmqvZxsvUyvSzqk+EmB33oYzPv+6n76p IWHwm1SNbBjQI2AlEmmW
=P2oY
-----END PGP SIGNATURE-----
Quoted 3 lines · Greg KH
Also realize our constraints. I don't want to, and in fact it's almost impossible to, upload the 3 compressed tarballs over a large number of internet connections that I have access to as I travel around the world.
How about signing just one compressed tarball/patch, and let other compressed versions be without signature files. Users who want to check signature will then download the one you signed.
Signature · 1 line
Jari Ruusu 1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9 DB 1D EB E3 24 0E A9 DD
Quoted 8 lines · Jari Ruusu
Quoted 3 lines · Greg KH
Also realize our constraints. I don't want to, and in fact it's almost impossible to, upload the 3 compressed tarballs over a large number of internet connections that I have access to as I travel around the world.
How about signing just one compressed tarball/patch, and let other compressed versions be without signature files. Users who want to check signature will then download the one you signed.
That's no difference from what we are doing today if you think it though.
greg k-h
Quoted 7 lines · Greg KH
Quoted 3 lines · Jari Ruusu
How about signing just one compressed tarball/patch, and let other compressed versions be without signature files. Users who want to check signature will then download the one you signed.
That's no difference from what we are doing today if you think it though.
You signing compressed version eliminates exposure to yet undiscovered decompressor bugs. Pre-emptively removing as many as possible attack surfaces is the correct securely engineered way of doing things. You signing decompressed version is opposite of that. You added new attack surface.
Decompressors are complex software. They almost certainly have bugs in evil-formatted data handling, forever. Reading data from untrusted internet and feeding that data to decompressor without any authentication, screams "REMOTE EXPLOIT ME", if you look at it from security point of view.
Quoted 3 lines
Yes, we are working on just that thing, and the foo.tar.sums file will be signed with the kernel.org "throwaway" key, so you can check that as well.
That sounds much better than current signing practice.
Signature · 1 line
Jari Ruusu 1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9 DB 1D EB E3 24 0E A9 DD
Quoted 4 lines · Greg KH
The real check, to verify that this tarball really came from "me" should be done on the uncompressed tarball, which is what I can sign, and it is something that you, or anyone else, can reliable duplicate on their own by just using git and not even downloading the tarball at all.
I'm OK on that part..
Quoted 5 lines
In other words, we just saved you a MASSIVE bandwidth transation for all of your future kernel downloads, and you can reliable know that the tarball you have in your system is what is on the kernel.org servers without you even having to download it yourself and run those decompression tools that you don't trus.
If you're building an automated process that will take a just-uploaded foo.tar and generate foo.tar.{bz2,gz,foozip}, can you add a step that would just do an 'md5sum foo.tar.* > foo.tar.sums'? Or sha256sum if you're worried about the crypto weakness issues with MD5. Personally, I'm more interested in the "Did I hit a network error that the TCP checksum didn't catch?" case.
No hurry, I know what a beast it can be to redesign systems of this scale. Just a would-be-nice...
Quoted 15 lines · On Tue, Oct 25, 2011 at 03:28:37AM -0400, [email protected] wrote:
Quoted 4 lines · Greg KH
The real check, to verify that this tarball really came from "me" should be done on the uncompressed tarball, which is what I can sign, and it is something that you, or anyone else, can reliable duplicate on their own by just using git and not even downloading the tarball at all.
I'm OK on that part..
Quoted 5 lines
In other words, we just saved you a MASSIVE bandwidth transation for all of your future kernel downloads, and you can reliable know that the tarball you have in your system is what is on the kernel.org servers without you even having to download it yourself and run those decompression tools that you don't trus.
Quoted 5 lines
If you're building an automated process that will take a just-uploaded foo.tar and generate foo.tar.{bz2,gz,foozip}, can you add a step that would just do an 'md5sum foo.tar.* > foo.tar.sums'? Or sha256sum if you're worried about the crypto weakness issues with MD5. Personally, I'm more interested in the "Did I hit a network error that the TCP checksum didn't catch?" case.
Yes, we are working on just that thing, and the foo.tar.sums file will be signed with the kernel.org "throwaway" key, so you can check that as well.
greg k-h
Quoted 2 lines · Greg KH
If you are really worried about decompressor bugs, then run them in a virtual machine/chroot :)
Of more concern than bugs are errors during download. Yes, TCP has a checksum, which is a CRC that quite frankly sucks when we're talking the amount of data that kernel.org moves. So there's a non-zero chance you'll get bad data downloaded. And you really want to do a more effective data check (MD5 or SHA sum, or a PGP signature) *before* you decompress, in case the corrupted data causes a spew of gigabytes of trash and fills your filesystem.