ovl: Add support for fs-verity checking of lowerdata
Subsystems (3)
v3 cover ovl: Add support for fs-verity checking of lowerdata
This patchset adds support for using fs-verity to validate lowerdata files by specifying an overlay.verity xattr on the metacopy files.
This is primarily motivated by the Composefs usecase, where there will be a read-only EROFS layer that contains redirect into a base data layer which has fs-verity enabled on all files. However, it is also useful in general if you want to ensure that the lowerdata files matches the expected content over time.
I have also added some tests for this feature to xfstests[1].
I'm also CC:ing the fsverity list and maintainers because there is one (tiny) fsverity change, and there may be interest in this usecase.
Changes since v2:
* Rebased on top of overlayfs-next
* We now alway do verity verification the first time the file content is used, rather than doing it at lookup time for the non-lazy lookup case.
Changes since v1:
* Rebased on v2 lazy lowerdata series
* Dropped the "validate" mount option variant. We now only support "off", "on" and "require", where "off" is the default.
* We now store the digest algorithm used in the overlay.verity xattr.
* Dropped ability to configure default verity options, as this could cause problems moving layers between machines.
* We now properly resolve dependent mount options by automatically enabling metacopy and redirect_dir if verity is on, or failing if the specified options conflict.
* Streamlined and fixed the handling of creds in ovl_ensure_verity_loaded().
* Renamed new helpers from ovl_entry_path_ to ovl_e_path_
[1] https://github.com/alexlarsson/xfstests/commits/verity-tests
Alexander Larsson (4):
fsverity: Export fsverity_get_digest
ovl: Add framework for verity support
ovl: Validate verity xattr when resolving lowerdata
ovl: Handle verity during copy-up
Documentation/filesystems/overlayfs.rst | 27 +++++ fs/overlayfs/copy_up.c | 33 +++++- fs/overlayfs/file.c | 8 +- fs/overlayfs/namei.c | 54 +++++++++- fs/overlayfs/overlayfs.h | 12 ++- fs/overlayfs/ovl_entry.h | 3 + fs/overlayfs/super.c | 79 +++++++++++++- fs/overlayfs/util.c | 133 ++++++++++++++++++++++++ fs/verity/measure.c | 1 + 9 files changed, 340 insertions(+), 10 deletions(-)
Signature · 1 line
2.40.1
PATCH v3 1/4 fsverity: Export fsverity_get_digest
Overlayfs needs to call this when built in module form, so we need to export the symbol. This uses EXPORT_SYMBOL_GPL like the other fsverity functions do.
--- fs/verity/measure.c | 1 + 1 file changed, 1 insertion(+)
fs/verity/measure.c +1 -0
@@ -85,3 +85,4 @@ int fsverity_get_digest(struct inode *inode, *alg = hash_alg->algo_id;return 0 ; } +EXPORT_SYMBOL_GPL (fsverity_get_digest);
Signature · 1 line
2.40.1
PATCH v3 2/4 ovl: Add framework for verity support
In reply to Alexander LarssonThis adds the scaffolding (docs, config, mount options) for supporting the new overlay xattr "overlay.verity", which contains a fs-verity digest. This is used for metacopy files, and the actual fs-verity digest of the lowerdata file needs to match it. The mount option "verity" specifies how this xattr is handled.
If you enable verity ("verity=on") all existing xattrs are validated before use, and during metacopy we generate verity xattr in the upper metacopy file (if the source file has verity enabled). This means later accesses can guarantee that the same data is used.
Additionally you can use "verity=require". In this mode all metacopy files must have a valid verity xattr. For this to work metadata copy-up must be able to create a verity xattr (so that later accesses are validated). Therefore, in this mode, if the lower data file doesn't have fs-verity enabled we fall back to a full copy rather than a metacopy.
Actual implementation follows in a separate commit.
--- Documentation/filesystems/overlayfs.rst | 27 +++++++++ fs/overlayfs/ovl_entry.h | 3 + fs/overlayfs/super.c | 74 ++++++++++++++++++++++++- 3 files changed, 102 insertions(+), 2 deletions(-)
Documentation/filesystems/overlayfs.rst +27 -0
@@ -406,6 +406,33 @@ when a "metacopy" file in one of the lower layers above it, has a "redirect" to the absolute path of the "lower data" file in the "data-only" lower layer. +fs-verity support +---------------------- + +When metadata copy up is used for a file, then the xattr +"trusted.overlay.verity" may be set on the metacopy file. This +specifies the expected fs-verity digest of the lowerdata file. This +may then be used to verify the content of the source file at the time +the file is opened. During metacopy copy up overlayfs can also set +this xattr. + +This is controlled by the "verity" mount option, which supports +these values: + +- "off": + The verity xattr is never used. This is the default if verity + option is not specified. +- "on": + Whenever a metacopy files specifies an expected digest, the + corresponding data file must match the specified digest. + When generating a metacopy file the verity xattr will be set + from the source file fs-verity digest (if it has one). +- "require": + Same as "on", but additionally all metacopy files must specify a + verity xattr. This means metadata copy up will only be used if + the data file has fs-verity enabled, otherwise a full copy-up is + used. + Sharing and copying layers --------------------------
fs/overlayfs/ovl_entry.h +3 -0
@@ -13,6 +13,9 @@ struct ovl_config {bool redirect_dir;bool redirect_follow;const char *redirect_mode; +bool verity; +bool require_verity; +const char *verity_mode;bool index;bool uuid;bool nfs_export;
fs/overlayfs/super.c +72 -2
158 lines not on this page · raw message
Signature · 1 line
2.40.1
Eric Biggers
maintainerRe: [PATCH v3 2/4] ovl: Add framework for verity support
Quoted 16 lines · Alexander Larsson
Documentation/filesystems/overlayfs.rst +16
-0+fs-verity support +---------------------- + +When metadata copy up is used for a file, then the xattr +"trusted.overlay.verity" may be set on the metacopy file. This +specifies the expected fs-verity digest of the lowerdata file. This +may then be used to verify the content of the source file at the time +the file is opened. During metacopy copy up overlayfs can also set +this xattr. + +This is controlled by the "verity" mount option, which supports +these values: + +- "off": + The verity xattr is never used. This is the default if verity + option is not specified.
Documentation/filesystems/overlayfs.rst when a "metacopy" file in one of the lower layers above it, has a "redirect" · line 434
Quoted 10 lines
Documentation/filesystems/overlayfs.rst +10
-0+- "on": + Whenever a metacopy files specifies an expected digest, the + corresponding data file must match the specified digest. + When generating a metacopy file the verity xattr will be set + from the source file fs-verity digest (if it has one). +- "require": + Same as "on", but additionally all metacopy files must specify a + verity xattr. This means metadata copy up will only be used if + the data file has fs-verity enabled, otherwise a full copy-up is + used.
It looks like my request for improved documentation was not taken, which is unfortunate and makes this patchset difficult to review.
- Eric
Amir Goldstein
maintainerRe: [PATCH v3 2/4] ovl: Add framework for verity support
Quoted 23 lines · Eric Biggers
Quoted 21 lines · Alexander Larsson
Documentation/filesystems/overlayfs.rst +21
-0+fs-verity support +---------------------- + +When metadata copy up is used for a file, then the xattr +"trusted.overlay.verity" may be set on the metacopy file. This +specifies the expected fs-verity digest of the lowerdata file. This +may then be used to verify the content of the source file at the time +the file is opened. During metacopy copy up overlayfs can also set +this xattr. + +This is controlled by the "verity" mount option, which supports +these values: + +- "off": + The verity xattr is never used. This is the default if verity + option is not specified. +- "on": + Whenever a metacopy files specifies an expected digest, the + corresponding data file must match the specified digest. + When generating a metacopy file the verity xattr will be set + from the source file fs-verity digest (if it has one).
Quoted 9 lines
Documentation/filesystems/overlayfs.rst when a "metacopy" file in one of the lower layers above it, has a "redirect" · line 434
Quoted 5 lines
+- "require":
Documentation/filesystems/overlayfs.rst +4
-0+ Same as "on", but additionally all metacopy files must specify a + verity xattr. This means metadata copy up will only be used if + the data file has fs-verity enabled, otherwise a full copy-up is + used.It looks like my request for improved documentation was not taken, which is unfortunate and makes this patchset difficult to review.
Which one?
IIRC, you had two requests.
One very broad to get the overlayfs.rst document up-to-date:
[1] https://lore.kernel.org/linux-unionfs/[email protected]/
But I assume you mean the specific request about this sentence:
[2] https://lore.kernel.org/linux-unionfs/[email protected]/
I must say, I re-read this paragraph and the email thread and I don't understand what it is that you are asking for.
I am not trying to dismiss your request, I am trying to understand. I understand that overlayfs.rst is not a high standard spec and that "metacopy files" is not well defined and I am not saying that we cannot improve the documentation.
But since you already spent the extra time understanding the inaccuracies of overlayfs.rst during v2 review and we already tried to explain the details of metacopy in v2 review, please elaborate what information is actually missing that made the review of v3 harder for you, so that we can try to improve those parts.
Let me try to tell this story in a different way
Imagine a feature:
send/recv email attachments by reference
instead of attaching files, drop them in a network share and send/recv a URL to that share with optional verity sig.
verity=off:
do not include verity sig when sending emails
do not verify verity sig when receiving emails
verity=on:
include verity sig when sending emails (if available) verify verity sig if it is included in received email
verity=require:
same as verity=on, but if verity sig is not included in received email, do not follow URL.
if verity sig not available when sending, attach the file itself instead of the URL -
that is what "otherwise a full copy-up" means in the context above
To me this sounds exactly like the documentation above. Is this story clear?
If it is, what in the doc above is missing to make it also clear?
Thanks,
Amir.
Eric Biggers
maintainerRe: [PATCH v3 2/4] ovl: Add framework for verity support
Quoted 34 lines · Amir Goldstein
Quoted 23 lines · Eric Biggers
Quoted 21 lines · Alexander Larsson
Documentation/filesystems/overlayfs.rst +21
-0+fs-verity support +---------------------- + +When metadata copy up is used for a file, then the xattr +"trusted.overlay.verity" may be set on the metacopy file. This +specifies the expected fs-verity digest of the lowerdata file. This +may then be used to verify the content of the source file at the time +the file is opened. During metacopy copy up overlayfs can also set +this xattr. + +This is controlled by the "verity" mount option, which supports +these values: + +- "off": + The verity xattr is never used. This is the default if verity + option is not specified. +- "on": + Whenever a metacopy files specifies an expected digest, the + corresponding data file must match the specified digest. + When generating a metacopy file the verity xattr will be set + from the source file fs-verity digest (if it has one).Quoted 9 lines
Documentation/filesystems/overlayfs.rst when a "metacopy" file in one of the lower layers above it, has a "redirect" · line 434
Quoted 5 lines
+- "require":
Documentation/filesystems/overlayfs.rst +4
-0+ Same as "on", but additionally all metacopy files must specify a + verity xattr. This means metadata copy up will only be used if + the data file has fs-verity enabled, otherwise a full copy-up is + used.It looks like my request for improved documentation was not taken, which is unfortunate and makes this patchset difficult to review.
Quoted 4 lines
Which one?
IIRC, you had two requests.
One very broad to get theoverlayfs.rstdocument up-to-date:
[1] https://lore.kernel.org/linux-unionfs/[email protected]/
That isn't an accurate summary of what I said. I actually pointed out two specific things that are confusing specifically in the context of this feature.
Quoted 2 lines
But I assume you mean the specific request about this sentence:
[2] https://lore.kernel.org/linux-unionfs/[email protected]/
And that was a third specific thing. I got a detailed response back
(https://lore.kernel.org/linux-unionfs/CAL7ro1GGAfdZG9cHDWE2vnhY5tSE=9MxYi_n_gJHRfaw7zMSgg@mail.gmail.com),
which was helpful. Unfortunately, the information in that response hasn't yet found its way into the documentation that is being proposed.
In general the proposed documentation reads like the audience is overlayfs developers. It doesn't describe the motivation for the feature or how to use it in each of the two use cases. Maybe that is intended, but it's not what I had expected to see.
Side note: the use of the passive voice, e.g. "the xattr may be set" and "the xattr may then be used to verify", should be avoided since it makes it unclear who/what is doing these actions.
- Eric
Re: [PATCH v3 2/4] ovl: Add framework for verity support
Quoted 52 lines · Eric Biggers
Quoted 34 lines · Amir Goldstein
Quoted 23 lines · Eric Biggers
Quoted 21 lines · Alexander Larsson
Documentation/filesystems/overlayfs.rst +21
-0+fs-verity support +---------------------- + +When metadata copy up is used for a file, then the xattr +"trusted.overlay.verity" may be set on the metacopy file. This +specifies the expected fs-verity digest of the lowerdata file. This +may then be used to verify the content of the source file at the time +the file is opened. During metacopy copy up overlayfs can also set +this xattr. + +This is controlled by the "verity" mount option, which supports +these values: + +- "off": + The verity xattr is never used. This is the default if verity + option is not specified. +- "on": + Whenever a metacopy files specifies an expected digest, the + corresponding data file must match the specified digest. + When generating a metacopy file the verity xattr will be set + from the source file fs-verity digest (if it has one).Quoted 9 lines
Documentation/filesystems/overlayfs.rst when a "metacopy" file in one of the lower layers above it, has a "redirect" · line 434
Quoted 5 lines
+- "require":
Documentation/filesystems/overlayfs.rst +4
-0+ Same as "on", but additionally all metacopy files must specify a + verity xattr. This means metadata copy up will only be used if + the data file has fs-verity enabled, otherwise a full copy-up is + used.It looks like my request for improved documentation was not taken, which is unfortunate and makes this patchset difficult to review.
Quoted 4 lines
Which one?
IIRC, you had two requests.
One very broad to get theoverlayfs.rstdocument up-to-date:
[1] https://lore.kernel.org/linux-unionfs/[email protected]/That isn't an accurate summary of what I said. I actually pointed out two specific things that are confusing specifically in the context of this feature.
Quoted 2 lines
But I assume you mean the specific request about this sentence:
[2] https://lore.kernel.org/linux-unionfs/[email protected]/And that was a third specific thing. I got a detailed response back
(https://lore.kernel.org/linux-unionfs/CAL7ro1GGAfdZG9cHDWE2vnhY5tSE=9MxYi_n_gJHRfaw7zMSgg@mail.gmail.com),
which was helpful. Unfortunately, the information in that response hasn't yet found its way into the documentation that is being proposed.
Quoted 4 lines
In general the proposed documentation reads like the audience is overlayfs developers. It doesn't describe the motivation for the feature or how to use it in each of the two use cases. Maybe that is intended, but it's not what I had expected to see.
I think your complaint is mostly related to the scope/goal of the overlayfs.rst file. The way it is currently written, and how the patch adds to it, it describes purely how overlayfs works "natively".
For example, when you change just the permissions of a file from the lower layer, then overlayfs generates a metacopy file with some special xattrs in the upper dir (rather than a full copy). It can also do things like set a redirect xattr on the metacopy file if the file changes name in the upper (due to a rename for example).
With the patch it may (depending on options) also set a verity xattr on the metacopy file so that later uses of the upper layer can ensure that the lower layer content file hasn't changed. This allows you to improve the robustness of overlayfs layers, such that if at a later time the lower directories accidentally change we will detect this when using the metacopy file rather than delivering the wrong data.
These kinds of uses are completely standalone, you just mount overlay with the right options and the kernel will do the right thing. There is no need to know the internal details of the xattrs.
The problem is that I also have a different usecase with composefs. This involves a completely different way of using overlayfs where you construct manually a filesystem with the xattrs that overlayfs reads, and then you mount this in a very special way (using a read-only filesystem on loopback). A combination of overlayfs features will give certain properties that are useful.
I feel like you expect that the overlayfs.rst document should describe the details of the composefs setup, but instead it (both the patch and the existing document) mainly describes the other kind of use.
In other words, if you are interested in using overlayfs with verity to increase the robustness against layer changes, then the document is probably enough. However, If you're interested in using overlayfs to implement more complex features like composefs, then the document is very weak in all sorts of details, not limited to the fs-verity part.
So, I don't think the current document is bad for what it does. But it should perhaps be amended with a more detailed description of the internals of the xattrs and their behaviour so that advanced users (i.e. developers) don't have to read the source code for such details. That seems like a general critique though, and not necessarily related to this particular patchset.
Quoted 1 line
Side note: the use of the passive voice, e.g. "the xattr may be set" and "the
xattr may then be used to verify", should be avoided since it makes it unclear who/what is doing these actions.
I'm not a native english speaker so I'm sure this can be written in a better way, but when it says:
When metadata copy up is used for a file, then the xattr
"trusted.overlay.verity" may be set on the metacopy file.
It really means:
In certain circumstances, depending on mount options and other
details, it may be the case that when a metacopy file is being created by overlayfs (as a side effect of a filesystem operation) the xattr "trusted.overlay.verity" will be set on the new metacopy file.
So, this is not really using "may" in the passive voice, but I see that it can be read that way. I'll try to reword this in a way that is more precise.
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]Amir Goldstein
maintainerRe: [PATCH v3 2/4] ovl: Add framework for verity support
In reply to Eric BiggersQuoted 44 lines · Eric Biggers
Quoted 34 lines · Amir Goldstein
Quoted 23 lines · Eric Biggers
Quoted 21 lines · Alexander Larsson
Documentation/filesystems/overlayfs.rst +21
-0+fs-verity support +---------------------- + +When metadata copy up is used for a file, then the xattr +"trusted.overlay.verity" may be set on the metacopy file. This +specifies the expected fs-verity digest of the lowerdata file. This +may then be used to verify the content of the source file at the time +the file is opened. During metacopy copy up overlayfs can also set +this xattr. + +This is controlled by the "verity" mount option, which supports +these values: + +- "off": + The verity xattr is never used. This is the default if verity + option is not specified. +- "on": + Whenever a metacopy files specifies an expected digest, the + corresponding data file must match the specified digest. + When generating a metacopy file the verity xattr will be set + from the source file fs-verity digest (if it has one).Quoted 9 lines
Documentation/filesystems/overlayfs.rst when a "metacopy" file in one of the lower layers above it, has a "redirect" · line 434
Quoted 5 lines
+- "require":
Documentation/filesystems/overlayfs.rst +4
-0+ Same as "on", but additionally all metacopy files must specify a + verity xattr. This means metadata copy up will only be used if + the data file has fs-verity enabled, otherwise a full copy-up is + used.It looks like my request for improved documentation was not taken, which is unfortunate and makes this patchset difficult to review.
Quoted 4 lines
Which one?
IIRC, you had two requests.
One very broad to get theoverlayfs.rstdocument up-to-date:
[1] https://lore.kernel.org/linux-unionfs/[email protected]/That isn't an accurate summary of what I said. I actually pointed out two specific things that are confusing specifically in the context of this feature.
Quoted 7 lines
Quoted 2 lines
But I assume you mean the specific request about this sentence:
[2] https://lore.kernel.org/linux-unionfs/[email protected]/And that was a third specific thing. I got a detailed response back
(https://lore.kernel.org/linux-unionfs/CAL7ro1GGAfdZG9cHDWE2vnhY5tSE=9MxYi_n_gJHRfaw7zMSgg@mail.gmail.com),
which was helpful. Unfortunately, the information in that response hasn't yet found its way into the documentation that is being proposed.
This response is mostly about composefs and as Alex wrote these details have no place in overlayfs.rst and not related to the proposed feature, which stands alone even without composefs.
Quoted 3 lines
In general the proposed documentation reads like the audience is overlayfs developers.
I never considered that overlayfs.rst is for an audience other than overlayfs developers or people that want to become overlayfs developers. It is not a user guide. If it were, it would have been a very bad one.
Quoted 4 lines
It doesn't describe the motivation for the feature or how to use it in each of the two use cases. Maybe that is intended, but it's not what I had expected to see.
Yeh, that's a valid point.
That is what I wanted to know - what exactly is missing. I guess this is the documented motivation:
"This may then be used to verify the content of the source file at the time the file is opened"
but it does not tell a complete chain of trust story.
How about something along the lines of:
"In the case that the upper layer can be trusted not to be tampered with while overlayfs is offline and some of the lower layers cannot be trusted not to be tampered with, the "verity" feature can protect against offline modification to lower files, whose metadata has been copied up to the upper layer (a.k.a "metacopy" files) ...."
It's generic language that what the patches do, regardless of the trust model of composefs and how it composes an overlayfs layers.
Quoted 4 lines
Side note: the use of the passive voice, e.g. "the xattr may be set" and "the xattr may then be used to verify", should be avoided since it makes it unclear who/what is doing these actions.
I am not a native English speaker and bad at documentation in general, so the only thing I can say is the passive-aggressive "patches welcome" ;-)
Thanks,
Amir.
Eric Biggers
maintainerRe: [PATCH v3 2/4] ovl: Add framework for verity support
Quoted 9 lines · Amir Goldstein
Quoted 3 lines
In general the proposed documentation reads like the audience is overlayfs developers.
I never considered that
overlayfs.rstis for an audience other than overlayfs developers or people that want to become overlayfs developers. It is not a user guide. If it were, it would have been a very bad one.
Quoted 8 lines
Quoted 4 lines
It doesn't describe the motivation for the feature or how to use it in each of the two use cases. Maybe that is intended, but it's not what I had expected to see.
Yeh, that's a valid point.
That is what I wanted to know - what exactly is missing. I guess this is the documented motivation:
Sure, but even if the document is just for kernel developers, it should still describe motivation and use cases, as those are important for userstanding.
Quoted 9 lines
"This may then be used to verify the content of the source file at the time the file is opened"
but it does not tell a complete chain of trust story.
How about something along the lines of:
"In the case that the upper layer can be trusted not to be tampered with while overlayfs is offline
So *online* tampering of the upper layer is fine?
Quoted 3 lines
and some of the lower layers cannot
be trusted not to be tampered with, the "verity" feature can protect against offline modification to lower files
Data of lower files, not simply "lower files", right?
Are *online* modifications to lower files indeed not in scope?
If the feature "can protect", then under what circumstances does it protect, and under what circumstances what does it not protect?
It would also be helpful to explain what specifically is meant by "protect". Does it mean that overlayfs prevents modifications to lower file data, or does it mean that overlayfs detects modifications to lower file data after they happen? If the latter, what happens when overlayfs detects a modification? What do userspace programs experience?
Quoted 5 lines
, whose metadata has been
copied up to the upper layer (a.k.a "metacopy" files) ...."It's generic language that what the patches do, regardless of the trust model of composefs and how it composes an overlayfs layers.
It's better, but it could use some more detail. See my comments above.
- Eric
Amir Goldstein
maintainerRe: [PATCH v3 2/4] ovl: Add framework for verity support
Quoted 26 lines · Eric Biggers
Quoted 9 lines · Amir Goldstein
Quoted 3 lines
In general the proposed documentation reads like the audience is overlayfs developers.
I never considered that
overlayfs.rstis for an audience other than overlayfs developers or people that want to become overlayfs developers. It is not a user guide. If it were, it would have been a very bad one.Quoted 8 lines
Quoted 4 lines
It doesn't describe the motivation for the feature or how to use it in each of the two use cases. Maybe that is intended, but it's not what I had expected to see.
Yeh, that's a valid point.
That is what I wanted to know - what exactly is missing. I guess this is the documented motivation:Sure, but even if the document is just for kernel developers, it should still describe motivation and use cases, as those are important for userstanding.
Quoted 3 lines
"This may then be used to verify the content of the source file at the time the file is opened"
Quoted 8 lines
Quoted 6 lines
but it does not tell a complete chain of trust story.
How about something along the lines of:
"In the case that the upper layer can be trusted not to be tampered with while overlayfs is offline
So *online* tampering of the upper layer is fine?
No, for the sake of this section, it would be easier to say that upper is completely trusted and completely under our control and that the feature only hardens overlayfs when lower is not under our full control.
In the case of composefs it's actually two lower layers, one trusted and one not trusted (and an optional trusted upper), but it does not matter IMO for the sake of explaining the basic feature.
Quoted 6 lines
Quoted 4 lines
and some of the lower layers cannot
be trusted not to be tampered with, the "verity" feature can protect against offline modification to lower filesData of lower files, not simply "lower files", right?
Right.
Quoted 2 lines
Are *online* modifications to lower files indeed not in scope?
They are. doc should not mention online.
Quoted 10 lines
If the feature "can protect", then under what circumstances does it protect, and under what circumstances what does it not protect?
It would also be helpful to explain what specifically is meant by "protect". Does it mean that overlayfs prevents modifications to lower file data, or does it mean that overlayfs detects modifications to lower file data after they happen? If the latter, what happens when overlayfs detects a modification? What do userspace programs experience?
Quoted 8 lines
Quoted 5 lines
, whose metadata has been
copied up to the upper layer (a.k.a "metacopy" files) ...."It's generic language that what the patches do, regardless of the trust model of composefs and how it composes an overlayfs layers.
It's better, but it could use some more detail. See my comments above.
Fair enough.
Alex,
Please incorporate Eric's feedback above to come up with a more detailed description than:
+This can be useful as a way to validate that a lower directory matches +the correct upper when it is re-mounted later. In case you can +guarantee that a overlayfs directory with verity xattrs is not +tampered with (for example using dm-verity or fs-verity) this can even +be used to guarantee the validity of an untrusted lower directory. +
On the specific points that Eric mentioned.
Thanks,
Amir.
Re: [PATCH v3 2/4] ovl: Add framework for verity support
Quoted 93 lines · Amir Goldstein
Quoted 26 lines · Eric Biggers
Quoted 9 lines · Amir Goldstein
Quoted 3 lines
In general the proposed documentation reads like the audience is overlayfs developers.
I never considered that
overlayfs.rstis for an audience other than overlayfs developers or people that want to become overlayfs developers. It is not a user guide. If it were, it would have been a very bad one.Quoted 8 lines
Quoted 4 lines
It doesn't describe the motivation for the feature or how to use it in each of the two use cases. Maybe that is intended, but it's not what I had expected to see.
Yeh, that's a valid point.
That is what I wanted to know - what exactly is missing. I guess this is the documented motivation:Sure, but even if the document is just for kernel developers, it should still describe motivation and use cases, as those are important for userstanding.
Quoted 3 lines
"This may then be used to verify the content of the source file at the time the file is opened"
Quoted 8 lines
Quoted 6 lines
but it does not tell a complete chain of trust story.
How about something along the lines of:
"In the case that the upper layer can be trusted not to be tampered with while overlayfs is offline
So *online* tampering of the upper layer is fine?
No, for the sake of this section, it would be easier to say that upper is completely trusted and completely under our control and that the feature only hardens overlayfs when lower is not under our full control.
In the case of composefs it's actually two lower layers, one trusted and one not trusted (and an optional trusted upper), but it does not matter IMO for the sake of explaining the basic feature.
Quoted 6 lines
Quoted 4 lines
and some of the lower layers cannot
be trusted not to be tampered with, the "verity" feature can protect against offline modification to lower filesData of lower files, not simply "lower files", right?
Right.
Quoted 2 lines
Are *online* modifications to lower files indeed not in scope?
They are. doc should not mention online.
Quoted 10 lines
If the feature "can protect", then under what circumstances does it protect, and under what circumstances what does it not protect?
It would also be helpful to explain what specifically is meant by "protect". Does it mean that overlayfs prevents modifications to lower file data, or does it mean that overlayfs detects modifications to lower file data after they happen? If the latter, what happens when overlayfs detects a modification? What do userspace programs experience?
Quoted 8 lines
Quoted 5 lines
, whose metadata has been
copied up to the upper layer (a.k.a "metacopy" files) ...."It's generic language that what the patches do, regardless of the trust model of composefs and how it composes an overlayfs layers.
It's better, but it could use some more detail. See my comments above.
Fair enough.
Alex,
Please incorporate Eric's feedback above to come up with a more detailed description than:
(quoted diff) +6
-0+This can be useful as a way to validate that a lower directory matches +the correct upper when it is re-mounted later. In case you can +guarantee that a overlayfs directory with verity xattrs is not +tampered with (for example using dm-verity or fs-verity) this can even +be used to guarantee the validity of an untrusted lower directory. +
Quoted 1 line
On the specific points that Eric mentioned.
I think maybe it's best we hash this out here first. What do you think about this version:
fs-verity support
----------------------
During metadata copy up of a lower file, if the source file has fs-verity enabled and overlay verity support is enabled, then the "trusted.overlay.verity" xattr is set on the new metacopy file. This specifies the expected fs-verity digest of the lowerdata file, which is used to verify the content of the lower file at the time the metacopy file is opened.
When a layer containing verity xattrs is used, it means that any such metacopy file in the upper layer is guaranteed to match the content that was in the lower at the time of the copy-up. If at any time (during a mount, after a remount, etc) such a file in the lower is replaced or modified in any way, then opening the corresponding file on overlayfs will result in EIO and a detailed error printed to the kernel logs. (Actually, due to caching the overlayfs mount might still see the previous file contents after a lower file is replaced under an active mount, but it will never see the wrong data.)
Verity can be used as a general robustness check to detect accidental changes in the overlayfs directories in use. But, with additional care it can also give more powerful guarantees. For example, if the upper layer is fully trusted (by using dm-verity or something similar), then an untrusted lower layer can be used to supply validated file content for all metacopy files. If additionally the untrusted lower directories are specified as "Data-only", then they can only supply such file content, and the entire mount can be trusted to match the upper layer.
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]Eric Biggers
maintainerRe: [PATCH v3 2/4] ovl: Add framework for verity support
Quoted 9 lines · Alexander Larsson
When a layer containing verity xattrs is used, it means that any such metacopy file in the upper layer is guaranteed to match the content that was in the lower at the time of the copy-up. If at any time (during a mount, after a remount, etc) such a file in the lower is replaced or modified in any way, then opening the corresponding file on overlayfs will result in EIO and a detailed error printed to the kernel logs. (Actually, due to caching the overlayfs mount might still see the previous file contents after a lower file is replaced under an active mount, but it will never see the wrong data.)
Well, the key point of fsverity is that data is not verified until it is actually read. At open time, the fsverity file digest is made available in constant time, and overlayfs will verify that. However, invalid data blocks are not reported until the data is actually read. The error that applications get is EIO for syscalls, and SIGBUS for memory-mapped reads, as mentioned at
https://www.kernel.org/doc/html/latest/filesystems/fsverity.html#accessing-verity-files
So overlayfs might report EIO at open time, or it might not report an error until the modified data is read. And in the latter case, presumably the error seen by the application matches the one for using fsverity natively?
You can link to the fsverity documentation somewhere if it would be helpful, but I'd still like the semantics of how this works on overlayfs to be documented.
- Eric
Re: [PATCH v3 2/4] ovl: Add framework for verity support
Quoted 19 lines · Eric Biggers
Quoted 9 lines · Alexander Larsson
When a layer containing verity xattrs is used, it means that any such metacopy file in the upper layer is guaranteed to match the content that was in the lower at the time of the copy-up. If at any time (during a mount, after a remount, etc) such a file in the lower is replaced or modified in any way, then opening the corresponding file on overlayfs will result in EIO and a detailed error printed to the kernel logs. (Actually, due to caching the overlayfs mount might still see the previous file contents after a lower file is replaced under an active mount, but it will never see the wrong data.)
Well, the key point of fsverity is that data is not verified until it is actually read. At open time, the fsverity file digest is made available in constant time, and overlayfs will verify that. However, invalid data blocks are not reported until the data is actually read. The error that applications get is EIO for syscalls, and SIGBUS for memory-mapped reads, as mentioned at
https://www.kernel.org/doc/html/latest/filesystems/fsverity.html#accessing-verity-files
Quoted 3 lines
So overlayfs might report EIO at open time, or it might not report an error until the modified data is read. And in the latter case, presumably the error seen by the application matches the one for using fsverity natively?
Yes, I'm aware of that, but do we need to describe this in the overlayfs documentation?
The text I wrote is describing the behaviour that overlayfs adds to the mix, and I sort of
assumed the late validation from fs-verity itself would be known about if the file already
has fs-verity enabled.
Quoted 2 lines
You can link to the fsverity documentation somewhere if it would be helpful, but I'd still like the semantics of how this works on overlayfs to be documented.
I guess just adding a link to that is not that bad. What about:
----
When a layer containing verity xattrs is used, it means that any such metacopy file in the upper layer is guaranteed to match the content that was in the lower at the time of the copy-up. If at any time (during a mount, after a remount, etc) such a file in the lower is replaced or modified in any way, then opening the corresponding file on overlayfs will result in EIO and a detailed error printed to the kernel logs. (Actually, due to caching the overlayfs mount might still see the previous file contents after a lower file is replaced under an active mount, but it will never see the wrong data.) In addition fs-verity will do late validation of the file content, as described in :ref:Documentation/filesystems/fsverity.rst <accessing_verity_files>.
---
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]Eric Biggers
maintainerRe: [PATCH v3 2/4] ovl: Add framework for verity support
Quoted 37 lines · Alexander Larsson
Quoted 19 lines · Eric Biggers
Quoted 9 lines · Alexander Larsson
When a layer containing verity xattrs is used, it means that any such metacopy file in the upper layer is guaranteed to match the content that was in the lower at the time of the copy-up. If at any time (during a mount, after a remount, etc) such a file in the lower is replaced or modified in any way, then opening the corresponding file on overlayfs will result in EIO and a detailed error printed to the kernel logs. (Actually, due to caching the overlayfs mount might still see the previous file contents after a lower file is replaced under an active mount, but it will never see the wrong data.)
Well, the key point of fsverity is that data is not verified until it is actually read. At open time, the fsverity file digest is made available in constant time, and overlayfs will verify that. However, invalid data blocks are not reported until the data is actually read. The error that applications get is EIO for syscalls, and SIGBUS for memory-mapped reads, as mentioned at
https://www.kernel.org/doc/html/latest/filesystems/fsverity.html#accessing-verity-filesQuoted 3 lines
So overlayfs might report EIO at open time, or it might not report an error until the modified data is read. And in the latter case, presumably the error seen by the application matches the one for using fsverity natively?
Yes, I'm aware of that, but do we need to describe this in the overlayfs documentation?
The text I wrote is describing the behaviour that overlayfs adds to the mix, and I sort of
assumed the late validation from fs-verity itself would be known about if the file already
has fs-verity enabled.Quoted 2 lines
You can link to the fsverity documentation somewhere if it would be helpful, but I'd still like the semantics of how this works on overlayfs to be documented.
I guess just adding a link to that is not that bad. What about:
Quoted 13 lines
----
When a layer containing verity xattrs is used, it means that any such metacopy file in the upper layer is guaranteed to match the content that was in the lower at the time of the copy-up. If at any time (during a mount, after a remount, etc) such a file in the lower is replaced or modified in any way, then opening the corresponding file on overlayfs will result in EIO and a detailed error printed to the kernel logs. (Actually, due to caching the overlayfs mount might still see the previous file contents after a lower file is replaced under an active mount, but it will never see the wrong data.) In addition fs-verity will do late validation of the file content, as described in :ref:Documentation/filesystems/fsverity.rst <accessing_verity_files>.
That still has the incorrect statement "If at any time (during a mount, after a remount, etc) such a file in the lower is replaced or modified in any way, then opening the corresponding file on overlayfs will result in EIO and a detailed error printed to the kernel logs." See my last email where I explained why that statement is not correct.
- Eric
Re: [PATCH v3 2/4] ovl: Add framework for verity support
Quoted 53 lines · Eric Biggers
Quoted 37 lines · Alexander Larsson
Quoted 19 lines · Eric Biggers
Quoted 9 lines · Alexander Larsson
When a layer containing verity xattrs is used, it means that any such metacopy file in the upper layer is guaranteed to match the content that was in the lower at the time of the copy-up. If at any time (during a mount, after a remount, etc) such a file in the lower is replaced or modified in any way, then opening the corresponding file on overlayfs will result in EIO and a detailed error printed to the kernel logs. (Actually, due to caching the overlayfs mount might still see the previous file contents after a lower file is replaced under an active mount, but it will never see the wrong data.)
Well, the key point of fsverity is that data is not verified until it is actually read. At open time, the fsverity file digest is made available in constant time, and overlayfs will verify that. However, invalid data blocks are not reported until the data is actually read. The error that applications get is EIO for syscalls, and SIGBUS for memory-mapped reads, as mentioned at
https://www.kernel.org/doc/html/latest/filesystems/fsverity.html#accessing-verity-filesQuoted 3 lines
So overlayfs might report EIO at open time, or it might not report an error until the modified data is read. And in the latter case, presumably the error seen by the application matches the one for using fsverity natively?
Yes, I'm aware of that, but do we need to describe this in the overlayfs documentation?
The text I wrote is describing the behaviour that overlayfs adds to the mix, and I sort of
assumed the late validation from fs-verity itself would be known about if the file already
has fs-verity enabled.Quoted 2 lines
You can link to the fsverity documentation somewhere if it would be helpful, but I'd still like the semantics of how this works on overlayfs to be documented.
I guess just adding a link to that is not that bad. What about:
Quoted 13 lines
----
When a layer containing verity xattrs is used, it means that any such metacopy file in the upper layer is guaranteed to match the content that was in the lower at the time of the copy-up. If at any time (during a mount, after a remount, etc) such a file in the lower is replaced or modified in any way, then opening the corresponding file on overlayfs will result in EIO and a detailed error printed to the kernel logs. (Actually, due to caching the overlayfs mount might still see the previous file contents after a lower file is replaced under an active mount, but it will never see the wrong data.) In addition fs-verity will do late validation of the file content, as described in :ref:Documentation/filesystems/fsverity.rst <accessing_verity_files>.
Quoted 5 lines
That still has the incorrect statement "If at any time (during a mount, after a remount, etc) such a file in the lower is replaced or modified in any way, then opening the corresponding file on overlayfs will result in EIO and a detailed error printed to the kernel logs." See my last email where I explained why that statement is not correct.
If the modification of the file happens via the kernel vfs API, then the new backing file will have either have no, or the wrong fs-verity digest, which will be reported by overlayfs on open. It is true that a modification on the block level which the vfs is unaware of will be reported at read time, by fs-verity (i.e. independent of overlayfs). This is what I meant by the "in addition .." part, but maybe that is not clear. I'll try to rephrase it.
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]PATCH v3 3/4 ovl: Validate verity xattr when resolving lowerdata
In reply to Alexander LarssonWhen accessing file data the first time we check the overlay.verity xattr on the metadata inode, and if set verify that the source lowerdata inode matches it (according to the verity options enabled). If the verity check passes we store this info in the inode flags as OVL_VERIFIED, so that we can avoid doing it again if the inode remains in memory.
The verification is done in ovl_maybe_validate_verity() which needs to be called in the same places as ovl_maybe_lookup_lowerdata(), so there is a new ovl_verify_lowerdata() helper that calls these in the right order, and all current callers of ovl_maybe_lookup_lowerdata() are changed to call it instead.
--- fs/overlayfs/copy_up.c | 2 +- fs/overlayfs/file.c | 8 ++-- fs/overlayfs/namei.c | 54 +++++++++++++++++++++- fs/overlayfs/overlayfs.h | 9 +++- fs/overlayfs/super.c | 5 ++- fs/overlayfs/util.c | 96 ++++++++++++++++++++++++++++++++++++++++ 6 files changed, 166 insertions(+), 8 deletions(-)
fs/overlayfs/copy_up.c +1 -1
@@ -1078,7 +1078,7 @@ static int ovl_copy_up_flags(struct dentry *dentry, int flags)* not very important to optimize this case, so do lazy lowerdata lookup * before any copy up, so we can do it before taking ovl_inode_lock(). */ - err =ovl_maybe_lookup_lowerdata (dentry); + err =ovl_verify_lowerdata (dentry);if (err)return err;
fs/overlayfs/file.c +4 -4
@@ -115,8 +115,8 @@ static int ovl_real_fdget_meta(const struct file *file, struct fd *real,if (allow_meta) {ovl_path_real (dentry, &realpath); }else { -/* lazy lookup of lowerdata */ - err =ovl_maybe_lookup_lowerdata (dentry); +/* lazy lookup and verify of lowerdata */ + err =ovl_verify_lowerdata (dentry);if (err)return err;
@@ -159,8 +159,8 @@ static int ovl_open(struct inode *inode, struct file *file)struct path realpath;int err; -/* lazy lookup of lowerdata */ - err =ovl_maybe_lookup_lowerdata (dentry); +/* lazy lookup and verify lowerdata */ + err =ovl_verify_lowerdata (dentry);if (err)return err;
fs/overlayfs/namei.c +53 -1
67 lines not on this page · raw message
fs/overlayfs/overlayfs.h +8 -1
@@ -38,6 +38,7 @@ enum ovl_xattr {OVL_XATTR_UPPER ,OVL_XATTR_METACOPY ,OVL_XATTR_PROTATTR , + OVL_XATTR_VERITY, }; enum ovl_inode_flag {
@@ -49,6 +50,7 @@ enum ovl_inode_flag {OVL_UPPERDATA ,/* Inode number will remain constant over copy up. */ OVL_CONST_INO , + OVL_VERIFIED, }; enum ovl_entry_flag {
@@ -460,6 +462,11 @@ int ovl_lock_rename_workdir(struct dentry *workdir, struct dentry *upperdir);int ovl_check_metacopy_xattr (struct ovl_fs *ofs,const struct path *path);bool ovl_is_metacopy_dentry (struct dentry *dentry);char *ovl_get_redirect_xattr (struct ovl_fs *ofs,const struct path *path,int padding); +int ovl_get_verity_xattr (struct ovl_fs *ofs,const struct path *path, +u8 *digest_buf,int *buf_length); +int ovl_validate_verity (struct ovl_fs *ofs, +struct path *metapath, +struct path *datapath);int ovl_sync_status (struct ovl_fs *ofs);static inline void ovl_set_flag (unsigned long flag,struct inode *inode)
@@ -559,7 +566,7 @@ struct dentry *ovl_get_index_fh(struct ovl_fs *ofs, struct ovl_fh *fh);struct dentry *ovl_lookup_index (struct ovl_fs *ofs,struct dentry *upper,struct dentry *origin,bool verify);int ovl_path_next (int idx,struct dentry *dentry,struct path *path); -int ovl_maybe_lookup_lowerdata (struct dentry *dentry); +int ovl_verify_lowerdata (struct dentry *dentry);struct dentry *ovl_lookup (struct inode *dir,struct dentry *dentry,unsigned int flags);bool ovl_lower_positive (struct dentry *dentry);
fs/overlayfs/super.c +4 -1
@@ -63,6 +63,7 @@ static struct dentry *ovl_d_real(struct dentry *dentry, const struct inode *inode) {struct dentry *real =NULL , *lower; +int err;/* It's an overlay file */ if (inode &&d_inode (dentry) == inode)
@@ -89,7 +90,9 @@ static struct dentry *ovl_d_real(struct dentry *dentry,* uprobes on offset within the file, so lowerdata should be available * when setting the uprobe. */ -ovl_maybe_lookup_lowerdata (dentry); + err =ovl_verify_lowerdata (dentry); +if (err) + goto bug; lower =ovl_dentry_lowerdata (dentry);if (!lower) goto bug;
fs/overlayfs/util.c +96 -0
121 lines not on this page · raw message
Signature · 1 line
2.40.1
Eric Biggers
maintainerRe: [PATCH v3 3/4] ovl: Validate verity xattr when resolving lowerdata
Quoted 41 lines · Alexander Larsson
fs/overlayfs/util.c +41
-0+int ovl_validate_verity (struct ovl_fs *ofs, +struct path *metapath, +struct path *datapath) +{ +u8 xattr_data[1 +FS_VERITY_MAX_DIGEST_SIZE ]; +u8 actual_digest[FS_VERITY_MAX_DIGEST_SIZE ]; +enum hash_algo verity_algo; +int xattr_len; +int err; + +if (!ofs->config.verity || +/* Verity only works on regular files */ + !S_ISREG (d_inode (metapath->dentry)->i_mode)) +return 0 ; + + xattr_len =sizeof (xattr_data); + err =ovl_get_verity_xattr (ofs, metapath, xattr_data, &xattr_len); +if (err == -ENODATA ) { +if (ofs->config.require_verity) { +pr_warn_ratelimited ("metacopy file '%pd' has no overlay.verity xattr\n" , + metapath->dentry); +return -EIO ; + } +return 0 ; + } +if (err <0 ) +return err; + + err =ovl_ensure_verity_loaded (datapath); +if (err <0 ) { +pr_warn_ratelimited ("lower file '%pd' failed to load fs-verity info\n" , + datapath->dentry); +return -EIO ; + } + + err =fsverity_get_digest (d_inode (datapath->dentry), actual_digest, &verity_algo); +if (err <0 ) { +pr_warn_ratelimited ("lower file '%pd' has no fs-verity digest\n" , datapath->dentry); +return -EIO ; + } +
fs/overlayfs/util.c char *ovl_get_redirect_xattr(struct ovl_fs *ofs, const struct path *path, int pa · line 1249
Quoted 10 lines
fs/overlayfs/util.c +10
-0+if (xattr_len !=1 + hash_digest_size[verity_algo] || + xattr_data[0 ] != (u8 ) verity_algo || +memcmp (xattr_data+1 , actual_digest, xattr_len -1 ) !=0 ) { +pr_warn_ratelimited ("lower file '%pd' has the wrong fs-verity digest\n" , + datapath->dentry); +return -EIO ; + } + +return 0 ; +}
This means the overlayfs verity xattr contains the algorithm ID of the fsverity digest as a HASH_ALGO_* value.
That works, but I think HASH_ALGO_* is somewhat of an IMA-ism. fsverity actually uses FS_VERITY_HASH_ALG_* everywhere else, including in the UAPI and in fsverity-utils which includes libfsverity
(https://git.kernel.org/pub/scm/fs/fsverity/fsverity-utils.git/tree/include/libfsverity.h).
I'm guessing that you used HASH_ALGO_* not because you really wanted to, but rather just because it's what fsverity_get_digest() happens to return, as IMA is currently its only user.
Can you consider
https://lore.kernel.org/r/[email protected] which would make fsverity_get_digest() support both types of IDs? Then you can use FS_VERITY_HASH_ALG_*, which I think would make things slightly easier for you.
- Eric
Re: [PATCH v3 3/4] ovl: Validate verity xattr when resolving lowerdata
Quoted 57 lines · Eric Biggers
Quoted 41 lines · Alexander Larsson
fs/overlayfs/util.c +41
-0+int ovl_validate_verity (struct ovl_fs *ofs, +struct path *metapath, +struct path *datapath) +{ +u8 xattr_data[1 +FS_VERITY_MAX_DIGEST_SIZE ]; +u8 actual_digest[FS_VERITY_MAX_DIGEST_SIZE ]; +enum hash_algo verity_algo; +int xattr_len; +int err; + +if (!ofs->config.verity || +/* Verity only works on regular files */ + !S_ISREG (d_inode (metapath->dentry)->i_mode)) +return 0 ; + + xattr_len =sizeof (xattr_data); + err =ovl_get_verity_xattr (ofs, metapath, xattr_data, &xattr_len); +if (err == -ENODATA ) { +if (ofs->config.require_verity) { +pr_warn_ratelimited ("metacopy file '%pd' has no overlay.verity xattr\n" , + metapath->dentry); +return -EIO ; + } +return 0 ; + } +if (err <0 ) +return err; + + err =ovl_ensure_verity_loaded (datapath); +if (err <0 ) { +pr_warn_ratelimited ("lower file '%pd' failed to load fs-verity info\n" , + datapath->dentry); +return -EIO ; + } + + err =fsverity_get_digest (d_inode (datapath->dentry), actual_digest, &verity_algo); +if (err <0 ) { +pr_warn_ratelimited ("lower file '%pd' has no fs-verity digest\n" , datapath->dentry); +return -EIO ; + } +fs/overlayfs/util.c char *ovl_get_redirect_xattr(struct ovl_fs *ofs, const struct path *path, int pa · line 1249
Quoted 10 lines
fs/overlayfs/util.c +10
-0+if (xattr_len !=1 + hash_digest_size[verity_algo] || + xattr_data[0 ] != (u8 ) verity_algo || +memcmp (xattr_data+1 , actual_digest, xattr_len -1 ) !=0 ) { +pr_warn_ratelimited ("lower file '%pd' has the wrong fs-verity digest\n" , + datapath->dentry); +return -EIO ; + } + +return 0 ; +}This means the overlayfs verity xattr contains the algorithm ID of the fsverity digest as a HASH_ALGO_* value.
Quoted 8 lines
That works, but I think HASH_ALGO_* is somewhat of an IMA-ism. fsverity actually uses FS_VERITY_HASH_ALG_* everywhere else, including in the UAPI and in fsverity-utils which includes libfsverity
(https://git.kernel.org/pub/scm/fs/fsverity/fsverity-utils.git/tree/include/libfsverity.h).I'm guessing that you used HASH_ALGO_* not because you really wanted to, but rather just because it's what
fsverity_get_digest()happens to return, as IMA is currently its only user.
Yeah, that is exactly why.
Quoted 4 lines
Can you consider
https://lore.kernel.org/r/[email protected] which would makefsverity_get_digest()support both types of IDs? Then you can use FS_VERITY_HASH_ALG_*, which I think would make things slightly easier for you.
Sounds very good to me. I'll rebase the patchset on top of it. Not sure how to best land this though, are you ok with this landing via overlayfs-next?
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]Eric Biggers
maintainerRe: [PATCH v3 3/4] ovl: Validate verity xattr when resolving lowerdata
Quoted 8 lines · Alexander Larsson
Quoted 4 lines
Can you consider
https://lore.kernel.org/r/[email protected] which would makefsverity_get_digest()support both types of IDs? Then you can use FS_VERITY_HASH_ALG_*, which I think would make things slightly easier for you.Sounds very good to me. I'll rebase the patchset on top of it. Not sure how to best land this though, are you ok with this landing via overlayfs-next?
If you're confident that this series will land in v6.4, then sure, you can apply my patch "fsverity: rework fsverity_get_digest() again" to overlayfs-next, instead of me taking it through fsverity/for-next. (Hopefully the IMA maintainer will ack it as well, as it touches security/integrity/.)
Just be careful about being overly-optimistic about features landing in the next release. I've had experience with cases like this before, where I didn't apply something for a reason like this, but then the series didn't make it in right away so it was worse than me just taking the patch in the first place.
I do see that the other prerequisites were just applied to overlayfs-next, so maybe this is good to go now. It's up to the other overlayfs folks.
- Eric
Eric Biggers
maintainerRe: [PATCH v3 3/4] ovl: Validate verity xattr when resolving lowerdata
Quoted 15 lines · Eric Biggers
Quoted 8 lines · Alexander Larsson
Quoted 4 lines
Can you consider
https://lore.kernel.org/r/[email protected] which would makefsverity_get_digest()support both types of IDs? Then you can use FS_VERITY_HASH_ALG_*, which I think would make things slightly easier for you.Sounds very good to me. I'll rebase the patchset on top of it. Not sure how to best land this though, are you ok with this landing via overlayfs-next?
If you're confident that this series will land in v6.4, then sure, you can apply my patch "fsverity: rework
fsverity_get_digest()again" to overlayfs-next, instead of me taking it through fsverity/for-next. (Hopefully the IMA maintainer will ack it as well, as it touchessecurity/integrity/.)
Quoted 7 lines
Just be careful about being overly-optimistic about features landing in the next release. I've had experience with cases like this before, where I didn't apply something for a reason like this, but then the series didn't make it in right away so it was worse than me just taking the patch in the first place.
I do see that the other prerequisites were just applied to overlayfs-next, so maybe this is good to go now. It's up to the other overlayfs folks.
I meant to say 6.5, not 6.4.
Anyway, just let me know if I should apply it or not, before it gets too late.
- Eric
Amir Goldstein
maintainerRe: [PATCH v3 3/4] ovl: Validate verity xattr when resolving lowerdata
Quoted 8 lines · Eric Biggers
Quoted 6 lines · Eric Biggers
Quoted 5 lines · Alexander Larsson
Quoted 4 lines
Can you consider
https://lore.kernel.org/r/[email protected] which would makefsverity_get_digest()support both types of IDs? Then you can use FS_VERITY_HASH_ALG_*, which I think would make things slightly easier for you.
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Sounds very good to me. I'll rebase the patchset on top of it. Not sure how to best land this though, are you ok with this landing via overlayfs-next?
If you're confident that this series will land in v6.4, then sure, you can apply my patch "fsverity: rework
fsverity_get_digest()again" to overlayfs-next, instead of me taking it through fsverity/for-next. (Hopefully the IMA maintainer will ack it as well, as it touchessecurity/integrity/.)
Mimi,
Can you please take a look:
https://lore.kernel.org/linux-integrity/[email protected]/
Quoted 6 lines
Quoted 6 lines
Just be careful about being overly-optimistic about features landing in the next release. I've had experience with cases like this before, where I didn't apply something for a reason like this, but then the series didn't make it in right away so it was worse than me just taking the patch in the first place.
Quoted 6 lines
Quoted 2 lines
I do see that the other prerequisites were just applied to overlayfs-next, so maybe this is good to go now. It's up to the other overlayfs folks.
I meant to say 6.5, not 6.4.
Anyway, just let me know if I should apply it or not, before it gets too late.
If you want to make sure that your patch lands in 6.5, you'd better take it through your tree.
If you provide a stable branch with the patch, Miklos will be able to merge your branch if he wishes to take Alex's patches for 6.5.
Thanks,
Amir.
Re: [PATCH v3 3/4] ovl: Validate verity xattr when resolving lowerdata
In reply to Eric BiggersQuoted 22 lines · Eric Biggers
Quoted 20 lines · Eric Biggers
Quoted 8 lines · Alexander Larsson
Quoted 4 lines
Can you consider
https://lore.kernel.org/r/[email protected] which would makefsverity_get_digest()support both types of IDs? Then you can use FS_VERITY_HASH_ALG_*, which I think would make things slightly easier for you.Sounds very good to me. I'll rebase the patchset on top of it. Not sure how to best land this though, are you ok with this landing via overlayfs-next?
If you're confident that this series will land in v6.4, then sure, you can apply my patch "fsverity: rework
fsverity_get_digest()again" to overlayfs-next, instead of me taking it through fsverity/for-next. (Hopefully the IMA maintainer will ack it as well, as it touchessecurity/integrity/.)Just be careful about being overly-optimistic about features landing in the next release. I've had experience with cases like this before, where I didn't apply something for a reason like this, but then the series didn't make it in right away so it was worse than me just taking the patch in the first place.
Quoted 6 lines
Quoted 2 lines
I do see that the other prerequisites were just applied to overlayfs-next, so maybe this is good to go now. It's up to the other overlayfs folks.
I meant to say 6.5, not 6.4.
Anyway, just let me know if I should apply it or not, before it gets too late.
Honestly, I have no idea about the timescale here. That is all up to miklos really. Maybe best to do as amir say and take it through your tree but on from a branch that milklos can merge into overlafs-next if he wants to take it this cycle.
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]PATCH v3 4/4 ovl: Handle verity during copy-up
In reply to Alexander LarssonDuring regular metacopy, if lowerdata file has fs-verity enabled, set the new overlay.verity xattr (if enabled).
During real data copy up, remove any old overlay.verity xattr.
If verity is required, and lowerdata does not have fs-verity enabled, fall back to full copy-up (or the generated metacopy would not validate).
--- fs/overlayfs/copy_up.c | 31 +++++++++++++++++++++++++++++++ fs/overlayfs/overlayfs.h | 3 +++ fs/overlayfs/util.c | 39 ++++++++++++++++++++++++++++++++++++++- 3 files changed, 72 insertions(+), 1 deletion(-)
fs/overlayfs/copy_up.c +31 -0
@@ -19,6 +19,7 @@#include <linux/fdtable.h> #include <linux/ratelimit.h> #include <linux/exportfs.h> +#include <linux/fsverity.h> #include "overlayfs.h" #define OVL_COPY_UP_CHUNK_SIZE (1 << 20)
@@ -643,6 +644,18 @@ static int ovl_copy_up_metadata(struct ovl_copy_up_ctx *c, struct dentry *temp)if (c->metacopy) { err =ovl_check_setxattr (ofs, temp,OVL_XATTR_METACOPY ,NULL ,0 , -EOPNOTSUPP ); + +/* Copy the verity digest if any so we can validate the copy-up later */ +if (!err) { +struct path lowerdatapath; + +ovl_path_lowerdata (c->dentry, &lowerdatapath); +if (WARN_ON_ONCE (lowerdatapath.dentry ==NULL )) + err = -EIO ; +else + err =ovl_set_verity_xattr_from (ofs, temp, &lowerdatapath); + } +if (err)return err; }
@@ -918,6 +931,19 @@ static bool ovl_need_meta_copy_up(struct dentry *dentry, umode_t mode,if (flags && ((OPEN_FMODE (flags) &FMODE_WRITE ) || (flags &O_TRUNC )))return false; +/* Fall back to full copy if no fsverity on source data and we require verity */ +if (ofs->config.require_verity) { +struct path lowerdata; + +ovl_path_lowerdata (dentry, &lowerdata); + +if (WARN_ON_ONCE (lowerdata.dentry ==NULL ) || +ovl_ensure_verity_loaded (&lowerdata) || + !fsverity_get_info (d_inode (lowerdata.dentry))) { +return false; + } + } +return true; }
@@ -984,6 +1010,11 @@ static int ovl_copy_up_meta_inode_data(struct ovl_copy_up_ctx *c)if (err) goto out_free; + err =ovl_removexattr (ofs, upperpath.dentry,OVL_XATTR_VERITY ); +if (err && err != -ENODATA ) + goto out_free; + + err =0 ;ovl_set_upperdata (d_inode (c->dentry)); out_free:kfree (capability);
fs/overlayfs/overlayfs.h +3 -0
@@ -462,11 +462,14 @@ int ovl_lock_rename_workdir(struct dentry *workdir, struct dentry *upperdir);int ovl_check_metacopy_xattr (struct ovl_fs *ofs,const struct path *path);bool ovl_is_metacopy_dentry (struct dentry *dentry);char *ovl_get_redirect_xattr (struct ovl_fs *ofs,const struct path *path,int padding); +int ovl_ensure_verity_loaded (struct path *path);int ovl_get_verity_xattr (struct ovl_fs *ofs,const struct path *path,u8 *digest_buf,int *buf_length);int ovl_validate_verity (struct ovl_fs *ofs,struct path *metapath,struct path *datapath); +int ovl_set_verity_xattr_from (struct ovl_fs *ofs,struct dentry *dst, +struct path *src);int ovl_sync_status (struct ovl_fs *ofs);static inline void ovl_set_flag (unsigned long flag,struct inode *inode)
fs/overlayfs/util.c +38 -1
@@ -1174,7 +1174,7 @@ int ovl_get_verity_xattr(struct ovl_fs *ofs, const struct path *path, }/* Call with mounter creds as it may open the file */ -static int ovl_ensure_verity_loaded (struct path *datapath) +int ovl_ensure_verity_loaded (struct path *datapath) {struct inode *inode =d_inode (datapath->dentry);const struct fsverity_info *vi;
@@ -1248,6 +1248,43 @@ int ovl_validate_verity(struct ovl_fs *ofs,return 0 ; } +int ovl_set_verity_xattr_from (struct ovl_fs *ofs,struct dentry *dst, +struct path *src) +{ +int err; +u8 src_digest[1 +FS_VERITY_MAX_DIGEST_SIZE ]; +enum hash_algo verity_algo; + +if (!ofs->config.verity || !S_ISREG (d_inode (dst)->i_mode)) +return 0 ; + + err = -EIO ; +if (src) { + err =ovl_ensure_verity_loaded (src); +if (err <0 ) { +pr_warn_ratelimited ("lower file '%pd' failed to load fs-verity info\n" , + src->dentry); +return -EIO ; + } + + err =fsverity_get_digest (d_inode (src->dentry), src_digest +1 , &verity_algo); + } +if (err == -ENODATA ) { +if (ofs->config.require_verity) { +pr_warn_ratelimited ("lower file '%pd' has no fs-verity digest\n" , + src->dentry); +return -EIO ; + } +return 0 ; + } +if (err <0 ) +return err; + + src_digest[0 ] = (u8 )verity_algo; +return ovl_check_setxattr (ofs, dst,OVL_XATTR_VERITY , + src_digest,1 + hash_digest_size[verity_algo], -EOPNOTSUPP ); +} +/* * ovl_sync_status() - Check fs sync status for volatile mounts *
Signature · 1 line
2.40.1
Amir Goldstein
maintainerRe: [PATCH v3 4/4] ovl: Handle verity during copy-up
Quoted 68 lines · Alexander Larsson
During regular metacopy, if lowerdata file has fs-verity enabled, set the new overlay.verity xattr (if enabled).
During real data copy up, remove any old overlay.verity xattr.
If verity is required, and lowerdata does not have fs-verity enabled, fall back to full copy-up (or the generated metacopy would not validate).
Signed-off-by Alexander Larsson <[email protected]>--- fs/overlayfs/copy_up.c | 31 +++++++++++++++++++++++++++++++ fs/overlayfs/overlayfs.h | 3 +++ fs/overlayfs/util.c | 39 ++++++++++++++++++++++++++++++++++++++- 3 files changed, 72 insertions(+), 1 deletion(-)fs/overlayfs/copy_up.c +26
-0@@ -19,6 +19,7 @@#include <linux/fdtable.h> #include <linux/ratelimit.h> #include <linux/exportfs.h> +#include <linux/fsverity.h> #include "overlayfs.h" #define OVL_COPY_UP_CHUNK_SIZE (1 << 20)@@ -643,6 +644,18 @@ static int ovl_copy_up_metadata(struct ovl_copy_up_ctx *c, struct dentry *temp)if (c->metacopy) { err =ovl_check_setxattr (ofs, temp,OVL_XATTR_METACOPY ,NULL ,0 , -EOPNOTSUPP ); + +/* Copy the verity digest if any so we can validate the copy-up later */ +if (!err) { +struct path lowerdatapath; + +ovl_path_lowerdata (c->dentry, &lowerdatapath); +if (WARN_ON_ONCE (lowerdatapath.dentry ==NULL )) + err = -EIO ; +else + err =ovl_set_verity_xattr_from (ofs, temp, &lowerdatapath); + } +if (err)return err; }@@ -918,6 +931,19 @@ static bool ovl_need_meta_copy_up(struct dentry *dentry, umode_t mode,if (flags && ((OPEN_FMODE (flags) &FMODE_WRITE ) || (flags &O_TRUNC )))return false; +/* Fall back to full copy if no fsverity on source data and we require verity */ +if (ofs->config.require_verity) { +struct path lowerdata; + +ovl_path_lowerdata (dentry, &lowerdata); + +if (WARN_ON_ONCE (lowerdata.dentry ==NULL ) || +ovl_ensure_verity_loaded (&lowerdata) || + !fsverity_get_info (d_inode (lowerdata.dentry))) { +return false; + } + } +return true; }
fs/overlayfs/copy_up.c static int ovl_copy_up_meta_inode_data(struct ovl_copy_up_ctx *c) · line 1014
Quoted 5 lines
fs/overlayfs/copy_up.c +2
-0@@ -984,6 +1010,11 @@ static int ovl_copy_up_meta_inode_data(struct ovl_copy_up_ctx *c)if (err) goto out_free; + err =ovl_removexattr (ofs, upperpath.dentry,OVL_XATTR_VERITY ); +if (err && err != -ENODATA )
-EOPNOTSUPP is also not a failure, although you could do:
if (!ofs->noxattr) ...
to avoid this error
Quoted 85 lines
fs/overlayfs/copy_up.c +3
-0+ goto out_free; + + err =0 ;ovl_set_upperdata (d_inode (c->dentry)); out_free:kfree (capability);fs/overlayfs/overlayfs.h +3
-0@@ -462,11 +462,14 @@ int ovl_lock_rename_workdir(struct dentry *workdir, struct dentry *upperdir);int ovl_check_metacopy_xattr (struct ovl_fs *ofs,const struct path *path);bool ovl_is_metacopy_dentry (struct dentry *dentry);char *ovl_get_redirect_xattr (struct ovl_fs *ofs,const struct path *path,int padding); +int ovl_ensure_verity_loaded (struct path *path);int ovl_get_verity_xattr (struct ovl_fs *ofs,const struct path *path,u8 *digest_buf,int *buf_length);int ovl_validate_verity (struct ovl_fs *ofs,struct path *metapath,struct path *datapath); +int ovl_set_verity_xattr_from (struct ovl_fs *ofs,struct dentry *dst, +struct path *src);int ovl_sync_status (struct ovl_fs *ofs);static inline void ovl_set_flag (unsigned long flag,struct inode *inode)fs/overlayfs/util.c +38
-2@@ -1174,7 +1174,7 @@ int ovl_get_verity_xattr(struct ovl_fs *ofs, const struct path *path, }/* Call with mounter creds as it may open the file */ -static int ovl_ensure_verity_loaded (struct path *datapath) +int ovl_ensure_verity_loaded (struct path *datapath) {struct inode *inode =d_inode (datapath->dentry);const struct fsverity_info *vi;@@ -1248,6 +1248,43 @@ int ovl_validate_verity(struct ovl_fs *ofs,return 0 ; } +int ovl_set_verity_xattr_from (struct ovl_fs *ofs,struct dentry *dst, +struct path *src) +{ +int err; +u8 src_digest[1 +FS_VERITY_MAX_DIGEST_SIZE ]; +enum hash_algo verity_algo; + +if (!ofs->config.verity || !S_ISREG (d_inode (dst)->i_mode)) +return 0 ; + + err = -EIO ; +if (src) { + err =ovl_ensure_verity_loaded (src); +if (err <0 ) { +pr_warn_ratelimited ("lower file '%pd' failed to load fs-verity info\n" , + src->dentry); +return -EIO ; + } + + err =fsverity_get_digest (d_inode (src->dentry), src_digest +1 , &verity_algo); + } +if (err == -ENODATA ) { +if (ofs->config.require_verity) { +pr_warn_ratelimited ("lower file '%pd' has no fs-verity digest\n" , + src->dentry); +return -EIO ; + } +return 0 ; + } +if (err <0 ) +return err; + + src_digest[0 ] = (u8 )verity_algo; +return ovl_check_setxattr (ofs, dst,OVL_XATTR_VERITY , + src_digest,1 + hash_digest_size[verity_algo], -EOPNOTSUPP ); +} +/* * ovl_sync_status() - Check fs sync status for volatile mounts * -- 2.40.1
Quoted 5 lines · Alexander Larsson
This patchset adds support for using fs-verity to validate lowerdata files by specifying an overlay.verity xattr on the metacopy files.
Quoted 7 lines
This is primarily motivated by the Composefs usecase, where there will be a read-only EROFS layer that contains redirect into a base data layer which has fs-verity enabled on all files. However, it is also useful in general if you want to ensure that the lowerdata files matches the expected content over time.
I have also added some tests for this feature to xfstests[1].
I can't remember if there is a good reason why your test does not include verify in a data-only layer.
I think this test coverage needs to be added.
Quoted 9 lines
I'm also CC:ing the fsverity list and maintainers because there is one (tiny) fsverity change, and there may be interest in this usecase.
Changes since v2:
* Rebased on top of overlayfs-next
* We now alway do verity verification the first time the file content is used, rather than doing it at lookup time for the non-lazy lookup case.
Aparat from the minor comment:
for the series.
Thanks,
Amir.
Quoted 36 lines
Changes since v1:
* Rebased on v2 lazy lowerdata series
* Dropped the "validate" mount option variant. We now only support "off", "on" and "require", where "off" is the default.
* We now store the digest algorithm used in the overlay.verity xattr.
* Dropped ability to configure default verity options, as this could cause problems moving layers between machines.
* We now properly resolve dependent mount options by automatically enabling metacopy andredirect_dirif verity is on, or failing if the specified options conflict.
* Streamlined and fixed the handling of creds inovl_ensure_verity_loaded().
* Renamed new helpers fromovl_entry_path_toovl_e_path_[1] https://github.com/alexlarsson/xfstests/commits/verity-tests
Alexander Larsson (4):
fsverity: Exportfsverity_get_digest
ovl: Add framework for verity support
ovl: Validate verity xattr when resolving lowerdata
ovl: Handle verity during copy-upDocumentation/filesystems/overlayfs.rst | 27 +++++ fs/overlayfs/copy_up.c | 33 +++++- fs/overlayfs/file.c | 8 +- fs/overlayfs/namei.c | 54 +++++++++- fs/overlayfs/overlayfs.h | 12 ++- fs/overlayfs/ovl_entry.h | 3 + fs/overlayfs/super.c | 79 +++++++++++++- fs/overlayfs/util.c | 133 ++++++++++++++++++++++++ fs/verity/measure.c | 1 + 9 files changed, 340 insertions(+), 10 deletions(-)--
2.40.1
Quoted 13 lines · Amir Goldstein
Quoted 11 lines · Alexander Larsson
This patchset adds support for using fs-verity to validate lowerdata files by specifying an overlay.verity xattr on the metacopy files.
This is primarily motivated by the Composefs usecase, where there will be a read-only EROFS layer that contains redirect into a base data layer which has fs-verity enabled on all files. However, it is also useful in general if you want to ensure that the lowerdata files matches the expected content over time.
Quoted 6 lines
Quoted 1 line
I have also added some tests for this feature to xfstests[1].
I can't remember if there is a good reason why your test does not include verify in a data-only layer.
I think this test coverage needs to be added.
Yeah. I'll add that.
Quoted 56 lines
Quoted 9 lines
I'm also CC:ing the fsverity list and maintainers because there is one (tiny) fsverity change, and there may be interest in this usecase.
Changes since v2:
* Rebased on top of overlayfs-next
* We now alway do verity verification the first time the file content is used, rather than doing it at lookup time for the non-lazy lookup case.Aparat from the minor comment:
Reviewed-by Amir Goldstein <[email protected]>for the series.
Thanks,
Amir.Quoted 36 lines
Changes since v1:
* Rebased on v2 lazy lowerdata series
* Dropped the "validate" mount option variant. We now only support "off", "on" and "require", where "off" is the default.
* We now store the digest algorithm used in the overlay.verity xattr.
* Dropped ability to configure default verity options, as this could cause problems moving layers between machines.
* We now properly resolve dependent mount options by automatically enabling metacopy andredirect_dirif verity is on, or failing if the specified options conflict.
* Streamlined and fixed the handling of creds inovl_ensure_verity_loaded().
* Renamed new helpers fromovl_entry_path_toovl_e_path_[1] https://github.com/alexlarsson/xfstests/commits/verity-tests
Alexander Larsson (4):
fsverity: Exportfsverity_get_digest
ovl: Add framework for verity support
ovl: Validate verity xattr when resolving lowerdata
ovl: Handle verity during copy-upDocumentation/filesystems/overlayfs.rst | 27 +++++ fs/overlayfs/copy_up.c | 33 +++++- fs/overlayfs/file.c | 8 +- fs/overlayfs/namei.c | 54 +++++++++- fs/overlayfs/overlayfs.h | 12 ++- fs/overlayfs/ovl_entry.h | 3 + fs/overlayfs/super.c | 79 +++++++++++++- fs/overlayfs/util.c | 133 ++++++++++++++++++++++++ fs/verity/measure.c | 1 + 9 files changed, 340 insertions(+), 10 deletions(-)--
2.40.1
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]Quoted 15 lines · Alexander Larsson
Quoted 13 lines · Amir Goldstein
Quoted 11 lines · Alexander Larsson
This patchset adds support for using fs-verity to validate lowerdata files by specifying an overlay.verity xattr on the metacopy files.
This is primarily motivated by the Composefs usecase, where there will be a read-only EROFS layer that contains redirect into a base data layer which has fs-verity enabled on all files. However, it is also useful in general if you want to ensure that the lowerdata files matches the expected content over time.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
I have also added some tests for this feature to xfstests[1].
I can't remember if there is a good reason why your test does not include verify in a data-only layer.
I think this test coverage needs to be added.
Yeah. I'll add that.
Updated the git branch with some lowerdata tests.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]
Amir Goldstein
maintainerQuoted 19 lines · Alexander Larsson
Quoted 17 lines · Alexander Larsson
Quoted 15 lines · Amir Goldstein
Quoted 12 lines · Alexander Larsson
This patchset adds support for using fs-verity to validate lowerdata files by specifying an overlay.verity xattr on the metacopy files.
This is primarily motivated by the Composefs usecase, where there will be a read-only EROFS layer that contains redirect into a base data layer which has fs-verity enabled on all files. However, it is also useful in general if you want to ensure that the lowerdata files matches the expected content over time.
I have also added some tests for this feature to xfstests[1].
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
I can't remember if there is a good reason why your test does not include verify in a data-only layer.
I think this test coverage needs to be added.
Yeah. I'll add that.
Updated the git branch with some lowerdata tests.
Nice!
Thanks,
Amir.
Amir Goldstein
maintainerQuoted 19 lines · Alexander Larsson
Quoted 17 lines · Alexander Larsson
Quoted 15 lines · Amir Goldstein
Quoted 12 lines · Alexander Larsson
This patchset adds support for using fs-verity to validate lowerdata files by specifying an overlay.verity xattr on the metacopy files.
This is primarily motivated by the Composefs usecase, where there will be a read-only EROFS layer that contains redirect into a base data layer which has fs-verity enabled on all files. However, it is also useful in general if you want to ensure that the lowerdata files matches the expected content over time.
I have also added some tests for this feature to xfstests[1].
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
I can't remember if there is a good reason why your test does not include verify in a data-only layer.
I think this test coverage needs to be added.
Yeah. I'll add that.
Updated the git branch with some lowerdata tests.
What do I need to do in order to enable verity on ext4 besides enabling FS_VERITY in the kernel?
I'm getting these on verity tests on ext4 in the default 4k config. _require_scratch_verity() doesn't mention any requirement other that 4K blocks and extent format files.
Thanks,
Amir.
BEGIN TEST 4k (10 tests): Ext4 4k block Wed Jun 14 06:04:25 UTC 2023
DEVICE: /dev/vdbEXT_MKFS_OPTIONS: -b 4096EXT_MOUNT_OPTIONS: -o block_validity
FSTYP -- ext4
PLATFORM -- Linux/x86_64 kvm-xfstests
6.4.0-rc2-xfstests-00026-g35774ba7f07c #1596 SMP PREEMPT_DYNAMIC Tue
Jun 13 18:16:59 IDT 2023
MKFS_OPTIONS -- -F -q -b 4096 /dev/vdcMOUNT_OPTIONS -- -o acl,user_xattr -o block_validity /dev/vdc /vdc
generic/572 [06:04:42] [06:04:47] [not run]
generic/572 -- ext4 verity isn't usable by default with these mkfs options
...
Eric Biggers
maintainerQuoted 17 lines · Amir Goldstein
What do I need to do in order to enable verity on ext4 besides enabling
FS_VERITYin the kernel?I'm getting these on verity tests on ext4 in the default 4k config.
_require_scratch_verity()doesn't mention any requirement other that 4K blocks and extent format files.Thanks,
Amir.BEGIN TEST 4k (10 tests): Ext4 4k block Wed Jun 14 06:04:25 UTC 2023
DEVICE:/dev/vdbEXT_MKFS_OPTIONS: -b 4096EXT_MOUNT_OPTIONS: -oblock_validity
FSTYP -- ext4
PLATFORM -- Linux/x86_64 kvm-xfstests
6.4.0-rc2-xfstests-00026-g35774ba7f07c #1596 SMPPREEMPT_DYNAMICTue
Quoted 7 lines
Jun 13 18:16:59 IDT 2023
MKFS_OPTIONS-- -F -q -b 4096/dev/vdcMOUNT_OPTIONS-- -o acl,user_xattr -oblock_validity/dev/vdc/vdcgeneric/572 [06:04:42] [06:04:47] [not run]
generic/572 -- ext4 verity isn't usable by default with these mkfs options
...
I don't know why it's not working for you. Is there anything helpful in /results/ext4/results-4k/generic/572.full?
- Eric
Quoted 48 lines · Amir Goldstein
Quoted 19 lines · Alexander Larsson
Quoted 17 lines · Alexander Larsson
Quoted 15 lines · Amir Goldstein
Quoted 12 lines · Alexander Larsson
This patchset adds support for using fs-verity to validate lowerdata files by specifying an overlay.verity xattr on the metacopy files.
This is primarily motivated by the Composefs usecase, where there will be a read-only EROFS layer that contains redirect into a base data layer which has fs-verity enabled on all files. However, it is also useful in general if you want to ensure that the lowerdata files matches the expected content over time.
I have also added some tests for this feature to xfstests[1].
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
I can't remember if there is a good reason why your test does not include verify in a data-only layer.
I think this test coverage needs to be added.
Yeah. I'll add that.
Updated the git branch with some lowerdata tests.
What do I need to do in order to enable verity on ext4 besides enabling
FS_VERITYin the kernel?I'm getting these on verity tests on ext4 in the default 4k config.
_require_scratch_verity()doesn't mention any requirement other that 4K blocks and extent format files.Thanks,
Amir.BEGIN TEST 4k (10 tests): Ext4 4k block Wed Jun 14 06:04:25 UTC 2023
DEVICE:/dev/vdbEXT_MKFS_OPTIONS: -b 4096EXT_MOUNT_OPTIONS: -oblock_validity
FSTYP -- ext4
PLATFORM -- Linux/x86_64 kvm-xfstests
6.4.0-rc2-xfstests-00026-g35774ba7f07c #1596 SMPPREEMPT_DYNAMICTue
Quoted 7 lines
Jun 13 18:16:59 IDT 2023
MKFS_OPTIONS-- -F -q -b 4096/dev/vdcMOUNT_OPTIONS-- -o acl,user_xattr -oblock_validity/dev/vdc/vdcgeneric/572 [06:04:42] [06:04:47] [not run]
generic/572 -- ext4 verity isn't usable by default with these mkfs options
...
You need to "tune2fs -O verity" (or pass -O verity to mkfs.ext4).
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]Amir Goldstein
maintainerQuoted 54 lines · Alexander Larsson
Quoted 52 lines · Amir Goldstein
Quoted 19 lines · Alexander Larsson
Quoted 17 lines · Alexander Larsson
Quoted 15 lines · Amir Goldstein
Quoted 12 lines · Alexander Larsson
This patchset adds support for using fs-verity to validate lowerdata files by specifying an overlay.verity xattr on the metacopy files.
This is primarily motivated by the Composefs usecase, where there will be a read-only EROFS layer that contains redirect into a base data layer which has fs-verity enabled on all files. However, it is also useful in general if you want to ensure that the lowerdata files matches the expected content over time.
I have also added some tests for this feature to xfstests[1].
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
I can't remember if there is a good reason why your test does not include verify in a data-only layer.
I think this test coverage needs to be added.
Yeah. I'll add that.
Updated the git branch with some lowerdata tests.
What do I need to do in order to enable verity on ext4 besides enabling
FS_VERITYin the kernel?I'm getting these on verity tests on ext4 in the default 4k config.
_require_scratch_verity()doesn't mention any requirement other that 4K blocks and extent format files.Thanks,
Amir.BEGIN TEST 4k (10 tests): Ext4 4k block Wed Jun 14 06:04:25 UTC 2023
DEVICE:/dev/vdbEXT_MKFS_OPTIONS: -b 4096EXT_MOUNT_OPTIONS: -oblock_validity
FSTYP -- ext4
PLATFORM -- Linux/x86_64 kvm-xfstests
6.4.0-rc2-xfstests-00026-g35774ba7f07c #1596 SMPPREEMPT_DYNAMICTueJun 13 18:16:59 IDT 2023
MKFS_OPTIONS-- -F -q -b 4096/dev/vdcMOUNT_OPTIONS-- -o acl,user_xattr -oblock_validity/dev/vdc/vdc
Quoted 6 lines
Quoted 3 lines
generic/572 [06:04:42] [06:04:47] [not run]
generic/572 -- ext4 verity isn't usable by default with these mkfs options
...You need to "tune2fs -O verity" (or pass -O verity to mkfs.ext4).
That was indeed missing in my setup, but it did not fix the problem.
Turned out that I had a very old version of fsverity installed in my kvm-xfstest test VM, where there is no --block-size option to fsverity enable would always fail.
Eric,
You may consider adding a check for minimal version of fsverity or check for support of --block-size option to make this error reason more clear for testers.
Ted,
FYI, FSVERITY_GIT in fstests-bld/config points to an out of date URL
How come there is no ext4/cfg/verity in fstests-bld?
Are you guys not testing fsverity with fstests-bld? Should we just add fsverity config or add verity to ext4/cfg/encrypt instead to avoid growing the test matrix?
I can send patches for fstests-bld fixing the above if you like.
Alex,
Verified that your verity-tests2 work as expected with v5 patches.
Thanks,
Amir.
Quoted 84 lines · Amir Goldstein
Quoted 54 lines · Alexander Larsson
Quoted 52 lines · Amir Goldstein
Quoted 19 lines · Alexander Larsson
Quoted 17 lines · Alexander Larsson
Quoted 15 lines · Amir Goldstein
Quoted 12 lines · Alexander Larsson
This patchset adds support for using fs-verity to validate lowerdata files by specifying an overlay.verity xattr on the metacopy files.
This is primarily motivated by the Composefs usecase, where there will be a read-only EROFS layer that contains redirect into a base data layer which has fs-verity enabled on all files. However, it is also useful in general if you want to ensure that the lowerdata files matches the expected content over time.
I have also added some tests for this feature to xfstests[1].
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
I can't remember if there is a good reason why your test does not include verify in a data-only layer.
I think this test coverage needs to be added.
Yeah. I'll add that.
Updated the git branch with some lowerdata tests.
What do I need to do in order to enable verity on ext4 besides enabling
FS_VERITYin the kernel?I'm getting these on verity tests on ext4 in the default 4k config.
_require_scratch_verity()doesn't mention any requirement other that 4K blocks and extent format files.Thanks,
Amir.BEGIN TEST 4k (10 tests): Ext4 4k block Wed Jun 14 06:04:25 UTC 2023
DEVICE:/dev/vdbEXT_MKFS_OPTIONS: -b 4096EXT_MOUNT_OPTIONS: -oblock_validity
FSTYP -- ext4
PLATFORM -- Linux/x86_64 kvm-xfstests
6.4.0-rc2-xfstests-00026-g35774ba7f07c #1596 SMPPREEMPT_DYNAMICTueJun 13 18:16:59 IDT 2023
MKFS_OPTIONS-- -F -q -b 4096/dev/vdcMOUNT_OPTIONS-- -o acl,user_xattr -oblock_validity/dev/vdc/vdcQuoted 6 lines
Quoted 3 lines
generic/572 [06:04:42] [06:04:47] [not run]
generic/572 -- ext4 verity isn't usable by default with these mkfs options
...You need to "tune2fs -O verity" (or pass -O verity to mkfs.ext4).
That was indeed missing in my setup, but it did not fix the problem.
Turned out that I had a very old version of fsverity installed in my kvm-xfstest test VM, where there is no --block-size option to fsverity enable would always fail.
Eric,
You may consider adding a check for minimal version of fsverity or check for support of --block-size option to make this error reason more clear for testers.
Ted,
FYI,
FSVERITY_GITin fstests-bld/config points to an out of date URLHow come there is no ext4/cfg/verity in fstests-bld?
Are you guys not testing fsverity with fstests-bld? Should we just add fsverity config or add verity to ext4/cfg/encrypt instead to avoid growing the test matrix?
Quoted 5 lines
I can send patches for fstests-bld fixing the above if you like.
Alex,
Verified that your verity-tests2 work as expected with v5 patches.
To be honest I have not validated that my changes to the shared verity code still works with the non-overlayfs tests. If you have a setup for it it would be great if you could try the regular ext4 w/ fs-veriy tests on top of the verity-test2 branch.
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]Amir Goldstein
maintainerQuoted 88 lines · Alexander Larsson
Quoted 86 lines · Amir Goldstein
Quoted 54 lines · Alexander Larsson
Quoted 52 lines · Amir Goldstein
Quoted 19 lines · Alexander Larsson
Quoted 17 lines · Alexander Larsson
Quoted 15 lines · Amir Goldstein
Quoted 12 lines · Alexander Larsson
This patchset adds support for using fs-verity to validate lowerdata files by specifying an overlay.verity xattr on the metacopy files.
This is primarily motivated by the Composefs usecase, where there will be a read-only EROFS layer that contains redirect into a base data layer which has fs-verity enabled on all files. However, it is also useful in general if you want to ensure that the lowerdata files matches the expected content over time.
I have also added some tests for this feature to xfstests[1].
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
I can't remember if there is a good reason why your test does not include verify in a data-only layer.
I think this test coverage needs to be added.
Yeah. I'll add that.
Updated the git branch with some lowerdata tests.
What do I need to do in order to enable verity on ext4 besides enabling
FS_VERITYin the kernel?I'm getting these on verity tests on ext4 in the default 4k config.
_require_scratch_verity()doesn't mention any requirement other that 4K blocks and extent format files.Thanks,
Amir.BEGIN TEST 4k (10 tests): Ext4 4k block Wed Jun 14 06:04:25 UTC 2023
DEVICE:/dev/vdbEXT_MKFS_OPTIONS: -b 4096EXT_MOUNT_OPTIONS: -oblock_validity
FSTYP -- ext4
PLATFORM -- Linux/x86_64 kvm-xfstests
6.4.0-rc2-xfstests-00026-g35774ba7f07c #1596 SMPPREEMPT_DYNAMICTueJun 13 18:16:59 IDT 2023
MKFS_OPTIONS-- -F -q -b 4096/dev/vdcMOUNT_OPTIONS-- -o acl,user_xattr -oblock_validity/dev/vdc/vdcQuoted 6 lines
Quoted 3 lines
generic/572 [06:04:42] [06:04:47] [not run]
generic/572 -- ext4 verity isn't usable by default with these mkfs options
...You need to "tune2fs -O verity" (or pass -O verity to mkfs.ext4).
That was indeed missing in my setup, but it did not fix the problem.
Turned out that I had a very old version of fsverity installed in my kvm-xfstest test VM, where there is no --block-size option to fsverity enable would always fail.
Eric,
You may consider adding a check for minimal version of fsverity or check for support of --block-size option to make this error reason more clear for testers.
Ted,
FYI,
FSVERITY_GITin fstests-bld/config points to an out of date URLHow come there is no ext4/cfg/verity in fstests-bld?
Are you guys not testing fsverity with fstests-bld? Should we just add fsverity config or add verity to ext4/cfg/encrypt instead to avoid growing the test matrix?I can send patches for fstests-bld fixing the above if you like.
Quoted 9 lines
Quoted 3 lines
Alex,
Verified that your verity-tests2 work as expected with v5 patches.
To be honest I have not validated that my changes to the shared verity code still works with the non-overlayfs tests. If you have a setup for it it would be great if you could try the regular ext4 w/ fs-veriy tests on top of the verity-test2 branch.
There is no problem with "./check -g verity" on ext4 those tests pass.
However, "./check -overlay -g generic/verity" fails several test:
Failures: generic/572 generic/573 generic/574 generic/575 generic/577 because _require_scratch_verity falsely claims that overlay (over ext4) supports verify, but then FS_IOC_ENABLE_VERITY actually fails during the test.
Instead of changing _require_scratch_verity() as you did, you should consider passing optional arguments, e.g.:
local fstyp=${1:-$FSTYP}
and calling it from _require_scratch_overlay_verity() with the $OVL_BASE_* values.
Thanks,
Amir.
Amir Goldstein
maintainerQuoted 4 lines · Amir Goldstein
Quoted 2 lines · Alexander Larsson
On Thu, Jun 22, 2023 at 11:37 AM Amir Goldstein <[email protected]> wrote:
...
Quoted 19 lines
Quoted 9 lines
Quoted 3 lines
Alex,
Verified that your verity-tests2 work as expected with v5 patches.
To be honest I have not validated that my changes to the shared verity code still works with the non-overlayfs tests. If you have a setup for it it would be great if you could try the regular ext4 w/ fs-veriy tests on top of the verity-test2 branch.
There is no problem with "./check -g verity" on ext4 those tests pass.
However, "./check -overlay -g generic/verity" fails several test:
Failures: generic/572 generic/573 generic/574 generic/575 generic/577 because_require_scratch_verityfalsely claims that overlay (over ext4) supports verify, but thenFS_IOC_ENABLE_VERITYactually fails during the test.
Quoted 6 lines
Instead of changing
_require_scratch_verity()as you did, you should consider passing optional arguments, e.g.:
local fstyp=${1:-$FSTYP}
and calling it from_require_scratch_overlay_verity()with the $OVL_BASE_* values.
FWIW, I pushed this solution to
https://github.com/amir73il/xfstests/commits/verity-tests
It's ugly, but it works.
Thanks,
Amir.
Cool, I wanted to look at this, but was on PTO last week. It looks good to me, and I synced this to:
https://github.com/alexlarsson/xfstests/commits/verity-tests
To avoid drift.
Quoted 41 lines · Amir Goldstein
Quoted 4 lines · Amir Goldstein
Quoted 2 lines · Alexander Larsson
On Thu, Jun 22, 2023 at 11:37 AM Amir Goldstein <[email protected]> wrote:
...
Quoted 19 lines
Quoted 9 lines
Quoted 3 lines
Alex,
Verified that your verity-tests2 work as expected with v5 patches.
To be honest I have not validated that my changes to the shared verity code still works with the non-overlayfs tests. If you have a setup for it it would be great if you could try the regular ext4 w/ fs-veriy tests on top of the verity-test2 branch.
There is no problem with "./check -g verity" on ext4 those tests pass.
However, "./check -overlay -g generic/verity" fails several test:
Failures: generic/572 generic/573 generic/574 generic/575 generic/577 because_require_scratch_verityfalsely claims that overlay (over ext4) supports verify, but thenFS_IOC_ENABLE_VERITYactually fails during the test.Quoted 6 lines
Instead of changing
_require_scratch_verity()as you did, you should consider passing optional arguments, e.g.:
local fstyp=${1:-$FSTYP}
and calling it from_require_scratch_overlay_verity()with the $OVL_BASE_* values.FWIW, I pushed this solution to
https://github.com/amir73il/xfstests/commits/verity-testsIt's ugly, but it works.
Thanks,
Amir.
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]Quoted 6 lines · Alexander Larsson
Cool, I wanted to look at this, but was on PTO last week. It looks good to me, and I synced this to:
https://github.com/alexlarsson/xfstests/commits/verity-tests
To avoid drift.
I pushed the overlay-verity series to overlayfs-next, so you may expect "finishing touches" bug reports from bots in the near future. As long as they are minor fixes, you can fix your branch and let me know and I will update overlayfs-next.
Miklos may yet have feedback on the final version, but I think all his comments were addressed including the ACK from Eric (thanks!).
Eric,
There was no posting of v5 that addressed your v4 review comments, so we do not have your RVB/ACK for patches 2-4.
Let me know if you want me to add that to the patches.
Alex,
wrt overlay verity-tests, please submit those tests along with the lowerdata tests to fstests anytime between now and the 6.6 merge window. The tests are properly equipped to check for the feature and testers can use them to test linux-next.
For test overlay/080 you may add:
For test overlay/079 you may retain my authorship or assume authorship I don't mind as it was co-authored and you took it to the finish line.
Thanks,
Amir.
Quoted 9 lines · Amir Goldstein
Quoted 6 lines · Alexander Larsson
Cool, I wanted to look at this, but was on PTO last week. It looks good to me, and I synced this to:
https://github.com/alexlarsson/xfstests/commits/verity-tests
To avoid drift.
Quoted 7 lines
I pushed the overlay-verity series to overlayfs-next, so you may expect "finishing touches" bug reports from bots in the near future. As long as they are minor fixes, you can fix your branch and let me know and I will update overlayfs-next.
Miklos may yet have feedback on the final version, but I think all his comments were addressed including the ACK from Eric (thanks!).
Nice!
Quoted 6 lines
Eric,
There was no posting of v5 that addressed your v4 review comments, so we do not have your RVB/ACK for patches 2-4.
Let me know if you want me to add that to the patches.
Quoted 6 lines
Alex,
wrt overlay verity-tests, please submit those tests along with the lowerdata tests to fstests anytime between now and the 6.6 merge window. The tests are properly equipped to check for the feature and testers can use them to test linux-next.
I'll have a look at this soon.
Quoted 8 lines
For test overlay/080 you may add:
Reviewed-by Amir Goldstein <[email protected]>For test overlay/079 you may retain my authorship or assume authorship I don't mind as it was co-authored and you took it to the finish line.
Thanks,
Amir.
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]Eric Biggers
maintainerHi Amir,
Quoted 25 lines · Amir Goldstein
Quoted 25 lines
Quoted 24 lines
What do I need to do in order to enable verity on ext4 besides enabling
FS_VERITYin the kernel?I'm getting these on verity tests on ext4 in the default 4k config.
_require_scratch_verity()doesn't mention any requirement other that 4K blocks and extent format files.Thanks,
Amir.BEGIN TEST 4k (10 tests): Ext4 4k block Wed Jun 14 06:04:25 UTC 2023
DEVICE:/dev/vdbEXT_MKFS_OPTIONS: -b 4096EXT_MOUNT_OPTIONS: -oblock_validity
FSTYP -- ext4
PLATFORM -- Linux/x86_64 kvm-xfstests
6.4.0-rc2-xfstests-00026-g35774ba7f07c #1596 SMPPREEMPT_DYNAMICTueJun 13 18:16:59 IDT 2023
MKFS_OPTIONS-- -F -q -b 4096/dev/vdcMOUNT_OPTIONS-- -o acl,user_xattr -oblock_validity/dev/vdc/vdcgeneric/572 [06:04:42] [06:04:47] [not run]
generic/572 -- ext4 verity isn't usable by default with these mkfs options
...
Quoted 8 lines
Quoted 2 lines
You need to "tune2fs -O verity" (or pass -O verity to mkfs.ext4).
That was indeed missing in my setup, but it did not fix the problem.
Turned out that I had a very old version of fsverity installed in my kvm-xfstest test VM, where there is no --block-size option to fsverity enable would always fail.
That would do it. So the tests were in fact skipping themselves as expected; just the skip message was not clear.
Quoted 6 lines
Eric,
You may consider adding a check for minimal version of fsverity or check for support of --block-size option to make this error reason more clear for testers.
I'm thinking it wouldn't be worth bothering, as the --block-size option was in the first actual release of fsverity-utils (v1.0). You must have pulled down a work-in-progress version of fsverity-utils back in 2018 or early 2019, well before the kernel support for fsverity was upstreamed, and then never updated it. I expect this issue affects very few people.
Quoted 3 lines
Ted,
FYI,
FSVERITY_GITin fstests-bld/config points to an out of date URL
It was already updated several months ago. Please run 'git pull'.
Quoted 2 lines
How come there is no ext4/cfg/verity in fstests-bld?
xfstests has a verity test *group*, which contains the tests that were written specifically to test verity.
An xfstests-bld test config is something that applies to all tests. Features like "encrypt" have an xfstests-bld test config since there is a way to enable/use those features in all tests. In the case of encrypt that means mounting the filesystem with "-o test_dummy_encryption".
In the case of verity, there is no way to enable/use verity in all tests. (It would be possible to always enable "-O verity" on the filesystem, but that does nothing to make verity actually be used/tested.) Hence, it doesn't have an xfstests-bld test config.
Quoted 1 line
Are you guys not testing fsverity with fstests-bld?
We are. The documented way to test fsverity is to use kvm-xfstests:
https://www.kernel.org/doc/html/latest/filesystems/fsverity.html#tests
Quoted 4 lines
Should we just add fsverity config or add verity to ext4/cfg/encrypt instead to avoid growing the test matrix?
I can send patches for fstests-bld fixing the above if you like.
As per the above, I don't think there is anything to fix.
- Eric
Amir Goldstein
maintainerQuoted 32 lines · Eric Biggers
Hi Amir,
Quoted 28 lines · Amir Goldstein
Quoted 27 lines
Quoted 17 lines
What do I need to do in order to enable verity on ext4 besides enabling
FS_VERITYin the kernel?I'm getting these on verity tests on ext4 in the default 4k config.
_require_scratch_verity()doesn't mention any requirement other that 4K blocks and extent format files.Thanks,
Amir.BEGIN TEST 4k (10 tests): Ext4 4k block Wed Jun 14 06:04:25 UTC 2023
DEVICE:/dev/vdbEXT_MKFS_OPTIONS: -b 4096EXT_MOUNT_OPTIONS: -oblock_validity
FSTYP -- ext4
PLATFORM -- Linux/x86_64 kvm-xfstests
6.4.0-rc2-xfstests-00026-g35774ba7f07c #1596 SMPPREEMPT_DYNAMICTueQuoted 7 lines
Jun 13 18:16:59 IDT 2023
MKFS_OPTIONS-- -F -q -b 4096/dev/vdcMOUNT_OPTIONS-- -o acl,user_xattr -oblock_validity/dev/vdc/vdcgeneric/572 [06:04:42] [06:04:47] [not run]
generic/572 -- ext4 verity isn't usable by default with these mkfs options
...You need to "tune2fs -O verity" (or pass -O verity to mkfs.ext4).
Quoted 9 lines
Quoted 5 lines
That was indeed missing in my setup, but it did not fix the problem.
Turned out that I had a very old version of fsverity installed in my kvm-xfstest test VM, where there is no --block-size option to fsverity enable would always fail.
That would do it. So the tests were in fact skipping themselves as expected; just the skip message was not clear.
Right.
Quoted 7 lines
Quoted 6 lines
Eric,
You may consider adding a check for minimal version of fsverity or check for support of --block-size option to make this error reason more clear for testers.
Quoted 6 lines
I'm thinking it wouldn't be worth bothering, as the --block-size option was in the first actual release of fsverity-utils (v1.0). You must have pulled down a work-in-progress version of fsverity-utils back in 2018 or early 2019, well before the kernel support for fsverity was upstreamed, and then never updated it. I expect this issue affects very few people.
Agree.
Quoted 6 lines
Quoted 3 lines
Ted,
FYI,
FSVERITY_GITin fstests-bld/config points to an out of date URLIt was already updated several months ago. Please run 'git pull'.
I thought I did. My bad.
Sorry for the noise.
Quoted 11 lines
Quoted 2 lines
How come there is no ext4/cfg/verity in fstests-bld?
xfstests has a verity test *group*, which contains the tests that were written specifically to test verity.
An xfstests-bld test config is something that applies to all tests. Features like "encrypt" have an xfstests-bld test config since there is a way to enable/use those features in all tests. In the case of encrypt that means mounting the filesystem with "-o
test_dummy_encryption".
Quoted 5 lines
In the case of verity, there is no way to enable/use verity in all tests. (It would be possible to always enable "-O verity" on the filesystem, but that does nothing to make verity actually be used/tested.) Hence, it doesn't have an xfstests-bld test config.
Got it.
In case of overlayfs over ext4 (i.e. -c ext4:overlay/small) it is needed to format ext4 in the beginning with -O verity because overlayfs tests do not currently have support for formatting the base fs differently per test.
This is the patch I intend to post.
Shout it you disagree:
commit 28d65d957c25eab85eb80b06724b215770a734a1
Author: Amir Goldstein <[email protected]> Date: Thu Jun 22 11:30:06 2023 +0300
test-appliance: enable verity for testing overlay over ext4
Add -O verity for ext4 formatted for overlay tests, so that overlay verity feature could be tested.
(quoted diff) +0 -0
b/test-appliance/files/root/fs/overlay/config
index 7c50b19..f252a70 100644
test-appliance/files/root/fs/overlay/config +1 -1
@@ -55,7 +55,7 @@ function __mkfs() case "$BASE_FSTYPE" in ext4) - /sbin/mke2fs -F -q -t ext4 "$dev" + /sbin/mke2fs -F -q -t ext4 -O verity "$dev" ;; xfs) mkfs.xfs -f -m rmapbt=1,reflink=1 "$dev"
Quoted 5 lines
Quoted 1 line
Are you guys not testing fsverity with fstests-bld?
We are. The documented way to test fsverity is to use kvm-xfstests:
https://www.kernel.org/doc/html/latest/filesystems/fsverity.html#tests
Quoted 7 lines
Quoted 4 lines
Should we just add fsverity config or add verity to ext4/cfg/encrypt instead to avoid growing the test matrix?
I can send patches for fstests-bld fixing the above if you like.
As per the above, I don't think there is anything to fix.
Right.
Expect for the patch above which is needed for running Alex's -g overlay/verity test
Thanks,
Amir.
Quoted 35 lines · Alexander Larsson
This patchset adds support for using fs-verity to validate lowerdata files by specifying an overlay.verity xattr on the metacopy files.
This is primarily motivated by the Composefs usecase, where there will be a read-only EROFS layer that contains redirect into a base data layer which has fs-verity enabled on all files. However, it is also useful in general if you want to ensure that the lowerdata files matches the expected content over time.
I have also added some tests for this feature to xfstests[1].
I'm also CC:ing the fsverity list and maintainers because there is one (tiny) fsverity change, and there may be interest in this usecase.
Changes since v2:
* Rebased on top of overlayfs-next
* We now alway do verity verification the first time the file content is used, rather than doing it at lookup time for the non-lazy lookup case.Changes since v1:
* Rebased on v2 lazy lowerdata series
* Dropped the "validate" mount option variant. We now only support "off", "on" and "require", where "off" is the default.
* We now store the digest algorithm used in the overlay.verity xattr.
* Dropped ability to configure default verity options, as this could cause problems moving layers between machines.
* We now properly resolve dependent mount options by automatically enabling metacopy andredirect_dirif verity is on, or failing if the specified options conflict.
* Streamlined and fixed the handling of creds inovl_ensure_verity_loaded().
* Renamed new helpers fromovl_entry_path_toovl_e_path_
Quoted 1 line
[1] https://github.com/alexlarsson/xfstests/commits/verity-tests
I pushed a new version of this branch with the following changes:
* Includes and uses the new fsverity_get_digest() rework from Eric
* The above means we now use the FS_VERITY_ALG_* enum values in the xattr
* Made the overlayfs.rst document change a bit more explicit on what
happens and by whom
* Ignore EOPNOTSUPP failure from removexattrs as pointed out by Amir
The previous patchset is available as the overlay-verity-v3 tag so you can compare the differences.
Eric Biggers
maintainerQuoted 12 lines · Alexander Larsson
I pushed a new version of this branch with the following changes:
* Includes and uses the new
fsverity_get_digest()rework from Eric
* The above means we now use the FS_VERITY_ALG_* enum values in the xattr
* Made theoverlayfs.rstdocument change a bit more explicit on what
happens and by whom
* Ignore EOPNOTSUPP failure from removexattrs as pointed out by AmirThe previous patchset is available as the overlay-verity-v3 tag so you can compare the differences.
Where can I find the new version?
- Eric
Quoted 11 lines · Eric Biggers
Quoted 9 lines · Alexander Larsson
I pushed a new version of this branch with the following changes:
* Includes and uses the new
fsverity_get_digest()rework from Eric
* The above means we now use the FS_VERITY_ALG_* enum values in the xattr
* Made theoverlayfs.rstdocument change a bit more explicit on what
happens and by whom
* Ignore EOPNOTSUPP failure from removexattrs as pointed out by Amir
Quoted 5 lines
Quoted 3 lines
The previous patchset is available as the overlay-verity-v3 tag so you can compare the differences.
Where can I find the new version?
Sorry, its at https://github.com/alexlarsson/linux/commits/overlay-verity
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]