Allow FUSE filesystems to provide out-of-band hashes to IMA

IMA wants to know what the hash of a file is, and currently does so by reading the entire file and generating the hash. Some filesystems may have the ability to store the hash in a secure manner resistant to offline attacks (eg, filesystem-level file signing), and in that case it's a performance win for IMA to be able to use that rather than having to re-hash everything. This patch simply adds VFS-level support for calling down to filesystems.

Signed-off-by Matthew Garrett <[email protected]>

---
 fs/read_write.c    | 24 ++++++++++++++++++++++++
 include/linux/fs.h |  6 +++++-
 2 files changed, 29 insertions(+), 1 deletion(-)

fs/read_write.c +24 -0

index 39b4a21dd933..9ba3ce4bb838 100644
@@ -2081,3 +2081,27 @@ int vfs_dedupe_file_range(struct file *file, struct file_dedupe_range *same)
 	return ret;
 }
 EXPORT_SYMBOL(vfs_dedupe_file_range);
+
+/**
+ * vfs_gethash - obtain a file's hash
+ * @file:	file structure in question
+ * @hash_algo:	the hash algorithm requested
+ * @buf:	buffer to return the hash in
+ * @size:	size allocated for the buffer by the caller
+ *
+ * This function allows filesystems that support securely storing the hash
+ * of a file to return it rather than forcing the kernel to recalculate it.
+ * Filesystems that cannot provide guarantees about the hash being resistant
+ * to offline attack should not implement this functionality.
+ *
+ * Returns 0 on success, -EOPNOTSUPP if the filesystem doesn't support it.
+ */
+int vfs_get_hash(struct file *file, enum hash_algo hash, uint8_t *buf,
+		 size_t size)
+{
+	if (!file->f_op->get_hash)
+		return -EOPNOTSUPP;
+
+	return file->f_op->get_hash(file, hash, buf, size);
+}
+EXPORT_SYMBOL(vfs_get_hash);

include/linux/fs.h +5 -1

index 6c0b4a1c22ff..540316cfd461 100644
@@ -40,6 +40,7 @@
 
 #include <asm/byteorder.h>
 #include <uapi/linux/fs.h>
+#include <uapi/linux/hash_info.h>
 
 struct backing_dev_info;
 struct bdi_writeback;

@@ -1764,6 +1765,8 @@ struct file_operations {
 	int (*dedupe_file_range)(struct file *, loff_t, struct file *, loff_t,
 			u64);
 	int (*fadvise)(struct file *, loff_t, loff_t, int);
+	int (*get_hash)(struct file *, enum hash_algo hash, uint8_t *buf,
+			size_t size);
 } __randomize_layout;
 
 struct inode_operations {

@@ -1838,7 +1841,8 @@ extern int vfs_dedupe_file_range(struct file *file,
 extern int vfs_dedupe_file_range_one(struct file *src_file, loff_t src_pos,
 				     struct file *dst_file, loff_t dst_pos,
 				     u64 len);
-
+extern int vfs_get_hash(struct file *file, enum hash_algo hash, uint8_t *buf,
+			size_t size);
 
 struct super_operations {
    	struct inode *(*alloc_inode)(struct super_block *sb);

Quoted 7 lines · Matthew Garrett

IMA wants to know what the hash of a file is, and currently does so by reading the entire file and generating the hash. Some filesystems may have the ability to store the hash in a secure manner resistant to offline attacks (eg, filesystem-level file signing), and in that case it's a performance win for IMA to be able to use that rather than having to re-hash everything. This patch simply adds VFS-level support for calling down to filesystems.

This patch description starts out saying that IMA needs the file hash without explaining why. Without that explanation, simply extracting the file hash included in the file signature might sound plausible, but kind of defeats the purpose of IMA.

Quoted 5 lines · Mimi Zohar

This patch description starts out saying that IMA needs the file hash without explaining why. Without that explanation, simply extracting the file hash included in the file signature might sound plausible, but kind of defeats the purpose of IMA.

I'm not sure how it defeats the purpose - IMA wants to know the hash of a file so it can either log it or compare it against a signature, and it currently obtains this hash by reading the entire file at measurement time. If the filesystem later returns different data then IMA won't notice, which allows a malicious filesystem to bypass the measurements - there's no guarantee that we won't evict large parts of the copy of an executable that IMA read, and the filesystem can give us back a modified page when we page it back in. So IMA fundamentally relies on the filesystem to be trustworthy, and if we rely on the filesystem to be trustworthy then we should be able to rely on it to accurately store and provide the hash of a file.

Quoted 8 lines · Matthew Garrett

Quoted 5 lines · Mimi Zohar

This patch description starts out saying that IMA needs the file hash without explaining why. Without that explanation, simply extracting the file hash included in the file signature might sound plausible, but kind of defeats the purpose of IMA.

Quoted 11 lines

I'm not sure how it defeats the purpose - IMA wants to know the hash of a file so it can either log it or compare it against a signature, and it currently obtains this hash by reading the entire file at measurement time. If the filesystem later returns different data then IMA won't notice, which allows a malicious filesystem to bypass the measurements - there's no guarantee that we won't evict large parts of the copy of an executable that IMA read, and the filesystem can give us back a modified page when we page it back in. So IMA fundamentally relies on the filesystem to be trustworthy, and if we rely on the filesystem to be trustworthy then we should be able to rely on it to accurately store and provide the hash of a file.

Oh, to clarify on the signature part of things - it would obviously be inappropriate to, say, just read the hash out of security.ima and hand that back.

But for a hypothetical case where the filesystem itself verifies the signature, then the filesystem would abort the transaction if the signature didn't match and it seems reasonable to avoid doing the validation twice (once up front and then again on every read)

Quoted 5 lines · Matthew Garrett

But for a hypothetical case where the filesystem itself verifies the signature, then the filesystem would abort the transaction if the signature didn't match and it seems reasonable to avoid doing the validation twice (once up front and then again on every read)

Right, this is a hypothetical scenario as far as I'm aware, since none of the filesystems are currently calculating and storing the file hash. The default should be for IMA to re-calculate the file hash.

Some filesystems may be able to provide hashes in an out of band manner, and allowing them to do so is a performance win. This is especially true of FUSE-based filesystems where otherwise we recalculate the hash on every measurement. Make use of this by default, but provide a parameter to force recalculation rather than trusting the filesystem.

Signed-off-by Matthew Garrett <[email protected]>

---
 Documentation/admin-guide/kernel-parameters.txt |  5 +++++
 security/integrity/ima/ima_crypto.c             | 11 +++++++++++
 2 files changed, 16 insertions(+)

Documentation/admin-guide/kernel-parameters.txt +5 -0

index 92eb1f42240d..617ae0f83b14 100644
@@ -1612,6 +1612,11 @@
 			different crypto accelerators. This option can be used
 			to achieve best performance for particular HW.
 
+	ima.force_hash= [IMA] Force hash calculation in IMA
+			Format: <bool>
+			Always calculate hashes rather than trusting the
+			filesystem to provide them to us.
+
 	init=		[KNL]
 			Format: <full_path>
 			Run specified binary instead of /sbin/init as init

security/integrity/ima/ima_crypto.c +4 -0

index 7e7e7e7c250a..f25259b2b6ec 100644
@@ -32,6 +32,10 @@ static unsigned long ima_ahash_minsize;
 module_param_named(ahash_minsize, ima_ahash_minsize, ulong, 0644);
 MODULE_PARM_DESC(ahash_minsize, "Minimum file size for ahash use");
 
+static bool ima_force_hash;
+module_param_named(force_hash, ima_force_hash, bool_enable_only, 0644);
+MODULE_PARM_DESC(force_hash, "Always calculate hashes");
+
 /* default is 0 - 1 page. */
 static int ima_maxorder;
 static unsigned int ima_bufsize = PAGE_SIZE;

security/integrity/ima/ima_crypto.c +6 -0

+		hash->length = hash_digest_size[hash->algo];
+		rc = vfs_get_hash(file, hash->algo, hash->digest, hash->length);
+		if (!rc)
+			return 0;
+	}
+
 	i_size = i_size_read(file_inode(file));
 
 	if (ima_ahash_minsize && i_size >= ima_ahash_minsize) {

Quoted 6 lines · Matthew Garrett

Some filesystems may be able to provide hashes in an out of band manner, and allowing them to do so is a performance win. This is especially true of FUSE-based filesystems where otherwise we recalculate the hash on every measurement. Make use of this by default, but provide a parameter to force recalculation rather than trusting the filesystem.

Quoted 1 line

Signed-off-by Matthew Garrett <[email protected]>

Support for not calculating the file hash would need to be finer grained than this, probably on a per mount basis. The default should be for IMA to always calculate the file hash, unless explicitly told not to.

IMA should never skip the file hash calculation if the filesystem is an untrusted mount (eg. SB_I_UNTRUSTED_MOUNTER).

Quoted 8 lines · Mimi Zohar

Quoted 6 lines · Matthew Garrett

Some filesystems may be able to provide hashes in an out of band manner, and allowing them to do so is a performance win. This is especially true of FUSE-based filesystems where otherwise we recalculate the hash on every measurement. Make use of this by default, but provide a parameter to force recalculation rather than trusting the filesystem.

Quoted 6 lines

Quoted 1 line

Signed-off-by Matthew Garrett <[email protected]>

Support for not calculating the file hash would need to be finer grained than this, probably on a per mount basis. The default should be for IMA to always calculate the file hash, unless explicitly told not to.

Ok, should this just be part of the IMA policy?

Quoted 3 lines

IMA should never skip the file hash calculation if the filesystem is an untrusted mount (eg. SB_I_UNTRUSTED_MOUNTER).

Ok.

Quoted 1 line · Matthew Garrett

Ok, should this just be part of the IMA policy?

How would you be able to differentiate between different FUSE filesystems for example?

Quoted 6 lines · Mimi Zohar

Quoted 1 line · Matthew Garrett

Ok, should this just be part of the IMA policy?

How would you be able to differentiate between different FUSE filesystems for example?

There's a couple of ways. We could extend the filesystem type matching logic to also check the subtype - you'd then need to enforce that at the LSM level in order to protect against untrusted filesystems spoofing the filesystem type. Alternatively, we could add an additional policy match type for mount point and iterate through s_mounts on the superblock - if any match, we could define the policy there?

Quoted 7 lines · Matthew Garrett

There's a couple of ways. We could extend the filesystem type matching logic to also check the subtype - you'd then need to enforce that at the LSM level in order to protect against untrusted filesystems spoofing the filesystem type. Alternatively, we could add an additional policy match type for mount point and iterate through s_mounts on the superblock - if any match, we could define the policy there?

The first method differentiates between different subtypes of FUSE filesystems, while the second method allows differentiating between the same type and subtype on different mount points. Both criteria are needed, but instead of the second method based on a mount point, perhaps based instead on a mount flag?

Trusted mount of permitted filesystem type and subtype, that is mounted with the defined mount flag.

Quoted 9 lines · Mimi Zohar

Quoted 7 lines · Matthew Garrett

There's a couple of ways. We could extend the filesystem type matching logic to also check the subtype - you'd then need to enforce that at the LSM level in order to protect against untrusted filesystems spoofing the filesystem type. Alternatively, we could add an additional policy match type for mount point and iterate through s_mounts on the superblock - if any match, we could define the policy there?

Quoted 5 lines

The first method differentiates between different subtypes of FUSE filesystems, while the second method allows differentiating between the same type and subtype on different mount points. Both criteria are needed, but instead of the second method based on a mount point, perhaps based instead on a mount flag?

Patch 3 already requires that the allow_gethash option be passed for this to work - I can restrict that to CAP_SYS_ADMIN?

Quoted 3 lines

Trusted mount of permitted filesystem type and subtype, that is mounted with the defined mount flag.

Ok, I'll write up a patch that allows policy matching of filesystem subtype as well as type and try to get that posted this week so we can discuss it in Edinburgh?

As of d77ccdc644a59b412d8e101576134c90a0aa6797, IMA will trigger a rehash of any file on a FUSE filesystem for every measurement. This has a significant impact on performance. Individual FUSE filesystems may have the ability to identify whether a file needs to be rehashed, and some may even have the ability to obtain the file hash out of band without needing the kernel to do it. Longer term, this may also be usable for other scenarios where a filesystem can provide a trustworthy hash (eg, fs-verity on ext4 could provide a hash and then later abort any read()s that discover that the file doesn't match the measurement).

Really, a security vs. performance argument?! I don't need to tell you of all people, that one of the basic tenents of trusted boot is calculating the actual file hash before use. Limiting the file hash re-calculation is one thing, but relying on some out of band method of obtaining the file hash without the kernel ever calculating it is totally different. The only exception will be for fs-verity, which will return not the file hash, but the file's Merkle tree root hash.

Quoted 7 lines · Mimi Zohar

Really, a security vs. performance argument?! I don't need to tell you of all people, that one of the basic tenents of trusted boot is calculating the actual file hash before use. Limiting the file hash re-calculation is one thing, but relying on some out of band method of obtaining the file hash without the kernel ever calculating it is totally different. The only exception will be for fs-verity, which will return not the file hash, but the file's Merkle tree root hash.

Using FUSE means you're inherently accepting the risk of TOCTOU. Having the kernel read everything once and hash it is no guarantee that the filesystem will return the same value on further reads, so if you're going to use FUSE in an environment where you're using IMA then you already need to assert that your filesystems are trustworthy.

Right, the correct behavior should be not to trust FUSE filesystems, but since we don't break userspace there is the "ima_policy=fail_securely" boot command line option.

Quoted 3 lines · Mimi Zohar

Right, the correct behavior should be not to trust FUSE filesystems, but since we don't break userspace there is the "ima_policy=fail_securely" boot command line option.

There seem to be two scenarios:

1) You trust FUSE mounts, perhaps because you have some other policy in place to ensure that only trusted binaries can mount stuff. In this scenario you already trust that the filesystem will give you consistent results when you read data from it -

it seems reasonable to
also trust it to give you back an accurate hash if you ask for one.

2) You don't trust FUSE mounts, in which case you pass ima_policy=fail_securely. This patch doesn't change that behaviour.

I agree that using FUSE in general is incompatible with IMA's goals, but it's possible to configure systems where you can ensure that only trustworthy code is involved. In that scenario this patch improves performance without compromising security.

Quoted 4 lines · Matthew Garrett

1) You trust FUSE mounts, perhaps because you have some other policy in place to ensure that only trusted binaries can mount stuff. In this scenario you already trust that the filesystem will give you consistent results when you read data from it -

In the trusted mount scenario, we trust the data should not change between calculating the file hash and reading the file data, making it similar to other local filesystems. Unlike other local filesystems, however, we can't detect when the file changes. For this reason we need to re-calculate the file hash to measure/appraise the file each time.

Quoted 2 lines

it seems reasonable to
also trust it to give you back an accurate hash if you ask for one.

Going from trusting the filesystem to behave properly, to trusting the file hash that the filesystem provides is a major leap. We don't do this today for any local filesystem.

Quoted 4 lines

I agree that using FUSE in general is incompatible with IMA's goals, but it's possible to configure systems where you can ensure that only trustworthy code is involved. In that scenario this patch improves performance without compromising security.

If you trust a FUSE filesystem to not only behave properly, but also to return file hashes, what is the value of measuring/appraising the files? Define a custom policy that doesn't measure/appraise files on FUSE filesystems.

Quoted 11 lines · Mimi Zohar

Quoted 4 lines · Matthew Garrett

I agree that using FUSE in general is incompatible with IMA's goals, but it's possible to configure systems where you can ensure that only trustworthy code is involved. In that scenario this patch improves performance without compromising security.

If you trust a FUSE filesystem to not only behave properly, but also to return file hashes, what is the value of measuring/appraising the files? Define a custom policy that doesn't measure/appraise files on FUSE filesystems.

We trust that the filesystem will return us accurate binaries and hashes, but we don't the binaries themselves may not be trustworthy - we want the same level of audit trail associated with their execution that we'd have for something run off local disk. We could certainly rearchitect our filesystems to generate audit events themselves, but we'd be duplicating functionality that already exists in the kernel.

Quoted 6 lines · Matthew Garrett

We trust that the filesystem will return us accurate binaries and hashes, but we don't the binaries themselves may not be trustworthy - we want the same level of audit trail associated with their execution that we'd have for something run off local disk. We could certainly rearchitect our filesystems to generate audit events themselves, but we'd be duplicating functionality that already exists in the kernel.

I'm really not comfortable with the FUSE filesystem calculating the file hash being used by IMA. Adding FUSE i_version support would have been better, instead of returning the actual file hash. Based on a mount option and the i_version, the kernel could then decide whether or not to limit re-calculating the file hash.

Quoted 9 lines · Mimi Zohar

Quoted 6 lines · Matthew Garrett

We trust that the filesystem will return us accurate binaries and hashes, but we don't the binaries themselves may not be trustworthy - we want the same level of audit trail associated with their execution that we'd have for something run off local disk. We could certainly rearchitect our filesystems to generate audit events themselves, but we'd be duplicating functionality that already exists in the kernel.

Quoted 5 lines

I'm really not comfortable with the FUSE filesystem calculating the file hash being used by IMA. Adding FUSE i_version support would have been better, instead of returning the actual file hash. Based on a mount option and the i_version, the kernel could then decide whether or not to limit re-calculating the file hash.

Well, there's a performance benefit as well - reading 500MB executables over the network is time consuming and otherwise mostly unnecessary. Given two solutions that have the same properties in terms of which components we need to trust, why not pick the one that's faster?

Quoted 5 lines · Matthew Garrett

Well, there's a performance benefit as well - reading 500MB executables over the network is time consuming and otherwise mostly unnecessary. Given two solutions that have the same properties in terms of which components we need to trust, why not pick the one that's faster?

With the existing cover letter, the purpose of this patch set should be to address the performance of calculating the file hash on trusted local FUSE mounted filesystems, not remote filesystems or fs-verity filesystems.

Quoted 8 lines · Mimi Zohar

Quoted 5 lines · Matthew Garrett

Well, there's a performance benefit as well - reading 500MB executables over the network is time consuming and otherwise mostly unnecessary. Given two solutions that have the same properties in terms of which components we need to trust, why not pick the one that's faster?

Quoted 4 lines

With the existing cover letter, the purpose of this patch set should be to address the performance of calculating the file hash on trusted local FUSE mounted filesystems, not remote filesystems or fs-verity filesystems.

The performance hit is more noticeable over remote filesystems, but we have large binaries that take several seconds to hash even on local filesystems. Would it be helpful to try to define the assumptions that IMA makes in terms of whether or not it produces trustworthy results? It feels like it's be easier to talk about this if we have a more formal set of conditions to take into consideration.

Quoted 7 lines

The performance hit is more noticeable over remote filesystems, but we have large binaries that take several seconds to hash even on local filesystems. Would it be helpful to try to define the assumptions that IMA makes in terms of whether or not it produces trustworthy results? It feels like it's be easier to talk about this if we have a more formal set of conditions to take into consideration.

[Cc'ing Chuck Lever]

Integrity of files on remote filesystems should probably be discussed in the context of fs-verity, not FUSE filesystems.

Do you want to continue the discussion here or perhaps as an LSS-EU BoF?

Quoted 8 lines · Mimi Zohar

Quoted 7 lines

The performance hit is more noticeable over remote filesystems, but we have large binaries that take several seconds to hash even on local filesystems. Would it be helpful to try to define the assumptions that IMA makes in terms of whether or not it produces trustworthy results? It feels like it's be easier to talk about this if we have a more formal set of conditions to take into consideration.

Quoted 4 lines

[Cc'ing Chuck Lever]

Integrity of files on remote filesystems should probably be discussed in the context of fs-verity, not FUSE filesystems.

Hm. We /could/ fake up fs-verity style behaviour, but we're not talking about files that are expected to be immutable so it would seem like there might be mismatched semantics there.

Quoted 3 lines · Matthew Garrett

Hm. We /could/ fake up fs-verity style behaviour, but we're not talking about files that are expected to be immutable so it would seem like there might be mismatched semantics there.

If these aren't immutable files, then the file hash needs to be calculated and re-calculated on file change. Trust between the kernel and FUSE isn't bi-directional. IMA already supports hardware crypto acceleration. (Refer to the "ima.ahash_minsize" and "ima.ahash_bufsize" boot command line options.) Why is it better that FUSE calculates the file hash than the kernel?