btrfs: simple quotas
PATCH 2/18 btrfs: fix start transaction qgroup rsv double free
In reply to Boris Burkovbtrfs_start_transaction reserves metadata space of the PERTRANS type before it identifies a transaction to start/join. This allows flushing when reserving that space without a deadlock. However, it results in a race which temporarily breaks qgroup rsv accounting.
T1 T2start_transactiondo_stuff
start_transaction
qgroup_reserve_meta_pertranscommit_transaction
qgroup_free_meta_all_pertrans
hit an error starting txn goto reserve_fail qgroup_free_meta_pertrans (already freed!)
The basic issue is that there is nothing preventing another commit from committing before start_transaction finishes (in fact sometimes we intentionally wait for it..) so any error path that frees the reserve is at risk of this race.
While this exact space was getting freed anyway, and it's not a huge deal to double free it (just a warning, the free code catches this), it can result in incorrectly freeing some other pertrans reservation in this same reservation, which could then lead to spuriously granting reservations we might not have the space for. Therefore, I do believe it is worth fixing.
To fix it, use the existing prealloc->pertrans conversion mechanism. When we first reserve the space, we reserve prealloc space and only when we are sure we have a transaction do we convert it to pertrans. This way any racing commits do not blow away our reservation, but we still get a pertrans reservation that is freed when _this_ transaction gets committed.
This issue can be reproduced by running generic/269 with either qgroups or squotas enabled via mkfs on the scratch device.
--- fs/btrfs/transaction.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-)
fs/btrfs/transaction.c +16 -3
@@ -591,8 +591,13 @@ start_transaction(struct btrfs_root *root, unsigned int num_items,u64 delayed_refs_bytes =0 ; qgroup_reserved = num_items * fs_info->nodesize; - ret =btrfs_qgroup_reserve_meta_pertrans (root, qgroup_reserved, - enforce_qgroups); +/* +* Use prealloc for now, as there might be a currently running +* transaction that could free this reserved space prematurely +* by committing. +*/ + ret =btrfs_qgroup_reserve_meta_prealloc (root, qgroup_reserved, + enforce_qgroups, false);if (ret)return ERR_PTR (ret);
@@ -705,6 +710,14 @@ start_transaction(struct btrfs_root *root, unsigned int num_items, h->reloc_reserved = reloc_reserved; } +/* +* Now that we have found a transaction to be a part of, convert the +* qgroup reservation from prealloc to pertrans. A different transaction +* can't race in and free our pertrans out from under us. +*/ +if (qgroup_reserved) +btrfs_qgroup_convert_reserved_meta (root, qgroup_reserved); + got_it:if (!current->journal_info) current->journal_info = h;
@@ -752,7 +765,7 @@ start_transaction(struct btrfs_root *root, unsigned int num_items,btrfs_block_rsv_release (fs_info, &fs_info->trans_block_rsv, num_bytes,NULL ); reserve_fail: -btrfs_qgroup_free_meta_pertrans (root, qgroup_reserved); +btrfs_qgroup_free_meta_prealloc (root, qgroup_reserved);return ERR_PTR (ret); }
Signature · 1 line
2.41.0
PATCH 1/18 btrfs: free qgroup rsv on io failure
In reply to Boris BurkovIf we do a write whose bio suffers an error, we will never reclaim the qgroup reserved space for it. We allocate the space in the write_iter codepath, then release the reservation as we allocate the ordered extent, but we only create a delayed ref if the ordered extent finishes. If it has an error, we simply leak the rsv. This is apparent in running any error injecting (dmerror) fstests like btrfs/146 or btrfs/160. Such tests fail due to dmesg on umount complaining about the leaked qgroup data space.
When we clean up other aspects of space on failed ordered_extents, also free the qgroup rsv.
--- fs/btrfs/inode.c | 7 +++++++ 1 file changed, 7 insertions(+)
fs/btrfs/inode.c +7 -0
@@ -3359,6 +3359,13 @@ int btrfs_finish_one_ordered(struct btrfs_ordered_extent *ordered_extent)btrfs_free_reserved_extent (fs_info, ordered_extent->disk_bytenr, ordered_extent->disk_num_bytes,1 ); +/* +* Actually free the qgroup rsv which was released when +* the ordered extent was created. +*/ +btrfs_qgroup_free_refroot (fs_info, inode->root->root_key.objectid, + ordered_extent->qgroup_rsv, +BTRFS_QGROUP_RSV_DATA ); } }
Signature · 1 line
2.41.0
cover btrfs: simple quotas
btrfs quota groups (qgroups) are a compelling feature of btrfs that
allow flexible control for limiting subvolume data and metadata usage.
However, due to btrfs's high level decision to tradeoff snapshot
performance against ref-counting performance, qgroups suffer from
non-trivial performance issues that make them unattractive in certain
workloads. Particularly, frequent backref walking during writes and
during commits can make operations increasingly expensive as the number
of snapshots scales up. For that reason, we have never been able to
commit to using qgroups in production at Meta, despite significant
interest from people running container workloads, where we would benefit
from protecting the rest of the host from a buggy application in a
container running away with disk usage. This patch series introduces a simplified version of qgroups called
simple quotas (squotas) which never computes global reference counts
for extents, and thus has similar performance characteristics to normal,
quotas disabled, btrfs. The "trick" is that in simple quotas mode, we
account all extents permanently to the subvolume in which they were
originally created. That allows us to make all accounting 1:1 with
extent item lifetime, removing the need to walk backrefs. However, this sacrifices the ability to compute shared vs. exclusive usage. It also results in counter-intuitive, though still predictable and simple,
accounting in the cases where an original extent is removed while a
shared copy still exists. Qgroups is able to detect that case and count
the remaining copy as an exclusive owner, while squotas is not. As a
result, squotas works best when the original extent is immutable and
outlives any clones.
==Format Change==
In order to track the original creating subvolume of a data extent in the face of reflinks, it is necessary to add additional accounting to the extent item. To save space, this is done with a new inline ref item. However, the downside of this approach is that it makes enabling squota an incompat change, denoted by the new incompat bit SIMPLE_QUOTA. When this bit is set and quotas are enabled, new extent items get the extra accounting, and freed extent items check for the accounting to find their creating subvolume. In addition, 1:1 with this incompat bit, the quota status item now tracks a "quota enablement generation" needed for properly handling deleting extents with predate enablement.
==API==
Squotas reuses the api of qgroups. The only difference is that when you enable quotas via btrfs quota enable, you pass the --simple flag. Squotas will always report exclusive == shared for each qgroup. Squotas deal with extent_item/metadata_item sizes and thus do not do anything special with compression. Squotas also introduce auto inheritance for nested subvols. The API is documented more fully in the documentation patches in btrfs-progs.
==Testing methodology==
Using updated btrfs-progs and fstests (relevant matching patch sets to be sent ASAP)
btrfs-progs: https://github.com/boryas/btrfs-progs/tree/squota-progs
fstests: https://github.com/boryas/fstests/tree/squota-test
I ran '-g auto' on fstests on the following configurations:
1a) baseline kernel/progs/fstests.
1b) squota kernel baseline progs/fstests.
2a) baseline kernel/progs/fstests. fstests configured to mkfs with quota 2b) squota kernel/progs/fstests. fstests configured to mkfs with squota
I compared 1a against 1b and 2a against 2b and detected no regressions. 2a/2b both exhibit regressions against 1a/1b which are largely issues with quota reservations in various complicated cases. I intend to run those down in the future, but they are not simple quota specific, as they are already broken with plain qgroups.
==Performance Testing==
I measured the performance of the change using fsperf. I ran with 3 configurations using the squota kernel:
- plain mkfs
- qgroup mkfs
- squota mkfs
And added a new performance test which creates 1000 files in a subvol, creates 100 snapshots of that subvol, then unshares extents in files in the snapshots. I measured write performance with fio and btrfs commit critical section performance side effects with bpftrace on 'wait_current_trans'.
The results for the test which measures unshare perf (unshare.py) with qgroup and squota compared to the baseline:
group test results
unshare results
metric baseline current stdev diff
========================================================================================avg_commit_ms 162.13 285.75 3.14 76.24%bg_count 16 16 0 0.00%
commits 378.20 379 1.92 0.21% elapsed 201.40 270.40 1.34 34.26%
end_state_mount_ns 26036211.60 26004593.60 2281065.40 -0.12%end_state_umount_ns 2.45e+09 2.55e+09 20740154.41 3.93%max_commit_ms 425.80 594 53.34 39.50%sys_cpu 0.10 0.06 0.06 -42.15%wait_current_trans_calls 2945.60 3405.20 47.08 15.60%wait_current_trans_ns_max 1.56e+08 3.43e+08 32659393.25 120.07%wait_current_trans_ns_mean 1974875.35 28588482.55 1557588.84 1347.61%
wait_current_trans_ns_min 232 232 25.88 0.00% wait_current_trans_ns_p50 718 740 22.80 3.06% wait_current_trans_ns_p95 7711770.20 2.21e+08 17241032.09 2761.19% wait_current_trans_ns_p99 67744932.29 2.68e+08 41275815.87 295.16% write_bw_bytes 653008.80 486344.40 4209.91 -25.52% write_clat_ns_mean 6251404.78 8406837.89 39779.15 34.48% write_clat_ns_p50 1656422.40 1643315.20 27415.68 -0.79% write_clat_ns_p99 1.90e+08 3.20e+08 2097152 68.62% write_io_kbytes 128000 128000 0 0.00% write_iops 159.43 118.74 1.03 -25.52% write_lat_ns_max 7.06e+08 9.80e+08 47324816.61 38.88% write_lat_ns_mean 6251503.06 8406936.06 39780.83 34.48% write_lat_ns_min 3354 4648 616.06 38.58%
squota test results
unshare results
metric baseline current stdev diff
========================================================================================avg_commit_ms 162.13 164.16 3.14 1.25%bg_count 16 0 0 -100.00%
commits 378.20 380.80 1.92 0.69% elapsed 201.40 208.20 1.34 3.38% end_state_mount_ns 26036211.60 25840729.60 2281065.40 -0.75% end_state_umount_ns 2.45e+09 3.01e+09 20740154.41 22.80% max_commit_ms 425.80 415.80 53.34 -2.35%
sys_cpu 0.10 0.08 0.06 -23.36%wait_current_trans_calls 2945.60 2981.60 47.08 1.22%wait_current_trans_ns_max 1.56e+08 1.12e+08 32659393.25 -27.86%wait_current_trans_ns_mean 1974875.35 1064734.76 1557588.84 -46.09%
wait_current_trans_ns_min 232 238 25.88 2.59% wait_current_trans_ns_p50 718 746 22.80 3.90% wait_current_trans_ns_p95 7711770.20 1567.60 17241032.09 -99.98% wait_current_trans_ns_p99 67744932.29 49880514.27 41275815.87 -26.37% write_bw_bytes 653008.80 631256 4209.91 -3.33% write_clat_ns_mean 6251404.78 6476816.06 39779.15 3.61% write_clat_ns_p50 1656422.40 1581056 27415.68 -4.55% write_clat_ns_p99 1.90e+08 1.94e+08 2097152 2.21% write_io_kbytes 128000 128000 0 0.00% write_iops 159.43 154.12 1.03 -3.33%
write_lat_ns_max 7.06e+08 7.65e+08 47324816.61 8.38%write_lat_ns_mean 6251503.06 6476912.76 39780.83 3.61%write_lat_ns_min 3354 4062 616.06 21.11%
And the same, but only showing results where the deviation was outside of a 95% confidence interval for the mean (default significance highlighting in fsperf):
qgroup test results
unshare results
metric baseline current stdev diff
========================================================================================
avg_commit_ms 162.13 285.75 3.14 76.24% elapsed 201.40 270.40 1.34 34.26% end_state_umount_ns 2.45e+09 2.55e+09 20740154.41 3.93% max_commit_ms 425.80 594 53.34 39.50% wait_current_trans_calls 2945.60 3405.20 47.08 15.60% wait_current_trans_ns_max 1.56e+08 3.43e+08 32659393.25 120.07% wait_current_trans_ns_mean 1974875.35 28588482.55 1557588.84 1347.61% wait_current_trans_ns_p95 7711770.20 2.21e+08 17241032.09 2761.19%
wait_current_trans_ns_p99 67744932.29 2.68e+08 41275815.87 295.16%write_bw_bytes 653008.80 486344.40 4209.91 -25.52%write_clat_ns_mean 6251404.78 8406837.89 39779.15 34.48%write_clat_ns_p99 1.90e+08 3.20e+08 2097152 68.62%write_iops 159.43 118.74 1.03 -25.52%
write_lat_ns_max 7.06e+08 9.80e+08 47324816.61 38.88% write_lat_ns_mean 6251503.06 8406936.06 39780.83 34.48% write_lat_ns_min 3354 4648 616.06 38.58%
squota test results
unshare results
metric baseline current stdev diff
========================================================================================
elapsed 201.40 208.20 1.34 3.38%end_state_umount_ns 2.45e+09 3.01e+09 20740154.41 22.80%write_bw_bytes 653008.80 631256 4209.91 -3.33%
write_clat_ns_mean 6251404.78 6476816.06 39779.15 3.61% write_clat_ns_p50 1656422.40 1581056 27415.68 -4.55%
write_clat_ns_p99 1.90e+08 1.94e+08 2097152 2.21%write_iops 159.43 154.12 1.03 -3.33%write_lat_ns_mean 6251503.06 6476912.76 39780.83 3.61%
Particularly noteworthy are the massive regressions to wait_current_trans in qgroup mode as well as the solid regressions to bandwidth, iops and write latency. The regressions/improvements in squotas are modest in comparison in line with the expectation. I am still investigating the squota umount regression, particularly whether it is in the umount's final commit and represents a real performance problem with squotas.
Boris Burkov (18):
btrfs: free qgroup rsv on io failure
btrfs: fix start transaction qgroup rsv double free
btrfs: introduce quota mode
btrfs: add new quota mode for simple quotas
btrfs: expose quota mode via sysfs
btrfs: flush reservations during quota disable
btrfs: create qgroup earlier in snapshot creation
btrfs: function for recording simple quota deltas
btrfs: rename tree_ref and data_ref owning_root
btrfs: track owning root in btrfs_ref
btrfs: track original extent owner in head_ref
btrfs: new inline ref storing owning subvol of data extents
btrfs: inline owner ref lookup helper
btrfs: record simple quota deltas
btrfs: simple quota auto hierarchy for nested subvols
btrfs: check generation when recording simple quota delta
btrfs: track metadata relocation cow with simple quota
btrfs: track data relocation with simple quota
fs/btrfs/accessors.h | 6 + fs/btrfs/backref.c | 3 + fs/btrfs/ctree.c | 22 ++- fs/btrfs/ctree.h | 1 + fs/btrfs/delayed-ref.c | 34 ++-- fs/btrfs/delayed-ref.h | 38 ++++- fs/btrfs/disk-io.c | 4 +- fs/btrfs/extent-tree.c | 231 +++++++++++++++++++++----- fs/btrfs/extent-tree.h | 6 +- fs/btrfs/file.c | 10 +- fs/btrfs/fs.h | 7 +- fs/btrfs/inode-item.c | 2 +- fs/btrfs/inode.c | 7 + fs/btrfs/ioctl.c | 6 +- fs/btrfs/print-tree.c | 12 ++ fs/btrfs/qgroup.c | 284 ++++++++++++++++++++++++++++---- fs/btrfs/qgroup.h | 28 +++- fs/btrfs/ref-verify.c | 7 +- fs/btrfs/relocation.c | 31 +++- fs/btrfs/root-tree.c | 2 +- fs/btrfs/sysfs.c | 26 +++ fs/btrfs/transaction.c | 37 ++++- fs/btrfs/tree-checker.c | 3 + fs/btrfs/tree-log.c | 3 +- include/uapi/linux/btrfs.h | 2 + include/uapi/linux/btrfs_tree.h | 13 ++ 26 files changed, 686 insertions(+), 139 deletions(-)
Signature · 1 line
2.41.0
PATCH 3/18 btrfs: introduce quota mode
In preparation for introducing simple quotas, change from a binary setting for quotas to an enum based mode. Initially, the possible modes are disabled/full. Full quotas is normal btrfs qgroups.
--- fs/btrfs/qgroup.c | 7 +++++++ fs/btrfs/qgroup.h | 6 ++++++ 2 files changed, 13 insertions(+)
fs/btrfs/qgroup.c +7 -0
@@ -30,6 +30,13 @@#include "root-tree.h" #include "tree-checker.h" +enum btrfs_qgroup_mode btrfs_qgroup_mode (struct btrfs_fs_info *fs_info) +{ +if (!test_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags)) +return BTRFS_QGROUP_MODE_DISABLED ; +return BTRFS_QGROUP_MODE_FULL ; +} +/* * Helpers to access qgroup reservation *
fs/btrfs/qgroup.h +6 -0
@@ -250,6 +250,12 @@ enum { };int btrfs_quota_enable (struct btrfs_fs_info *fs_info); +enum btrfs_qgroup_mode { +BTRFS_QGROUP_MODE_DISABLED , +BTRFS_QGROUP_MODE_FULL , +}; + +enum btrfs_qgroup_mode btrfs_qgroup_mode (struct btrfs_fs_info *fs_info);int btrfs_quota_disable (struct btrfs_fs_info *fs_info);int btrfs_qgroup_rescan (struct btrfs_fs_info *fs_info);void btrfs_qgroup_rescan_resume (struct btrfs_fs_info *fs_info);
Signature · 1 line
2.41.0
PATCH 4/18 btrfs: add new quota mode for simple quotas
In reply to Boris BurkovAdd a new quota mode called "simple quotas". It can be enabled by the existing quota enable ioctl via an unused field, and sets an incompat bit, as the implementation of simple quotas will make backwards incompatible changes to the disk format of the extent tree.
--- fs/btrfs/delayed-ref.c | 4 +- fs/btrfs/fs.h | 5 +-- fs/btrfs/ioctl.c | 2 +- fs/btrfs/qgroup.c | 88 ++++++++++++++++++++++++++------------ fs/btrfs/qgroup.h | 4 +- fs/btrfs/root-tree.c | 2 +- fs/btrfs/transaction.c | 4 +- include/uapi/linux/btrfs.h | 2 + 8 files changed, 74 insertions(+), 37 deletions(-)
fs/btrfs/delayed-ref.c +2 -2
@@ -898,7 +898,7 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans,return -ENOMEM ; } -if (test_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags) && +if (btrfs_qgroup_mode (fs_info) !=BTRFS_QGROUP_MODE_DISABLED && !generic_ref->skip_qgroup) { record =kzalloc (sizeof (*record),GFP_NOFS );if (!record) {
@@ -1002,7 +1002,7 @@ int btrfs_add_delayed_data_ref(struct btrfs_trans_handle *trans,return -ENOMEM ; } -if (test_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags) && +if (btrfs_qgroup_mode (fs_info) !=BTRFS_QGROUP_MODE_DISABLED && !generic_ref->skip_qgroup) { record =kzalloc (sizeof (*record),GFP_NOFS );if (!record) {
fs/btrfs/fs.h +2 -3
@@ -218,7 +218,8 @@ enum {BTRFS_FEATURE_INCOMPAT_NO_HOLES | \BTRFS_FEATURE_INCOMPAT_METADATA_UUID | \BTRFS_FEATURE_INCOMPAT_RAID1C34 | \ - BTRFS_FEATURE_INCOMPAT_ZONED) +BTRFS_FEATURE_INCOMPAT_ZONED | \ + BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA) #ifdef CONFIG_BTRFS_DEBUG/*
@@ -233,7 +234,6 @@ enum {#define BTRFS_FEATURE_INCOMPAT_SUPP \ (BTRFS_FEATURE_INCOMPAT_SUPP_STABLE) -#endif #define BTRFS_FEATURE_INCOMPAT_SAFE_SET \
@@ -790,7 +790,6 @@ struct btrfs_fs_info {struct lockdep_map btrfs_state_change_map[4 ];struct lockdep_map btrfs_trans_pending_ordered_map;struct lockdep_map btrfs_ordered_extent_map; -#ifdef CONFIG_BTRFS_FS_REF_VERIFY spinlock_t ref_verify_lock;struct rb_root block_tree;
fs/btrfs/ioctl.c +1 -1
@@ -3691,7 +3691,7 @@ static long btrfs_ioctl_quota_ctl(struct file *file, void __user *arg)switch (sa->cmd) {case BTRFS_QUOTA_CTL_ENABLE : - ret =btrfs_quota_enable (fs_info); + ret =btrfs_quota_enable (fs_info, sa);break ;case BTRFS_QUOTA_CTL_DISABLE : ret =btrfs_quota_disable (fs_info);
fs/btrfs/qgroup.c +61 -27
303 lines not on this page · raw message
fs/btrfs/qgroup.h +3 -1
@@ -249,13 +249,15 @@ enum { ENUM_BIT(QGROUP_FREE), }; -int btrfs_quota_enable (struct btrfs_fs_info *fs_info);enum btrfs_qgroup_mode {BTRFS_QGROUP_MODE_DISABLED ,BTRFS_QGROUP_MODE_FULL , +BTRFS_QGROUP_MODE_SIMPLE };enum btrfs_qgroup_mode btrfs_qgroup_mode (struct btrfs_fs_info *fs_info); +int btrfs_quota_enable (struct btrfs_fs_info *fs_info, +struct btrfs_ioctl_quota_ctl_args *quota_ctl_args);int btrfs_quota_disable (struct btrfs_fs_info *fs_info);int btrfs_qgroup_rescan (struct btrfs_fs_info *fs_info);void btrfs_qgroup_rescan_resume (struct btrfs_fs_info *fs_info);
fs/btrfs/root-tree.c +1 -1
@@ -508,7 +508,7 @@ int btrfs_subvolume_reserve_metadata(struct btrfs_root *root,struct btrfs_fs_info *fs_info = root->fs_info;struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; -if (test_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags)) { +if (btrfs_qgroup_mode (fs_info) !=BTRFS_QGROUP_MODE_DISABLED ) {/* One for parent inode, two for dir entries */ qgroup_num_bytes =3 * fs_info->nodesize; ret = btrfs_qgroup_reserve_meta_prealloc(root,
fs/btrfs/transaction.c +2 -2
@@ -1523,11 +1523,11 @@ static int qgroup_account_snapshot(struct btrfs_trans_handle *trans,int ret;/* -* Save some performance in the case that qgroups are not +* Save some performance in the case that full qgroups are not * enabled. If this check races with the ioctl, rescan will * kick in anyway. */ -if (!test_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags)) +if (btrfs_qgroup_mode (fs_info) !=BTRFS_QGROUP_MODE_FULL )return 0 ;/*
include/uapi/linux/btrfs.h +2 -0
@@ -333,6 +333,7 @@ struct btrfs_ioctl_fs_info_args {#define BTRFS_FEATURE_INCOMPAT_RAID1C34 (1ULL << 11)#define BTRFS_FEATURE_INCOMPAT_ZONED (1ULL << 12)#define BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2 (1ULL << 13) +#define BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA (1ULL << 14)struct btrfs_ioctl_feature_flags {__u64 compat_flags;
@@ -753,6 +754,7 @@ struct btrfs_ioctl_get_dev_stats {#define BTRFS_QUOTA_CTL_ENABLE 1#define BTRFS_QUOTA_CTL_DISABLE 2#define BTRFS_QUOTA_CTL_RESCAN__NOTUSED 3 +#define BTRFS_QUOTA_CTL_ENABLE_SIMPLE_QUOTA (1UL)struct btrfs_ioctl_quota_ctl_args {__u64 cmd;__u64 status;
Signature · 1 line
2.41.0
PATCH 5/18 btrfs: expose quota mode via sysfs
In reply to Boris BurkovAdd a new sysfs file
/sys/fs/btrfs/<uuid>/qgroups/mode
which prints out the mode qgroups is running in. The possible modes are disabled, qgroup, and squota
--- fs/btrfs/sysfs.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
fs/btrfs/sysfs.c +26 -0
@@ -2086,6 +2086,31 @@ static ssize_t qgroup_enabled_show(struct kobject *qgroups_kobj, }BTRFS_ATTR (qgroups, enabled, qgroup_enabled_show); +static ssize_t qgroup_mode_show (struct kobject *qgroups_kobj, +struct kobj_attribute *a, +char *buf) +{ +struct btrfs_fs_info *fs_info =to_fs_info (qgroups_kobj->parent); +char *mode ="" ; + +spin_lock (&fs_info->qgroup_lock); +switch (btrfs_qgroup_mode (fs_info)) { +case BTRFS_QGROUP_MODE_DISABLED : + mode ="disabled" ; +break ; +case BTRFS_QGROUP_MODE_FULL : + mode ="qgroup" ; +break ; +case BTRFS_QGROUP_MODE_SIMPLE : + mode ="squota" ; +break ; + } +spin_unlock (&fs_info->qgroup_lock); + +return sysfs_emit (buf,"%s\n" , mode); +} +BTRFS_ATTR (qgroups, mode, qgroup_mode_show); +static ssize_t qgroup_inconsistent_show (struct kobject *qgroups_kobj,struct kobj_attribute *a,char *buf)
@@ -2148,6 +2173,7 @@ static struct attribute *qgroups_attrs[] = {BTRFS_ATTR_PTR (qgroups, enabled),BTRFS_ATTR_PTR (qgroups, inconsistent),BTRFS_ATTR_PTR (qgroups, drop_subtree_threshold), +BTRFS_ATTR_PTR (qgroups, mode), NULL };ATTRIBUTE_GROUPS (qgroups);
Signature · 1 line
2.41.0
PATCH 6/18 btrfs: flush reservations during quota disable
In reply to Boris BurkovThe following sequence:
enable simple quotas
do some writes
reserve space
create ordered_extent
release rsv (store rsv_bytes in OE, mark QGROUP_RESERVED bits)
disable quotas
enable simple quotas
set qgroup rsv to 0 on all subvolsordered_extent finishes
create delayed ref with rsv_bytes from before
run delayed ref
record_simple_quota_delta
free rsv_bytes (0 -> -rsv_delta)results in us reliably underflowing the subvolume's qgroup rsv counter, because disabling/re-enabling quotas toggles reservation counters down to 0, but does not remove other file system state which represents successful acquisition of qgroup rsv space. Specifically metadata rsv counters on the root object and rsv_bytes on ordered_extent objects that have released their reservation as well as the corresponding QGROUP_RESERVED extent bits.
Normal qgroups gets away with this, I believe because it forces more work to happen on transaction commit, but I am not certain it is totally safe from the ordered_extent/leaked extent bit variant. Simple quotas hits this reliably.
The intent of the fix is to make disable take the time to clear that external to qgroups state as well: after flipping off the quota bit on fs_info, flush delalloc and ordered extents, clearing the extent bits along the way. This makes it so there are no ordered extents or meta prealloc hanging around from the first enablement period during the second.
--- fs/btrfs/qgroup.c | 47 ++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 44 insertions(+), 3 deletions(-)
fs/btrfs/qgroup.c +44 -3
@@ -1247,6 +1247,40 @@ int btrfs_quota_enable(struct btrfs_fs_info *fs_info,return ret; } +/* +* It is possible to have outstanding ordered extents +* which reserved bytes before we disabled. We need to fully flush +* delalloc, ordered extents, and a commit to ensure that +* we don't leak such reservations, only to have them come back +* if we re-enable. +* +* i.e.: +* enable simple quotas +* reserve space +* release it, store rsv_bytes in OE +* disable quotas +* enable simple quotas (qgroup rsv are all 0) +* OE finishes +* run delayed refs +* free rsv_bytes, resulting in miscounting or even underflow +*/ +static int flush_reservations (struct btrfs_fs_info *fs_info) +{ +struct btrfs_trans_handle *trans; +int ret; + + ret =btrfs_start_delalloc_roots (fs_info,LONG_MAX , false); +if (ret) +return ret; +btrfs_wait_ordered_roots (fs_info,U64_MAX ,0 , (u64)-1 ); + trans =btrfs_join_transaction (fs_info->tree_root); +if (IS_ERR (trans)) +return PTR_ERR (trans); +btrfs_commit_transaction (trans); + +return ret; +} +int btrfs_quota_disable (struct btrfs_fs_info *fs_info) {struct btrfs_root *quota_root;
@@ -1291,6 +1325,10 @@ int btrfs_quota_disable(struct btrfs_fs_info *fs_info)clear_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags);btrfs_qgroup_wait_for_completion (fs_info, false); + ret =flush_reservations (fs_info); +if (ret) + goto out; +/* * 1 For the root item *
@@ -1351,7 +1389,7 @@ int btrfs_quota_disable(struct btrfs_fs_info *fs_info)if (ret && trans)btrfs_end_transaction (trans);else if (trans) - ret =btrfs_end_transaction (trans); + ret =btrfs_commit_transaction (trans);mutex_unlock (&fs_info->cleaner_mutex);return ret;
@@ -3954,8 +3992,11 @@ static int __btrfs_qgroup_release_data(struct btrfs_inode *inode,int trace_op =QGROUP_RELEASE ;int ret; -if (btrfs_qgroup_mode (inode->root->fs_info) ==BTRFS_QGROUP_MODE_DISABLED ) -return 0 ; +if (btrfs_qgroup_mode (inode->root->fs_info) ==BTRFS_QGROUP_MODE_DISABLED ) { +extent_changeset_init (&changeset); +return clear_record_extent_bits (&inode->io_tree, start, start + len -1 , +EXTENT_QGROUP_RESERVED , &changeset); + }/* In release case, we shouldn't have @reserved */ WARN_ON (!free && reserved);
Signature · 1 line
2.41.0
PATCH 7/18 btrfs: create qgroup earlier in snapshot creation
In reply to Boris BurkovPull creating the qgroup earlier in the snapshot. This allows simple quotas qgroups to see all the metadata writes related to the snapshot being created and to be born with the root node accounted.
--- fs/btrfs/qgroup.c | 3 +++ fs/btrfs/transaction.c | 5 +++++ 2 files changed, 8 insertions(+)
fs/btrfs/qgroup.c +3 -0
@@ -1670,6 +1670,9 @@ int btrfs_create_qgroup(struct btrfs_trans_handle *trans, u64 qgroupid)struct btrfs_qgroup *qgroup;int ret =0 ; +if (btrfs_qgroup_mode (fs_info) ==BTRFS_QGROUP_MODE_DISABLED ) +return 0 ; +mutex_lock (&fs_info->qgroup_ioctl_lock);if (!fs_info->quota_root) { ret = -ENOTCONN ;
fs/btrfs/transaction.c +5 -0
@@ -1716,6 +1716,11 @@ static noinline int create_pending_snapshot(struct btrfs_trans_handle *trans, }btrfs_release_path (path); + ret =btrfs_create_qgroup (trans, objectid); +if (ret) { +btrfs_abort_transaction (trans, ret); + goto fail; + }/* * pull in the delayed directory update * and the delayed inode item
Signature · 1 line
2.41.0
PATCH 8/18 btrfs: function for recording simple quota deltas
In reply to Boris BurkovRather than re-computing shared/exclusive ownership based on backrefs and walking roots for implicit backrefs, simple quotas does an increment when creating an extent and a decrement when deleting it. Add the API for the extent item code to use to track those events.
Also add a helper function to make collecting parent qgroups in a ulist easier for functions like this.
--- fs/btrfs/qgroup.c | 82 +++++++++++++++++++++++++++++++++++++++++++++++ fs/btrfs/qgroup.h | 11 ++++++- 2 files changed, 92 insertions(+), 1 deletion(-)
fs/btrfs/qgroup.c +82 -0
@@ -333,6 +333,44 @@ static int del_relation_rb(struct btrfs_fs_info *fs_info,return -ENOENT ; } +static int qgroup_collect_parents (struct btrfs_qgroup *qgroup, +struct ulist *ul) +{ +struct ulist_iterator uiter; +struct ulist_node *unode; +struct btrfs_qgroup_list *glist; +struct btrfs_qgroup *qg; +bool err_free = false; +int ret =0 ; + +if (!ul) { + ul =ulist_alloc (GFP_KERNEL ); + err_free = true; + }else { +ulist_reinit (ul); + } + + ret =ulist_add (ul, qgroup->qgroupid, +qgroup_to_aux (qgroup),GFP_ATOMIC ); +if (ret <0 ) + goto out; +ULIST_ITER_INIT (&uiter); +while ((unode =ulist_next (ul, &uiter))) { + qg =unode_aux_to_qgroup (unode); +list_for_each_entry (glist, &qg->groups, next_group) { + ret =ulist_add (ul, glist->group->qgroupid, +qgroup_to_aux (glist->group),GFP_ATOMIC ); +if (ret <0 ) + goto out; + } + } + ret =0 ; +out: +if (ret && err_free) +ulist_free (ul); +return ret; +} +#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS int btrfs_verify_qgroup_counts (struct btrfs_fs_info *fs_info,u64 qgroupid,u64 rfer,u64 excl)
@@ -4531,3 +4569,47 @@ void btrfs_qgroup_destroy_extent_records(struct btrfs_transaction *trans)kfree (entry); } } + +int btrfs_record_simple_quota_delta (struct btrfs_fs_info *fs_info, +struct btrfs_simple_quota_delta *delta) +{ +int ret; +struct ulist *ul = fs_info->qgroup_ulist; +struct btrfs_qgroup *qgroup; +struct ulist_iterator uiter; +struct ulist_node *unode; +struct btrfs_qgroup *qg; +u64 root = delta->root; +u64 num_bytes = delta->num_bytes; +int sign = delta->is_inc ?1 :-1 ; + +if (btrfs_qgroup_mode (fs_info) !=BTRFS_QGROUP_MODE_SIMPLE ) +return 0 ; + +if (!is_fstree (root)) +return 0 ; + +spin_lock (&fs_info->qgroup_lock); + qgroup =find_qgroup_rb (fs_info, root); +if (!qgroup) { + ret = -ENOENT ; + goto out; + } + ret =qgroup_collect_parents (qgroup, ul); +if (ret) + goto out; + +ULIST_ITER_INIT (&uiter); +while ((unode =ulist_next (ul, &uiter))) { + qg =unode_aux_to_qgroup (unode); + qg->excl += num_bytes * sign; + qg->rfer += num_bytes * sign; +qgroup_dirty (fs_info, qg); + } + +out: +spin_unlock (&fs_info->qgroup_lock); +if (!ret && delta->rsv_bytes) +btrfs_qgroup_free_refroot (fs_info, root, delta->rsv_bytes,BTRFS_QGROUP_RSV_DATA ); +return ret; +}
fs/btrfs/qgroup.h +10 -1
@@ -235,6 +235,14 @@ struct btrfs_qgroup {struct kobject kobj; }; +struct btrfs_simple_quota_delta { +u64 root;/* The fstree root this delta counts against */ +u64 num_bytes;/* The number of bytes in the extent being counted */ +u64 rsv_bytes;/* The number of bytes reserved for this extent */ +bool is_inc;/* Whether we are using or freeing the extent */ +bool is_data;/* Whether the extent is data or metadata */ +}; +static inline u64 btrfs_qgroup_subvolid (u64 qgroupid) {return (qgroupid & ((1ULL <<BTRFS_QGROUP_LEVEL_SHIFT ) -1 ));
@@ -447,5 +455,6 @@ int btrfs_qgroup_trace_subtree_after_cow(struct btrfs_trans_handle *trans, struct btrfs_root *root, struct extent_buffer *eb);void btrfs_qgroup_destroy_extent_records (struct btrfs_transaction *trans);bool btrfs_check_quota_leak (struct btrfs_fs_info *fs_info); - +int btrfs_record_simple_quota_delta (struct btrfs_fs_info *fs_info, +struct btrfs_simple_quota_delta *delta);#endif
Signature · 1 line
2.41.0
PATCH 9/18 btrfs: rename tree_ref and data_ref owning_root
In reply to Boris Burkovcommit 113479d5b8eb ("btrfs: rename root fields in delayed refs structs") changed these from ref_root to owning_root. However, there are many circumstances where that name is not really accurate and the root on the ref struct _is_ the referring root. In general, these are not the owning root, though it does happen in some ref merging cases involving overwrites during snapshots and similar.
Simple quotas cares quite a bit about tracking the original owner of an extent through delayed refs, so rename these back to free up the name for the real owning root (which will live on the generic btrfs_ref and the head ref)
--- fs/btrfs/delayed-ref.c | 10 +++++----- fs/btrfs/delayed-ref.h | 12 ++++++------ fs/btrfs/extent-tree.c | 10 +++++----- fs/btrfs/ref-verify.c | 4 ++-- 4 files changed, 18 insertions(+), 18 deletions(-)
fs/btrfs/delayed-ref.c +5 -5
@@ -885,7 +885,7 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans,u64 parent = generic_ref->parent;u8 ref_type; - is_system = (generic_ref->tree_ref.owning_root ==BTRFS_CHUNK_TREE_OBJECTID ); + is_system = (generic_ref->tree_ref.ref_root ==BTRFS_CHUNK_TREE_OBJECTID );ASSERT (generic_ref->type ==BTRFS_REF_METADATA && generic_ref->action); ref =kmem_cache_alloc (btrfs_delayed_tree_ref_cachep,GFP_NOFS );
@@ -914,14 +914,14 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans, ref_type =BTRFS_TREE_BLOCK_REF_KEY ;init_delayed_ref_common (fs_info, &ref->node, bytenr, num_bytes, - generic_ref->tree_ref.owning_root, action, + generic_ref->tree_ref.ref_root, action, ref_type); - ref->root = generic_ref->tree_ref.owning_root; + ref->root = generic_ref->tree_ref.ref_root; ref->parent = parent; ref->level = level;init_delayed_ref_head (head_ref, record, bytenr, num_bytes, - generic_ref->tree_ref.owning_root,0 , action, + generic_ref->tree_ref.ref_root,0 , action, false, is_system); head_ref->extent_op = extent_op;
@@ -974,7 +974,7 @@ int btrfs_add_delayed_data_ref(struct btrfs_trans_handle *trans,u64 bytenr = generic_ref->bytenr;u64 num_bytes = generic_ref->len;u64 parent = generic_ref->parent; -u64 ref_root = generic_ref->data_ref.owning_root; +u64 ref_root = generic_ref->data_ref.ref_root;u64 owner = generic_ref->data_ref.ino;u64 offset = generic_ref->data_ref.offset;u8 ref_type;
fs/btrfs/delayed-ref.h +6 -6
@@ -188,8 +188,8 @@ enum btrfs_ref_type {struct btrfs_data_ref {/* For EXTENT_DATA_REF */ -/* Original root this data extent belongs to */ -u64 owning_root; +/* Root which owns this data reference */ +u64 ref_root;/* Inode which refers to this data extent */ u64 ino;
@@ -212,11 +212,11 @@ struct btrfs_tree_ref {int level;/* -* Root which owns this tree block. +* Root which owns this tree block reference. * * For TREE_BLOCK_REF (skinny metadata, either inline or keyed) */ -u64 owning_root; +u64 ref_root;/* For non-skinny metadata, no special member needed */ };
@@ -294,7 +294,7 @@ static inline void btrfs_init_tree_ref(struct btrfs_ref *generic_ref, generic_ref->real_root = mod_root ?: root;#endif generic_ref->tree_ref.level = level; - generic_ref->tree_ref.owning_root = root; + generic_ref->tree_ref.ref_root = root; generic_ref->type =BTRFS_REF_METADATA ;if (skip_qgroup || !(is_fstree (root) && (!mod_root ||is_fstree (mod_root))))
@@ -312,7 +312,7 @@ static inline void btrfs_init_data_ref(struct btrfs_ref *generic_ref,/* If @real_root not set, use @root as fallback */ generic_ref->real_root = mod_root ?: ref_root;#endif - generic_ref->data_ref.owning_root = ref_root; + generic_ref->data_ref.ref_root = ref_root; generic_ref->data_ref.ino = ino; generic_ref->data_ref.offset = offset; generic_ref->type =BTRFS_REF_DATA ;
fs/btrfs/extent-tree.c +5 -5
@@ -1386,7 +1386,7 @@ int btrfs_inc_extent_ref(struct btrfs_trans_handle *trans,ASSERT (generic_ref->type !=BTRFS_REF_NOT_SET && generic_ref->action);BUG_ON (generic_ref->type ==BTRFS_REF_METADATA && - generic_ref->tree_ref.owning_root ==BTRFS_TREE_LOG_OBJECTID ); + generic_ref->tree_ref.ref_root ==BTRFS_TREE_LOG_OBJECTID );if (generic_ref->type ==BTRFS_REF_METADATA ) ret =btrfs_add_delayed_tree_ref (trans, generic_ref,NULL );
@@ -3329,9 +3329,9 @@ int btrfs_free_extent(struct btrfs_trans_handle *trans, struct btrfs_ref *ref)* tree, just update pinning info and exit early. */ if ((ref->type ==BTRFS_REF_METADATA && - ref->tree_ref.owning_root ==BTRFS_TREE_LOG_OBJECTID ) || + ref->tree_ref.ref_root ==BTRFS_TREE_LOG_OBJECTID ) || (ref->type ==BTRFS_REF_DATA && - ref->data_ref.owning_root ==BTRFS_TREE_LOG_OBJECTID )) { + ref->data_ref.ref_root ==BTRFS_TREE_LOG_OBJECTID )) {/* unlocks the pinned mutex */ btrfs_pin_extent (trans, ref->bytenr, ref->len,1 ); ret =0 ;
@@ -3342,9 +3342,9 @@ int btrfs_free_extent(struct btrfs_trans_handle *trans, struct btrfs_ref *ref) }if (!((ref->type ==BTRFS_REF_METADATA && - ref->tree_ref.owning_root ==BTRFS_TREE_LOG_OBJECTID ) || + ref->tree_ref.ref_root ==BTRFS_TREE_LOG_OBJECTID ) || (ref->type ==BTRFS_REF_DATA && - ref->data_ref.owning_root ==BTRFS_TREE_LOG_OBJECTID ))) + ref->data_ref.ref_root ==BTRFS_TREE_LOG_OBJECTID )))btrfs_ref_tree_mod (fs_info, ref);return ret;
fs/btrfs/ref-verify.c +2 -2
@@ -681,10 +681,10 @@ int btrfs_ref_tree_mod(struct btrfs_fs_info *fs_info,if (generic_ref->type ==BTRFS_REF_METADATA ) {if (!parent) - ref_root = generic_ref->tree_ref.owning_root; + ref_root = generic_ref->tree_ref.ref_root; owner = generic_ref->tree_ref.level; }else if (!parent) { - ref_root = generic_ref->data_ref.owning_root; + ref_root = generic_ref->data_ref.ref_root; owner = generic_ref->data_ref.ino; offset = generic_ref->data_ref.offset; }
Signature · 1 line
2.41.0
PATCH 10/18 btrfs: track owning root in btrfs_ref
In reply to Boris BurkovWhile data extents require us to store additional inline refs to track the original owner on free, this information is available implicitly for metadata. It is found in the owner field of the header of the tree block. Even if other trees refer to this block and the original ref goes away, we will not rewrite that header field, so it will reliably give the original owner.
In addition, there is a relocation case where a new data extent needs to have an owning root separate from the referring root wired through delayed refs.
To use it for recording simple quota deltas, we need to wire this root id through from when we create the delayed ref until we fully process it. Store it in the generic btrfs_ref struct of the delayed ref.
--- fs/btrfs/delayed-ref.c | 7 ++++--- fs/btrfs/delayed-ref.h | 13 +++++++++++-- fs/btrfs/extent-tree.c | 19 ++++++++++++------- fs/btrfs/file.c | 10 +++++----- fs/btrfs/inode-item.c | 2 +- fs/btrfs/relocation.c | 16 +++++++++------- fs/btrfs/tree-log.c | 3 ++- 7 files changed, 44 insertions(+), 26 deletions(-)
fs/btrfs/delayed-ref.c +4 -3
@@ -840,7 +840,7 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans,static void init_delayed_ref_common (struct btrfs_fs_info *fs_info,struct btrfs_delayed_ref_node *ref,u64 bytenr,u64 num_bytes,u64 ref_root, -int action,u8 ref_type) +int action,u8 ref_type,u64 owning_root) {u64 seq =0 ;
@@ -857,6 +857,7 @@ static void init_delayed_ref_common(struct btrfs_fs_info *fs_info, ref->action = action; ref->seq = seq; ref->type = ref_type; + ref->owning_root = owning_root;RB_CLEAR_NODE (&ref->ref_node);INIT_LIST_HEAD (&ref->add_list); }
@@ -915,7 +916,7 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans,init_delayed_ref_common (fs_info, &ref->node, bytenr, num_bytes, generic_ref->tree_ref.ref_root, action, - ref_type); + ref_type, generic_ref->owning_root); ref->root = generic_ref->tree_ref.ref_root; ref->parent = parent; ref->level = level;
@@ -989,7 +990,7 @@ int btrfs_add_delayed_data_ref(struct btrfs_trans_handle *trans,else ref_type =BTRFS_EXTENT_DATA_REF_KEY ;init_delayed_ref_common (fs_info, &ref->node, bytenr, num_bytes, - ref_root, action, ref_type); + ref_root, action, ref_type, ref_root); ref->root = ref_root; ref->parent = parent; ref->objectid = owner;
fs/btrfs/delayed-ref.h +11 -2
@@ -32,6 +32,12 @@ struct btrfs_delayed_ref_node {/* seq number to keep track of insertion order */ u64 seq; +/* +* root which originally allocated this extent and owns it for +* simple quota accounting purposes. +*/ +u64 owning_root; +/* ref count on this data structure */ refcount_t refs;
@@ -239,6 +245,7 @@ struct btrfs_ref {#endif u64 bytenr;u64 len; +u64 owning_root;/* Bytenr of the parent tree block */ u64 parent;
@@ -278,16 +285,18 @@ static inline u64 btrfs_calc_delayed_ref_bytes(const struct btrfs_fs_info *fs_in }static inline void btrfs_init_generic_ref (struct btrfs_ref *generic_ref, -int action,u64 bytenr,u64 len,u64 parent) +int action,u64 bytenr,u64 len,u64 parent,u64 owning_root) { generic_ref->action = action; generic_ref->bytenr = bytenr; generic_ref->len = len; generic_ref->parent = parent; + generic_ref->owning_root = owning_root; }static inline void btrfs_init_tree_ref (struct btrfs_ref *generic_ref, -int level,u64 root,u64 mod_root,bool skip_qgroup) +int level,u64 root,u64 mod_root, +bool skip_qgroup) {#ifdef CONFIG_BTRFS_FS_REF_VERIFY /* If @real_root not set, use @root as fallback */
fs/btrfs/extent-tree.c +12 -7
@@ -2410,7 +2410,7 @@ static int __btrfs_mod_ref(struct btrfs_trans_handle *trans, num_bytes =btrfs_file_extent_disk_num_bytes (buf, fi); key.offset -=btrfs_file_extent_offset (buf, fi);btrfs_init_generic_ref (&generic_ref, action, bytenr, - num_bytes, parent); + num_bytes, parent, ref_root);btrfs_init_data_ref (&generic_ref, ref_root, key.objectid, key.offset, root->root_key.objectid, for_reloc);
@@ -2424,7 +2424,7 @@ static int __btrfs_mod_ref(struct btrfs_trans_handle *trans, bytenr =btrfs_node_blockptr (buf, i); num_bytes = fs_info->nodesize;btrfs_init_generic_ref (&generic_ref, action, bytenr, - num_bytes, parent); + num_bytes, parent, ref_root);btrfs_init_tree_ref (&generic_ref, level -1 , ref_root, root->root_key.objectid, for_reloc);if (inc)
@@ -3242,7 +3242,7 @@ void btrfs_free_tree_block(struct btrfs_trans_handle *trans,int ret;btrfs_init_generic_ref (&generic_ref,BTRFS_DROP_DELAYED_REF , - buf->start, buf->len, parent); + buf->start, buf->len, parent,btrfs_header_owner (buf));btrfs_init_tree_ref (&generic_ref,btrfs_header_level (buf), root_id,0 , false);
@@ -4699,12 +4699,16 @@ int btrfs_alloc_reserved_file_extent(struct btrfs_trans_handle *trans, struct btrfs_key *ins) {struct btrfs_ref generic_ref = {0 }; +u64 root_objectid = root->root_key.objectid; +u64 owning_root = root_objectid; + +BUG_ON (root_objectid ==BTRFS_TREE_LOG_OBJECTID );BUG_ON (root->root_key.objectid ==BTRFS_TREE_LOG_OBJECTID );btrfs_init_generic_ref (&generic_ref,BTRFS_ADD_DELAYED_EXTENT , - ins->objectid, ins->offset,0 ); -btrfs_init_data_ref (&generic_ref, root->root_key.objectid, owner, + ins->objectid, ins->offset,0 , owning_root); +btrfs_init_data_ref (&generic_ref, root_objectid, owner, offset,0 , false);btrfs_ref_tree_mod (root->fs_info, &generic_ref);
@@ -4916,7 +4920,7 @@ struct extent_buffer *btrfs_alloc_tree_block(struct btrfs_trans_handle *trans, extent_op->level = level;btrfs_init_generic_ref (&generic_ref,BTRFS_ADD_DELAYED_EXTENT , - ins.objectid, ins.offset, parent); + ins.objectid, ins.offset, parent,btrfs_header_owner (buf));btrfs_init_tree_ref (&generic_ref, level, root_objectid, root->root_key.objectid, false);btrfs_ref_tree_mod (fs_info, &generic_ref);
@@ -5336,8 +5340,9 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans, wc->drop_level = level;find_next_key (path, level, &wc->drop_progress); +// TODO owning root on generic ref!? btrfs_init_generic_ref (&ref,BTRFS_DROP_DELAYED_REF , bytenr, - fs_info->nodesize, parent); + fs_info->nodesize, parent,btrfs_header_owner (next));btrfs_init_tree_ref (&ref, level -1 , root->root_key.objectid,0 , false); ret =btrfs_free_extent (trans, &ref);
fs/btrfs/file.c +5 -5
@@ -373,7 +373,7 @@ int btrfs_drop_extents(struct btrfs_trans_handle *trans,if (update_refs && disk_bytenr >0 ) {btrfs_init_generic_ref (&ref,BTRFS_ADD_DELAYED_REF , - disk_bytenr, num_bytes,0 ); + disk_bytenr, num_bytes,0 , root->root_key.objectid); btrfs_init_data_ref(&ref, root->root_key.objectid, new_key.objectid,
@@ -463,7 +463,7 @@ int btrfs_drop_extents(struct btrfs_trans_handle *trans, }else if (update_refs && disk_bytenr >0 ) {btrfs_init_generic_ref (&ref,BTRFS_DROP_DELAYED_REF , - disk_bytenr, num_bytes,0 ); + disk_bytenr, num_bytes,0 , root->root_key.objectid); btrfs_init_data_ref(&ref, root->root_key.objectid, key.objectid,
@@ -745,7 +745,7 @@ int btrfs_mark_extent_written(struct btrfs_trans_handle *trans,btrfs_mark_buffer_dirty (leaf);btrfs_init_generic_ref (&ref,BTRFS_ADD_DELAYED_REF , bytenr, - num_bytes,0 ); + num_bytes,0 , root->root_key.objectid);btrfs_init_data_ref (&ref, root->root_key.objectid, ino, orig_offset,0 , false); ret =btrfs_inc_extent_ref (trans, &ref);
@@ -771,7 +771,7 @@ int btrfs_mark_extent_written(struct btrfs_trans_handle *trans, other_start = end; other_end =0 ;btrfs_init_generic_ref (&ref,BTRFS_DROP_DELAYED_REF , bytenr, - num_bytes,0 ); + num_bytes,0 , root->root_key.objectid);btrfs_init_data_ref (&ref, root->root_key.objectid, ino, orig_offset,0 , false); if (extent_mergeable(leaf, path->slots[0] + 1,
@@ -2294,7 +2294,7 @@ static int btrfs_insert_replace_extent(struct btrfs_trans_handle *trans,btrfs_init_generic_ref (&ref,BTRFS_ADD_DELAYED_REF , extent_info->disk_offset, - extent_info->disk_len,0 ); + extent_info->disk_len,0 , root->root_key.objectid); ref_offset = extent_info->file_offset - extent_info->data_offset;btrfs_init_data_ref (&ref, root->root_key.objectid,btrfs_ino (inode), ref_offset,0 , false);
fs/btrfs/inode-item.c +1 -1
@@ -676,7 +676,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, bytes_deleted += extent_num_bytes;btrfs_init_generic_ref (&ref,BTRFS_DROP_DELAYED_REF , - extent_start, extent_num_bytes,0 ); + extent_start, extent_num_bytes,0 , root->root_key.objectid);btrfs_init_data_ref (&ref,btrfs_header_owner (leaf), control->ino, extent_offset, root->root_key.objectid, false);
fs/btrfs/relocation.c +9 -7
@@ -1158,7 +1158,7 @@ int replace_file_extents(struct btrfs_trans_handle *trans, key.offset -=btrfs_file_extent_offset (leaf, fi);btrfs_init_generic_ref (&ref,BTRFS_ADD_DELAYED_REF , new_bytenr, - num_bytes, parent); + num_bytes, parent, root->root_key.objectid);btrfs_init_data_ref (&ref,btrfs_header_owner (leaf), key.objectid, key.offset, root->root_key.objectid, false);
@@ -1169,7 +1169,7 @@ int replace_file_extents(struct btrfs_trans_handle *trans, }btrfs_init_generic_ref (&ref,BTRFS_DROP_DELAYED_REF , bytenr, - num_bytes, parent); + num_bytes, parent, root->root_key.objectid);btrfs_init_data_ref (&ref,btrfs_header_owner (leaf), key.objectid, key.offset, root->root_key.objectid, false);
@@ -1382,7 +1382,8 @@ int replace_path(struct btrfs_trans_handle *trans, struct reloc_control *rc,btrfs_mark_buffer_dirty (path->nodes[level]);btrfs_init_generic_ref (&ref,BTRFS_ADD_DELAYED_REF , old_bytenr, - blocksize, path->nodes[level]->start); + blocksize, path->nodes[level]->start, + src->root_key.objectid);btrfs_init_tree_ref (&ref, level -1 , src->root_key.objectid,0 , true); ret =btrfs_inc_extent_ref (trans, &ref);
@@ -1391,7 +1392,7 @@ int replace_path(struct btrfs_trans_handle *trans, struct reloc_control *rc,break ; }btrfs_init_generic_ref (&ref,BTRFS_ADD_DELAYED_REF , new_bytenr, - blocksize,0 ); + blocksize,0 , dest->root_key.objectid);btrfs_init_tree_ref (&ref, level -1 , dest->root_key.objectid,0 , true); ret =btrfs_inc_extent_ref (trans, &ref);
@@ -1401,7 +1402,8 @@ int replace_path(struct btrfs_trans_handle *trans, struct reloc_control *rc, }btrfs_init_generic_ref (&ref,BTRFS_DROP_DELAYED_REF , new_bytenr, - blocksize, path->nodes[level]->start); + blocksize, path->nodes[level]->start, + src->root_key.objectid);btrfs_init_tree_ref (&ref, level -1 , src->root_key.objectid,0 , true); ret =btrfs_free_extent (trans, &ref);
@@ -1411,7 +1413,7 @@ int replace_path(struct btrfs_trans_handle *trans, struct reloc_control *rc, }btrfs_init_generic_ref (&ref,BTRFS_DROP_DELAYED_REF , old_bytenr, - blocksize,0 ); + blocksize,0 , src->root_key.objectid);btrfs_init_tree_ref (&ref, level -1 , dest->root_key.objectid,0 , true); ret =btrfs_free_extent (trans, &ref);
@@ -2491,7 +2493,7 @@ static int do_relocation(struct btrfs_trans_handle *trans,btrfs_init_generic_ref (&ref,BTRFS_ADD_DELAYED_REF , node->eb->start, blocksize, - upper->eb->start); + upper->eb->start,btrfs_header_owner (upper->eb));btrfs_init_tree_ref (&ref, node->level,btrfs_header_owner (upper->eb), root->root_key.objectid, false);
fs/btrfs/tree-log.c +2 -1
@@ -767,7 +767,8 @@ static noinline int replay_one_extent(struct btrfs_trans_handle *trans, }else if (ret ==0 ) {btrfs_init_generic_ref (&ref,BTRFS_ADD_DELAYED_REF , - ins.objectid, ins.offset,0 ); + ins.objectid, ins.offset,0 , + root->root_key.objectid);btrfs_init_data_ref (&ref, root->root_key.objectid, key->objectid, offset,0 , false);
Signature · 1 line
2.41.0
PATCH 11/18 btrfs: track original extent owner in head_ref
In reply to Boris BurkovSimple quotas requires tracking the original creating root of any given extent. This gets complicated when multiple subvolumes create overlapping/contradictory refs in the same transaction. For example, due to modifying or deleting an extent while also snapshotting it.
To resolve this in a general way, take advantage of the fact that we are essentially already tracking this for handling releasing reservations. The head ref coalesces the various refs and uses must_insert_reserved to check if it needs to create an extent/free reservation. Store the ref that set must_insert_reserved as the owning ref on the head ref.
Note that this can result in writing an extent for the very first time with an owner different from its only ref, but it will look the same as if you first created it with the original owning ref, then added the other ref, then removed the owning ref.
--- fs/btrfs/delayed-ref.c | 10 ++++++---- fs/btrfs/delayed-ref.h | 7 +++++++ 2 files changed, 13 insertions(+), 4 deletions(-)
fs/btrfs/delayed-ref.c +6 -4
@@ -632,6 +632,7 @@ static noinline void update_existing_head_ref(struct btrfs_trans_handle *trans,* Set it again here */ existing->must_insert_reserved = update->must_insert_reserved; + existing->owning_root = update->owning_root;/* * update the num_bytes so we make sure the accounting
@@ -694,7 +695,7 @@ static void init_delayed_ref_head(struct btrfs_delayed_ref_head *head_ref,struct btrfs_qgroup_extent_record *qrecord, u64 bytenr, u64 num_bytes, u64 ref_root, u64 reserved, int action, bool is_data, - bool is_system) + bool is_system, u64 owning_root) { int count_mod = 1;bool must_insert_reserved = false;
@@ -735,6 +736,7 @@ static void init_delayed_ref_head(struct btrfs_delayed_ref_head *head_ref, head_ref->num_bytes = num_bytes; head_ref->ref_mod = count_mod; head_ref->must_insert_reserved = must_insert_reserved; + head_ref->owning_root = owning_root; head_ref->is_data = is_data; head_ref->is_system = is_system; head_ref->ref_tree =RB_ROOT_CACHED ;
@@ -923,7 +925,7 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans,init_delayed_ref_head (head_ref, record, bytenr, num_bytes, generic_ref->tree_ref.ref_root,0 , action, - false, is_system); + false, is_system, generic_ref->owning_root); head_ref->extent_op = extent_op; delayed_refs = &trans->transaction->delayed_refs;
@@ -1015,7 +1017,7 @@ int btrfs_add_delayed_data_ref(struct btrfs_trans_handle *trans, }init_delayed_ref_head (head_ref, record, bytenr, num_bytes, ref_root, - reserved, action, true, false); + reserved, action, true, false, generic_ref->owning_root); head_ref->extent_op =NULL ; delayed_refs = &trans->transaction->delayed_refs;
@@ -1061,7 +1063,7 @@ int btrfs_add_delayed_extent_op(struct btrfs_trans_handle *trans,return -ENOMEM ;init_delayed_ref_head (head_ref,NULL , bytenr, num_bytes,0 ,0 , -BTRFS_UPDATE_DELAYED_HEAD , false, false); +BTRFS_UPDATE_DELAYED_HEAD , false, false,0 ); head_ref->extent_op = extent_op; delayed_refs = &trans->transaction->delayed_refs;
fs/btrfs/delayed-ref.h +7 -0
@@ -123,6 +123,13 @@ struct btrfs_delayed_ref_head {* the free has happened. */ bool must_insert_reserved; + +/* +* The root which triggered the allocation when +* must_insert_reserved is true +*/ +u64 owning_root; +bool is_data;bool is_system;bool processing;
Signature · 1 line
2.41.0
PATCH 12/18 btrfs: new inline ref storing owning subvol of data extents
In reply to Boris BurkovIn order to implement simple quota groups, we need to be able to associate a data extent with the subvolume that created it. Once you account for reflink, this information cannot be recovered without explicitly storing it. Options for storing it are:
- a new key/item
- a new extent inline ref item
The former is backwards compatible, but wastes space, the latter is incompat, but is efficient in space and reuses the existing inline ref machinery, while only abusing it a tiny amount -- specifically, the new item is not a ref, per-se.
--- fs/btrfs/accessors.h | 4 +++ fs/btrfs/backref.c | 3 ++ fs/btrfs/extent-tree.c | 51 ++++++++++++++++++++++++++------- fs/btrfs/print-tree.c | 12 ++++++++ fs/btrfs/ref-verify.c | 3 ++ fs/btrfs/tree-checker.c | 3 ++ include/uapi/linux/btrfs_tree.h | 6 ++++ 7 files changed, 71 insertions(+), 11 deletions(-)
fs/btrfs/accessors.h +4 -0
@@ -350,6 +350,8 @@ BTRFS_SETGET_FUNCS(extent_data_ref_count, struct btrfs_extent_data_ref, count, 3 BTRFS_SETGET_FUNCS(shared_data_ref_count, struct btrfs_shared_data_ref, count, 32); +BTRFS_SETGET_FUNCS(extent_owner_ref_root_id, struct btrfs_extent_owner_ref, root_id, 64); + BTRFS_SETGET_FUNCS(extent_inline_ref_type, struct btrfs_extent_inline_ref, type,8 ); BTRFS_SETGET_FUNCS(extent_inline_ref_offset, struct btrfs_extent_inline_ref,
@@ -366,6 +368,8 @@ static inline u32 btrfs_extent_inline_ref_size(int type)if (type ==BTRFS_EXTENT_DATA_REF_KEY )return sizeof (struct btrfs_extent_data_ref ) + offsetof(struct btrfs_extent_inline_ref , offset); +if (type ==BTRFS_EXTENT_OWNER_REF_KEY ) +return sizeof (struct btrfs_extent_inline_ref );return 0 ; }
fs/btrfs/backref.c +3 -0
@@ -1129,6 +1129,9 @@ static int add_inline_refs(struct btrfs_backref_walk_ctx *ctx, count, sc, GFP_NOFS);break ; } +case BTRFS_EXTENT_OWNER_REF_KEY : +WARN_ON (!btrfs_fs_incompat (ctx->fs_info,SIMPLE_QUOTA )); +break ;default :WARN_ON (1 ); }
fs/btrfs/extent-tree.c +40 -11
144 lines not on this page · raw message
fs/btrfs/print-tree.c +12 -0
@@ -80,12 +80,20 @@ static void print_extent_data_ref(const struct extent_buffer *eb, btrfs_extent_data_ref_count(eb, ref)); } +static void print_extent_owner_ref (const struct extent_buffer *eb, +struct btrfs_extent_owner_ref *ref) +{ +WARN_ON (!btrfs_fs_incompat (eb->fs_info,SIMPLE_QUOTA )); +pr_cont ("extent data owner root %llu\n" ,btrfs_extent_owner_ref_root_id (eb, ref)); +} +static void print_extent_item (const struct extent_buffer *eb,int slot,int type) {struct btrfs_extent_item *ei;struct btrfs_extent_inline_ref *iref;struct btrfs_extent_data_ref *dref;struct btrfs_shared_data_ref *sref; +struct btrfs_extent_owner_ref *oref;struct btrfs_disk_key key;unsigned long end;unsigned long ptr;
@@ -159,6 +167,10 @@ static void print_extent_item(const struct extent_buffer *eb, int slot, int type"\t\t\t(parent %llu not aligned to sectorsize %u)\n" , offset, eb->fs_info->sectorsize);break ; +case BTRFS_EXTENT_OWNER_REF_KEY : + oref = (struct btrfs_extent_owner_ref *)(&iref->offset); +print_extent_owner_ref (eb, oref); +break ;default :pr_cont ("(extent %llu has INVALID ref type %d)\n" , eb->start, type);
fs/btrfs/ref-verify.c +3 -0
@@ -485,6 +485,9 @@ static int process_extent_item(struct btrfs_fs_info *fs_info, ret =add_shared_data_ref (fs_info, offset, count, key->objectid, key->offset);break ; +case BTRFS_EXTENT_OWNER_REF_KEY : +WARN_ON (!btrfs_fs_incompat (fs_info,SIMPLE_QUOTA )); +break ;default :btrfs_err (fs_info,"invalid key type in iref" ); ret = -EINVAL ;
fs/btrfs/tree-checker.c +3 -0
@@ -1451,6 +1451,9 @@ static int check_extent_item(struct extent_buffer *leaf, } inline_refs +=btrfs_shared_data_ref_count (leaf, sref);break ; +case BTRFS_EXTENT_OWNER_REF_KEY : +WARN_ON (!btrfs_fs_incompat (fs_info,SIMPLE_QUOTA )); +break ;default :extent_err (leaf, slot,"unknown inline ref type: %u" , inline_type);
include/uapi/linux/btrfs_tree.h +6 -0
@@ -226,6 +226,8 @@#define BTRFS_SHARED_DATA_REF_KEY 184 +#define BTRFS_EXTENT_OWNER_REF_KEY 190 +/* * block groups give us hints into the extent allocation trees. Which * blocks are free etc etc
@@ -783,6 +785,10 @@ struct btrfs_shared_data_ref {__le32 count; } __attribute__ ((__packed__)); +struct btrfs_extent_owner_ref { +u64 root_id; +} __attribute__ ((__packed__ )); +struct btrfs_extent_inline_ref {__u8 type;__le64 offset;
Signature · 1 line
2.41.0
PATCH 13/18 btrfs: inline owner ref lookup helper
In reply to Boris BurkovInline ref parsing is a bit tricky and relies on a decent amount of implicit information, so I think it is beneficial to have a helper function for reading the owner ref, if only to "document" the format, along with the write path.
The main subtlety of note which I was missing by open-coding this was that it is important to check whether or not inline refs are present *at all*. i.e., if we are writing out a new extent under squotas, we will always use a big enough item for the inline ref and have it. However, it is possible that some random item predating squotas will not have any inline refs. In that case, trying to read the "type" field of the first inline ref will just be reading garbage in the form of whatever is in the next item.
This will be used by the extent free-ing path, which looks up data extent owners as well as a relocation path which needs to grab the owner before relocating an extent.
--- fs/btrfs/extent-tree.c | 51 ++++++++++++++++++++++++++++++++++++++++++ fs/btrfs/extent-tree.h | 3 +++ 2 files changed, 54 insertions(+)
fs/btrfs/extent-tree.c +51 -0
@@ -2821,6 +2821,57 @@ int btrfs_finish_extent_commit(struct btrfs_trans_handle *trans)return 0 ; } +/* +* Helper to parse an extent item's inline extents looking for a simple +* quotas owner ref. +* +* @fs_info - the btrfs_fs_info for this mount +* @leaf - a leaf in the extent tree containing the extent item +* @slot - the slot in the leaf where the extent item is found +* +* Returns the objectid of the root that originally allocated the extent item +* if the inline owner ref is expected and present, otherwise 0. +* +* If an extent item has an owner ref item, it will be the first +* inline ref item. Therefore the logic is to check whether there are +* any inline ref items, then check the type of the first one. +* +*/ +u64 btrfs_get_extent_owner_root (struct btrfs_fs_info *fs_info, +struct extent_buffer *leaf, +int slot) +{ +struct btrfs_extent_item *ei; +struct btrfs_extent_inline_ref *iref; +struct btrfs_extent_owner_ref *oref; +unsigned long ptr; +unsigned long end; +int type; + +if (!btrfs_fs_incompat (fs_info,SIMPLE_QUOTA )) +return 0 ; + + ei = btrfs_item_ptr(leaf, slot, struct btrfs_extent_item); + ptr = (unsigned long )(ei +1 ); + end = (unsigned long )ei +btrfs_item_size (leaf, slot); + +/* No inline ref items of any kind, can't check type */ +if (ptr == end) +return 0 ; + + iref = (struct btrfs_extent_inline_ref *)ptr; + type =btrfs_get_extent_inline_ref_type (leaf, iref,BTRFS_REF_TYPE_ANY ); + +/* We found an owner ref, get the root out of it */ +if (type ==BTRFS_EXTENT_OWNER_REF_KEY ) { + oref = (struct btrfs_extent_owner_ref *)(&iref->offset); +return btrfs_extent_owner_ref_root_id (leaf, oref); + } + +/* We have inline refs, but not an owner ref */ +return 0 ; +} +static int do_free_extent_accounting (struct btrfs_trans_handle *trans,u64 bytenr,u64 num_bytes,bool is_data) {
fs/btrfs/extent-tree.h +3 -0
@@ -144,6 +144,9 @@ int btrfs_set_disk_extent_flags(struct btrfs_trans_handle *trans, struct extent_buffer *eb, u64 flags);int btrfs_free_extent (struct btrfs_trans_handle *trans,struct btrfs_ref *ref); +u64 btrfs_get_extent_owner_root (struct btrfs_fs_info *fs_info, +struct extent_buffer *leaf, +int slot);int btrfs_free_reserved_extent (struct btrfs_fs_info *fs_info,u64 start,u64 len,int delalloc);int btrfs_pin_reserved_extent (struct btrfs_trans_handle *trans,u64 start,u64 len);
Signature · 1 line
2.41.0
PATCH 14/18 btrfs: record simple quota deltas
In reply to Boris BurkovAt the moment that we run delayed refs, we make the final ref-count based decision on creating/removing extent (and metadata) items. Therefore, it is exactly the spot to hook up simple quotas.
There are a few important subtleties to the fields we must collect to accurately track simple quotas, particularly when removing an extent. When removing a data extent, the ref could be in any tree (due to reflink, for example) and so we need to recover the owning root id from the owner ref item. When removing a metadata extent, we know the owning root from the owner field in the header when we create the delayed ref, so we can recover it from there.
We must also be careful to handle reservations properly to not leaked reserved space. The happy path is freeing the reservation when the simple quota delta runs on a data extent. If that doesn't happen, due to refs canceling out or some error, the ref head already has the must_insert_reserved machinery to handle this, so we piggy back on that and use it to clean up the reserved data.
--- fs/btrfs/delayed-ref.c | 3 ++ fs/btrfs/delayed-ref.h | 6 ++++ fs/btrfs/extent-tree.c | 79 +++++++++++++++++++++++++++++++++++++----- 3 files changed, 80 insertions(+), 8 deletions(-)
fs/btrfs/delayed-ref.c +3 -0
@@ -735,6 +735,9 @@ static void init_delayed_ref_head(struct btrfs_delayed_ref_head *head_ref, head_ref->bytenr = bytenr; head_ref->num_bytes = num_bytes; head_ref->ref_mod = count_mod; + head_ref->reserved_bytes =0 ; +if (reserved) + head_ref->reserved_bytes = reserved; head_ref->must_insert_reserved = must_insert_reserved; head_ref->owning_root = owning_root; head_ref->is_data = is_data;
fs/btrfs/delayed-ref.h +6 -0
@@ -110,6 +110,12 @@ struct btrfs_delayed_ref_head {*/ int ref_mod; +/* +* Track reserved bytes when setting must_insert_reserved. +* On success or cleanup, we will need to free the reservation. +*/ +u64 reserved_bytes; +/* * when a new extent is allocated, it is just reserved in memory * The actual extent isn't inserted into the extent allocation tree
fs/btrfs/extent-tree.c +71 -8
194 lines not on this page · raw message
Signature · 1 line
2.41.0
PATCH 15/18 btrfs: simple quota auto hierarchy for nested subvols
In reply to Boris BurkovConsider the following sequence:
- enable quotas
- create subvol S id 256 at dir outer/
- create a qgroup 1/100
- add 0/256 (S's auto qgroup) to 1/100
- create subvol T id 257 at dir outer/inner/
With full qgroups, there is no relationship between 0/257 and either of 0/256 or 1/100. There is an inherit feature that the creator of inner/ can use to specify it ought to be in 1/100.
Simple quotas are targeted at container isolation, where such automatic inheritance for not necessarily trusted/controlled nested subvol creation would be quite helpful. Therefore, add a new default behavior for simple quotas: when you create a nested subvol, automatically inherit as parents any parents of the qgroup of the subvol the new inode is going in.
In our example, 257/0 would also be under 1/100, allowing easy control of a total quota over an arbitrary hierarchy of subvolumes.
I think this _might_ be a generally useful behavior, so it could be interesting to put it behind a new inheritance flag that simple quotas always use while traditional quotas let the user specify, but this is a minimally intrusive change to start.
--- fs/btrfs/ioctl.c | 2 +- fs/btrfs/qgroup.c | 46 +++++++++++++++++++++++++++++++++++++++--- fs/btrfs/qgroup.h | 6 +++--- fs/btrfs/transaction.c | 13 ++++++++---- 4 files changed, 56 insertions(+), 11 deletions(-)
fs/btrfs/ioctl.c +1 -1
@@ -652,7 +652,7 @@ static noinline int create_subvol(struct mnt_idmap *idmap,/* Tree log can't currently deal with an inode which is a new root. */ btrfs_set_log_full_commit (trans); - ret =btrfs_qgroup_inherit (trans,0 , objectid, inherit); + ret =btrfs_qgroup_inherit (trans,0 , objectid, root->root_key.objectid, inherit);if (ret) goto out;
fs/btrfs/qgroup.c +43 -3
76 lines not on this page · raw message
fs/btrfs/qgroup.h +3 -3
@@ -271,8 +271,7 @@ int btrfs_qgroup_rescan(struct btrfs_fs_info *fs_info);void btrfs_qgroup_rescan_resume (struct btrfs_fs_info *fs_info);int btrfs_qgroup_wait_for_completion (struct btrfs_fs_info *fs_info,bool interruptible); -int btrfs_add_qgroup_relation (struct btrfs_trans_handle *trans,u64 src, -u64 dst); +int btrfs_add_qgroup_relation (struct btrfs_trans_handle *trans,u64 src,u64 dst);int btrfs_del_qgroup_relation (struct btrfs_trans_handle *trans,u64 src,u64 dst);int btrfs_create_qgroup (struct btrfs_trans_handle *trans,u64 qgroupid);
@@ -366,7 +365,8 @@ int btrfs_qgroup_account_extent(struct btrfs_trans_handle *trans, u64 bytenr,int btrfs_qgroup_account_extents (struct btrfs_trans_handle *trans);int btrfs_run_qgroups (struct btrfs_trans_handle *trans);int btrfs_qgroup_inherit (struct btrfs_trans_handle *trans,u64 srcid, -u64 objectid,struct btrfs_qgroup_inherit *inherit); +u64 objectid,u64 inode_rootid, +struct btrfs_qgroup_inherit *inherit);void btrfs_qgroup_free_refroot (struct btrfs_fs_info *fs_info,u64 ref_root,u64 num_bytes,enum btrfs_qgroup_rsv_type type);
fs/btrfs/transaction.c +9 -4
@@ -1523,13 +1523,14 @@ static int qgroup_account_snapshot(struct btrfs_trans_handle *trans,int ret;/* -* Save some performance in the case that full qgroups are not +* Save some performance in the case that qgroups are not * enabled. If this check races with the ioctl, rescan will * kick in anyway. */ if (btrfs_qgroup_mode (fs_info) !=BTRFS_QGROUP_MODE_FULL )return 0 ; +/* * Ensure dirty @src will be committed. Or, after coming * commit_fs_roots() and switch_commit_roots(), any dirty but not
@@ -1566,7 +1567,7 @@ static int qgroup_account_snapshot(struct btrfs_trans_handle *trans,/* Now qgroup are all updated, we can inherit it to new qgroups */ ret =btrfs_qgroup_inherit (trans, src->root_key.objectid, dst_objectid, - inherit); + parent->root_key.objectid, inherit);if (ret <0 ) goto out;
@@ -1832,8 +1833,12 @@ static noinline int create_pending_snapshot(struct btrfs_trans_handle *trans,* To co-operate with that hack, we do hack again. * Or snapshot will be greatly slowed down by a subtree qgroup rescan */ - ret =qgroup_account_snapshot (trans, root, parent_root, - pending->inherit, objectid); +if (btrfs_qgroup_mode (fs_info) ==BTRFS_QGROUP_MODE_FULL ) + ret =qgroup_account_snapshot (trans, root, parent_root, + pending->inherit, objectid); +else if (btrfs_qgroup_mode (fs_info) ==BTRFS_QGROUP_MODE_SIMPLE ) + ret =btrfs_qgroup_inherit (trans, root->root_key.objectid, objectid, + parent_root->root_key.objectid, pending->inherit);if (ret <0 ) goto fail;
Signature · 1 line
2.41.0
PATCH 16/18 btrfs: check generation when recording simple quota delta
In reply to Boris BurkovSimple quotas count extents only from the moment the feature is enabled. Therefore, if we do something like:
1. create subvol S
2. write F in S
3. enable quotas
4. remove F
5. write G in S
then after 3. and 4. we would expect the simple quota usage of S to be 0 (putting aside some metadata extents that might be written) and after 5., it should be the size of G plus metadata. Therefore, we need to be able to determine whether a particular quota delta we are processing predates simple quota enablement.
To do this, store the transaction id when quotas were enabled. In fs_info for immediate use and in the quota status item to make it recoverable on mount. When we see a delta, check if the generation of the extent item is less than that of quota enablement. If so, we should ignore the delta from this extent.
--- fs/btrfs/accessors.h | 2 ++ fs/btrfs/extent-tree.c | 4 ++++ fs/btrfs/fs.h | 2 ++ fs/btrfs/qgroup.c | 17 +++++++++++++---- fs/btrfs/qgroup.h | 1 + include/uapi/linux/btrfs_tree.h | 7 +++++++ 6 files changed, 29 insertions(+), 4 deletions(-)
fs/btrfs/accessors.h +2 -0
@@ -971,6 +971,8 @@ BTRFS_SETGET_FUNCS(qgroup_status_flags, struct btrfs_qgroup_status_item, flags, 64); BTRFS_SETGET_FUNCS(qgroup_status_rescan, struct btrfs_qgroup_status_item, rescan,64 ); +BTRFS_SETGET_FUNCS(qgroup_status_enable_gen, struct btrfs_qgroup_status_item, + enable_gen,64 );/* btrfs_qgroup_info_item */ BTRFS_SETGET_FUNCS(qgroup_info_generation, struct btrfs_qgroup_info_item,
fs/btrfs/extent-tree.c +4 -0
@@ -1533,6 +1533,7 @@ static int run_delayed_data_ref(struct btrfs_trans_handle *trans, .rsv_bytes = href->reserved_bytes, .is_data = true, .is_inc = true, + .generation = trans->transid, };if (extent_op)
@@ -1696,6 +1697,7 @@ static int run_delayed_tree_ref(struct btrfs_trans_handle *trans, .rsv_bytes = 0, .is_data = false, .is_inc = true, + .generation = trans->transid, };BUG_ON (!extent_op || !extent_op->update_flags);
@@ -3231,6 +3233,7 @@ static int __btrfs_free_extent(struct btrfs_trans_handle *trans, .rsv_bytes = 0, .is_data = is_data, .is_inc = false, + .generation = btrfs_extent_generation(leaf, ei), };/* In this branch refs == 1 */
@@ -4866,6 +4869,7 @@ int btrfs_alloc_logged_file_extent(struct btrfs_trans_handle *trans,struct btrfs_simple_quota_delta delta = { .root = root_objectid, .num_bytes = ins->offset, + .generation = trans->transid, .rsv_bytes =0 , .is_data = true, .is_inc = true,
fs/btrfs/fs.h +2 -0
@@ -802,6 +802,8 @@ struct btrfs_fs_info {spinlock_t eb_leak_lock;struct list_head allocated_ebs;#endif + +u64 quota_enable_gen; }; static inline void btrfs_set_last_root_drop_gen(struct btrfs_fs_info *fs_info,
fs/btrfs/qgroup.c +13 -4
@@ -468,8 +468,9 @@ int btrfs_read_qgroup_config(struct btrfs_fs_info *fs_info)btrfs_err (fs_info,"qgroup generation mismatch, marked as inconsistent" ); } - fs_info->qgroup_flags =btrfs_qgroup_status_flags (l, - ptr); +if (btrfs_fs_incompat (fs_info,SIMPLE_QUOTA )) + fs_info->quota_enable_gen =btrfs_qgroup_status_enable_gen (l, ptr); + fs_info->qgroup_flags =btrfs_qgroup_status_flags (l, ptr); rescan_progress =btrfs_qgroup_status_rescan (l, ptr); goto next1; }
@@ -1115,6 +1116,10 @@ int btrfs_quota_enable(struct btrfs_fs_info *fs_info, ptr =btrfs_item_ptr (leaf, path->slots[0 ], struct btrfs_qgroup_status_item);btrfs_set_qgroup_status_generation (leaf, ptr, trans->transid); +if (simple) { +btrfs_set_fs_incompat (fs_info,SIMPLE_QUOTA ); +btrfs_set_qgroup_status_enable_gen (leaf, ptr, trans->transid); + }btrfs_set_qgroup_status_version (leaf, ptr,BTRFS_QGROUP_STATUS_VERSION ); fs_info->qgroup_flags =BTRFS_QGROUP_STATUS_FLAG_ON ;if (!simple)
@@ -1210,6 +1215,8 @@ int btrfs_quota_enable(struct btrfs_fs_info *fs_info, goto out_free_path; } + fs_info->quota_enable_gen = trans->transid; +mutex_unlock (&fs_info->qgroup_ioctl_lock);/* * Commit the transaction while not holding qgroup_ioctl_lock, to avoid
@@ -1234,8 +1241,6 @@ int btrfs_quota_enable(struct btrfs_fs_info *fs_info,spin_lock (&fs_info->qgroup_lock); fs_info->quota_root = quota_root;set_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags); -if (simple) -btrfs_set_fs_incompat (fs_info,SIMPLE_QUOTA );spin_unlock (&fs_info->qgroup_lock);/* Skip rescan for simple qgroups */
@@ -4629,6 +4634,10 @@ int btrfs_record_simple_quota_delta(struct btrfs_fs_info *fs_info,if (!is_fstree (root))return 0 ; +/* If the extent predates enabling quotas, don't count it. */ +if (delta->generation < fs_info->quota_enable_gen) +return 0 ; +spin_lock (&fs_info->qgroup_lock); qgroup =find_qgroup_rb (fs_info, root);if (!qgroup) {
fs/btrfs/qgroup.h +1 -0
@@ -241,6 +241,7 @@ struct btrfs_simple_quota_delta {u64 rsv_bytes;/* The number of bytes reserved for this extent */ bool is_inc;/* Whether we are using or freeing the extent */ bool is_data;/* Whether the extent is data or metadata */ +u64 generation;/* The generation the extent was created in */ };static inline u64 btrfs_qgroup_subvolid (u64 qgroupid)
include/uapi/linux/btrfs_tree.h +7 -0
@@ -1230,6 +1230,13 @@ struct btrfs_qgroup_status_item {* of the scan. It contains a logical address */ __le64 rescan; + +/* +* the generation when quotas are enabled. Used by simple quotas to +* avoid decrementing when freeing an extent that was written before +* enable. +*/ +__le64 enable_gen; } __attribute__ ((__packed__));struct btrfs_qgroup_info_item {
Signature · 1 line
2.41.0
PATCH 17/18 btrfs: track metadata relocation cow with simple quota
In reply to Boris BurkovRelocation cows metadata blocks in two cases for the reloc root:
- copying the subvol root item when creating the reloc root
- copying a btree node when there is a cow during relocation
In both cases, the resulting btree node hits an abnormal code path with respect to the owner field in its btrfs_header. It first creates the root item for the new objectid, which populates the reloc root id, and it at this point that delayed refs are created.
Later, it fully copies the old node into the new node (including the original owner field) which overwrites it. This results in a simple quotas mismatch where we run the delayed ref for the reloc root which has no simple quota effect (reloc root is not an fstree) but when we ultimately delete the node, the owner is the real original fstree and we do free the space.
To work around this without tampering with the behavior of relocation, add a parameter to btrfs_add_tree_block that lets the relocation code path specify a different owning root than the "operating" root (in this case, owning root is the real root and the operating root is the reloc root). These can naturally be plumbed into delayed refs that have the same concept.
Note that this is a double count in some sense, but a relatively natural one, as there are really two extents, and the old one will be deleted soon. This is consistent with how data relocation extents are accounted by simple quotas.
--- fs/btrfs/ctree.c | 22 ++++++++++++++-------- fs/btrfs/disk-io.c | 4 ++-- fs/btrfs/extent-tree.c | 8 ++++++-- fs/btrfs/extent-tree.h | 3 ++- fs/btrfs/ioctl.c | 2 +- 5 files changed, 25 insertions(+), 14 deletions(-)
fs/btrfs/ctree.c +14 -8
@@ -316,6 +316,7 @@ int btrfs_copy_root(struct btrfs_trans_handle *trans,int ret =0 ;int level;struct btrfs_disk_key disk_key; +u64 reloc_src_root =0 ;WARN_ON (test_bit (BTRFS_ROOT_SHAREABLE , &root->state) && trans->transid != fs_info->running_transaction->transid);
@@ -328,9 +329,11 @@ int btrfs_copy_root(struct btrfs_trans_handle *trans,else btrfs_node_key (buf, &disk_key,0 ); +if (new_root_objectid ==BTRFS_TREE_RELOC_OBJECTID ) + reloc_src_root =btrfs_header_owner (buf); cow =btrfs_alloc_tree_block (trans, root,0 , new_root_objectid, &disk_key, level, buf->start,0 , -BTRFS_NESTING_NEW_ROOT ); +BTRFS_NESTING_NEW_ROOT , reloc_src_root);if (IS_ERR (cow))return PTR_ERR (cow);
@@ -522,6 +525,7 @@ static noinline int __btrfs_cow_block(struct btrfs_trans_handle *trans,int last_ref =0 ;int unlock_orig =0 ;u64 parent_start =0 ; +u64 reloc_src_root =0 ;if (*cow_ret == buf) unlock_orig =1 ;
@@ -540,12 +544,14 @@ static noinline int __btrfs_cow_block(struct btrfs_trans_handle *trans,else btrfs_node_key (buf, &disk_key,0 ); -if ((root->root_key.objectid ==BTRFS_TREE_RELOC_OBJECTID ) && parent) - parent_start = parent->start; - +if (root->root_key.objectid ==BTRFS_TREE_RELOC_OBJECTID ) { +if (parent) + parent_start = parent->start; + reloc_src_root =btrfs_header_owner (buf); + } cow =btrfs_alloc_tree_block (trans, root, parent_start, root->root_key.objectid, &disk_key, level, - search_start, empty_size, nest); + search_start, empty_size, nest, reloc_src_root);if (IS_ERR (cow))return PTR_ERR (cow);
@@ -2956,7 +2962,7 @@ static noinline int insert_new_root(struct btrfs_trans_handle *trans, c =btrfs_alloc_tree_block (trans, root,0 , root->root_key.objectid, &lower_key, level, root->node->start,0 , -BTRFS_NESTING_NEW_ROOT ); +BTRFS_NESTING_NEW_ROOT ,0 );if (IS_ERR (c))return PTR_ERR (c);
@@ -3100,7 +3106,7 @@ static noinline int split_node(struct btrfs_trans_handle *trans, split =btrfs_alloc_tree_block (trans, root,0 , root->root_key.objectid, &disk_key, level, c->start,0 , -BTRFS_NESTING_SPLIT ); +BTRFS_NESTING_SPLIT ,0 );if (IS_ERR (split))return PTR_ERR (split);
@@ -3853,7 +3859,7 @@ static noinline int split_leaf(struct btrfs_trans_handle *trans, right =btrfs_alloc_tree_block (trans, root,0 , root->root_key.objectid, &disk_key,0 , l->start,0 , num_doubles ?BTRFS_NESTING_NEW_ROOT : -BTRFS_NESTING_SPLIT ); +BTRFS_NESTING_SPLIT ,0 );if (IS_ERR (right))return PTR_ERR (right);
fs/btrfs/disk-io.c +2 -2
@@ -862,7 +862,7 @@ struct btrfs_root *btrfs_create_tree(struct btrfs_trans_handle *trans, root->root_key.offset =0 ; leaf =btrfs_alloc_tree_block (trans, root,0 , objectid,NULL ,0 ,0 ,0 , -BTRFS_NESTING_NORMAL ); +BTRFS_NESTING_NORMAL ,0 );if (IS_ERR (leaf)) { ret =PTR_ERR (leaf); leaf =NULL ;
@@ -939,7 +939,7 @@ int btrfs_alloc_log_tree_node(struct btrfs_trans_handle *trans,*/ leaf =btrfs_alloc_tree_block (trans, root,0 ,BTRFS_TREE_LOG_OBJECTID , -NULL ,0 ,0 ,0 ,BTRFS_NESTING_NORMAL ); +NULL ,0 ,0 ,0 ,BTRFS_NESTING_NORMAL ,0 );if (IS_ERR (leaf))return PTR_ERR (leaf);
fs/btrfs/extent-tree.c +6 -2
@@ -5005,7 +5005,8 @@ struct extent_buffer *btrfs_alloc_tree_block(struct btrfs_trans_handle *trans,const struct btrfs_disk_key *key, int level, u64 hint, u64 empty_size, - enum btrfs_lock_nesting nest) + enum btrfs_lock_nesting nest, + u64 reloc_src_root) { struct btrfs_fs_info *fs_info = root->fs_info;struct btrfs_key ins;
@@ -5017,6 +5018,7 @@ struct extent_buffer *btrfs_alloc_tree_block(struct btrfs_trans_handle *trans,int ret;u32 blocksize = fs_info->nodesize;bool skinny_metadata =btrfs_fs_incompat (fs_info,SKINNY_METADATA ); +u64 owning_root;#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS if (btrfs_is_testing (fs_info)) {
@@ -5043,11 +5045,13 @@ struct extent_buffer *btrfs_alloc_tree_block(struct btrfs_trans_handle *trans, ret =PTR_ERR (buf); goto out_free_reserved; } + owning_root =btrfs_header_owner (buf);if (root_objectid ==BTRFS_TREE_RELOC_OBJECTID ) {if (parent ==0 ) parent = ins.objectid; flags |=BTRFS_BLOCK_FLAG_FULL_BACKREF ; + owning_root = reloc_src_root; }else BUG_ON (parent >0 );
@@ -5067,7 +5071,7 @@ struct extent_buffer *btrfs_alloc_tree_block(struct btrfs_trans_handle *trans, extent_op->level = level;btrfs_init_generic_ref (&generic_ref,BTRFS_ADD_DELAYED_EXTENT , - ins.objectid, ins.offset, parent,btrfs_header_owner (buf)); + ins.objectid, ins.offset, parent, owning_root);btrfs_init_tree_ref (&generic_ref, level, root_objectid, root->root_key.objectid, false);btrfs_ref_tree_mod (fs_info, &generic_ref);
fs/btrfs/extent-tree.h +2 -1
@@ -121,7 +121,8 @@ struct extent_buffer *btrfs_alloc_tree_block(struct btrfs_trans_handle *trans,const struct btrfs_disk_key *key, int level, u64 hint, u64 empty_size, - enum btrfs_lock_nesting nest); + enum btrfs_lock_nesting nest, + u64 reloc_src_root); void btrfs_free_tree_block(struct btrfs_trans_handle *trans, u64 root_id, struct extent_buffer *buf,
fs/btrfs/ioctl.c +1 -1
@@ -657,7 +657,7 @@ static noinline int create_subvol(struct mnt_idmap *idmap, goto out; leaf =btrfs_alloc_tree_block (trans, root,0 , objectid,NULL ,0 ,0 ,0 , -BTRFS_NESTING_NORMAL ); +BTRFS_NESTING_NORMAL ,0 );if (IS_ERR (leaf)) { ret =PTR_ERR (leaf); goto out;
Signature · 1 line
2.41.0
PATCH 18/18 btrfs: track data relocation with simple quota
In reply to Boris BurkovRelocation data allocations are quite tricky for simple quotas. The basic data relocation sequence is (ignoring details that aren't relevant to this fix):
- create a fake relocation data fs root
- create a fake relocation inode in that root
- foreach data extent:
- preallocate a data extent on behalf of the fake inode
- copy over the data
- foreach extent
- swap the refs so that the original file extent now refers to the new extent item
- drop the fake root, dropping its refs on the old extents, which lets us delete them.
Done naively, this results in storing an extent item in the extent tree whose owner_ref points at the relocation data root and a no-op squota recording, since the reloc root is not a legit fstree. So far, that's OK. The problem comes when you do the swap, and leave an extent item owned by this bogus root as the real permanent extents of the file. If the file then drops that ref, we free it and no-op account that against the fake relocation root. Essentially, this means that relocation is simple quota "extent laundering", since we re-own the extents into a fake root.
Simple quotas very intentionally doesn't have a mechanism for transferring ownership of extents, as that is exactly the complicated thing we are trying to avoid with the new design. Further, it cannot be correctly done in this case, since at the time you create the new "real" refs, there is no way to know which was the original owner before relocation unless we track it.
Therefore, it makes more sense to trick the preallocation to handle relocation as a special case and note the proper owner ref from the beginning. That way, we never write out an extent item without the correct owner ref that it will eventually have.
This could be done by wiring a special root parameter all the way through the allocation code path, but to avoid that special case touching all the code, take advantage of the serial nature of relocation to store the src root on the relocation root object. Then when we finish the prealloc, if it happens to be this case, prepare the delayed ref appropriately.
This is obviously a smelly bit of code, but I think it is the best solution to the problem, given the relocation implementation.
--- fs/btrfs/ctree.h | 1 + fs/btrfs/extent-tree.c | 13 +++++++------ fs/btrfs/relocation.c | 15 +++++++++++++++ 3 files changed, 23 insertions(+), 6 deletions(-)
fs/btrfs/ctree.h +1 -0
@@ -333,6 +333,7 @@ struct btrfs_root {#ifdef CONFIG_BTRFS_DEBUG struct list_head leak_list;#endif +u64 relocation_src_root; };static inline bool btrfs_root_readonly (const struct btrfs_root *root)
fs/btrfs/extent-tree.c +7 -6
@@ -57,7 +57,7 @@ static void __run_delayed_extent_op(struct btrfs_delayed_extent_op *extent_op,static int alloc_reserved_file_extent (struct btrfs_trans_handle *trans,u64 parent,u64 root_objectid,u64 flags,u64 owner,u64 offset, -struct btrfs_key *ins,int ref_mod); +struct btrfs_key *ins,int ref_mod,u64 oref_root);static int alloc_reserved_tree_block (struct btrfs_trans_handle *trans,struct btrfs_delayed_ref_node *node,struct btrfs_delayed_extent_op *extent_op);
@@ -1541,7 +1541,7 @@ static int run_delayed_data_ref(struct btrfs_trans_handle *trans, ret =alloc_reserved_file_extent (trans, parent, ref_root, flags, ref->objectid, ref->offset, &ins, - node->ref_mod); + node->ref_mod, href->owning_root);if (!ret) ret =btrfs_record_simple_quota_delta (trans->fs_info, &delta); }else if (node->action ==BTRFS_ADD_DELAYED_REF ) {
@@ -4683,7 +4683,7 @@ static int alloc_reserved_extent(struct btrfs_trans_handle *trans, u64 bytenr,static int alloc_reserved_file_extent (struct btrfs_trans_handle *trans,u64 parent,u64 root_objectid,u64 flags,u64 owner,u64 offset, -struct btrfs_key *ins,int ref_mod) +struct btrfs_key *ins,int ref_mod,u64 oref_root) {struct btrfs_fs_info *fs_info = trans->fs_info;struct btrfs_root *extent_root;
@@ -4731,7 +4731,7 @@ static int alloc_reserved_file_extent(struct btrfs_trans_handle *trans,if (simple_quota) {btrfs_set_extent_inline_ref_type (leaf, iref,BTRFS_EXTENT_OWNER_REF_KEY ); oref = (struct btrfs_extent_owner_ref *)(&iref->offset); -btrfs_set_extent_owner_ref_root_id (leaf, oref, root_objectid); +btrfs_set_extent_owner_ref_root_id (leaf, oref, oref_root); iref = (struct btrfs_extent_inline_ref *)(oref +1 ); }btrfs_set_extent_inline_ref_type (leaf, iref, type);
@@ -4842,7 +4842,8 @@ int btrfs_alloc_reserved_file_extent(struct btrfs_trans_handle *trans,BUG_ON (root_objectid ==BTRFS_TREE_LOG_OBJECTID ); -BUG_ON (root->root_key.objectid ==BTRFS_TREE_LOG_OBJECTID ); +if (btrfs_is_data_reloc_root (root) &&is_fstree (root->relocation_src_root)) + owning_root = root->relocation_src_root;btrfs_init_generic_ref (&generic_ref,BTRFS_ADD_DELAYED_EXTENT , ins->objectid, ins->offset,0 , owning_root);
@@ -4899,7 +4900,7 @@ int btrfs_alloc_logged_file_extent(struct btrfs_trans_handle *trans,spin_unlock (&space_info->lock); ret =alloc_reserved_file_extent (trans,0 , root_objectid,0 , owner, - offset, ins,1 ); + offset, ins,1 , root_objectid);if (ret)btrfs_pin_extent (trans, ins->objectid, ins->offset,1 ); ret =btrfs_record_simple_quota_delta (fs_info, &delta);
fs/btrfs/relocation.c +15 -0
@@ -3665,6 +3665,21 @@ static noinline_for_stack int relocate_block_group(struct reloc_control *rc) struct btrfs_extent_item); flags =btrfs_extent_flags (path->nodes[0 ], ei); +/* +* If we are relocating a simple quota owned extent item, we need +* to note the owner on the reloc data root so that when we +* allocate the replacement item, we can attribute it to the +* correct eventual owner (rather than the reloc data root) +*/ +if (btrfs_qgroup_mode (fs_info) ==BTRFS_QGROUP_MODE_SIMPLE ) { +struct btrfs_root *root =BTRFS_I (rc->data_inode)->root; +u64 owning_root_id =btrfs_get_extent_owner_root (fs_info, + path->nodes[0 ], + path->slots[0 ]); + + root->relocation_src_root = owning_root_id; + } +if (flags &BTRFS_EXTENT_FLAG_TREE_BLOCK ) { ret =add_tree_block (rc, &key, path, &blocks); } else if (rc->stage == UPDATE_DATA_PTRS &&
Signature · 1 line
2.41.0
Re: [PATCH 01/18] btrfs: free qgroup rsv on io failure
In reply to Boris BurkovQuoted 9 lines · Boris Burkov
If we do a write whose bio suffers an error, we will never reclaim the qgroup reserved space for it. We allocate the space in the
write_itercodepath, then release the reservation as we allocate the ordered extent, but we only create a delayed ref if the ordered extent finishes. If it has an error, we simply leak the rsv. This is apparent in running any error injecting (dmerror) fstests like btrfs/146 or btrfs/160. Such tests fail due to dmesg on umount complaining about the leaked qgroup data space.
Quoted 4 lines
When we clean up other aspects of space on failed
ordered_extents, also free the qgroup rsv.Signed-off-by Boris Burkov <[email protected]>
Thanks,
Josef
Re: [PATCH 02/18] btrfs: fix start transaction qgroup rsv double free
In reply to Boris BurkovQuoted 34 lines · Boris Burkov
btrfs_start_transactionreserves metadata space of the PERTRANS type before it identifies a transaction to start/join. This allows flushing when reserving that space without a deadlock. However, it results in a race which temporarily breaks qgroup rsv accounting.T1 T2
start_transactiondo_stuff
start_transaction
qgroup_reserve_meta_pertranscommit_transaction
qgroup_free_meta_all_pertrans
hit an error starting txn gotoreserve_failqgroup_free_meta_pertrans(already freed!)The basic issue is that there is nothing preventing another commit from committing before
start_transactionfinishes (in fact sometimes we intentionally wait for it..) so any error path that frees the reserve is at risk of this race.While this exact space was getting freed anyway, and it's not a huge deal to double free it (just a warning, the free code catches this), it can result in incorrectly freeing some other pertrans reservation in this same reservation, which could then lead to spuriously granting reservations we might not have the space for. Therefore, I do believe it is worth fixing.
To fix it, use the existing
prealloc->pertransconversion mechanism. When we first reserve the space, we reserve prealloc space and only when we are sure we have a transaction do we convert it to pertrans. This way any racing commits do not blow away our reservation, but we still get a pertrans reservation that is freed when _this_ transaction gets committed.
Quoted 4 lines
This issue can be reproduced by running generic/269 with either qgroups or squotas enabled via mkfs on the scratch device.
Signed-off-by Boris Burkov <[email protected]>
Thanks,
Josef
Re: [PATCH 03/18] btrfs: introduce quota mode
In reply to Boris BurkovQuoted 5 lines · Boris Burkov
In preparation for introducing simple quotas, change from a binary setting for quotas to an enum based mode. Initially, the possible modes are disabled/full. Full quotas is normal btrfs qgroups.
Signed-off-by Boris Burkov <[email protected]>
Thanks,
Josef
Re: [PATCH 04/18] btrfs: add new quota mode for simple quotas
In reply to Boris BurkovQuoted 96 lines · Boris Burkov
Add a new quota mode called "simple quotas". It can be enabled by the existing quota enable ioctl via an unused field, and sets an incompat bit, as the implementation of simple quotas will make backwards incompatible changes to the disk format of the extent tree.
Signed-off-by Boris Burkov <[email protected]>--- fs/btrfs/delayed-ref.c | 4 +- fs/btrfs/fs.h | 5 +-- fs/btrfs/ioctl.c | 2 +- fs/btrfs/qgroup.c | 88 ++++++++++++++++++++++++++------------ fs/btrfs/qgroup.h | 4 +- fs/btrfs/root-tree.c | 2 +- fs/btrfs/transaction.c | 4 +- include/uapi/linux/btrfs.h | 2 + 8 files changed, 74 insertions(+), 37 deletions(-)fs/btrfs/delayed-ref.c +2
-2@@ -898,7 +898,7 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans,return -ENOMEM ; } -if (test_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags) && +if (btrfs_qgroup_mode (fs_info) !=BTRFS_QGROUP_MODE_DISABLED && !generic_ref->skip_qgroup) { record =kzalloc (sizeof (*record),GFP_NOFS );if (!record) {@@ -1002,7 +1002,7 @@ int btrfs_add_delayed_data_ref(struct btrfs_trans_handle *trans,return -ENOMEM ; } -if (test_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags) && +if (btrfs_qgroup_mode (fs_info) !=BTRFS_QGROUP_MODE_DISABLED && !generic_ref->skip_qgroup) { record =kzalloc (sizeof (*record),GFP_NOFS );if (!record) {fs/btrfs/fs.h +2
-3@@ -218,7 +218,8 @@ enum {BTRFS_FEATURE_INCOMPAT_NO_HOLES | \BTRFS_FEATURE_INCOMPAT_METADATA_UUID | \BTRFS_FEATURE_INCOMPAT_RAID1C34 | \ - BTRFS_FEATURE_INCOMPAT_ZONED) +BTRFS_FEATURE_INCOMPAT_ZONED | \ + BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA) #ifdef CONFIG_BTRFS_DEBUG/* @@ -233,7 +234,6 @@ enum {#define BTRFS_FEATURE_INCOMPAT_SUPP \ (BTRFS_FEATURE_INCOMPAT_SUPP_STABLE) -#endif #define BTRFS_FEATURE_INCOMPAT_SAFE_SET \@@ -790,7 +790,6 @@ struct btrfs_fs_info {struct lockdep_map btrfs_state_change_map[4 ];struct lockdep_map btrfs_trans_pending_ordered_map;struct lockdep_map btrfs_ordered_extent_map; -#ifdef CONFIG_BTRFS_FS_REF_VERIFY spinlock_t ref_verify_lock;struct rb_root block_tree;fs/btrfs/ioctl.c +1
-1@@ -3691,7 +3691,7 @@ static long btrfs_ioctl_quota_ctl(struct file *file, void __user *arg)switch (sa->cmd) {case BTRFS_QUOTA_CTL_ENABLE : - ret =btrfs_quota_enable (fs_info); + ret =btrfs_quota_enable (fs_info, sa);break ;case BTRFS_QUOTA_CTL_DISABLE : ret =btrfs_quota_disable (fs_info);fs/btrfs/qgroup.c +2
-0@@ -3,6 +3,8 @@* Copyright (C) 2011 STRATO. All rights reserved. */ +#include "linux/btrfs.h" +#include <linux/btrfs.h> #include <linux/sched.h> #include <linux/pagemap.h> #include <linux/writeback.h>
fs/btrfs/qgroup.c line 16
Quoted 6 lines
fs/btrfs/qgroup.c +0
-1@@ -10,7 +12,6 @@#include <linux/rbtree.h> #include <linux/slab.h> #include <linux/workqueue.h> -#include <linux/btrfs.h> #include <linux/sched/mm.h>
Something wonky happened here.
<snip>
Quoted 58 lines
fs/btrfs/qgroup.c +10
-4#include "ctree.h" @@ -34,6 +35,8 @@ enum btrfs_qgroup_mode btrfs_qgroup_mode(struct btrfs_fs_info *fs_info) {if (!test_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags))return BTRFS_QGROUP_MODE_DISABLED ; +if (btrfs_fs_incompat (fs_info,SIMPLE_QUOTA )) +return BTRFS_QGROUP_MODE_SIMPLE ;return BTRFS_QGROUP_MODE_FULL ; }@@ -347,6 +350,8 @@ int btrfs_verify_qgroup_counts(struct btrfs_fs_info *fs_info, u64 qgroupid,static void qgroup_mark_inconsistent (struct btrfs_fs_info *fs_info) { +if (btrfs_qgroup_mode (fs_info) ==BTRFS_QGROUP_MODE_SIMPLE ) +return ; fs_info->qgroup_flags |= (BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT |BTRFS_QGROUP_RUNTIME_FLAG_CANCEL_RESCAN |BTRFS_QGROUP_RUNTIME_FLAG_NO_ACCOUNTING );@@ -368,7 +373,7 @@ int btrfs_read_qgroup_config(struct btrfs_fs_info *fs_info)u64 flags =0 ;u64 rescan_progress =0 ; -if (!test_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags)) +if (btrfs_qgroup_mode (fs_info) ==BTRFS_QGROUP_MODE_DISABLED )return 0 ; fs_info->qgroup_ulist =ulist_alloc (GFP_KERNEL );@@ -419,7 +424,8 @@ int btrfs_read_qgroup_config(struct btrfs_fs_info *fs_info) goto out; }if (btrfs_qgroup_status_generation (l, ptr) != - fs_info->generation) { + fs_info->generation && +btrfs_qgroup_mode (fs_info) !=BTRFS_QGROUP_MODE_SIMPLE ) {qgroup_mark_inconsistent (fs_info);btrfs_err (fs_info,"qgroup generation mismatch, marked as inconsistent" );@@ -557,7 +563,7 @@ bool btrfs_check_quota_leak(struct btrfs_fs_info *fs_info)struct rb_node *node;bool ret = false; -if (!test_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags)) +if (btrfs_qgroup_mode (fs_info) ==BTRFS_QGROUP_MODE_DISABLED )return ret;/* * Since we're unmounting, there is no race and no need to grab qgroup @@ -956,7 +962,8 @@ static int btrfs_clean_quota_tree(struct btrfs_trans_handle *trans,return ret; } -int btrfs_quota_enable (struct btrfs_fs_info *fs_info) +int btrfs_quota_enable (struct btrfs_fs_info *fs_info, +struct btrfs_ioctl_quota_ctl_args *quota_ctl_args) {struct btrfs_root *quota_root;struct btrfs_root *tree_root = fs_info->tree_root;
fs/btrfs/qgroup.c int btrfs_quota_enable(struct btrfs_fs_info *fs_info) · line 978
Quoted 4 lines
fs/btrfs/qgroup.c +1
-0@@ -968,6 +975,7 @@ int btrfs_quota_enable(struct btrfs_fs_info *fs_info)struct btrfs_qgroup *qgroup =NULL ;struct btrfs_trans_handle *trans =NULL ;struct ulist *ulist =NULL ; +bool simple = quota_ctl_args->status ==BTRFS_QUOTA_CTL_ENABLE_SIMPLE_QUOTA ;
Why are you using status here? Just use it as a different command.
<snip>
fs/btrfs/qgroup.c int btrfs_qgroup_inherit(struct btrfs_trans_handle *trans, u64 srcid, · line 3017
Quoted 10 lines
fs/btrfs/qgroup.c +1
-2@@ -3010,11 +3033,10 @@ int btrfs_qgroup_inherit(struct btrfs_trans_handle *trans, u64 srcid,qgroup_dirty (fs_info, dstgroup); } -if (srcid) { +if (srcid &&btrfs_qgroup_mode (fs_info) ==BTRFS_QGROUP_MODE_FULL ) { srcgroup =find_qgroup_rb (fs_info, srcid);if (!srcgroup) goto unlock; -
Errant change.
<snip>
Quoted 13 lines
include/uapi/linux/btrfs.h +1
-0@@ -333,6 +333,7 @@ struct btrfs_ioctl_fs_info_args {#define BTRFS_FEATURE_INCOMPAT_RAID1C34 (1ULL << 11)#define BTRFS_FEATURE_INCOMPAT_ZONED (1ULL << 12)#define BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2 (1ULL << 13) +#define BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA (1ULL << 14)struct btrfs_ioctl_feature_flags {__u64 compat_flags;
include/uapi/linux/btrfs.h struct btrfs_ioctl_get_dev_stats { · line 760
Quoted 7 lines
include/uapi/linux/btrfs.h +1
-0@@ -753,6 +754,7 @@ struct btrfs_ioctl_get_dev_stats {#define BTRFS_QUOTA_CTL_ENABLE 1#define BTRFS_QUOTA_CTL_DISABLE 2#define BTRFS_QUOTA_CTL_RESCAN__NOTUSED 3 +#define BTRFS_QUOTA_CTL_ENABLE_SIMPLE_QUOTA (1UL)struct btrfs_ioctl_quota_ctl_args {__u64 cmd;__u64 status;
Yeah this just has cmd and status, and cmd can be whatever we want, so just make ENABLE_SIMPLE_QUOTA 4 and use it in cmd. Thanks,
Josef
Re: [PATCH 05/18] btrfs: expose quota mode via sysfs
In reply to Boris BurkovQuoted 6 lines · Boris Burkov
Add a new sysfs file
/sys/fs/btrfs/<uuid>/qgroups/mode
which prints out the mode qgroups is running in. The possible modes are disabled, qgroup, and squotaSigned-off-by Boris Burkov <[email protected]>
but you should also include another patch for BTRFS_FEAT_ATTR_INCOMPAT() so it shows up in /sys/fs/btrfs/features as well. Thanks,
Josef
Re: [PATCH 06/18] btrfs: flush reservations during quota disable
In reply to Boris BurkovQuoted 28 lines · Boris Burkov
The following sequence:
enable simple quotas
do some writes
reserve space
createordered_extent
release rsv (storersv_bytesin OE, markQGROUP_RESERVEDbits)
disable quotas
enable simple quotas
set qgroup rsv to 0 on all subvolsordered_extentfinishes
create delayed ref withrsv_bytesfrom before
run delayed refrecord_simple_quota_delta free rsv_bytes (0 -> -rsv_delta)results in us reliably underflowing the subvolume's qgroup rsv counter, because disabling/re-enabling quotas toggles reservation counters down to 0, but does not remove other file system state which represents successful acquisition of qgroup rsv space. Specifically metadata rsv counters on the root object and
rsv_bytesonordered_extentobjects that have released their reservation as well as the correspondingQGROUP_RESERVEDextent bits.Normal qgroups gets away with this, I believe because it forces more work to happen on transaction commit, but I am not certain it is totally safe from the ordered_extent/leaked extent bit variant. Simple quotas hits this reliably.
Quoted 7 lines
The intent of the fix is to make disable take the time to clear that external to qgroups state as well: after flipping off the quota bit on
fs_info, flush delalloc and ordered extents, clearing the extent bits along the way. This makes it so there are no ordered extents or meta prealloc hanging around from the first enablement period during the second.Signed-off-by Boris Burkov <[email protected]>
Thanks,
Josef
Re: [PATCH 07/18] btrfs: create qgroup earlier in snapshot creation
In reply to Boris BurkovQuoted 29 lines · Boris Burkov
Pull creating the qgroup earlier in the snapshot. This allows simple quotas qgroups to see all the metadata writes related to the snapshot being created and to be born with the root node accounted.
Signed-off-by Boris Burkov <[email protected]>--- fs/btrfs/qgroup.c | 3 +++ fs/btrfs/transaction.c | 5 +++++ 2 files changed, 8 insertions(+)fs/btrfs/qgroup.c +3
-0@@ -1670,6 +1670,9 @@ int btrfs_create_qgroup(struct btrfs_trans_handle *trans, u64 qgroupid)struct btrfs_qgroup *qgroup;int ret =0 ; +if (btrfs_qgroup_mode (fs_info) ==BTRFS_QGROUP_MODE_DISABLED ) +return 0 ; +mutex_lock (&fs_info->qgroup_ioctl_lock);if (!fs_info->quota_root) { ret = -ENOTCONN ;fs/btrfs/transaction.c +0
-0
fs/btrfs/transaction.c static noinline int create_pending_snapshot(struct btrfs_trans_handle *trans, · line 1723
Quoted 8 lines
fs/btrfs/transaction.c +5
-0@@ -1716,6 +1716,11 @@ static noinline int create_pending_snapshot(struct btrfs_trans_handle *trans, }btrfs_release_path (path); + ret =btrfs_create_qgroup (trans, objectid); +if (ret) { +btrfs_abort_transaction (trans, ret); + goto fail; + }
Newline please.
How is this ok with normal qgroups? We weren't creating a qgroup at snapshot creation time at all it seems, so I don't understand how this is ok for qgroups. Thanks,
Josef
Re: [PATCH 08/18] btrfs: function for recording simple quota deltas
In reply to Boris BurkovQuoted 28 lines · Boris Burkov
Rather than re-computing shared/exclusive ownership based on backrefs and walking roots for implicit backrefs, simple quotas does an increment when creating an extent and a decrement when deleting it. Add the API for the extent item code to use to track those events.
Also add a helper function to make collecting parent qgroups in a ulist easier for functions like this.
Signed-off-by Boris Burkov <[email protected]>--- fs/btrfs/qgroup.c | 82 +++++++++++++++++++++++++++++++++++++++++++++++ fs/btrfs/qgroup.h | 11 ++++++- 2 files changed, 92 insertions(+), 1 deletion(-)fs/btrfs/qgroup.c +6
-0@@ -333,6 +333,44 @@ static int del_relation_rb(struct btrfs_fs_info *fs_info,return -ENOENT ; } +static int qgroup_collect_parents (struct btrfs_qgroup *qgroup, +struct ulist *ul) +{ +struct ulist_iterator uiter; +struct ulist_node *unode; +struct btrfs_qgroup_list *glist;
fs/btrfs/qgroup.c static int del_relation_rb(struct btrfs_fs_info *fs_info, · line 351
Quoted 10 lines
fs/btrfs/qgroup.c +10
-0+struct btrfs_qgroup *qg; +bool err_free = false; +int ret =0 ; + +if (!ul) { + ul =ulist_alloc (GFP_KERNEL ); + err_free = true; + }else { +ulist_reinit (ul); + }
You're calling this under a spin_lock, so GFP_KERNEL isn't allowed here. Additionally it doesn't seem like you ever call it with a NULL ulist, so probably just get rid of the above.
fs/btrfs/qgroup.c static int del_relation_rb(struct btrfs_fs_info *fs_info, · line 354
Quoted 3 lines
fs/btrfs/qgroup.c +3
-0+ + ret =ulist_add (ul, qgroup->qgroupid, +qgroup_to_aux (qgroup),GFP_ATOMIC );
We're going to hit ENOMEM in production here, we used to hit it all the time with the extent state stuff. How will this affect us? Will it screw up accounting? If so we need to figure out a way to back up out of the lock and pre-allocate a node to use in this case.
Quoted 23 lines
fs/btrfs/qgroup.c +19
-0+if (ret <0 ) + goto out; +ULIST_ITER_INIT (&uiter); +while ((unode =ulist_next (ul, &uiter))) { + qg =unode_aux_to_qgroup (unode); +list_for_each_entry (glist, &qg->groups, next_group) { + ret =ulist_add (ul, glist->group->qgroupid, +qgroup_to_aux (glist->group),GFP_ATOMIC ); +if (ret <0 ) + goto out; + } + } + ret =0 ; +out: +if (ret && err_free) +ulist_free (ul); +return ret; +} +#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS int btrfs_verify_qgroup_counts (struct btrfs_fs_info *fs_info,u64 qgroupid,u64 rfer,u64 excl)
fs/btrfs/qgroup.c void btrfs_qgroup_destroy_extent_records(struct btrfs_transaction *trans) · line 4577
Quoted 9 lines
fs/btrfs/qgroup.c +6
-0@@ -4531,3 +4569,47 @@ void btrfs_qgroup_destroy_extent_records(struct btrfs_transaction *trans)kfree (entry); } } + +int btrfs_record_simple_quota_delta (struct btrfs_fs_info *fs_info, +struct btrfs_simple_quota_delta *delta) +{ +int ret; +struct ulist *ul = fs_info->qgroup_ulist;
This isn't NULL if we're enabled right? Thanks,
Josef
Re: [PATCH 09/18] btrfs: rename tree_ref and data_ref owning_root
In reply to Boris BurkovQuoted 7 lines · Boris Burkov
commit
113479d5b8eb("btrfs: rename root fields in delayed refs structs") changed these fromref_roottoowning_root. However, there are many circumstances where that name is not really accurate and the root on the ref struct _is_ the referring root. In general, these are not the owning root, though it does happen in some ref merging cases involving overwrites during snapshots and similar.
Quoted 6 lines
Simple quotas cares quite a bit about tracking the original owner of an extent through delayed refs, so rename these back to free up the name for the real owning root (which will live on the generic
btrfs_refand the head ref)Signed-off-by Boris Burkov <[email protected]>
Thanks,
Josef
Re: [PATCH 10/18] btrfs: track owning root in btrfs_ref
In reply to Boris BurkovQuoted 143 lines · Boris Burkov
While data extents require us to store additional inline refs to track the original owner on free, this information is available implicitly for metadata. It is found in the owner field of the header of the tree block. Even if other trees refer to this block and the original ref goes away, we will not rewrite that header field, so it will reliably give the original owner.
In addition, there is a relocation case where a new data extent needs to have an owning root separate from the referring root wired through delayed refs.
To use it for recording simple quota deltas, we need to wire this root id through from when we create the delayed ref until we fully process it. Store it in the generic
btrfs_refstruct of the delayed ref.Signed-off-by Boris Burkov <[email protected]>--- fs/btrfs/delayed-ref.c | 7 ++++--- fs/btrfs/delayed-ref.h | 13 +++++++++++-- fs/btrfs/extent-tree.c | 19 ++++++++++++------- fs/btrfs/file.c | 10 +++++----- fs/btrfs/inode-item.c | 2 +- fs/btrfs/relocation.c | 16 +++++++++------- fs/btrfs/tree-log.c | 3 ++- 7 files changed, 44 insertions(+), 26 deletions(-)fs/btrfs/delayed-ref.c +4
-3@@ -840,7 +840,7 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans,static void init_delayed_ref_common (struct btrfs_fs_info *fs_info,struct btrfs_delayed_ref_node *ref,u64 bytenr,u64 num_bytes,u64 ref_root, -int action,u8 ref_type) +int action,u8 ref_type,u64 owning_root) {u64 seq =0 ;@@ -857,6 +857,7 @@ static void init_delayed_ref_common(struct btrfs_fs_info *fs_info, ref->action = action; ref->seq = seq; ref->type = ref_type; + ref->owning_root = owning_root;RB_CLEAR_NODE (&ref->ref_node);INIT_LIST_HEAD (&ref->add_list); }@@ -915,7 +916,7 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans,init_delayed_ref_common (fs_info, &ref->node, bytenr, num_bytes, generic_ref->tree_ref.ref_root, action, - ref_type); + ref_type, generic_ref->owning_root); ref->root = generic_ref->tree_ref.ref_root; ref->parent = parent; ref->level = level;@@ -989,7 +990,7 @@ int btrfs_add_delayed_data_ref(struct btrfs_trans_handle *trans,else ref_type =BTRFS_EXTENT_DATA_REF_KEY ;init_delayed_ref_common (fs_info, &ref->node, bytenr, num_bytes, - ref_root, action, ref_type); + ref_root, action, ref_type, ref_root); ref->root = ref_root; ref->parent = parent; ref->objectid = owner;fs/btrfs/delayed-ref.h +11
-2@@ -32,6 +32,12 @@ struct btrfs_delayed_ref_node {/* seq number to keep track of insertion order */ u64 seq; +/* +* root which originally allocated this extent and owns it for +* simple quota accounting purposes. +*/ +u64 owning_root; +/* ref count on this data structure */ refcount_t refs;@@ -239,6 +245,7 @@ struct btrfs_ref {#endif u64 bytenr;u64 len; +u64 owning_root;/* Bytenr of the parent tree block */ u64 parent;@@ -278,16 +285,18 @@ static inline u64 btrfs_calc_delayed_ref_bytes(const struct btrfs_fs_info *fs_in }static inline void btrfs_init_generic_ref (struct btrfs_ref *generic_ref, -int action,u64 bytenr,u64 len,u64 parent) +int action,u64 bytenr,u64 len,u64 parent,u64 owning_root) { generic_ref->action = action; generic_ref->bytenr = bytenr; generic_ref->len = len; generic_ref->parent = parent; + generic_ref->owning_root = owning_root; }static inline void btrfs_init_tree_ref (struct btrfs_ref *generic_ref, -int level,u64 root,u64 mod_root,bool skip_qgroup) +int level,u64 root,u64 mod_root, +bool skip_qgroup) {#ifdef CONFIG_BTRFS_FS_REF_VERIFY /* If @real_root not set, use @root as fallback */ fs/btrfs/extent-tree.c +3
-3@@ -2410,7 +2410,7 @@ static int __btrfs_mod_ref(struct btrfs_trans_handle *trans, num_bytes =btrfs_file_extent_disk_num_bytes (buf, fi); key.offset -=btrfs_file_extent_offset (buf, fi);btrfs_init_generic_ref (&generic_ref, action, bytenr, - num_bytes, parent); + num_bytes, parent, ref_root);btrfs_init_data_ref (&generic_ref, ref_root, key.objectid, key.offset, root->root_key.objectid, for_reloc);@@ -2424,7 +2424,7 @@ static int __btrfs_mod_ref(struct btrfs_trans_handle *trans, bytenr =btrfs_node_blockptr (buf, i); num_bytes = fs_info->nodesize;btrfs_init_generic_ref (&generic_ref, action, bytenr, - num_bytes, parent); + num_bytes, parent, ref_root);btrfs_init_tree_ref (&generic_ref, level -1 , ref_root, root->root_key.objectid, for_reloc);if (inc)@@ -3242,7 +3242,7 @@ void btrfs_free_tree_block(struct btrfs_trans_handle *trans,int ret;btrfs_init_generic_ref (&generic_ref,BTRFS_DROP_DELAYED_REF , - buf->start, buf->len, parent); + buf->start, buf->len, parent,btrfs_header_owner (buf));btrfs_init_tree_ref (&generic_ref,btrfs_header_level (buf), root_id,0 , false);
fs/btrfs/extent-tree.c int btrfs_alloc_reserved_file_extent(struct btrfs_trans_handle *trans, · line 4706
Quoted 8 lines
fs/btrfs/extent-tree.c +4
-0@@ -4699,12 +4699,16 @@ int btrfs_alloc_reserved_file_extent(struct btrfs_trans_handle *trans, struct btrfs_key *ins) {struct btrfs_ref generic_ref = {0 }; +u64 root_objectid = root->root_key.objectid; +u64 owning_root = root_objectid; + +BUG_ON (root_objectid ==BTRFS_TREE_LOG_OBJECTID );
This is a duplicate check of what's checked below. Also we don't want to add new BUG_ON()'s, we want to add ASSERT()'s. Thanks,
Josef
Re: [PATCH 11/18] btrfs: track original extent owner in head_ref
In reply to Boris BurkovQuoted 11 lines · Boris Burkov
Simple quotas requires tracking the original creating root of any given extent. This gets complicated when multiple subvolumes create overlapping/contradictory refs in the same transaction. For example, due to modifying or deleting an extent while also snapshotting it.
To resolve this in a general way, take advantage of the fact that we are essentially already tracking this for handling releasing reservations. The head ref coalesces the various refs and uses
must_insert_reservedto check if it needs to create an extent/free reservation. Store the ref that setmust_insert_reservedas the owning ref on the head ref.
Quoted 6 lines
Note that this can result in writing an extent for the very first time with an owner different from its only ref, but it will look the same as if you first created it with the original owning ref, then added the other ref, then removed the owning ref.
Signed-off-by Boris Burkov <[email protected]>
Reviewed-by; Josef Bacik <[email protected]>
Thanks,
Josef
Re: [PATCH 12/18] btrfs: new inline ref storing owning subvol of data extents
In reply to Boris BurkovQuoted 7 lines · Boris Burkov
In order to implement simple quota groups, we need to be able to associate a data extent with the subvolume that created it. Once you account for reflink, this information cannot be recovered without explicitly storing it. Options for storing it are:
- a new key/item
- a new extent inline ref item
Quoted 6 lines
The former is backwards compatible, but wastes space, the latter is incompat, but is efficient in space and reuses the existing inline ref machinery, while only abusing it a tiny amount -- specifically, the new item is not a ref, per-se.
Signed-off-by Boris Burkov <[email protected]>
Thanks,
Josef
Re: [PATCH 13/18] btrfs: inline owner ref lookup helper
In reply to Boris BurkovQuoted 14 lines · Boris Burkov
Inline ref parsing is a bit tricky and relies on a decent amount of implicit information, so I think it is beneficial to have a helper function for reading the owner ref, if only to "document" the format, along with the write path.
The main subtlety of note which I was missing by open-coding this was that it is important to check whether or not inline refs are present *at all*. i.e., if we are writing out a new extent under squotas, we will always use a big enough item for the inline ref and have it. However, it is possible that some random item predating squotas will not have any inline refs. In that case, trying to read the "type" field of the first inline ref will just be reading garbage in the form of whatever is in the next item.
Quoted 5 lines
This will be used by the extent free-ing path, which looks up data extent owners as well as a relocation path which needs to grab the owner before relocating an extent.
Signed-off-by Boris Burkov <[email protected]>
Thanks,
Josef
Re: [PATCH 14/18] btrfs: record simple quota deltas
In reply to Boris BurkovQuoted 31 lines · Boris Burkov
At the moment that we run delayed refs, we make the final ref-count based decision on creating/removing extent (and metadata) items. Therefore, it is exactly the spot to hook up simple quotas.
There are a few important subtleties to the fields we must collect to accurately track simple quotas, particularly when removing an extent. When removing a data extent, the ref could be in any tree (due to reflink, for example) and so we need to recover the owning root id from the owner ref item. When removing a metadata extent, we know the owning root from the owner field in the header when we create the delayed ref, so we can recover it from there.
We must also be careful to handle reservations properly to not leaked reserved space. The happy path is freeing the reservation when the simple quota delta runs on a data extent. If that doesn't happen, due to refs canceling out or some error, the ref head already has the
must_insert_reservedmachinery to handle this, so we piggy back on that and use it to clean up the reserved data.Signed-off-by Boris Burkov <[email protected]>--- fs/btrfs/delayed-ref.c | 3 ++ fs/btrfs/delayed-ref.h | 6 ++++ fs/btrfs/extent-tree.c | 79 +++++++++++++++++++++++++++++++++++++----- 3 files changed, 80 insertions(+), 8 deletions(-)fs/btrfs/delayed-ref.c +0
-0
fs/btrfs/delayed-ref.c static void init_delayed_ref_head(struct btrfs_delayed_ref_head *head_ref, · line 740
Quoted 6 lines
fs/btrfs/delayed-ref.c +3
-0@@ -735,6 +735,9 @@ static void init_delayed_ref_head(struct btrfs_delayed_ref_head *head_ref, head_ref->bytenr = bytenr; head_ref->num_bytes = num_bytes; head_ref->ref_mod = count_mod; + head_ref->reserved_bytes =0 ; +if (reserved) + head_ref->reserved_bytes = reserved;
This is just
head_ref->reserved_bytes = reserved;
Thanks,
Josef
Re: [PATCH 15/18] btrfs: simple quota auto hierarchy for nested subvols
In reply to Boris BurkovQuoted 74 lines · Boris Burkov
Consider the following sequence:
- enable quotas
- create subvol S id 256 at dir outer/
- create a qgroup 1/100
- add 0/256 (S's auto qgroup) to 1/100
- create subvol T id 257 at dir outer/inner/With full qgroups, there is no relationship between 0/257 and either of 0/256 or 1/100. There is an inherit feature that the creator of inner/ can use to specify it ought to be in 1/100.
Simple quotas are targeted at container isolation, where such automatic inheritance for not necessarily trusted/controlled nested subvol creation would be quite helpful. Therefore, add a new default behavior for simple quotas: when you create a nested subvol, automatically inherit as parents any parents of the qgroup of the subvol the new inode is going in.
In our example, 257/0 would also be under 1/100, allowing easy control of a total quota over an arbitrary hierarchy of subvolumes.
I think this _might_ be a generally useful behavior, so it could be interesting to put it behind a new inheritance flag that simple quotas always use while traditional quotas let the user specify, but this is a minimally intrusive change to start.
Signed-off-by Boris Burkov <[email protected]>--- fs/btrfs/ioctl.c | 2 +- fs/btrfs/qgroup.c | 46 +++++++++++++++++++++++++++++++++++++++--- fs/btrfs/qgroup.h | 6 +++--- fs/btrfs/transaction.c | 13 ++++++++---- 4 files changed, 56 insertions(+), 11 deletions(-)fs/btrfs/ioctl.c +1
-1@@ -652,7 +652,7 @@ static noinline int create_subvol(struct mnt_idmap *idmap,/* Tree log can't currently deal with an inode which is a new root. */ btrfs_set_log_full_commit (trans); - ret =btrfs_qgroup_inherit (trans,0 , objectid, inherit); + ret =btrfs_qgroup_inherit (trans,0 , objectid, root->root_key.objectid, inherit);if (ret) goto out;fs/btrfs/qgroup.c +10
-2@@ -1557,8 +1557,7 @@ static int quick_update_accounting(struct btrfs_fs_info *fs_info,return ret; } -int btrfs_add_qgroup_relation (struct btrfs_trans_handle *trans,u64 src, -u64 dst) +int btrfs_add_qgroup_relation (struct btrfs_trans_handle *trans,u64 src,u64 dst) {struct btrfs_fs_info *fs_info = trans->fs_info;struct btrfs_qgroup *parent;@@ -2998,6 +2997,42 @@ int btrfs_run_qgroups(struct btrfs_trans_handle *trans)return ret; } +static int qgroup_auto_inherit (struct btrfs_fs_info *fs_info, +u64 inode_rootid, +struct btrfs_qgroup_inherit **inherit) +{ +int i =0 ; +u64 num_qgroups =0 ; +struct btrfs_qgroup *inode_qg; +struct btrfs_qgroup_list *qg_list; +
fs/btrfs/qgroup.c int btrfs_run_qgroups(struct btrfs_trans_handle *trans) · line 3018
Quoted 10 lines
fs/btrfs/qgroup.c +10
-0+if (*inherit) +return -EEXIST ; + + inode_qg =find_qgroup_rb (fs_info, inode_rootid); +if (!inode_qg) +return -ENOENT ; + +list_for_each_entry (qg_list, &inode_qg->groups, next_group) { + ++num_qgroups; + }
You can simplify this with
num_qgroups = list_count_nodes(&inode_qg->groups);
Thanks,
Josef
Re: [PATCH 16/18] btrfs: check generation when recording simple quota delta
In reply to Boris BurkovQuoted 14 lines · Boris Burkov
Simple quotas count extents only from the moment the feature is enabled. Therefore, if we do something like:
1. create subvol S
2. write F in S
3. enable quotas
4. remove F
5. write G in Sthen after 3. and 4. we would expect the simple quota usage of S to be 0 (putting aside some metadata extents that might be written) and after 5., it should be the size of G plus metadata. Therefore, we need to be able to determine whether a particular quota delta we are processing predates simple quota enablement.
Quoted 7 lines
To do this, store the transaction id when quotas were enabled. In
fs_infofor immediate use and in the quota status item to make it recoverable on mount. When we see a delta, check if the generation of the extent item is less than that of quota enablement. If so, we should ignore the delta from this extent.Signed-off-by Boris Burkov <[email protected]>
Thanks,
Josef
Re: [PATCH 17/18] btrfs: track metadata relocation cow with simple quota
In reply to Boris BurkovQuoted 23 lines · Boris Burkov
Relocation cows metadata blocks in two cases for the reloc root:
- copying the subvol root item when creating the reloc root
- copying a btree node when there is a cow during relocationIn both cases, the resulting btree node hits an abnormal code path with respect to the owner field in its
btrfs_header. It first creates the root item for the new objectid, which populates the reloc root id, and it at this point that delayed refs are created.Later, it fully copies the old node into the new node (including the original owner field) which overwrites it. This results in a simple quotas mismatch where we run the delayed ref for the reloc root which has no simple quota effect (reloc root is not an fstree) but when we ultimately delete the node, the owner is the real original fstree and we do free the space.
To work around this without tampering with the behavior of relocation, add a parameter to
btrfs_add_tree_blockthat lets the relocation code path specify a different owning root than the "operating" root (in this case, owning root is the real root and the operating root is the reloc root). These can naturally be plumbed into delayed refs that have the same concept.
Quoted 6 lines
Note that this is a double count in some sense, but a relatively natural one, as there are really two extents, and the old one will be deleted soon. This is consistent with how data relocation extents are accounted by simple quotas.
Signed-off-by Boris Burkov <[email protected]>
Thanks,
Josef
Re: [PATCH 18/18] btrfs: track data relocation with simple quota
In reply to Boris BurkovQuoted 137 lines · Boris Burkov
Relocation data allocations are quite tricky for simple quotas. The basic data relocation sequence is (ignoring details that aren't relevant to this fix):
- create a fake relocation data fs root
- create a fake relocation inode in that root
- foreach data extent:
- preallocate a data extent on behalf of the fake inode
- copy over the data
- foreach extent
- swap the refs so that the original file extent now refers to the new extent item
- drop the fake root, dropping its refs on the old extents, which lets us delete them.Done naively, this results in storing an extent item in the extent tree whose
owner_refpoints at the relocation data root and a no-op squota recording, since the reloc root is not a legit fstree. So far, that's OK. The problem comes when you do the swap, and leave an extent item owned by this bogus root as the real permanent extents of the file. If the file then drops that ref, we free it and no-op account that against the fake relocation root. Essentially, this means that relocation is simple quota "extent laundering", since we re-own the extents into a fake root.Simple quotas very intentionally doesn't have a mechanism for transferring ownership of extents, as that is exactly the complicated thing we are trying to avoid with the new design. Further, it cannot be correctly done in this case, since at the time you create the new "real" refs, there is no way to know which was the original owner before relocation unless we track it.
Therefore, it makes more sense to trick the preallocation to handle relocation as a special case and note the proper owner ref from the beginning. That way, we never write out an extent item without the correct owner ref that it will eventually have.
This could be done by wiring a special root parameter all the way through the allocation code path, but to avoid that special case touching all the code, take advantage of the serial nature of relocation to store the src root on the relocation root object. Then when we finish the prealloc, if it happens to be this case, prepare the delayed ref appropriately.
This is obviously a smelly bit of code, but I think it is the best solution to the problem, given the relocation implementation.
Signed-off-by Boris Burkov <[email protected]>--- fs/btrfs/ctree.h | 1 + fs/btrfs/extent-tree.c | 13 +++++++------ fs/btrfs/relocation.c | 15 +++++++++++++++ 3 files changed, 23 insertions(+), 6 deletions(-)fs/btrfs/ctree.h +1
-0@@ -333,6 +333,7 @@ struct btrfs_root {#ifdef CONFIG_BTRFS_DEBUG struct list_head leak_list;#endif +u64 relocation_src_root; };static inline bool btrfs_root_readonly (const struct btrfs_root *root)fs/btrfs/extent-tree.c +7
-6@@ -57,7 +57,7 @@ static void __run_delayed_extent_op(struct btrfs_delayed_extent_op *extent_op,static int alloc_reserved_file_extent (struct btrfs_trans_handle *trans,u64 parent,u64 root_objectid,u64 flags,u64 owner,u64 offset, -struct btrfs_key *ins,int ref_mod); +struct btrfs_key *ins,int ref_mod,u64 oref_root);static int alloc_reserved_tree_block (struct btrfs_trans_handle *trans,struct btrfs_delayed_ref_node *node,struct btrfs_delayed_extent_op *extent_op);@@ -1541,7 +1541,7 @@ static int run_delayed_data_ref(struct btrfs_trans_handle *trans, ret =alloc_reserved_file_extent (trans, parent, ref_root, flags, ref->objectid, ref->offset, &ins, - node->ref_mod); + node->ref_mod, href->owning_root);if (!ret) ret =btrfs_record_simple_quota_delta (trans->fs_info, &delta); }else if (node->action ==BTRFS_ADD_DELAYED_REF ) {@@ -4683,7 +4683,7 @@ static int alloc_reserved_extent(struct btrfs_trans_handle *trans, u64 bytenr,static int alloc_reserved_file_extent (struct btrfs_trans_handle *trans,u64 parent,u64 root_objectid,u64 flags,u64 owner,u64 offset, -struct btrfs_key *ins,int ref_mod) +struct btrfs_key *ins,int ref_mod,u64 oref_root) {struct btrfs_fs_info *fs_info = trans->fs_info;struct btrfs_root *extent_root;@@ -4731,7 +4731,7 @@ static int alloc_reserved_file_extent(struct btrfs_trans_handle *trans,if (simple_quota) {btrfs_set_extent_inline_ref_type (leaf, iref,BTRFS_EXTENT_OWNER_REF_KEY ); oref = (struct btrfs_extent_owner_ref *)(&iref->offset); -btrfs_set_extent_owner_ref_root_id (leaf, oref, root_objectid); +btrfs_set_extent_owner_ref_root_id (leaf, oref, oref_root); iref = (struct btrfs_extent_inline_ref *)(oref +1 ); }btrfs_set_extent_inline_ref_type (leaf, iref, type);@@ -4842,7 +4842,8 @@ int btrfs_alloc_reserved_file_extent(struct btrfs_trans_handle *trans,BUG_ON (root_objectid ==BTRFS_TREE_LOG_OBJECTID ); -BUG_ON (root->root_key.objectid ==BTRFS_TREE_LOG_OBJECTID ); +if (btrfs_is_data_reloc_root (root) &&is_fstree (root->relocation_src_root)) + owning_root = root->relocation_src_root;btrfs_init_generic_ref (&generic_ref,BTRFS_ADD_DELAYED_EXTENT , ins->objectid, ins->offset,0 , owning_root);@@ -4899,7 +4900,7 @@ int btrfs_alloc_logged_file_extent(struct btrfs_trans_handle *trans,spin_unlock (&space_info->lock); ret =alloc_reserved_file_extent (trans,0 , root_objectid,0 , owner, - offset, ins,1 ); + offset, ins,1 , root_objectid);if (ret)btrfs_pin_extent (trans, ins->objectid, ins->offset,1 ); ret =btrfs_record_simple_quota_delta (fs_info, &delta);fs/btrfs/relocation.c +5
-0@@ -3665,6 +3665,21 @@ static noinline_for_stack int relocate_block_group(struct reloc_control *rc) struct btrfs_extent_item); flags =btrfs_extent_flags (path->nodes[0 ], ei); +/* +* If we are relocating a simple quota owned extent item, we need +* to note the owner on the reloc data root so that when we +* allocate the replacement item, we can attribute it to the +* correct eventual owner (rather than the reloc data root)
fs/btrfs/relocation.c static noinline_for_stack int relocate_block_group(struct reloc_control *rc) · line 3682
Quoted 10 lines
fs/btrfs/relocation.c +10
-0+*/ +if (btrfs_qgroup_mode (fs_info) ==BTRFS_QGROUP_MODE_SIMPLE ) { +struct btrfs_root *root =BTRFS_I (rc->data_inode)->root; +u64 owning_root_id =btrfs_get_extent_owner_root (fs_info, + path->nodes[0 ], + path->slots[0 ]); + + root->relocation_src_root = owning_root_id; + } +
This is almost correct but can mess up if we have adjacent extents that are owned by different roots. If you look further down we move the extents via relocate_data_extent(), which will cluster together ranges to limit the number of preallocations you do. You're going to have to add a check in there for the owning_root_id != root->relocation_src_root, do the prealloc, and then set root->relocation_src_root in there and carry on. Thanks,
Josef
Re: [PATCH 07/18] btrfs: create qgroup earlier in snapshot creation
In reply to Josef BacikQuoted 41 lines · Josef Bacik
Quoted 29 lines · Boris Burkov
Pull creating the qgroup earlier in the snapshot. This allows simple quotas qgroups to see all the metadata writes related to the snapshot being created and to be born with the root node accounted.
Signed-off-by Boris Burkov <[email protected]>--- fs/btrfs/qgroup.c | 3 +++ fs/btrfs/transaction.c | 5 +++++ 2 files changed, 8 insertions(+)fs/btrfs/qgroup.c +3
-0@@ -1670,6 +1670,9 @@ int btrfs_create_qgroup(struct btrfs_trans_handle *trans, u64 qgroupid)struct btrfs_qgroup *qgroup;int ret =0 ; +if (btrfs_qgroup_mode (fs_info) ==BTRFS_QGROUP_MODE_DISABLED ) +return 0 ; +mutex_lock (&fs_info->qgroup_ioctl_lock);if (!fs_info->quota_root) { ret = -ENOTCONN ;fs/btrfs/transaction.c +0
-0fs/btrfs/transaction.c static noinline int create_pending_snapshot(struct btrfs_trans_handle *trans, · line 1723
Quoted 8 lines
fs/btrfs/transaction.c +5
-0@@ -1716,6 +1716,11 @@ static noinline int create_pending_snapshot(struct btrfs_trans_handle *trans, }btrfs_release_path (path); + ret =btrfs_create_qgroup (trans, objectid); +if (ret) { +btrfs_abort_transaction (trans, ret); + goto fail; + }Newline please.
Quoted 2 lines
How is this ok with normal qgroups? We weren't creating a qgroup at snapshot creation time at all it seems, so I don't understand how this is ok for qgroups.
qgroup_account_snapshot calls btrfs_qgroup_inherit which contains a separate implementation of qgroup creation.
Quoted 3 lines
Thanks,
Josef
Re: [PATCH 07/18] btrfs: create qgroup earlier in snapshot creation
Quoted 42 lines · Boris Burkov
Quoted 41 lines · Josef Bacik
Quoted 29 lines · Boris Burkov
Pull creating the qgroup earlier in the snapshot. This allows simple quotas qgroups to see all the metadata writes related to the snapshot being created and to be born with the root node accounted.
Signed-off-by Boris Burkov <[email protected]>--- fs/btrfs/qgroup.c | 3 +++ fs/btrfs/transaction.c | 5 +++++ 2 files changed, 8 insertions(+)fs/btrfs/qgroup.c +3
-0@@ -1670,6 +1670,9 @@ int btrfs_create_qgroup(struct btrfs_trans_handle *trans, u64 qgroupid)struct btrfs_qgroup *qgroup;int ret =0 ; +if (btrfs_qgroup_mode (fs_info) ==BTRFS_QGROUP_MODE_DISABLED ) +return 0 ; +mutex_lock (&fs_info->qgroup_ioctl_lock);if (!fs_info->quota_root) { ret = -ENOTCONN ;fs/btrfs/transaction.c +0
-0fs/btrfs/transaction.c static noinline int create_pending_snapshot(struct btrfs_trans_handle *trans, · line 1723
Quoted 8 lines
fs/btrfs/transaction.c +5
-0@@ -1716,6 +1716,11 @@ static noinline int create_pending_snapshot(struct btrfs_trans_handle *trans, }btrfs_release_path (path); + ret =btrfs_create_qgroup (trans, objectid); +if (ret) { +btrfs_abort_transaction (trans, ret); + goto fail; + }Newline please.
Quoted 5 lines
Quoted 2 lines
How is this ok with normal qgroups? We weren't creating a qgroup at snapshot creation time at all it seems, so I don't understand how this is ok for qgroups.
qgroup_account_snapshotcallsbtrfs_qgroup_inheritwhich contains a separate implementation of qgroup creation.
Which it still does, so I'm confused as to how this is ok. How do we not get an EEXIST when we do the btrfs_qgroup_inherit? Thanks,
Josef
Re: [PATCH 10/18] btrfs: track owning root in btrfs_ref
In reply to Josef BacikQuoted 143 lines · Josef Bacik
Quoted 142 lines · Boris Burkov
While data extents require us to store additional inline refs to track the original owner on free, this information is available implicitly for metadata. It is found in the owner field of the header of the tree block. Even if other trees refer to this block and the original ref goes away, we will not rewrite that header field, so it will reliably give the original owner.
In addition, there is a relocation case where a new data extent needs to have an owning root separate from the referring root wired through delayed refs.
To use it for recording simple quota deltas, we need to wire this root id through from when we create the delayed ref until we fully process it. Store it in the generic
btrfs_refstruct of the delayed ref.Signed-off-by Boris Burkov <[email protected]>--- fs/btrfs/delayed-ref.c | 7 ++++--- fs/btrfs/delayed-ref.h | 13 +++++++++++-- fs/btrfs/extent-tree.c | 19 ++++++++++++------- fs/btrfs/file.c | 10 +++++----- fs/btrfs/inode-item.c | 2 +- fs/btrfs/relocation.c | 16 +++++++++------- fs/btrfs/tree-log.c | 3 ++- 7 files changed, 44 insertions(+), 26 deletions(-)fs/btrfs/delayed-ref.c +4
-3@@ -840,7 +840,7 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans,static void init_delayed_ref_common (struct btrfs_fs_info *fs_info,struct btrfs_delayed_ref_node *ref,u64 bytenr,u64 num_bytes,u64 ref_root, -int action,u8 ref_type) +int action,u8 ref_type,u64 owning_root) {u64 seq =0 ;@@ -857,6 +857,7 @@ static void init_delayed_ref_common(struct btrfs_fs_info *fs_info, ref->action = action; ref->seq = seq; ref->type = ref_type; + ref->owning_root = owning_root;RB_CLEAR_NODE (&ref->ref_node);INIT_LIST_HEAD (&ref->add_list); }@@ -915,7 +916,7 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans,init_delayed_ref_common (fs_info, &ref->node, bytenr, num_bytes, generic_ref->tree_ref.ref_root, action, - ref_type); + ref_type, generic_ref->owning_root); ref->root = generic_ref->tree_ref.ref_root; ref->parent = parent; ref->level = level;@@ -989,7 +990,7 @@ int btrfs_add_delayed_data_ref(struct btrfs_trans_handle *trans,else ref_type =BTRFS_EXTENT_DATA_REF_KEY ;init_delayed_ref_common (fs_info, &ref->node, bytenr, num_bytes, - ref_root, action, ref_type); + ref_root, action, ref_type, ref_root); ref->root = ref_root; ref->parent = parent; ref->objectid = owner;fs/btrfs/delayed-ref.h +11
-2@@ -32,6 +32,12 @@ struct btrfs_delayed_ref_node {/* seq number to keep track of insertion order */ u64 seq; +/* +* root which originally allocated this extent and owns it for +* simple quota accounting purposes. +*/ +u64 owning_root; +/* ref count on this data structure */ refcount_t refs;@@ -239,6 +245,7 @@ struct btrfs_ref {#endif u64 bytenr;u64 len; +u64 owning_root;/* Bytenr of the parent tree block */ u64 parent;@@ -278,16 +285,18 @@ static inline u64 btrfs_calc_delayed_ref_bytes(const struct btrfs_fs_info *fs_in }static inline void btrfs_init_generic_ref (struct btrfs_ref *generic_ref, -int action,u64 bytenr,u64 len,u64 parent) +int action,u64 bytenr,u64 len,u64 parent,u64 owning_root) { generic_ref->action = action; generic_ref->bytenr = bytenr; generic_ref->len = len; generic_ref->parent = parent; + generic_ref->owning_root = owning_root; }static inline void btrfs_init_tree_ref (struct btrfs_ref *generic_ref, -int level,u64 root,u64 mod_root,bool skip_qgroup) +int level,u64 root,u64 mod_root, +bool skip_qgroup) {#ifdef CONFIG_BTRFS_FS_REF_VERIFY /* If @real_root not set, use @root as fallback */ fs/btrfs/extent-tree.c +3
-3@@ -2410,7 +2410,7 @@ static int __btrfs_mod_ref(struct btrfs_trans_handle *trans, num_bytes =btrfs_file_extent_disk_num_bytes (buf, fi); key.offset -=btrfs_file_extent_offset (buf, fi);btrfs_init_generic_ref (&generic_ref, action, bytenr, - num_bytes, parent); + num_bytes, parent, ref_root);btrfs_init_data_ref (&generic_ref, ref_root, key.objectid, key.offset, root->root_key.objectid, for_reloc);@@ -2424,7 +2424,7 @@ static int __btrfs_mod_ref(struct btrfs_trans_handle *trans, bytenr =btrfs_node_blockptr (buf, i); num_bytes = fs_info->nodesize;btrfs_init_generic_ref (&generic_ref, action, bytenr, - num_bytes, parent); + num_bytes, parent, ref_root);btrfs_init_tree_ref (&generic_ref, level -1 , ref_root, root->root_key.objectid, for_reloc);if (inc)@@ -3242,7 +3242,7 @@ void btrfs_free_tree_block(struct btrfs_trans_handle *trans,int ret;btrfs_init_generic_ref (&generic_ref,BTRFS_DROP_DELAYED_REF , - buf->start, buf->len, parent); + buf->start, buf->len, parent,btrfs_header_owner (buf));btrfs_init_tree_ref (&generic_ref,btrfs_header_level (buf), root_id,0 , false);
Quoted 12 lines
fs/btrfs/extent-tree.c int btrfs_alloc_reserved_file_extent(struct btrfs_trans_handle *trans, · line 4706
Quoted 9 lines
fs/btrfs/extent-tree.c +4
-0@@ -4699,12 +4699,16 @@ int btrfs_alloc_reserved_file_extent(struct btrfs_trans_handle *trans, struct btrfs_key *ins) {struct btrfs_ref generic_ref = {0 }; +u64 root_objectid = root->root_key.objectid; +u64 owning_root = root_objectid; + +BUG_ON (root_objectid ==BTRFS_TREE_LOG_OBJECTID );This is a duplicate check of what's checked below. Also we don't want to add new BUG_ON()'s, we want to add ASSERT()'s. Thanks,
Oops. I just meant to use root_objectid instead of root->root_key.objectid and somehow left in the old line, my bad.
I assume we don't want to change these existing BUG_ONs to ASSERTs without specific justification, though.
Quoted 2 lines
Josef
Re: [PATCH 07/18] btrfs: create qgroup earlier in snapshot creation
In reply to Josef BacikQuoted 46 lines · Josef Bacik
Quoted 45 lines · Boris Burkov
Quoted 43 lines · Josef Bacik
Quoted 29 lines · Boris Burkov
Pull creating the qgroup earlier in the snapshot. This allows simple quotas qgroups to see all the metadata writes related to the snapshot being created and to be born with the root node accounted.
Signed-off-by Boris Burkov <[email protected]>--- fs/btrfs/qgroup.c | 3 +++ fs/btrfs/transaction.c | 5 +++++ 2 files changed, 8 insertions(+)fs/btrfs/qgroup.c +3
-0@@ -1670,6 +1670,9 @@ int btrfs_create_qgroup(struct btrfs_trans_handle *trans, u64 qgroupid)struct btrfs_qgroup *qgroup;int ret =0 ; +if (btrfs_qgroup_mode (fs_info) ==BTRFS_QGROUP_MODE_DISABLED ) +return 0 ; +mutex_lock (&fs_info->qgroup_ioctl_lock);if (!fs_info->quota_root) { ret = -ENOTCONN ;fs/btrfs/transaction.c +0
-0fs/btrfs/transaction.c static noinline int create_pending_snapshot(struct btrfs_trans_handle *trans, · line 1723
Quoted 8 lines
fs/btrfs/transaction.c +5
-0@@ -1716,6 +1716,11 @@ static noinline int create_pending_snapshot(struct btrfs_trans_handle *trans, }btrfs_release_path (path); + ret =btrfs_create_qgroup (trans, objectid); +if (ret) { +btrfs_abort_transaction (trans, ret); + goto fail; + }Newline please.
How is this ok with normal qgroups? We weren't creating a qgroup at snapshot creation time at all it seems, so I don't understand how this is ok for qgroups.
Quoted 7 lines
Quoted 2 lines
qgroup_account_snapshotcallsbtrfs_qgroup_inheritwhich contains a separate implementation of qgroup creation.Which it still does, so I'm confused as to how this is ok. How do we not get an EEXIST when we do the
btrfs_qgroup_inherit? Thanks,Josef
add_qgroup_item ignores EEXIST. I believe this should be exercised by a user intentionally creating a qgroup for a subvolume before creating the subvolume (I believe that is explicitly supported, though it does seem to rely on guessing the subvol ID?!)
from man btrfs-qgroup:
"For the 0/<subvolume id> qgroup, a qgroup can be created even before the subvolume is created."