Validate journal log items during log recovery.
Subsystems (2)
cover Validate journal log items during log recovery.
Log recovery reads log items straight from the journal and acts on them with very little checking. Validation of each item type is scattered and much of it runs only as the item is being applied to the filesystem. Corrupt or malicious log contents can therefore be partially acted on before anything notices they are wrong, and the checks that do exist are inconsistent from one item type to the next.
This series makes log item validation a first-class, up-front step. As each item is decoded from the journal it is fully validated before any recovery code acts on it, so downstream recovery can trust the item's structure and contents. The validation is driven by per-item-type callbacks on the recovery item operations, so every item type is checked the same way: the region count is checked against what the item type and its logged fields require, each region is bounds checked, and the whole item is checked for internal consistency. Existing validation that was scattered through the commit handlers is brought forward into these callbacks, and the now-redundant checks are removed so the commit handlers are left to do only replay.
The operation header decode and the transaction header are validated the same way, so the record framing is checked before the items within it are assembled.
The scope is the on-disk log format and the in-core structures recovery builds from it. Every log item type is covered. The validation is what can be checked from the log item alone together with stable mount and superblock geometry.
Some checks cannot be made at decode time and are deliberately left to the first runtime access after recovery. Realtime group bounds are not available because the realtime inodes are not set up until after recovery. Filesystem block number bounds cannot be trusted at decode time because a growfs superblock update is not replayed into the in-core superblock until pass two. Hence extent and pointer block range validation will need to be reworked before it can be relied on. More detail is below under known issues.
The known issues will take a little time to be addressed, so I'd prefer to get feedback on the rest of the patchset before I spend time reworking that aspect of it. Early(-ish) feedback on the structure and implementation as it currently stands is more important right now.
The patchset passes all of fstests on 1k and 4k block sizes (except for the couple of growfs/recovery stress tests) without regressions, so it largely works for known good journals. Anyone who has been fuzzing journal structures with LLMs and/or modified syzbots are welcome to try it and see what holes they can find in it.
The entire series can be pulled from here:
https://git.kernel.org/pub/scm/linux/kernel/git/dgc/linux-xfs.git log-verification-1
Known issues:
1. Validation runs repeatedly on every recovery pass
Except for the CRC validation pass.
Pass 1 needs to validate all the transaction/checkpoint structures as we walk the entire active journal to find cancellation items. We also need to validate buffer and quotaoff items as tehy are recorded in cancelation tables.
Pass 2 readahead needs items that are read ahead to be validated sufficiently to ensure the location being read from disk is likely to be OK.
Pass 2 needs to fully validate all the items being recovered, before they are recovered.
Some of the item recovery also reformats the items being recovered; things like 32bit vs 64bit incore structure differences are being resolved at item validation time so they don't need to be handled differently by validation or recovery code. This means that validation needs to run on those items on every pass that they will have something run against them. This needs more thought; inodes are the main issue here, and largely they are only parsed in pass 2.
Hence we might be able to split up validation into pass-1 + pass-2 variants. pass-1 needs to validate the number and size of regions matches the format header to ensure checkpoint decoding is always valid. pass-2 can then do the full item validation.
2. Geometry validation versus growfs and shrink
This is an extension of issue 1, in that validation occurs before replay. The geometry items validate against is the geometry in effect at the start of recovery. A geometry change only takes effect when the primary superblock buffer is replayed, which happens during transaction recovery in the second pass, not during checkpoint region decoding.
A growfs operation logs the superblock geometry change ahead of any metadata that uses the new geometry. A shrink also logs the superblock geometry change after relevant internal metadata has been updated and/or invalidated. Hence if the superblock geometry changes during recovery, item validation at region decode time will fail to validate geometry-based fields correctly.
This affects any item that carries a type that encodes geometry information, such as inode numbers, disk addresses, and extents, including the realtime variants. Structural validation that does not depend on geometry is unaffected.
As the geometry change only takes effect during transaction recovery, any validation that depends on it must run after the superblock buffer has been replayed.
This causes issues for readahead validation, which decodes addresses and submits IO, potentially for addresses outside the range of the initial superblock geometry. Readahead silently discards reads beyond the end of the filesystem, but we shouldn't issue them in the first place.
Even in pass 2, the checkpoint where the geometry change is recorded cannot be validated until recovery even though items in the checkpoint are correctly ordered either side of the change.
Hence it is likely that we need two layers of item validation for pass 2 - the decode time validation of everything except disk addresses, and recovery time validation of addresses. This will order all disk address based validation correctly against grow/shrink operations.
3. fold ->complete() into ->validate_item()
The xlog_recover_item_ops ->complete() callback was added early on, when it was not entirely clear how item validation would end u plooking. In the end, it is used by only one special item type:
the transaction header.
->validate_item() already runs on every item at that same point, so ->complete() probably should be folded into it. ->validate_item() would perform the header copy and signal that the item is consumed, removing the need for ->complete() altogether.
Commits:
28f113a xfs: set the unmount record operation header length a61b098 xfs: add log recovery item validation design document 06c7593 xfs: lift transaction header parsing out of the region assembly a68100c xfs: track transaction header decode state explicitly 24e21ca xfs: track the log item being rebuilt with an explicit pointer d91765d xfs: complete log items explicitly during recovery 1927333 xfs: add log item validation infrastructure to log recovery cacb405 xfs: recover the transaction header as a validated item type 0703ad7 xfs: factor out xlog_recover_validate_ophdr() fc50f95 xfs: validate operation header length and alignment during recovery 3d16b7b xfs: validate the operation header padding during recovery fe25cc9 xfs: validate the operation header flags during recovery c2268a5 xfs: enforce zero length control records during recovery 7ea3fb8 xfs: validate the unmount record during recovery dfd687d xfs: handle zero length continuation op headers during recovery 74f2613 xfs: bound the assembled region size during recovery 1de7ba8 xfs: dump the corrupt region when log item validation fails 3c87e627 xfs: validate fixed size log items during recovery 8a79bbf xfs: validate extent free log items during recovery 1bb8381 xfs: validate and convert extent free intents during recovery 426c695 xfs: validate and convert extent free done items during recovery 425e959 xfs: validate rmap update intents during recovery 2549fef xfs: validate rmap update done items during recovery 80bb17f xfs: validate refcount update intents during recovery d10d3f8 xfs: validate refcount update done items during recovery 6061b3b xfs: validate bmap update intents during recovery aa3c496 xfs: validate bmap update done items during recovery 27521ef xfs: validate mapping exchange intents during recovery 3773819 xfs: validate mapping exchange done items during recovery e3b1f31 xfs: validate dquot items during recovery 96008da xfs: validate attr done items during recovery b21e835 xfs: validate attr intents during recovery e91bfa9 xfs: convert inode items to host format during recovery 2a6df3a xfs: allow xfs_imap() to map without an inode btree lookup c661e55 xfs: validate the inode item format header during recovery 71dd594 xfs: validate the logged inode core during recovery 0a30aae xfs: validate the logged inode forks during recovery e1d4af6 xfs: validate logged local fork contents during recovery 88362e8 xfs: validate logged extent fork contents during recovery 1f1bb06 xfs: allow a long format btree root to be checked without a cursor 7e1242f xfs: validate logged btree fork roots during recovery 34f073c xfs: validate logged metadata btree fork roots during recovery 4a383f7 xfs: validate buffer item region count during recovery 9503237 xfs: validate buffer item regions during recovery cc5eed7 xfs: validate the whole buffer item during recovery f5855c0 xfs: remove redundant buffer checks from recovery replay e0a8bd9 xfs: fail recovery on a buffer type mismatch 44cad35 xfs: validate logged dquot buffers during recovery
Diffstat:
Documentation/filesystems/xfs/index.rst | 1 +- Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst | 514 ++++- fs/xfs/libxfs/xfs_bmap.c | 48 +- fs/xfs/libxfs/xfs_bmap.h | 8 +- fs/xfs/libxfs/xfs_btree.c | 50 +- fs/xfs/libxfs/xfs_btree.h | 4 +- fs/xfs/libxfs/xfs_dquot_buf.c | 64 +- fs/xfs/libxfs/xfs_format.h | 8 +- fs/xfs/libxfs/xfs_ialloc.c | 7 +- fs/xfs/libxfs/xfs_inode_buf.c | 19 +- fs/xfs/libxfs/xfs_inode_buf.h | 2 +- fs/xfs/libxfs/xfs_inode_fork.c | 25 +- fs/xfs/libxfs/xfs_log_format.h | 8 +- fs/xfs/libxfs/xfs_log_recover.h | 64 +- fs/xfs/libxfs/xfs_quota_defs.h | 2 +- fs/xfs/scrub/inode_repair.c | 14 +- fs/xfs/xfs_attr_item.c | 283 +- fs/xfs/xfs_bmap_item.c | 109 +- fs/xfs/xfs_buf_item_recover.c | 349 +- fs/xfs/xfs_dquot_item_recover.c | 118 +- fs/xfs/xfs_exchmaps_item.c | 46 +- fs/xfs/xfs_extfree_item.c | 368 ++- fs/xfs/xfs_icache.h | 2 +- fs/xfs/xfs_icreate_item.c | 18 +- fs/xfs/xfs_inode.h | 43 +- fs/xfs/xfs_inode_item_recover.c | 979 +++++-- fs/xfs/xfs_log.c | 16 +- fs/xfs/xfs_log_recover.c | 764 ++++- fs/xfs/xfs_refcount_item.c | 163 +- fs/xfs/xfs_rmap_item.c | 163 +- 30 files changed, 3410 insertions(+), 849 deletions(-)
PATCH 1/48 xfs: set the unmount record operation header length
The unmount record is written to the log with a hand built operation header passed directly to xlog_write() through xlog_write_one_vec(). The operation header length was never set, so the unmount record is written with a zero length header even though it carries an unmount log format structure. Recovery does not check the unmount record, so the incorrect length goes unnoticed.
Before commit db357078b0423e ("xfs: introduce xlog_write_full()") the operation header length was rewritten correctly by xlog_write(), hence the incorrectly built unmount header went unnoticed. After that commit, xlog_write() never rewrote operation header lengths for regions that fit fully into the iclog. This change exposed the problem with the unmount record header construction.
Set the operation header length when building the unmount record. xlog_write_one_vec() now validates that the operation header length is set correctly for the region being written, and shuts down the filesystem if a malformed record write is detected.
--- fs/xfs/xfs_log.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
fs/xfs/xfs_log.c +16 -0
@@ -811,6 +811,7 @@ xlog_write_one_vec(struct xfs_log_iovec *reg, struct xlog_ticket *ticket) { + struct xlog_op_header *ophdr = reg->i_addr; struct xfs_log_vec lv = { .lv_niovecs = 1, .lv_iovecp = reg,
@@ -818,6 +819,20 @@ xlog_write_one_vec( };LIST_HEAD (lv_chain); +/* +* The caller builds the operation header itself, and xlog_write() no +* longer rewrites the length when the region is written whole. Make sure +* the length the caller recorded in the header matches the region, so a +* bad header is caught here rather than being written to the log. +*/ +if (reg->i_len !=sizeof (*ophdr) +be32_to_cpu (ophdr->oh_len)) { +xfs_alert (log->l_mp, +"%s: region length %u does not match op header length %u" , + __func__, reg->i_len,be32_to_cpu (ophdr->oh_len)); +xlog_force_shutdown (log,SHUTDOWN_CORRUPT_INCORE ); +return -EFSCORRUPTED ; + } +/* account for space used by record data */ ticket->t_curr_res -= lv.lv_bytes;
@@ -842,6 +857,7 @@ xlog_write_unmount_record( .ophdr = { .oh_clientid = XFS_LOG, .oh_tid = cpu_to_be32(ticket->t_tid), + .oh_len = cpu_to_be32(sizeof(struct xfs_unmount_log_format)), .oh_flags = XLOG_UNMOUNT_TRANS, }, .ulf = {
Signature · 1 line
2.55.0
PATCH 2/48 xfs: add log recovery item validation design document
In reply to Dave ChinnerAdd a design document describing a systematic approach to validating log items during journal recovery. Log item data read from the journal cannot be fully trusted - corruption from torn writes, hardware errors or software bugs can produce invalid type codes, wrong region counts, truncated regions or internally inconsistent format structures that the current recovery code largely uses unchecked.
The document describes a three-layer validation scheme (generic region header decode, per-region type-specific validation, and full cross-region item validation), the treatment of the transaction header as a validated region type, and a set of additional safety fixes for the generic region assembly code. It also lays out a phased implementation plan.
Changes:
- Add Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst
- Add the new document to the XFS documentation index
Created-by-AI: Claude Opus 4.8 (1M context)
Prompt: Let's start by converting the existing verification doc to kernel .rst format and moving it to Documentation/filesystems/xfs/ and adding it to the relevant doc index file. The file should be tracked in a new patch.
Refinements:
- Reordered the Phase 1 implementation plan so the ops-struct validation infrastructure (min_regions/max_regions/min_hdr_len fields and the validate_region callback) is added first (Patch 1), followed by the transaction header ops entry that uses it (Patch 2). Downstream patches renumbered accordingly so each patch builds cleanly on its own.
--- Documentation/filesystems/xfs/index.rst | 1 + .../xfs-log-recovery-validation-design.rst | 477 ++++++++++++++++++ 2 files changed, 478 insertions(+) create mode 100644 Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst
Documentation/filesystems/xfs/index.rst +1 -0
@@ -9,6 +9,7 @@ XFS Filesystem Documentation :numbered: xfs-delayed-logging-design + xfs-log-recovery-validation-design xfs-maintainer-entry-profile xfs-self-describing-metadata xfs-online-fsck-design
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst new file +477 -0
477 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 3/48 xfs: lift transaction header parsing out of the region assembly
In reply to Dave ChinnerThe transaction header at the start of a recovered transaction is not a log item, but its parsing is currently tangled into the generic log item region
assembly: xlog_recover_add_to_trans() and xlog_recover_add_to_cont_trans() each carry a special list_empty() branch that decodes the header instead of a log item region.
Lift all of that into a single xlog_recover_init_new_trans() helper called from xlog_recovery_process_trans() before the log item switch. When the item queue is empty the incoming op record is (a fragment of) the transaction header, so it is routed to the new helper; XLOG_WAS_CONT_TRANS distinguishes a continuation fragment from the initial one.
The two former header branches are merged. Their common length validation is done once: a fragment must be at least 4 bytes (all regions are 32 bit aligned, and 4 bytes is the minimum needed to read the magic number) and no larger than the whole header. The magic number is only checked on the initial fragment. The bad-magic warning now reports the observed and expected values. The redundant ASSERT(0) calls on the corruption paths are removed so debug builds exercise the error paths.
This is a behaviour-preserving refactor: the header is still reassembled directly into r_theader and the placeholder item is still added to the queue when the header completes, exactly as before. It removes the header special cases from the region assembly functions so they can be simplified in following patches.
Changes:
- Add xlog_recover_init_new_trans() combining the two header decode paths
- Route header ophdrs to it from xlog_recovery_process_trans() via the empty item queue check, above the log item switch
- Remove the transaction header branches from xlog_recover_add_to_trans() and xlog_recover_add_to_cont_trans()
Created-by-AI: Claude Opus 4.8 (1M context)
Prompt: ok, I think we need to do a bit of refactoring before this new infra patch. We need to lift the empty transaction case up out of add_to_trans and add_to_cont_trans. [Refactor patch 1 of 4: lift transaction header parsing.]
--- fs/xfs/xfs_log_recover.c | 113 +++++++++++++++++++++++++-------------- 1 file changed, 73 insertions(+), 40 deletions(-)
fs/xfs/xfs_log_recover.c +73 -40
@@ -2108,35 +2108,76 @@ xlog_recover_add_item(list_add_tail (&item->ri_list, head); } +/* +* Decode the transaction header at the start of a new transaction. +* +* The transaction header is not a log item. It can be arbitrarily split across +* op records, so it may arrive whole in a single op record, or as an initial +* fragment (cont == false) followed by one or more continuation fragments +* (cont == true). Reassemble it directly into r_theader where the rest of +* recovery expects to find it. +*/ STATIC int -xlog_recover_add_to_cont_trans ( +xlog_recover_init_new_trans (struct xlog *log,struct xlog_recover *trans,char *dp, -int len) +int len, +bool cont) { -struct xlog_recover_item *item; -char *ptr, *old_ptr; -int old_len; +char *ptr;/* -* If the transaction is empty, the header was split across this and the -* previous record. Copy the rest of the header. +* All regions are 32 bit aligned, so the smallest valid fragment is 4 +* bytes; that is also the minimum needed to read the magic number. A +* fragment larger than the whole header is corruption in either case. */ -if (list_empty (&trans->r_itemq)) { -ASSERT (len <=sizeof (struct xfs_trans_header )); -if (len >sizeof (struct xfs_trans_header )) { -xfs_warn (log->l_mp,"%s: bad header length" , __func__); +if (len <4 || len >sizeof (struct xfs_trans_header )) { +xfs_warn (log->l_mp,"%s: bad header length %d" , __func__, len); +return -EFSCORRUPTED ; + } + +if (!cont) { +/* +* Initial fragment. The magic number is at the start of the +* header, which lands at the start of r_theader. +*/ +if (*(uint *)dp !=XFS_TRANS_HEADER_MAGIC ) { +xfs_warn (log->l_mp, +"%s: bad header magic number 0x%x, expected 0x%x" , + __func__, *(uint *)dp,XFS_TRANS_HEADER_MAGIC );return -EFSCORRUPTED ; } - -xlog_recover_add_item (&trans->r_itemq); + ptr = (char *)&trans->r_theader; + }else { +/* continuation fragment completes the tail of the header */ ptr = (char *)&trans->r_theader +sizeof (struct xfs_trans_header ) - len; -memcpy (ptr, dp, len); -return 0 ; } +/* +* Add an item to the transaction once the header is complete: for an +* unsplit header that is this initial fragment (len == full header), +* and for a split header it is the continuation fragment that finishes +* it. +*/ +if (cont || len ==sizeof (struct xfs_trans_header )) +xlog_recover_add_item (&trans->r_itemq); +memcpy (ptr, dp, len); +return 0 ; +} + +STATIC int +xlog_recover_add_to_cont_trans ( +struct xlog *log, +struct xlog_recover *trans, +char *dp, +int len) +{ +struct xlog_recover_item *item; +char *ptr, *old_ptr; +int old_len; +/* take the tail entry */ item = list_entry(trans->r_itemq.prev, struct xlog_recover_item, ri_list);
@@ -2180,31 +2221,6 @@ xlog_recover_add_to_trans(if (!len)return 0 ; -if (list_empty (&trans->r_itemq)) { -/* we need to catch log corruptions here */ -if (*(uint *)dp !=XFS_TRANS_HEADER_MAGIC ) { -xfs_warn (log->l_mp,"%s: bad header magic number" , - __func__); -ASSERT (0 ); -return -EFSCORRUPTED ; - } - -if (len >sizeof (struct xfs_trans_header )) { -xfs_warn (log->l_mp,"%s: bad header length" , __func__); -ASSERT (0 ); -return -EFSCORRUPTED ; - } - -/* -* The transaction header can be arbitrarily split across op -* records. If we don't have the whole thing here, copy what we -* do have and handle the rest in the next record. -*/ -if (len ==sizeof (struct xfs_trans_header )) -xlog_recover_add_item (&trans->r_itemq); -memcpy (&trans->r_theader, dp, len); -return 0 ; - } ptr =xlog_kvmalloc (len);memcpy (ptr, dp, len);
@@ -2302,6 +2318,21 @@ xlog_recovery_process_trans(if (flags &XLOG_WAS_CONT_TRANS ) flags &= ~XLOG_CONTINUE_TRANS ; +/* +* An empty item queue means we are still decoding the transaction +* header at the start of the transaction rather than a log item. The +* header can arrive whole (flags == 0), as an initial fragment +* (XLOG_CONTINUE_TRANS) or as a continuation fragment +* (XLOG_WAS_CONT_TRANS); XLOG_WAS_CONT_TRANS distinguishes the +* continuation. +*/ +if (list_empty (&trans->r_itemq) && + (!flags || (flags & (XLOG_CONTINUE_TRANS |XLOG_WAS_CONT_TRANS )))) { + error =xlog_recover_init_new_trans (log, trans, dp, len, + flags &XLOG_WAS_CONT_TRANS ); + goto out_error; + } +/* * Callees must not free the trans structure. We'll decide if we need to * free it or not based on the operation being done and it's result.
@@ -2335,6 +2366,8 @@ xlog_recovery_process_trans( error = -EFSCORRUPTED ;break ; } + +out_error:if (error || freeit)xlog_recover_free_trans (trans);return error;
Signature · 1 line
2.55.0
PATCH 4/48 xfs: track transaction header decode state explicitly
In reply to Dave Chinnerxlog_recovery_process_trans() decides whether an incoming op record is (a fragment of) the transaction header or a log item region by checking whether the transaction's item queue is empty. That is an indirect signal: it happens to be true only while the header is being assembled because no item is added until the header is complete.
Replace it with an explicit bool r_hdr_decoded in struct xlog_recover. It is set once the transaction header has been fully assembled in xlog_recover_init_new_trans() - the same point at which the placeholder item is added - and the routing check in xlog_recovery_process_trans() now tests !r_hdr_decoded instead of list_empty(&trans->r_itemq).
r_hdr_decoded is zero-initialised to false when the transaction is allocated, so no explicit initialisation is required. No functional change.
Changes:
- Add r_hdr_decoded to struct xlog_recover
- Set it when the transaction header is fully decoded in xlog_recover_init_new_trans()
- Use !r_hdr_decoded rather than an empty item queue to detect header ophdrs in xlog_recovery_process_trans()
Created-by-AI: Claude Opus 4.8 (1M context)
Prompt: The addition of r_hdr_decoded to replace the list_empty() check is the second patch. [Refactor patch 2 of 4.]
--- fs/xfs/libxfs/xfs_log_recover.h | 1 + fs/xfs/xfs_log_recover.c | 24 ++++++++++++++---------- 2 files changed, 15 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_log_recover.h +1 -0
@@ -115,6 +115,7 @@ struct xlog_recover {int r_state;/* not needed */ xfs_lsn_t r_lsn;/* xact lsn */ struct list_head r_itemq;/* q for items */ +bool r_hdr_decoded;/* trans header fully decoded */ };#define ITEM_TYPE (i) (*(unsigned short *)(i)->ri_buf[0].iov_base)
fs/xfs/xfs_log_recover.c +14 -10
@@ -2155,15 +2155,19 @@ xlog_recover_init_new_trans(sizeof (struct xfs_trans_header ) - len; } +memcpy (ptr, dp, len); +/* -* Add an item to the transaction once the header is complete: for an -* unsplit header that is this initial fragment (len == full header), +* The header is complete once the full struct has been assembled: for +* an unsplit header that is this initial fragment (len == full header), * and for a split header it is the continuation fragment that finishes -* it. +* it. Mark it decoded and add the item that subsequent log item regions +* will be decoded into. */ -if (cont || len ==sizeof (struct xfs_trans_header )) +if (cont || len ==sizeof (struct xfs_trans_header )) { + trans->r_hdr_decoded = true;xlog_recover_add_item (&trans->r_itemq); -memcpy (ptr, dp, len); + }return 0 ; }
@@ -2319,14 +2323,14 @@ xlog_recovery_process_trans( flags &= ~XLOG_CONTINUE_TRANS ;/* -* An empty item queue means we are still decoding the transaction -* header at the start of the transaction rather than a log item. The -* header can arrive whole (flags == 0), as an initial fragment -* (XLOG_CONTINUE_TRANS) or as a continuation fragment +* Until the transaction header has been fully decoded we are still +* assembling it at the start of the transaction rather than decoding a +* log item. The header can arrive whole (flags == 0), as an initial +* fragment (XLOG_CONTINUE_TRANS) or as a continuation fragment * (XLOG_WAS_CONT_TRANS); XLOG_WAS_CONT_TRANS distinguishes the * continuation. */ -if (list_empty (&trans->r_itemq) && +if (!trans->r_hdr_decoded && (!flags || (flags & (XLOG_CONTINUE_TRANS |XLOG_WAS_CONT_TRANS )))) { error =xlog_recover_init_new_trans (log, trans, dp, len, flags &XLOG_WAS_CONT_TRANS );
Signature · 1 line
2.55.0
PATCH 5/48 xfs: track the log item being rebuilt with an explicit pointer
In reply to Dave ChinnerThe log item currently being reassembled from journal op records is found by taking the tail entry of the transaction's item queue with list_entry(trans->r_itemq.prev, ...). This is repeated in several places and relies on the item being rebuilt always being the last one on the queue.
Track it explicitly instead. Add a struct xlog_recover_item *r_cur_item to struct xlog_recover, have xlog_recover_add_item() record and return the item it creates, and use trans->r_cur_item wherever the code previously fetched the tail entry.
r_cur_item is zero-initialised to NULL when the transaction is allocated and is set when the first item is added as the transaction header completes, so it is always valid by the time log item regions are decoded. No functional change.
Changes:
- Add r_cur_item to struct xlog_recover
- xlog_recover_add_item() now takes the trans, sets r_cur_item and returns the new item
- Replace list_entry(r_itemq.prev) lookups in xlog_recover_add_to_trans() and xlog_recover_add_to_cont_trans() with trans->r_cur_item
Created-by-AI: Claude Opus 4.8 (1M context)
Prompt: The third patch is adding r_cur_item to replace the list_entry(.prev) mechanism for tracking the current item. [Refactor patch 3 of 4.] Refinement: have xlog_recover_add_item() return the new item.
--- fs/xfs/libxfs/xfs_log_recover.h | 1 + fs/xfs/xfs_log_recover.c | 29 +++++++++++++---------------- 2 files changed, 14 insertions(+), 16 deletions(-)
fs/xfs/libxfs/xfs_log_recover.h +1 -0
@@ -116,6 +116,7 @@ struct xlog_recover {xfs_lsn_t r_lsn;/* xact lsn */ struct list_head r_itemq;/* q for items */ bool r_hdr_decoded;/* trans header fully decoded */ +struct xlog_recover_item *r_cur_item;/* item currently being rebuilt */ };#define ITEM_TYPE (i) (*(unsigned short *)(i)->ri_buf[0].iov_base)
fs/xfs/xfs_log_recover.c +13 -16
@@ -2097,15 +2097,19 @@ xlog_recover_commit_trans(return error; } -STATIC void +STATIC struct xlog_recover_item *xlog_recover_add_item ( -struct list_head *head) +struct xlog_recover *trans) {struct xlog_recover_item *item; item = kzalloc_obj(struct xlog_recover_item, GFP_KERNEL | __GFP_NOFAIL);INIT_LIST_HEAD (&item->ri_list); -list_add_tail (&item->ri_list, head); +list_add_tail (&item->ri_list, &trans->r_itemq); + +/* this is now the item being rebuilt */ + trans->r_cur_item = item; +return item; }/*
@@ -2166,7 +2170,7 @@ xlog_recover_init_new_trans(*/ if (cont || len ==sizeof (struct xfs_trans_header )) { trans->r_hdr_decoded = true; -xlog_recover_add_item (&trans->r_itemq); +xlog_recover_add_item (trans); }return 0 ; }
@@ -2178,14 +2182,10 @@ xlog_recover_add_to_cont_trans( char *dp, int len) { -struct xlog_recover_item *item; +struct xlog_recover_item *item = trans->r_cur_item;char *ptr, *old_ptr;int old_len; -/* take the tail entry */ - item = list_entry(trans->r_itemq.prev, struct xlog_recover_item, - ri_list); - old_ptr = item->ri_buf[item->ri_cnt-1 ].iov_base; old_len = item->ri_buf[item->ri_cnt-1 ].iov_len;
@@ -2230,15 +2230,12 @@ xlog_recover_add_to_trans(memcpy (ptr, dp, len); in_f = (struct xfs_inode_log_format *)ptr; -/* take the tail entry */ - item = list_entry(trans->r_itemq.prev, struct xlog_recover_item, - ri_list); +/* the item currently being rebuilt */ + item = trans->r_cur_item;if (item->ri_total !=0 && item->ri_total == item->ri_cnt) { -/* tail item is in use, get a new one */ -xlog_recover_add_item (&trans->r_itemq); - item =list_entry (trans->r_itemq.prev, - struct xlog_recover_item, ri_list); +/* current item is full, start a new one */ + item =xlog_recover_add_item (trans); }if (item->ri_total ==0 ) {/* first region to be added */
Signature · 1 line
2.55.0
PATCH 6/48 xfs: complete log items explicitly during recovery
In reply to Dave ChinnerLog item completion during recovery was detected lazily: an item was added to the transaction item queue as soon as its first region arrived, and the code only noticed the item was full when the next region turned up and found the current item's region count had reached its total.
Rework this into explicit item start and completion. An item is now created off the item queue (xlog_recover_add_item() no longer links it) and tracked only via r_cur_item while it is being rebuilt. When its final region has been fully decoded the item is moved onto the item queue and r_cur_item is cleared by xlog_recover_complete_item(); the next region decoded then finds no current item and starts a new one.
Region completion is driven by a new region_complete argument, derived from XLOG_CONTINUE_TRANS in xlog_recovery_process_trans() before the container flags are masked. A region (and hence potentially its item) is only complete once it is not a partial continuation fragment. This replaces the previous implicit "item is done because its region count matched" detection with explicit last-region state, and lets both xlog_recover_add_to_trans() and xlog_recover_add_to_cont_trans() complete an item at the point its final region finishes.
The transaction header is a single region, so it is now marked decoded when its region completes rather than via a length comparison, and it no longer creates a placeholder item for the first log item to reuse. As a result the first log item region always starts a fresh item, so the first-region setup (reading ilf_size and allocating ri_buf) moves entirely into the new-item path and the ri_total == 0 special case goes away.
Because an item being rebuilt is no longer on the item queue, xlog_recover_free_trans() also frees a non-NULL r_cur_item. The per-item teardown is factored into xlog_recover_free_item() for reuse.
Changes:
- xlog_recover_add_item() no longer links the item to the queue
- Add xlog_recover_complete_item() to push a finished item and clear r_cur_item
- Plumb a region_complete flag into init_new_trans/add_to_trans/ add_to_cont_trans and complete items when their final region finishes
- Mark the transaction header decoded on region completion; drop its placeholder item and the first-region ri_total == 0 handling
- Factor xlog_recover_free_item() and free a dangling r_cur_item in xlog_recover_free_trans()
Created-by-AI: Claude Opus 4.8 (1M context)
Prompt: The fourth patch is reworking add_to_trans/cont_trans to change the way the items are added to the list (i.e. from 'add next at end of current' to 'add at end of current, clear r_cur_item, next sees r_cur_item == NULL so creates new'). [Refactor patch 4 of 4.]
Refinements:
- Moved the first-region ilf_size/ri_buf setup entirely into the no-current-item branch and removed the now-impossible ri_total == 0 case.
- Drove header and item completion off an explicit region_complete flag rather than length/region-count heuristics.
--- fs/xfs/xfs_log_recover.c | 128 +++++++++++++++++++++++++++++---------- 1 file changed, 96 insertions(+), 32 deletions(-)
fs/xfs/xfs_log_recover.c +96 -32
@@ -2105,13 +2105,31 @@ xlog_recover_add_item( item = kzalloc_obj(struct xlog_recover_item, GFP_KERNEL | __GFP_NOFAIL);INIT_LIST_HEAD (&item->ri_list); -list_add_tail (&item->ri_list, &trans->r_itemq); -/* this is now the item being rebuilt */ +/* +* The item is not placed on the transaction's item queue until it has +* been fully decoded (see xlog_recover_complete_item()). Until then it +* is tracked as the item currently being rebuilt. +*/ trans->r_cur_item = item;return item; } +/* +* A log item has been fully decoded from the journal. Move it onto the +* transaction's item queue and clear the current item so the next region +* decoded starts a new item. +*/ +static void +xlog_recover_complete_item ( +struct xlog_recover *trans) +{ +struct xlog_recover_item *item = trans->r_cur_item; + +list_add_tail (&item->ri_list, &trans->r_itemq); + trans->r_cur_item =NULL ; +} +/* * Decode the transaction header at the start of a new transaction. *
@@ -2127,7 +2145,8 @@ xlog_recover_init_new_trans(struct xlog_recover *trans, char *dp, int len, - bool cont) + bool cont, + bool region_complete) { char *ptr;
@@ -2162,16 +2181,13 @@ xlog_recover_init_new_trans(memcpy (ptr, dp, len);/* -* The header is complete once the full struct has been assembled: for -* an unsplit header that is this initial fragment (len == full header), -* and for a split header it is the continuation fragment that finishes -* it. Mark it decoded and add the item that subsequent log item regions -* will be decoded into. +* The transaction header is a single region, so it is fully decoded +* once the region is complete. The header is not a log item, so nothing +* is added to the item queue here; the first log item region decoded +* after this will start the first item. */ -if (cont || len ==sizeof (struct xfs_trans_header )) { +if (region_complete) trans->r_hdr_decoded = true; -xlog_recover_add_item (trans); - }return 0 ; }
@@ -2180,7 +2196,8 @@ xlog_recover_add_to_cont_trans(struct xlog *log, struct xlog_recover *trans, char *dp, - int len) + int len, + bool region_complete) { struct xlog_recover_item *item = trans->r_cur_item;char *ptr, *old_ptr;
@@ -2196,6 +2213,14 @@ xlog_recover_add_to_cont_trans( item->ri_buf[item->ri_cnt-1 ].iov_len += len; item->ri_buf[item->ri_cnt-1 ].iov_base = ptr;trace_xfs_log_recover_item_add_cont (log, trans, item,0 ); + +/* +* The continued region is only complete once its final fragment has +* been appended. When that finishes the item's last region, the item is +* complete. +*/ +if (region_complete && item->ri_cnt == item->ri_total) +xlog_recover_complete_item (trans);return 0 ; }
@@ -2217,9 +2242,9 @@ xlog_recover_add_to_trans(struct xlog *log, struct xlog_recover *trans, char *dp, - int len) + int len, + bool region_complete) { - struct xfs_inode_log_format *in_f; /* any will do */ struct xlog_recover_item *item;char *ptr;
@@ -2228,17 +2253,17 @@ xlog_recover_add_to_trans( ptr =xlog_kvmalloc (len);memcpy (ptr, dp, len); - in_f = (struct xfs_inode_log_format *)ptr; -/* the item currently being rebuilt */ +/* +* If there is no item currently being rebuilt, this region is the first +* region of a new item. Start the item and use its first region to +* determine how many regions it has. +*/ item = trans->r_cur_item; -if (item->ri_total !=0 && - item->ri_total == item->ri_cnt) { -/* current item is full, start a new one */ - item =xlog_recover_add_item (trans); - } +if (!item) { +struct xfs_inode_log_format *in_f =/* any will do */ + (struct xfs_inode_log_format *)ptr; -if (item->ri_total ==0 ) {/* first region to be added */ if (in_f->ilf_size ==0 || in_f->ilf_size >XLOG_MAX_REGIONS_IN_ITEM ) { xfs_warn(log->l_mp,
@@ -2249,6 +2274,7 @@ xlog_recover_add_to_trans(return -EFSCORRUPTED ; } + item =xlog_recover_add_item (trans); item->ri_total = in_f->ilf_size; item->ri_buf =kzalloc_objs (*item->ri_buf, item->ri_total,GFP_KERNEL | __GFP_NOFAIL);
@@ -2268,6 +2294,14 @@ xlog_recover_add_to_trans( item->ri_buf[item->ri_cnt].iov_len = len; item->ri_cnt++;trace_xfs_log_recover_item_add (log, trans, item,0 ); + +/* +* The item is complete once its final region has been fully decoded. If +* this region continues into the next op record it is not complete yet, +* so the item cannot be either. +*/ +if (region_complete && item->ri_cnt == item->ri_total) +xlog_recover_complete_item (trans);return 0 ; }
@@ -2276,24 +2310,40 @@ xlog_recover_add_to_trans(* * Remember that EFIs, EFDs, and IUNLINKs are handled later. */ +static void +xlog_recover_free_item ( +struct xlog_recover_item *item) +{ +int i; + +/* Free the regions in the item. */ +for (i =0 ; i < item->ri_cnt; i++) +kvfree (item->ri_buf[i].iov_base); +/* Free the item itself */ +kfree (item->ri_buf); +kfree (item); +} + STATIC voidxlog_recover_free_trans (struct xlog_recover *trans) {struct xlog_recover_item *item, *n; -int i;hlist_del_init (&trans->r_list);list_for_each_entry_safe (item, n, &trans->r_itemq, ri_list) { -/* Free the regions in the item. */ list_del (&item->ri_list); -for (i =0 ; i < item->ri_cnt; i++) -kvfree (item->ri_buf[i].iov_base); -/* Free the item itself */ -kfree (item->ri_buf); -kfree (item); +xlog_recover_free_item (item); } + +/* +* An item still being rebuilt has not been added to the item queue yet, +* so free it separately. +*/ +if (trans->r_cur_item) +xlog_recover_free_item (trans->r_cur_item); +/* Free the transaction recover structure */ kfree (trans); }
@@ -2313,6 +2363,17 @@ xlog_recovery_process_trans( {int error =0 ;bool freeit = false; +bool region_complete; + +/* +* If XLOG_CONTINUE_TRANS is set the current region is only a partial +* fragment that continues into the next op record, so it cannot be +* completed yet. The region is complete once this flag is clear, which +* is the case for both an unsplit region and the final (XLOG_END_TRANS) +* fragment of a split region. Capture this before the flags are masked +* below. +*/ + region_complete = !(flags &XLOG_CONTINUE_TRANS );/* mask off ophdr transaction container flags */ flags &= ~XLOG_END_TRANS ;
@@ -2330,7 +2391,8 @@ xlog_recovery_process_trans(if (!trans->r_hdr_decoded && (!flags || (flags & (XLOG_CONTINUE_TRANS |XLOG_WAS_CONT_TRANS )))) { error =xlog_recover_init_new_trans (log, trans, dp, len, - flags &XLOG_WAS_CONT_TRANS ); + flags &XLOG_WAS_CONT_TRANS , + region_complete); goto out_error; }
@@ -2342,10 +2404,12 @@ xlog_recovery_process_trans(/* expected flag values */ case 0 :case XLOG_CONTINUE_TRANS : - error =xlog_recover_add_to_trans (log, trans, dp, len); + error =xlog_recover_add_to_trans (log, trans, dp, len, + region_complete);break ;case XLOG_WAS_CONT_TRANS : - error =xlog_recover_add_to_cont_trans (log, trans, dp, len); + error =xlog_recover_add_to_cont_trans (log, trans, dp, len, + region_complete);break ;case XLOG_COMMIT_TRANS : error = xlog_recover_commit_trans(log, trans, pass,
Signature · 1 line
2.55.0
PATCH 7/48 xfs: add log item validation infrastructure to log recovery
In reply to Dave ChinnerLog recovery decodes log items straight out of the journal and trusts the region layout it finds there. Corrupt or malicious journal contents can therefore drive recovery with bogus type codes, region counts and header lengths before any validation happens. Add the infrastructure that later patches will use to validate each item as it is decoded, and wire the generic validation call sites into the region assembly path so per-type patches only need to supply the checks.
struct xlog_recover_item_ops gains a min_hdr_len field (the minimum valid length of the format header in ri_buf[0]) and four optional callbacks:validate_nregions() returns the number of regions to assemble for the item; validate_region() checks a single region once it has been fully assembled; validate_item() checks the whole item once all of its regions are present; and complete() is a per-type completion hook run once the item is fully assembled and validated.
The region count is a per-type method rather than a static bound because not all recovered types describe their regions with the same "type/size" format header that log items use. Each such type needs to extract and validate its own region count. Types that do use the common layout supply no method and fall back to the generic xlog_recover_nregions() helper, which accepts any 0 < count <= XLOG_MAX_REGIONS_IN_ITEM.
Item completion is driven through a shared path. xlog_recover_complete_region() runs the min_hdr_len check on the format header and the type's validate_region() callback once a region is complete, then completes the item via xlog_recover_complete_item() - which runs validate_item() - once the final region has been assembled. A region is complete when it is not a partial continuation fragment, tracked with a region_complete flag derived from XLOG_CONTINUE_TRANS in xlog_recovery_process_trans() and passed into both xlog_recover_add_to_trans() and xlog_recover_add_to_cont_trans(). Running the header and region checks only at region-completion time means a region split across op records is not rejected while it is still being assembled.
A completed item is queued on the transaction for replay by default. The complete() hook lets a type do type-specific work at completion first, and it may consume the item there and then rather than have it queued for later
recovery: returning XLOG_RECOVER_ITEM_CONSUMED frees the item instead of queuing it, while returning 0 falls through to the default queuing. This supports items that need finalising during decode, including pseudo-items that are consumed rather than replayed. No type supplies the hook yet.
The ops lookup, unknown-type rejection and region-count validation move into xlog_recover_add_item() as part of this, so the lookup is no longer done in xlog_recover_reorder_trans() and xlog_find_item_ops() now takes the type code directly. No item type supplies the new field or callbacks yet, so the generic helper is always used and there is no functional change beyond the unknown-type rejection moving earlier in decode.
--- .../xfs-log-recovery-validation-design.rst | 124 +++++---- fs/xfs/libxfs/xfs_log_recover.h | 53 ++++ fs/xfs/xfs_log_recover.c | 238 +++++++++++++----- 3 files changed, 298 insertions(+), 117 deletions(-)
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst +73 -51
@@ -70,8 +70,9 @@ c) Look up the item ops via ``xlog_find_item_ops()`` d) If the type is unknown, reject immediately with ``-EFSCORRUPTED`` e) Store ``item->ri_ops`` at this point (currently done in ``reorder_trans``) -f) Validate ``ilf_size`` against ``ops->min_regions`` and - ``ops->max_regions`` +f) Validate the region count via ``ops->validate_nregions()`` (or the + generic ``xlog_recover_nregions()`` helper if the type does not supply + one), and use its return value as the number of regions to assemble g) Validate ``len >= ops->min_hdr_len`` (the minimum format header size) Transaction header as a validated type
@@ -110,36 +111,46 @@ ops entry by having the generic code recognise it as a special case at step (b) and apply its fixed constraints directly. Either approach works; the ops entry is cleaner but the special case is simpler. -New fields in ``struct xlog_recover_item_ops``:: +New members in ``struct xlog_recover_item_ops``:: - uint16_t min_regions; /* minimum valid ri_total */ - uint16_t max_regions; /* maximum valid ri_total */ uint16_t min_hdr_len; /* minimum ri_buf[0].iov_len */ - -These are compile-time constants per item type. Examples: - -=========== =========== =========== ============================== -Item Type min_regions max_regions min_hdr_len -=========== =========== =========== ============================== -TRANS_HDR 1 1 sizeof(xfs_trans_header) -BUF 2 XLOG_MAX.. sizeof(xfs_buf_log_format) -INODE 2 4 sizeof(xfs_inode_log_format) -DQUOT 2 2 sizeof(xfs_dq_logformat) -EFI 1 1 sizeof(xfs_efi_log_format) -EFD 1 1 sizeof(xfs_efd_log_format) -RUI 1 1 sizeof(xfs_rui_log_format) -RUD 1 1 sizeof(xfs_rud_log_format) -CUI 1 1 sizeof(xfs_cui_log_format) -CUD 1 1 sizeof(xfs_cud_log_format) -BUI 1 1 sizeof(xfs_bui_log_format) -BUD 1 1 sizeof(xfs_bud_log_format) -ATTRI 2 5 sizeof(xfs_attri_log_format) -ATTRD 1 1 sizeof(xfs_attrd_log_format) -XMI 1 1 sizeof(xfs_xmi_log_format) -XMD 1 1 sizeof(xfs_xmd_log_format) -ICREATE 1 1 sizeof(xfs_icreate_log) -QUOTAOFF 1 1 sizeof(xfs_qoff_logformat) -=========== =========== =========== ============================== + int (*validate_nregions)(struct xlog *log, uint16_t nregions); + +``min_hdr_len`` is a compile-time constant per item type. The region count +is validated by the ``validate_nregions()`` method rather than a static +min/max pair: it is given the raw count read from the journal and returns +the number of regions to assemble, or a negative error. This is a method, +not a bound, because not all recovered types describe their regions with the +same "type/size" format header that log items use, so each such type must +extract and validate its own count. Types that do use the common layout +supply no method and fall back to the generic ``xlog_recover_nregions()`` +helper, which accepts any ``0 < count <= XLOG_MAX_REGIONS_IN_ITEM``. + +Examples of the region count each type should accept and the minimum header +length: + +=========== ===================== ============================== +Item Type regions min_hdr_len +=========== ===================== ============================== +TRANS_HDR 1 (magic-derived) sizeof(xfs_trans_header) +BUF 2 .. XLOG_MAX.. sizeof(xfs_buf_log_format) +INODE 2 .. 4 sizeof(xfs_inode_log_format) +DQUOT 2 sizeof(xfs_dq_logformat) +EFI 1 sizeof(xfs_efi_log_format) +EFD 1 sizeof(xfs_efd_log_format) +RUI 1 sizeof(xfs_rui_log_format) +RUD 1 sizeof(xfs_rud_log_format) +CUI 1 sizeof(xfs_cui_log_format) +CUD 1 sizeof(xfs_cud_log_format) +BUI 1 sizeof(xfs_bui_log_format) +BUD 1 sizeof(xfs_bud_log_format) +ATTRI 2 .. 5 sizeof(xfs_attri_log_format) +ATTRD 1 sizeof(xfs_attrd_log_format) +XMI 1 sizeof(xfs_xmi_log_format) +XMD 1 sizeof(xfs_xmd_log_format) +ICREATE 1 sizeof(xfs_icreate_log) +QUOTAOFF 1 sizeof(xfs_qoff_logformat) +=========== ===================== ============================== (RT variants same as their non-RT counterparts.)
@@ -355,8 +366,9 @@ region assembly code: - Check ``len >= 4`` before reading the type and size fields (except for the zero-length transaction header continuation case) - - After looking up ops, validate ``ilf_size`` against ops constraints - before using it for the ``kzalloc_objs`` allocation + - After looking up ops, validate the region count via + ``ops->validate_nregions()`` (or the generic helper) and use its + return value for the ``kzalloc_objs`` allocation 3. ``xlog_recover_process_data()``:
@@ -372,25 +384,34 @@ region assembly code: Implementation Plan =================== -The infrastructure fields and callbacks in ``struct -xlog_recover_item_ops`` (the ``min_regions``/``max_regions``/``min_hdr_len`` -fields and the ``validate_region`` callback) are introduced first, before -the transaction header ops entry that uses them. Later patches populate -those fields and callbacks for the remaining item types and wire up the -generic call sites. Each patch builds cleanly and is independently +The infrastructure in ``struct xlog_recover_item_ops`` (the ``min_hdr_len`` +field and the ``validate_nregions``, ``validate_region``, ``validate_item`` +and ``complete`` callbacks) is introduced first, along with the generic +region and item completion path that calls them, before the transaction +header ops entry that uses them. Later patches populate those callbacks for +the remaining item types. Each patch builds cleanly and is independently testable. Phase 1: Generic infrastructure, early ops lookup, transaction header --------------------------------------------------------------------- Patch 1: Add validation infrastructure to the ops struct - - Add the ``min_regions``/``max_regions``/``min_hdr_len`` fields to - ``struct xlog_recover_item_ops`` - - Add the ``validate_region`` callback to ``struct - xlog_recover_item_ops`` - - No behaviour change yet: the fields are zero and the callback is - NULL for all existing item types; nothing reads them until later - patches + - Add the ``min_hdr_len`` field and the ``validate_nregions``, + ``validate_region``, ``validate_item`` and ``complete`` callbacks to + ``struct xlog_recover_item_ops`` (plus the ``XLOG_RECOVER_ITEM_CONSUMED`` + return value for ``complete``) + - Fold the start of a new item into ``xlog_recover_add_item()``, which + takes the type and region count, looks up the ops, rejects unknown + types, validates the region count (via ``validate_nregions`` or the + generic ``xlog_recover_nregions()`` helper) and returns the item or an + ERR_PTR(); remove the ops lookup from ``reorder_trans`` + - Add the generic region/item completion path + (``xlog_recover_complete_region()``/``xlog_recover_complete_item()``) + that runs ``min_hdr_len``, ``validate_region``, ``validate_item`` and + the ``complete`` hook, driven from both region assembly functions + - No item type supplies the new field or callbacks yet, so the generic + helper is always used and there is no behaviour change beyond the + unknown-type rejection moving earlier Patch 2: Treat the transaction header as a validated region type - Add an ops entry for the transaction header keyed on the low 16 bits
@@ -414,12 +435,13 @@ Patch 3: Move item ops lookup to add_to_trans (first region decode) - Remove the ops lookup from ``xlog_recover_reorder_trans()`` (it becomes a simple NULL check / assertion) -Patch 4: Populate min_regions/max_regions/min_hdr_len for all item types - - The three fields were added to ``struct xlog_recover_item_ops`` in - Patch 1; populate them for all remaining item types - - Add generic checks in ``add_to_trans`` after ops lookup: - ``ilf_size >= ops->min_regions && ilf_size <= ops->max_regions``, - ``len >= ops->min_hdr_len`` (for non-continuation first regions) +Patch 4: Populate validate_nregions/min_hdr_len for all item types + - The ``min_hdr_len`` field and ``validate_nregions`` callback were added + to ``struct xlog_recover_item_ops`` in Patch 1; populate them for all + remaining item types (each ``validate_nregions`` enforces the type's + region-count range and returns the count) + - The generic ``min_hdr_len`` check already runs at region-completion + time from Patch 1 Patch 5: Fix generic safety issues - ``add_to_cont_trans``: check ``ri_cnt > 0`` and ``ri_buf != NULL``
fs/xfs/libxfs/xfs_log_recover.h +53 -0
73 lines not on this page · raw message
fs/xfs/xfs_log_recover.c +172 -66
322 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 8/48 xfs: recover the transaction header as a validated item type
In reply to Dave ChinnerThe transaction header that starts each recovered transaction was decoded by a bespoke path: xlog_recovery_process_trans() routed the first op records of a transaction to xlog_recover_init_new_trans() based on an r_hdr_decoded flag, which reassembled the header into r_theader with its own inline magic and length checks. This is a second, parallel copy of the region assembly and validation logic that the generic item path already provides, and it is not subject to the item validation infrastructure.
Decode the header through the generic item machinery instead. The header is laid out so that the first two 16 bit words at the front of its single region
- which xlog_recover_add_item() reads as the item type and region count - overlap the 32 bit th_magic field. Register an xlog_recover_item_ops keyed on the magic so the header is assembled, validated and completed exactly like any other item, and consumed via its ->complete() method (copied into the transaction and freed rather than queued for replay).
The XFS log is written in host byte order, so which half of th_magic is read as the type depends on the endianness of the host that wrote the log. Two ops entries are registered, one for each order; on any given host only one of the two type codes is matched. The type-count method validates that the other half of the magic is present and returns a region count of 1, the region validation checks the full magic and exact header length once assembled, and min_hdr_len rejects a short header.
With the header handled as an item, xlog_recover_init_new_trans(), the r_hdr_decoded flag and the special-case dispatch in xlog_recovery_process_trans() are all removed; a header op record now falls through to xlog_recover_add_to_trans()/add_to_cont_trans() like any other region.
--- .../xfs-log-recovery-validation-design.rst | 89 ++++---- fs/xfs/libxfs/xfs_log_recover.h | 1 - fs/xfs/xfs_log_recover.c | 199 +++++++++++------- 3 files changed, 173 insertions(+), 116 deletions(-)
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst +52 -37
@@ -81,35 +81,42 @@ Transaction header as a validated type The transaction header (``struct xfs_trans_header``) is currently decoded inline in ``add_to_trans`` with bespoke magic number and length checks, and its continuation is handled separately in ``add_to_cont_trans``. Instead, -treat it as a regular validated type within the same framework: +treat it as a regular validated type within the same framework by giving it +an ``xlog_recover_item_ops`` entry: -- Add a pseudo item type (e.g. ``XFS_LI_TRANS_HDR`` or use the magic - number ``XFS_TRANS_HEADER_MAGIC`` as the type code) with its own - ``xlog_recover_item_ops`` entry. -- ``min_regions = 1``, ``max_regions = 1`` - ``min_hdr_len = sizeof(struct xfs_trans_header)`` +- ``validate_nregions`` always returns a region count of 1 (see the + endianness note below) - ``validate_region`` checks ``iov_len == sizeof(struct xfs_trans_header)`` and the magic number -- No ``commit_pass1``/``commit_pass2`` callbacks — after validation, the - decoded transaction header is copied to ``trans->r_theader`` as before - -This eliminates the special-case parsing in ``add_to_trans`` and -``add_to_cont_trans`` for the transaction header. The zero-length first -fragment case is handled uniformly: it arrives as a continuation -(``oh_len == 0``, ``XLOG_CONTINUE_TRANS`` set), and the continuation -infrastructure assembles the complete region before ``validate_region`` -checks it. - -The ``ilf_size`` field of the transaction header format is a bit different -— ``xfs_trans_header`` uses ``th_num_items`` rather than the generic -``ilf_size`` at offset 2. Since the transaction header always has exactly -1 region, we don't need to read ``ilf_size`` from it. The -``ops->min_regions == ops->max_regions == 1`` is sufficient. - -Alternatively, the transaction header could be handled without a full -ops entry by having the generic code recognise it as a special case -at step (b) and apply its fixed constraints directly. Either approach -works; the ops entry is cleaner but the special case is simpler. +- ``complete`` copies the decoded header to ``trans->r_theader`` and + returns ``XLOG_RECOVER_ITEM_CONSUMED`` so it is freed rather than queued + for replay — the header is not a log item and has no + ``commit_pass1``/``commit_pass2`` handler + +The header has no format header of the ``type/size`` shape: its first 32 +bits are ``th_magic``. When ``add_to_trans`` reads the first two 16 bit +words as the type and region count, they land on the two halves of the +magic. The type-half is used as the ops key, and ``validate_nregions`` +checks that the count-half holds the other half of the magic before +returning 1 (the header always has a single region). + +Because the XFS log is written in host byte order, which half of the magic +is read as the type depends on the endianness of the host that wrote the +log. Register two ops entries, one for each order (``item_type`` of +``0x414e``/``0x5452``, with the complementary half checked by +``validate_nregions``); on any given host only one is ever matched, and +neither type code collides with an ``XFS_LI_*`` value. + +Routing the header through the generic assembly removes the special-case +parsing in ``add_to_trans`` and ``add_to_cont_trans``, and lets +``xlog_recover_init_new_trans()``, the ``r_hdr_decoded`` flag and the +header dispatch branch in ``xlog_recovery_process_trans()`` be deleted — a +header op record now falls through to the generic region assembly like any +other. The zero-length first fragment case is handled uniformly: it arrives +as a continuation (``oh_len == 0``, ``XLOG_CONTINUE_TRANS`` set), and the +continuation infrastructure assembles the complete region before +``validate_region`` checks it. New members in ``struct xlog_recover_item_ops``::
@@ -414,19 +421,20 @@ Patch 1: Add validation infrastructure to the ops struct unknown-type rejection moving earlier Patch 2: Treat the transaction header as a validated region type - - Add an ops entry for the transaction header keyed on the low 16 bits - of ``XFS_TRANS_HEADER_MAGIC``, with ``min_regions = 1``, - ``max_regions = 1``, ``min_hdr_len = sizeof(struct xfs_trans_header)`` - and a ``validate_region`` that checks ``iov_len`` and the magic - number + - Add two ops entries for the transaction header, keyed on the two + halves of ``XFS_TRANS_HEADER_MAGIC`` (one per host byte order), each + with ``min_hdr_len = sizeof(struct xfs_trans_header)``, a + ``validate_nregions`` that checks the other half of the magic and + returns 1, a ``validate_region`` that checks ``iov_len`` and the magic + number, and a ``complete`` that copies the header to + ``trans->r_theader`` and returns ``XLOG_RECOVER_ITEM_CONSUMED`` - Remove the bespoke transaction header parsing from ``add_to_trans`` and ``add_to_cont_trans``; route through the generic region assembly - - Handle the zero-length first fragment case (``len == 0`` with empty - ``r_itemq``) by deferring to the continuation path - - Add the ``validate_region`` call site(s) needed for the transaction - header (generic call sites for all other types are wired in Patch 7) - - After validation, copy the decoded header to ``trans->r_theader`` - as before + - Delete ``xlog_recover_init_new_trans()``, the ``r_hdr_decoded`` flag + and the header dispatch branch in ``xlog_recovery_process_trans()`` + - The zero-length first fragment case is handled by the continuation + path (see the note below on ``add_to_cont_trans`` needing a + ``r_cur_item != NULL`` guard, added in Patch 5) Patch 3: Move item ops lookup to add_to_trans (first region decode) - Look up and store ``ri_ops`` when ``ri_total == 0``
@@ -444,7 +452,14 @@ Patch 4: Populate validate_nregions/min_hdr_len for all item types time from Patch 1 Patch 5: Fix generic safety issues - - ``add_to_cont_trans``: check ``ri_cnt > 0`` and ``ri_buf != NULL`` + - ``add_to_cont_trans``: check there is an item under assembly + (``trans->r_cur_item != NULL``) before dereferencing it. A corrupt + log whose first op record is a continuation fragment + (``XLOG_WAS_CONT_TRANS`` with no preceding initial fragment) reaches + ``add_to_cont_trans`` with no current item; reject it rather than + dereferencing NULL. (Now that the transaction header is decoded as an + item in Patch 2, this also covers the header's leading-continuation + case, which the old ``xlog_recover_init_new_trans()`` handled itself.) - ``add_to_cont_trans``: bounds-check accumulated region size before ``kvrealloc`` - ``process_data``: validate ``oh_len`` alignment
fs/xfs/libxfs/xfs_log_recover.h +0 -1
@@ -168,7 +168,6 @@ struct xlog_recover {int r_state;/* not needed */ xfs_lsn_t r_lsn;/* xact lsn */ struct list_head r_itemq;/* q for items */ -bool r_hdr_decoded;/* trans header fully decoded */ struct xlog_recover_item *r_cur_item;/* item currently being rebuilt */ };
fs/xfs/xfs_log_recover.c +121 -78
224 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 9/48 xfs: factor out xlog_recover_validate_ophdr()
In reply to Dave ChinnerFactor the existing operation header checks out of xlog_recover_process_ophdr() into a dedicated helper to centralise ophdr validation. Later patches add validation of the remaining ophdr fields here. The ASSERT(0) and WARN_ON(1) are dropped as they add nothing over the logged message and returned error, and they fire on expected corrupt input.
--- fs/xfs/xfs_log_recover.c | 55 ++++++++++++++++++++++++++-------------- 1 file changed, 36 insertions(+), 19 deletions(-)
fs/xfs/xfs_log_recover.c +36 -19
@@ -2638,39 +2638,56 @@ xlog_recover_ophdr_to_trans(return NULL ; } +/* +* Validate an operation header decoded from the journal before it is used to +* drive transaction recovery. The ophdr and its data region come straight from +* the on-disk log and cannot be trusted, so check the fields for known values +* and obvious corruption here, in one place, before anything acts on them. +*/ STATIC int -xlog_recover_process_ophdr ( +xlog_recover_validate_ophdr (struct xlog *log, -struct hlist_head rhash[], -struct xlog_rec_header *rhead,struct xlog_op_header *ohead,char *dp, -char *end, -int pass, -struct list_head *buffer_list) +char *end) { -struct xlog_recover *trans; -unsigned int len; -int error; +unsigned int len =be32_to_cpu (ohead->oh_len); + +/* Check the ophdr contains all the data it is supposed to contain. */ +if (dp + len > end) { +xfs_warn (log->l_mp,"%s: bad length 0x%x" , __func__, len); +return -EFSCORRUPTED ; + }/* Do we understand who wrote this op? */ if (ohead->oh_clientid !=XFS_TRANSACTION && ohead->oh_clientid !=XFS_LOG ) {xfs_warn (log->l_mp,"%s: bad clientid 0x%x" , __func__, ohead->oh_clientid); -ASSERT (0 );return -EFSCORRUPTED ; } -/* -* Check the ophdr contains all the data it is supposed to contain. -*/ - len =be32_to_cpu (ohead->oh_len); -if (dp + len > end) { -xfs_warn (log->l_mp,"%s: bad length 0x%x" , __func__, len); -WARN_ON (1 ); -return -EFSCORRUPTED ; - } +return 0 ; +} + +STATIC int +xlog_recover_process_ophdr ( +struct xlog *log, +struct hlist_head rhash[], +struct xlog_rec_header *rhead, +struct xlog_op_header *ohead, +char *dp, +char *end, +int pass, +struct list_head *buffer_list) +{ +struct xlog_recover *trans; +unsigned int len =be32_to_cpu (ohead->oh_len); +int error; + + error =xlog_recover_validate_ophdr (log, ohead, dp, end); +if (error) +return error; trans =xlog_recover_ophdr_to_trans (rhash, rhead, ohead);if (!trans) {
Signature · 1 line
2.55.0
PATCH 10/48 xfs: validate operation header length and alignment during recovery
In reply to Dave ChinnerThe operation header length read from the journal was only checked against the end of the log record, so an implausibly large or misaligned length was still trusted to drive region assembly and memory allocation.
Bound the length against the largest object that can be logged in a single region and require it to be 32 bit aligned. Move the in-place ophdr decode into the validation helper as well, so a misaligned or out-of-bounds header is rejected before it is dereferenced rather than after.
--- fs/xfs/xfs_log_recover.c | 115 +++++++++++++++++++++++++++------------ 1 file changed, 81 insertions(+), 34 deletions(-)
fs/xfs/xfs_log_recover.c +81 -34
@@ -2639,24 +2639,69 @@ xlog_recover_ophdr_to_trans( }/* -* Validate an operation header decoded from the journal before it is used to -* drive transaction recovery. The ophdr and its data region come straight from -* the on-disk log and cannot be trusted, so check the fields for known values -* and obvious corruption here, in one place, before anything acts on them. +* Decode and validate an operation header from the journal. dp points at the +* ophdr within the log record and end is the end of the record data. The ophdr +* and its data region come straight from the on-disk log and cannot be trusted, +* so decode the header in place and check its fields for known values and +* obvious corruption here, in one place, before anything acts on them. +* +* Returns the validated ophdr, or an ERR_PTR() on corruption. */ -STATIC int +STATIC struct xlog_op_header *xlog_recover_validate_ophdr (struct xlog *log, -struct xlog_op_header *ohead,char *dp,char *end) { -unsigned int len =be32_to_cpu (ohead->oh_len); +struct xlog_op_header *ohead; +unsigned int len; + +/* +* All log regions are 32 bit aligned, so the ophdr must be too. A +* misaligned ophdr means the walk over the record has lost sync with +* the on-disk layout. +*/ +if (dp !=PTR_ALIGN (dp,sizeof (uint32_t ))) { +xfs_warn (log->l_mp,"%s: unaligned op header" , __func__); +return ERR_PTR (-EFSCORRUPTED ); + } + +/* The ophdr itself must fit within the record. */ +if (dp +sizeof (*ohead) > end) { +xfs_warn (log->l_mp,"%s: op header overrun" , __func__); +return ERR_PTR (-EFSCORRUPTED ); + } + + ohead = (struct xlog_op_header *)dp; + dp +=sizeof (*ohead); + len =be32_to_cpu (ohead->oh_len);/* Check the ophdr contains all the data it is supposed to contain. */ if (dp + len > end) {xfs_warn (log->l_mp,"%s: bad length 0x%x" , __func__, len); -return -EFSCORRUPTED ; +return ERR_PTR (-EFSCORRUPTED ); + } + +/* +* All log regions are 32 bit aligned, so the length must be too. A +* misaligned length desyncs the walk over the ophdrs in the record. +*/ +if (!IS_ALIGNED (len,sizeof (uint32_t ))) { +xfs_warn (log->l_mp,"%s: unaligned length 0x%x" , __func__, len); +return ERR_PTR (-EFSCORRUPTED ); + } + +/* +* A single region cannot be larger than the largest object we log in +* one region. That is a whole buffer in a buffer log item (up to +* XFS_MAX_BLOCKSIZE, e.g. a 64k directory block on a 4k block size +* filesystem) or a remote attribute value (up to XFS_XATTR_SIZE_MAX, +* logged in its own region); both are 64k. +*/ +BUILD_BUG_ON (XFS_XATTR_SIZE_MAX >XFS_MAX_BLOCKSIZE ); +if (len >XFS_MAX_BLOCKSIZE ) { +xfs_warn (log->l_mp,"%s: length 0x%x too large" , __func__, len); +return ERR_PTR (-EFSCORRUPTED ); }/* Do we understand who wrote this op? */
@@ -2664,10 +2709,10 @@ xlog_recover_validate_ophdr( ohead->oh_clientid != XFS_LOG) {xfs_warn (log->l_mp,"%s: bad clientid 0x%x" , __func__, ohead->oh_clientid); -return -EFSCORRUPTED ; +return ERR_PTR (-EFSCORRUPTED ); } -return 0 ; +return ohead; }STATIC int
@@ -2675,24 +2720,28 @@ xlog_recover_process_ophdr(struct xlog *log, struct hlist_head rhash[], struct xlog_rec_header *rhead, - struct xlog_op_header *ohead, char *dp, char *end, int pass, struct list_head *buffer_list) { + struct xlog_op_header *ohead;struct xlog_recover *trans; -unsigned int len =be32_to_cpu (ohead->oh_len); +unsigned int len;int error; - error =xlog_recover_validate_ophdr (log, ohead, dp, end); -if (error) -return error; + ohead =xlog_recover_validate_ophdr (log, dp, end); +if (IS_ERR (ohead)) +return PTR_ERR (ohead); + +/* the data region follows the ophdr */ + dp +=sizeof (*ohead); + len =be32_to_cpu (ohead->oh_len); trans =xlog_recover_ophdr_to_trans (rhash, rhead, ohead);if (!trans) {/* nothing to do, so skip over this ophdr */ -return 0 ; +return sizeof (*ohead) + len; }/*
@@ -2726,8 +2775,11 @@ xlog_recover_process_ophdr( log->l_recovery_lsn = trans->r_lsn; } -return xlog_recovery_process_trans (log, trans, dp, len, - ohead->oh_flags, pass, buffer_list); + error =xlog_recovery_process_trans (log, trans, dp, len, + ohead->oh_flags, pass, buffer_list); +if (error) +return error; +return sizeof (*ohead) + len; }/*
@@ -2748,10 +2800,9 @@ xlog_recover_process_data(int pass, struct list_head *buffer_list) { - struct xlog_op_header *ohead; char *end;int num_logops; -int error; +int consumed; end = dp +be32_to_cpu (rhead->h_len); num_logops =be32_to_cpu (rhead->h_num_logops);
@@ -2762,21 +2813,17 @@ xlog_recover_process_data(trace_xfs_log_recover_record (log, rhead, pass);while ((dp < end) && num_logops) { +/* +* Decode, validate and process the next ophdr and its data +* region. On success the number of bytes consumed from the +* record is returned; errors abort recovery. +*/ + consumed =xlog_recover_process_ophdr (log, rhash, rhead, dp, + end, pass, buffer_list); +if (consumed <0 ) +return consumed; - ohead = (struct xlog_op_header *)dp; - dp +=sizeof (*ohead); -if (dp > end) { -xfs_warn (log->l_mp,"%s: op header overrun" , __func__); -return -EFSCORRUPTED ; - } - -/* errors will abort recovery */ - error =xlog_recover_process_ophdr (log, rhash, rhead, ohead, - dp, end, pass, buffer_list); -if (error) -return error; - - dp +=be32_to_cpu (ohead->oh_len); + dp += consumed; num_logops--; }return 0 ;
Signature · 1 line
2.55.0
PATCH 11/48 xfs: validate the operation header padding during recovery
In reply to Dave ChinnerThe operation header padding field is always written as zero, so a non-zero value in a recovered ophdr is a reliable indication of corruption that recovery currently ignores.
Reject a non-zero padding field during ophdr validation.
--- fs/xfs/xfs_log_recover.c | 7 +++++++ 1 file changed, 7 insertions(+)
fs/xfs/xfs_log_recover.c +7 -0
@@ -2712,6 +2712,13 @@ xlog_recover_validate_ophdr(return ERR_PTR (-EFSCORRUPTED ); } +/* The padding is always written as zero. */ +if (ohead->oh_res2 !=0 ) { +xfs_warn (log->l_mp,"%s: bad padding 0x%x" , + __func__, ohead->oh_res2); +return ERR_PTR (-EFSCORRUPTED ); + } +return ohead; }
Signature · 1 line
2.55.0
PATCH 12/48 xfs: validate the operation header flags during recovery
In reply to Dave ChinnerRecovery acted on the operation header flags without checking them for unknown bits or impossible combinations, so corruption in the flags could misdirect transaction assembly rather than being detected.
Reject any oh_flags value the log writer does not produce, accepting only the known individual flags and the valid continuation combinations.
--- fs/xfs/xfs_log_recover.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)
fs/xfs/xfs_log_recover.c +21 -0
@@ -2719,6 +2719,27 @@ xlog_recover_validate_ophdr(return ERR_PTR (-EFSCORRUPTED ); } +/* +* Only the flag combinations the log writer actually produces are +* valid. Reject unknown flags and impossible combinations. END is only +* ever set together with WAS_CONT, and START, COMMIT and UNMOUNT each +* stand alone. +*/ +switch (ohead->oh_flags) { +case 0 : +case XLOG_START_TRANS : +case XLOG_COMMIT_TRANS : +case XLOG_CONTINUE_TRANS : +case XLOG_WAS_CONT_TRANS |XLOG_CONTINUE_TRANS : +case XLOG_WAS_CONT_TRANS |XLOG_END_TRANS : +case XLOG_UNMOUNT_TRANS : +break ; +default : +xfs_warn (log->l_mp,"%s: bad flags 0x%x" , + __func__, ohead->oh_flags); +return ERR_PTR (-EFSCORRUPTED ); + } +return ohead; }
Signature · 1 line
2.55.0
PATCH 13/48 xfs: enforce zero length control records during recovery
In reply to Dave ChinnerStart and commit records only bracket a transaction and carry no data of their own, but recovery only asserted this for start records and did not check it for commit records, so a corrupt non-zero length on either was accepted on production builds.
Enforce the zero length for both in the operation header flag validation and drop the now redundant assert.
--- fs/xfs/xfs_log_recover.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)
fs/xfs/xfs_log_recover.c +11 -3
@@ -2618,8 +2618,6 @@ xlog_recover_ophdr_to_trans(if (!(ohead->oh_flags &XLOG_START_TRANS ))return NULL ; -ASSERT (be32_to_cpu (ohead->oh_len) ==0 ); -/* * This is a new transaction so allocate a new recovery container to * hold the recovery ops that will follow.
@@ -2726,9 +2724,19 @@ xlog_recover_validate_ophdr(* stand alone. */ switch (ohead->oh_flags) { -case 0 :case XLOG_START_TRANS :case XLOG_COMMIT_TRANS : +/* +* Start and commit records only bracket a transaction and carry +* no data of their own. +*/ +if (len !=0 ) { +xfs_warn (log->l_mp,"%s: bad control record length 0x%x" , + __func__, len); +return ERR_PTR (-EFSCORRUPTED ); + } +break ; +case 0 :case XLOG_CONTINUE_TRANS :case XLOG_WAS_CONT_TRANS |XLOG_CONTINUE_TRANS :case XLOG_WAS_CONT_TRANS |XLOG_END_TRANS :
Signature · 1 line
2.55.0
PATCH 14/48 xfs: validate the unmount record during recovery
In reply to Dave ChinnerThe unmount record is the only record the log writes itself rather than through a transaction, and the log item processing loop skips it, so its contents were never validated on recovery even though a clean unmount is detected by inspecting it during the head and tail scan.
Add xlog_recover_validate_unmount_record() to check the record is written by the log, is the expected length, and carries the expected magic and zeroed padding. Call it from the operation header validation, and route the head and tail scan through the operation header validation as well so a corrupt unmount record is detected there rather than being mistaken for a clean unmount.
The unmount record length was incorrectly written as zero for a period of time. A zero length record has no payload to check, so accept it with a warning so that filesystems written by those kernels still recover.
--- fs/xfs/xfs_log_recover.c | 70 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 67 insertions(+), 3 deletions(-)
fs/xfs/xfs_log_recover.c +67 -3
@@ -42,6 +42,8 @@ xlog_clear_stale_blocks( STATIC intxlog_do_recovery_pass (struct xlog *,xfs_daddr_t ,xfs_daddr_t ,int ,xfs_daddr_t *); +STATIC struct xlog_op_header * +xlog_recover_validate_ophdr (struct xlog *log,char *dp,char *end);/* * Sector aligned buffer routines for buffer create/read/write/access
@@ -1169,8 +1171,11 @@ xlog_check_unmount_rec(if (error)return error; - op_head = (struct xlog_op_header *)offset; -if (op_head->oh_flags &XLOG_UNMOUNT_TRANS ) { + op_head =xlog_recover_validate_ophdr (log, offset, + offset +BBSIZE ); +if (IS_ERR (op_head)) +return PTR_ERR (op_head); +if (op_head->oh_flags ==XLOG_UNMOUNT_TRANS ) {/* * Set tail and last sync so that newly written log * records will point recovery to after the current
@@ -2636,6 +2641,60 @@ xlog_recover_ophdr_to_trans(return NULL ; } +/* +* Validate an unmount record. The unmount record is the only record written by +* the log itself rather than by a transaction, and the log item processing loop +* skips it, so it can only be validated here. It is a single fixed size region, +* written by the log with the XFS_LOG clientid, containing an unmount format +* structure with a known magic number and zeroed padding. +* +* The ophdr has already been validated, so oh_len is known to be within the +* record. The unmount format length was incorrectly written for a period of +* time as zero rather than the region length. Such a record has no payload to +* read, so accept it with a warning and stop, so that filesystems written by +* those kernels still recover. +*/ +STATIC int +xlog_recover_validate_unmount_record ( +struct xlog *log, +struct xlog_op_header *ohead) +{ +struct xfs_unmount_log_format *ulf; +unsigned int len =be32_to_cpu (ohead->oh_len); + +if (ohead->oh_clientid !=XFS_LOG ) { +xfs_warn (log->l_mp,"%s: bad unmount clientid 0x%x" , + __func__, ohead->oh_clientid); +return -EFSCORRUPTED ; + } + +if (len ==0 ) { +xfs_warn (log->l_mp, +"%s: zero length unmount record recovered." , __func__); +return 0 ; + } + +if (len !=sizeof (*ulf)) { +xfs_warn (log->l_mp,"%s: bad unmount record length 0x%x" , + __func__, len); +return -EFSCORRUPTED ; + } + + ulf = (struct xfs_unmount_log_format *)(ohead +1 ); +if (ulf->magic !=XLOG_UNMOUNT_TYPE ) { +xfs_warn (log->l_mp,"%s: bad unmount magic 0x%x, expected 0x%x" , + __func__, ulf->magic,XLOG_UNMOUNT_TYPE ); +return -EFSCORRUPTED ; + } + +if (ulf->pad1 !=0 || ulf->pad2 !=0 ) { +xfs_warn (log->l_mp,"%s: bad unmount padding" , __func__); +return -EFSCORRUPTED ; + } + +return 0 ; +} +/* * Decode and validate an operation header from the journal. dp points at the * ophdr within the log record and end is the end of the record data. The ophdr
@@ -2653,6 +2712,7 @@ xlog_recover_validate_ophdr( {struct xlog_op_header *ohead;unsigned int len; +int error;/* * All log regions are 32 bit aligned, so the ophdr must be too. A
@@ -2736,11 +2796,15 @@ xlog_recover_validate_ophdr(return ERR_PTR (-EFSCORRUPTED ); }break ; +case XLOG_UNMOUNT_TRANS : + error =xlog_recover_validate_unmount_record (log, ohead); +if (error) +return ERR_PTR (error); +break ;case 0 :case XLOG_CONTINUE_TRANS :case XLOG_WAS_CONT_TRANS |XLOG_CONTINUE_TRANS :case XLOG_WAS_CONT_TRANS |XLOG_END_TRANS : -case XLOG_UNMOUNT_TRANS :break ;default : xfs_warn(log->l_mp, "%s: bad flags 0x%x",
Signature · 1 line
2.55.0
PATCH 15/48 xfs: handle zero length continuation op headers during recovery
In reply to Dave ChinnerRecovery has always had to cope with zero length op headers, but only ever handled them as a special case in the bespoke transaction header decode. Routing the transaction header through the generic item assembly dropped that handling, so handle zero length continuation op headers generically for all items instead.
A zero length op header that opens a new item carries nothing to decode, so the item cannot be initialised until the continuation that follows arrives. Leave a sentinel in r_cur_item for this case so the continuation is decoded as the first region of the new item. Zero length op headers on subsequent regions are handled naturally by the region iterator. A continuation that reaches the append path with no item under assembly is now treated as a corrupt log.
--- fs/xfs/libxfs/xfs_log_recover.h | 9 +++++++ fs/xfs/xfs_log_recover.c | 48 +++++++++++++++++++++++++++------ 2 files changed, 49 insertions(+), 8 deletions(-)
fs/xfs/libxfs/xfs_log_recover.h +9 -0
@@ -171,6 +171,15 @@ struct xlog_recover {struct xlog_recover_item *r_cur_item;/* item currently being rebuilt */ }; +/* +* The current recovery item can sometimes start with a zero length continuation +* op header. In this case there is nothing to decode, so the recovery item +* cannot be initialised yet. Leave this sentinel in r_cur_item instead so the +* continuation initialises the recovery item from the start of the continued +* region data. +*/ +#define XLOG_RECOVER_CONT_ITEM ((struct xlog_recover_item *)1UL) +#define ITEM_TYPE (i) (*(unsigned short *)(i)->ri_buf[0].iov_base)#define XLOG_RECOVER_CRCPASS 0
fs/xfs/xfs_log_recover.c +40 -8
85 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 16/48 xfs: bound the assembled region size during recovery
In reply to Dave ChinnerA region split across continuation op headers is grown one fragment at a time with no limit on the total size, so a corrupt continuation stream can drive an unbounded allocation during recovery.
Add XLOG_MAX_REGION_SIZE - the largest object logged in a single region - as a format level bound usable by both the log write and recovery paths, and reject a continuation once the assembled region would exceed it. Use the same constant for the existing per-op-header length check.
--- fs/xfs/libxfs/xfs_log_format.h | 8 ++++++++ fs/xfs/xfs_log_recover.c | 21 ++++++++++++--------- 2 files changed, 20 insertions(+), 9 deletions(-)
fs/xfs/libxfs/xfs_log_format.h +8 -0
@@ -38,6 +38,14 @@ typedef uint32_t xlog_tid_t;#define XLOG_HEADER_SIZE 512 +/* +* The largest object that is logged in a single region: a whole buffer in a +* buffer log item, or a remote attribute value. This bounds the size of any +* single log region both when writing the log and when reassembling regions +* during recovery. +*/ +#define XLOG_MAX_REGION_SIZE max(XFS_MAX_BLOCKSIZE, XFS_XATTR_SIZE_MAX) +/* Minimum number of transactions that must fit in the log (defined by mkfs) */ #define XFS_MIN_LOG_FACTOR 3
fs/xfs/xfs_log_recover.c +12 -9
@@ -2408,6 +2408,16 @@ xlog_recover_add_to_cont_trans( old_ptr = item->ri_buf[item->ri_cnt-1 ].iov_base; old_len = item->ri_buf[item->ri_cnt-1 ].iov_len; +/* +* Bound the assembled region size before growing the buffer so a corrupt +* continuation stream cannot drive an unbounded allocation. +*/ +if (old_len + len >XLOG_MAX_REGION_SIZE ) { +xfs_warn (log->l_mp,"%s: region too large (0x%x)" , + __func__, old_len + len); +return -EFSCORRUPTED ; + } + ptr =kvrealloc (old_ptr, len + old_len,GFP_KERNEL );if (!ptr)return -ENOMEM ;
@@ -2781,15 +2791,8 @@ xlog_recover_validate_ophdr(return ERR_PTR (-EFSCORRUPTED ); } -/* -* A single region cannot be larger than the largest object we log in -* one region. That is a whole buffer in a buffer log item (up to -* XFS_MAX_BLOCKSIZE, e.g. a 64k directory block on a 4k block size -* filesystem) or a remote attribute value (up to XFS_XATTR_SIZE_MAX, -* logged in its own region); both are 64k. -*/ -BUILD_BUG_ON (XFS_XATTR_SIZE_MAX >XFS_MAX_BLOCKSIZE ); -if (len >XFS_MAX_BLOCKSIZE ) { +/* A single region cannot be larger than the largest object we log. */ +if (len >XLOG_MAX_REGION_SIZE ) {xfs_warn (log->l_mp,"%s: length 0x%x too large" , __func__, len);return ERR_PTR (-EFSCORRUPTED ); }
Signature · 1 line
2.55.0
PATCH 17/48 xfs: dump the corrupt region when log item validation fails
In reply to Dave ChinnerLog item validation reports failures with a type specific warning, but that warning does not include the region contents. The existing recovery code dumped the offending buffer via XFS_CORRUPTION_ERROR for diagnostics, and that is being removed as per-type validation replaces it.
Emit XFS_CORRUPTION_ERROR from the region completion path when validation fails, so the type specific warning is accompanied by a hex dump of the relevant buffer. A per-region failure dumps that region; a whole-item failure dumps the format header. This is added before the per-type validation so those patches can drop their own corruption dumps.
--- fs/xfs/xfs_log_recover.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-)
fs/xfs/xfs_log_recover.c +17 -6
@@ -2353,7 +2353,8 @@ xlog_recover_complete_region( struct xlog_recover *trans) {struct xlog_recover_item *item = trans->r_cur_item; -int error; +int region_to_dump =0 ; +int error =0 ;/* * The format header (region 0) must be at least the minimum size for
@@ -2366,19 +2367,29 @@ xlog_recover_complete_region("log item type 0x%x header too short (%zu < %u)" , item->ri_ops->item_type, item->ri_buf[0 ].iov_len, item->ri_ops->min_hdr_len); -return -EFSCORRUPTED ; + error = -EFSCORRUPTED ; + goto out_error; }if (item->ri_ops->validate_region) { error = item->ri_ops->validate_region (log, item, item->ri_cnt -1 ); -if (error) -return error; +if (error) { +/* dump the bad region rather than the log format header */ + region_to_dump = item->ri_cnt -1 ; + goto out_error; + } }if (item->ri_cnt == item->ri_total) -return xlog_recover_complete_item (log, trans); -return 0 ; + error =xlog_recover_complete_item (log, trans); + +out_error: +if (error == -EFSCORRUPTED ) +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , log->l_mp, + item->ri_buf[region_to_dump].iov_base, + item->ri_buf[region_to_dump].iov_len); +return error; }STATIC int
Signature · 1 line
2.55.0
PATCH 18/48 xfs: validate fixed size log items during recovery
In reply to Dave ChinnerThe inode create and quota off log items are each a single fixed size region. Recovery trusted the region layout decoded from the journal without checking that it matched.
Validate both against their fixed shape as they are decoded. A shared region count check rejects anything other than a single region, and each type checks its one region is exactly the format structure size.
--- fs/xfs/libxfs/xfs_log_recover.h | 1 + fs/xfs/xfs_dquot_item_recover.c | 18 ++++++++++++++++++ fs/xfs/xfs_icreate_item.c | 18 ++++++++++++++++++ fs/xfs/xfs_log_recover.c | 25 ++++++++++++++++++++----- 4 files changed, 57 insertions(+), 5 deletions(-)
fs/xfs/libxfs/xfs_log_recover.h +1 -0
@@ -196,6 +196,7 @@ int xlog_recover_iget_handle(struct xfs_mount *mp, xfs_ino_t ino, uint32_t gen, struct xfs_inode **ipp);void xlog_recover_release_intent (struct xlog *log,unsigned short intent_type,uint64_t intent_id); +int xlog_recover_single_region (struct xlog *log,uint16_t nregions);int xlog_alloc_buf_cancel_table (struct xlog *log);void xlog_free_buf_cancel_table (struct xlog *log);
fs/xfs/xfs_dquot_item_recover.c +18 -0
@@ -208,8 +208,26 @@ xlog_recover_quotaoff_commit_pass1(return 0 ; } +static int +xlog_recover_quotaoff_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +if (item->ri_buf[region_index].iov_len != +sizeof (struct xfs_qoff_logformat )) { +xfs_warn (log->l_mp,"QOFF: bad item region length %zu recovered." , + item->ri_buf[region_index].iov_len); +return -EFSCORRUPTED ; + } +return 0 ; +} +const struct xlog_recover_item_ops xlog_quotaoff_item_ops = { .item_type =XFS_LI_QUOTAOFF , + .min_hdr_len =sizeof (struct xfs_qoff_logformat ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_quotaoff_validate_region, .commit_pass1 = xlog_recover_quotaoff_commit_pass1,/* nothing to commit in pass2 */ };
fs/xfs/xfs_icreate_item.c +18 -0
@@ -253,8 +253,26 @@ xlog_recover_icreate_commit_pass2( length, be32_to_cpu(icl->icl_gen)); } +static int +xlog_recover_icreate_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +if (item->ri_buf[region_index].iov_len != +sizeof (struct xfs_icreate_log )) { +xfs_warn (log->l_mp,"ICR: bad item region length %zu recovered." , + item->ri_buf[region_index].iov_len); +return -EFSCORRUPTED ; + } +return 0 ; +} +const struct xlog_recover_item_ops xlog_icreate_item_ops = { .item_type =XFS_LI_ICREATE , + .min_hdr_len =sizeof (struct xfs_icreate_log ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_icreate_validate_region, .reorder = xlog_recover_icreate_reorder, .commit_pass2 = xlog_recover_icreate_commit_pass2, };
fs/xfs/xfs_log_recover.c +20 -5
@@ -2224,14 +2224,25 @@ xlog_recover_nregions( struct xlog *log, uint16_t nregions) { -if (nregions ==0 || nregions >XLOG_MAX_REGIONS_IN_ITEM ) { -xfs_warn (log->l_mp, -"bad number of regions (%d) in inode log format" , nregions); +if (nregions ==0 || nregions >XLOG_MAX_REGIONS_IN_ITEM )return -EFSCORRUPTED ; - }return nregions; } +/* +* ->validate_nregions() helper for the many item types that are made up of a +* single region. +*/ +int +xlog_recover_single_region ( +struct xlog *log, +uint16_t nregions) +{ +if (nregions !=1 ) +return -EFSCORRUPTED ; +return 1 ; +} +/* * Start decoding a new log item. The item type and region count come from the * type/size fields at the front of the item's first region. Look up the item
@@ -2271,8 +2282,12 @@ xlog_recover_add_item( valid_nregions = ops->validate_nregions (log, nregions); else valid_nregions =xlog_recover_nregions (log, nregions); -if (valid_nregions <0 ) +if (valid_nregions <0 ) { +xfs_warn (log->l_mp, +"%s: bad region count %u for log item type 0x%x" , + __func__, nregions, item_type);return ERR_PTR (valid_nregions); + } item = kzalloc_obj(struct xlog_recover_item, GFP_KERNEL | __GFP_NOFAIL);INIT_LIST_HEAD (&item->ri_list);
Signature · 1 line
2.55.0
PATCH 19/48 xfs: validate extent free log items during recovery
In reply to Dave ChinnerThe extent free intent and done items are single region items whose length depends on the extent count declared in the region. Recovery only checked this in the commit handlers, after the item had already been decoded and initialised. Validate the structure up front as the region is decoded instead. The data and realtime variants share these checks as their formats are identical.
The commit handlers and xfs_efi_copy_format() can now trust the structure, so the redundant checks are removed from them. xfs_efi_copy_format() no longer has a failure case and returns void.
--- fs/xfs/xfs_extfree_item.c | 217 ++++++++++++++++++++++++-------------- 1 file changed, 136 insertions(+), 81 deletions(-)
fs/xfs/xfs_extfree_item.c +136 -81
@@ -180,58 +180,58 @@ xfs_efi_init(* one of which will be the native format for this kernel. * It will handle the conversion of formats if necessary. */ -STATIC int +STATIC voidxfs_efi_copy_format (struct kvec *buf,struct xfs_efi_log_format *dst_efi_fmt) {struct xfs_efi_log_format *src_efi_fmt = buf->iov_base; -uint len, len32, len64, i; - - len =xfs_efi_log_format_sizeof (src_efi_fmt->efi_nextents); - len32 =xfs_efi_log_format32_sizeof (src_efi_fmt->efi_nextents); - len64 =xfs_efi_log_format64_sizeof (src_efi_fmt->efi_nextents); +struct xfs_efi_log_format_32 *src_efi_fmt_32; +struct xfs_efi_log_format_64 *src_efi_fmt_64; +uint nextents = src_efi_fmt->efi_nextents; +uint i; -if (buf->iov_len == len) { +/* +* The buffer has already been validated as one of the three formats, so +* pick which one it is by its length and copy it into the native form. +*/ +if (buf->iov_len ==xfs_efi_log_format_sizeof (nextents)) {memcpy (dst_efi_fmt, src_efi_fmt, offsetof(struct xfs_efi_log_format , efi_extents)); -for (i =0 ; i < src_efi_fmt->efi_nextents; i++) +for (i =0 ; i < nextents; i++)memcpy (&dst_efi_fmt->efi_extents[i], &src_efi_fmt->efi_extents[i],sizeof (struct xfs_extent )); -return 0 ; - }else if (buf->iov_len == len32) { -struct xfs_efi_log_format_32 *src_efi_fmt_32 = buf->iov_base; +return ; + } +if (buf->iov_len ==xfs_efi_log_format32_sizeof (nextents)) { + src_efi_fmt_32 = buf->iov_base; dst_efi_fmt->efi_type = src_efi_fmt_32->efi_type; dst_efi_fmt->efi_size = src_efi_fmt_32->efi_size; dst_efi_fmt->efi_nextents = src_efi_fmt_32->efi_nextents; dst_efi_fmt->efi_id = src_efi_fmt_32->efi_id; -for (i =0 ; i < dst_efi_fmt->efi_nextents; i++) { +for (i =0 ; i < nextents; i++) { dst_efi_fmt->efi_extents[i].ext_start = src_efi_fmt_32->efi_extents[i].ext_start; dst_efi_fmt->efi_extents[i].ext_len = src_efi_fmt_32->efi_extents[i].ext_len; } -return 0 ; - }else if (buf->iov_len == len64) { -struct xfs_efi_log_format_64 *src_efi_fmt_64 = buf->iov_base; - - dst_efi_fmt->efi_type = src_efi_fmt_64->efi_type; - dst_efi_fmt->efi_size = src_efi_fmt_64->efi_size; - dst_efi_fmt->efi_nextents = src_efi_fmt_64->efi_nextents; - dst_efi_fmt->efi_id = src_efi_fmt_64->efi_id; -for (i =0 ; i < dst_efi_fmt->efi_nextents; i++) { - dst_efi_fmt->efi_extents[i].ext_start = - src_efi_fmt_64->efi_extents[i].ext_start; - dst_efi_fmt->efi_extents[i].ext_len = - src_efi_fmt_64->efi_extents[i].ext_len; - } -return 0 ; +return ; + } + +ASSERT (buf->iov_len ==xfs_efi_log_format64_sizeof (nextents)); + src_efi_fmt_64 = buf->iov_base; + dst_efi_fmt->efi_type = src_efi_fmt_64->efi_type; + dst_efi_fmt->efi_size = src_efi_fmt_64->efi_size; + dst_efi_fmt->efi_nextents = src_efi_fmt_64->efi_nextents; + dst_efi_fmt->efi_id = src_efi_fmt_64->efi_id; +for (i =0 ; i < nextents; i++) { + dst_efi_fmt->efi_extents[i].ext_start = + src_efi_fmt_64->efi_extents[i].ext_start; + dst_efi_fmt->efi_extents[i].ext_len = + src_efi_fmt_64->efi_extents[i].ext_len; } -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW ,NULL , buf->iov_base, - buf->iov_len); -return -EFSCORRUPTED ; }static inline struct xfs_efd_log_item *EFD_ITEM (struct xfs_log_item *lip)
@@ -864,22 +864,11 @@ xlog_recover_efi_commit_pass2(struct xfs_mount *mp = log->l_mp;struct xfs_efi_log_item *efip;struct xfs_efi_log_format *efi_formatp; -int error; efi_formatp = item->ri_buf[0 ].iov_base; -if (item->ri_buf[0 ].iov_len <xfs_efi_log_format_sizeof (0 )) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); -return -EFSCORRUPTED ; - } - efip =xfs_efi_init (mp,ITEM_TYPE (item), efi_formatp->efi_nextents); - error =xfs_efi_copy_format (&item->ri_buf[0 ], &efip->efi_format); -if (error) { -xfs_efi_item_free (efip); -return error; - } +xfs_efi_copy_format (&item->ri_buf[0 ], &efip->efi_format);atomic_set (&efip->efi_next_extent, efi_formatp->efi_nextents); xlog_recover_intent_item(log, &efip->efi_item, lsn,
@@ -887,8 +876,60 @@ xlog_recover_efi_commit_pass2(return 0 ; } +/* +* The efi is a single region whose length must match the extent count for one +* of the native, 32 bit or 64 bit on-disk formats. This is shared by the data +* and realtime efi items, which have identical formats. +*/ +static int +xlog_recover_efi_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +struct xfs_efi_log_format *efi_formatp = + item->ri_buf[region_index].iov_base; +size_t len = item->ri_buf[region_index].iov_len; +uint32_t nextents; + +if (len <xfs_efi_log_format_sizeof (0 )) { +xfs_warn (log->l_mp,"EFI: bad item region length %zu recovered." , + len); +return -EFSCORRUPTED ; + } + +/* +* Bound the extent count before the exact size checks below so a corrupt +* count cannot overflow the size calculations. This limit is a fuzzy +* upper bound using the smallest extent encoding to prevent overflow; it +* does not need to be exact as the size checks that follow enforce the +* precise length for each format. +*/ + nextents = efi_formatp->efi_nextents; +if (nextents ==0 || + nextents >XLOG_MAX_REGION_SIZE /sizeof (struct xfs_extent_32 )) { +xfs_warn (log->l_mp,"EFI: bad extent count %u recovered." , + nextents); +return -EFSCORRUPTED ; + } + +if (len !=xfs_efi_log_format_sizeof (nextents) && + len !=xfs_efi_log_format32_sizeof (nextents) && + len !=xfs_efi_log_format64_sizeof (nextents)) { +xfs_warn (log->l_mp, +"EFI: item region length %zu does not match %u extents recovered." , + len, nextents); +return -EFSCORRUPTED ; + } + +return 0 ; +} +const struct xlog_recover_item_ops xlog_efi_item_ops = { .item_type =XFS_LI_EFI , + .min_hdr_len =sizeof (struct xfs_efi_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_efi_validate_region, .commit_pass2 = xlog_recover_efi_commit_pass2, };
@@ -903,22 +944,11 @@ xlog_recover_rtefi_commit_pass2(struct xfs_mount *mp = log->l_mp;struct xfs_efi_log_item *efip;struct xfs_efi_log_format *efi_formatp; -int error; efi_formatp = item->ri_buf[0 ].iov_base; -if (item->ri_buf[0 ].iov_len <xfs_efi_log_format_sizeof (0 )) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); -return -EFSCORRUPTED ; - } - efip =xfs_efi_init (mp,ITEM_TYPE (item), efi_formatp->efi_nextents); - error =xfs_efi_copy_format (&item->ri_buf[0 ], &efip->efi_format); -if (error) { -xfs_efi_item_free (efip); -return error; - } +xfs_efi_copy_format (&item->ri_buf[0 ], &efip->efi_format);atomic_set (&efip->efi_next_extent, efi_formatp->efi_nextents); xlog_recover_intent_item(log, &efip->efi_item, lsn,
@@ -941,6 +971,9 @@ xlog_recover_rtefi_commit_pass2(const struct xlog_recover_item_ops xlog_rtefi_item_ops = { .item_type =XFS_LI_EFI_RT , + .min_hdr_len =sizeof (struct xfs_efi_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_efi_validate_region, .commit_pass2 = xlog_recover_rtefi_commit_pass2, };
@@ -959,31 +992,66 @@ xlog_recover_efd_commit_pass2( xfs_lsn_t lsn) {struct xfs_efd_log_format *efd_formatp; -int buflen = item->ri_buf[0 ].iov_len; efd_formatp = item->ri_buf[0 ].iov_base; -if (buflen <sizeof (struct xfs_efd_log_format )) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , log->l_mp, - efd_formatp, buflen); +xlog_recover_release_intent (log,XFS_LI_EFI , efd_formatp->efd_efi_id); +return 0 ; +} + +/* +* The efd is a single region whose length must match the extent count for the +* 32 bit or 64 bit on-disk format. This is shared by the data and realtime efd +* items, which have identical formats. +*/ +static int +xlog_recover_efd_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +struct xfs_efd_log_format *efd_formatp = + item->ri_buf[region_index].iov_base; +size_t len = item->ri_buf[region_index].iov_len; +uint32_t nextents; + +if (len <sizeof (struct xfs_efd_log_format )) { +xfs_warn (log->l_mp,"EFD: bad item region length %zu recovered." , + len);return -EFSCORRUPTED ; } -if (item->ri_buf[0 ].iov_len !=xfs_efd_log_format32_sizeof ( - efd_formatp->efd_nextents) && - item->ri_buf[0 ].iov_len !=xfs_efd_log_format64_sizeof ( - efd_formatp->efd_nextents)) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , log->l_mp, - efd_formatp, buflen); +/* +* Bound the extent count before the exact size checks below so a corrupt +* count cannot overflow the size calculations. This limit is a fuzzy +* upper bound using the smallest extent encoding to prevent overflow; it +* does not need to be exact as the size checks that follow enforce the +* precise length for each format. +*/ + nextents = efd_formatp->efd_nextents; +if (nextents ==0 || + nextents >XLOG_MAX_REGION_SIZE /sizeof (struct xfs_extent_32 )) { +xfs_warn (log->l_mp,"EFD: bad extent count %u recovered." , + nextents); +return -EFSCORRUPTED ; + } + +if (len !=xfs_efd_log_format32_sizeof (nextents) && + len !=xfs_efd_log_format64_sizeof (nextents)) { +xfs_warn (log->l_mp, +"EFD: item region length %zu does not match %u extents recovered." , + len, nextents);return -EFSCORRUPTED ; } -xlog_recover_release_intent (log,XFS_LI_EFI , efd_formatp->efd_efi_id);return 0 ; }const struct xlog_recover_item_ops xlog_efd_item_ops = { .item_type =XFS_LI_EFD , + .min_hdr_len =sizeof (struct xfs_efd_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_efd_validate_region, .commit_pass2 = xlog_recover_efd_commit_pass2, };
@@ -996,25 +1064,9 @@ xlog_recover_rtefd_commit_pass2( xfs_lsn_t lsn) {struct xfs_efd_log_format *efd_formatp; -int buflen = item->ri_buf[0 ].iov_len; efd_formatp = item->ri_buf[0 ].iov_base; -if (buflen <sizeof (struct xfs_efd_log_format )) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , log->l_mp, - efd_formatp, buflen); -return -EFSCORRUPTED ; - } - -if (item->ri_buf[0 ].iov_len !=xfs_efd_log_format32_sizeof ( - efd_formatp->efd_nextents) && - item->ri_buf[0 ].iov_len !=xfs_efd_log_format64_sizeof ( - efd_formatp->efd_nextents)) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , log->l_mp, - efd_formatp, buflen); -return -EFSCORRUPTED ; - } -xlog_recover_release_intent (log,XFS_LI_EFI_RT , efd_formatp->efd_efi_id);return 0 ;
@@ -1025,5 +1077,8 @@ xlog_recover_rtefd_commit_pass2(const struct xlog_recover_item_ops xlog_rtefd_item_ops = { .item_type =XFS_LI_EFD_RT , + .min_hdr_len =sizeof (struct xfs_efd_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_efd_validate_region, .commit_pass2 = xlog_recover_rtefd_commit_pass2, };
Signature · 1 line
2.55.0
PATCH 20/48 xfs: validate and convert extent free intents during recovery
In reply to Dave ChinnerThe extents in a recovered extent free intent were only checked for validity during intent replay, after the item had been decoded, copied to host format and queued. A corrupt extent was therefore carried a long way into recovery before being rejected.
Validate the extents up front and convert the item to host format in ->validate_item(), so the rest of recovery iterates the item as a host structure without regard for the on-disk format. The conversion allocates a new buffer only when the on-disk format differs in size from the host format.
The commit handlers then copy the already host format item directly, and the extent validity check is dropped from intent replay. The format conversion no longer needs its host format input case and is renamed accordingly. This is shared by the data and realtime extent free items.
--- fs/xfs/xfs_extfree_item.c | 88 +++++++++++++++++++++++---------------- 1 file changed, 51 insertions(+), 37 deletions(-)
fs/xfs/xfs_extfree_item.c +51 -37
@@ -174,14 +174,13 @@ xfs_efi_init( }/* -* Copy an EFI format buffer from the given buf, and into the destination -* EFI format structure. -* The given buffer can be in 32 bit or 64 bit form (which has different padding), -* one of which will be the native format for this kernel. -* It will handle the conversion of formats if necessary. +* Convert an on-disk EFI format buffer into the given host format structure. +* The buffer is in 32 bit or 64 bit on-disk form (which have different padding); +* a buffer already in host format does not need conversion and is not passed +* here. The buffer has already been validated as one of these two formats. */ STATIC void -xfs_efi_copy_format ( +xfs_efi_to_host_format (struct kvec *buf,struct xfs_efi_log_format *dst_efi_fmt) {
@@ -191,20 +190,6 @@ xfs_efi_copy_format(uint nextents = src_efi_fmt->efi_nextents;uint i; -/* -* The buffer has already been validated as one of the three formats, so -* pick which one it is by its length and copy it into the native form. -*/ -if (buf->iov_len ==xfs_efi_log_format_sizeof (nextents)) { -memcpy (dst_efi_fmt, src_efi_fmt, - offsetof(struct xfs_efi_log_format , efi_extents)); -for (i =0 ; i < nextents; i++) -memcpy (&dst_efi_fmt->efi_extents[i], - &src_efi_fmt->efi_extents[i], -sizeof (struct xfs_extent )); -return ; - } -if (buf->iov_len ==xfs_efi_log_format32_sizeof (nextents)) { src_efi_fmt_32 = buf->iov_base; dst_efi_fmt->efi_type = src_efi_fmt_32->efi_type;
@@ -660,23 +645,10 @@ xfs_extent_free_recover_work(int error =0 ;bool isrt =xfs_efi_item_isrt (lip); -/* -* First check the validity of the extents described by the EFI. If -* any are bad, then assume that all are bad and just toss the EFI. -* Mixing RT and non-RT extents in the same EFI item is not allowed. -*/ -for (i =0 ; i < efip->efi_format.efi_nextents; i++) { -if (!xfs_efi_validate_ext (mp, isrt, - &efip->efi_format.efi_extents[i])) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - &efip->efi_format, -sizeof (efip->efi_format)); -return -EFSCORRUPTED ; - } - +/* The extents were validated when the efi was recovered. */ +for (i =0 ; i < efip->efi_format.efi_nextents; i++)xfs_efi_recover_work (mp, dfp, isrt, &efip->efi_format.efi_extents[i]); - } resv =xlog_recover_resv (&M_RES (mp)->tr_itruncate); error =xfs_trans_alloc (mp, &resv,0 ,0 ,0 , &tp);
@@ -868,7 +840,7 @@ xlog_recover_efi_commit_pass2( efi_formatp = item->ri_buf[0 ].iov_base; efip =xfs_efi_init (mp,ITEM_TYPE (item), efi_formatp->efi_nextents); -xfs_efi_copy_format (&item->ri_buf[0 ], &efip->efi_format); +memcpy (&efip->efi_format, efi_formatp, item->ri_buf[0 ].iov_len);atomic_set (&efip->efi_next_extent, efi_formatp->efi_nextents); xlog_recover_intent_item(log, &efip->efi_item, lsn,
@@ -925,11 +897,52 @@ xlog_recover_efi_validate_region(return 0 ; } +/* +* Convert the efi to host format and validate its contents. Later recovery then +* iterates ri_buf[0] as a host struct xfs_efi_log_format without caring about +* the on-disk format. The conversion only needs to allocate a new buffer when +* the on-disk format differs in size from the host format; a buffer already in +* host format is validated in place. +*/ +static int +xlog_recover_efi_validate_item ( +struct xlog *log, +struct xlog_recover_item *item) +{ +struct xfs_efi_log_format *efi_formatp = item->ri_buf[0 ].iov_base; +uint32_t nextents = efi_formatp->efi_nextents; +size_t host_len =xfs_efi_log_format_sizeof (nextents); +bool isrt =ITEM_TYPE (item) ==XFS_LI_EFI_RT ; +uint32_t i; + +if (item->ri_buf[0 ].iov_len != host_len) { + efi_formatp =xlog_kvmalloc (host_len); +xfs_efi_to_host_format (&item->ri_buf[0 ], efi_formatp); +kvfree (item->ri_buf[0 ].iov_base); + item->ri_buf[0 ].iov_base = efi_formatp; + item->ri_buf[0 ].iov_len = host_len; + } + +for (i =0 ; i < nextents; i++) { +struct xfs_extent *extp = &efi_formatp->efi_extents[i]; + +if (!xfs_efi_validate_ext (log->l_mp, isrt, extp)) { +xfs_warn (log->l_mp, +"EFI: bad extent %u (start 0x%llx, len 0x%x) recovered." , + i, extp->ext_start, extp->ext_len); +return -EFSCORRUPTED ; + } + } + +return 0 ; +} +const struct xlog_recover_item_ops xlog_efi_item_ops = { .item_type =XFS_LI_EFI , .min_hdr_len =sizeof (struct xfs_efi_log_format ), .validate_nregions = xlog_recover_single_region, .validate_region = xlog_recover_efi_validate_region, + .validate_item = xlog_recover_efi_validate_item, .commit_pass2 = xlog_recover_efi_commit_pass2, };
@@ -948,7 +961,7 @@ xlog_recover_rtefi_commit_pass2( efi_formatp = item->ri_buf[0 ].iov_base; efip =xfs_efi_init (mp,ITEM_TYPE (item), efi_formatp->efi_nextents); -xfs_efi_copy_format (&item->ri_buf[0 ], &efip->efi_format); +memcpy (&efip->efi_format, efi_formatp, item->ri_buf[0 ].iov_len);atomic_set (&efip->efi_next_extent, efi_formatp->efi_nextents); xlog_recover_intent_item(log, &efip->efi_item, lsn,
@@ -974,6 +987,7 @@ const struct xlog_recover_item_ops xlog_rtefi_item_ops = { .min_hdr_len = sizeof(struct xfs_efi_log_format), .validate_nregions = xlog_recover_single_region, .validate_region = xlog_recover_efi_validate_region, + .validate_item = xlog_recover_efi_validate_item, .commit_pass2 = xlog_recover_rtefi_commit_pass2, };
Signature · 1 line
2.55.0
PATCH 21/48 xfs: validate and convert extent free done items during recovery
In reply to Dave ChinnerAn extent free done item carries the same extents as the intent it completes, but recovery never converted it to host format or checked those extents. The on-disk format was left in place and its contents trusted.
Convert the item to host format and validate its extents in ->validate_item(), mirroring the extent free intent item. The conversion allocates a new buffer only when the on-disk format differs in size from the host format. This is shared by the data and realtime extent free done items.
This conversion and validation is being done now so that, in future, the EFD extents can be checked against the extents in the matched EFI. This will ensure a corruption of the efd_efi_id does not result in completion of an incorrect EFI.
--- fs/xfs/xfs_extfree_item.c | 85 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+)
fs/xfs/xfs_extfree_item.c +85 -0
@@ -219,6 +219,52 @@ xfs_efi_to_host_format( } } +/* +* Convert an on-disk EFD format buffer into the given host format structure. +* The buffer is in 32 bit or 64 bit on-disk form (which have different padding); +* a buffer already in host format does not need conversion and is not passed +* here. The buffer has already been validated as one of these two formats. +*/ +STATIC void +xfs_efd_to_host_format ( +struct kvec *buf, +struct xfs_efd_log_format *dst_efd_fmt) +{ +struct xfs_efd_log_format *src_efd_fmt = buf->iov_base; +struct xfs_efd_log_format_32 *src_efd_fmt_32; +struct xfs_efd_log_format_64 *src_efd_fmt_64; +uint nextents = src_efd_fmt->efd_nextents; +uint i; + +if (buf->iov_len ==xfs_efd_log_format32_sizeof (nextents)) { + src_efd_fmt_32 = buf->iov_base; + dst_efd_fmt->efd_type = src_efd_fmt_32->efd_type; + dst_efd_fmt->efd_size = src_efd_fmt_32->efd_size; + dst_efd_fmt->efd_nextents = src_efd_fmt_32->efd_nextents; + dst_efd_fmt->efd_efi_id = src_efd_fmt_32->efd_efi_id; +for (i =0 ; i < nextents; i++) { + dst_efd_fmt->efd_extents[i].ext_start = + src_efd_fmt_32->efd_extents[i].ext_start; + dst_efd_fmt->efd_extents[i].ext_len = + src_efd_fmt_32->efd_extents[i].ext_len; + } +return ; + } + +ASSERT (buf->iov_len ==xfs_efd_log_format64_sizeof (nextents)); + src_efd_fmt_64 = buf->iov_base; + dst_efd_fmt->efd_type = src_efd_fmt_64->efd_type; + dst_efd_fmt->efd_size = src_efd_fmt_64->efd_size; + dst_efd_fmt->efd_nextents = src_efd_fmt_64->efd_nextents; + dst_efd_fmt->efd_efi_id = src_efd_fmt_64->efd_efi_id; +for (i =0 ; i < nextents; i++) { + dst_efd_fmt->efd_extents[i].ext_start = + src_efd_fmt_64->efd_extents[i].ext_start; + dst_efd_fmt->efd_extents[i].ext_len = + src_efd_fmt_64->efd_extents[i].ext_len; + } +} +static inline struct xfs_efd_log_item *EFD_ITEM (struct xfs_log_item *lip) { return container_of(lip, struct xfs_efd_log_item, efd_item);
@@ -1061,11 +1107,49 @@ xlog_recover_efd_validate_region(return 0 ; } +/* +* Convert the efd to host format and validate its contents, as for the efi. +* The efd extents mirror those of the efi it completes. +*/ +static int +xlog_recover_efd_validate_item ( +struct xlog *log, +struct xlog_recover_item *item) +{ +struct xfs_efd_log_format *efd_formatp = item->ri_buf[0 ].iov_base; +uint32_t nextents = efd_formatp->efd_nextents; +size_t host_len =xfs_efd_log_format_sizeof (nextents); +bool isrt =ITEM_TYPE (item) ==XFS_LI_EFD_RT ; +uint32_t i; + +if (item->ri_buf[0 ].iov_len != host_len) { + efd_formatp =xlog_kvmalloc (host_len); +xfs_efd_to_host_format (&item->ri_buf[0 ], efd_formatp); +kvfree (item->ri_buf[0 ].iov_base); + item->ri_buf[0 ].iov_base = efd_formatp; + item->ri_buf[0 ].iov_len = host_len; + } + +for (i =0 ; i < nextents; i++) { +struct xfs_extent *extp = &efd_formatp->efd_extents[i]; + +if (!xfs_efi_validate_ext (log->l_mp, isrt, extp)) { +xfs_warn (log->l_mp, +"EFD: bad extent %u (start 0x%llx, len 0x%x) recovered." , + i, extp->ext_start, extp->ext_len); +return -EFSCORRUPTED ; + } + } + +return 0 ; +} +const struct xlog_recover_item_ops xlog_efd_item_ops = { .item_type =XFS_LI_EFD , .min_hdr_len =sizeof (struct xfs_efd_log_format ), .validate_nregions = xlog_recover_single_region, .validate_region = xlog_recover_efd_validate_region, + .validate_item = xlog_recover_efd_validate_item, .commit_pass2 = xlog_recover_efd_commit_pass2, };
@@ -1094,5 +1178,6 @@ const struct xlog_recover_item_ops xlog_rtefd_item_ops = { .min_hdr_len = sizeof(struct xfs_efd_log_format), .validate_nregions = xlog_recover_single_region, .validate_region = xlog_recover_efd_validate_region, + .validate_item = xlog_recover_efd_validate_item, .commit_pass2 = xlog_recover_rtefd_commit_pass2, };
Signature · 1 line
2.55.0
PATCH 22/48 xfs: validate rmap update intents during recovery
In reply to Dave ChinnerAn rmap update intent is a single region item whose length depends on its extent count, and whose extents carry rmap owner, offset and state that recovery must trust. These were only checked during intent replay, after the item had been decoded and queued.
Validate the region length as it is decoded, and validate the rmap extents in ->validate_item(). The rmap map extent has fixed width fields, so there is a single on-disk format and no conversion is required. The commit handlers and intent replay can then trust the item, so the redundant checks are removed from them. This is shared by the data and realtime rmap update items.
--- fs/xfs/xfs_rmap_item.c | 119 ++++++++++++++++++++++++++--------------- 1 file changed, 77 insertions(+), 42 deletions(-)
fs/xfs/xfs_rmap_item.c +77 -42
@@ -578,23 +578,10 @@ xfs_rmap_recover_work(int i;int error =0 ; -/* -* First check the validity of the extents described by the -* RUI. If any are bad, then assume that all are bad and -* just toss the RUI. -*/ -for (i =0 ; i < ruip->rui_format.rui_nextents; i++) { -if (!xfs_rui_validate_map (mp, isrt, - &ruip->rui_format.rui_extents[i])) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - &ruip->rui_format, -sizeof (ruip->rui_format)); -return -EFSCORRUPTED ; - } - +/* The extents were validated when the rui was recovered. */ +for (i =0 ; i < ruip->rui_format.rui_nextents; i++)xfs_rui_recover_work (mp, dfp, isrt, &ruip->rui_format.rui_extents[i]); - } resv =xlog_recover_resv (&M_RES (mp)->tr_itruncate);if (isrt)
@@ -746,34 +733,92 @@ xlog_recover_rui_commit_pass2(struct xfs_mount *mp = log->l_mp;struct xfs_rui_log_item *ruip;struct xfs_rui_log_format *rui_formatp; -size_t len; rui_formatp = item->ri_buf[0 ].iov_base; -if (item->ri_buf[0 ].iov_len <xfs_rui_log_format_sizeof (0 )) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); + ruip =xfs_rui_init (mp,ITEM_TYPE (item), rui_formatp->rui_nextents); +xfs_rui_copy_format (&ruip->rui_format, rui_formatp); +atomic_set (&ruip->rui_next_extent, rui_formatp->rui_nextents); + +xlog_recover_intent_item (log, &ruip->rui_item, lsn, + &xfs_rmap_update_defer_type); +return 0 ; +} + +/* +* The rui is a single region whose length must match its extent count. The +* rmap map extent has fixed width fields, so there is a single on-disk format. +* This is shared by the data and realtime rui items, which have identical +* formats. +*/ +static int +xlog_recover_rui_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +struct xfs_rui_log_format *rui_formatp = + item->ri_buf[region_index].iov_base; +size_t len = item->ri_buf[region_index].iov_len; +uint32_t nextents; + +if (len <xfs_rui_log_format_sizeof (0 )) { +xfs_warn (log->l_mp,"RUI: bad item region length %zu recovered." , + len);return -EFSCORRUPTED ; } - len =xfs_rui_log_format_sizeof (rui_formatp->rui_nextents); -if (item->ri_buf[0 ].iov_len != len) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); +/* +* Bound the extent count before the exact size check below so a corrupt +* count cannot overflow the size calculation. This limit is a fuzzy +* upper bound to prevent overflow; the size check that follows enforces +* the precise length. +*/ + nextents = rui_formatp->rui_nextents; +if (nextents ==0 || + nextents >XLOG_MAX_REGION_SIZE /sizeof (struct xfs_map_extent )) { +xfs_warn (log->l_mp,"RUI: bad extent count %u recovered." , + nextents);return -EFSCORRUPTED ; } - ruip =xfs_rui_init (mp,ITEM_TYPE (item), rui_formatp->rui_nextents); -xfs_rui_copy_format (&ruip->rui_format, rui_formatp); -atomic_set (&ruip->rui_next_extent, rui_formatp->rui_nextents); +if (len !=xfs_rui_log_format_sizeof (nextents)) { +xfs_warn (log->l_mp, +"RUI: item region length %zu does not match %u extents recovered." , + len, nextents); +return -EFSCORRUPTED ; + } + +return 0 ; +} + +/* Validate the rmap update extents in the rui. */ +static int +xlog_recover_rui_validate_item ( +struct xlog *log, +struct xlog_recover_item *item) +{ +struct xfs_rui_log_format *rui_formatp = item->ri_buf[0 ].iov_base; +bool isrt =ITEM_TYPE (item) ==XFS_LI_RUI_RT ; +uint32_t i; + +for (i =0 ; i < rui_formatp->rui_nextents; i++) { +if (!xfs_rui_validate_map (log->l_mp, isrt, + &rui_formatp->rui_extents[i])) { +xfs_warn (log->l_mp,"RUI: bad extent %u recovered." , i); +return -EFSCORRUPTED ; + } + } -xlog_recover_intent_item (log, &ruip->rui_item, lsn, - &xfs_rmap_update_defer_type);return 0 ; }const struct xlog_recover_item_ops xlog_rui_item_ops = { .item_type =XFS_LI_RUI , + .min_hdr_len =sizeof (struct xfs_rui_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_rui_validate_region, + .validate_item = xlog_recover_rui_validate_item, .commit_pass2 = xlog_recover_rui_commit_pass2, };
@@ -788,23 +833,9 @@ xlog_recover_rtrui_commit_pass2(struct xfs_mount *mp = log->l_mp;struct xfs_rui_log_item *ruip;struct xfs_rui_log_format *rui_formatp; -size_t len; rui_formatp = item->ri_buf[0 ].iov_base; -if (item->ri_buf[0 ].iov_len <xfs_rui_log_format_sizeof (0 )) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); -return -EFSCORRUPTED ; - } - - len =xfs_rui_log_format_sizeof (rui_formatp->rui_nextents); -if (item->ri_buf[0 ].iov_len != len) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); -return -EFSCORRUPTED ; - } - ruip =xfs_rui_init (mp,ITEM_TYPE (item), rui_formatp->rui_nextents);xfs_rui_copy_format (&ruip->rui_format, rui_formatp);atomic_set (&ruip->rui_next_extent, rui_formatp->rui_nextents);
@@ -829,6 +860,10 @@ xlog_recover_rtrui_commit_pass2(const struct xlog_recover_item_ops xlog_rtrui_item_ops = { .item_type =XFS_LI_RUI_RT , + .min_hdr_len =sizeof (struct xfs_rui_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_rui_validate_region, + .validate_item = xlog_recover_rui_validate_item, .commit_pass2 = xlog_recover_rtrui_commit_pass2, };
Signature · 1 line
2.55.0
PATCH 23/48 xfs: validate rmap update done items during recovery
In reply to Dave ChinnerAn rmap update done item is a single fixed size region, but recovery only checked its length in the commit handler and never checked its padding.
Validate the region length and require the padding to be zero as the item is decoded. The commit handlers can then trust the item, so the length check is removed from them. This is shared by the data and realtime rmap update done items.
--- fs/xfs/xfs_rmap_item.c | 44 ++++++++++++++++++++++++++++++++---------- 1 file changed, 34 insertions(+), 10 deletions(-)
fs/xfs/xfs_rmap_item.c +34 -10
@@ -884,18 +884,45 @@ xlog_recover_rud_commit_pass2(struct xfs_rud_log_format *rud_formatp; rud_formatp = item->ri_buf[0 ].iov_base; -if (item->ri_buf[0 ].iov_len !=sizeof (struct xfs_rud_log_format )) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , log->l_mp, - rud_formatp, item->ri_buf[0 ].iov_len); +xlog_recover_release_intent (log,XFS_LI_RUI , rud_formatp->rud_rui_id); +return 0 ; +} + +/* +* The rud is a single fixed size region. This is shared by the data and +* realtime rud items, which have identical formats. +*/ +static int +xlog_recover_rud_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +struct xfs_rud_log_format *rud_formatp = + item->ri_buf[region_index].iov_base; + +if (item->ri_buf[region_index].iov_len != +sizeof (struct xfs_rud_log_format )) { +xfs_warn (log->l_mp,"RUD: bad item region length %zu recovered." , + item->ri_buf[region_index].iov_len); +return -EFSCORRUPTED ; + } + +/* The padding is always written as zero. */ +if (rud_formatp->__pad !=0 ) { +xfs_warn (log->l_mp,"RUD: bad padding 0x%x recovered." , + rud_formatp->__pad);return -EFSCORRUPTED ; } -xlog_recover_release_intent (log,XFS_LI_RUI , rud_formatp->rud_rui_id);return 0 ; }const struct xlog_recover_item_ops xlog_rud_item_ops = { .item_type =XFS_LI_RUD , + .min_hdr_len =sizeof (struct xfs_rud_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_rud_validate_region, .commit_pass2 = xlog_recover_rud_commit_pass2, };
@@ -910,12 +937,6 @@ xlog_recover_rtrud_commit_pass2(struct xfs_rud_log_format *rud_formatp; rud_formatp = item->ri_buf[0 ].iov_base; -if (item->ri_buf[0 ].iov_len !=sizeof (struct xfs_rud_log_format )) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , log->l_mp, - rud_formatp, item->ri_buf[0 ].iov_len); -return -EFSCORRUPTED ; - } -xlog_recover_release_intent (log,XFS_LI_RUI_RT , rud_formatp->rud_rui_id);return 0 ;
@@ -926,5 +947,8 @@ xlog_recover_rtrud_commit_pass2(const struct xlog_recover_item_ops xlog_rtrud_item_ops = { .item_type =XFS_LI_RUD_RT , + .min_hdr_len =sizeof (struct xfs_rud_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_rud_validate_region, .commit_pass2 = xlog_recover_rtrud_commit_pass2, };
Signature · 1 line
2.55.0
PATCH 24/48 xfs: validate refcount update intents during recovery
In reply to Dave ChinnerA refcount update intent is a single region item whose length depends on its extent count, and whose extents carry refcount operation state that recovery must trust. These were only checked during intent replay, after the item had been decoded and queued.
Validate the region length as it is decoded, and validate the refcount extents in ->validate_item(). The commit handlers and intent replay can then trust the item, so the redundant checks are removed from them. This is shared by the data and realtime refcount update items.
--- fs/xfs/xfs_refcount_item.c | 119 ++++++++++++++++++++++++------------- 1 file changed, 77 insertions(+), 42 deletions(-)
fs/xfs/xfs_refcount_item.c +77 -42
@@ -513,23 +513,10 @@ xfs_refcount_recover_work(int i;int error =0 ; -/* -* First check the validity of the extents described by the -* CUI. If any are bad, then assume that all are bad and -* just toss the CUI. -*/ -for (i =0 ; i < cuip->cui_format.cui_nextents; i++) { -if (!xfs_cui_validate_phys (mp, isrt, - &cuip->cui_format.cui_extents[i])) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - &cuip->cui_format, -sizeof (cuip->cui_format)); -return -EFSCORRUPTED ; - } - +/* The extents were validated when the cui was recovered. */ +for (i =0 ; i < cuip->cui_format.cui_nextents; i++)xfs_cui_recover_work (mp, dfp, isrt, &cuip->cui_format.cui_extents[i]); - }/* * Under normal operation, refcount updates are deferred, so we
@@ -717,34 +704,92 @@ xlog_recover_cui_commit_pass2(struct xfs_mount *mp = log->l_mp;struct xfs_cui_log_item *cuip;struct xfs_cui_log_format *cui_formatp; -size_t len; cui_formatp = item->ri_buf[0 ].iov_base; -if (item->ri_buf[0 ].iov_len <xfs_cui_log_format_sizeof (0 )) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); + cuip =xfs_cui_init (mp,ITEM_TYPE (item), cui_formatp->cui_nextents); +xfs_cui_copy_format (&cuip->cui_format, cui_formatp); +atomic_set (&cuip->cui_next_extent, cui_formatp->cui_nextents); + +xlog_recover_intent_item (log, &cuip->cui_item, lsn, + &xfs_refcount_update_defer_type); +return 0 ; +} + +/* +* The cui is a single region whose length must match its extent count. The +* refcount phys extent has fixed width fields, so there is a single on-disk +* format. This is shared by the data and realtime cui items, which have +* identical formats. +*/ +static int +xlog_recover_cui_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +struct xfs_cui_log_format *cui_formatp = + item->ri_buf[region_index].iov_base; +size_t len = item->ri_buf[region_index].iov_len; +uint32_t nextents; + +if (len <xfs_cui_log_format_sizeof (0 )) { +xfs_warn (log->l_mp,"CUI: bad item region length %zu recovered." , + len);return -EFSCORRUPTED ; } - len =xfs_cui_log_format_sizeof (cui_formatp->cui_nextents); -if (item->ri_buf[0 ].iov_len != len) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); +/* +* Bound the extent count before the exact size check below so a corrupt +* count cannot overflow the size calculation. This limit is a fuzzy +* upper bound to prevent overflow; the size check that follows enforces +* the precise length. +*/ + nextents = cui_formatp->cui_nextents; +if (nextents ==0 || + nextents >XLOG_MAX_REGION_SIZE /sizeof (struct xfs_phys_extent )) { +xfs_warn (log->l_mp,"CUI: bad extent count %u recovered." , + nextents);return -EFSCORRUPTED ; } - cuip =xfs_cui_init (mp,ITEM_TYPE (item), cui_formatp->cui_nextents); -xfs_cui_copy_format (&cuip->cui_format, cui_formatp); -atomic_set (&cuip->cui_next_extent, cui_formatp->cui_nextents); +if (len !=xfs_cui_log_format_sizeof (nextents)) { +xfs_warn (log->l_mp, +"CUI: item region length %zu does not match %u extents recovered." , + len, nextents); +return -EFSCORRUPTED ; + } + +return 0 ; +} + +/* Validate the refcount update extents in the cui. */ +static int +xlog_recover_cui_validate_item ( +struct xlog *log, +struct xlog_recover_item *item) +{ +struct xfs_cui_log_format *cui_formatp = item->ri_buf[0 ].iov_base; +bool isrt =ITEM_TYPE (item) ==XFS_LI_CUI_RT ; +uint32_t i; + +for (i =0 ; i < cui_formatp->cui_nextents; i++) { +if (!xfs_cui_validate_phys (log->l_mp, isrt, + &cui_formatp->cui_extents[i])) { +xfs_warn (log->l_mp,"CUI: bad extent %u recovered." , i); +return -EFSCORRUPTED ; + } + } -xlog_recover_intent_item (log, &cuip->cui_item, lsn, - &xfs_refcount_update_defer_type);return 0 ; }const struct xlog_recover_item_ops xlog_cui_item_ops = { .item_type =XFS_LI_CUI , + .min_hdr_len =sizeof (struct xfs_cui_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_cui_validate_region, + .validate_item = xlog_recover_cui_validate_item, .commit_pass2 = xlog_recover_cui_commit_pass2, };
@@ -759,23 +804,9 @@ xlog_recover_rtcui_commit_pass2(struct xfs_mount *mp = log->l_mp;struct xfs_cui_log_item *cuip;struct xfs_cui_log_format *cui_formatp; -size_t len; cui_formatp = item->ri_buf[0 ].iov_base; -if (item->ri_buf[0 ].iov_len <xfs_cui_log_format_sizeof (0 )) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); -return -EFSCORRUPTED ; - } - - len =xfs_cui_log_format_sizeof (cui_formatp->cui_nextents); -if (item->ri_buf[0 ].iov_len != len) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); -return -EFSCORRUPTED ; - } - cuip =xfs_cui_init (mp,ITEM_TYPE (item), cui_formatp->cui_nextents);xfs_cui_copy_format (&cuip->cui_format, cui_formatp);atomic_set (&cuip->cui_next_extent, cui_formatp->cui_nextents);
@@ -800,6 +831,10 @@ xlog_recover_rtcui_commit_pass2(const struct xlog_recover_item_ops xlog_rtcui_item_ops = { .item_type =XFS_LI_CUI_RT , + .min_hdr_len =sizeof (struct xfs_cui_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_cui_validate_region, + .validate_item = xlog_recover_cui_validate_item, .commit_pass2 = xlog_recover_rtcui_commit_pass2, };
Signature · 1 line
2.55.0
PATCH 25/48 xfs: validate refcount update done items during recovery
In reply to Dave ChinnerA refcount update done item is a single fixed size region, but recovery only checked its length in the commit handler and never checked its padding.
Validate the region length and require the padding to be zero as the item is decoded. The commit handlers can then trust the item, so the length check is removed from them. This is shared by the data and realtime refcount update done items.
--- fs/xfs/xfs_refcount_item.c | 44 +++++++++++++++++++++++++++++--------- 1 file changed, 34 insertions(+), 10 deletions(-)
fs/xfs/xfs_refcount_item.c +34 -10
@@ -855,18 +855,45 @@ xlog_recover_cud_commit_pass2(struct xfs_cud_log_format *cud_formatp; cud_formatp = item->ri_buf[0 ].iov_base; -if (item->ri_buf[0 ].iov_len !=sizeof (struct xfs_cud_log_format )) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , log->l_mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); +xlog_recover_release_intent (log,XFS_LI_CUI , cud_formatp->cud_cui_id); +return 0 ; +} + +/* +* The cud is a single fixed size region. This is shared by the data and +* realtime cud items, which have identical formats. +*/ +static int +xlog_recover_cud_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +struct xfs_cud_log_format *cud_formatp = + item->ri_buf[region_index].iov_base; + +if (item->ri_buf[region_index].iov_len != +sizeof (struct xfs_cud_log_format )) { +xfs_warn (log->l_mp,"CUD: bad item region length %zu recovered." , + item->ri_buf[region_index].iov_len); +return -EFSCORRUPTED ; + } + +/* The padding is always written as zero. */ +if (cud_formatp->__pad !=0 ) { +xfs_warn (log->l_mp,"CUD: bad padding 0x%x recovered." , + cud_formatp->__pad);return -EFSCORRUPTED ; } -xlog_recover_release_intent (log,XFS_LI_CUI , cud_formatp->cud_cui_id);return 0 ; }const struct xlog_recover_item_ops xlog_cud_item_ops = { .item_type =XFS_LI_CUD , + .min_hdr_len =sizeof (struct xfs_cud_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_cud_validate_region, .commit_pass2 = xlog_recover_cud_commit_pass2, };
@@ -881,12 +908,6 @@ xlog_recover_rtcud_commit_pass2(struct xfs_cud_log_format *cud_formatp; cud_formatp = item->ri_buf[0 ].iov_base; -if (item->ri_buf[0 ].iov_len !=sizeof (struct xfs_cud_log_format )) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , log->l_mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); -return -EFSCORRUPTED ; - } -xlog_recover_release_intent (log,XFS_LI_CUI_RT , cud_formatp->cud_cui_id);return 0 ;
@@ -897,5 +918,8 @@ xlog_recover_rtcud_commit_pass2(const struct xlog_recover_item_ops xlog_rtcud_item_ops = { .item_type =XFS_LI_CUD_RT , + .min_hdr_len =sizeof (struct xfs_cud_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_cud_validate_region, .commit_pass2 = xlog_recover_rtcud_commit_pass2, };
Signature · 1 line
2.55.0
PATCH 26/48 xfs: validate bmap update intents during recovery
In reply to Dave ChinnerA bmap update intent is a single region item carrying one mapping operation whose fork, owner and extent recovery must trust. These were only checked during intent replay, after the item had been decoded and queued.
Validate the region length as it is decoded, and validate the mapping in ->validate_item(). xfs_bui_validate() now takes the format structure so it can run against the decoded region. The commit handler and intent replay can then trust the item, so the redundant checks are removed from them.
--- fs/xfs/xfs_bmap_item.c | 77 +++++++++++++++++++++++++++++------------- 1 file changed, 53 insertions(+), 24 deletions(-)
fs/xfs/xfs_bmap_item.c +53 -24
@@ -418,15 +418,15 @@ xfs_bmap_update_abort_intent(static inline bool xfs_bui_validate (struct xfs_mount *mp, -struct xfs_bui_log_item *buip) +struct xfs_bui_log_format *bui_formatp) {struct xfs_map_extent *map;/* Only one mapping operation per BUI... */ -if (buip->bui_format.bui_nextents !=XFS_BUI_MAX_FAST_EXTENTS ) +if (bui_formatp->bui_nextents !=XFS_BUI_MAX_FAST_EXTENTS )return false; - map = &buip->bui_format.bui_extents[0 ]; + map = &bui_formatp->bui_extents[0 ];if (map->me_flags & ~XFS_BMAP_EXTENT_FLAGS )return false;
@@ -505,12 +505,7 @@ xfs_bmap_recover_work(int iext_delta;int error =0 ; -if (!xfs_bui_validate (mp, buip)) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - &buip->bui_format,sizeof (buip->bui_format)); -return -EFSCORRUPTED ; - } - +/* The bui was validated when it was recovered. */ map = &buip->bui_format.bui_extents[0 ]; work =xfs_bui_recover_work (mp, dfp, &ip, map);if (IS_ERR (work))
@@ -650,40 +645,74 @@ xlog_recover_bui_commit_pass2(struct xfs_mount *mp = log->l_mp;struct xfs_bui_log_item *buip;struct xfs_bui_log_format *bui_formatp; -size_t len; bui_formatp = item->ri_buf[0 ].iov_base; -if (item->ri_buf[0 ].iov_len <xfs_bui_log_format_sizeof (0 )) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); + buip =xfs_bui_init (mp); +xfs_bui_copy_format (&buip->bui_format, bui_formatp); +atomic_set (&buip->bui_next_extent, bui_formatp->bui_nextents); + +xlog_recover_intent_item (log, &buip->bui_item, lsn, + &xfs_bmap_update_defer_type); +return 0 ; +} + +/* +* The bui is a single region with exactly one mapping operation, so its length +* must match a single extent. +*/ +static int +xlog_recover_bui_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +struct xfs_bui_log_format *bui_formatp = + item->ri_buf[region_index].iov_base; +size_t len = item->ri_buf[region_index].iov_len; + +if (len <xfs_bui_log_format_sizeof (0 )) { +xfs_warn (log->l_mp,"BUI: bad item region length %zu recovered." , + len);return -EFSCORRUPTED ; }if (bui_formatp->bui_nextents !=XFS_BUI_MAX_FAST_EXTENTS ) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); +xfs_warn (log->l_mp,"BUI: bad extent count %u recovered." , + bui_formatp->bui_nextents);return -EFSCORRUPTED ; } - len =xfs_bui_log_format_sizeof (bui_formatp->bui_nextents); -if (item->ri_buf[0 ].iov_len != len) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); +if (len !=xfs_bui_log_format_sizeof (bui_formatp->bui_nextents)) { +xfs_warn (log->l_mp, +"BUI: item region length %zu does not match %u extents recovered." , + len, bui_formatp->bui_nextents);return -EFSCORRUPTED ; } - buip =xfs_bui_init (mp); -xfs_bui_copy_format (&buip->bui_format, bui_formatp); -atomic_set (&buip->bui_next_extent, bui_formatp->bui_nextents); +return 0 ; +} + +/* Validate the bmap update mapping in the bui. */ +static int +xlog_recover_bui_validate_item ( +struct xlog *log, +struct xlog_recover_item *item) +{ +if (!xfs_bui_validate (log->l_mp, item->ri_buf[0 ].iov_base)) { +xfs_warn (log->l_mp,"BUI: bad mapping recovered." ); +return -EFSCORRUPTED ; + } -xlog_recover_intent_item (log, &buip->bui_item, lsn, - &xfs_bmap_update_defer_type);return 0 ; }const struct xlog_recover_item_ops xlog_bui_item_ops = { .item_type =XFS_LI_BUI , + .min_hdr_len =sizeof (struct xfs_bui_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_bui_validate_region, + .validate_item = xlog_recover_bui_validate_item, .commit_pass2 = xlog_recover_bui_commit_pass2, };
Signature · 1 line
2.55.0
PATCH 27/48 xfs: validate bmap update done items during recovery
In reply to Dave ChinnerA bmap update done item is a single fixed size region, but recovery only checked its length in the commit handler and never checked its padding.
Validate the region length and require the padding to be zero as the item is decoded. The commit handler can then trust the item, so the length check is removed from it.
--- fs/xfs/xfs_bmap_item.c | 32 ++++++++++++++++++++++++++++---- 1 file changed, 28 insertions(+), 4 deletions(-)
fs/xfs/xfs_bmap_item.c +28 -4
@@ -733,17 +733,41 @@ xlog_recover_bud_commit_pass2(struct xfs_bud_log_format *bud_formatp; bud_formatp = item->ri_buf[0 ].iov_base; -if (item->ri_buf[0 ].iov_len !=sizeof (struct xfs_bud_log_format )) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , log->l_mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); +xlog_recover_release_intent (log,XFS_LI_BUI , bud_formatp->bud_bui_id); +return 0 ; +} + +/* The bud is a single fixed size region. */ +static int +xlog_recover_bud_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +struct xfs_bud_log_format *bud_formatp = + item->ri_buf[region_index].iov_base; + +if (item->ri_buf[region_index].iov_len != +sizeof (struct xfs_bud_log_format )) { +xfs_warn (log->l_mp,"BUD: bad item region length %zu recovered." , + item->ri_buf[region_index].iov_len); +return -EFSCORRUPTED ; + } + +/* The padding is always written as zero. */ +if (bud_formatp->__pad !=0 ) { +xfs_warn (log->l_mp,"BUD: bad padding 0x%x recovered." , + bud_formatp->__pad);return -EFSCORRUPTED ; } -xlog_recover_release_intent (log,XFS_LI_BUI , bud_formatp->bud_bui_id);return 0 ; }const struct xlog_recover_item_ops xlog_bud_item_ops = { .item_type =XFS_LI_BUD , + .min_hdr_len =sizeof (struct xfs_bud_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_bud_validate_region, .commit_pass2 = xlog_recover_bud_commit_pass2, };
Signature · 1 line
2.55.0
PATCH 28/48 xfs: validate mapping exchange intents during recovery
In reply to Dave ChinnerA mapping exchange intent is a single fixed size region describing an exchange between two files, but its contents were only checked during intent replay, after the item had been decoded and queued.
Validate the region length as it is decoded, and validate the exchange parameters in ->validate_item(). xfs_xmi_validate() now takes the format structure so it can run against the decoded region. The commit handler and intent replay can then trust the item, so the redundant checks are removed from them.
--- fs/xfs/xfs_exchmaps_item.c | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-)
fs/xfs/xfs_exchmaps_item.c +3 -11
@@ -324,14 +324,12 @@ xfs_exchmaps_abort_intent(static inline bool xfs_xmi_validate (struct xfs_mount *mp, -struct xfs_xmi_log_item *xmi_lip) +struct xfs_xmi_log_format *xlf) { -struct xfs_xmi_log_format *xlf = &xmi_lip->xmi_format; -if (!xfs_has_exchange_range (mp))return false; -if (xmi_lip->xmi_format.__pad !=0 ) +if (xlf->__pad !=0 )return false;if (xlf->xmi_flags & ~XFS_EXCHMAPS_LOGGED_FLAGS )
@@ -447,13 +445,7 @@ xfs_exchmaps_recover_work(struct xfs_inode *ip1, *ip2;int error =0 ; -if (!xfs_xmi_validate (mp, xmi_lip)) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - &xmi_lip->xmi_format, -sizeof (xmi_lip->xmi_format)); -return -EFSCORRUPTED ; - } - +/* The xmi was validated when it was recovered. */ xmi =xfs_xmi_item_recover_intent (mp, dfp, &xmi_lip->xmi_format, &req, &ip1, &ip2);if (IS_ERR (xmi))
Signature · 1 line
2.55.0
PATCH 29/48 xfs: validate mapping exchange done items during recovery
In reply to Dave ChinnerA mapping exchange done item is a single fixed size region, but recovery only checked its length in the commit handler and never checked its padding.
Validate the region length and require the padding to be zero as the item is decoded. The commit handler can then trust the item, so the length check is removed from it.
--- fs/xfs/xfs_exchmaps_item.c | 32 ++++++++++++++++++++++++++++---- 1 file changed, 28 insertions(+), 4 deletions(-)
fs/xfs/xfs_exchmaps_item.c +28 -4
@@ -607,17 +607,41 @@ xlog_recover_xmd_commit_pass2(struct xfs_xmd_log_format *xmd_formatp; xmd_formatp = item->ri_buf[0 ].iov_base; -if (item->ri_buf[0 ].iov_len !=sizeof (struct xfs_xmd_log_format )) { -XFS_ERROR_REPORT (__func__,XFS_ERRLEVEL_LOW , - -EFSCORRUPTED , log->l_mp); +xlog_recover_release_intent (log,XFS_LI_XMI , xmd_formatp->xmd_xmi_id); +return 0 ; +} + +/* The xmd is a single fixed size region. */ +static int +xlog_recover_xmd_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +struct xfs_xmd_log_format *xmd_formatp = + item->ri_buf[region_index].iov_base; + +if (item->ri_buf[region_index].iov_len != +sizeof (struct xfs_xmd_log_format )) { +xfs_warn (log->l_mp,"XMD: bad item region length %zu recovered." , + item->ri_buf[region_index].iov_len); +return -EFSCORRUPTED ; + } + +/* The padding is always written as zero. */ +if (xmd_formatp->__pad !=0 ) { +xfs_warn (log->l_mp,"XMD: bad padding 0x%x recovered." , + xmd_formatp->__pad);return -EFSCORRUPTED ; } -xlog_recover_release_intent (log,XFS_LI_XMI , xmd_formatp->xmd_xmi_id);return 0 ; }const struct xlog_recover_item_ops xlog_xmd_item_ops = { .item_type =XFS_LI_XMD , + .min_hdr_len =sizeof (struct xfs_xmd_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_xmd_validate_region, .commit_pass2 = xlog_recover_xmd_commit_pass2, };
Signature · 1 line
2.55.0
PATCH 30/48 xfs: validate dquot items during recovery
In reply to Dave ChinnerA dquot item is a format header region followed by the dquot record region. Recovery validated the dquot record with xfs_dquot_verify() but never checked the log format header, and only loosely checked the record region size, all during commit rather than as the item was decoded. The buffer address and offset in the format header were used to read a buffer without any bounds checking.
Validate the region layout and the whole item up front as it is decoded, so commit and readahead can trust the item. The redundant checks are removed from them.
--- fs/xfs/xfs_dquot_item_recover.c | 100 +++++++++++++++++++++++++------- 1 file changed, 78 insertions(+), 22 deletions(-)
fs/xfs/xfs_dquot_item_recover.c +78 -22
@@ -35,11 +35,6 @@ xlog_recover_dquot_ra_pass2(return ; recddq = item->ri_buf[1 ].iov_base; -if (recddq ==NULL ) -return ; -if (item->ri_buf[1 ].iov_len <sizeof (struct xfs_disk_dquot )) -return ; - type = recddq->d_type &XFS_DQTYPE_REC_MASK ;ASSERT (type);if (log->l_quotaoffs_flag & type)
@@ -80,15 +75,6 @@ xlog_recover_dquot_commit_pass2(return 0 ; recddq = item->ri_buf[1 ].iov_base; -if (recddq ==NULL ) { -xfs_alert (log->l_mp,"NULL dquot in %s." , __func__); -return -EFSCORRUPTED ; - } -if (item->ri_buf[1 ].iov_len <sizeof (struct xfs_disk_dquot )) { -xfs_alert (log->l_mp,"dquot too small (%zd) in %s." , - item->ri_buf[1 ].iov_len, __func__); -return -EFSCORRUPTED ; - }/* * This type of quotas was turned off, so ignore this record.
@@ -109,14 +95,6 @@ xlog_recover_dquot_commit_pass2(* removed since the last mount - ENOSYS. */ dq_f = item->ri_buf[0 ].iov_base; -ASSERT (dq_f); - fa =xfs_dquot_verify (mp, recddq, dq_f->qlf_id); -if (fa) { -xfs_alert (mp,"corrupt dquot ID 0x%x in log at %pS" , - dq_f->qlf_id, fa); -return -EFSCORRUPTED ; - } -ASSERT (dq_f->qlf_len ==1 );/* * At this point we are assuming that the dquots have been allocated
@@ -175,8 +153,86 @@ xlog_recover_dquot_commit_pass2(return error; } +/* A dquot item is a format header region followed by the dquot record. */ +static int +xlog_recover_dquot_nregions ( +struct xlog *log, +uint16_t nregions) +{ +if (nregions !=2 ) +return -EFSCORRUPTED ; +return 2 ; +} + +static int +xlog_recover_dquot_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +size_t len = item->ri_buf[region_index].iov_len; +size_t expected; + +if (region_index ==0 ) + expected =sizeof (struct xfs_dq_logformat ); +else + expected =sizeof (struct xfs_disk_dquot ); + +if (len != expected) { +xfs_warn (log->l_mp, +"DQUOT: bad region %d length %zu recovered." , + region_index, len); +return -EFSCORRUPTED ; + } + +return 0 ; +} + +static int +xlog_recover_dquot_validate_item ( +struct xlog *log, +struct xlog_recover_item *item) +{ +struct xfs_mount *mp = log->l_mp; +struct xfs_dq_logformat *dq_f = item->ri_buf[0 ].iov_base; +struct xfs_disk_dquot *ddq = item->ri_buf[1 ].iov_base; +xfs_failaddr_t fa; + +if (dq_f->qlf_len !=XFS_DQUOT_CLUSTER_SIZE_FSB ) { +xfs_warn (mp,"DQUOT: bad buffer length %d recovered." , + dq_f->qlf_len); +return -EFSCORRUPTED ; + } + +if (dq_f->qlf_boffset +sizeof (struct xfs_dqblk ) > +XFS_FSB_TO_B (mp,XFS_DQUOT_CLUSTER_SIZE_FSB )) { +xfs_warn (mp,"DQUOT: bad buffer offset %u recovered." , + dq_f->qlf_boffset); +return -EFSCORRUPTED ; + } + +if (!xfs_verify_fsbno (mp,XFS_DADDR_TO_FSB (mp, dq_f->qlf_blkno))) { +xfs_warn (mp,"DQUOT: bad buffer block 0x%llx recovered." , + dq_f->qlf_blkno); +return -EFSCORRUPTED ; + } + + fa =xfs_dquot_verify (mp, ddq, dq_f->qlf_id); +if (fa) { +xfs_warn (mp,"DQUOT: dquot id 0x%x failed validation at %pS" , + dq_f->qlf_id, fa); +return -EFSCORRUPTED ; + } + +return 0 ; +} +const struct xlog_recover_item_ops xlog_dquot_item_ops = { .item_type =XFS_LI_DQUOT , + .min_hdr_len =sizeof (struct xfs_dq_logformat ), + .validate_nregions = xlog_recover_dquot_nregions, + .validate_region = xlog_recover_dquot_validate_region, + .validate_item = xlog_recover_dquot_validate_item, .ra_pass2 = xlog_recover_dquot_ra_pass2, .commit_pass2 = xlog_recover_dquot_commit_pass2, };
Signature · 1 line
2.55.0
PATCH 31/48 xfs: validate attr done items during recovery
In reply to Dave ChinnerAn attr done item is a single fixed size region, but recovery only checked its length in the commit handler and never checked its padding.
Validate the region length and require the padding to be zero as the item is decoded. The commit handler can then trust the item, so the length check is removed from it.
--- fs/xfs/xfs_attr_item.c | 34 +++++++++++++++++++++++++++++----- 1 file changed, 29 insertions(+), 5 deletions(-)
fs/xfs/xfs_attr_item.c +29 -5
@@ -1167,14 +1167,35 @@ xlog_recover_attrd_commit_pass2(struct xfs_attrd_log_format *attrd_formatp; attrd_formatp = item->ri_buf[0 ].iov_base; -if (item->ri_buf[0 ].iov_len !=sizeof (struct xfs_attrd_log_format )) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , log->l_mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); +xlog_recover_release_intent (log,XFS_LI_ATTRI , + attrd_formatp->alfd_alf_id); +return 0 ; +} + +/* The attrd is a single fixed size region. */ +static int +xlog_recover_attrd_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +struct xfs_attrd_log_format *attrd_formatp = + item->ri_buf[region_index].iov_base; + +if (item->ri_buf[region_index].iov_len != +sizeof (struct xfs_attrd_log_format )) { +xfs_warn (log->l_mp,"ATTRD: bad item region length %zu recovered." , + item->ri_buf[region_index].iov_len); +return -EFSCORRUPTED ; + } + +/* The padding is always written as zero. */ +if (attrd_formatp->__pad !=0 ) { +xfs_warn (log->l_mp,"ATTRD: bad padding 0x%x recovered." , + attrd_formatp->__pad);return -EFSCORRUPTED ; } -xlog_recover_release_intent (log,XFS_LI_ATTRI , - attrd_formatp->alfd_alf_id);return 0 ; }
@@ -1203,5 +1224,8 @@ static const struct xfs_item_ops xfs_attrd_item_ops = {const struct xlog_recover_item_ops xlog_attrd_item_ops = { .item_type =XFS_LI_ATTRD , + .min_hdr_len =sizeof (struct xfs_attrd_log_format ), + .validate_nregions = xlog_recover_single_region, + .validate_region = xlog_recover_attrd_validate_region, .commit_pass2 = xlog_recover_attrd_commit_pass2, };
Signature · 1 line
2.55.0
PATCH 32/48 xfs: validate attr intents during recovery
In reply to Dave ChinnerAn attr intent is a format header region followed by up to four name and value regions, the number and contents of which depend on the operation it describes. Recovery validated all of this inline in the commit handler, interleaved with building the in-core item, so the item was validated only as it was being set up for replay.
Validate the attr intent as it is decoded instead. Region lengths are bounded as the regions are assembled. Exact region length validation is done during item validation. The commit handler is then left to do only the replay setup:
pull the validated name and value regions out of the item and build the in-core intent from them.
--- fs/xfs/xfs_attr_item.c | 249 +++++++++++++++++++++++++---------------- 1 file changed, 151 insertions(+), 98 deletions(-)
fs/xfs/xfs_attr_item.c +151 -98
@@ -1002,151 +1002,200 @@ xlog_recover_attri_commit_pass2( {struct xfs_mount *mp = log->l_mp;struct xfs_attri_log_item *attrip; -struct xfs_attri_log_format *attri_formatp; +struct xfs_attri_log_format *attri_formatp = item->ri_buf[0 ].iov_base;struct xfs_attri_log_nameval *nv;const void *attr_name;const void *attr_value =NULL ;const void *attr_new_name =NULL ;const void *attr_new_value =NULL ; -size_t len;unsigned int name_len =0 ;unsigned int value_len =0 ;unsigned int new_name_len =0 ;unsigned int new_value_len =0 ; -unsigned int op, i =0 ; +unsigned int i =1 ; -/* Validate xfs_attri_log_format before the large memory allocation */ - len =sizeof (struct xfs_attri_log_format ); -if (item->ri_buf[i].iov_len != len) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len); -return -EFSCORRUPTED ; +/* +* The item has already been validated, so pull the name and value +* lengths out of the format header. The regions present depend on the +* operation being performed. +*/ +switch (xfs_attr_log_item_op (attri_formatp)) { +case XFS_ATTRI_OP_FLAGS_PPTR_REMOVE : +case XFS_ATTRI_OP_FLAGS_PPTR_SET : +case XFS_ATTRI_OP_FLAGS_SET : +case XFS_ATTRI_OP_FLAGS_REPLACE : + name_len = attri_formatp->alfi_name_len; + value_len = attri_formatp->alfi_value_len; +break ; +case XFS_ATTRI_OP_FLAGS_REMOVE : + name_len = attri_formatp->alfi_name_len; +break ; +case XFS_ATTRI_OP_FLAGS_PPTR_REPLACE : + name_len = attri_formatp->alfi_old_name_len; + new_name_len = attri_formatp->alfi_new_name_len; + new_value_len = value_len = attri_formatp->alfi_value_len; +break ; } - attri_formatp = item->ri_buf[i].iov_base; -if (!xfs_attri_validate (mp, attri_formatp)) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - attri_formatp, len); +/* Pull the name and value regions out in the order they are logged. */ + attr_name = item->ri_buf[i++].iov_base; +if (new_name_len >0 ) + attr_new_name = item->ri_buf[i++].iov_base; +if (value_len >0 ) + attr_value = item->ri_buf[i++].iov_base; +if (new_value_len >0 ) + attr_new_value = item->ri_buf[i++].iov_base; + +/* +* Memory alloc failure will cause replay to abort. We attach the +* name/value buffer to the recovered incore log item and drop our +* reference. +*/ + nv =xfs_attri_log_nameval_alloc (attr_name, name_len, + attr_new_name, new_name_len, + attr_value, value_len, + attr_new_value, new_value_len); + + attrip =xfs_attri_init (mp, nv); +memcpy (&attrip->attri_format, attri_formatp, +sizeof (struct xfs_attri_log_format )); + +xlog_recover_intent_item (log, &attrip->attri_item, lsn, + &xfs_attr_defer_type); +xfs_attri_log_nameval_put (nv); +return 0 ; +} + +/* +* An attri has a format header region followed by up to four name and value +* regions, the exact number depending on the operation it describes. The exact +* count for the operation is checked once the format header is decoded in +* ->validate_item(); here we only bound the count to the possible range. +*/ +static int +xlog_recover_attri_nregions ( +struct xlog *log, +uint16_t nregions) +{ +if (nregions <2 || nregions >5 )return -EFSCORRUPTED ; +return nregions; +} + +/* +* Bound each region by its largest possible size. Region 0 is the fixed size +* format header. Region 1 is always a name, so no larger than a maximum length +* attr name. Regions 2 to 4 may be a name or a value, so no larger than a +* maximum length attr value. This detects an unbounded continuation of a region +* before the exact per-operation sizes are checked in ->validate_item(). +*/ +static int +xlog_recover_attri_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +size_t len = item->ri_buf[region_index].iov_len; + +if (region_index ==0 ) { +if (len !=sizeof (struct xfs_attri_log_format )) + goto out_error; +return 0 ; } -/* Check the number of log iovecs makes sense for the op code. */ - op =xfs_attr_log_item_op (attri_formatp); -switch (op) { +if (region_index ==1 ) { +if (len >xlog_calc_iovec_len (XATTR_NAME_MAX )) + goto out_error; +return 0 ; + } + +if (len >xlog_calc_iovec_len (XATTR_SIZE_MAX )) + goto out_error; +return 0 ; + +out_error: +xfs_warn (log->l_mp,"ATTRI: bad region %d length %zu recovered." , + region_index, len); +return -EFSCORRUPTED ; +} + +/* +* Validate the whole attri: the format header, the exact region count for the +* operation it describes, and the name and value regions that operation +* requires. Once this passes, the rest of recovery can walk the regions without +* checking them. +*/ +static int +xlog_recover_attri_validate_item ( +struct xlog *log, +struct xlog_recover_item *item) +{ +struct xfs_mount *mp = log->l_mp; +struct xfs_attri_log_format *attri_formatp = item->ri_buf[0 ].iov_base; +unsigned int name_len =0 ; +unsigned int value_len =0 ; +unsigned int new_name_len =0 ; +unsigned int new_value_len =0 ; +unsigned int i =1 ; + +if (!xfs_attri_validate (mp, attri_formatp)) +return -EFSCORRUPTED ; + +/* Check the number of regions makes sense for the op code. */ +switch (xfs_attr_log_item_op (attri_formatp)) {case XFS_ATTRI_OP_FLAGS_PPTR_REMOVE :case XFS_ATTRI_OP_FLAGS_PPTR_SET : -/* Log item, attr name, attr value */ -if (item->ri_total !=3 ) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - attri_formatp, len); +/* Format header, attr name, attr value */ +if (item->ri_total !=3 )return -EFSCORRUPTED ; - } name_len = attri_formatp->alfi_name_len; value_len = attri_formatp->alfi_value_len;break ;case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE : -/* Log item, attr name, optional attr value */ -if (item->ri_total !=2 + !!attri_formatp->alfi_value_len) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - attri_formatp, len); +/* Format header, attr name, optional attr value */ +if (item->ri_total != (2 + !!attri_formatp->alfi_value_len))return -EFSCORRUPTED ; - } name_len = attri_formatp->alfi_name_len; value_len = attri_formatp->alfi_value_len;break ;case XFS_ATTRI_OP_FLAGS_REMOVE : -/* Log item, attr name */ -if (item->ri_total !=2 ) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - attri_formatp, len); +/* Format header, attr name */ +if (item->ri_total !=2 )return -EFSCORRUPTED ; - } name_len = attri_formatp->alfi_name_len;break ;case XFS_ATTRI_OP_FLAGS_PPTR_REPLACE : -/* -* Log item, attr name, new attr name, attr value, new attr -* value -*/ -if (item->ri_total !=5 ) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - attri_formatp, len); +/* Format header, attr name, new name, attr value, new value */ +if (item->ri_total !=5 )return -EFSCORRUPTED ; - } name_len = attri_formatp->alfi_old_name_len; new_name_len = attri_formatp->alfi_new_name_len; new_value_len = value_len = attri_formatp->alfi_value_len;break ;default : -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - attri_formatp, len);return -EFSCORRUPTED ; } - i++; -/* Validate the attr name */ - attr_name =xfs_attri_validate_name_iovec (mp, attri_formatp, - &item->ri_buf[i], name_len); -if (!attr_name) +if (!xfs_attri_validate_name_iovec (mp, attri_formatp, &item->ri_buf[i++], + name_len))return -EFSCORRUPTED ; - i++; - -/* Validate the new attr name */ -if (new_name_len >0 ) { - attr_new_name =xfs_attri_validate_name_iovec (mp, - attri_formatp, &item->ri_buf[i], - new_name_len); -if (!attr_new_name) -return -EFSCORRUPTED ; - i++; - } - -/* Validate the attr value, if present */ -if (value_len !=0 ) { - attr_value =xfs_attri_validate_value_iovec (mp, attri_formatp, - &item->ri_buf[i], value_len); -if (!attr_value) -return -EFSCORRUPTED ; - i++; - } - -/* Validate the new attr value, if present */ -if (new_value_len !=0 ) { - attr_new_value =xfs_attri_validate_value_iovec (mp, - attri_formatp, &item->ri_buf[i], - new_value_len); -if (!attr_new_value) -return -EFSCORRUPTED ; - i++; - } -/* -* Make sure we got the correct number of buffers for the operation -* that we just loaded. -*/ -if (i != item->ri_total) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, - attri_formatp, len); +if (new_name_len >0 && + !xfs_attri_validate_name_iovec (mp, attri_formatp, &item->ri_buf[i++], + new_name_len))return -EFSCORRUPTED ; - } -/* -* Memory alloc failure will cause replay to abort. We attach the -* name/value buffer to the recovered incore log item and drop our -* reference. -*/ - nv =xfs_attri_log_nameval_alloc (attr_name, name_len, - attr_new_name, new_name_len, - attr_value, value_len, - attr_new_value, new_value_len); +if (value_len >0 && + !xfs_attri_validate_value_iovec (mp, attri_formatp, &item->ri_buf[i++], + value_len)) +return -EFSCORRUPTED ; - attrip =xfs_attri_init (mp, nv); -memcpy (&attrip->attri_format, attri_formatp, len); +if (new_value_len >0 && + !xfs_attri_validate_value_iovec (mp, attri_formatp, &item->ri_buf[i++], + new_value_len)) +return -EFSCORRUPTED ; -xlog_recover_intent_item (log, &attrip->attri_item, lsn, - &xfs_attr_defer_type); -xfs_attri_log_nameval_put (nv);return 0 ; }
@@ -1210,6 +1259,10 @@ static const struct xfs_item_ops xfs_attri_item_ops = {const struct xlog_recover_item_ops xlog_attri_item_ops = { .item_type =XFS_LI_ATTRI , + .min_hdr_len =sizeof (struct xfs_attri_log_format ), + .validate_nregions = xlog_recover_attri_nregions, + .validate_region = xlog_recover_attri_validate_region, + .validate_item = xlog_recover_attri_validate_item, .commit_pass2 = xlog_recover_attri_commit_pass2, };
Signature · 1 line
2.55.0
PATCH 33/48 xfs: convert inode items to host format during recovery
In reply to Dave ChinnerThe inode item format header is logged in either the host format or the older 32 bit on-disk format. Recovery converted between them in the commit handler and carried a separately allocated copy through the rest of the function.
Convert the format header to host format in ->validate_item() as the item is decoded, so the rest of recovery treats region 0 as a host format header without regard for the on-disk format. The commit handler and readahead now use the header directly. The separate allocation and its cleanup are removed.
--- fs/xfs/xfs_inode_item_recover.c | 108 +++++++++++++++++++++++--------- 1 file changed, 79 insertions(+), 29 deletions(-)
fs/xfs/xfs_inode_item_recover.c +79 -29
@@ -30,17 +30,10 @@ xlog_recover_inode_ra_pass2(struct xlog *log, struct xlog_recover_item *item) { -if (item->ri_buf[0 ].iov_len ==sizeof (struct xfs_inode_log_format )) { -struct xfs_inode_log_format *ilfp = item->ri_buf[0 ].iov_base; + struct xfs_inode_log_format *ilfp = item->ri_buf[0].iov_base; -xlog_buf_readahead (log, ilfp->ilf_blkno, ilfp->ilf_len, - &xfs_inode_buf_ra_ops); - }else { -struct xfs_inode_log_format_32 *ilfp = item->ri_buf[0 ].iov_base; - -xlog_buf_readahead (log, ilfp->ilf_blkno, ilfp->ilf_len, - &xfs_inode_buf_ra_ops); - } +xlog_buf_readahead (log, ilfp->ilf_blkno, ilfp->ilf_len, + &xfs_inode_buf_ra_ops); }/*
@@ -304,6 +297,75 @@ xlog_recover_inode_dbroot(return 0 ; } +/* +* An inode item is a format header region, the log dinode core, and up to two +* fork regions, so it has two to four regions. The exact count depends on the +* fields logged and is checked once the format header is decoded. +*/ +static int +xlog_recover_inode_nregions ( +struct xlog *log, +uint16_t nregions) +{ +if (nregions <2 || nregions >4 ) +return -EFSCORRUPTED ; +return nregions; +} + +/* +* The format header (region 0) is logged in either the host format or the 32 +* bit on-disk format, so it must match one of those two sizes. +*/ +static int +xlog_recover_inode_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +size_t len = item->ri_buf[region_index].iov_len; + +if (region_index >0 ) +return 0 ; + +if (len !=sizeof (struct xfs_inode_log_format ) && + len !=sizeof (struct xfs_inode_log_format_32 )) { +xfs_warn (log->l_mp, +"INODE: bad format region length %zu recovered." , len); +return -EFSCORRUPTED ; + } + +return 0 ; +} + +/* +* Convert the format header to host format so the rest of recovery can treat +* region 0 as a struct xfs_inode_log_format without regard for the on-disk +* format. A header already in host format is left in place. +*/ +static int +xlog_recover_inode_validate_item ( +struct xlog *log, +struct xlog_recover_item *item) +{ +struct xfs_inode_log_format *in_f; +int error; + +if (item->ri_buf[0 ].iov_len ==sizeof (struct xfs_inode_log_format )) +return 0 ; + + in_f = kmalloc_obj(struct xfs_inode_log_format, GFP_KERNEL | __GFP_NOFAIL); + error =xfs_inode_item_format_convert (&item->ri_buf[0 ], in_f); +if (error) { +kfree (in_f); +return error; + } + +kvfree (item->ri_buf[0 ].iov_base); + item->ri_buf[0 ].iov_base = in_f; + item->ri_buf[0 ].iov_len =sizeof (struct xfs_inode_log_format ); +return 0 ; +} + STATIC int xlog_recover_inode_commit_pass2( struct xlog *log,
@@ -311,7 +373,7 @@ xlog_recover_inode_commit_pass2( struct xlog_recover_item *item, xfs_lsn_t current_lsn) { -struct xfs_inode_log_format *in_f; +struct xfs_inode_log_format *in_f = item->ri_buf[0 ].iov_base;struct xfs_mount *mp = log->l_mp;struct xfs_buf *bp;struct xfs_dinode *dip;
@@ -323,35 +385,22 @@ xlog_recover_inode_commit_pass2(uint fields;struct xfs_log_dinode *ldip;uint isize; -int need_free =0 ;xfs_failaddr_t fa; -if (item->ri_buf[0 ].iov_len ==sizeof (struct xfs_inode_log_format )) { - in_f = item->ri_buf[0 ].iov_base; - }else { - in_f = kmalloc_obj(struct xfs_inode_log_format, -GFP_KERNEL | __GFP_NOFAIL); - need_free =1 ; - error =xfs_inode_item_format_convert (&item->ri_buf[0 ], in_f); -if (error) - goto error; - } -/* * Inode buffers can be freed, look out for it, * and do not replay the inode. */ if (xlog_is_buffer_cancelled (log, in_f->ilf_blkno, in_f->ilf_len)) { - error =0 ;trace_xfs_log_recover_inode_cancel (log, in_f); - goto error; +return 0 ; }trace_xfs_log_recover_inode_recover (log, in_f); error =xfs_buf_read (mp->m_ddev_targp, in_f->ilf_blkno, in_f->ilf_len,0 , &bp, &xfs_inode_buf_ops);if (error) - goto error; +return error;ASSERT (in_f->ilf_fields &XFS_ILOG_CORE ); dip =xfs_buf_offset (bp, in_f->ilf_boffset);
@@ -608,14 +657,15 @@ xlog_recover_inode_commit_pass2( out_release:xfs_buf_relse (bp); -error: -if (need_free) -kfree (in_f);return error; }const struct xlog_recover_item_ops xlog_inode_item_ops = { .item_type =XFS_LI_INODE , + .min_hdr_len =sizeof (struct xfs_inode_log_format_32 ), + .validate_nregions = xlog_recover_inode_nregions, + .validate_region = xlog_recover_inode_validate_region, + .validate_item = xlog_recover_inode_validate_item, .ra_pass2 = xlog_recover_inode_ra_pass2, .commit_pass2 = xlog_recover_inode_commit_pass2, };
Signature · 1 line
2.55.0
PATCH 34/48 xfs: allow xfs_imap() to map without an inode btree lookup
In reply to Dave Chinnerxfs_imap() maps an inode number to its cluster buffer location. For the common inode cluster geometries it does this by arithmetic, but for the general case it falls back to an inode btree lookup that reads the AGI and walks the inode btree.
Some callers cannot allow that lookup. Log recovery in particular wants to cross-check a logged inode buffer location against the one derived from the inode number, but it runs before the filesystem is fully available and must not issue btree lookups.
Add an XFS_IMAP_NO_LOOKUP flag. When it is set, xfs_imap() maps the inode by arithmetic alone and returns -EAGAIN instead of performing the btree lookup, so the caller can tell that arithmetic mapping was not possible for this geometry.
--- fs/xfs/libxfs/xfs_ialloc.c | 7 +++++++ fs/xfs/xfs_icache.h | 2 ++ 2 files changed, 9 insertions(+)
fs/xfs/libxfs/xfs_ialloc.c +7 -0
@@ -2449,6 +2449,10 @@ xfs_imap_lookup(/* * Return the location of the inode in imap, for mapping it into a buffer. +* +* If the caller cannot allow a btree lookup to be performed, then it should set +* XFS_IMAP_NO_LOOKUP. If arithmetic mapping cannot be performed, we return +* -EAGAIN to inform the caller mapping failed due to a lookup being required. */ int xfs_imap(
@@ -2536,6 +2540,9 @@ xfs_imap(*/ } +if (flags &XFS_IMAP_NO_LOOKUP ) +return -EAGAIN ; + error =xfs_imap_lookup (pag, tp, agino, agbno, &chunk_agbno, &offset_agbno, flags);if (error)
fs/xfs/xfs_icache.h +2 -0
@@ -41,6 +41,8 @@ struct xfs_icwalk {#define XFS_IGET_INCORE (1U << 3)/* Return -EAGAIN immediately if the inode is unavailable. */ #define XFS_IGET_NORETRY (1U << 4) +/* Map the inode by arithmetic only, never by an inode btree lookup. */ +#define XFS_IMAP_NO_LOOKUP (1U << 5)int xfs_iget (struct xfs_mount *mp,struct xfs_trans *tp,xfs_ino_t ino,uint flags,uint lock_flags,xfs_inode_t **ipp);
Signature · 1 line
2.55.0
PATCH 35/48 xfs: validate the inode item format header during recovery
In reply to Dave ChinnerThe inode item format header describes how many regions the item has and where the inode lives on disk. Recovery trusted these values, guarding only a few of them with asserts in the commit handler while walking the regions and reading the inode buffer.
Validate the format header as the item is decoded, so the rest of recovery can trust the region count and buffer location it describes. The header is self-consistent with the fields it logs, and the buffer location is fixed by the inode number, so both are checked here. Deriving the location uses xfs_imap() with XFS_IMAP_NO_LOOKUP so recovery never issues an inode btree lookup.
The host format conversion is split into its own helper so validation always runs against a host format header.
--- fs/xfs/xfs_inode_item_recover.c | 123 ++++++++++++++++++++++++++++++-- 1 file changed, 116 insertions(+), 7 deletions(-)
fs/xfs/xfs_inode_item_recover.c +116 -7
@@ -21,6 +21,8 @@#include "xfs_log_priv.h" #include "xfs_log_recover.h" #include "xfs_icache.h" +#include "xfs_ialloc.h" +#include "xfs_ag.h" #include "xfs_bmap_btree.h" #include "xfs_rtrmap_btree.h" #include "xfs_rtrefcount_btree.h"
@@ -343,29 +345,136 @@ xlog_recover_inode_validate_region(* format. A header already in host format is left in place. */ static int -xlog_recover_inode_validate_item ( -struct xlog *log, +xlog_recover_inode_convert_format (struct xlog_recover_item *item) { -struct xfs_inode_log_format *in_f; +struct xfs_inode_log_format *ilfp;int error;if (item->ri_buf[0 ].iov_len ==sizeof (struct xfs_inode_log_format ))return 0 ; - in_f = kmalloc_obj(struct xfs_inode_log_format, GFP_KERNEL | __GFP_NOFAIL); - error =xfs_inode_item_format_convert (&item->ri_buf[0 ], in_f); + ilfp = kmalloc_obj(struct xfs_inode_log_format, GFP_KERNEL | __GFP_NOFAIL); + error =xfs_inode_item_format_convert (&item->ri_buf[0 ], ilfp);if (error) { -kfree (in_f); +kfree (ilfp);return error; }kvfree (item->ri_buf[0 ].iov_base); - item->ri_buf[0 ].iov_base = in_f; + item->ri_buf[0 ].iov_base = ilfp; item->ri_buf[0 ].iov_len =sizeof (struct xfs_inode_log_format );return 0 ; } +/* +* The buffer location logged in the format header is fixed by the inode number, +* so derive it and check the header matches. The inode cluster geometry is not +* always resolvable by arithmetic, and a btree lookup cannot be run this early +* in recovery, so an unresolvable geometry only checks the buffer length. +*/ +static int +xlog_recover_inode_check_location ( +struct xlog *log, +struct xfs_inode_log_format *ilfp) +{ +struct xfs_mount *mp = log->l_mp; +struct xfs_perag *pag; +struct xfs_imap imap; +xfs_daddr_t blkno; +int len; +int error; + + len =XFS_FSB_TO_BB (mp,M_IGEO (mp)->blocks_per_cluster); +if (ilfp->ilf_len != len) { +xfs_warn (mp, +"INODE: inode 0x%llx buffer length 0x%x does not match expected 0x%x recovered." , + ilfp->ilf_ino, ilfp->ilf_len, len); +return -EFSCORRUPTED ; + } + + pag =xfs_perag_get (mp,XFS_INO_TO_AGNO (mp, ilfp->ilf_ino)); + error =xfs_imap (pag,NULL , ilfp->ilf_ino, &imap,XFS_IMAP_NO_LOOKUP ); +xfs_perag_put (pag); +if (error == -EAGAIN ) +return 0 ; +if (error) +return error; + + blkno =XFS_AGB_TO_DADDR (mp,XFS_INO_TO_AGNO (mp, ilfp->ilf_ino), + imap.im_agbno); +if (ilfp->ilf_blkno != blkno) { +xfs_warn (mp, +"INODE: inode 0x%llx buffer block 0x%llx does not match expected 0x%llx recovered." , + ilfp->ilf_ino, ilfp->ilf_blkno, blkno); +return -EFSCORRUPTED ; + } +if (ilfp->ilf_boffset != imap.im_boffset) { +xfs_warn (mp, +"INODE: inode 0x%llx buffer offset 0x%x does not match expected 0x%x recovered." , + ilfp->ilf_ino, ilfp->ilf_boffset, imap.im_boffset); +return -EFSCORRUPTED ; + } + +return 0 ; +} + +/* +* Validate the format header once it is in host format. The item size records +* how many regions the item should have, the set of logged fields fixes what +* that count must be, and the inode number fixes the buffer location, so all of +* these are checked against the header here. +*/ +static int +xlog_recover_inode_validate_item ( +struct xlog *log, +struct xlog_recover_item *item) +{ +struct xfs_inode_log_format *ilfp; +unsigned int nregions; +int error; + + error =xlog_recover_inode_convert_format (item); +if (error) +return error; + ilfp = item->ri_buf[0 ].iov_base; + +if (!xfs_verify_ino (log->l_mp, ilfp->ilf_ino)) { +xfs_warn (log->l_mp,"INODE: bad inode number 0x%llx recovered." , + ilfp->ilf_ino); +return -EFSCORRUPTED ; + } + +if (!(ilfp->ilf_fields &XFS_ILOG_CORE )) { +xfs_warn (log->l_mp, +"INODE: inode 0x%llx has no core logged, fields 0x%x recovered." , + ilfp->ilf_ino, ilfp->ilf_fields); +return -EFSCORRUPTED ; + } + +if (ilfp->ilf_size != item->ri_total) { +xfs_warn (log->l_mp, +"INODE: inode 0x%llx item size %u does not match region count %u recovered." , + ilfp->ilf_ino, ilfp->ilf_size, item->ri_total); +return -EFSCORRUPTED ; + } + + nregions =2 ; +if (ilfp->ilf_fields &XFS_ILOG_DFORK ) + nregions++; +if (ilfp->ilf_fields &XFS_ILOG_AFORK ) + nregions++; +if (ilfp->ilf_size != nregions) { +xfs_warn (log->l_mp, +"INODE: inode 0x%llx item size %u does not match region count %u the fields 0x%x define." , + ilfp->ilf_ino, ilfp->ilf_size, nregions, + ilfp->ilf_fields); +return -EFSCORRUPTED ; + } + +return xlog_recover_inode_check_location (log, ilfp); +} + STATIC int xlog_recover_inode_commit_pass2( struct xlog *log,
Signature · 1 line
2.55.0
PATCH 36/48 xfs: validate the logged inode core during recovery
In reply to Dave ChinnerThe log dinode core carries the inode fields being recovered into the on-disk inode. Recovery validated only a handful of these fields, scattered through the commit handler as it set up the replay, so most of the core was trusted and written back without being checked.
Add xfs_log_dinode_verify() that verifies the fields carried in the log dinode core. Call it during item validation once the core region length has been checked, so the whole core is trusted before replay begins.
The fork offset check is common to the on-disk and log dinode, so xfs_dinode_verify_forkoff() is recast as xfs_inode_verify_forkoff() taking the offset and format as arguments and shared between them. The extent count helper already present in recovery is converted to the failaddr idiom so it composes as a sub-check of the new verifier.
The core checks the commit handler performed are removed, and the core region length is now checked exactly against the size the running filesystem logs. The on-disk inode magic check in the commit handler is also removed as the inode buffer verifier run by the buffer read already performs it.
--- fs/xfs/libxfs/xfs_inode_buf.c | 19 +- fs/xfs/libxfs/xfs_inode_buf.h | 2 + fs/xfs/xfs_inode_item_recover.c | 362 ++++++++++++++++++++++---------- 3 files changed, 259 insertions(+), 124 deletions(-)
fs/xfs/libxfs/xfs_inode_buf.c +10 -9
@@ -477,17 +477,18 @@ xfs_dinode_verify_fork(return NULL ; } -static xfs_failaddr_t -xfs_dinode_verify_forkoff ( -struct xfs_dinode *dip, -struct xfs_mount *mp) +xfs_failaddr_t +xfs_inode_verify_forkoff ( +struct xfs_mount *mp, +uint8_t forkoff, +uint8_t format) { -if (!dip->di_forkoff) +if (!forkoff)return NULL ; -switch (dip->di_format) { +switch (format) {case XFS_DINODE_FMT_DEV : -if (dip->di_forkoff != (roundup (sizeof (xfs_dev_t),8 ) >>3 )) +if (forkoff != (roundup (sizeof (xfs_dev_t),8 ) >>3 ))return __this_address;break ;case XFS_DINODE_FMT_META_BTREE :
@@ -497,7 +498,7 @@ xfs_dinode_verify_forkoff(case XFS_DINODE_FMT_LOCAL :/* fall through ... */ case XFS_DINODE_FMT_EXTENTS :/* fall through ... */ case XFS_DINODE_FMT_BTREE : -if (dip->di_forkoff >= (XFS_LITINO (mp) >>3 )) +if (forkoff >= (XFS_LITINO (mp) >>3 ))return __this_address;break ;default :
@@ -683,7 +684,7 @@ xfs_dinode_verify(return __this_address;/* check for illegal values of forkoff */ - fa =xfs_dinode_verify_forkoff (dip, mp); + fa =xfs_inode_verify_forkoff (mp, dip->di_forkoff, dip->di_format);if (fa)return fa;
fs/xfs/libxfs/xfs_inode_buf.h +2 -0
@@ -35,6 +35,8 @@ xfs_failaddr_t xfs_inode_validate_extsize(struct xfs_mount *mp,xfs_failaddr_t xfs_inode_validate_cowextsize (struct xfs_mount *mp,uint32_t cowextsize,uint16_t mode,uint16_t flags,uint64_t flags2); +xfs_failaddr_t xfs_inode_verify_forkoff (struct xfs_mount *mp,uint8_t forkoff, +uint8_t format);static inline uint64_t xfs_inode_encode_bigtime (struct timespec64 tv) {
fs/xfs/xfs_inode_item_recover.c +247 -115
443 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 37/48 xfs: validate the logged inode forks during recovery
In reply to Dave ChinnerAn inode item logs a data and/or attribute fork region alongside the core. The commit handler trusted these regions. It guarded them only with asserts as it copied each fork into the on-disk inode. A fork whose format, size or placement did not agree with the core was replayed regardless.
Verify the logged forks during item validation, once the core has been verified, so the fork is trusted before replay begins. The contents of the fork regions are not inspected here. A device inode is handled separately as it holds its rdev in the format header and logs no data fork region. The asserts the commit handler used are removed.
The fork size calculation was duplicated between the on-disk inode macros and the in-memory inode helpers. Add __xfs_inode_fork_size(), keyed on the scalar fork offset, and route the macros, the in-memory helpers and the new log dinode check through it so the calculation exists in one place.
--- fs/xfs/libxfs/xfs_format.h | 8 +- fs/xfs/libxfs/xfs_inode_fork.c | 2 +- fs/xfs/xfs_inode.h | 43 +++++--- fs/xfs/xfs_inode_item_recover.c | 168 +++++++++++++++++++++++++++++--- 4 files changed, 185 insertions(+), 36 deletions(-)
fs/xfs/libxfs/xfs_format.h +3 -5
@@ -1099,13 +1099,11 @@ enum xfs_dinode_fmt {#define XFS_DFORK_BOFF (dip) ((int)((dip)->di_forkoff << 3))#define XFS_DFORK_DSIZE (dip,mp) \ - ((dip)->di_forkoff ? XFS_DFORK_BOFF(dip) : XFS_LITINO(mp)) + __xfs_inode_fork_size(mp, (dip)->di_forkoff, XFS_DATA_FORK)#define XFS_DFORK_ASIZE (dip,mp) \ - ((dip)->di_forkoff ? XFS_LITINO(mp) - XFS_DFORK_BOFF(dip) : 0) + __xfs_inode_fork_size(mp, (dip)->di_forkoff, XFS_ATTR_FORK)#define XFS_DFORK_SIZE (dip,mp,w) \ - ((w) == XFS_DATA_FORK ? \ - XFS_DFORK_DSIZE(dip, mp) : \ - XFS_DFORK_ASIZE(dip, mp)) + __xfs_inode_fork_size(mp, (dip)->di_forkoff, w)#define XFS_DFORK_MAXEXT (dip, mp, w) \ (XFS_DFORK_SIZE(dip, mp, w) / sizeof(struct xfs_bmbt_rec))
fs/xfs/libxfs/xfs_inode_fork.c +1 -1
@@ -86,7 +86,7 @@ xfs_iformat_local(*/ if (unlikely (size >XFS_DFORK_SIZE (dip, ip->i_mount, whichfork))) {xfs_warn (ip->i_mount, -"corrupt inode %llu (bad size %d for local fork, size = %zd)." , +"corrupt inode %llu (bad size %d for local fork, size = %u)." , (unsigned long long )I_INO (ip), size,XFS_DFORK_SIZE (dip, ip->i_mount, whichfork)); xfs_inode_verifier_error(ip, -EFSCORRUPTED,
fs/xfs/xfs_inode.h +28 -15
61 lines not on this page · raw message
fs/xfs/xfs_inode_item_recover.c +153 -15
204 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 38/48 xfs: validate logged local fork contents during recovery
In reply to Dave ChinnerA local format fork carries a shortform directory, an extended attribute structure or a symlink target directly in the inode. Recovery copied these into the inode without inspecting them, so a corrupt shortform structure was replayed as it stood.
Validate the contents of a logged local fork once the fork has been verified against the core. The shortform directory and attribute verifiers are reused directly as they already operate on the on-disk form. The logged symlink target is validated by a recovery local helper because the shared verifier expects the null terminated in-core form, which the logged target does not have.
--- fs/xfs/xfs_inode_item_recover.c | 53 +++++++++++++++++++++++++++++---- 1 file changed, 48 insertions(+), 5 deletions(-)
fs/xfs/xfs_inode_item_recover.c +48 -5
@@ -24,6 +24,8 @@#include "xfs_ialloc.h" #include "xfs_ag.h" #include "xfs_dir2.h" +#include "xfs_dir2_priv.h" +#include "xfs_attr_leaf.h" #include "xfs_metafile.h" #include "xfs_bmap_btree.h" #include "xfs_rtrmap_btree.h"
@@ -441,6 +443,45 @@ xlog_dinode_verify(return NULL ; } +/* +* Verify a logged shortform symlink target. The logged target is the on-disk +* form, which is not null terminated, so this cannot use +* xfs_symlink_shortform_verify(). +*/ +static xfs_failaddr_t +xlog_dinode_verify_symlink ( +void *sfp, +unsigned int size) +{ +if (size ==0 || size >XFS_SYMLINK_MAXLEN ) +return __this_address; + +/* The target must not contain a null. */ +if (memchr (sfp,0 , size)) +return __this_address; + +return NULL ; +} + +/* +* Verify the contents of a logged local format data fork. A directory holds a +* shortform directory, and a symlink holds its target. A regular file has no +* local format data fork. +*/ +static xfs_failaddr_t +xlog_dinode_verify_local_dfork ( +struct xfs_mount *mp, +struct xfs_log_dinode *ldip, +void *src, +unsigned int size) +{ +if (S_ISDIR (ldip->di_mode)) +return xfs_dir2_sf_verify (mp, src, size); +if (S_ISLNK (ldip->di_mode)) +return xlog_dinode_verify_symlink (src, size); +return __this_address; +} +/* * Verify the logged data fork against the core and the fork region that carries * it. The core has already been verified, so the mode and fork offset can be
@@ -449,9 +490,9 @@ xlog_dinode_verify(* the fork was logged, the field logged for it must match that format, and the * region holding it must be the size the format header records. A local or * extents fork is copied verbatim into the inode fork, so it must also fit -* within the space the fork offset leaves for it. A btree fork is logged in the -* larger expanded form and compressed into the fork on replay, so it does not -* get the fit check. The contents of the region are not inspected. +* within the space the fork offset leaves for it, and a local fork must hold +* valid contents. A btree fork is logged in the larger expanded form and +* compressed into the fork on replay, so it does not get the fit check. */ static xfs_failaddr_t xlog_dinode_verify_dfork(
@@ -490,7 +531,8 @@ xlog_dinode_verify_dfork(return __this_address;if (ilfp->ilf_dsize > fork_size)return __this_address; -break ; +return xlog_dinode_verify_local_dfork (mp, ldip, + item->ri_buf[2 ].iov_base, ilfp->ilf_dsize);case XFS_DINODE_FMT_EXTENTS :if (field !=XFS_ILOG_DEXT )return __this_address;
@@ -547,7 +589,8 @@ xlog_dinode_verify_afork(return __this_address;if (ilfp->ilf_asize > fork_size)return __this_address; -break ; +return xfs_attr_shortform_verify (item->ri_buf[region].iov_base, + ilfp->ilf_asize);case XFS_DINODE_FMT_EXTENTS :if (field !=XFS_ILOG_AEXT )return __this_address;
Signature · 1 line
2.55.0
PATCH 39/48 xfs: validate logged extent fork contents during recovery
In reply to Dave ChinnerAn extents format fork is logged as a packed array of on-disk extent records. Recovery copied this array into the inode without inspecting it, so corrupt extent records were replayed as they stood.
The existing verifiers open code the extent array walk for validation, so factor that walk into xfs_bmap_validate_extent_array() and use it in all three places. The on-disk reader now dumps the whole fork on a bad record rather than the single record, so the failing record is still captured.
xfs_bmap_complain_bad_rec() is recast to take a mount and inode number rather than an incore inode, so the array walk can report a bad record on behalf of any caller. The walk reports against the inode number it is passed, and stays silent when passed NULLFSINO.
--- fs/xfs/libxfs/xfs_bmap.c | 48 ++++++++++++++++++++++++++++----- fs/xfs/libxfs/xfs_bmap.h | 8 ++++-- fs/xfs/libxfs/xfs_inode_fork.c | 23 ++++++++-------- fs/xfs/scrub/inode_repair.c | 14 +++------- fs/xfs/xfs_inode_item_recover.c | 32 ++++++++++++++++++++-- 5 files changed, 91 insertions(+), 34 deletions(-)
fs/xfs/libxfs/xfs_bmap.c +41 -7
@@ -1090,12 +1090,12 @@ struct xfs_iread_state {int xfs_bmap_complain_bad_rec ( -struct xfs_inode *ip, +struct xfs_mount *mp, +xfs_ino_t ino,int whichfork,xfs_failaddr_t fa,const struct xfs_bmbt_irec *irec) { -struct xfs_mount *mp = ip->i_mount;const char *forkname;switch (whichfork) {
@@ -1107,7 +1107,7 @@ xfs_bmap_complain_bad_rec(xfs_warn (mp,"Bmap BTree record corruption in inode 0x%llx %s fork detected at %pS!" , -I_INO (ip), forkname, fa); + ino, forkname, fa); xfs_warn(mp, "Offset 0x%llx, start block 0x%llx, block count 0x%llx state 0x%x", irec->br_startoff, irec->br_startblock, irec->br_blockcount,
@@ -1160,8 +1160,8 @@ xfs_iread_bmbt_block("xfs_iread_extents(2)" , frp, sizeof(*frp), fa);xfs_bmap_mark_sick (ip, whichfork); -return xfs_bmap_complain_bad_rec (ip, whichfork, fa, - &new); +return xfs_bmap_complain_bad_rec (ip->i_mount, +I_INO (ip), whichfork, fa, &new); }xfs_iext_insert (ip, &ir->icur, &new,xfs_bmap_fork_to_state (whichfork));
@@ -6135,6 +6135,39 @@ xfs_bmap_validate_extent( XFS_IS_REALTIME_INODE(ip), whichfork, irec); } +/* +* Check a packed on-disk array of extent records. Returns the address of the +* first bad record, or NULL if they are all good. If ino is not NULLFSINO, a +* bad record is also reported against that inode. +*/ +xfs_failaddr_t +xfs_bmap_validate_extent_array ( +struct xfs_mount *mp, +xfs_ino_t ino, +bool rtfile, +int whichfork, +struct xfs_bmbt_rec *recs, +xfs_extnum_t nrecs) +{ +xfs_extnum_t i; + +for (i =0 ; i < nrecs; i++) { +struct xfs_bmbt_irec irec; +xfs_failaddr_t fa; + +xfs_bmbt_disk_get_all (&recs[i], &irec); + fa =xfs_bmap_validate_extent_raw (mp, rtfile, whichfork, &irec); +if (fa) { +if (ino !=NULLFSINO ) +xfs_bmap_complain_bad_rec (mp, ino, whichfork, + fa, &irec); +return fa; + } + } + +return NULL ; +} +/* * Used in xfs_itruncate_extents(). This is the maximum number of extents * freed from a file in a single transaction.
@@ -6196,8 +6229,9 @@ xfs_bmap_query_range_helper( &irec);if (fa) {xfs_btree_mark_sick (cur); -return xfs_bmap_complain_bad_rec (cur->bc_ino.ip, - cur->bc_ino.whichfork, fa, &irec); +return xfs_bmap_complain_bad_rec (cur->bc_mp, +I_INO (cur->bc_ino.ip), cur->bc_ino.whichfork, + fa, &irec); }return query->fn (cur, &irec, query->priv);
fs/xfs/libxfs/xfs_bmap.h +6 -2
@@ -275,8 +275,12 @@ xfs_failaddr_t xfs_bmap_validate_extent_raw(struct xfs_mount *mp, bool rtfile, int whichfork, struct xfs_bmbt_irec *irec);xfs_failaddr_t xfs_bmap_validate_extent (struct xfs_inode *ip,int whichfork,struct xfs_bmbt_irec *irec); -int xfs_bmap_complain_bad_rec (struct xfs_inode *ip,int whichfork, -xfs_failaddr_t fa,const struct xfs_bmbt_irec *irec); +xfs_failaddr_t xfs_bmap_validate_extent_array (struct xfs_mount *mp, +xfs_ino_t ino,bool rtfile,int whichfork, +struct xfs_bmbt_rec *recs,xfs_extnum_t nrecs); +int xfs_bmap_complain_bad_rec (struct xfs_mount *mp,xfs_ino_t ino, +int whichfork,xfs_failaddr_t fa, +const struct xfs_bmbt_irec *irec); int xfs_bmapi_remap(struct xfs_trans *tp, struct xfs_inode *ip, xfs_fileoff_t bno, xfs_filblks_t len, xfs_fsblock_t startblock,
fs/xfs/libxfs/xfs_inode_fork.c +11 -12
@@ -138,23 +138,22 @@ xfs_iformat_extents( ifp->if_data =NULL ; ifp->if_height =0 ;if (size) { +xfs_failaddr_t fa; + dp = (xfs_bmbt_rec_t *)XFS_DFORK_PTR (dip, whichfork); + fa =xfs_bmap_validate_extent_array (mp,I_INO (ip), +XFS_IS_REALTIME_INODE (ip), whichfork, dp, nex); +if (fa) { +xfs_inode_verifier_error (ip, -EFSCORRUPTED , +"xfs_iformat_extents(2)" , dp, size, fa); +xfs_inode_mark_sick (ip,XFS_SICK_INO_CORE ); +return -EFSCORRUPTED ; + } +xfs_iext_first (ifp, &icur);for (i =0 ; i < nex; i++, dp++) { -xfs_failaddr_t fa; -xfs_bmbt_disk_get_all (dp, &new); - fa =xfs_bmap_validate_extent (ip, whichfork, &new); -if (fa) { -xfs_inode_verifier_error (ip, -EFSCORRUPTED , -"xfs_iformat_extents(2)" , - dp,sizeof (*dp), fa); -xfs_inode_mark_sick (ip,XFS_SICK_INO_CORE ); -return xfs_bmap_complain_bad_rec (ip, whichfork, - fa, &new); - } -xfs_iext_insert (ip, &icur, &new, state);trace_xfs_read_extent (ip, &icur, state, _THIS_IP_);xfs_iext_next (ifp, &icur);
fs/xfs/scrub/inode_repair.c +3 -11
@@ -869,11 +869,9 @@ xrep_dinode_bad_extents_fork( unsigned int dfork_size, int whichfork) { -struct xfs_bmbt_irec new;struct xfs_bmbt_rec *dp;xfs_extnum_t nex;bool isrt; -unsigned int i; nex =xfs_dfork_nextents (dip, whichfork);if (nex > dfork_size /sizeof (struct xfs_bmbt_rec ))
@@ -882,15 +880,9 @@ xrep_dinode_bad_extents_fork( dp =XFS_DFORK_PTR (dip, whichfork); isrt = dip->di_flags &cpu_to_be16 (XFS_DIFLAG_REALTIME ); -for (i =0 ; i < nex; i++, dp++) { -xfs_failaddr_t fa; - -xfs_bmbt_disk_get_all (dp, &new); - fa =xfs_bmap_validate_extent_raw (sc->mp, isrt, whichfork, - &new); -if (fa) -return true; - } +if (xfs_bmap_validate_extent_array (sc->mp,NULLFSINO , isrt, whichfork, + dp, nex)) +return true;return false; }
fs/xfs/xfs_inode_item_recover.c +30 -2
@@ -27,6 +27,7 @@#include "xfs_dir2_priv.h" #include "xfs_attr_leaf.h" #include "xfs_metafile.h" +#include "xfs_bmap.h" #include "xfs_bmap_btree.h" #include "xfs_rtrmap_btree.h" #include "xfs_rtrefcount_btree.h"
@@ -443,6 +444,31 @@ xlog_dinode_verify(return NULL ; } +/* +* Verify a logged extents fork. The region is a packed array of on-disk extent +* records, so its size fixes the number of records it holds. +* +* The region size has already been bounds checked against the inode fork size, +* so it is known to fit within the fork and does not need rechecking here. +*/ +static xfs_failaddr_t +xlog_dinode_verify_extents ( +struct xfs_mount *mp, +struct xfs_log_dinode *ldip, +int whichfork, +void *src, +unsigned int size) +{ +bool isrt; + +if (size ==0 || (size %sizeof (struct xfs_bmbt_rec ))) +return __this_address; + + isrt = ldip->di_flags &XFS_DIFLAG_REALTIME ; +return xfs_bmap_validate_extent_array (mp, ldip->di_ino, isrt, whichfork, + src, size /sizeof (struct xfs_bmbt_rec )); +} +/* * Verify a logged shortform symlink target. The logged target is the on-disk * form, which is not null terminated, so this cannot use
@@ -538,7 +564,8 @@ xlog_dinode_verify_dfork(return __this_address;if (ilfp->ilf_dsize > fork_size)return __this_address; -break ; +return xlog_dinode_verify_extents (mp, ldip,XFS_DATA_FORK , + item->ri_buf[2 ].iov_base, ilfp->ilf_dsize);case XFS_DINODE_FMT_BTREE :case XFS_DINODE_FMT_META_BTREE :if (field !=XFS_ILOG_DBROOT )
@@ -596,7 +623,8 @@ xlog_dinode_verify_afork(return __this_address;if (ilfp->ilf_asize > fork_size)return __this_address; -break ; +return xlog_dinode_verify_extents (mp, ldip,XFS_ATTR_FORK , + item->ri_buf[region].iov_base, ilfp->ilf_asize);case XFS_DINODE_FMT_BTREE :if (field !=XFS_ILOG_ABROOT )return __this_address;
Signature · 1 line
2.55.0
PATCH 40/48 xfs: allow a long format btree root to be checked without a cursor
In reply to Dave ChinnerThe long format btree block header check derives the block magic and the maximum record count from a btree cursor. An inode rooted btree logged to the journal has no cursor available during recovery, so this check cannot be used to validate a logged btree root.
Rework the header check to take the magic and maximum records as arguments rather than reading them from a cursor. The cursor based callers derive both from the cursor and pass them in, so their behaviour is unchanged.
Add xfs_btree_check_fsblock_root_hdr() for callers that hold a long format btree root but no cursor. It checks the root against the supplied magic, maximum records and level range, and confirms the root has no siblings.
--- fs/xfs/libxfs/xfs_btree.c | 50 ++++++++++++++++++++++++++++++--------- fs/xfs/libxfs/xfs_btree.h | 4 ++++ 2 files changed, 43 insertions(+), 11 deletions(-)
fs/xfs/libxfs/xfs_btree.c +39 -11
@@ -123,13 +123,13 @@ xfs_btree_check_agblock_siblings(static xfs_failaddr_t __xfs_btree_check_lblock_hdr ( -struct xfs_btree_cur *cur, +struct xfs_mount *mp, +uint32_t magic, +unsigned int maxrecs,struct xfs_btree_block *block,int level,struct xfs_buf *bp) { -struct xfs_mount *mp = cur->bc_mp; -if (xfs_has_crc (mp)) {if (!uuid_equal (&block->bb_u.l.bb_uuid, &mp->m_sb.sb_meta_uuid))return __this_address;
@@ -140,12 +140,11 @@ __xfs_btree_check_lblock_hdr(return __this_address; } -if (be32_to_cpu (block->bb_magic) !=xfs_btree_magic (mp, cur->bc_ops)) +if (be32_to_cpu (block->bb_magic) != magic)return __this_address;if (be16_to_cpu (block->bb_level) != level)return __this_address; -if (be16_to_cpu (block->bb_numrecs) > - cur->bc_ops->get_maxrecs (cur, level)) +if (be16_to_cpu (block->bb_numrecs) > maxrecs)return __this_address;return NULL ;
@@ -157,16 +156,17 @@ __xfs_btree_check_lblock_hdr(*/ static xfs_failaddr_t __xfs_btree_check_fsblock ( -struct xfs_btree_cur *cur, +struct xfs_mount *mp, +uint32_t magic, +unsigned int maxrecs,struct xfs_btree_block *block,int level,struct xfs_buf *bp) { -struct xfs_mount *mp = cur->bc_mp;xfs_failaddr_t fa;xfs_fsblock_t fsb; - fa =__xfs_btree_check_lblock_hdr (cur, block, level, bp); + fa =__xfs_btree_check_lblock_hdr (mp, magic, maxrecs, block, level, bp);if (fa)return fa;
@@ -191,6 +191,29 @@ __xfs_btree_check_fsblock(return fa; } +/* +* Check the header of a long format btree root that sits in an inode fork, +* without needing a btree cursor. The block is a root, so it has no siblings. +* The caller supplies the block magic, the maximum records for the block, and +* the valid level range for the btree. +*/ +xfs_failaddr_t +xfs_btree_check_fsblock_root_hdr ( +struct xfs_mount *mp, +uint32_t magic, +unsigned int maxrecs, +struct xfs_btree_block *block, +unsigned int minlevel, +unsigned int maxlevel) +{ +unsigned int level =be16_to_cpu (block->bb_level); + +if (level < minlevel || level > maxlevel) +return __this_address; + +return __xfs_btree_check_fsblock (mp, magic, maxrecs, block, level,NULL ); +} +/* * Check an in-memory btree block header. Return the address of the failing * check, or NULL if everything is ok.
@@ -202,11 +225,13 @@ __xfs_btree_check_memblock(int level, struct xfs_buf *bp) { + struct xfs_mount *mp = cur->bc_mp;struct xfs_buftarg *btp = cur->bc_mem.xfbtree->target;xfs_failaddr_t fa;xfbno_t bno; - fa =__xfs_btree_check_lblock_hdr (cur, block, level, bp); + fa =__xfs_btree_check_lblock_hdr (mp,xfs_btree_magic (mp, cur->bc_ops), + cur->bc_ops->get_maxrecs (cur, level), block, level, bp);if (fa)return fa;
@@ -277,7 +302,10 @@ __xfs_btree_check_block(case XFS_BTREE_TYPE_AG :return __xfs_btree_check_agblock (cur, block, level, bp);case XFS_BTREE_TYPE_INODE : -return __xfs_btree_check_fsblock (cur, block, level, bp); +return __xfs_btree_check_fsblock (cur->bc_mp, +xfs_btree_magic (cur->bc_mp, cur->bc_ops), + cur->bc_ops->get_maxrecs (cur, level), + block, level, bp);default :ASSERT (0 );return __this_address;
fs/xfs/libxfs/xfs_btree.h +4 -0
@@ -345,6 +345,10 @@ xfs_btree_cur_sizeof(unsigned int nlevels)xfs_failaddr_t __xfs_btree_check_block (struct xfs_btree_cur *cur,struct xfs_btree_block *block,int level,struct xfs_buf *bp); +xfs_failaddr_t xfs_btree_check_fsblock_root_hdr (struct xfs_mount *mp, +uint32_t magic,unsigned int maxrecs, +struct xfs_btree_block *block,unsigned int minlevel, +unsigned int maxlevel);int __xfs_btree_check_ptr (struct xfs_btree_cur *cur,const union xfs_btree_ptr *ptr,int index,int level);
Signature · 1 line
2.55.0
PATCH 41/48 xfs: validate logged btree fork roots during recovery
In reply to Dave ChinnerA btree format fork is logged as the expanded in-core bmap btree root, which is a normal btree root block whose length is only as long as the fork region that holds it. Recovery copied this root into the inode without inspecting it, so a corrupt root was replayed as it stood.
Validate the logged bmap btree root once the fork has been verified against the core. The root is checked as a short btree root block whose record count fills the fork region exactly, and its keys and pointers are checked to be in range. The metadata btree roots are a distinct form and remain validated by a following patch.
--- fs/xfs/xfs_inode_item_recover.c | 62 ++++++++++++++++++++++++++++++++- 1 file changed, 61 insertions(+), 1 deletion(-)
fs/xfs/xfs_inode_item_recover.c +61 -1
@@ -29,6 +29,7 @@#include "xfs_metafile.h" #include "xfs_bmap.h" #include "xfs_bmap_btree.h" +#include "xfs_btree.h" #include "xfs_rtrmap_btree.h" #include "xfs_rtrefcount_btree.h"
@@ -469,6 +470,60 @@ xlog_dinode_verify_extents( src, size / sizeof(struct xfs_bmbt_rec)); } +/* +* Verify a logged bmap btree root. The logged root is the expanded in-core +* block form, not the compact on-disk root, so it is a normal btree root block +* whose length is only as long as the fork region that holds it. The block +* length is fixed by the record count, so the region size fixes how many +* records the root can hold. +* +* The region size has already been bounds checked against the inode fork size, +* so it is known to fit within the fork and does not need rechecking here. +*/ +static xfs_failaddr_t +xlog_dinode_verify_bmbt_root ( +struct xfs_mount *mp, +struct xfs_log_dinode *ldip, +int whichfork, +void *src, +unsigned int size) +{ +struct xfs_btree_block *block = src; +xfs_failaddr_t fa; +uint32_t magic; +unsigned int numrecs; +unsigned int i; + +if (size <xfs_bmbt_block_len (mp)) +return __this_address; + +/* The root block is never a leaf, so it always has keys and pointers. */ + magic =xfs_has_crc (mp) ?XFS_BMAP_CRC_MAGIC :XFS_BMAP_MAGIC ; + fa =xfs_btree_check_fsblock_root_hdr (mp, magic, +xfs_bmbt_maxrecs (mp, size, false), block,1 , + mp->m_bm_maxlevels[whichfork]); +if (fa) +return fa; + +/* The record count must be non-zero and exactly fill the region. */ + numrecs =be16_to_cpu (block->bb_numrecs); +if (numrecs ==0 || size !=xfs_bmap_broot_space_calc (mp, numrecs)) +return __this_address; + +for (i =1 ; i <= numrecs; i++) { +struct xfs_bmbt_key *key =xfs_bmbt_key_addr (mp, block, i); +xfs_bmbt_ptr_t *ptr =xfs_bmap_broot_ptr_addr (mp, block, + i, size); + +if (!xfs_verify_fileoff (be64_to_cpu (key->br_startoff))) +return __this_address; +if (!xfs_verify_fsbno (mp,be64_to_cpu (*ptr))) +return __this_address; + } + +return NULL ; +} +/* * Verify a logged shortform symlink target. The logged target is the on-disk * form, which is not null terminated, so this cannot use
@@ -567,6 +622,10 @@ xlog_dinode_verify_dfork(return xlog_dinode_verify_extents (mp, ldip,XFS_DATA_FORK , item->ri_buf[2 ].iov_base, ilfp->ilf_dsize);case XFS_DINODE_FMT_BTREE : +if (field !=XFS_ILOG_DBROOT ) +return __this_address; +return xlog_dinode_verify_bmbt_root (mp, ldip,XFS_DATA_FORK , + item->ri_buf[2 ].iov_base, ilfp->ilf_dsize);case XFS_DINODE_FMT_META_BTREE :if (field !=XFS_ILOG_DBROOT )return __this_address;
@@ -628,7 +687,8 @@ xlog_dinode_verify_afork(case XFS_DINODE_FMT_BTREE :if (field !=XFS_ILOG_ABROOT )return __this_address; -break ; +return xlog_dinode_verify_bmbt_root (mp, ldip,XFS_ATTR_FORK , + item->ri_buf[region].iov_base, ilfp->ilf_asize); }return NULL ;
Signature · 1 line
2.55.0
PATCH 42/48 xfs: validate logged metadata btree fork roots during recovery
In reply to Dave ChinnerA metadata btree format data fork holds an rt rmap or rt refcount btree root, logged as the expanded in-core btree root like a bmap btree root. Recovery copied this root into the inode without inspecting it, so a corrupt root was replayed as it stood.
Validate the logged metadata btree root once the fork has been verified against the core. The root is checked as a short btree root block whose record count fills the fork region exactly, and its node pointers are checked to be in range. The record and key contents are relative to a realtime group, which is not set up until after log recovery, so their validation is left to the first runtime access to the root block.
--- fs/xfs/xfs_inode_item_recover.c | 129 +++++++++++++++++++++++++++++++- 1 file changed, 128 insertions(+), 1 deletion(-)
fs/xfs/xfs_inode_item_recover.c +128 -1
@@ -524,6 +524,132 @@ xlog_dinode_verify_bmbt_root(return NULL ; } +/* +* Verify a logged rt rmap btree root, as for the bmap btree root. The node +* pointers are filesystem block numbers and are checked here. +*/ +static xfs_failaddr_t +xlog_dinode_verify_rtrmap_root ( +struct xfs_mount *mp, +struct xfs_log_dinode *ldip, +void *src, +unsigned int size) +{ +struct xfs_btree_block *block = src; +xfs_failaddr_t fa; +unsigned int level; +unsigned int numrecs; +unsigned int i; + +if (size <XFS_RTRMAP_BLOCK_LEN ) +return __this_address; + + fa =xfs_btree_check_fsblock_root_hdr (mp,XFS_RTRMAP_CRC_MAGIC , +xfs_rtrmapbt_maxrecs (mp, size, +be16_to_cpu (block->bb_level) ==0 ), + block,0 , mp->m_rtrmap_maxlevels); +if (fa) +return fa; + + level =be16_to_cpu (block->bb_level); + numrecs =be16_to_cpu (block->bb_numrecs); +if (numrecs ==0 || + size !=xfs_rtrmap_broot_space_calc (level, numrecs)) +return __this_address; + +/* +* A leaf root holds records whose bounds are relative to a realtime +* group. The realtime groups are not set up until long after log +* recovery runs, so the leaf contents cannot be validated here. That is +* deferred to the first runtime access to the root block. +*/ +if (level ==0 ) +return NULL ; + +for (i =1 ; i <= numrecs; i++) { +xfs_rtrmap_ptr_t *ptr =xfs_rtrmap_broot_ptr_addr (mp, + block, i, size); + +if (!xfs_verify_fsbno (mp,be64_to_cpu (*ptr))) +return __this_address; + } + +return NULL ; +} + +/* +* Verify a logged rt refcount btree root, as for the rt rmap btree root. +*/ +static xfs_failaddr_t +xlog_dinode_verify_rtrefcount_root ( +struct xfs_mount *mp, +struct xfs_log_dinode *ldip, +void *src, +unsigned int size) +{ +struct xfs_btree_block *block = src; +xfs_failaddr_t fa; +unsigned int level; +unsigned int numrecs; +unsigned int i; + +if (size <XFS_RTREFCOUNT_BLOCK_LEN ) +return __this_address; + + fa =xfs_btree_check_fsblock_root_hdr (mp,XFS_RTREFC_CRC_MAGIC , +xfs_rtrefcountbt_maxrecs (mp, size, +be16_to_cpu (block->bb_level) ==0 ), + block,0 , mp->m_rtrefc_maxlevels -1 ); +if (fa) +return fa; + + level =be16_to_cpu (block->bb_level); + numrecs =be16_to_cpu (block->bb_numrecs); +if (numrecs ==0 || + size !=xfs_rtrefcount_broot_space_calc (level, numrecs)) +return __this_address; + +/* +* A leaf root holds records whose bounds are relative to a realtime +* group. The realtime groups are not set up until long after log +* recovery runs, so the leaf contents cannot be validated here. That is +* deferred to the first runtime access to the root block. +*/ +if (level ==0 ) +return NULL ; + +for (i =1 ; i <= numrecs; i++) { +xfs_rtrefcount_ptr_t *ptr =xfs_rtrefcount_broot_ptr_addr (mp, + block, i, size); + +if (!xfs_verify_fsbno (mp,be64_to_cpu (*ptr))) +return __this_address; + } + +return NULL ; +} + +/* +* Verify a logged metadata btree root. The metadata btree type is fixed by the +* inode, so it is chosen from the log dinode metadata type. +*/ +static xfs_failaddr_t +xlog_dinode_verify_metabtree_root ( +struct xfs_mount *mp, +struct xfs_log_dinode *ldip, +void *src, +unsigned int size) +{ +switch (ldip->di_metatype) { +case XFS_METAFILE_RTRMAP : +return xlog_dinode_verify_rtrmap_root (mp, ldip, src, size); +case XFS_METAFILE_RTREFCOUNT : +return xlog_dinode_verify_rtrefcount_root (mp, ldip, src, size); +default : +return __this_address; + } +} +/* * Verify a logged shortform symlink target. The logged target is the on-disk * form, which is not null terminated, so this cannot use
@@ -629,7 +755,8 @@ xlog_dinode_verify_dfork(case XFS_DINODE_FMT_META_BTREE :if (field !=XFS_ILOG_DBROOT )return __this_address; -break ; +return xlog_dinode_verify_metabtree_root (mp, ldip, + item->ri_buf[2 ].iov_base, ilfp->ilf_dsize); }return NULL ;
Signature · 1 line
2.55.0
PATCH 43/48 xfs: validate buffer item region count during recovery
In reply to Dave ChinnerValidate the number of regions in a buffer log item as it is decoded from the journal. A cancelled buffer logs only its format header, so the valid range spans the whole range a log item can hold.
--- fs/xfs/xfs_buf_item_recover.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
fs/xfs/xfs_buf_item_recover.c +17 -0
@@ -1173,8 +1173,25 @@ xlog_recover_buf_commit_pass2(return 0 ; } +/* +* A buffer item is a format header region followed by a region for each dirty +* range of the buffer. However, a cancelled buffer only logs a format header, +* hence the valid region count range spans the entire range a log item can +* legally have. +*/ +static int +xlog_recover_buf_nregions ( +struct xlog *log, +uint16_t nregions) +{ +if (nregions <1 || nregions >XLOG_MAX_REGIONS_IN_ITEM ) +return -EFSCORRUPTED ; +return nregions; +} +const struct xlog_recover_item_ops xlog_buf_item_ops = { .item_type =XFS_LI_BUF , + .validate_nregions = xlog_recover_buf_nregions, .reorder = xlog_recover_buf_reorder, .ra_pass2 = xlog_recover_buf_ra_pass2, .commit_pass1 = xlog_recover_buf_commit_pass1,
Signature · 1 line
2.55.0
PATCH 44/48 xfs: validate buffer item regions during recovery
In reply to Dave ChinnerValidate each region of a buffer log item as it is decoded from the journal. The format header must hold a dirty bitmap that fits within the region, and each dirty range region must cover whole chunks of the buffer.
The fixed prefix of the format header is checked through min_hdr_len, so the region check only has to bound the used part of the dirty bitmap against the region.
--- fs/xfs/xfs_buf_item_recover.c | 46 +++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+)
fs/xfs/xfs_buf_item_recover.c +46 -0
@@ -1189,9 +1189,55 @@ xlog_recover_buf_nregions(return nregions; } +/* +* Region 0 is the format header. The fixed prefix is already checked by +* min_hdr_len, so check that the used part of the dirty bitmap does not exceed +* the bitmap and fits within the region. Later regions are the dirty ranges of +* the buffer, each covering whole chunks of it. The region length has already +* been bounds checked against XLOG_MAX_REGION_SIZE during assembly, so only the +* lower bound and chunk alignment are checked here. +*/ +static int +xlog_recover_buf_validate_region ( +struct xlog *log, +struct xlog_recover_item *item, +int region_index) +{ +size_t len = item->ri_buf[region_index].iov_len; + +if (region_index ==0 ) { +struct xfs_buf_log_format *blf = item->ri_buf[0 ].iov_base; + +if (blf->blf_map_size >XFS_BLF_DATAMAP_SIZE ) { +xfs_warn (log->l_mp, +"BUF: bad dirty bitmap size %u recovered." , + blf->blf_map_size); +return -EFSCORRUPTED ; + } +if (!xfs_buf_log_check_iovec (&item->ri_buf[0 ])) { +xfs_warn (log->l_mp, +"BUF: bad format region length %zu recovered." , + len); +return -EFSCORRUPTED ; + } +return 0 ; + } + +if (len ==0 || (len %XFS_BLF_CHUNK )) { +xfs_warn (log->l_mp, +"BUF: bad region %d length %zu recovered." , + region_index, len); +return -EFSCORRUPTED ; + } + +return 0 ; +} +const struct xlog_recover_item_ops xlog_buf_item_ops = { .item_type =XFS_LI_BUF , + .min_hdr_len = offsetof(struct xfs_buf_log_format , blf_data_map), .validate_nregions = xlog_recover_buf_nregions, + .validate_region = xlog_recover_buf_validate_region, .reorder = xlog_recover_buf_reorder, .ra_pass2 = xlog_recover_buf_ra_pass2, .commit_pass1 = xlog_recover_buf_commit_pass1,
Signature · 1 line
2.55.0
PATCH 45/48 xfs: validate the whole buffer item during recovery
In reply to Dave ChinnerValidate a buffer log item once all its regions are decoded from the journal, so the rest of recovery can trust it. The item size must match the number of regions attached, and the buffer must lie wholly within the data device.
A cancelled buffer is not replayed and logs only its format header, so it is required to be a single region and nothing further is checked. Any other buffer must record a valid buffer type that agrees with its type flags, and its dirty regions must correspond to the dirty bitmap without describing more than a buffer's worth of data.
The buffer block range is checked against the data device size held in the buftarg rather than the superblock geometry. Deriving a shared daddr range helper is left for later.
--- fs/xfs/xfs_buf_item_recover.c | 153 ++++++++++++++++++++++++++++++++++ 1 file changed, 153 insertions(+)
fs/xfs/xfs_buf_item_recover.c +153 -0
@@ -1233,11 +1233,164 @@ xlog_recover_buf_validate_region(return 0 ; } +/* The buffer type flags. A buffer has at most one of these set. */ +#define XFS_BLF_TYPE_MASK \ + (XFS_BLF_INODE_BUF | XFS_BLF_UDQUOT_BUF | XFS_BLF_PDQUOT_BUF | \ + XFS_BLF_GDQUOT_BUF) + +/* +* The buffer type is recorded both in the top bits of the format flags and in +* the type flags. Check the two agree: a buffer carries at most one type flag, +* and it must be the one the type field describes. +*/ +static xfs_failaddr_t +xlog_recover_buf_verify_type ( +uint16_t blft, +unsigned int type) +{ +if (blft <=XFS_BLFT_UNKNOWN_BUF || blft >=XFS_BLFT_MAX_BUF ) +return __this_address; + +switch (blft) { +case XFS_BLFT_DINO_BUF : +if (type !=XFS_BLF_INODE_BUF ) +return __this_address; +break ; +case XFS_BLFT_UDQUOT_BUF : +if (type !=XFS_BLF_UDQUOT_BUF ) +return __this_address; +break ; +case XFS_BLFT_PDQUOT_BUF : +if (type !=XFS_BLF_PDQUOT_BUF ) +return __this_address; +break ; +case XFS_BLFT_GDQUOT_BUF : +if (type !=XFS_BLF_GDQUOT_BUF ) +return __this_address; +break ; +default : +if (type) +return __this_address; +break ; + } + +return NULL ; +} + +/* +* Validate the whole buffer item once all regions are assembled. The item size +* must match the number of regions attached. A cancelled buffer is not +* replayed, so it logs only the format header. Any other buffer must have a +* valid type, and its dirty regions must match the dirty bitmap and not +* describe more than a buffer's worth of data. +*/ +static int +xlog_recover_buf_validate_item ( +struct xlog *log, +struct xlog_recover_item *item) +{ +struct xfs_buf_log_format *blf = item->ri_buf[0 ].iov_base; +xfs_daddr_t eofs = log->l_mp->m_ddev_targp->bt_nr_sectors; +xfs_failaddr_t fa; +size_t total =0 ; +unsigned int ri =1 ; +int first_bit; + +if (blf->blf_size != item->ri_total) { +xfs_warn (log->l_mp, +"BUF: item size %u does not match region count %u recovered." , + blf->blf_size, item->ri_total); +return -EFSCORRUPTED ; + } + +/* +* The buffer length is an invariant: a buffer cannot be larger than the +* most data that can be logged in a single region. +*/ +if (blf->blf_len ==0 ||BBTOB (blf->blf_len) >XLOG_MAX_REGION_SIZE ) { +xfs_warn (log->l_mp, +"BUF: bad buffer length %u recovered." , blf->blf_len); +return -EFSCORRUPTED ; + } + +/* +* The buffer must lie wholly within the data device. Guard against the +* end wrapping so an out of range buffer cannot pass the bounds check. +*/ +if (blf->blf_blkno <0 || + blf->blf_blkno >= eofs || + blf->blf_blkno + blf->blf_len < blf->blf_blkno || + blf->blf_blkno + blf->blf_len > eofs) { +xfs_warn (log->l_mp, +"BUF: bad buffer block 0x%llx length %u recovered." , + blf->blf_blkno, blf->blf_len); +return -EFSCORRUPTED ; + } + +/* +* A cancelled buffer overrides everything else. It is not replayed, so +* it logs only the format header regardless of what the dirty bitmap +* holds, and nothing else needs to be checked. +*/ +if (blf->blf_flags &XFS_BLF_CANCEL ) { +if (blf->blf_size !=1 ) { +xfs_warn (log->l_mp, +"BUF: cancelled buffer has %u regions recovered." , + blf->blf_size); +return -EFSCORRUPTED ; + } +return 0 ; + } + + fa =xlog_recover_buf_verify_type (xfs_blft_from_flags (blf), + blf->blf_flags &XFS_BLF_TYPE_MASK ); +if (fa) { +xfs_warn (log->l_mp, +"BUF: type %u and flags 0x%x disagree recovered at %pS" , +xfs_blft_from_flags (blf), blf->blf_flags, fa); +return -EFSCORRUPTED ; + } + +/* +* Each dirty region covers one contiguous run of set bits in the dirty +* bitmap. Walk the bitmap and the regions together: each region size +* must match the length of the run it covers, each run must lie within +* the buffer, and the regions must exactly cover the bitmap. The dirty +* data across all regions cannot be larger than the buffer itself. +*/ + first_bit =xfs_next_bit (blf->blf_data_map, blf->blf_map_size,0 ); +while (first_bit !=-1 ) { +int nbits =xfs_contig_bits (blf->blf_data_map, + blf->blf_map_size, first_bit); + +if (ri >= item->ri_total || + item->ri_buf[ri].iov_len != (nbits <<XFS_BLF_SHIFT ) || + ((first_bit + nbits) <<XFS_BLF_SHIFT ) >BBTOB (blf->blf_len)) + goto bad_map; + + total += item->ri_buf[ri].iov_len; + ri++; + first_bit =xfs_next_bit (blf->blf_data_map, blf->blf_map_size, + first_bit + nbits +1 ); + } + +if (ri != item->ri_total || total >BBTOB (blf->blf_len)) + goto bad_map; + +return 0 ; + +bad_map: +xfs_warn (log->l_mp, +"BUF: dirty bitmap does not match regions recovered." ); +return -EFSCORRUPTED ; +} +const struct xlog_recover_item_ops xlog_buf_item_ops = { .item_type =XFS_LI_BUF , .min_hdr_len = offsetof(struct xfs_buf_log_format , blf_data_map), .validate_nregions = xlog_recover_buf_nregions, .validate_region = xlog_recover_buf_validate_region, + .validate_item = xlog_recover_buf_validate_item, .reorder = xlog_recover_buf_reorder, .ra_pass2 = xlog_recover_buf_ra_pass2, .commit_pass1 = xlog_recover_buf_commit_pass1,
Signature · 1 line
2.55.0
PATCH 46/48 xfs: remove redundant buffer checks from recovery replay
In reply to Dave ChinnerThe buffer item is now fully validated as it is decoded from the journal, so the replay paths can trust its regions and dirty bitmap. The bitmap and region consistency checks scattered through the buffer and inode buffer replay loops are now redundant.
Remove the region and bitmap checks from the replay loops. The checks on buffer contents, which cannot be validated from the log item alone, remain.
--- fs/xfs/xfs_buf_item_recover.c | 40 ----------------------------------- 1 file changed, 40 deletions(-)
fs/xfs/xfs_buf_item_recover.c +0 -40
@@ -485,38 +485,6 @@ xlog_recover_do_reg_buffer(break ; nbits =xfs_contig_bits (buf_f->blf_data_map, buf_f->blf_map_size, bit); -ASSERT (nbits >0 ); -ASSERT (item->ri_buf[i].iov_base !=NULL ); -ASSERT (item->ri_buf[i].iov_len %XFS_BLF_CHUNK ==0 ); -/* -* The bitmap is only trustworthy to the extent that it -* describes a region that actually fits inside the buffer we -* read in based on the (attacker-controlled) blf_len. Do not -* rely on an ASSERT() for this -- it compiles away entirely on -* non-DEBUG kernels, which is exactly where this matters, so -* validate it for real and abort recovery of this buffer rather -* than copying past the end of it. -*/ -if (XFS_IS_CORRUPT (mp,BBTOB (bp->b_length) < - ((uint )bit <<XFS_BLF_SHIFT ) + - (nbits <<XFS_BLF_SHIFT ))) { -xfs_alert (mp, -"Bad buffer log item dirty bitmap (bit %d, nbits %d) for %d-byte buffer at daddr 0x%llx." , - bit, nbits,BBTOB (bp->b_length), -xfs_buf_daddr (bp)); -return -EFSCORRUPTED ; - } - -/* -* The dirty regions logged in the buffer, even though -* contiguous, may span multiple chunks. This is because the -* dirty region may span a physical page boundary in a buffer -* and hence be split into two separate vectors for writing into -* the log. Hence we need to trim nbits back to the length of -* the current region being copied out of the log. -*/ -if (item->ri_buf[i].iov_len < (nbits <<XFS_BLF_SHIFT )) - nbits = item->ri_buf[i].iov_len >>XFS_BLF_SHIFT ;/* * Do a sanity check if this is a dquot buffer. Just checking
@@ -555,9 +523,6 @@ xlog_recover_do_reg_buffer( bit += nbits; } -/* Shouldn't be any more regions */ -ASSERT (i == item->ri_total); -xlog_recover_validate_buf_type (mp, bp, buf_f, current_lsn);return 0 ; }
@@ -674,7 +639,6 @@ xlog_recover_do_inode_buffer( nbits =xfs_contig_bits (buf_f->blf_data_map, buf_f->blf_map_size, bit); -ASSERT (nbits >0 ); reg_buf_offset = bit <<XFS_BLF_SHIFT ; reg_buf_bytes = nbits <<XFS_BLF_SHIFT ; item_index++;
@@ -688,10 +652,6 @@ xlog_recover_do_inode_buffer(if (next_unlinked_offset < reg_buf_offset)continue ; -ASSERT (item->ri_buf[item_index].iov_base !=NULL ); -ASSERT ((item->ri_buf[item_index].iov_len %XFS_BLF_CHUNK ) ==0 ); -ASSERT ((reg_buf_offset + reg_buf_bytes) <=BBTOB (bp->b_length)); -/* * The current logged region contains a copy of the * current di_next_unlinked field. Extract its value
Signature · 1 line
2.55.0
PATCH 47/48 xfs: fail recovery on a buffer type mismatch
In reply to Dave ChinnerWhen recovering a buffer, xlog_recover_validate_buf_type() checks that the buffer type recorded in the log item agrees with the magic number of the buffer being replayed into. A mismatch left bp->b_ops unset and was only reported with a warning and an ASSERT. On a production kernel recovery then queued the buffer with no verifier attached, so xfs_buf_verify_write() warned about the missing operations but still wrote the mismatched recovery data to disk.
Return an error from xlog_recover_validate_buf_type() on a type mismatch instead of asserting. The replay path propagates the error so recovery fails rather than writing out an unverifiable buffer. The skip path, where the buffer on disk is newer than the log item being recovered, ignores the result:
later changes to that buffer are still to be replayed and may change its type, so a mismatch against the current log item is expected there and must not warn.
--- fs/xfs/xfs_buf_item_recover.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-)
fs/xfs/xfs_buf_item_recover.c +16 -7
@@ -212,7 +212,7 @@ xlog_recover_buf_commit_pass1(* the first 32 bits of the buffer (most blocks), * inside a struct xfs_da_blkinfo at the start of the buffer. */ -static void +static int xlog_recover_validate_buf_type( struct xfs_mount *mp, struct xfs_buf *bp,
@@ -234,7 +234,7 @@ xlog_recover_validate_buf_type(* just avoid the verification stage for non-crc filesystems */ if (!xfs_has_crc (mp)) -return ; +return 0 ; magic32 =be32_to_cpu (*(__be32 *)bp->b_addr); magic16 =be16_to_cpu (*(__be16 *)bp->b_addr);
@@ -424,14 +424,22 @@ xlog_recover_validate_buf_type(/* * Nothing else to do in the case of a NULL current LSN as this means * the buffer is more recent than the change in the log and will be -* skipped. +* skipped. The buffer contents are newer than this format, so later +* changes to this buffer still to be replayed may change its type. A +* mismatch against this format is therefore expected and not a corruption. */ if (current_lsn ==NULLCOMMITLSN ) -return ; +return 0 ; +/* +* The buffer type does not match the magic number in the buffer we are +* about to replay into. The write verifier keyed off the type would +* reject it, so fail recovery rather than submit a buffer that cannot be +* verified. +*/ if (warnmsg) {xfs_warn (mp, warnmsg); -ASSERT (0 ); +return -EFSCORRUPTED ; }/*
@@ -452,6 +460,8 @@ xlog_recover_validate_buf_type( bip = bp->b_log_item; bip->bli_item.li_lsn = current_lsn; } + +return 0 ; }/*
@@ -523,8 +533,7 @@ xlog_recover_do_reg_buffer( bit += nbits; } -xlog_recover_validate_buf_type (mp, bp, buf_f, current_lsn); -return 0 ; +return xlog_recover_validate_buf_type (mp, bp, buf_f, current_lsn); }/*
Signature · 1 line
2.55.0
PATCH 48/48 xfs: validate logged dquot buffers during recovery
In reply to Dave ChinnerA dquot buffer is only ever logged by xfs_qm_init_dquot_blk(), which initialises and logs the whole buffer in a single region. So a logged dquot buffer has exactly two regions, the format header and one data region covering the entire buffer, and that region holds a full chunk of valid dquots.
Validate this in the buffer item validate_item callback. A buffer carrying a dquot type flag must have that shape, and every dquot in the logged chunk is verified so a corrupt dquot is never replayed to disk. The dquot content check in the replay path only looked at the first dquot and relied on the replay loop for the region bounds, so it is removed and the replay loop reduces to copying the logged regions into the buffer.
To share the per-dquot verification, xfs_dquot_verify_chunk() is factored out of xfs_dquot_buf_verify(). It walks a chunk of dquots and, on failure, updates its dquot pointer to the offending dquot so the caller can report it. The buffer read verifier is otherwise unchanged.
--- fs/xfs/libxfs/xfs_dquot_buf.c | 64 ++++++++++++++++++------------- fs/xfs/libxfs/xfs_quota_defs.h | 2 + fs/xfs/xfs_buf_item_recover.c | 70 +++++++++++++++++++--------------- 3 files changed, 78 insertions(+), 58 deletions(-)
fs/xfs/libxfs/xfs_dquot_buf.c +37 -27
@@ -179,6 +179,38 @@ xfs_dquot_buf_verify_crc(return true; } +/* +* Verify every dquot in a full chunk of dquots. We don't know what the id of +* the first dquot is supposed to be, just that they should be increasing +* monotonically within the chunk. If the first id is corrupt, then it will fail +* on the second dquot in the chunk so corruptions could point to the wrong +* dquot in this case. On failure @dqb is updated to point at the failing dquot +* so the caller can report it. +*/ +xfs_failaddr_t +xfs_dquot_verify_chunk ( +struct xfs_mount *mp, +struct xfs_dqblk **dqb, +unsigned int ndquots) +{ +xfs_failaddr_t fa; +xfs_dqid_t id =0 ; +unsigned int i; + +for (i =0 ; i < ndquots; i++) { +if (i ==0 ) + id =be32_to_cpu ((*dqb)[i].dd_diskdq.d_id); + + fa =xfs_dqblk_verify (mp, &(*dqb)[i], id + i); +if (fa) { + *dqb = &(*dqb)[i]; +return fa; + } + } + +return NULL ; +} + STATIC xfs_failaddr_t xfs_dquot_buf_verify( struct xfs_mount *mp,
@@ -187,9 +219,7 @@ xfs_dquot_buf_verify( {struct xfs_dqblk *dqb = bp->b_addr;xfs_failaddr_t fa; -xfs_dqid_t id =0 ;int ndquots; -int i;/* * if we are in log recovery, the quota subsystem has not been
@@ -201,32 +231,12 @@ xfs_dquot_buf_verify(else ndquots =xfs_calc_dquots_per_chunk (bp->b_length); -/* -* On the first read of the buffer, verify that each dquot is valid. -* We don't know what the id of the dquot is supposed to be, just that -* they should be increasing monotonically within the buffer. If the -* first id is corrupt, then it will fail on the second dquot in the -* buffer so corruptions could point to the wrong dquot in this case. -*/ -for (i =0 ; i < ndquots; i++) { -struct xfs_disk_dquot *ddq; + fa =xfs_dquot_verify_chunk (mp, &dqb, ndquots); +if (fa && !readahead) +xfs_buf_verifier_error (bp, -EFSCORRUPTED , __func__, dqb, +sizeof (struct xfs_dqblk ), fa); - ddq = &dqb[i].dd_diskdq; - -if (i ==0 ) - id =be32_to_cpu (ddq->d_id); - - fa =xfs_dqblk_verify (mp, &dqb[i], id + i); -if (fa) { -if (!readahead) -xfs_buf_verifier_error (bp, -EFSCORRUPTED , - __func__, &dqb[i], -sizeof (struct xfs_dqblk ), fa); -return fa; - } - } - -return NULL ; +return fa; } static xfs_failaddr_t
fs/xfs/libxfs/xfs_quota_defs.h +2 -0
@@ -132,6 +132,8 @@ extern xfs_failaddr_t xfs_dquot_verify(struct xfs_mount *mp, struct xfs_disk_dquot *ddq, xfs_dqid_t id);extern xfs_failaddr_t xfs_dqblk_verify (struct xfs_mount *mp,struct xfs_dqblk *dqb,xfs_dqid_t id); +xfs_failaddr_t xfs_dquot_verify_chunk (struct xfs_mount *mp, +struct xfs_dqblk **dqb,unsigned int ndquots);extern int xfs_calc_dquots_per_chunk (unsigned int nbblks);extern void xfs_dqblk_repair (struct xfs_mount *mp,struct xfs_dqblk *dqb,xfs_dqid_t id,xfs_dqtype_t type);
fs/xfs/xfs_buf_item_recover.c +39 -31
98 lines not on this page · raw message
Signature · 1 line
2.55.0
Re: [PATCH 02/48] xfs: add log recovery item validation design document
In reply to Dave ChinnerQuoted 17 lines · Dave Chinner
Add a design document describing a systematic approach to validating log items during journal recovery. Log item data read from the journal cannot be fully trusted - corruption from torn writes, hardware errors or software bugs can produce invalid type codes, wrong region counts, truncated regions or internally inconsistent format structures that the current recovery code largely uses unchecked.
The document describes a three-layer validation scheme (generic region header decode, per-region type-specific validation, and full cross-region item validation), the treatment of the transaction header as a validated region type, and a set of additional safety fixes for the generic region assembly code. It also lays out a phased implementation plan.
Changes:
- AddDocumentation/filesystems/xfs/xfs-log-recovery-validation-design.rst
- Add the new document to the XFS documentation index
Quoted 5 lines
Created-by-AI: Claude Opus 4.8 (1M context)
Prompt: Let's start by converting the existing verification doc to kernel .rst format and moving it to
Documentation/filesystems/xfs/and adding it to the relevant doc index file. The file should be tracked in a new patch.
Eheheh.
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst
Quoted 29 lines
Refinements:
- Reordered the Phase 1 implementation plan so the ops-struct validation infrastructure (min_regions/max_regions/min_hdr_len fields and thevalidate_regioncallback) is added first (Patch 1), followed by the transaction header ops entry that uses it (Patch 2). Downstream patches renumbered accordingly so each patch builds cleanly on its own.--- Documentation/filesystems/xfs/index.rst | 1 + .../xfs-log-recovery-validation-design.rst | 477 ++++++++++++++++++ 2 files changed, 478 insertions(+) create mode 100644 Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rstDocumentation/filesystems/xfs/index.rst +1
-0@@ -9,6 +9,7 @@ XFS Filesystem Documentation :numbered: xfs-delayed-logging-design + xfs-log-recovery-validation-design xfs-maintainer-entry-profile xfs-self-describing-metadata xfs-online-fsck-designDocumentation/filesystems/xfs/xfs-log-recovery-validation-design.rst new file +0
-0
The document sounds pretty reasonable for a first pass at hardening the log recovery code. This is a good start.
<snip>
Quoted 2 lines
+Layer 3: Full item validation (type-specific, cross-region)
+-----------------------------------------------------------
<snip>
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst line 331
Quoted 2 lines
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst new file +2
-0+Other intent/done items + similar field-level validation
But please be more specific.
Quoted 8 lines
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst new file +8
-0+ICREATE + - Already has validation in + ``xlog_recover_icreate_commit_pass2()``, move the checks to + ``validate_item`` + +QUOTAOFF + - Verify ``qf_flags`` contains only valid quota flags +
<snip>
Quoted 12 lines
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst new file +12
-0+Phase 3: Full item validation +----------------------------- + +Patch M: Add validate_item callback infrastructure + - Add the callback to the ops struct + - Call it from ``xlog_recover_reorder_trans()`` after ops verification + - Return ``-EFSCORRUPTED`` on failure, aborting recovery + +Patches M+1 to M+N: Implement validate_item for each item type + - Move existing validation out of ``commit_pass2`` into + ``validate_item`` where possible (e.g. ICREATE field checks) + - Add new cross-region validation
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst line 477
Quoted 10 lines
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst new file +10
-0+ - Order: simple types first, complex types last + - INODE validation is the most complex (fork format vs region size) + +Phase 4: Cleanup +---------------- + +- Remove redundant validation from ``commit_pass1``/``commit_pass2`` that + is now covered by ``validate_region``/``validate_item`` +- Add ``ASSERT()``\ s in commit handlers to verify validation has run +- Review and update error messages for consistency
Why is the plan for splitting scope across M+N patches relevant for a design document that will live in the kernel?
--D
Re: [PATCH 02/48] xfs: add log recovery item validation design document
Quoted 18 lines · Darrick J. Wong
Quoted 17 lines · Dave Chinner
Add a design document describing a systematic approach to validating log items during journal recovery. Log item data read from the journal cannot be fully trusted - corruption from torn writes, hardware errors or software bugs can produce invalid type codes, wrong region counts, truncated regions or internally inconsistent format structures that the current recovery code largely uses unchecked.
The document describes a three-layer validation scheme (generic region header decode, per-region type-specific validation, and full cross-region item validation), the treatment of the transaction header as a validated region type, and a set of additional safety fixes for the generic region assembly code. It also lays out a phased implementation plan.
Changes:
- AddDocumentation/filesystems/xfs/xfs-log-recovery-validation-design.rst
- Add the new document to the XFS documentation index
Quoted 7 lines
Quoted 5 lines
Created-by-AI: Claude Opus 4.8 (1M context)
Prompt: Let's start by converting the existing verification doc to kernel .rst format and moving it to
Documentation/filesystems/xfs/and adding it to the relevant doc index file. The file should be tracked in a new patch.Eheheh.
Yeah, I forgot this was written about 3 months ago before I completely rewrote all the patch header generation skills. If you read through the patch headers, you'll probably spot changes in the way the read that coincide with refinements to the skills...
i.e. my workflow has been rapidly evolving over the past few months, and this patch series spans quite a lot of change and refinement...
Quoted 8 lines
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst
Quoted 5 lines
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst new file +0
-0The document sounds pretty reasonable for a first pass at hardening the log recovery code. This is a good start.
OK, that's good to hear.
Quoted 11 lines
<snip>
Quoted 2 lines
+Layer 3: Full item validation (type-specific, cross-region)
+-----------------------------------------------------------<snip>
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst line 331
Quoted 2 lines
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst new file +2
-0+Other intent/done items + similar field-level validationBut please be more specific.
Yup, it needs to be revised to match what that actually meant :)
Quoted 17 lines
Quoted 17 lines
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst new file +17
-0+Phase 3: Full item validation +----------------------------- + +Patch M: Add validate_item callback infrastructure + - Add the callback to the ops struct + - Call it from ``xlog_recover_reorder_trans()`` after ops verification + - Return ``-EFSCORRUPTED`` on failure, aborting recovery + +Patches M+1 to M+N: Implement validate_item for each item type + - Move existing validation out of ``commit_pass2`` into + ``validate_item`` where possible (e.g. ICREATE field checks) + - Add new cross-region validation + - Order: simple types first, complex types last + - INODE validation is the most complex (fork format vs region size) + +Phase 4: Cleanup +----------------
Quoted 8 lines
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst line 477
Quoted 5 lines
Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst new file +5
-0+ +- Remove redundant validation from ``commit_pass1``/``commit_pass2`` that + is now covered by ``validate_region``/``validate_item`` +- Add ``ASSERT()``\ s in commit handlers to verify validation has run +- Review and update error messages for consistencyWhy is the plan for splitting scope across M+N patches relevant for a design document that will live in the kernel?
It's not. This document was derived from the original planning work I did in the LLM. I forgot to remove this stuff once the LLM didn't need it as a reference anymore. I'll match sure the design doc is updated before the next version.
FWIW, the reason for writing the design/planning doc in this way was to give the LLM something to re-reference whilst working through the patch series. The plannig and coding wasn't all done in one session. The work was also large enough in scope that frequent context compaction was required to manage context size and minimise token cache re-read costs. I simply forgot to tell it to clean up the design doc as the work was completed.
I've since changed how I record the implementation plan for new development - the design docs still exist for the "LLM needs guidance" reasons, but I use LLM session memory for maintaining the implementation plan now. Hence the LLM now automatically removes work from the plan as it is complete, and I don't have to remember to tell it to do so anymore. That's likely why I forgot about this altogether...
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]