folio changes during io results in CIFS send failures
Hi David and Steve,
When running the fio write workload
fio --name=smb-fio \ --directory=/mnt \ --rw=write \ --bs=4k \ --size=4k \ --ioengine=io_uring \ --direct=0 \ --numjobs=1 \ --iodepth=1 \ --time_based=1 \ --runtime=500 \ --unlink=0 \ --group_reporting=1 \ --filename_format="$file_fmt" \ --output-format=json \ --output=smb-fio.json
on a mounted CIFS share I saw the messages
CIFS: VFS: sign fail cmd 0x9 message id 0x49127 CIFS: VFS: \\win22.bira.test SMB signature verification returned error = -13
After debugging the signed request, sent request, signature and recalculated signature on sent request I found out that a folio content change is happening between signature and sending the data, while the signature is not changing. Some debug messages for reference:
CIFS: sign-write-req-sig-changed conn=1 mid=0x49127 hdr_mid=0x49127 flags=0x8 cc=64 cr=72 tid=0x5 sid=0x2880074000071 iter_1=4194304 iter_2=4194304 sig_1=92eeb2b661423fc65c86401a4c14888f sig_2=8e53f3f500d42122183c8c74227520e7 CIFS: sign-write-req-delta conn=3 mid=0xccb4 first_diff=168 last_diff=3754 diff_bytes=24 diff_ranges=8 len_1=4208 len_2=4208 iter_1=4096 iter_2=4096 CIFS: sign-write-req-delta-range conn=3 mid=0xccb4 range=0 off=168 len=3 buf_1=1d5e730f000000000000000000000000b62c7d9d5f374dcc11c33b6c0fd37332 buf_2=9db87c0f000000000000000000000000b62c7d9d5f374dcc11c33b6c0fd37332 ^^^^^^
first_diff above shows the first changed byte is payload offset 56 (112 byte header) and we can see a 3-byte change there. This same 3-byte change repeats every 512 bytes in the payload.
After digging into netfs code, I noticed the folios are not kept locked while a subrequest is being filled, and maybe this allows changes to this folio even though it has not been marked as ended writeback.
In my case, the subrequest can be much larger than a single folio, so netfs accumulates several 4k folios before issuing the network write.
So a minimal change that prevented this unwanted change during write is the following -- *although I'm likely missing something*.
If you think this makes sense I can send a formal patch.
fs/netfs/write_collect.c +2 -0
@@ -63,6 +63,8 @@ int netfs_folio_written_back(struct folio *folio)struct netfs_group *group =NULL ;int gcount =0 ; +folio_unlock (folio); +if ((finfo =netfs_folio_info (folio))) {/* Streaming writes cannot be redirtied whilst under writeback, * so discard the streaming record.
fs/netfs/write_issue.c +0 -2
@@ -347,7 +347,6 @@ static int netfs_write_folio(struct netfs_io_request *wreq,/* mmap beyond eof. */ _debug ("beyond eof" );folio_start_writeback (folio); -folio_unlock (folio); wreq->nr_group_rel +=netfs_folio_written_back (folio);netfs_put_group_many (wreq->group, wreq->nr_group_rel); wreq->nr_group_rel =0 ;
@@ -415,7 +414,6 @@ static int netfs_write_folio(struct netfs_io_request *wreq,netfs_issue_write (wreq, cache);folio_start_writeback (folio); -folio_unlock (folio);if (fgroup ==NETFS_FOLIO_COPY_TO_CACHE ) {if (!cache->avail) {
Signature · 1 line
Henrique