Skip to contentSkip to conversation

libceph: Reset sparse-read on fault

PATCH In review Sam Edwards 2 participants 2 messages Dec 31, 2025 – Jan 5

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 libceph: Reset sparse-read on fault

When a fault occurs, the connection is abandoned, reestablished, and any pending operations are retried. The OSD client tracks the progress of a sparse-read reply using a separate state machine, largely independent of the messenger's state.

If a connection is lost mid-payload or the sparse-read state machine returns an error, the sparse-read state is not reset. The OSD client will then interpret the beginning of a new reply as the continuation of the old one. If this makes the sparse-read machinery enter a failure state, it may never recover, producing loops like:

libceph: [0] got 0 extents
libceph: data len 142248331 != extent len 0
libceph: osd0 (1)...:6801 socket error on read
libceph: data len 142248331 != extent len 0
libceph: osd0 (1)...:6801 socket error on read

Therefore, reset the sparse-read state in osd_fault(), ensuring retries start from a clean state.

---
 net/ceph/osd_client.c | 3 +++
 1 file changed, 3 insertions(+)
net/ceph/osd_client.c +3 -0
index 3667319b949d..1a7be2f615dc 100644
@@ -4281,6 +4281,9 @@ static void osd_fault(struct ceph_connection *con)
 		goto out_unlock;
 	}
 
+	osd->o_sparse_op_idx = -1;
+	ceph_init_sparse_read(&osd->o_sparse_read);
+
 	if (!reopen_osd(osd))
 		kick_osd_requests(osd);
 	maybe_request_map(osdc);
Signature · 1 line
2.51.2
maintainer
#1 lore raw
Quoted 42 lines · Sam Edwards

When a fault occurs, the connection is abandoned, reestablished, and any pending operations are retried. The OSD client tracks the progress of a sparse-read reply using a separate state machine, largely independent of the messenger's state.

If a connection is lost mid-payload or the sparse-read state machine returns an error, the sparse-read state is not reset. The OSD client will then interpret the beginning of a new reply as the continuation of the old one. If this makes the sparse-read machinery enter a failure state, it may never recover, producing loops like:

libceph: [0] got 0 extents
libceph: data len 142248331 != extent len 0
libceph: osd0 (1)...:6801 socket error on read
libceph: data len 142248331 != extent len 0
libceph: osd0 (1)...:6801 socket error on read

Therefore, reset the sparse-read state in osd_fault(), ensuring retries start from a clean state.

---
 net/ceph/osd_client.c | 3 +++
 1 file changed, 3 insertions(+)
net/ceph/osd_client.c +3 -1
index 3667319b949d..1a7be2f615dc 100644
@@ -4281,6 +4281,9 @@ static void osd_fault(struct ceph_connection *con)
                goto out_unlock;
        }
 
+       osd->o_sparse_op_idx = -1;
+       ceph_init_sparse_read(&osd->o_sparse_read);
+
        if (!reopen_osd(osd))
                kick_osd_requests(osd);
        maybe_request_map(osdc);
--
Quoted 2 lines

2.51.2

Hi Sam,

Good catch! Applied (with the sad note that support for sparse reads is officially the most problematic patchset that ever made it into libceph).

Thanks,

Ilya