Skip to contentSkip to conversation

scsi_debug/tlibio/lio_write_buffer: Always return total amount of written bytes

PATCH Superseded Konstantin Khorenko via ltp 2 participants 5 messages May 16, 2023 – May 18, 2023 Library code

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 scsi_debug/tlibio/lio_write_buffer: Always return total amount of written bytes

Sometimes we got failures like:
growfiles(gf217): 65884 growfiles.c/2262: 104203 tlibio.c/744 write(3, buf, 5000) returned=2288 growfiles(gf217): 65884 growfiles.c/1765: 104203 Hit max errors value of 1
gf217 1 TFAIL : growfiles.c:134: Test failed

Which looked strange as partial write is something usual and valid. It turned out that lio_write_buffer() has the code cycle writes in case of a partial write happens, but it anyway returns the amount of bytes written by the LAST write.

And upper growfile() consider the returned amount from lio_write_buffer() to be less than it tried to write and fails the testcase.

Fix lio_write_buffer() to always return total bytes written, even in case partial writes.

---
 lib/tlibio.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
lib/tlibio.c +4 -1
index cc110d1c9..8298e2de9 100644
@@ -539,6 +539,8 @@ int lio_write_buffer(int fd,		/* open file descriptor */
 		long wrd)	/* to allow future features, use zero for now */
 {
 	int ret = 0;		/* syscall return or used to get random method */
+	int totally_written = 0;/* as we cycle writes in case of partial writes, */
+				/* we have to report up total bytes written */
 	char *io_type;		/* Holds string of type of io */
 	int omethod = method;
 	int listio_cmd;		/* Holds the listio/lio_listio cmd */
@@ -745,13 +747,14 @@ int lio_write_buffer(int fd,		/* open file descriptor */
 						fd, size, ret);
 					size -= ret;
 					buffer += ret;
+					totally_written += ret;
 				} else {
 					if (Debug_level > 1)
 						printf
 						    ("DEBUG %s/%d: write completed without error (ret %d)\n",
 						     __FILE__, __LINE__, ret);
 
-					return ret;
+					return totally_written + ret;
 				}
 			}
 			wait4sync_io(fd, 0);
Signature · 5 lines
2.31.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#1 lore raw

PATCH 2/2 scsi_debug/tlibio/lio_read_buffer: Always return total amount of read bytes

This issue behind this patch is noticed while fixing the lio_write_buffer().

Here in lio_read_buffer() we have similar situation: in case of a partial read, we cycle, but lio_read_buffer() returns back the amount of bytes read during the last read() call while it's expected to return the whole amount of read bytes.

---
 lib/tlibio.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
lib/tlibio.c +4 -1
index 8298e2de9..604663307 100644
@@ -1112,6 +1112,8 @@ int lio_read_buffer(int fd,	/* open file descriptor */
 		long wrd)	/* to allow future features, use zero for now */
 {
 	int ret = 0;		/* syscall return or used to get random method */
+	int totally_read = 0;	/* as we cycle reads in case of partial reads, */
+				/* we have to report up total bytes read */
 	char *io_type;		/* Holds string of type of io */
 	int listio_cmd;		/* Holds the listio/lio_listio cmd */
 	int omethod = method;
@@ -1323,13 +1325,14 @@ int lio_read_buffer(int fd,	/* open file descriptor */
 						fd, size, ret);
 					size -= ret;
 					buffer += ret;
+					totally_read += ret;
 				} else {
 					if (Debug_level > 1)
 						printf
 						    ("DEBUG %s/%d: read completed without error (ret %d)\n",
 						     __FILE__, __LINE__, ret);
 
-					return ret;
+					return totally_read + ret;
 				}
 			}
 			wait4sync_io(fd, 1);
Signature · 5 lines
2.31.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
Reviewed-by on v1 2/2
#2 lore raw

Re: [LTP] [PATCH 2/2] scsi_debug/tlibio/lio_read_buffer: Always return total amount of read bytes

Quoted 18 lines

This issue behind this patch is noticed while fixing the lio_write_buffer().

Here in lio_read_buffer() we have similar situation: in case of a partial read, we cycle, but lio_read_buffer() returns back the amount of bytes read during the last read() call while it's expected to return the whole amount of read bytes.

---
 lib/tlibio.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
lib/tlibio.c +0 -0
index 8298e2de9..604663307 100644

lib/tlibio.c int lio_read_buffer(int fd, /* open file descriptor */ · line 1116

Quoted 5 lines
lib/tlibio.c +2 -0
@@ -1112,6 +1112,8 @@ int lio_read_buffer(int fd,	/* open file descriptor */
 		long wrd)	/* to allow future features, use zero for now */
 {
 	int ret = 0;		/* syscall return or used to get random method */
+	int totally_read = 0;	/* as we cycle reads in case of partial reads, */
+				/* we have to report up total bytes read */

Again, multiline comment would be more readable:

	/* as we cycle reads in case of partial reads,
	 * we have to report up total bytes read */
	int totally_read = 0;

Kind regards,
Petr

Quoted 19 lines
lib/tlibio.c +2 -1
 	char *io_type;		/* Holds string of type of io */
 	int listio_cmd;		/* Holds the listio/lio_listio cmd */
 	int omethod = method;
@@ -1323,13 +1325,14 @@ int lio_read_buffer(int fd,	/* open file descriptor */
 						fd, size, ret);
 					size -= ret;
 					buffer += ret;
+					totally_read += ret;
 				} else {
 					if (Debug_level > 1)
 						printf
 						    ("DEBUG %s/%d: read completed without error (ret %d)\n",
 						     __FILE__, __LINE__, ret);
 
-					return ret;
+					return totally_read + ret;
 				}
 			}
 			wait4sync_io(fd, 1);
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Reviewed-by on v1 1/2
#3 lore raw
In reply to Konstantin Khorenko via ltp

Hi Konstantin,

Quoted 4 lines

Sometimes we got failures like:
growfiles(gf217): 65884 growfiles.c/2262: 104203 tlibio.c/744 write(3, buf, 5000) returned=2288 growfiles(gf217): 65884 growfiles.c/1765: 104203 Hit max errors value of 1
gf217 1 TFAIL : growfiles.c:134: Test failed

I wonder on which circumstances do you get this error. Running on container?

Quoted 22 lines

Which looked strange as partial write is something usual and valid. It turned out that lio_write_buffer() has the code cycle writes in case of a partial write happens, but it anyway returns the amount of bytes written by the LAST write.

And upper growfile() consider the returned amount from lio_write_buffer() to be less than it tried to write and fails the testcase.

Fix lio_write_buffer() to always return total bytes written, even in case partial writes.

---
 lib/tlibio.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
lib/tlibio.c +0 -0
index cc110d1c9..8298e2de9 100644

lib/tlibio.c int lio_write_buffer(int fd, /* open file descriptor */ · line 543

Quoted 5 lines
lib/tlibio.c +2 -0
@@ -539,6 +539,8 @@ int lio_write_buffer(int fd,		/* open file descriptor */
 		long wrd)	/* to allow future features, use zero for now */
 {
 	int ret = 0;		/* syscall return or used to get random method */
+	int totally_written = 0;/* as we cycle writes in case of partial writes, */
+				/* we have to report up total bytes written */

Nit: I'd do multiline comment (much more readable):

	/* as we cycle writes in case of partial writes,
	 * we have to report up total bytes written */
	int totally_written = 0;

Otherwise LGTM.

lib/tlibio.c int lio_write_buffer(int fd, /* open file descriptor */ · line 755

Quoted 13 lines
lib/tlibio.c +1 -0
 	char *io_type;		/* Holds string of type of io */
 	int omethod = method;
 	int listio_cmd;		/* Holds the listio/lio_listio cmd */
@@ -745,13 +747,14 @@ int lio_write_buffer(int fd,		/* open file descriptor */
 						fd, size, ret);
 					size -= ret;
 					buffer += ret;
+					totally_written += ret;
 				} else {
 					if (Debug_level > 1)
 						printf
 						    ("DEBUG %s/%d: write completed without error (ret %d)\n",
 						     __FILE__, __LINE__, ret);

BTW growfiles.c and other files based on tlibio.c would deserve big cleanup and rewrite into new API.

Kind regards,
Petr

Quoted 5 lines
lib/tlibio.c +1 -1
-					return ret;
+					return totally_written + ret;
 				}
 			}
 			wait4sync_io(fd, 0);
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#4 lore raw
Quoted 8 lines · Petr Vorel

Hi Konstantin,

Quoted 4 lines

Sometimes we got failures like:
growfiles(gf217): 65884 growfiles.c/2262: 104203 tlibio.c/744 write(3, buf, 5000) returned=2288 growfiles(gf217): 65884 growfiles.c/1765: 104203 Hit max errors value of 1
gf217 1 TFAIL : growfiles.c:134: Test failed

I wonder on which circumstances do you get this error. Running on container?

Yes, your guess is correct. :) Our QA runs ltp in Containers and VMs under high load, i mean when many Containers/VMs host on a physical Node and run in parallel different tests.

Quoted 17 lines
Quoted 16 lines

Which looked strange as partial write is something usual and valid. It turned out that lio_write_buffer() has the code cycle writes in case of a partial write happens, but it anyway returns the amount of bytes written by the LAST write.

And upper growfile() consider the returned amount from lio_write_buffer() to be less than it tried to write and fails the testcase.

Fix lio_write_buffer() to always return total bytes written, even in case partial writes.

---
  lib/tlibio.c | 5 ++++-
  1 file changed, 4 insertions(+), 1 deletion(-)
Quoted 12 lines

lib/tlibio.c int lio_write_buffer(int fd, /* open file descriptor */ · line 543

Quoted 10 lines
lib/tlibio.c +2 -0
index cc110d1c9..8298e2de9 100644
@@ -539,6 +539,8 @@ int lio_write_buffer(int fd,		/* open file descriptor */
  		long wrd)	/* to allow future features, use zero for now */
  {
  	int ret = 0;		/* syscall return or used to get random method */
+	int totally_written = 0;/* as we cycle writes in case of partial writes, */
+				/* we have to report up total bytes written */

Nit: I'd do multiline comment (much more readable):

No problem, will redo and resend.

Thank you for the review!

Best regards,
Konstantin Khorenko

Quoted 34 lines
	/* as we cycle writes in case of partial writes,
	 * we have to report up total bytes written */
	int totally_written = 0;

Otherwise LGTM.

lib/tlibio.c int lio_write_buffer(int fd, /* open file descriptor */ · line 755

Quoted 13 lines
lib/tlibio.c +1 -0
  	char *io_type;		/* Holds string of type of io */
  	int omethod = method;
  	int listio_cmd;		/* Holds the listio/lio_listio cmd */
@@ -745,13 +747,14 @@ int lio_write_buffer(int fd,		/* open file descriptor */
  						fd, size, ret);
  					size -= ret;
  					buffer += ret;
+					totally_written += ret;
  				} else {
  					if (Debug_level > 1)
  						printf
  						    ("DEBUG %s/%d: write completed without error (ret %d)\n",
  						     __FILE__, __LINE__, ret);

BTW growfiles.c and other files based on tlibio.c would deserve big cleanup and rewrite into new API.

Kind regards,
Petr

Quoted 5 lines
lib/tlibio.c +1 -1
-					return ret;
+					return totally_written + ret;
  				}
  			}
  			wait4sync_io(fd, 0);
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp