Skip to contentSkip to conversation

um: Fix WRITE_ZEROES in the UBD Driver

PATCH In review Frédéric Danis 2 participants 9 messages Jan 25, 2022 – Feb 16, 2022 User-mode Linux (UML)

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 um: Fix WRITE_ZEROES in the UBD Driver

Call to fallocate with FALLOC_FL_PUNCH_HOLE on a device backed by a sparse file can end up by missing data, zeroes data range, if the underlying file is used with a tool like bmaptool which will referenced only used spaces.

---
 arch/um/drivers/ubd_kern.c  | 8 +++++++-
 arch/um/include/shared/os.h | 1 +
 arch/um/os-Linux/file.c     | 9 +++++++++
 3 files changed, 17 insertions(+), 1 deletion(-)
arch/um/drivers/ubd_kern.c +7 -1
index 69d2d0049a61..b03269faef71 100644
@@ -1526,13 +1526,19 @@ static void do_io(struct io_thread_req *req, struct io_desc *desc)
 			}
 			break;
 		case REQ_OP_DISCARD:
-		case REQ_OP_WRITE_ZEROES:
 			n = os_falloc_punch(req->fds[bit], off, len);
 			if (n) {
 				req->error = map_error(-n);
 				return;
 			}
 			break;
+		case REQ_OP_WRITE_ZEROES:
+			n = os_falloc_zeroes(req->fds[bit], off, len);
+			if (n) {
+				req->error = map_error(-n);
+				return;
+			}
+			break;
 		default:
 			WARN_ON_ONCE(1);
 			req->error = BLK_STS_NOTSUPP;
arch/um/include/shared/os.h +1 -0
index 00214059d9ec..fafde1d5416e 100644
@@ -168,6 +168,7 @@ extern unsigned os_major(unsigned long long dev);
 extern unsigned os_minor(unsigned long long dev);
 extern unsigned long long os_makedev(unsigned major, unsigned minor);
 extern int os_falloc_punch(int fd, unsigned long long offset, int count);
+extern int os_falloc_zeroes(int fd, unsigned long long offset, int count);
 extern int os_eventfd(unsigned int initval, int flags);
 extern int os_sendmsg_fds(int fd, const void *buf, unsigned int len,
 			  const int *fds, unsigned int fds_num);
arch/um/os-Linux/file.c +9 -0
index e4421dbc4c36..fc4450db59bd 100644
@@ -625,6 +625,15 @@ int os_falloc_punch(int fd, unsigned long long offset, int len)
 	return n;
 }
 
+int os_falloc_zeroes(int fd, unsigned long long offset, int len)
+{
+	int n = fallocate(fd, FALLOC_FL_ZERO_RANGE|FALLOC_FL_KEEP_SIZE, offset, len);
+
+	if (n < 0)
+		return -errno;
+	return n;
+}
+
 int os_eventfd(unsigned int initval, int flags)
 {
 	int fd = eventfd(initval, flags);
Signature · 7 lines
2.25.1


_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um
maintainer
#1 lore raw
Quoted 3 lines · Frédéric Danis

Call to fallocate with FALLOC_FL_PUNCH_HOLE on a device backed by a sparse file can end up by missing data, zeroes data range, if the underlying file is used with a tool like bmaptool which will referenced only used spaces.

Can you elaborate on when do you miss data.

A file with a hole created using FALLOC_FL_PUNCH_HOLE when reading from the hole should return zeroes.

If it returns anything else or if the zeroed range exceeds the one specified in falloc - that is an underlying fs bug.

Quoted 65 lines
---
  arch/um/drivers/ubd_kern.c  | 8 +++++++-
  arch/um/include/shared/os.h | 1 +
  arch/um/os-Linux/file.c     | 9 +++++++++
  3 files changed, 17 insertions(+), 1 deletion(-)
arch/um/drivers/ubd_kern.c +7 -1
index 69d2d0049a61..b03269faef71 100644
@@ -1526,13 +1526,19 @@ static void do_io(struct io_thread_req *req, struct io_desc *desc)
  			}
  			break;
  		case REQ_OP_DISCARD:
-		case REQ_OP_WRITE_ZEROES:
  			n = os_falloc_punch(req->fds[bit], off, len);
  			if (n) {
  				req->error = map_error(-n);
  				return;
  			}
  			break;
+		case REQ_OP_WRITE_ZEROES:
+			n = os_falloc_zeroes(req->fds[bit], off, len);
+			if (n) {
+				req->error = map_error(-n);
+				return;
+			}
+			break;
  		default:
  			WARN_ON_ONCE(1);
  			req->error = BLK_STS_NOTSUPP;
arch/um/include/shared/os.h +1 -0
index 00214059d9ec..fafde1d5416e 100644
@@ -168,6 +168,7 @@ extern unsigned os_major(unsigned long long dev);
  extern unsigned os_minor(unsigned long long dev);
  extern unsigned long long os_makedev(unsigned major, unsigned minor);
  extern int os_falloc_punch(int fd, unsigned long long offset, int count);
+extern int os_falloc_zeroes(int fd, unsigned long long offset, int count);
  extern int os_eventfd(unsigned int initval, int flags);
  extern int os_sendmsg_fds(int fd, const void *buf, unsigned int len,
  			  const int *fds, unsigned int fds_num);
arch/um/os-Linux/file.c +9 -0
index e4421dbc4c36..fc4450db59bd 100644
@@ -625,6 +625,15 @@ int os_falloc_punch(int fd, unsigned long long offset, int len)
  	return n;
  }
 
+int os_falloc_zeroes(int fd, unsigned long long offset, int len)
+{
+	int n = fallocate(fd, FALLOC_FL_ZERO_RANGE|FALLOC_FL_KEEP_SIZE, offset, len);
+
+	if (n < 0)
+		return -errno;
+	return n;
+}
+
  int os_eventfd(unsigned int initval, int flags)
  {
  	int fd = eventfd(initval, flags);
Signature · 9 lines
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/


_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um
#2 lore raw

Hi Anton,

Quoted 10 lines · Anton Ivanov
Quoted 3 lines · Frédéric Danis

Call to fallocate with FALLOC_FL_PUNCH_HOLE on a device backed by a sparse file can end up by missing data, zeroes data range, if the underlying file is used with a tool like bmaptool which will referenced only used spaces.

Can you elaborate on when do you miss data.

A file with a hole created using FALLOC_FL_PUNCH_HOLE when reading from the hole should return zeroes.

If it returns anything else or if the zeroed range exceeds the one specified in falloc - that is an underlying fs bug.

I use bmaptool to create a block map of the used part of a raw system image file, then compress the file and at the end use again bmaptool to flash the file to a SDCard.

iiuc fallocate() call with FALLOC_FL_PUNCH_HOLE doesn't "register" the space, as it is zeroed, so when bmaptool creates the block map it doesn't reference the "used holes", and when I flash the SDCard those parts are missing, so the filesystem on the SDCard is corrupted.

Quoted 71 lines
Quoted 69 lines

---
  arch/um/drivers/ubd_kern.c  | 8 +++++++-
  arch/um/include/shared/os.h | 1 +
  arch/um/os-Linux/file.c     | 9 +++++++++

  3 files changed, 17 insertions(+), 1 deletion(-)
arch/um/drivers/ubd_kern.c +0 -0
index 69d2d0049a61..b03269faef71 100644
@@ -1526,13 +1526,19 @@ static void do_io(struct io_thread_req *req, 
 struct io_desc *desc)
               }
               break;

          case REQ_OP_DISCARD:
-        case REQ_OP_WRITE_ZEROES:
              n = os_falloc_punch(req->fds[bit], off, len); if (n) {
                  req->error = map_error(-n); return;
              }
              break;
+        case REQ_OP_WRITE_ZEROES:
+            n = os_falloc_zeroes(req->fds[bit], off, len); +            if (n) {
+                req->error = map_error(-n);
+                return;
+            }
+            break;
          default:
              WARN_ON_ONCE(1);
              req->error = BLK_STS_NOTSUPP;

arch/um/include/shared/os.h +1 -0
index 00214059d9ec..fafde1d5416e 100644
@@ -168,6 +168,7 @@ extern unsigned os_major(unsigned long long dev);
   extern unsigned os_minor(unsigned long long dev);
   extern unsigned long long os_makedev(unsigned major, unsigned minor);
   extern int os_falloc_punch(int fd, unsigned long long offset, int count);
+extern int os_falloc_zeroes(int fd, unsigned long long offset, int 
 count);
   extern int os_eventfd(unsigned int initval, int flags);
   extern int os_sendmsg_fds(int fd, const void *buf, unsigned int len,

                const int *fds, unsigned int fds_num);

arch/um/os-Linux/file.c +8 -0
index e4421dbc4c36..fc4450db59bd 100644
@@ -625,6 +625,15 @@ int os_falloc_punch(int fd, unsigned long long 
 offset, int len)
       return n;
   }
   +int os_falloc_zeroes(int fd, unsigned long long offset, int len)
+{
+    int n = fallocate(fd, FALLOC_FL_ZERO_RANGE|FALLOC_FL_KEEP_SIZE, 
 offset, len);
+
+    if (n < 0)
+        return -errno;
+    return n;
+}
+
   int os_eventfd(unsigned int initval, int flags)

  {
      int fd = eventfd(initval, flags);

Signature · 12 lines
Frédéric Danis
Senior Software Engineer

Collabora Ltd.
Platinum Building, St John's Innovation Park, Cambridge CB4 0DS, United Kingdom
Registered in England & Wales, no. 5513718


_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um
maintainer
#3 lore raw
Quoted 10 lines · Frédéric Danis

Hi Anton,

Quoted 7 lines · Anton Ivanov
Quoted 3 lines · Frédéric Danis

Call to fallocate with FALLOC_FL_PUNCH_HOLE on a device backed by a sparse file can end up by missing data, zeroes data range, if the underlying file is used with a tool like bmaptool which will referenced only used spaces.

Can you elaborate on when do you miss data.

Quoted 7 lines
Quoted 3 lines

A file with a hole created using FALLOC_FL_PUNCH_HOLE when reading from the hole should return zeroes.

If it returns anything else or if the zeroed range exceeds the one specified in falloc - that is an underlying fs bug.

I use bmaptool to create a block map of the used part of a raw system image file, then compress the file and at the end use again bmaptool to flash the file to a SDCard.

iiuc fallocate() call with FALLOC_FL_PUNCH_HOLE doesn't "register" the space, as it is zeroed, so when bmaptool creates the block map it doesn't reference the "used holes", and when I flash the SDCard those parts are missing, so the filesystem on the SDCard is corrupted.

I can see what you are doing and I will OK the patch.

However, I still do not understand why it does not work.

bmaptool creates a map of the used blocks.

When copying, it should create the "missing" as sparse, right?

A sparse "hole" when read should be always read as zero. If it is not being read as zero when compressing - bug. Ditto at the next stage when flashing.

IMHO there is an underlying bug outside UML here somewhere.

A

Quoted 69 lines
Quoted 68 lines
Quoted 66 lines

---
  arch/um/drivers/ubd_kern.c  | 8 +++++++-
  arch/um/include/shared/os.h | 1 +
  arch/um/os-Linux/file.c     | 9 +++++++++

  3 files changed, 17 insertions(+), 1 deletion(-)
arch/um/drivers/ubd_kern.c +0 -0
index 69d2d0049a61..b03269faef71 100644
@@ -1526,13 +1526,19 @@ static void do_io(struct io_thread_req *req, struct io_desc *desc)
               }
               break;

          case REQ_OP_DISCARD:
-        case REQ_OP_WRITE_ZEROES:
              n = os_falloc_punch(req->fds[bit], off, len); if (n) {
                  req->error = map_error(-n); return;
              }
              break;
+        case REQ_OP_WRITE_ZEROES:
+            n = os_falloc_zeroes(req->fds[bit], off, len); +            if (n) {
+                req->error = map_error(-n);
+                return;
+            }
+            break;
          default:
              WARN_ON_ONCE(1);
              req->error = BLK_STS_NOTSUPP;

arch/um/include/shared/os.h +1 -0
index 00214059d9ec..fafde1d5416e 100644
@@ -168,6 +168,7 @@ extern unsigned os_major(unsigned long long dev);
   extern unsigned os_minor(unsigned long long dev);
   extern unsigned long long os_makedev(unsigned major, unsigned minor);
   extern int os_falloc_punch(int fd, unsigned long long offset, int count);
+extern int os_falloc_zeroes(int fd, unsigned long long offset, int count);
   extern int os_eventfd(unsigned int initval, int flags);
   extern int os_sendmsg_fds(int fd, const void *buf, unsigned int len,
                 const int *fds, unsigned int fds_num);
arch/um/os-Linux/file.c +8 -0
index e4421dbc4c36..fc4450db59bd 100644
@@ -625,6 +625,15 @@ int os_falloc_punch(int fd, unsigned long long offset, int len)
       return n;
   }
   +int os_falloc_zeroes(int fd, unsigned long long offset, int len)
+{
+    int n = fallocate(fd, FALLOC_FL_ZERO_RANGE|FALLOC_FL_KEEP_SIZE, offset, len);
+
+    if (n < 0)
+        return -errno;
+    return n;
+}
+
   int os_eventfd(unsigned int initval, int flags)
   {
       int fd = eventfd(initval, flags);
Signature · 8 lines
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/

_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um
#4 lore raw

Hi Anton,

Quoted 21 lines · Anton Ivanov
Quoted 10 lines · Frédéric Danis

Hi Anton,

Quoted 7 lines · Anton Ivanov
Quoted 3 lines · Frédéric Danis

Call to fallocate with FALLOC_FL_PUNCH_HOLE on a device backed by a sparse file can end up by missing data, zeroes data range, if the underlying file is used with a tool like bmaptool which will referenced only used spaces.

Can you elaborate on when do you miss data.

Quoted 7 lines
Quoted 3 lines

A file with a hole created using FALLOC_FL_PUNCH_HOLE when reading from the hole should return zeroes.

If it returns anything else or if the zeroed range exceeds the one specified in falloc - that is an underlying fs bug.

I use bmaptool to create a block map of the used part of a raw system image file, then compress the file and at the end use again bmaptool to flash the file to a SDCard.

iiuc fallocate() call with FALLOC_FL_PUNCH_HOLE doesn't "register" the space, as it is zeroed, so when bmaptool creates the block map it doesn't reference the "used holes", and when I flash the SDCard those parts are missing, so the filesystem on the SDCard is corrupted.

Quoted 7 lines

I can see what you are doing and I will OK the patch.

However, I still do not understand why it does not work.

bmaptool creates a map of the used blocks.

When copying, it should create the "missing" as sparse, right?

It depends:
- if you're creating a file, in this case there's no problem as bmaptool will create holes which will be seen as zeroed spaces,
- or if you're flashing a device like a SDCard, in this case it only writes the referenced blocks and let untouched the other parts which are certainly not full of zeroes, this is why it needs that intentional zeroed parts are not just holes.

I'm not sure to be clear :(, bmaptool is better explained at https://github.com/intel/bmap-tools/

Quoted 2 lines

A sparse "hole" when read should be always read as zero. If it is not being read as zero when compressing - bug. Ditto at the next stage when flashing.

Compression is optional, I have the same problem if I use bmaptool to flash the uncompressed file as it will copy to device only the referenced blocks in the block map.

Quoted 4 lines

IMHO there is an underlying bug outside UML here somewhere.

A

Regards,

Fred

Quoted 75 lines
Quoted 74 lines
Quoted 73 lines
Quoted 71 lines

---
  arch/um/drivers/ubd_kern.c  | 8 +++++++-
  arch/um/include/shared/os.h | 1 +
  arch/um/os-Linux/file.c     | 9 +++++++++

  3 files changed, 17 insertions(+), 1 deletion(-)
arch/um/drivers/ubd_kern.c +0 -0
index 69d2d0049a61..b03269faef71 100644
@@ -1526,13 +1526,19 @@ static void do_io(struct io_thread_req 
 *req, struct io_desc *desc)
               }
               break;

          case REQ_OP_DISCARD:
-        case REQ_OP_WRITE_ZEROES:
              n = os_falloc_punch(req->fds[bit], off, len); if (n) {
                  req->error = map_error(-n); return;
              }
              break;
+        case REQ_OP_WRITE_ZEROES:
+            n = os_falloc_zeroes(req->fds[bit], off, len); +            if (n) {
+                req->error = map_error(-n);
+                return;
+            }
+            break;
          default:
              WARN_ON_ONCE(1);
              req->error = BLK_STS_NOTSUPP;

arch/um/include/shared/os.h +1 -0
index 00214059d9ec..fafde1d5416e 100644
@@ -168,6 +168,7 @@ extern unsigned os_major(unsigned long long dev);
   extern unsigned os_minor(unsigned long long dev);
   extern unsigned long long os_makedev(unsigned major, unsigned minor);
   extern int os_falloc_punch(int fd, unsigned long long offset, int count);
+extern int os_falloc_zeroes(int fd, unsigned long long offset, int 
 count);
   extern int os_eventfd(unsigned int initval, int flags);
   extern int os_sendmsg_fds(int fd, const void *buf, unsigned int len,

                const int *fds, unsigned int fds_num);

arch/um/os-Linux/file.c +8 -0
index e4421dbc4c36..fc4450db59bd 100644
@@ -625,6 +625,15 @@ int os_falloc_punch(int fd, unsigned long long 
 offset, int len)
       return n;
   }
   +int os_falloc_zeroes(int fd, unsigned long long offset, int len)
+{
+    int n = fallocate(fd, 
 FALLOC_FL_ZERO_RANGE|FALLOC_FL_KEEP_SIZE, offset, len);
+
+    if (n < 0)
+        return -errno;
+    return n;
+}
+
   int os_eventfd(unsigned int initval, int flags)

  {
      int fd = eventfd(initval, flags);

Signature · 12 lines
Frédéric Danis
Senior Software Engineer

Collabora Ltd.
Platinum Building, St John's Innovation Park, Cambridge CB4 0DS, United Kingdom
Registered in England & Wales, no. 5513718


_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um
maintainer
#5 lore raw
Quoted 28 lines · Frédéric Danis

Hi Anton,

Quoted 25 lines · Anton Ivanov
Quoted 10 lines · Frédéric Danis

Hi Anton,

Quoted 7 lines · Anton Ivanov
Quoted 3 lines · Frédéric Danis

Call to fallocate with FALLOC_FL_PUNCH_HOLE on a device backed by a sparse file can end up by missing data, zeroes data range, if the underlying file is used with a tool like bmaptool which will referenced only used spaces.

Can you elaborate on when do you miss data.

Quoted 7 lines
Quoted 3 lines

A file with a hole created using FALLOC_FL_PUNCH_HOLE when reading from the hole should return zeroes.

If it returns anything else or if the zeroed range exceeds the one specified in falloc - that is an underlying fs bug.

I use bmaptool to create a block map of the used part of a raw system image file, then compress the file and at the end use again bmaptool to flash the file to a SDCard.

iiuc fallocate() call with FALLOC_FL_PUNCH_HOLE doesn't "register" the space, as it is zeroed, so when bmaptool creates the block map it doesn't reference the "used holes", and when I flash the SDCard those parts are missing, so the filesystem on the SDCard is corrupted.

I can see what you are doing and I will OK the patch.

However, I still do not understand why it does not work.

Quoted 7 lines
Quoted 3 lines

bmaptool creates a map of the used blocks.

When copying, it should create the "missing" as sparse, right?

It depends:
- if you're creating a file, in this case there's no problem as bmaptool will create holes which will be seen as zeroed spaces,
- or if you're flashing a device like a SDCard, in this case it only writes the referenced blocks and let untouched the other parts which are certainly not full of zeroes, this is why it needs that intentional zeroed parts are not just holes.

OK, I understood it now.

Quoted 7 lines

I'm not sure to be clear :(, bmaptool is better explained at https://github.com/intel/bmap-tools/

Quoted 2 lines

A sparse "hole" when read should be always read as zero. If it is not being read as zero when compressing - bug. Ditto at the next stage when flashing.

Compression is optional, I have the same problem if I use bmaptool to flash the uncompressed file as it will copy to device only the referenced blocks in the block map.

OK, got it.

Quoted 81 lines
Quoted 5 lines

IMHO there is an underlying bug outside UML here somewhere.

A

Regards,

Fred

Quoted 71 lines
Quoted 70 lines
Quoted 69 lines
Quoted 67 lines

---
  arch/um/drivers/ubd_kern.c  | 8 +++++++-
  arch/um/include/shared/os.h | 1 +
  arch/um/os-Linux/file.c     | 9 +++++++++

  3 files changed, 17 insertions(+), 1 deletion(-)
arch/um/drivers/ubd_kern.c +0 -0
index 69d2d0049a61..b03269faef71 100644
@@ -1526,13 +1526,19 @@ static void do_io(struct io_thread_req *req, struct io_desc *desc)
               }
               break;

          case REQ_OP_DISCARD:
-        case REQ_OP_WRITE_ZEROES:
              n = os_falloc_punch(req->fds[bit], off, len); if (n) {
                  req->error = map_error(-n); return;
              }
              break;
+        case REQ_OP_WRITE_ZEROES:
+            n = os_falloc_zeroes(req->fds[bit], off, len); +            if (n) {
+                req->error = map_error(-n);
+                return;
+            }
+            break;
          default:
              WARN_ON_ONCE(1);
              req->error = BLK_STS_NOTSUPP;

arch/um/include/shared/os.h +1 -0
index 00214059d9ec..fafde1d5416e 100644
@@ -168,6 +168,7 @@ extern unsigned os_major(unsigned long long dev);
   extern unsigned os_minor(unsigned long long dev);
   extern unsigned long long os_makedev(unsigned major, unsigned minor);
   extern int os_falloc_punch(int fd, unsigned long long offset, int count);
+extern int os_falloc_zeroes(int fd, unsigned long long offset, int count);
   extern int os_eventfd(unsigned int initval, int flags);
   extern int os_sendmsg_fds(int fd, const void *buf, unsigned int len,
                 const int *fds, unsigned int fds_num);
arch/um/os-Linux/file.c +8 -0
index e4421dbc4c36..fc4450db59bd 100644
@@ -625,6 +625,15 @@ int os_falloc_punch(int fd, unsigned long long offset, int len)
       return n;
   }
   +int os_falloc_zeroes(int fd, unsigned long long offset, int len)
+{
+    int n = fallocate(fd, FALLOC_FL_ZERO_RANGE|FALLOC_FL_KEEP_SIZE, offset, len);
+
+    if (n < 0)
+        return -errno;
+    return n;
+}
+
   int os_eventfd(unsigned int initval, int flags)
   {
       int fd = eventfd(initval, flags);
Signature · 9 lines
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/


_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um
maintainer Acked-by on v1 1/1
#6 lore raw
In reply to Frédéric Danis
Quoted 59 lines · Frédéric Danis

Call to fallocate with FALLOC_FL_PUNCH_HOLE on a device backed by a sparse file can end up by missing data, zeroes data range, if the underlying file is used with a tool like bmaptool which will referenced only used spaces.

---
  arch/um/drivers/ubd_kern.c  | 8 +++++++-
  arch/um/include/shared/os.h | 1 +
  arch/um/os-Linux/file.c     | 9 +++++++++
  3 files changed, 17 insertions(+), 1 deletion(-)
arch/um/drivers/ubd_kern.c +7 -1
index 69d2d0049a61..b03269faef71 100644
@@ -1526,13 +1526,19 @@ static void do_io(struct io_thread_req *req, struct io_desc *desc)
  			}
  			break;
  		case REQ_OP_DISCARD:
-		case REQ_OP_WRITE_ZEROES:
  			n = os_falloc_punch(req->fds[bit], off, len);
  			if (n) {
  				req->error = map_error(-n);
  				return;
  			}
  			break;
+		case REQ_OP_WRITE_ZEROES:
+			n = os_falloc_zeroes(req->fds[bit], off, len);
+			if (n) {
+				req->error = map_error(-n);
+				return;
+			}
+			break;
  		default:
  			WARN_ON_ONCE(1);
  			req->error = BLK_STS_NOTSUPP;
arch/um/include/shared/os.h +1 -0
index 00214059d9ec..fafde1d5416e 100644
@@ -168,6 +168,7 @@ extern unsigned os_major(unsigned long long dev);
  extern unsigned os_minor(unsigned long long dev);
  extern unsigned long long os_makedev(unsigned major, unsigned minor);
  extern int os_falloc_punch(int fd, unsigned long long offset, int count);
+extern int os_falloc_zeroes(int fd, unsigned long long offset, int count);
  extern int os_eventfd(unsigned int initval, int flags);
  extern int os_sendmsg_fds(int fd, const void *buf, unsigned int len,
  			  const int *fds, unsigned int fds_num);
arch/um/os-Linux/file.c +2 -0
index e4421dbc4c36..fc4450db59bd 100644
@@ -625,6 +625,15 @@ int os_falloc_punch(int fd, unsigned long long offset, int len)
  	return n;
  }
 
+int os_falloc_zeroes(int fd, unsigned long long offset, int len)
+{

arch/um/os-Linux/file.c int os_falloc_punch(int fd, unsigned long long offset, int len) · line 639

Quoted 10 lines
arch/um/os-Linux/file.c +7 -0
+	int n = fallocate(fd, FALLOC_FL_ZERO_RANGE|FALLOC_FL_KEEP_SIZE, offset, len);
+
+	if (n < 0)
+		return -errno;
+	return n;
+}
+
  int os_eventfd(unsigned int initval, int flags)
  {
  	int fd = eventfd(initval, flags);
Signature · 8 lines
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/

_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um
#7 lore raw

Hi,

Quoted 69 lines · Anton Ivanov
Quoted 66 lines · Frédéric Danis

Call to fallocate with FALLOC_FL_PUNCH_HOLE on a device backed by a sparse file can end up by missing data, zeroes data range, if the underlying file is used with a tool like bmaptool which will referenced only used spaces.

---
  arch/um/drivers/ubd_kern.c  | 8 +++++++-
  arch/um/include/shared/os.h | 1 +
  arch/um/os-Linux/file.c     | 9 +++++++++

  3 files changed, 17 insertions(+), 1 deletion(-)
arch/um/drivers/ubd_kern.c +0 -0
index 69d2d0049a61..b03269faef71 100644
@@ -1526,13 +1526,19 @@ static void do_io(struct io_thread_req *req, 
 struct io_desc *desc)
               }
               break;

          case REQ_OP_DISCARD:
-        case REQ_OP_WRITE_ZEROES:
              n = os_falloc_punch(req->fds[bit], off, len); if (n) {
                  req->error = map_error(-n); return;
              }
              break;
+        case REQ_OP_WRITE_ZEROES:
+            n = os_falloc_zeroes(req->fds[bit], off, len); +            if (n) {
+                req->error = map_error(-n);
+                return;
+            }
+            break;
          default:
              WARN_ON_ONCE(1);
              req->error = BLK_STS_NOTSUPP;

arch/um/include/shared/os.h +1 -0
index 00214059d9ec..fafde1d5416e 100644
@@ -168,6 +168,7 @@ extern unsigned os_major(unsigned long long dev);
   extern unsigned os_minor(unsigned long long dev);
   extern unsigned long long os_makedev(unsigned major, unsigned minor);
   extern int os_falloc_punch(int fd, unsigned long long offset, int count);
+extern int os_falloc_zeroes(int fd, unsigned long long offset, int 
 count);
   extern int os_eventfd(unsigned int initval, int flags);
   extern int os_sendmsg_fds(int fd, const void *buf, unsigned int len,

                const int *fds, unsigned int fds_num);

arch/um/os-Linux/file.c +6 -0
index e4421dbc4c36..fc4450db59bd 100644
@@ -625,6 +625,15 @@ int os_falloc_punch(int fd, unsigned long long 
 offset, int len)
       return n;
   }
   +int os_falloc_zeroes(int fd, unsigned long long offset, int len)
+{
+    int n = fallocate(fd, FALLOC_FL_ZERO_RANGE|FALLOC_FL_KEEP_SIZE, 
 offset, len);
+
+    if (n < 0)
+        return -errno;
+    return n;
Quoted 9 lines
Quoted 6 lines
arch/um/os-Linux/file.c +2 -0
+}
+
   int os_eventfd(unsigned int initval, int flags)

  {
      int fd = eventfd(initval, flags);

What should be next step to get this merged?

Regards,

Fred

Signature · 12 lines
Frédéric Danis
Senior Software Engineer

Collabora Ltd.
Platinum Building, St John's Innovation Park, Cambridge CB4 0DS, United Kingdom
Registered in England & Wales, no. 5513718


_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um
maintainer
#8 lore raw
Quoted 81 lines · Frédéric Danis

Hi,

Quoted 69 lines · Anton Ivanov
Quoted 66 lines · Frédéric Danis

Call to fallocate with FALLOC_FL_PUNCH_HOLE on a device backed by a sparse file can end up by missing data, zeroes data range, if the underlying file is used with a tool like bmaptool which will referenced only used spaces.

---
  arch/um/drivers/ubd_kern.c  | 8 +++++++-
  arch/um/include/shared/os.h | 1 +
  arch/um/os-Linux/file.c     | 9 +++++++++

  3 files changed, 17 insertions(+), 1 deletion(-)
arch/um/drivers/ubd_kern.c +0 -0
index 69d2d0049a61..b03269faef71 100644
@@ -1526,13 +1526,19 @@ static void do_io(struct io_thread_req *req, 
 struct io_desc *desc)
               }
               break;

          case REQ_OP_DISCARD:
-        case REQ_OP_WRITE_ZEROES:
              n = os_falloc_punch(req->fds[bit], off, len); if (n) {
                  req->error = map_error(-n); return;
              }
              break;
+        case REQ_OP_WRITE_ZEROES:
+            n = os_falloc_zeroes(req->fds[bit], off, len); +            if (n) {
+                req->error = map_error(-n);
+                return;
+            }
+            break;
          default:
              WARN_ON_ONCE(1);
              req->error = BLK_STS_NOTSUPP;

arch/um/include/shared/os.h +1 -0
index 00214059d9ec..fafde1d5416e 100644
@@ -168,6 +168,7 @@ extern unsigned os_major(unsigned long long dev);
   extern unsigned os_minor(unsigned long long dev);
   extern unsigned long long os_makedev(unsigned major, unsigned minor);
   extern int os_falloc_punch(int fd, unsigned long long offset, int count);
+extern int os_falloc_zeroes(int fd, unsigned long long offset, int 
 count);
   extern int os_eventfd(unsigned int initval, int flags);
   extern int os_sendmsg_fds(int fd, const void *buf, unsigned int len,

                const int *fds, unsigned int fds_num);

arch/um/os-Linux/file.c +6 -0
index e4421dbc4c36..fc4450db59bd 100644
@@ -625,6 +625,15 @@ int os_falloc_punch(int fd, unsigned long long 
 offset, int len)
       return n;
   }
   +int os_falloc_zeroes(int fd, unsigned long long offset, int len)
+{
+    int n = fallocate(fd, FALLOC_FL_ZERO_RANGE|FALLOC_FL_KEEP_SIZE, 
 offset, len);
+
+    if (n < 0)
+        return -errno;
+    return n;
Quoted 9 lines
Quoted 6 lines
arch/um/os-Linux/file.c +2 -0
+}
+
   int os_eventfd(unsigned int initval, int flags)

  {
      int fd = eventfd(initval, flags);

Quoted 1 line

What should be next step to get this merged?

I reviewed it and acked it.

Richard decides which on the actual merge.

A.

Quoted 5 lines

Regards,

Fred

Signature · 8 lines
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/

_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um