Skip to contentSkip to conversation

squashfs-tools: Add -offset option to skip n bytes at the start of the file

PATCH Ritesh Raj Sarraf 2 participants 3 messages Jan 29, 2017 – Jan 31, 2017 2 messages not in the archive VFS

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): probonopd/AppImageKit/issues/[email protected]

A message that is not in the archive (sent off-list or not yet received): probonopd/AppImageKit/issues/339/[email protected]

#2 lore raw
Quoted 4 lines · probonopd

Yes, I am not happy about that.
You could either push your change to the squashfs folks
Sent a PR half a year ago but no reaction: plougher/squashfs-tools#13 - is something wrong with it?

That repo seems inactive. So does the git repo at kernel.org. but I would rely on that one.

Let's try fresh now. Given there's been not much development in the upstream repo, I've just taken your patch, as is, for submission.

Quoted 3 lines

Or you could just carry the delta and ask the user to fetch the squashfs-tools source themselves
Do you mean "carry the delta" as in "provide a diff"?

Doing now.

Dear Squashfs Team,

Can you please review/include the attached patch into squashfs-tools ? The mentioned changes are needed by AppImageKit.

What is AppImage

The AppImage format is a format for packaging applications in a way that allows them to run on a variety of different target systems (base operating systems, distributions) without further modification.

https://en.wikipedia.org/wiki/AppImage

AppImageKit is a concrete implementation of the AppImage format and provides tools such as appimagetool and appimaged for conveniently handling AppImages.

https://github.com/probonopd/AppImageKit

Signature · 3 lines
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."

From f72f32b0fe7929e71edd9e6589fa1fcec1d86adc Mon Sep 17 00:00:00 2001
From: probonopd <[email protected]>
Date: Thu, 15 Sep 2016 18:24:10 +0200
Subject: [PATCH 1/2] Add -offset option to skip n bytes at the start of the
input file

---
 squashfs-tools/unsquashfs.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
squashfs-tools/unsquashfs.c +12 -1
index 1323dd6..b6ba452 100644
@@ -42,6 +42,7 @@ struct cache *fragment_cache, *data_cache;
 struct queue *to_reader, *to_inflate, *to_writer, *from_writer;
 pthread_t *thread, *inflator_thread;
 pthread_mutex_t	fragment_mutex;
+static off_t squashfs_start_offset = 0;
 
 /* user options that control parallelisation */
 int processors = -1;
@@ -631,7 +632,7 @@ int read_fs_bytes(int fd, long long byte, int bytes, void *buff)
 	TRACE("read_bytes: reading from position 0x%llx, bytes %d\n", byte,
 		bytes);
 
-	if(lseek(fd, off, SEEK_SET) == -1) {
+	if(lseek(fd, off + squashfs_start_offset, SEEK_SET) == -1) {
 		ERROR("Lseek failed because %s\n", strerror(errno));
 		return FALSE;
 	}
@@ -2544,6 +2545,14 @@ int main(int argc, char *argv[])
 				exit(1);
 			}
 			dest = argv[i];
+        		} else if (strcmp(argv[i], "-offset") == 0 ||
+				strcmp(argv[i], "-o") == 0) {
+			if(++i == argc) {
+				fprintf(stderr, "%s: -offset missing argument\n",
+					argv[0]);
+				exit(1);
+			}
+			squashfs_start_offset = (off_t)atol(argv[i]);
 		} else if(strcmp(argv[i], "-processors") == 0 ||
 				strcmp(argv[i], "-p") == 0) {
 			if((++i == argc) || 
@@ -2636,6 +2645,8 @@ int main(int argc, char *argv[])
 				"copyright information\n");
 			ERROR("\t-d[est] <pathname>\tunsquash to <pathname>, "
 				"default \"squashfs-root\"\n");
+			ERROR("\t-o[ffset] <bytes>\tskip <bytes> at start of input file, "
+				"default \"0\"\n");
 			ERROR("\t-n[o-progress]\t\tdon't display the progress "
 				"bar\n");
 			ERROR("\t-no[-xattrs]\t\tdon't extract xattrs in file system"

From 5a498ad24dcfeac9f3d747e894f22901f3ac10ef Mon Sep 17 00:00:00 2001
From: probonopd <[email protected]>
Date: Thu, 15 Sep 2016 21:09:52 +0200
Subject: [PATCH 2/2] Add -offset function to skip n bytes at the beginning of
the squashfs file

---
 squashfs-tools/mksquashfs.c | 25 +++++++++++++++++++------
 1 file changed, 19 insertions(+), 6 deletions(-)
squashfs-tools/mksquashfs.c +19 -6
index d221c35..92b6a31 100644
@@ -98,6 +98,7 @@ int old_exclude = TRUE;
 int use_regex = FALSE;
 int nopad = FALSE;
 int exit_on_error = FALSE;
+static off_t squashfs_start_offset = 0;
 
 long long global_uid = -1, global_gid = -1;
 
@@ -516,9 +517,9 @@ int read_fs_bytes(int fd, long long byte, int bytes, void *buff)
 
 	pthread_cleanup_push((void *) pthread_mutex_unlock, &pos_mutex);
 	pthread_mutex_lock(&pos_mutex);
-	if(lseek(fd, off, SEEK_SET) == -1) {
+	if(lseek(fd, off+squashfs_start_offset, SEEK_SET) == -1) {
 		ERROR("read_fs_bytes: Lseek on destination failed because %s, "
-			"offset=0x%llx\n", strerror(errno), off);
+			"offset=0x%llx\n", strerror(errno), off+squashfs_start_offset);
 		res = 0;
 	} else if(read_bytes(fd, buff, bytes) < bytes) {
 		ERROR("Read on destination failed\n");
@@ -557,10 +558,10 @@ void write_destination(int fd, long long byte, int bytes, void *buff)
 	pthread_cleanup_push((void *) pthread_mutex_unlock, &pos_mutex);
 	pthread_mutex_lock(&pos_mutex);
 
-	if(lseek(fd, off, SEEK_SET) == -1) {
+	if(lseek(fd, off+squashfs_start_offset, SEEK_SET) == -1) {
 		ERROR("write_destination: Lseek on destination "
 			"failed because %s, offset=0x%llx\n", strerror(errno),
-			off);
+			off+squashfs_start_offset);
 		BAD_ERROR("Probably out of space on output %s\n",
 			block_device ? "block device" : "filesystem");
 	}
@@ -2315,9 +2316,9 @@ void *writer(void *arg)
 		pthread_cleanup_push((void *) pthread_mutex_unlock, &pos_mutex);
 		pthread_mutex_lock(&pos_mutex);
 
-		if(lseek(fd, off, SEEK_SET) == -1) {
+		if(lseek(fd, off+squashfs_start_offset, SEEK_SET) == -1) {
 			ERROR("writer: Lseek on destination failed because "
-				"%s, offset=0x%llx\n", strerror(errno), off);
+				"%s, offset=0x%llx\n", strerror(errno), off+squashfs_start_offset);
 			BAD_ERROR("Probably out of space on output "
 				"%s\n", block_device ? "block device" :
 				"filesystem");
@@ -5341,6 +5342,15 @@ int main(int argc, char *argv[])
 			force_progress = TRUE;
 		else if(strcmp(argv[i], "-no-exports") == 0)
 			exportable = FALSE;
+        else if(strcmp(argv[i], "-offset") == 0 ||
+				strcmp(argv[i], "-o") ==0) {
+			if(++i == argc) {
+				ERROR("%s: %s offset missing argument\n", argv[0],
+							argv[i - 1]);
+				exit(1);
+			}
+			squashfs_start_offset = (off_t)atol(argv[i]);
+        }        
 		else if(strcmp(argv[i], "-processors") == 0) {
 			if((++i == argc) || !parse_num(argv[i], &processors)) {
 				ERROR("%s: -processors missing or invalid "
@@ -5641,6 +5651,9 @@ int main(int argc, char *argv[])
 			ERROR("\nMiscellaneous options:\n");
 			ERROR("-root-owned\t\talternative name for -all-root"
 				"\n");
+			ERROR("-o <offset>\t\tSkip <offset> bytes at the "
+				"beginning of the file.\n\t\t\t"
+				"Default 0 bytes\n");            
 			ERROR("-noInodeCompression\talternative name for -noI"
 				"\n");
 			ERROR("-noDataCompression\talternative name for -noD"
#3 lore raw
Quoted 9 lines · Ritesh Raj Sarraf
Quoted 4 lines · probonopd

Yes, I am not happy about that.
You could either push your change to the squashfs folks
Sent a PR half a year ago but no reaction: plougher/squashfs-tools#13 - is something wrong with it?

That repo seems inactive. So does the git repo at kernel.org. but I would rely on that one.

The repo at github is dead, I only used it during the time kernel.org was offline due to the hack.

The repo at kernel.org is the official repo. It hasn't been updated in a while simply because the tools are considered stable, and I have not had a burning desire to add new features for a while.

Quoted 11 lines

Let's try fresh now. Given there's been not much development in the upstream repo, I've just taken your patch, as is, for submission.

Quoted 3 lines

Or you could just carry the delta and ask the user to fetch the squashfs-tools source themselves
Do you mean "carry the delta" as in "provide a diff"?

Doing now.

Dear Squashfs Team,

That's me.

Quoted 3 lines

Can you please review/include the attached patch into squashfs-tools ? The mentioned changes are needed by AppImageKit.

Well from a quick look at the attached patches, I see no reason not to pull them.

I'll have a look over at github because is the first I've seen of these patches, and I'll see if I can pull them from there.

If not, I'll ask the author to send them to me at this address.

[email protected] is dead because my ISP arbitrarily ceased email support sometime ago (and I'd had that email for 20 years). It would appear phillip@squashfs,org.uk is still redirected to that, and I'll fix that.

Phillip

Quoted 19 lines

What is AppImage

The AppImage format is a format for packaging applications in a way that allows them to run on a variety of different target systems (base operating systems, distributions) without further modification.

https://en.wikipedia.org/wiki/AppImage

AppImageKit is a concrete implementation of the AppImage format and provides tools such as appimagetool and appimaged for conveniently handling AppImages.

https://github.com/probonopd/AppImageKit

--
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."

#4 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hello Phillip,

Good to hear from you. When the emails bounced, I was wondering if you are still active or not.

Quoted 6 lines · Phillip Lougher

The repo at kernel.org is the official repo.  It hasn't been updated in a while
simply because the tools are considered stable, and I have not had a burning desire to add new features for a while.

Cool. Thanks.

Quoted 5 lines
Quoted 4 lines

Can you please review/include the attached patch into squashfs-tools ? The mentioned changes are needed by AppImageKit.

Quoted 8 lines

Well from a quick look at the attached patches, I see no reason not to pull them.

I'll have a look over at github because is the first I've seen of these patches, and I'll see if I can pull them from there.

If not, I'll ask the author to send them to me at this address.

Great. The author (probono AT puredarwin.org) is copied on this email.

Quoted 5 lines

[email protected] is dead because my ISP arbitrarily ceased email support sometime ago (and I'd had that email for 20 years). It would appear phillip@squashfs,org.uk is still redirected to that, and I'll fix that.

I used to maintain my own mail server but eventually gave up (amount of spam kept the machine alive all the while) and switched to GApps.

- --
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEQCVDstmIVAB/Yn02pjpYo/LhdWkFAliQYmYACgkQpjpYo/Lh dWlblg/8DxMhZEDN8CnL1ZslyIdlTtz9aFPQ73qjNSHvNKx+zO0KhgOC7dSbB/uU BPlNBXlGcJ9q4rIVcgxEmETIUpag4aZND8bkfnX4rVffq8/Q0rPHNjknlkT8Wvah UjsyqIBP5PgAlyJKNr2ukl5TXGk+OTpcAnWgZBEdsrJbFBF/GxRnGkHxr7DZEjkK D2gPs2GTK6SF21+Mj2nkuNWzf2ofX1amf5ZQyOoTOOO0WBRquTQIKDGk5eM13+5o +yP16UGUD60ADJfeSYXSAbLCRzIRNPc++9jmcyccpTdU5zSYGzeO5+UyH7wlTF5Q lHPA0W5ss7xq2bW2iDM/+iPB75YY9iHGXsyWt66sHZ28BFR32KdoxzmQlKnV2+cT QOBSDbGto+f64CbN+ufLROIu5g89G6LNOu1PIH6KVY4f7z2zCZK8VRvcL/XQschp 8+jc67fSv+go8yOY2C3VG1JYSds7vsb7TyJZQKxbEMQC0vkoO5R6olbCMNb+8O1A kPk0eozr/GnWm1/xqXUMT9ZEhqeicQeTa2MXnPWypgfJnTxbkA4PuDUid1YUtpJ0 xNFJOSKu6SdFOgH2GkO8nXFlB8DOfvlulATdUJJyHa3ThPzo3EGPss9vZU8Gj34g CJBovBqkNp3Azu26ksQrEMK2UP/BF9LaiKpPgTrhLkULWnH7nBQ= =RCdF
-----END PGP SIGNATURE-----