Transferring files over tftp

Hello,

I am facing tftp issues when trying to transfer files > 2GB. This ends up in a crash:

barebox@Phytec phyCORE AM335x:/ cp /mnt/tftp/2GB_plus_1M .
eth0: 100Mbps full duplex link detected
T T T T T T T T T T T T T T read: Connection timed out unable to handle paging request at address 0x200d319a

pc : [<9fe0587a>]    lr : [<9fe058cb>]

sp : 9ffef810  ip : ffffffff  fp : 9004d4d0
r10: 00000000  r9 : 9004e410  r8 : 00000006
r7 : 00000000  r6 : 8fefeec0  r5 : a79cdbf6  r4 : 200d318e
r3 : 0000001a  r2 : 00000019  r1 : 9004f40c  r0 : 8fefeec0
Flags: nzCv  IRQs off  FIQs on  Mode SVC_32
[<9fe0587a>] (remove_free_block+0xe/0x3e) from [<9fe058cb>] (block_remove+0x21/0x24) [<9fe058cb>] (block_remove+0x21/0x24) from [<9fe058eb>] (block_merge_next+0x1d/0x34) [<9fe058eb>] (block_merge_next+0x1d/0x34) from [<9fe05b73>] (tlsf_free+0x55/0x62) [<9fe05b73>] (tlsf_free+0x55/0x62) from [<9fe408e7>] (copy_file+0x77/0x13c) [<9fe408e7>] (copy_file+0x77/0x13c) from [<9fe33c41>] (do_cp+0x101/0x124) [<9fe33c41>] (do_cp+0x101/0x124) from [<9fe0318d>] (execute_command+0x21/0x48) [<9fe0318d>] (execute_command+0x21/0x48) from [<9fe07c81>] (run_list_real+0x5a9/0x608) [<9fe07c81>] (run_list_real+0x5a9/0x608) from [<9fe075dd>] (parse_stream_outer+0x105/0x1 5c)
[<9fe075dd>] (parse_stream_outer+0x105/0x15c) from [<9fe07ea3>] (run_shell+0x33/0x60) [<9fe07ea3>] (run_shell+0x33/0x60) from [<9fe0318d>] (execute_command+0x21/0x48) [<9fe0318d>] (execute_command+0x21/0x48) from [<9fe07c81>] (run_list_real+0x5a9/0x608) [<9fe07c81>] (run_list_real+0x5a9/0x608) from [<9fe0796d>] (run_list_real+0x295/0x608)

[<9fe5603d>] (unwind_backtrace+0x1/0x58) from [<9fe00cf1>] (panic+0x1d/0x34) [<9fe00cf1>] (panic+0x1d/0x34) from [<9fe543ad>] (do_exception+0xd/0x10) [<9fe543ad>] (do_exception+0xd/0x10) from [<9fe5440d>] (do_data_abort+0x21/0x2c) [<9fe5440d>] (do_data_abort+0x21/0x2c) from [<9fe54134>] (do_abort_6+0x48/0x54)

Copying a file <2GB works even without timeouts.

Causing this seems to be commit:

commit e3e55fa3aa698bb92fa4362a36e102749c13c124
Author: Sascha Hauer <[email protected]>
Date:   Thu Aug 30 12:27:00 2018 +0200

    fs: tftp: improve file size handling

    Previously we used FILE_SIZE_STREAM unconditionally. Instead, fill the inode size with a valid filesize if we have one and only if not fall back to FILE_SIZE_STREAM.

Reverting the commit reveals a second issue. After 2GB the transfer breaks up silently. Maybe some overflow? Any idea where to look at this further?

Hi Teresa,

Quoted 8 lines · Teresa Remmet

Hello,

I am facing tftp issues when trying to transfer files > 2GB. This ends up in a crash:

Quoted 15 lines

Causing this seems to be commit:

commit e3e55fa3aa698bb92fa4362a36e102749c13c124
Author: Sascha Hauer <[email protected]>
Date:   Thu Aug 30 12:27:00 2018 +0200

    fs: tftp: improve file size handling

    Previously we used FILE_SIZE_STREAM unconditionally. Instead, fill the
    inode size with a valid filesize if we have one and only if not fall
    back to FILE_SIZE_STREAM.

Quoted 3 lines

Reverting the commit reveals a second issue. After 2GB the transfer breaks up silently.
Maybe some overflow? Any idea where to look at this further?

As a first step try the attached patch which makes the filesize in tftp
a 64bit type so that we at least can handle the filesize numbers.

I tried transferring a file > 2GiB here and it seemed to work, although
I don't have a board with that amount of memory, so I had to write to /dev/null. I don't know if that makes a difference.

Also you probably begin to hit the limits of the tftp servers. I realized that our tftp server (atftpd) only reports the filesize for files smaller than 2GiB, with bigger files it only reports '0'.

I think other tftp servers begin to make problems when the 16bit block
counter overflows, so with working 2GiB files you are lucky already ;)

Sascha

--------------------------------------8<-----------------------------
-----

From bf3de3391c958c9c44fcfa54b2c1a6a29e68a5a4 Mon Sep 17 00:00:00 2001
From: Sascha Hauer <[email protected]>
Date: Fri, 16 Nov 2018 08:24:06 +0100
Subject: [PATCH] fs: tftp: Make filesize a 64bit type

Signed-off-by Sascha Hauer <[email protected]>

---
 fs/tftp.c | 8 ++++----

 1 file changed, 4 insertions(+), 4 deletions(-)

fs/tftp.c +1 -1

index 0d9ee6effd..8131ab27b3 100644
@@ -80,7 +80,7 @@ struct file_priv {
  	int state;
  	int err;
  	char *filename;
-	int filesize;
+	loff_t filesize;
  	uint64_t resend_timeout;
  	uint64_t progress_timeout;
  	struct kfifo *fifo;

@@ -136,7 +136,7 @@ static int tftp_send(struct file_priv *priv)

  "timeout%c"
  "%d%c"
  "tsize%c"

fs/tftp.c +1 -1

-				"%d%c"
+				"%lld%c"

  "blksize%c" "1432", priv->filename + 1, 0,

fs/tftp.c +1 -1

@@ -235,7 +235,7 @@ static void tftp_parse_oack(struct file_priv
 *priv, unsigned char *pkt, int len)
  		if (val > s + len)
  			return;
  		if (!strcmp(opt, "tsize"))
-			priv->filesize = simple_strtoul(val, NULL,
 10);
+			priv->filesize = simple_strtoull(val, NULL,
 10);

  if (!strcmp(opt, "blksize")) priv->blocksize = simple_strtoul(val, NULL, 10);
  pr_debug("OACK opt: %s val: %s\n", opt, val);

fs/tftp.c +0 -0

@@ -658,7 +658,7 @@ static struct dentry *tftp_lookup(struct inode
 *dir, struct dentry *dentry,

  struct fs_device_d *fsdev = container_of(sb, struct fs_device_d, sb);
  struct inode *inode;
  struct file_priv *priv;

fs/tftp.c +1 -1

-	int filesize;
+	loff_t filesize;

  priv = tftp_do_open(&fsdev->dev, O_RDONLY, dentry); if (IS_ERR(priv))
--
2.19.1

--------------------------------------8<-----------------------------
-----

From bf3de3391c958c9c44fcfa54b2c1a6a29e68a5a4 Mon Sep 17 00:00:00 2001
From: Sascha Hauer <[email protected]>
Date: Fri, 16 Nov 2018 08:24:06 +0100
Subject: [PATCH] fs: tftp: Make filesize a 64bit type

Signed-off-by Sascha Hauer <[email protected]>

---
 fs/tftp.c | 8 ++++----

 1 file changed, 4 insertions(+), 4 deletions(-)

fs/tftp.c +1 -1

index 0d9ee6effd..8131ab27b3 100644
@@ -80,7 +80,7 @@ struct file_priv {
  	int state;
  	int err;
  	char *filename;
-	int filesize;
+	loff_t filesize;
  	uint64_t resend_timeout;
  	uint64_t progress_timeout;
  	struct kfifo *fifo;

@@ -136,7 +136,7 @@ static int tftp_send(struct file_priv *priv)

  "timeout%c"
  "%d%c"
  "tsize%c"

fs/tftp.c +1 -1

-				"%d%c"
+				"%lld%c"

  "blksize%c" "1432", priv->filename + 1, 0,

fs/tftp.c +1 -1

@@ -235,7 +235,7 @@ static void tftp_parse_oack(struct file_priv
 *priv, unsigned char *pkt, int len)
  		if (val > s + len)
  			return;
  		if (!strcmp(opt, "tsize"))
-			priv->filesize = simple_strtoul(val, NULL,
 10);
+			priv->filesize = simple_strtoull(val, NULL,
 10);

  if (!strcmp(opt, "blksize")) priv->blocksize = simple_strtoul(val, NULL, 10);
  pr_debug("OACK opt: %s val: %s\n", opt, val);

fs/tftp.c +0 -0

@@ -658,7 +658,7 @@ static struct dentry *tftp_lookup(struct inode
 *dir, struct dentry *dentry,

  struct fs_device_d *fsdev = container_of(sb, struct fs_device_d, sb);
  struct inode *inode;
  struct file_priv *priv;

fs/tftp.c +1 -1

-	int filesize;
+	loff_t filesize;

  priv = tftp_do_open(&fsdev->dev, O_RDONLY, dentry); if (IS_ERR(priv))
--
2.19.1

Hello Sascha,

Am Freitag, den 16.11.2018, 08:32 +0100 schrieb Sascha Hauer:

...

Quoted 10 lines

--------------------------------------8<-----------------------------
-----

From bf3de3391c958c9c44fcfa54b2c1a6a29e68a5a4 Mon Sep 17 00:00:00 2001
From: Sascha Hauer <[email protected]>
Date: Fri, 16 Nov 2018 08:24:06 +0100
Subject: [PATCH] fs: tftp: Make filesize a 64bit type

Quoted 1 line

Signed-off-by Sascha Hauer <[email protected]>

I miss this patch on next. As it fixes my problem. Is there any reason why it is not applied?