Skip to contentSkip to conversation

add TCP and HTTP for downloading images

PATCH Superseded Ying-Chun Liu (PaulLiu) 4 participants 8 messages Jul 8, 2022 – Oct 20, 2022

Subsystems (2)

v17 of 20 Superseded Latest is v20 →

lore
Share & tools
lore
#0 lore raw

v17 cover add TCP and HTTP for downloading images

From: "Ying-Chun Liu (PaulLiu)" <[email protected]>

This patch is a refresh from previous patches made by Duncan Hare <DuncanCHare at yahoo.com>. I've contacted him and continue to work on this patch.

This patch introduce a TCP stack with SACK. And a simple wget command to download images from http server.

v1-v12: Made by Duncan, didn't tracked.
v13: Fix some issues which is reviewed by Christian
v14: Add options to enable/disable SACK.
v15: Fix various syntax errors reviewed by Michal.
Remove magic numbers. Use kernel-doc format.
v16: Add more kernel-doc. Fix more double spaces.
v17: Fix wget with address timeout issue reported by Ramon.

Ying-Chun Liu (PaulLiu) (2):
net: Add TCP protocol
net: Add wget application

 cmd/Kconfig        |   7 +
 cmd/net.c          |  13 +
 include/net.h      |  38 ++-
 include/net/tcp.h  | 312 ++++++++++++++++++++
 include/net/wget.h |  22 ++
 net/Kconfig        |  16 +
 net/Makefile       |   2 +
 net/net.c          |  36 +++
 net/tcp.c          | 720 +++++++++++++++++++++++++++++++++++++++++++++
 net/wget.c         | 428 +++++++++++++++++++++++++++
 10 files changed, 1584 insertions(+), 10 deletions(-)
 create mode 100644 include/net/tcp.h
 create mode 100644 include/net/wget.h
 create mode 100644 net/tcp.c
 create mode 100644 net/wget.c
Signature · 1 line
2.35.1
#1 lore raw

PATCH v17 1/2 net: Add TCP protocol

From: "Ying-Chun Liu (PaulLiu)" <[email protected]>

Currently file transfers are done using tftp or NFS both over udp. This requires a request to be sent from client (u-boot) to the boot server.

The current standard is TCP with selective acknowledgment.

Cc Christian Gmeiner <[email protected]>
Cc Joe Hershberger <[email protected]>
Cc Michal Simek <[email protected]>
Cc Ramon Fried <[email protected]>

---
v1-v12: Made by Duncan, didn't tracked.
v13: Fix some issues which is reviewed by Christian
v14: Add options to enable/disable SACK.
v15: Fix various syntax errors reviewed by Michal.
Remove magic numbers. Use kernel-doc format.
v16: Add more kernel-doc. Fix more double spaces.

---
 include/net.h     |  36 ++-
 include/net/tcp.h | 312 ++++++++++++++++++++
 net/Kconfig       |  16 ++
 net/Makefile      |   1 +
 net/net.c         |  30 ++
 net/tcp.c         | 720 ++++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 1106 insertions(+), 9 deletions(-)
 create mode 100644 include/net/tcp.h
 create mode 100644 net/tcp.c
include/net.h +27 -9
index e3889a0bc8..b7bac86cf1 100644
@@ -365,6 +365,7 @@ struct vlan_ethernet_hdr {
 #define PROT_NCSI	0x88f8		/* NC-SI control packets        */
 
 #define IPPROTO_ICMP	 1	/* Internet Control Message Protocol	*/
+#define IPPROTO_TCP	6	/* Transmission Control Protocol	*/
 #define IPPROTO_UDP	17	/* User Datagram Protocol		*/
 
 /*
@@ -688,19 +689,36 @@ static inline void net_send_packet(uchar *pkt, int len)
 	(void) eth_send(pkt, len);
 }
 
-/*
- * Transmit "net_tx_packet" as UDP packet, performing ARP request if needed
- *  (ether will be populated)
- *
- * @param ether Raw packet buffer
- * @param dest IP address to send the datagram to
- * @param dport Destination UDP port
- * @param sport Source UDP port
- * @param payload_len Length of data after the UDP header
+/**
+ * net_send_ip_packet() - Transmit "net_tx_packet" as UDP or TCP packet,
+ *                        send ARP request if needed (ether will be populated)
+ * @ether: Raw packet buffer
+ * @dest: IP address to send the datagram to
+ * @dport: Destination UDP port
+ * @sport: Source UDP port
+ * @payload_len: Length of data after the UDP header
+ * @action: TCP action to be performed
+ * @tcp_seq_num: TCP sequence number of this transmission
+ * @tcp_ack_num: TCP stream acknolegement number
+ *
+ * Return: 0 on success, other value on failure
  */
 int net_send_ip_packet(uchar *ether, struct in_addr dest, int dport, int sport,
 		       int payload_len, int proto, u8 action, u32 tcp_seq_num,
 		       u32 tcp_ack_num);
+/**
+ * net_send_tcp_packet() - Transmit TCP packet.
+ * @payload_len: length of payload
+ * @dport: Destination TCP port
+ * @sport: Source TCP port
+ * @action: TCP action to be performed
+ * @tcp_seq_num: TCP sequence number of this transmission
+ * @tcp_ack_num: TCP stream acknolegement number
+ *
+ * Return: 0 on success, other value on failure
+ */
+int net_send_tcp_packet(int payload_len, int dport, int sport, u8 action,
+			u32 tcp_seq_num, u32 tcp_ack_num);
 int net_send_udp_packet(uchar *ether, struct in_addr dest, int dport,
 			int sport, int payload_len);
 
include/net/tcp.h new file +312 -0

312 lines not on this page · raw message

net/Kconfig +16 -0
index 564ea8b2d2..414ae8b422 100644
@@ -168,6 +168,22 @@ config BOOTP_SERVERIP
 	  variable, not the BOOTP server. This affects the operation of both
 	  bootp and tftp.
 
+config PROT_TCP
+	bool "TCP stack"
+	help
+	  Enable a generic tcp framework that allows defining a custom
+	  handler for tcp protocol.
+
+config PROT_TCP_SACK
+	bool "TCP SACK support"
+	depends on PROT_TCP
+	help
+	  TCP protocol with SACK. SACK means selective acknowledgements.
+	  By turning this option on TCP will learn what segments are already
+	  received. So that it improves TCP's retransmission efficiency.
+	  This option should be turn on if you want to achieve the fastest
+	  file transfer possible.
+
 endif   # if NET
 
 config SYS_RX_ETH_BUFFER
net/Makefile +1 -0
index 6c812502d3..d131d1cb1a 100644
@@ -30,6 +30,7 @@ obj-$(CONFIG_CMD_TFTPBOOT) += tftp.o
 obj-$(CONFIG_UDP_FUNCTION_FASTBOOT)  += fastboot.o
 obj-$(CONFIG_CMD_WOL)  += wol.o
 obj-$(CONFIG_PROT_UDP) += udp.o
+obj-$(CONFIG_PROT_TCP) += tcp.o
 
 # Disable this warning as it is triggered by:
 # sprintf(buf, index ? "foo%d" : "foo", index)
net/net.c +30 -0
index 81905f6315..751d0251be 100644
@@ -116,6 +116,7 @@
 #if defined(CONFIG_CMD_WOL)
 #include "wol.h"
 #endif
+#include <net/tcp.h>
 
 /** BOOTP EXTENTIONS **/
 
@@ -386,6 +387,8 @@ int net_init(void)
 
 		/* Only need to setup buffer pointers once. */
 		first_call = 0;
+		if (IS_ENABLED(CONFIG_PROT_TCP))
+			tcp_set_tcp_state(TCP_CLOSED);
 	}
 
 	return net_init_loop();
@@ -816,6 +819,16 @@ int net_send_udp_packet(uchar *ether, struct in_addr dest, int dport, int sport,
 				  IPPROTO_UDP, 0, 0, 0);
 }
 
+#if defined(CONFIG_PROT_TCP)
+int net_send_tcp_packet(int payload_len, int dport, int sport, u8 action,
+			u32 tcp_seq_num, u32 tcp_ack_num)
+{
+	return net_send_ip_packet(net_server_ethaddr, net_server_ip, dport,
+				  sport, payload_len, IPPROTO_TCP, action,
+				  tcp_seq_num, tcp_ack_num);
+}
+#endif
+
 int net_send_ip_packet(uchar *ether, struct in_addr dest, int dport, int sport,
 		       int payload_len, int proto, u8 action, u32 tcp_seq_num,
 		       u32 tcp_ack_num)
@@ -847,6 +860,14 @@ int net_send_ip_packet(uchar *ether, struct in_addr dest, int dport, int sport,
 				   payload_len);
 		pkt_hdr_size = eth_hdr_size + IP_UDP_HDR_SIZE;
 		break;
+#if defined(CONFIG_PROT_TCP)
+	case IPPROTO_TCP:
+		pkt_hdr_size = eth_hdr_size
+			+ tcp_set_tcp_header(pkt + eth_hdr_size, dport, sport,
+					     payload_len, action, tcp_seq_num,
+					     tcp_ack_num);
+		break;
+#endif
 	default:
 		return -EINVAL;
 	}
@@ -1256,6 +1277,15 @@ void net_process_received_packet(uchar *in_packet, int len)
 		if (ip->ip_p == IPPROTO_ICMP) {
 			receive_icmp(ip, len, src_ip, et);
 			return;
+#if defined(CONFIG_PROT_TCP)
+		} else if (ip->ip_p == IPPROTO_TCP) {
+			debug_cond(DEBUG_DEV_PKT,
+				   "TCP PH (to=%pI4, from=%pI4, len=%d)\n",
+				   &dst_ip, &src_ip, len);
+
+			rxhand_tcp_f((union tcp_build_pkt *)ip, len);
+			return;
+#endif
 		} else if (ip->ip_p != IPPROTO_UDP) {	/* Only UDP packets */
 			return;
 		}
net/tcp.c new file +720 -0

720 lines not on this page · raw message

Signature · 1 line
2.35.1
Reviewed-by on v17 1/2
#2 lore raw

Re: [PATCH v17 1/2] net: Add TCP protocol

Hi Ying-Chun,

Quoted 23 lines · Ying-Chun Liu (PaulLiu)

From: "Ying-Chun Liu (PaulLiu)" <[email protected]>

Currently file transfers are done using tftp or NFS both over udp. This requires a request to be sent from client (u-boot) to the boot server.

The current standard is TCP with selective acknowledgment.

Cc Christian Gmeiner <[email protected]>
Cc Joe Hershberger <[email protected]>
Cc Michal Simek <[email protected]>
Cc Ramon Fried <[email protected]>

---
v1-v12: Made by Duncan, didn't tracked.
v13: Fix some issues which is reviewed by Christian
v14: Add options to enable/disable SACK.
v15: Fix various syntax errors reviewed by Michal.
Remove magic numbers. Use kernel-doc format.
v16: Add more kernel-doc. Fix more double spaces.

Quoted 10 lines
---
 include/net.h     |  36 ++-
 include/net/tcp.h | 312 ++++++++++++++++++++
 net/Kconfig       |  16 ++
 net/Makefile      |   1 +
 net/net.c         |  30 ++
 net/tcp.c         | 720 ++++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 1106 insertions(+), 9 deletions(-)
 create mode 100644 include/net/tcp.h
 create mode 100644 net/tcp.c

This looks good to me.

Can we get a test for this? Perhaps a fake Ethernet driver in sandbox / drivers/net?

#3 lore raw

Re: [PATCH v17 1/2] net: Add TCP protocol

Hi Simon,

I think it is a bit hard for me to test it right now. It seems that we need to setup a fake HTTP server?
It can be easily done by some python scripts but I'm not sure how to start. Is there some example on testing the network commands? Like tftp or nfs?

Yours,
Paul

Quoted 45 lines · Simon Glass

Hi Ying-Chun,

Quoted 23 lines · Ying-Chun Liu (PaulLiu)

From: "Ying-Chun Liu (PaulLiu)" <[email protected]>

Currently file transfers are done using tftp or NFS both over udp. This requires a request to be sent from client (u-boot) to the boot server.

The current standard is TCP with selective acknowledgment.

Cc Christian Gmeiner <[email protected]>
Cc Joe Hershberger <[email protected]>
Cc Michal Simek <[email protected]>
Cc Ramon Fried <[email protected]>

---
v1-v12: Made by Duncan, didn't tracked.
v13: Fix some issues which is reviewed by Christian
v14: Add options to enable/disable SACK.
v15: Fix various syntax errors reviewed by Michal.
Remove magic numbers. Use kernel-doc format.
v16: Add more kernel-doc. Fix more double spaces.

Quoted 10 lines
---
 include/net.h     |  36 ++-
 include/net/tcp.h | 312 ++++++++++++++++++++
 net/Kconfig       |  16 ++
 net/Makefile      |   1 +
 net/net.c         |  30 ++
 net/tcp.c         | 720 ++++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 1106 insertions(+), 9 deletions(-)
 create mode 100644 include/net/tcp.h
 create mode 100644 net/tcp.c

This looks good to me.

Can we get a test for this? Perhaps a fake Ethernet driver in sandbox / drivers/net?

#4 lore raw

Re: [PATCH v17 1/2] net: Add TCP protocol

Hi PaulLiu,

Quoted 6 lines · PaulLiu

Hi Simon,

I think it is a bit hard for me to test it right now. It seems that we need to setup a fake HTTP server? It can be easily done by some python scripts but I'm not sure how to start.
Is there some example on testing the network commands? Like tftp or nfs?

See test/dm/eth.c for some examples.

It is better if you can have the test contained, rather than requiring starting up a separate server. E.g. you can write a small test in C which sends an http request, handles the reply and then checks that the data is received. At present sandbox supports storing one packet, but that could be extended if not enough.

Regards,
Simon

Quoted 51 lines

Yours,
Paul

Quoted 45 lines · Simon Glass

Hi Ying-Chun,

Quoted 23 lines · Ying-Chun Liu (PaulLiu)

From: "Ying-Chun Liu (PaulLiu)" <[email protected]>

Currently file transfers are done using tftp or NFS both over udp. This requires a request to be sent from client (u-boot) to the boot server.

The current standard is TCP with selective acknowledgment.

Cc Christian Gmeiner <[email protected]>
Cc Joe Hershberger <[email protected]>
Cc Michal Simek <[email protected]>
Cc Ramon Fried <[email protected]>

---
v1-v12: Made by Duncan, didn't tracked.
v13: Fix some issues which is reviewed by Christian
v14: Add options to enable/disable SACK.
v15: Fix various syntax errors reviewed by Michal.
Remove magic numbers. Use kernel-doc format.
v16: Add more kernel-doc. Fix more double spaces.

Quoted 10 lines
---
 include/net.h     |  36 ++-
 include/net/tcp.h | 312 ++++++++++++++++++++
 net/Kconfig       |  16 ++
 net/Makefile      |   1 +
 net/net.c         |  30 ++
 net/tcp.c         | 720 ++++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 1106 insertions(+), 9 deletions(-)
 create mode 100644 include/net/tcp.h
 create mode 100644 net/tcp.c

This looks good to me.

Can we get a test for this? Perhaps a fake Ethernet driver in sandbox / drivers/net?

#5 lore raw

Re: [PATCH v17 1/2] net: Add TCP protocol

I used a server on Linux for testing.
Sent from Yahoo Mail on Android

On Tue, Oct 18, 2022 at 9:59, Simon Glass<[email protected]> wrote: Hi PaulLiu,

Quoted 6 lines · PaulLiu

Hi Simon,

I think it is a bit hard for me to test it right now. It seems that we need to setup a fake HTTP server? It can be easily done by some python scripts but I'm not sure how to start.
Is there some example on testing the network commands? Like tftp or nfs?

See test/dm/eth.c for some examples.

It is better if you can have the test contained, rather than requiring starting up a separate server. E.g. you can write a small test in C which sends an http request, handles the reply and then checks that the data is received. At present sandbox supports storing one packet, but that could be extended if not enough.

Regards,
Simon

Quoted 51 lines

Yours,
Paul

Quoted 45 lines · Simon Glass

Hi Ying-Chun,

Quoted 27 lines · Ying-Chun Liu (PaulLiu)

From: "Ying-Chun Liu (PaulLiu)" <[email protected]>

Currently file transfers are done using tftp or NFS both over udp. This requires a request to be sent from client (u-boot) to the boot server.

The current standard is TCP with selective acknowledgment.

Cc Christian Gmeiner <[email protected]>
Cc Joe Hershberger <[email protected]>
Cc Michal Simek <[email protected]>
Cc Ramon Fried <[email protected]>

---
v1-v12: Made by Duncan, didn't tracked.
v13: Fix some issues which is reviewed by Christian
v14: Add options to enable/disable SACK.
v15: Fix various syntax errors reviewed by Michal. Remove magic numbers. Use kernel-doc format.
v16: Add more kernel-doc. Fix more double spaces.
---
  include/net.h    |  36 ++- include/net/tcp.h | 312 ++++++++++++++++++++ net/Kconfig      |  16 ++

Quoted 6 lines
  net/Makefile      |  1 +
  net/net.c        |  30 ++
  net/tcp.c        | 720 ++++++++++++++++++++++++++++++++++++++++++++++
  6 files changed, 1106 insertions(+), 9 deletions(-)

  create mode 100644 include/net/tcp.h
  create mode 100644 net/tcp.c

This looks good to me.

Can we get a test for this? Perhaps a fake Ethernet driver in sandbox / drivers/net?

#6 lore raw

PATCH v17 2/2 net: Add wget application

In reply to Ying-Chun Liu (PaulLiu)

From: "Ying-Chun Liu (PaulLiu)" <[email protected]>

This commit adds a simple wget command that can download files from http server.

The command syntax is
wget ${loadaddr} <path of the file from server>

Cc Christian Gmeiner <[email protected]>
Cc Joe Hershberger <[email protected]>
Cc Michal Simek <[email protected]>
Cc Ramon Fried <[email protected]>

---
v1-v12: Made by Duncan, didn't tracked.
v13: Fix some issues which is reviewed by Christian
v14: Add options to enable/disable SACK.
v15: Fix various syntax errors reviewed by Michal.
Remove magic numbers. Use kernel-doc format.
v16: Add more kernel-doc. Fix more double spaces.
v17: Fix wget with address timeout issue reported by Ramon.

---
 cmd/Kconfig        |   7 +
 cmd/net.c          |  13 ++
 include/net.h      |   2 +-
 include/net/wget.h |  22 +++
 net/Makefile       |   1 +
 net/net.c          |   6 +
 net/wget.c         | 428 +++++++++++++++++++++++++++++++++++++++++++++
 7 files changed, 478 insertions(+), 1 deletion(-)
 create mode 100644 include/net/wget.h
 create mode 100644 net/wget.c
cmd/Kconfig +7 -0
index 09193b61b9..5a5f2f1df8 100644
@@ -1715,6 +1715,13 @@ config NFS_TIMEOUT
 	  "ERROR: Cannot umount" in nfs command, try longer timeout such as
 	  10000.
 
+config CMD_WGET
+	bool "wget"
+	select TCP
+	help
+	  wget is a simple command to download kernel, or other files,
+	  from a http server over TCP.
+
 config CMD_MII
 	bool "mii"
 	imply CMD_MDIO
cmd/net.c +13 -0
index 3619c843d8..60fd785061 100644
@@ -124,6 +124,19 @@ U_BOOT_CMD(
 );
 #endif
 
+#if defined(CONFIG_CMD_WGET)
+static int do_wget(struct cmd_tbl *cmdtp, int flag, int argc, char * const argv[])
+{
+	return netboot_common(WGET, cmdtp, argc, argv);
+}
+
+U_BOOT_CMD(
+	wget,   3,      1,      do_wget,
+	"boot image via network using HTTP protocol",
+	"[loadAddress] [[hostIPaddr:]path and image name]"
+);
+#endif
+
 static void netboot_update_env(void)
 {
 	char tmp[22];
include/net.h +1 -1
index b7bac86cf1..fb80a8981d 100644
@@ -559,7 +559,7 @@ extern int		net_restart_wrap;	/* Tried all network devices */
 
 enum proto_t {
 	BOOTP, RARP, ARP, TFTPGET, DHCP, PING, DNS, NFS, CDP, NETCONS, SNTP,
-	TFTPSRV, TFTPPUT, LINKLOCAL, FASTBOOT, WOL, UDP
+	TFTPSRV, TFTPPUT, LINKLOCAL, FASTBOOT, WOL, UDP, WGET
 };
 
 extern char	net_boot_file_name[1024];/* Boot File name */
include/net/wget.h new file +22 -0
new file mode 100644
index 0000000000..da0920de11
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Duncan Hare Copyright 2017
+ */
+
+/**
+ * wget_start() - begin wget
+ */
+void wget_start(void);
+
+enum wget_state {
+	WGET_CLOSED,
+	WGET_CONNECTING,
+	WGET_CONNECTED,
+	WGET_TRANSFERRING,
+	WGET_TRANSFERRED
+};
+
+#define DEBUG_WGET		0	/* Set to 1 for debug messages */
+#define SERVER_PORT		80
+#define WGET_RETRY_COUNT	30
+#define WGET_TIMEOUT		2000UL
net/Makefile +1 -0
index d131d1cb1a..4f757a224c 100644
@@ -31,6 +31,7 @@ obj-$(CONFIG_UDP_FUNCTION_FASTBOOT)  += fastboot.o
 obj-$(CONFIG_CMD_WOL)  += wol.o
 obj-$(CONFIG_PROT_UDP) += udp.o
 obj-$(CONFIG_PROT_TCP) += tcp.o
+obj-$(CONFIG_CMD_WGET) += wget.o
 
 # Disable this warning as it is triggered by:
 # sprintf(buf, index ? "foo%d" : "foo", index)
net/net.c +6 -0
index 751d0251be..cee2ff8100 100644
@@ -117,6 +117,7 @@
 #include "wol.h"
 #endif
 #include <net/tcp.h>
+#include <net/wget.h>
 
 /** BOOTP EXTENTIONS **/
 
@@ -505,6 +506,11 @@ restart:
 			nfs_start();
 			break;
 #endif
+#if defined(CONFIG_CMD_WGET)
+		case WGET:
+			wget_start();
+			break;
+#endif
 #if defined(CONFIG_CMD_CDP)
 		case CDP:
 			cdp_start();
net/wget.c new file +428 -0

428 lines not on this page · raw message

Signature · 1 line
2.35.1
Reviewed-by on v17 2/2
#7 lore raw

Re: [PATCH v17 2/2] net: Add wget application

Quoted 23 lines · Ying-Chun Liu (PaulLiu)

From: "Ying-Chun Liu (PaulLiu)" <[email protected]>

This commit adds a simple wget command that can download files from http server.

The command syntax is
wget ${loadaddr} <path of the file from server>

Cc Christian Gmeiner <[email protected]>
Cc Joe Hershberger <[email protected]>
Cc Michal Simek <[email protected]>
Cc Ramon Fried <[email protected]>

---
v1-v12: Made by Duncan, didn't tracked.
v13: Fix some issues which is reviewed by Christian
v14: Add options to enable/disable SACK.
v15: Fix various syntax errors reviewed by Michal.
Remove magic numbers. Use kernel-doc format.
v16: Add more kernel-doc. Fix more double spaces.
v17: Fix wget with address timeout issue reported by Ramon.

Quoted 11 lines
---
 cmd/Kconfig        |   7 +
 cmd/net.c          |  13 ++
 include/net.h      |   2 +-
 include/net/wget.h |  22 +++
 net/Makefile       |   1 +
 net/net.c          |   6 +
 net/wget.c         | 428 +++++++++++++++++++++++++++++++++++++++++++++
 7 files changed, 478 insertions(+), 1 deletion(-)
 create mode 100644 include/net/wget.h
 create mode 100644 net/wget.c

This series needs something in doc/cmd and test/

Regards,
Simon