Why netboot:
Subsystems (2)
v11 cover Why netboot:
From: Duncan Hare <[email protected]>
Central management, including logs and change control, coupled with with enhanced security and unauthorized change detection and remediation by exposing a small attack surface.
Why TCP with Selective Acknowledgement:
Currently file transfer are done using tftp or NFS both over udp. This requires a request to be sent from client (u-boot) to the boot server.
For a 4 Mbyte kernel, with a 1k block size this requires 4,000 request for a block.
Using a large block size, one greater than the Ethernet maximum frame size limitation, would require fragmentation, which u-boot supports. However missing fragment recovery requires timeout detection and re-transmission requests for missing fragments.
UDP is ideally suited to fast single packet exchanges, inquiry/response, for example dns, becuse of the lack of connection overhead.
UDP as a file transport mechanism is slow, even in low latency networks, because file transfer with udp requires poll/response mechanism to provide transfer integrity.
In networks with large latency, for example: the internet, UDP is even slower. What is a 30 second transfer on a local boot server and LAN increase to over 3 minutes, because of all the requests/response traffic.
This was anticipated in the evolution of the IP protocols and TCP was developed and then enhanced for high latency high bandwidth networks.
The current standard is TCP with selective acknowledgment.
In our testing we have reduce kernel transmission time to around 0.4 seconds for a 4Mbyte kernel, with a 100 Mbps downlink.
Why http and wget:
HTTP is the most efficient file retrieval protocol in common use. The client send a single request, after TCP connection, to receive a file of any length.
WGET is the application which implements http file transfer outside browsers as a file transfer protocol. Versions of wget exists on many operating systems.
Changes in v11:
Add TCP with Selective Acknowledgement (SACK)
Adding wget
Duncan Hare (3):
Adding TCP and wget into u-boot
Adding TCP
Add wget, for fast file transfer.
cmd/Kconfig | 7 + cmd/net.c | 13 + include/net.h | 15 +- include/net/tcp.h | 227 +++++++++++++++++ include/net/wget.h | 19 ++ net/Kconfig | 6 + net/Makefile | 2 + net/net.c | 79 +++++- net/ping.c | 9 +- net/tcp.c | 700 +++++++++++++++++++++++++++++++++++++++++++++++++++++ net/wget.c | 418 ++++++++++++++++++++++++++++++++ 11 files changed, 1475 insertions(+), 20 deletions(-) create mode 100644 include/net/tcp.h create mode 100644 include/net/wget.h create mode 100644 net/tcp.c create mode 100644 net/wget.c
Signature · 1 line
2.11.0
PATCH v11 1/3 Adding TCP and wget into u-boot
From: Duncan Hare <[email protected]>
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Initial changes for adding TCP
Meet change control standards after review
<<<<
---
Consolidatind UDP header functions to make it easier to add TCP versions of the same, while reusing IP portions. This patch should not change any behaviors.
Add a protocol parameter to ip packet sending in net.c Add UDP protocol for current applications to minimize code changes to existing ping.c
All the code is new, and not copied from any source.
Changes in v11: None
include/net.h | 7 ++++++- net/net.c | 32 +++++++++++++++++++++++--------- net/ping.c | 9 ++------- 3 files changed, 31 insertions(+), 17 deletions(-)
include/net.h +6 -1
@@ -593,7 +593,8 @@ int net_set_ether(uchar *xet, const uchar *dest_ethaddr, uint prot);int net_update_ether (struct ethernet_hdr *et,uchar *addr,uint prot);/* Set IP header */ -void net_set_ip_header (uchar *pkt,struct in_addr dest,struct in_addr source); +void net_set_ip_header (uchar *pkt,struct in_addr dest,struct in_addr source, +u16 pkt_len,u8 prot);void net_set_udp_header (uchar *pkt,struct in_addr dest,int dport,int sport,int len);
@@ -667,6 +668,10 @@ static inline void net_send_packet(uchar *pkt, int len)* @param sport Source UDP port * @param payload_len Length of data after the UDP header */ +int net_send_ip_packet (uchar *ether,struct in_addr dest,int dport,int sport, +int payload_len,int proto,u8 action,u32 tcp_seq_num, +u32 tcp_ack_num); +int net_send_udp_packet (uchar *ether,struct in_addr dest,int dport,int sport,int payload_len);
net/net.c +23 -9
@@ -777,6 +777,14 @@ void net_set_timeout_handler(ulong iv, thand_f *f)int net_send_udp_packet (uchar *ether,struct in_addr dest,int dport,int sport,int payload_len) { +return net_send_ip_packet (ether, dest, dport, sport, payload_len, +IPPROTO_UDP ,0 ,0 ,0 ); +} + +int net_send_ip_packet (uchar *ether,struct in_addr dest,int dport,int sport, +int payload_len,int proto,u8 action,u32 tcp_seq_num, +u32 tcp_ack_num) +{uchar *pkt;int eth_hdr_size;int pkt_hdr_size;
@@ -797,9 +805,14 @@ int net_send_udp_packet(uchar *ether, struct in_addr dest, int dport, int sport, pkt = (uchar *)net_tx_packet; eth_hdr_size =net_set_ether (pkt, ether,PROT_IP ); - pkt += eth_hdr_size; -net_set_udp_header (pkt, dest, dport, sport, payload_len); - pkt_hdr_size = eth_hdr_size +IP_UDP_HDR_SIZE ; +switch (proto) { +case IPPROTO_UDP : +net_set_udp_header (pkt + eth_hdr_size, dest, + dport, sport, payload_len); + pkt_hdr_size = eth_hdr_size +IP_UDP_HDR_SIZE ; +break ; +default :return -EINVAL ; + }/* if MAC address was not discovered yet, do an ARP request */ if (memcmp (ether, net_null_ethaddr,6 ) ==0 ) {
@@ -1424,7 +1437,8 @@ int net_update_ether(struct ethernet_hdr *et, uchar *addr, uint prot) } } -void net_set_ip_header (uchar *pkt,struct in_addr dest,struct in_addr source) +void net_set_ip_header (uchar *pkt,struct in_addr dest,struct in_addr source, +u16 pkt_len,u8 prot) {struct ip_udp_hdr *ip = (struct ip_udp_hdr *)pkt;
@@ -1434,15 +1448,17 @@ void net_set_ip_header(uchar *pkt, struct in_addr dest, struct in_addr source)/* IP_HDR_SIZE / 4 (not including UDP) */ ip->ip_hl_v =0x45 ; ip->ip_tos =0 ; - ip->ip_len =htons (IP_HDR_SIZE ); + ip->ip_len =htons (pkt_len); ip->ip_id =htons (net_ip_id++); ip->ip_off =htons (IP_FLAGS_DFRAG );/* Don't fragment */ ip->ip_ttl =255 ; + ip->ip_p = prot; ip->ip_sum =0 ;/* already in network byte order */ net_copy_ip ((void *)&ip->ip_src, &source);/* already in network byte order */ net_copy_ip ((void *)&ip->ip_dst, &dest); + ip->ip_sum =compute_ip_checksum (ip,IP_HDR_SIZE ); } void net_set_udp_header(uchar *pkt, struct in_addr dest, int dport, int sport,
@@ -1458,10 +1474,8 @@ void net_set_udp_header(uchar *pkt, struct in_addr dest, int dport, int sport,if (len &1 ) pkt[IP_UDP_HDR_SIZE + len] =0 ; -net_set_ip_header (pkt, dest, net_ip); - ip->ip_len =htons (IP_UDP_HDR_SIZE + len); - ip->ip_p =IPPROTO_UDP ; - ip->ip_sum =compute_ip_checksum (ip,IP_HDR_SIZE ); +net_set_ip_header (pkt, dest, net_ip,IP_UDP_HDR_SIZE + len, +IPPROTO_UDP ); ip->udp_src =htons (sport); ip->udp_dst =htons (dport);
net/ping.c +2 -7
@@ -20,16 +20,11 @@ struct in_addr net_ping_ip;static void set_icmp_header (uchar *pkt,struct in_addr dest) {/* -* Construct an IP and ICMP header. +* Construct an ICMP header. */ -struct ip_hdr *ip = (struct ip_hdr *)pkt;struct icmp_hdr *icmp = (struct icmp_hdr *)(pkt +IP_HDR_SIZE ); -net_set_ip_header (pkt, dest, net_ip); - - ip->ip_len =htons (IP_ICMP_HDR_SIZE ); - ip->ip_p =IPPROTO_ICMP ; - ip->ip_sum =compute_ip_checksum (ip,IP_HDR_SIZE ); +net_set_ip_header (pkt, dest, net_ip,IP_ICMP_HDR_SIZE ,IPPROTO_ICMP ); icmp->type =ICMP_ECHO_REQUEST ; icmp->code =0 ;
Signature · 1 line
2.11.0
PATCH v11 2/3 Adding TCP
In reply to DH at synoia.comFrom: Duncan Hare <[email protected]>
All the code is new, and not copied from any source.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
For Multi-hop fast netwoks, (aka Long Fat networks) SACK provides continued stream teansfer without delaying the transfer for a misssing packet.
<<<<
---
Clean code post review.
Changes in v11:
Add TCP with Selective Acknowledgement (SACK)
include/net.h | 12 +- include/net/tcp.h | 227 ++++++++++++++++++ net/Kconfig | 6 + net/Makefile | 1 + net/tcp.c | 700 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 943 insertions(+), 3 deletions(-) create mode 100644 include/net/tcp.h create mode 100644 net/tcp.c
include/net.h +9 -3
@@ -1,4 +1,4 @@ -/* SPDX-License-Identifier: GPL-2.0 */ +// SPDX-License-Identifier: GPL-2.0 /* * LiMon Monitor (LiMon) - Network. *
@@ -25,7 +25,9 @@* alignment in memory. * */ - +#if defined(CONFIG_TCP )/* Protected becuse UDP uses less buffers */ +#define CONFIG_SYS_RX_ETH_BUFFER 12 +#endif #ifdef CONFIG_SYS_RX_ETH_BUFFER # define PKTBUFSRX CONFIG_SYS_RX_ETH_BUFFER#else
@@ -84,7 +86,7 @@ enum eth_state_t { };#ifdef CONFIG_DM_ETH -/** +/* * struct eth_pdata - Platform data for Ethernet MAC controllers * * @iobase: The base address of the hardware registers
@@ -350,6 +352,7 @@ struct vlan_ethernet_hdr {#define PROT_PPP_SES 0x8864/* PPPoE session messages */ #define IPPROTO_ICMP 1/* Internet Control Message Protocol */ +#define IPPROTO_TCP 6/* Transmission Control Protocol */ #define IPPROTO_UDP 17/* User Datagram Protocol */ /*
@@ -675,6 +678,9 @@ int net_send_ip_packet(uchar *ether, struct in_addr dest, int dport, int sport,int net_send_udp_packet (uchar *ether,struct in_addr dest,int dport,int sport,int payload_len); +int net_send_tcp_packet (int payload_len,int dport,int sport,u8 action, +u32 tcp_seq_num,u32 tcp_ack_num); +/* Processes a received packet */ void net_process_received_packet (uchar *in_packet,int len);
include/net/tcp.h new file +227 -0
227 lines not on this page · raw message
net/Kconfig +6 -0
@@ -22,4 +22,10 @@ config NETCONSOLE Support the 'nc' input/output device for networked console. See README.NetConsole for details. +config TCP + bool "TCP stack" + help + TCP protocol support with SACK for wget. Selecting this will provide + the fastest file transfer possible. + endif # if NET
net/Makefile +1 -0
@@ -23,6 +23,7 @@ obj-$(CONFIG_CMD_PING) += ping.o obj-$(CONFIG_CMD_RARP) += rarp.o obj-$(CONFIG_CMD_SNTP) += sntp.o obj-$(CONFIG_CMD_TFTPBOOT) += tftp.o +obj-$(CONFIG_TCP) += tcp.o # Disable this warning as it is triggered by: # sprintf(buf, index ? "foo%d" : "foo", index)
net/tcp.c new file +700 -0
700 lines not on this page · raw message
Signature · 1 line
2.11.0
PATCH v11 3/3 Add wget, for fast file transfer.
In reply to DH at synoia.comFrom: Duncan Hare <[email protected]>
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
<<<<
---
All the code is new, and not copied from any source.
Changes in v11:
Adding wget
cmd/Kconfig | 7 + cmd/net.c | 13 ++ include/net.h | 8 +- include/net/wget.h | 19 +++ net/Makefile | 1 + net/net.c | 49 ++++++- net/wget.c | 418 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 508 insertions(+), 7 deletions(-) create mode 100644 include/net/wget.h create mode 100644 net/wget.c
cmd/Kconfig +7 -0
@@ -1156,6 +1156,13 @@ config CMD_NFS help Boot image via network using NFS protocol. +config CMD_WGET + bool "wget" + select TCP + help + Download kernel, or other files, from a web server over TCP. + Fast file transfer over networks with latency. + config CMD_MII bool "mii" help
cmd/net.c +13 -0
@@ -113,6 +113,19 @@ U_BOOT_CMD( );#endif +#if defined(CONFIG_CMD_WGET ) +static int do_wget (cmd_tbl_t *cmdtp,int flag,int argc,char *const argv[]) +{ +return netboot_common (WGET , cmdtp, argc, argv); +} + +U_BOOT_CMD ( + wget,3 ,1 , do_wget, +"boot image via network using HTTP protocol" , +"[loadAddress] [[hostIPaddr:]path and image name]" +); +#endif +static void netboot_update_env (void ) {char tmp[22 ];
include/net.h +3 -5
@@ -25,9 +25,7 @@* alignment in memory. * */ -#if defined(CONFIG_TCP )/* Protected becuse UDP uses less buffers */ -#define CONFIG_SYS_RX_ETH_BUFFER 12 -#endif +#ifdef CONFIG_SYS_RX_ETH_BUFFER # define PKTBUFSRX CONFIG_SYS_RX_ETH_BUFFER#else
@@ -86,7 +84,7 @@ enum eth_state_t { };#ifdef CONFIG_DM_ETH -/* +/** * struct eth_pdata - Platform data for Ethernet MAC controllers * * @iobase: The base address of the hardware registers
@@ -538,7 +536,7 @@ extern int net_restart_wrap; /* Tried all network devices */enum proto_t {BOOTP ,RARP ,ARP ,TFTPGET ,DHCP ,PING ,DNS ,NFS ,CDP ,NETCONS ,SNTP , -TFTPSRV ,TFTPPUT ,LINKLOCAL +TFTPSRV ,TFTPPUT ,LINKLOCAL ,WGET };extern char net_boot_file_name[1024 ];/* Boot File name */
include/net/wget.h new file +19 -0
@@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* +* Duncan Hare Copyright 2017 +*/ + +void wget_start (void );/* Begin wget */ + +enum WGET_STATE { +WGET_CLOSED , +WGET_CONNECTING , +WGET_CONNECTED , +WGET_TRANSFERRING , +WGET_TRANSFERRED +}; + +#define DEBUG_WGET 0/* Set to 1 for debug messges */ +#define SERVER_PORT 80 +#define WGET_RETRY_COUNT 30 +#define WGET_TIMEOUT 2000UL
net/Makefile +1 -0
@@ -24,6 +24,7 @@ obj-$(CONFIG_CMD_RARP) += rarp.o obj-$(CONFIG_CMD_SNTP) += sntp.o obj-$(CONFIG_CMD_TFTPBOOT) += tftp.o obj-$(CONFIG_TCP) += tcp.o +obj-$(CONFIG_CMD_WGET) += wget.o # Disable this warning as it is triggered by: # sprintf(buf, index ? "foo%d" : "foo", index)
net/net.c +47 -2
97 lines not on this page · raw message
net/wget.c new file +418 -0
418 lines not on this page · raw message
Signature · 1 line
2.11.0
v11 cover Why netboot:
In reply to DH at synoia.comHi Duncan,
I tried a test of applying this series and the first patch doesn't apply cleanly and checkpatch complains about Missing or malformed SPDX-License-Identifier.
Can you rebase on top of the current upstream master and resend?
Quoted 1 line · On Sun, May 20, 2018 at 6:41 PM, <[email protected]> wrote:
From: Duncan Hare <[email protected]>
Is there a reason you are not using (what appears to be) your company email address in your git configuration?
Quoted 87 lines
Central management, including logs and change control, coupled with with enhanced security and unauthorized change detection and remediation by exposing a small attack surface.
Why TCP with Selective Acknowledgement:
Currently file transfer are done using tftp or NFS both over udp. This requires a request to be sent from client (u-boot) to the boot server.
For a 4 Mbyte kernel, with a 1k block size this requires 4,000 request for a block.
Using a large block size, one greater than the Ethernet maximum frame size limitation, would require fragmentation, which u-boot supports. However missing fragment recovery requires timeout detection and re-transmission requests for missing fragments.
UDP is ideally suited to fast single packet exchanges, inquiry/response, for example dns, becuse of the lack of connection overhead.
UDP as a file transport mechanism is slow, even in low latency networks, because file transfer with udp requires poll/response mechanism to provide transfer integrity.
In networks with large latency, for example: the internet, UDP is even slower. What is a 30 second transfer on a local boot server and LAN increase to over 3 minutes, because of all the requests/response traffic.
This was anticipated in the evolution of the IP protocols and TCP was developed and then enhanced for high latency high bandwidth networks.
The current standard is TCP with selective acknowledgment.
In our testing we have reduce kernel transmission time to around 0.4 seconds for a 4Mbyte kernel, with a 100 Mbps downlink.
Why http and wget:
HTTP is the most efficient file retrieval protocol in common use. The client send a single request, after TCP connection, to receive a file of any length.
WGET is the application which implements http file transfer outside browsers as a file transfer protocol. Versions of wget exists on many operating systems.
Changes in v11:
Add TCP with Selective Acknowledgement (SACK)
Adding wgetDuncan Hare (3):
Adding TCP and wget into u-boot
Adding TCP
Add wget, for fast file transfer.cmd/Kconfig | 7 + cmd/net.c | 13 + include/net.h | 15 +- include/net/tcp.h | 227 +++++++++++++++++ include/net/wget.h | 19 ++ net/Kconfig | 6 + net/Makefile | 2 + net/net.c | 79 +++++- net/ping.c | 9 +- net/tcp.c | 700 +++++++++++++++++++++++++++++++++++++++++++++++++++++ net/wget.c | 418 ++++++++++++++++++++++++++++++++ 11 files changed, 1475 insertions(+), 20 deletions(-) create mode 100644 include/net/tcp.h create mode 100644 include/net/wget.h create mode 100644 net/tcp.c create mode 100644 net/wget.c--
2.11.0_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de
https://lists.denx.de/listinfo/u-boot