net: support QUIC crypto
Subsystems (6)
A message that is not in the archive (sent off-list or not yet received): Adel Abouchaev <[email protected]
cover net: support QUIC crypto
QUIC requires end to end encryption of the data. The application usually prepares the data in clear text, encrypts and calls send() which implies multiple copies of the data before the packets hit the networking stack. Similar to kTLS, QUIC kernel offload of cryptography reduces the memory pressure by reducing the number of copies.
The scope of kernel support is limited to the symmetric cryptography, leaving the handshake to the user space library. For QUIC in particular, the application packets that require symmetric cryptography are the 1RTT packets with short headers. Kernel will encrypt the application packets on transmission and decrypt on receive. This series implements Tx only, because in QUIC server applications Tx outweighs Rx by orders of magnitude.
Supporting the combination of QUIC and GSO requires the application to correctly place the data and the kernel to correctly slice it. The encryption process appends an arbitrary number of bytes (tag) to the end of the message to authenticate it. The GSO value should include this overhead, the offload would then subtract the tag size to parse the input on Tx before chunking and encrypting it.
With the kernel cryptography, the buffer copy operation is conjoined with the encryption operation. The memory bandwidth is reduced by 5-8%. When devices supporting QUIC encryption in hardware come to the market, we will be able to free further 7% of CPU utilization which is used today for crypto operations.
Adel Abouchaev (6):
Documentation on QUIC kernel Tx crypto.
Define QUIC specific constants, control and data plane structures Add UDP ULP operations, initialization and handling prototype
functions.
Implement QUIC offload functions
Add flow counters and Tx processing error counter
Add self tests for ULP operations, flow setup and crypto tests
Documentation/networking/quic.rst | 176 +++ include/net/inet_sock.h | 2 + include/net/netns/mib.h | 3 + include/net/quic.h | 59 + include/net/snmp.h | 6 + include/net/udp.h | 33 + include/uapi/linux/quic.h | 61 + include/uapi/linux/snmp.h | 11 + include/uapi/linux/udp.h | 4 + net/Kconfig | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 14 + net/ipv4/udp_ulp.c | 190 ++++ net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1446 ++++++++++++++++++++++++ net/quic/quic_proc.c | 45 + tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/Makefile | 2 +- tools/testing/selftests/net/quic.c | 1024 +++++++++++++++++ tools/testing/selftests/net/quic.sh | 45 + 22 files changed, 3149 insertions(+), 2 deletions(-) create mode 100644 Documentation/networking/quic.rst create mode 100644 include/net/quic.h create mode 100644 include/uapi/linux/quic.h create mode 100644 net/ipv4/udp_ulp.c create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c create mode 100644 net/quic/quic_proc.c create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
Signature · 1 line
2.30.2
RFC 1/6 net: Documentation on QUIC kernel Tx crypto.
Adding Documentation/networking/quic.rst file to describe kernel QUIC code.
--- Documentation/networking/quic.rst | 176 ++++++++++++++++++++++++++++++ 1 file changed, 176 insertions(+) create mode 100644 Documentation/networking/quic.rst
Documentation/networking/quic.rst new file +176 -0
@@ -0,0 +1,176 @@ +.. _kernel_quic: + +=========== +KERNEL QUIC +=========== + +Overview +======== + +QUIC is a secure general-purpose transport protocol that creates a stateful +interaction between a client and a server. QUIC provides end-to-end integrity +and confidentiality. Refer to RFC 9000 for more information on QUIC. + +The kernel Tx side offload covers the encryption of the application streams +in the kernel rather than in the application. These packets are 1RTT packets +in QUIC connection. Encryption of every other packets is still done by the +QUIC library in user space. + + + +User Interface +============== + +Creating a QUIC connection +-------------------------- + +QUIC connection originates and terminates in the application, using one of many +available QUIC libraries. The code instantiates QUIC client and QUIC server in +some form and configures them to use certain addresses and ports for the +source and destination. The client and server negotiate the set of keys to +protect the communication during different phases of the connection, maintain +the connection and perform congestion control. + +Requesting to add QUIC Tx kernel encryption to the connection +------------------------------------------------------------- + +Each flow that should be encrypted by the kernel needs to be registered with +the kernel using socket API. A setsockopt() call on the socket creates an +association between the QUIC connection ID of the flow with the encryption +parameters for the crypto operations: + +.. code_block:: c + struct quic_connection_info conn_info; + char conn_id[5] = {0x01, 0x02, 0x03, 0x04, 0x05}; + const size_t conn_id_len = sizeof(conn_id); + char conn_key[16] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f}; + char conn_iv[12] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b}; + char conn_hdr_key[16] = {0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, + 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f}; + + conn_info.cipher_type = TLS_CIPHER_AES_GCM_128; + + memset(&conn_info.key, 0, sizeof(struct quic_connection_info_key)); + conn_info.key.conn_id_length = 5; + memcpy(&conn_info.key.conn_id[QUIC_MAX_CONNECTION_ID_SIZE - conn_id_len], + &conn_id, conn_id_len); + + memcpy(&conn_info.payload_key, conn_key, sizeof(conn_key)); + memcpy(&conn_info.payload_iv, conn_iv, sizeof(conn_iv)); + memcpy(&conn_info.header_key, conn_hdr_key, sizeof(conn_hdr_key)); + + setsockopt(fd, SOL_UDP, UDP_QUIC_ADD_TX_CONNECTION, &conn_info, + sizeof(conn_info)); + + +Requesting to remove QUIC Tx kernel crypto offload control messages +------------------------------------------------------------------- + +All flows are removed when the socket is closed. To request an explicit remove +of the offload for the connection during the lifetime of the socket the process +is similar to adding the flow. Only the connection ID and its length are +necessary to supply to remove the connection from the offload: + +.. code_block:: c + + memset(&conn_info.key, 0, sizeof(struct quic_connection_info_key)); + conn_info.key.conn_id_length = 5; + memcpy(&conn_info.key.conn_id[QUIC_MAX_CONNECTION_ID_SIZE - conn_id_len], + &conn_id, conn_id_len); + setsockopt(fd, SOL_UDP, UDP_QUIC_DEL_TX_CONNECTION, &conn_info, + sizeof(conn_info)); + +Sending QUIC application data +----------------------------- + +For QUIC Tx encryption offload, the application should use sendmsg() socket +call and provide ancillary data with information on connection ID length and +offload flags for the kernel to perform the encryption and GSO support if +requested. + +.. code_block:: c + + size_t cmsg_tx_len = sizeof(struct quic_tx_ancillary_data); + uint8_t cmsg_buf[CMSG_SPACE(cmsg_tx_len)]; + struct quic_tx_ancillary_data * anc_data; + size_t quic_data_len = 4500; + struct cmsghdr * cmsg_hdr; + char quic_data[9000]; + struct iovec iov[2]; + int send_len = 9000; + struct msghdr msg; + int err; + + iov[0].iov_base = quic_data; + iov[0].iov_len = quic_data_len; + iov[1].iov_base = quic_data + 4500; + iov[1].iov_len = quic_data_len; + + if (client.addr.sin_family == AF_INET) { + msg.msg_name = &client.addr; + msg.msg_namelen = sizeof(client.addr); + } else { + msg.msg_name = &client.addr6; + msg.msg_namelen = sizeof(client.addr6); + } + + msg.msg_iov = iov; + msg.msg_iovlen = 2; + msg.msg_control = cmsg_buf; + msg.msg_controllen = sizeof(cmsg_buf); + cmsg_hdr = CMSG_FIRSTHDR(&msg); + cmsg_hdr->cmsg_level = IPPROTO_UDP; + cmsg_hdr->cmsg_type = UDP_QUIC_ENCRYPT; + cmsg_hdr->cmsg_len = CMSG_LEN(cmsg_tx_len); + anc_data = CMSG_DATA(cmsg_hdr); + anc_data->flags = 0; + anc_data->next_pkt_num = 0x0d65c9; + anc_data->conn_id_length = conn_id_len; + err = sendmsg(self->sfd, &msg, 0); + +QUIC Tx offload in kernel will read the data from userspace, encrypt and +copy it to the ciphertext within the same operation. + + +Sending QUIC application data with GSO +-------------------------------------- +When GSO is in use, the kernel will use the GSO fragment size as the target +for ciphertext. The packets from the user space should align on the boundary +of GSO fragment size minus the size of the tag for the chosen cipher. For the +GSO fragment 1200, the plain packets should follow each other at every 1184 +bytes, given the tag size of 16. After the encryption, the rest of the UDP +and IP stacks will follow the defined value of GSO fragment which will include +the trailing tag bytes. + +To set up GSO fragmentation: + +.. code_block:: c + setsockopt(self->sfd, SOL_UDP, UDP_SEGMENT, &frag_size, sizeof(frag_size)); + +If the GSO fragment size is provided in ancillary data within the sendmsg() +call, the value in ancillary data will take precedence over the segment size +provided in setsockopt to split the payload into packets. This is consistent +with the UDP stack behavior. + +Integrating to userspace QUIC libraries +--------------------------------------- + +Userspace QUIC libraries integration would depend on the implementation of the +QUIC protocol. For MVFST library, the control plane is integrated into the +handshake callbacks to properly configure the flows into the socket; and the +data plane is integrated into the methods that perform encryption and send +the packets to the batch scheduler for transmissions to the socket. + +MVFST library can be found at https://github.com/facebookincubator/mvfst. + +Statistics +========== + +QUIC Tx offload to the kernel has counters reflected in /proc/net/quic_stat: + + QuicCurrTxSw - number of currently active kernel offloaded QUIC connections + QuicTxSw - accumulative total number of offloaded QUIC connections + QuicTxSwError - accumulative total number of errors during QUIC Tx offload to + kernel
Signature · 1 line
2.30.2
RFC 2/6 net: Define QUIC specific constants, control and data plane structures
In reply to Adel AbouchaevDefine control and data plane structures to pass in control plane for flow add/remove and during packet send within ancillary data. Define constants to use within SOL_UDP to program QUIC sockets.
--- include/uapi/linux/quic.h | 61 +++++++++++++++++++++++++++++++++++++++ include/uapi/linux/udp.h | 3 ++ 2 files changed, 64 insertions(+) create mode 100644 include/uapi/linux/quic.h
include/uapi/linux/quic.h new file +61 -0
@@ -0,0 +1,61 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) */ + +#ifndef _UAPI_LINUX_QUIC_H +#define _UAPI_LINUX_QUIC_H + +#include <linux/types.h> +#include <linux/tls.h> + +#define QUIC_MAX_CONNECTION_ID_SIZE 20 + +/* Side by side data for QUIC egress operations */ +#define QUIC_BYPASS_ENCRYPTION 0x01 + +struct quic_tx_ancillary_data { +__aligned_u64 next_pkt_num; +__u8 flags; +__u8 conn_id_length; +}; + +struct quic_connection_info_key { +__u8 conn_id[QUIC_MAX_CONNECTION_ID_SIZE ]; +__u8 conn_id_length; +}; + +struct quic_aes_gcm_128 { +__u8 header_key[TLS_CIPHER_AES_GCM_128_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_GCM_128_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_GCM_128_IV_SIZE ]; +}; + +struct quic_aes_gcm_256 { +__u8 header_key[TLS_CIPHER_AES_GCM_256_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_GCM_256_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_GCM_256_IV_SIZE ]; +}; + +struct quic_aes_ccm_128 { +__u8 header_key[TLS_CIPHER_AES_CCM_128_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_CCM_128_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_CCM_128_IV_SIZE ]; +}; + +struct quic_chacha20_poly1305 { +__u8 header_key[TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_CHACHA20_POLY1305_IV_SIZE ]; +}; + +struct quic_connection_info { +__u16 cipher_type; +struct quic_connection_info_key key; +union { +struct quic_aes_gcm_128 aes_gcm_128; +struct quic_aes_gcm_256 aes_gcm_256; +struct quic_aes_ccm_128 aes_ccm_128; +struct quic_chacha20_poly1305 chacha20_poly1305; + }; +}; + +#endif +
include/uapi/linux/udp.h +3 -0
@@ -34,6 +34,9 @@ struct udphdr {#define UDP_NO_CHECK6_RX 102/* Disable accpeting checksum for UDP6 */ #define UDP_SEGMENT 103/* Set GSO segmentation size */ #define UDP_GRO 104/* This socket can receive UDP GRO packets */ +#define UDP_QUIC_ADD_TX_CONNECTION 106/* Add QUIC Tx crypto offload */ +#define UDP_QUIC_DEL_TX_CONNECTION 107/* Del QUIC Tx crypto offload */ +#define UDP_QUIC_ENCRYPT 108/* QUIC encryption parameters */ /* UDP encapsulation types */ #define UDP_ENCAP_ESPINUDP_NON_IKE 1/* draft-ietf-ipsec-nat-t-ike-00/01 */
Signature · 1 line
2.30.2
RFC 3/6 net: Add UDP ULP operations, initialization and handling prototype functions.
In reply to Adel AbouchaevDefine functions to add UDP ULP handling, registration with UDP protocol and supporting data structures. Create structure for QUIC ULP and add empty prototype functions to support it.
--- include/net/inet_sock.h | 2 + include/net/udp.h | 33 +++++++ include/uapi/linux/udp.h | 1 + net/Kconfig | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 6 ++ net/ipv4/udp_ulp.c | 190 +++++++++++++++++++++++++++++++++++++++ 8 files changed, 236 insertions(+), 1 deletion(-) create mode 100644 net/ipv4/udp_ulp.c
include/net/inet_sock.h +2 -0
@@ -238,6 +238,8 @@ struct inet_sock {__be32 mc_addr;struct ip_mc_socklist __rcu *mc_list;struct inet_cork_full cork; +const struct udp_ulp_ops *udp_ulp_ops; +void __rcu *ulp_data; };#define IPCORK_OPT 1/* ip-options has been held in ipcork.opt */
include/net/udp.h +33 -0
@@ -523,4 +523,37 @@ struct proto *udp_bpf_get_proto(struct sock *sk, struct sk_psock *psock);int udp_bpf_update_proto (struct sock *sk,struct sk_psock *psock,bool restore);#endif +/* +* Interface for adding Upper Level Protocols over UDP +*/ + +#define UDP_ULP_NAME_MAX 16 +#define UDP_ULP_MAX 128 + +struct udp_ulp_ops { +struct list_head list; + +/* initialize ulp */ +int (*init)(struct sock *sk); +/* cleanup ulp */ +void (*release)(struct sock *sk); + +char name[UDP_ULP_NAME_MAX ]; +struct module *owner; +}; + +int udp_register_ulp (struct udp_ulp_ops *type); +void udp_unregister_ulp (struct udp_ulp_ops *type); +int udp_set_ulp (struct sock *sk,const char *name); +void udp_get_available_ulp (char *buf,size_t len); +void udp_cleanup_ulp (struct sock *sk); +int udp_setsockopt_ulp (struct sock *sk,sockptr_t optval, +unsigned int optlen); +int udp_getsockopt_ulp (struct sock *sk,char __user *optval, +int __user *optlen); + +#define MODULE_ALIAS_UDP_ULP (name)\ + __MODULE_INFO(alias, alias_userspace, name);\ + __MODULE_INFO(alias, alias_udp_ulp, "udp-ulp-" name) +#endif /* _UDP_H */
include/uapi/linux/udp.h +1 -0
@@ -34,6 +34,7 @@ struct udphdr {#define UDP_NO_CHECK6_RX 102/* Disable accpeting checksum for UDP6 */ #define UDP_SEGMENT 103/* Set GSO segmentation size */ #define UDP_GRO 104/* This socket can receive UDP GRO packets */ +#define UDP_ULP 105/* Attach ULP to a UDP socket */ #define UDP_QUIC_ADD_TX_CONNECTION 106/* Add QUIC Tx crypto offload */ #define UDP_QUIC_DEL_TX_CONNECTION 107/* Del QUIC Tx crypto offload */ #define UDP_QUIC_ENCRYPT 108/* QUIC encryption parameters */
net/Kconfig +1 -0
@@ -63,6 +63,7 @@ menu "Networking options" source "net/packet/Kconfig" source "net/unix/Kconfig" source "net/tls/Kconfig" +source "net/quic/Kconfig" source "net/xfrm/Kconfig" source "net/iucv/Kconfig" source "net/smc/Kconfig"
net/Makefile +1 -0
@@ -16,6 +16,7 @@ obj-y += ethernet/ 802/ sched/ netlink/ bpf/ ethtool/ obj-$(CONFIG_NETFILTER) += netfilter/ obj-$(CONFIG_INET) += ipv4/ obj-$(CONFIG_TLS) += tls/ +obj-$(CONFIG_QUIC) += quic/ obj-$(CONFIG_XFRM) += xfrm/ obj-$(CONFIG_UNIX_SCM) += unix/ obj-y += ipv6/
net/ipv4/Makefile +2 -1
@@ -14,7 +14,8 @@ obj-y := route.o inetpeer.o protocol.o \ udp_offload.o arp.o icmp.o devinet.o af_inet.o igmp.o \ fib_frontend.o fib_semantics.o fib_trie.o fib_notifier.o \ inet_fragment.o ping.o ip_tunnel_core.o gre_offload.o \ - metrics.o netlink.o nexthop.o udp_tunnel_stub.o + metrics.o netlink.o nexthop.o udp_tunnel_stub.o \ + udp_ulp.o obj-$(CONFIG_BPFILTER) += bpfilter/
net/ipv4/udp.c +6 -0
@@ -2778,6 +2778,9 @@ int udp_lib_setsockopt(struct sock *sk, int level, int optname, up->pcflag |=UDPLITE_RECV_CC ;break ; +case UDP_ULP : +return udp_setsockopt_ulp (sk, optval, optlen); +default : err = -ENOPROTOOPT ;break ;
@@ -2846,6 +2849,9 @@ int udp_lib_getsockopt(struct sock *sk, int level, int optname, val = up->pcrlen;break ; +case UDP_ULP : +return udp_getsockopt_ulp (sk, optval, optlen); +default :return -ENOPROTOOPT ; }
net/ipv4/udp_ulp.c new file +190 -0
190 lines not on this page · raw message
Signature · 1 line
2.30.2
RFC 4/6 net: Implement QUIC offload functions
In reply to Adel AbouchaevAdd connection hash to the context do support add, remove operations on QUIC connections for the control plane and lookup for the data plane. Implement setsockopt and add placeholders to add and delete Tx connections.
--- include/net/quic.h | 49 ++ net/ipv4/udp.c | 8 + net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1400 ++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 1481 insertions(+) create mode 100644 include/net/quic.h create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c
include/net/quic.h new file +49 -0
@@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef INCLUDE_NET_QUIC_H +#define INCLUDE_NET_QUIC_H + +#include <linux/mutex.h> +#include <linux/rhashtable.h> +#include <linux/skmsg.h> +#include <uapi/linux/quic.h> + +#define QUIC_MAX_SHORT_HEADER_SIZE 25 +#define QUIC_MAX_CONNECTION_ID_SIZE 20 +#define QUIC_HDR_MASK_SIZE 16 +#define QUIC_MAX_GSO_FRAGS 16 + +// Maximum IV and nonce sizes should be in sync with supported ciphers. +#define QUIC_CIPHER_MAX_IV_SIZE 12 +#define QUIC_CIPHER_MAX_NONCE_SIZE 16 + +/* Side by side data for QUIC egress operations */ +#define QUIC_ANCILLARY_FLAGS (QUIC_BYPASS_ENCRYPTION) + +#define QUIC_MAX_IOVEC_SEGMENTS 8 +#define QUIC_MAX_SG_ALLOC_ELEMENTS 32 +#define QUIC_MAX_PLAIN_PAGES 16 +#define QUIC_MAX_CIPHER_PAGES_ORDER 4 + +struct quic_internal_crypto_context { +struct quic_connection_info conn_info; +struct crypto_skcipher *header_tfm; +struct crypto_aead *packet_aead; +}; + +struct quic_connection_rhash { +struct rhash_head node; +struct quic_internal_crypto_context crypto_ctx; +struct rcu_head rcu; +}; + +struct quic_context { +struct proto *sk_proto; +struct rhashtable tx_connections; +struct scatterlist sg_alloc[QUIC_MAX_SG_ALLOC_ELEMENTS ]; +struct page *cipher_page; +struct mutex sendmsg_mux; +struct rcu_head rcu; +}; + +#endif
net/ipv4/udp.c +8 -0
@@ -113,6 +113,7 @@#include <net/sock_reuseport.h> #include <net/addrconf.h> #include <net/udp_tunnel.h> +#include <uapi/linux/quic.h> #if IS_ENABLED (CONFIG_IPV6 )#include <net/ipv6_stubs.h> #endif
@@ -1009,6 +1010,13 @@ static int __udp_cmsg_send(struct cmsghdr *cmsg, u16 *gso_size)return -EINVAL ; *gso_size = *(__u16 *)CMSG_DATA (cmsg);return 0 ; +case UDP_QUIC_ENCRYPT : +/* This option is handled in UDP_ULP and is only checked +* here for the bypass bit +*/ +if (cmsg->cmsg_len !=CMSG_LEN (sizeof (struct quic_tx_ancillary_data ))) +return -EINVAL ; +return 0 ;default :return -EINVAL ; }
net/quic/Kconfig new file +16 -0
@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# QUIC configuration +# +config QUIC + tristate "QUIC encryption offload" + depends on INET + select CRYPTO + select CRYPTO_AES + select CRYPTO_GCM + help + Enable kernel support for QUIC crypto offload. Currently only TX + encryption offload is supported. The kernel will perform + copy-during-encryption. + + If unsure, say N.
net/quic/Makefile new file +8 -0
@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Makefile for the QUIC subsystem +# + +obj-$(CONFIG_QUIC) += quic.o + +quic-y := quic_main.o
net/quic/quic_main.c new file +1400 -0
1,400 lines not on this page · raw message
Signature · 1 line
2.30.2
RFC 5/6 net: Add flow counters and Tx processing error counter
In reply to Adel AbouchaevAdded flow counters. Total flow counter is accumulative, the current shows the number of flows currently in flight, the error counters is accumulating the number of errors during Tx processing.
--- include/net/netns/mib.h | 3 +++ include/net/quic.h | 10 +++++++++ include/net/snmp.h | 6 +++++ include/uapi/linux/snmp.h | 11 ++++++++++ net/quic/Makefile | 2 +- net/quic/quic_main.c | 46 +++++++++++++++++++++++++++++++++++++++ net/quic/quic_proc.c | 45 ++++++++++++++++++++++++++++++++++++++ 7 files changed, 122 insertions(+), 1 deletion(-) create mode 100644 net/quic/quic_proc.c
include/net/netns/mib.h +3 -0
@@ -24,6 +24,9 @@ struct netns_mib {#if IS_ENABLED (CONFIG_TLS ) DEFINE_SNMP_STAT(struct linux_tls_mib, tls_statistics);#endif +#if IS_ENABLED (CONFIG_QUIC ) + DEFINE_SNMP_STAT(struct linux_quic_mib, quic_statistics); +#endif #ifdef CONFIG_MPTCP DEFINE_SNMP_STAT(struct mptcp_mib, mptcp_statistics);#endif
include/net/quic.h +10 -0
@@ -25,6 +25,16 @@#define QUIC_MAX_PLAIN_PAGES 16#define QUIC_MAX_CIPHER_PAGES_ORDER 4 +#define __QUIC_INC_STATS (net, field) \ + __SNMP_INC_STATS((net)->mib.quic_statistics, field) +#define QUIC_INC_STATS (net, field) \ + SNMP_INC_STATS((net)->mib.quic_statistics, field) +#define QUIC_DEC_STATS (net, field) \ + SNMP_DEC_STATS((net)->mib.quic_statistics, field) + +int __net_init quic_proc_init (struct net *net); +void __net_exit quic_proc_fini(struct net *net); +struct quic_internal_crypto_context {struct quic_connection_info conn_info;struct crypto_skcipher *header_tfm;
include/net/snmp.h +6 -0
@@ -117,6 +117,12 @@ struct linux_tls_mib {unsigned long mibs[LINUX_MIB_TLSMAX ]; }; +/* Linux QUIC */ +#define LINUX_MIB_QUICMAX __LINUX_MIB_QUICMAX +struct linux_quic_mib { +unsigned long mibs[LINUX_MIB_QUICMAX ]; +}; +#define DEFINE_SNMP_STAT (type, name) \ __typeof__(type) __percpu *name#define DEFINE_SNMP_STAT_ATOMIC (type, name) \
include/uapi/linux/snmp.h +11 -0
@@ -347,4 +347,15 @@ enum__LINUX_MIB_TLSMAX }; +/* linux QUIC mib definitions */ +enum +{ +LINUX_MIB_QUICNUM =0 , +LINUX_MIB_QUICCURRTXSW ,/* QuicCurrTxSw */ +LINUX_MIB_QUICTXSW ,/* QuicTxSw */ +LINUX_MIB_QUICTXSWERROR ,/* QuicTxSwError */ + __LINUX_MIB_QUICMAX +}; + +#endif /* _LINUX_SNMP_H */
net/quic/Makefile +1 -1
@@ -5,4 +5,4 @@ obj-$(CONFIG_QUIC) += quic.o -quic-y := quic_main.o +quic-y := quic_main.o quic_proc.o
net/quic/quic_main.c +46 -0
88 lines not on this page · raw message
net/quic/quic_proc.c new file +45 -0
45 lines not on this page · raw message
Signature · 1 line
2.30.2
RFC 6/6 net: Add self tests for ULP operations, flow setup and crypto tests
In reply to Adel AbouchaevAdd self tests for ULP operations, flow setup and crypto tests.
--- tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/Makefile | 2 +- tools/testing/selftests/net/quic.c | 1024 ++++++++++++++++++++++++ tools/testing/selftests/net/quic.sh | 45 ++ 4 files changed, 1071 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
tools/testing/selftests/net/.gitignore +1 -0
@@ -38,3 +38,4 @@ ioam6_parser toeplitz tun cmsg_sender +quic
tools/testing/selftests/net/Makefile +1 -1
@@ -54,7 +54,7 @@ TEST_GEN_FILES += ipsec TEST_GEN_FILES += ioam6_parser TEST_GEN_FILES += gro TEST_GEN_PROGS = reuseport_bpf reuseport_bpf_cpu reuseport_bpf_numa -TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls tun +TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls tun quic TEST_GEN_FILES += toeplitz TEST_GEN_FILES += cmsg_sender TEST_GEN_FILES += stress_reuseport_listen
tools/testing/selftests/net/quic.c new file +1024 -0
1,024 lines not on this page · raw message
tools/testing/selftests/net/quic.sh new file +45 -0
45 lines not on this page · raw message
Signature · 1 line
2.30.2
Quoted 21 lines · Adel Abouchaev
QUIC requires end to end encryption of the data. The application usually prepares the data in clear text, encrypts and calls
send()which implies multiple copies of the data before the packets hit the networking stack. Similar to kTLS, QUIC kernel offload of cryptography reduces the memory pressure by reducing the number of copies.The scope of kernel support is limited to the symmetric cryptography, leaving the handshake to the user space library. For QUIC in particular, the application packets that require symmetric cryptography are the 1RTT packets with short headers. Kernel will encrypt the application packets on transmission and decrypt on receive. This series implements Tx only, because in QUIC server applications Tx outweighs Rx by orders of magnitude.
Supporting the combination of QUIC and GSO requires the application to correctly place the data and the kernel to correctly slice it. The encryption process appends an arbitrary number of bytes (tag) to the end of the message to authenticate it. The GSO value should include this overhead, the offload would then subtract the tag size to parse the input on Tx before chunking and encrypting it.
Quoted 6 lines
With the kernel cryptography, the buffer copy operation is conjoined with the encryption operation. The memory bandwidth is reduced by 5-8%. When devices supporting QUIC encryption in hardware come to the market, we will be able to free further 7% of CPU utilization which is used today for crypto operations.
Hi,
I can't apply this series on top of current net-next. On what commit on net-next this series is based?
Signature · 1 line
An old man doll... just what I always wanted! - Clara
cover net: support QUIC crypto
In reply to a message not in the archiveQUIC requires end to end encryption of the data. The application usually prepares the data in clear text, encrypts and calls send() which implies multiple copies of the data before the packets hit the networking stack. Similar to kTLS, QUIC kernel offload of cryptography reduces the memory pressure by reducing the number of copies.
The scope of kernel support is limited to the symmetric cryptography, leaving the handshake to the user space library. For QUIC in particular, the application packets that require symmetric cryptography are the 1RTT packets with short headers. Kernel will encrypt the application packets on transmission and decrypt on receive. This series implements Tx only, because in QUIC server applications Tx outweighs Rx by orders of magnitude.
Supporting the combination of QUIC and GSO requires the application to correctly place the data and the kernel to correctly slice it. The encryption process appends an arbitrary number of bytes (tag) to the end of the message to authenticate it. The GSO value should include this overhead, the offload would then subtract the tag size to parse the input on Tx before chunking and encrypting it.
With the kernel cryptography, the buffer copy operation is conjoined with the encryption operation. The memory bandwidth is reduced by 5-8%. When devices supporting QUIC encryption in hardware come to the market, we will be able to free further 7% of CPU utilization which is used today for crypto operations.
Adel Abouchaev (6):
Documentation on QUIC kernel Tx crypto.
Define QUIC specific constants, control and data plane structures Add UDP ULP operations, initialization and handling prototype
functions.
Implement QUIC offload functions
Add flow counters and Tx processing error counter
Add self tests for ULP operations, flow setup and crypto tests
Documentation/networking/quic.rst | 176 +++ include/net/inet_sock.h | 2 + include/net/netns/mib.h | 3 + include/net/quic.h | 59 + include/net/snmp.h | 6 + include/net/udp.h | 33 + include/uapi/linux/quic.h | 61 + include/uapi/linux/snmp.h | 11 + include/uapi/linux/udp.h | 4 + net/Kconfig | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 14 + net/ipv4/udp_ulp.c | 190 ++++ net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1446 ++++++++++++++++++++++++ net/quic/quic_proc.c | 45 + tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/Makefile | 2 +- tools/testing/selftests/net/quic.c | 1024 +++++++++++++++++ tools/testing/selftests/net/quic.sh | 45 + 22 files changed, 3149 insertions(+), 2 deletions(-) create mode 100644 Documentation/networking/quic.rst create mode 100644 include/net/quic.h create mode 100644 include/uapi/linux/quic.h create mode 100644 net/ipv4/udp_ulp.c create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c create mode 100644 net/quic/quic_proc.c create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
Signature · 1 line
2.30.2
RFC 1/6 net: Documentation on QUIC kernel Tx crypto.
Adding Documentation/networking/quic.rst file to describe kernel QUIC code.
--- Documentation/networking/quic.rst | 176 ++++++++++++++++++++++++++++++ 1 file changed, 176 insertions(+) create mode 100644 Documentation/networking/quic.rst
Documentation/networking/quic.rst new file +176 -0
@@ -0,0 +1,176 @@ +.. _kernel_quic: + +=========== +KERNEL QUIC +=========== + +Overview +======== + +QUIC is a secure general-purpose transport protocol that creates a stateful +interaction between a client and a server. QUIC provides end-to-end integrity +and confidentiality. Refer to RFC 9000 for more information on QUIC. + +The kernel Tx side offload covers the encryption of the application streams +in the kernel rather than in the application. These packets are 1RTT packets +in QUIC connection. Encryption of every other packets is still done by the +QUIC library in user space. + + + +User Interface +============== + +Creating a QUIC connection +-------------------------- + +QUIC connection originates and terminates in the application, using one of many +available QUIC libraries. The code instantiates QUIC client and QUIC server in +some form and configures them to use certain addresses and ports for the +source and destination. The client and server negotiate the set of keys to +protect the communication during different phases of the connection, maintain +the connection and perform congestion control. + +Requesting to add QUIC Tx kernel encryption to the connection +------------------------------------------------------------- + +Each flow that should be encrypted by the kernel needs to be registered with +the kernel using socket API. A setsockopt() call on the socket creates an +association between the QUIC connection ID of the flow with the encryption +parameters for the crypto operations: + +.. code_block:: c + struct quic_connection_info conn_info; + char conn_id[5] = {0x01, 0x02, 0x03, 0x04, 0x05}; + const size_t conn_id_len = sizeof(conn_id); + char conn_key[16] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f}; + char conn_iv[12] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b}; + char conn_hdr_key[16] = {0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, + 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f}; + + conn_info.cipher_type = TLS_CIPHER_AES_GCM_128; + + memset(&conn_info.key, 0, sizeof(struct quic_connection_info_key)); + conn_info.key.conn_id_length = 5; + memcpy(&conn_info.key.conn_id[QUIC_MAX_CONNECTION_ID_SIZE - conn_id_len], + &conn_id, conn_id_len); + + memcpy(&conn_info.payload_key, conn_key, sizeof(conn_key)); + memcpy(&conn_info.payload_iv, conn_iv, sizeof(conn_iv)); + memcpy(&conn_info.header_key, conn_hdr_key, sizeof(conn_hdr_key)); + + setsockopt(fd, SOL_UDP, UDP_QUIC_ADD_TX_CONNECTION, &conn_info, + sizeof(conn_info)); + + +Requesting to remove QUIC Tx kernel crypto offload control messages +------------------------------------------------------------------- + +All flows are removed when the socket is closed. To request an explicit remove +of the offload for the connection during the lifetime of the socket the process +is similar to adding the flow. Only the connection ID and its length are +necessary to supply to remove the connection from the offload: + +.. code_block:: c + + memset(&conn_info.key, 0, sizeof(struct quic_connection_info_key)); + conn_info.key.conn_id_length = 5; + memcpy(&conn_info.key.conn_id[QUIC_MAX_CONNECTION_ID_SIZE - conn_id_len], + &conn_id, conn_id_len); + setsockopt(fd, SOL_UDP, UDP_QUIC_DEL_TX_CONNECTION, &conn_info, + sizeof(conn_info)); + +Sending QUIC application data +----------------------------- + +For QUIC Tx encryption offload, the application should use sendmsg() socket +call and provide ancillary data with information on connection ID length and +offload flags for the kernel to perform the encryption and GSO support if +requested. + +.. code_block:: c + + size_t cmsg_tx_len = sizeof(struct quic_tx_ancillary_data); + uint8_t cmsg_buf[CMSG_SPACE(cmsg_tx_len)]; + struct quic_tx_ancillary_data * anc_data; + size_t quic_data_len = 4500; + struct cmsghdr * cmsg_hdr; + char quic_data[9000]; + struct iovec iov[2]; + int send_len = 9000; + struct msghdr msg; + int err; + + iov[0].iov_base = quic_data; + iov[0].iov_len = quic_data_len; + iov[1].iov_base = quic_data + 4500; + iov[1].iov_len = quic_data_len; + + if (client.addr.sin_family == AF_INET) { + msg.msg_name = &client.addr; + msg.msg_namelen = sizeof(client.addr); + } else { + msg.msg_name = &client.addr6; + msg.msg_namelen = sizeof(client.addr6); + } + + msg.msg_iov = iov; + msg.msg_iovlen = 2; + msg.msg_control = cmsg_buf; + msg.msg_controllen = sizeof(cmsg_buf); + cmsg_hdr = CMSG_FIRSTHDR(&msg); + cmsg_hdr->cmsg_level = IPPROTO_UDP; + cmsg_hdr->cmsg_type = UDP_QUIC_ENCRYPT; + cmsg_hdr->cmsg_len = CMSG_LEN(cmsg_tx_len); + anc_data = CMSG_DATA(cmsg_hdr); + anc_data->flags = 0; + anc_data->next_pkt_num = 0x0d65c9; + anc_data->conn_id_length = conn_id_len; + err = sendmsg(self->sfd, &msg, 0); + +QUIC Tx offload in kernel will read the data from userspace, encrypt and +copy it to the ciphertext within the same operation. + + +Sending QUIC application data with GSO +-------------------------------------- +When GSO is in use, the kernel will use the GSO fragment size as the target +for ciphertext. The packets from the user space should align on the boundary +of GSO fragment size minus the size of the tag for the chosen cipher. For the +GSO fragment 1200, the plain packets should follow each other at every 1184 +bytes, given the tag size of 16. After the encryption, the rest of the UDP +and IP stacks will follow the defined value of GSO fragment which will include +the trailing tag bytes. + +To set up GSO fragmentation: + +.. code_block:: c + setsockopt(self->sfd, SOL_UDP, UDP_SEGMENT, &frag_size, sizeof(frag_size)); + +If the GSO fragment size is provided in ancillary data within the sendmsg() +call, the value in ancillary data will take precedence over the segment size +provided in setsockopt to split the payload into packets. This is consistent +with the UDP stack behavior. + +Integrating to userspace QUIC libraries +--------------------------------------- + +Userspace QUIC libraries integration would depend on the implementation of the +QUIC protocol. For MVFST library, the control plane is integrated into the +handshake callbacks to properly configure the flows into the socket; and the +data plane is integrated into the methods that perform encryption and send +the packets to the batch scheduler for transmissions to the socket. + +MVFST library can be found at https://github.com/facebookincubator/mvfst. + +Statistics +========== + +QUIC Tx offload to the kernel has counters reflected in /proc/net/quic_stat: + + QuicCurrTxSw - number of currently active kernel offloaded QUIC connections + QuicTxSw - accumulative total number of offloaded QUIC connections + QuicTxSwError - accumulative total number of errors during QUIC Tx offload to + kernel
Signature · 1 line
2.30.2
Re: [RFC net-next 1/6] net: Documentation on QUIC kernel Tx crypto.
Documentation/networking/quic.rst line 176
Quoted 9 lines
Documentation/networking/quic.rst new file +9
-0+Statistics +========== + +QUIC Tx offload to the kernel has counters reflected in /proc/net/quic_stat: + + QuicCurrTxSw - number of currently active kernel offloaded QUIC connections + QuicTxSw - accumulative total number of offloaded QUIC connections + QuicTxSwError - accumulative total number of errors during QUIC Tx offload to + kernel
netlink messages please, not /proc for statistics. netlink is the preferred way to configure and report about the network stack.
Andrew
Re: [RFC net-next 1/6] net: Documentation on QUIC kernel Tx crypto.
Andrew,
Could you add more to your comment? The /proc was used similarly to kTLS. Netlink is better, though, unsure how ULP stats would fit in it.
Cheers,
Adel.
Quoted 13 lines · Andrew Lunn
Documentation/networking/quic.rst line 176
Quoted 9 lines
Documentation/networking/quic.rst new file +9
-0+Statistics +========== + +QUIC Tx offload to the kernel has counters reflected in /proc/net/quic_stat: + + QuicCurrTxSw - number of currently active kernel offloaded QUIC connections + QuicTxSw - accumulative total number of offloaded QUIC connections + QuicTxSwError - accumulative total number of errors during QUIC Tx offload to + kernelnetlink messages please, not /proc for statistics. netlink is the preferred way to configure and report about the network stack.
Andrew
Re: [RFC net-next 1/6] net: Documentation on QUIC kernel Tx crypto.
Quoted 4 lines · Adel Abouchaev
Andrew,
Could you add more to your comment? The /proc was used similarly to kTLS. Netlink is better, though, unsure how ULP stats would fit in it.
How do tools like ss(1) retrieve the protocol summary statistics? Do they still use /proc, or netlink?
Andrew
Re: [RFC net-next 1/6] net: Documentation on QUIC kernel Tx crypto.
Looking at https://github.com/shemminger/iproute2/blob/main/misc/ss.c#L589 the ss.c still uses proc/.
Adel.
Quoted 9 lines · Andrew Lunn
Quoted 4 lines · Adel Abouchaev
Andrew,
Could you add more to your comment? The /proc was used similarly to kTLS.
Netlink is better, though, unsure how ULP stats would fit in it.How do tools like ss(1) retrieve the protocol summary statistics? Do they still use /proc, or netlink?
Andrew
Re: [RFC net-next 1/6] net: Documentation on QUIC kernel Tx crypto.
Quoted 5 lines · Adel Abouchaev
Looking at
https://github.com/shemminger/iproute2/blob/main/misc/ss.c#L589 thess.cstill uses proc/.
Only for legacy reasons.
ss -t for sure will use netlink first, then fallback to /proc
New counters should use netlink, please.
Quoted 12 lines
Adel.
Quoted 9 lines · Andrew Lunn
Quoted 4 lines · Adel Abouchaev
Andrew,
Could you add more to your comment? The /proc was used similarly to kTLS.
Netlink is better, though, unsure how ULP stats would fit in it.How do tools like ss(1) retrieve the protocol summary statistics? Do they still use /proc, or netlink?
Andrew
Jakub Kicinski
maintainerRe: [RFC net-next 1/6] net: Documentation on QUIC kernel Tx crypto.
Quoted 6 lines · On Thu, 4 Aug 2022 10:00:37 -0700 Eric Dumazet wrote:
Quoted 3 lines · Adel Abouchaev
Looking at
https://github.com/shemminger/iproute2/blob/main/misc/ss.c#L589 thess.cstill uses proc/.Only for legacy reasons.
That but in all honesty also the fact that a proc file is pretty easy and self-describing while the historic netlink families are undocumented code salads.
Quoted 3 lines
ss -t for sure will use netlink first, then fallback to /proc
New counters should use netlink, please.
Just to be sure I'm not missing anything - we're talking about some new netlink, right? Is there an existing place for "overall prot family stats" over netlink today?
Re: [RFC net-next 1/6] net: Documentation on QUIC kernel Tx crypto.
Quoted 13 lines · Jakub Kicinski
Quoted 6 lines · On Thu, 4 Aug 2022 10:00:37 -0700 Eric Dumazet wrote:
Quoted 3 lines · Adel Abouchaev
Looking at
https://github.com/shemminger/iproute2/blob/main/misc/ss.c#L589 thess.cstill uses proc/.Only for legacy reasons.
That but in all honesty also the fact that a proc file is pretty easy and self-describing while the historic netlink families are undocumented code salads.
Quoted 7 lines
Quoted 3 lines
ss -t for sure will use netlink first, then fallback to /proc
New counters should use netlink, please.
Just to be sure I'm not missing anything - we're talking about some new netlink, right? Is there an existing place for "overall prot family stats" over netlink today?
I thought we were speaking of dumping ULP info on a per UDP socket basis.
If this is about new SNMP counters, then sure, /proc is fine I guess.
Jonathan Corbet
maintainerRe: [RFC net-next 1/6] net: Documentation on QUIC kernel Tx crypto.
In reply to Adel AbouchaevQuoted 8 lines · Adel Abouchaev
Adding
Documentation/networking/quic.rstfile to describe kernel QUIC code.Signed-off-by Adel Abouchaev <[email protected]>--- Documentation/networking/quic.rst | 176 ++++++++++++++++++++++++++++++ 1 file changed, 176 insertions(+) create mode 100644 Documentation/networking/quic.rst
When you add a new RST file, you need to add it to the index.rst as well or it won't be pulled into the docs build.
Also...this all looks like user-space API documentation, so might Documentation/userspace-api be a better place for it?
Thanks,
jon
RFC 2/6 net: Define QUIC specific constants, control and data plane structures
In reply to Adel AbouchaevDefine control and data plane structures to pass in control plane for flow add/remove and during packet send within ancillary data. Define constants to use within SOL_UDP to program QUIC sockets.
--- include/uapi/linux/quic.h | 61 +++++++++++++++++++++++++++++++++++++++ include/uapi/linux/udp.h | 3 ++ 2 files changed, 64 insertions(+) create mode 100644 include/uapi/linux/quic.h
include/uapi/linux/quic.h new file +61 -0
@@ -0,0 +1,61 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) */ + +#ifndef _UAPI_LINUX_QUIC_H +#define _UAPI_LINUX_QUIC_H + +#include <linux/types.h> +#include <linux/tls.h> + +#define QUIC_MAX_CONNECTION_ID_SIZE 20 + +/* Side by side data for QUIC egress operations */ +#define QUIC_BYPASS_ENCRYPTION 0x01 + +struct quic_tx_ancillary_data { +__aligned_u64 next_pkt_num; +__u8 flags; +__u8 conn_id_length; +}; + +struct quic_connection_info_key { +__u8 conn_id[QUIC_MAX_CONNECTION_ID_SIZE ]; +__u8 conn_id_length; +}; + +struct quic_aes_gcm_128 { +__u8 header_key[TLS_CIPHER_AES_GCM_128_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_GCM_128_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_GCM_128_IV_SIZE ]; +}; + +struct quic_aes_gcm_256 { +__u8 header_key[TLS_CIPHER_AES_GCM_256_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_GCM_256_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_GCM_256_IV_SIZE ]; +}; + +struct quic_aes_ccm_128 { +__u8 header_key[TLS_CIPHER_AES_CCM_128_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_CCM_128_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_CCM_128_IV_SIZE ]; +}; + +struct quic_chacha20_poly1305 { +__u8 header_key[TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_CHACHA20_POLY1305_IV_SIZE ]; +}; + +struct quic_connection_info { +__u16 cipher_type; +struct quic_connection_info_key key; +union { +struct quic_aes_gcm_128 aes_gcm_128; +struct quic_aes_gcm_256 aes_gcm_256; +struct quic_aes_ccm_128 aes_ccm_128; +struct quic_chacha20_poly1305 chacha20_poly1305; + }; +}; + +#endif +
include/uapi/linux/udp.h +3 -0
@@ -34,6 +34,9 @@ struct udphdr {#define UDP_NO_CHECK6_RX 102/* Disable accpeting checksum for UDP6 */ #define UDP_SEGMENT 103/* Set GSO segmentation size */ #define UDP_GRO 104/* This socket can receive UDP GRO packets */ +#define UDP_QUIC_ADD_TX_CONNECTION 106/* Add QUIC Tx crypto offload */ +#define UDP_QUIC_DEL_TX_CONNECTION 107/* Del QUIC Tx crypto offload */ +#define UDP_QUIC_ENCRYPT 108/* QUIC encryption parameters */ /* UDP encapsulation types */ #define UDP_ENCAP_ESPINUDP_NON_IKE 1/* draft-ietf-ipsec-nat-t-ike-00/01 */
Signature · 1 line
2.30.2
RFC 3/6 net: Add UDP ULP operations, initialization and handling prototype functions.
In reply to Adel AbouchaevDefine functions to add UDP ULP handling, registration with UDP protocol and supporting data structures. Create structure for QUIC ULP and add empty prototype functions to support it.
--- include/net/inet_sock.h | 2 + include/net/udp.h | 33 +++++++ include/uapi/linux/udp.h | 1 + net/Kconfig | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 6 ++ net/ipv4/udp_ulp.c | 190 +++++++++++++++++++++++++++++++++++++++ 8 files changed, 236 insertions(+), 1 deletion(-) create mode 100644 net/ipv4/udp_ulp.c
include/net/inet_sock.h +2 -0
@@ -238,6 +238,8 @@ struct inet_sock {__be32 mc_addr;struct ip_mc_socklist __rcu *mc_list;struct inet_cork_full cork; +const struct udp_ulp_ops *udp_ulp_ops; +void __rcu *ulp_data; };#define IPCORK_OPT 1/* ip-options has been held in ipcork.opt */
include/net/udp.h +33 -0
@@ -523,4 +523,37 @@ struct proto *udp_bpf_get_proto(struct sock *sk, struct sk_psock *psock);int udp_bpf_update_proto (struct sock *sk,struct sk_psock *psock,bool restore);#endif +/* +* Interface for adding Upper Level Protocols over UDP +*/ + +#define UDP_ULP_NAME_MAX 16 +#define UDP_ULP_MAX 128 + +struct udp_ulp_ops { +struct list_head list; + +/* initialize ulp */ +int (*init)(struct sock *sk); +/* cleanup ulp */ +void (*release)(struct sock *sk); + +char name[UDP_ULP_NAME_MAX ]; +struct module *owner; +}; + +int udp_register_ulp (struct udp_ulp_ops *type); +void udp_unregister_ulp (struct udp_ulp_ops *type); +int udp_set_ulp (struct sock *sk,const char *name); +void udp_get_available_ulp (char *buf,size_t len); +void udp_cleanup_ulp (struct sock *sk); +int udp_setsockopt_ulp (struct sock *sk,sockptr_t optval, +unsigned int optlen); +int udp_getsockopt_ulp (struct sock *sk,char __user *optval, +int __user *optlen); + +#define MODULE_ALIAS_UDP_ULP (name)\ + __MODULE_INFO(alias, alias_userspace, name);\ + __MODULE_INFO(alias, alias_udp_ulp, "udp-ulp-" name) +#endif /* _UDP_H */
include/uapi/linux/udp.h +1 -0
@@ -34,6 +34,7 @@ struct udphdr {#define UDP_NO_CHECK6_RX 102/* Disable accpeting checksum for UDP6 */ #define UDP_SEGMENT 103/* Set GSO segmentation size */ #define UDP_GRO 104/* This socket can receive UDP GRO packets */ +#define UDP_ULP 105/* Attach ULP to a UDP socket */ #define UDP_QUIC_ADD_TX_CONNECTION 106/* Add QUIC Tx crypto offload */ #define UDP_QUIC_DEL_TX_CONNECTION 107/* Del QUIC Tx crypto offload */ #define UDP_QUIC_ENCRYPT 108/* QUIC encryption parameters */
net/Kconfig +1 -0
@@ -63,6 +63,7 @@ menu "Networking options" source "net/packet/Kconfig" source "net/unix/Kconfig" source "net/tls/Kconfig" +source "net/quic/Kconfig" source "net/xfrm/Kconfig" source "net/iucv/Kconfig" source "net/smc/Kconfig"
net/Makefile +1 -0
@@ -16,6 +16,7 @@ obj-y += ethernet/ 802/ sched/ netlink/ bpf/ ethtool/ obj-$(CONFIG_NETFILTER) += netfilter/ obj-$(CONFIG_INET) += ipv4/ obj-$(CONFIG_TLS) += tls/ +obj-$(CONFIG_QUIC) += quic/ obj-$(CONFIG_XFRM) += xfrm/ obj-$(CONFIG_UNIX_SCM) += unix/ obj-y += ipv6/
net/ipv4/Makefile +2 -1
@@ -14,7 +14,8 @@ obj-y := route.o inetpeer.o protocol.o \ udp_offload.o arp.o icmp.o devinet.o af_inet.o igmp.o \ fib_frontend.o fib_semantics.o fib_trie.o fib_notifier.o \ inet_fragment.o ping.o ip_tunnel_core.o gre_offload.o \ - metrics.o netlink.o nexthop.o udp_tunnel_stub.o + metrics.o netlink.o nexthop.o udp_tunnel_stub.o \ + udp_ulp.o obj-$(CONFIG_BPFILTER) += bpfilter/
net/ipv4/udp.c +6 -0
@@ -2778,6 +2778,9 @@ int udp_lib_setsockopt(struct sock *sk, int level, int optname, up->pcflag |=UDPLITE_RECV_CC ;break ; +case UDP_ULP : +return udp_setsockopt_ulp (sk, optval, optlen); +default : err = -ENOPROTOOPT ;break ;
@@ -2846,6 +2849,9 @@ int udp_lib_getsockopt(struct sock *sk, int level, int optname, val = up->pcrlen;break ; +case UDP_ULP : +return udp_getsockopt_ulp (sk, optval, optlen); +default :return -ENOPROTOOPT ; }
net/ipv4/udp_ulp.c new file +190 -0
190 lines not on this page · raw message
Signature · 1 line
2.30.2
RFC 4/6 net: Implement QUIC offload functions
In reply to Adel AbouchaevAdd connection hash to the context do support add, remove operations on QUIC connections for the control plane and lookup for the data plane. Implement setsockopt and add placeholders to add and delete Tx connections.
--- include/net/quic.h | 49 ++ net/ipv4/udp.c | 8 + net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1400 ++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 1481 insertions(+) create mode 100644 include/net/quic.h create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c
include/net/quic.h new file +49 -0
@@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef INCLUDE_NET_QUIC_H +#define INCLUDE_NET_QUIC_H + +#include <linux/mutex.h> +#include <linux/rhashtable.h> +#include <linux/skmsg.h> +#include <uapi/linux/quic.h> + +#define QUIC_MAX_SHORT_HEADER_SIZE 25 +#define QUIC_MAX_CONNECTION_ID_SIZE 20 +#define QUIC_HDR_MASK_SIZE 16 +#define QUIC_MAX_GSO_FRAGS 16 + +// Maximum IV and nonce sizes should be in sync with supported ciphers. +#define QUIC_CIPHER_MAX_IV_SIZE 12 +#define QUIC_CIPHER_MAX_NONCE_SIZE 16 + +/* Side by side data for QUIC egress operations */ +#define QUIC_ANCILLARY_FLAGS (QUIC_BYPASS_ENCRYPTION) + +#define QUIC_MAX_IOVEC_SEGMENTS 8 +#define QUIC_MAX_SG_ALLOC_ELEMENTS 32 +#define QUIC_MAX_PLAIN_PAGES 16 +#define QUIC_MAX_CIPHER_PAGES_ORDER 4 + +struct quic_internal_crypto_context { +struct quic_connection_info conn_info; +struct crypto_skcipher *header_tfm; +struct crypto_aead *packet_aead; +}; + +struct quic_connection_rhash { +struct rhash_head node; +struct quic_internal_crypto_context crypto_ctx; +struct rcu_head rcu; +}; + +struct quic_context { +struct proto *sk_proto; +struct rhashtable tx_connections; +struct scatterlist sg_alloc[QUIC_MAX_SG_ALLOC_ELEMENTS ]; +struct page *cipher_page; +struct mutex sendmsg_mux; +struct rcu_head rcu; +}; + +#endif
net/ipv4/udp.c +8 -0
@@ -113,6 +113,7 @@#include <net/sock_reuseport.h> #include <net/addrconf.h> #include <net/udp_tunnel.h> +#include <uapi/linux/quic.h> #if IS_ENABLED (CONFIG_IPV6 )#include <net/ipv6_stubs.h> #endif
@@ -1009,6 +1010,13 @@ static int __udp_cmsg_send(struct cmsghdr *cmsg, u16 *gso_size)return -EINVAL ; *gso_size = *(__u16 *)CMSG_DATA (cmsg);return 0 ; +case UDP_QUIC_ENCRYPT : +/* This option is handled in UDP_ULP and is only checked +* here for the bypass bit +*/ +if (cmsg->cmsg_len !=CMSG_LEN (sizeof (struct quic_tx_ancillary_data ))) +return -EINVAL ; +return 0 ;default :return -EINVAL ; }
net/quic/Kconfig new file +16 -0
@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# QUIC configuration +# +config QUIC + tristate "QUIC encryption offload" + depends on INET + select CRYPTO + select CRYPTO_AES + select CRYPTO_GCM + help + Enable kernel support for QUIC crypto offload. Currently only TX + encryption offload is supported. The kernel will perform + copy-during-encryption. + + If unsure, say N.
net/quic/Makefile new file +8 -0
@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Makefile for the QUIC subsystem +# + +obj-$(CONFIG_QUIC) += quic.o + +quic-y := quic_main.o
net/quic/quic_main.c new file +1400 -0
1,400 lines not on this page · raw message
Signature · 1 line
2.30.2
RFC 5/6 net: Add flow counters and Tx processing error counter
In reply to Adel AbouchaevAdded flow counters. Total flow counter is accumulative, the current shows the number of flows currently in flight, the error counters is accumulating the number of errors during Tx processing.
--- include/net/netns/mib.h | 3 +++ include/net/quic.h | 10 +++++++++ include/net/snmp.h | 6 +++++ include/uapi/linux/snmp.h | 11 ++++++++++ net/quic/Makefile | 2 +- net/quic/quic_main.c | 46 +++++++++++++++++++++++++++++++++++++++ net/quic/quic_proc.c | 45 ++++++++++++++++++++++++++++++++++++++ 7 files changed, 122 insertions(+), 1 deletion(-) create mode 100644 net/quic/quic_proc.c
include/net/netns/mib.h +3 -0
@@ -24,6 +24,9 @@ struct netns_mib {#if IS_ENABLED (CONFIG_TLS ) DEFINE_SNMP_STAT(struct linux_tls_mib, tls_statistics);#endif +#if IS_ENABLED (CONFIG_QUIC ) + DEFINE_SNMP_STAT(struct linux_quic_mib, quic_statistics); +#endif #ifdef CONFIG_MPTCP DEFINE_SNMP_STAT(struct mptcp_mib, mptcp_statistics);#endif
include/net/quic.h +10 -0
@@ -25,6 +25,16 @@#define QUIC_MAX_PLAIN_PAGES 16#define QUIC_MAX_CIPHER_PAGES_ORDER 4 +#define __QUIC_INC_STATS (net, field) \ + __SNMP_INC_STATS((net)->mib.quic_statistics, field) +#define QUIC_INC_STATS (net, field) \ + SNMP_INC_STATS((net)->mib.quic_statistics, field) +#define QUIC_DEC_STATS (net, field) \ + SNMP_DEC_STATS((net)->mib.quic_statistics, field) + +int __net_init quic_proc_init (struct net *net); +void __net_exit quic_proc_fini(struct net *net); +struct quic_internal_crypto_context {struct quic_connection_info conn_info;struct crypto_skcipher *header_tfm;
include/net/snmp.h +6 -0
@@ -117,6 +117,12 @@ struct linux_tls_mib {unsigned long mibs[LINUX_MIB_TLSMAX ]; }; +/* Linux QUIC */ +#define LINUX_MIB_QUICMAX __LINUX_MIB_QUICMAX +struct linux_quic_mib { +unsigned long mibs[LINUX_MIB_QUICMAX ]; +}; +#define DEFINE_SNMP_STAT (type, name) \ __typeof__(type) __percpu *name#define DEFINE_SNMP_STAT_ATOMIC (type, name) \
include/uapi/linux/snmp.h +11 -0
@@ -347,4 +347,15 @@ enum__LINUX_MIB_TLSMAX }; +/* linux QUIC mib definitions */ +enum +{ +LINUX_MIB_QUICNUM =0 , +LINUX_MIB_QUICCURRTXSW ,/* QuicCurrTxSw */ +LINUX_MIB_QUICTXSW ,/* QuicTxSw */ +LINUX_MIB_QUICTXSWERROR ,/* QuicTxSwError */ + __LINUX_MIB_QUICMAX +}; + +#endif /* _LINUX_SNMP_H */
net/quic/Makefile +1 -1
@@ -5,4 +5,4 @@ obj-$(CONFIG_QUIC) += quic.o -quic-y := quic_main.o +quic-y := quic_main.o quic_proc.o
net/quic/quic_main.c +46 -0
88 lines not on this page · raw message
net/quic/quic_proc.c new file +45 -0
45 lines not on this page · raw message
Signature · 1 line
2.30.2
RFC 6/6 net: Add self tests for ULP operations, flow setup and crypto tests
In reply to Adel AbouchaevAdd self tests for ULP operations, flow setup and crypto tests.
--- tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/Makefile | 2 +- tools/testing/selftests/net/quic.c | 1024 ++++++++++++++++++++++++ tools/testing/selftests/net/quic.sh | 45 ++ 4 files changed, 1071 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
tools/testing/selftests/net/.gitignore +1 -0
@@ -38,3 +38,4 @@ ioam6_parser toeplitz tun cmsg_sender +quic
tools/testing/selftests/net/Makefile +1 -1
@@ -54,7 +54,7 @@ TEST_GEN_FILES += ipsec TEST_GEN_FILES += ioam6_parser TEST_GEN_FILES += gro TEST_GEN_PROGS = reuseport_bpf reuseport_bpf_cpu reuseport_bpf_numa -TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls tun +TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls tun quic TEST_GEN_FILES += toeplitz TEST_GEN_FILES += cmsg_sender TEST_GEN_FILES += stress_reuseport_listen
tools/testing/selftests/net/quic.c new file +1024 -0
1,024 lines not on this page · raw message
tools/testing/selftests/net/quic.sh new file +45 -0
45 lines not on this page · raw message
Signature · 1 line
2.30.2
v2 cover net: support QUIC crypto
In reply to a message not in the archiveQUIC requires end to end encryption of the data. The application usually prepares the data in clear text, encrypts and calls send() which implies multiple copies of the data before the packets hit the networking stack. Similar to kTLS, QUIC kernel offload of cryptography reduces the memory pressure by reducing the number of copies.
The scope of kernel support is limited to the symmetric cryptography, leaving the handshake to the user space library. For QUIC in particular, the application packets that require symmetric cryptography are the 1RTT packets with short headers. Kernel will encrypt the application packets on transmission and decrypt on receive. This series implements Tx only, because in QUIC server applications Tx outweighs Rx by orders of magnitude.
Supporting the combination of QUIC and GSO requires the application to correctly place the data and the kernel to correctly slice it. The encryption process appends an arbitrary number of bytes (tag) to the end of the message to authenticate it. The GSO value should include this overhead, the offload would then subtract the tag size to parse the input on Tx before chunking and encrypting it.
With the kernel cryptography, the buffer copy operation is conjoined with the encryption operation. The memory bandwidth is reduced by 5-8%. When devices supporting QUIC encryption in hardware come to the market, we will be able to free further 7% of CPU utilization which is used today for crypto operations.
Adel Abouchaev (6):
Documentation on QUIC kernel Tx crypto.
Define QUIC specific constants, control and data plane structures Add UDP ULP operations, initialization and handling prototype
functions.
Implement QUIC offload functions
Add flow counters and Tx processing error counter
Add self tests for ULP operations, flow setup and crypto tests
v2: Moved the inner QUIC Kconfig from the ULP patch to QUIC patch.
v2: Updated the tests to match the uAPI context structure fields.
v2: Formatted the quic.rst document.
Documentation/networking/index.rst | 1 + Documentation/networking/quic.rst | 186 +++ include/net/inet_sock.h | 2 + include/net/netns/mib.h | 3 + include/net/quic.h | 59 + include/net/snmp.h | 6 + include/net/udp.h | 33 + include/uapi/linux/quic.h | 61 + include/uapi/linux/snmp.h | 11 + include/uapi/linux/udp.h | 4 + net/Kconfig | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 14 + net/ipv4/udp_ulp.c | 190 ++++ net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1446 ++++++++++++++++++++++++ net/quic/quic_proc.c | 45 + tools/testing/selftests/net/.gitignore | 3 +- tools/testing/selftests/net/Makefile | 2 +- tools/testing/selftests/net/quic.c | 1024 +++++++++++++++++ tools/testing/selftests/net/quic.sh | 45 + 23 files changed, 3161 insertions(+), 3 deletions(-) create mode 100644 Documentation/networking/quic.rst create mode 100644 include/net/quic.h create mode 100644 include/uapi/linux/quic.h create mode 100644 net/ipv4/udp_ulp.c create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c create mode 100644 net/quic/quic_proc.c create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
Signature · 1 line
2.30.2
RFC v2 1/6 Documentation on QUIC kernel Tx crypto.
Adding Documentation/networking/quic.rst file to describe kernel QUIC code.
---
v2: Added quic.rst reference to the index.rst file; identation in quic.rst file.
--- Documentation/networking/index.rst | 1 + Documentation/networking/quic.rst | 186 +++++++++++++++++++++++++++++ 2 files changed, 187 insertions(+) create mode 100644 Documentation/networking/quic.rst
Documentation/networking/index.rst +1 -0
@@ -90,6 +90,7 @@ Contents: plip ppp_generic proc_net_tcp + quic radiotap-headers rds regulatory
Documentation/networking/quic.rst new file +186 -0
186 lines not on this page · raw message
Signature · 1 line
2.30.2
Re: [RFC net-next v2 1/6] Documentation on QUIC kernel Tx crypto.
Quoted 3 lines · Adel Abouchaev
Adding
Documentation/networking/quic.rstfile to describe kernel QUIC code.
Better say "Add documentation for kernel QUIC code".
Quoted 194 lines
Documentation/networking/index.rst +1
-0@@ -90,6 +90,7 @@ Contents: plip ppp_generic proc_net_tcp + quic radiotap-headers rds regulatoryDocumentation/networking/quic.rst new file +176
-0@@ -0,0 +1,186 @@ +.. _kernel_quic: + +=========== +KERNEL QUIC +=========== + +Overview +======== + +QUIC is a secure general-purpose transport protocol that creates a stateful +interaction between a client and a server. QUIC provides end-to-end integrity +and confidentiality. Refer to RFC 9000 for more information on QUIC. + +The kernel Tx side offload covers the encryption of the application streams +in the kernel rather than in the application. These packets are 1RTT packets +in QUIC connection. Encryption of every other packets is still done by the +QUIC library in user space. + + + +User Interface +============== + +Creating a QUIC connection +-------------------------- + +QUIC connection originates and terminates in the application, using one of many +available QUIC libraries. The code instantiates QUIC client and QUIC server in +some form and configures them to use certain addresses and ports for the +source and destination. The client and server negotiate the set of keys to +protect the communication during different phases of the connection, maintain +the connection and perform congestion control. + +Requesting to add QUIC Tx kernel encryption to the connection +------------------------------------------------------------- + +Each flow that should be encrypted by the kernel needs to be registered with +the kernel using socket API. A setsockopt() call on the socket creates an +association between the QUIC connection ID of the flow with the encryption +parameters for the crypto operations: + +.. code-block:: c + + struct quic_connection_info conn_info; + char conn_id[5] = {0x01, 0x02, 0x03, 0x04, 0x05}; + const size_t conn_id_len = sizeof(conn_id); + char conn_key[16] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f}; + char conn_iv[12] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b}; + char conn_hdr_key[16] = {0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, + 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f + }; + + conn_info.cipher_type = TLS_CIPHER_AES_GCM_128; + + memset(&conn_info.key, 0, sizeof(struct quic_connection_info_key)); + conn_info.key.conn_id_length = 5; + memcpy(&conn_info.key.conn_id[QUIC_MAX_CONNECTION_ID_SIZE + - conn_id_len], + &conn_id, conn_id_len); + + memcpy(&conn_info.payload_key, conn_key, sizeof(conn_key)); + memcpy(&conn_info.payload_iv, conn_iv, sizeof(conn_iv)); + memcpy(&conn_info.header_key, conn_hdr_key, sizeof(conn_hdr_key)); + + setsockopt(fd, SOL_UDP, UDP_QUIC_ADD_TX_CONNECTION, &conn_info, + sizeof(conn_info)); + + +Requesting to remove QUIC Tx kernel crypto offload control messages +------------------------------------------------------------------- + +All flows are removed when the socket is closed. To request an explicit remove +of the offload for the connection during the lifetime of the socket the process +is similar to adding the flow. Only the connection ID and its length are +necessary to supply to remove the connection from the offload: + +.. code-block:: c + + memset(&conn_info.key, 0, sizeof(struct quic_connection_info_key)); + conn_info.key.conn_id_length = 5; + memcpy(&conn_info.key.conn_id[QUIC_MAX_CONNECTION_ID_SIZE + - conn_id_len], + &conn_id, conn_id_len); + setsockopt(fd, SOL_UDP, UDP_QUIC_DEL_TX_CONNECTION, &conn_info, + sizeof(conn_info)); + +Sending QUIC application data +----------------------------- + +For QUIC Tx encryption offload, the application should use sendmsg() socket +call and provide ancillary data with information on connection ID length and +offload flags for the kernel to perform the encryption and GSO support if +requested. + +.. code-block:: c + + size_t cmsg_tx_len = sizeof(struct quic_tx_ancillary_data); + uint8_t cmsg_buf[CMSG_SPACE(cmsg_tx_len)]; + struct quic_tx_ancillary_data * anc_data; + size_t quic_data_len = 4500; + struct cmsghdr * cmsg_hdr; + char quic_data[9000]; + struct iovec iov[2]; + int send_len = 9000; + struct msghdr msg; + int err; + + iov[0].iov_base = quic_data; + iov[0].iov_len = quic_data_len; + iov[1].iov_base = quic_data + 4500; + iov[1].iov_len = quic_data_len; + + if (client.addr.sin_family == AF_INET) { + msg.msg_name = &client.addr; + msg.msg_namelen = sizeof(client.addr); + } else { + msg.msg_name = &client.addr6; + msg.msg_namelen = sizeof(client.addr6); + } + + msg.msg_iov = iov; + msg.msg_iovlen = 2; + msg.msg_control = cmsg_buf; + msg.msg_controllen = sizeof(cmsg_buf); + cmsg_hdr = CMSG_FIRSTHDR(&msg); + cmsg_hdr->cmsg_level = IPPROTO_UDP; + cmsg_hdr->cmsg_type = UDP_QUIC_ENCRYPT; + cmsg_hdr->cmsg_len = CMSG_LEN(cmsg_tx_len); + anc_data = CMSG_DATA(cmsg_hdr); + anc_data->flags = 0; + anc_data->next_pkt_num = 0x0d65c9; + anc_data->conn_id_length = conn_id_len; + err = sendmsg(self->sfd, &msg, 0); + +QUIC Tx offload in kernel will read the data from userspace, encrypt and +copy it to the ciphertext within the same operation. + + +Sending QUIC application data with GSO +-------------------------------------- +When GSO is in use, the kernel will use the GSO fragment size as the target +for ciphertext. The packets from the user space should align on the boundary +of GSO fragment size minus the size of the tag for the chosen cipher. For the +GSO fragment 1200, the plain packets should follow each other at every 1184 +bytes, given the tag size of 16. After the encryption, the rest of the UDP +and IP stacks will follow the defined value of GSO fragment which will include +the trailing tag bytes. + +To set up GSO fragmentation: + +.. code-block:: c + + setsockopt(self->sfd, SOL_UDP, UDP_SEGMENT, &frag_size, + sizeof(frag_size)); + +If the GSO fragment size is provided in ancillary data within the sendmsg() +call, the value in ancillary data will take precedence over the segment size +provided in setsockopt to split the payload into packets. This is consistent +with the UDP stack behavior. + +Integrating to userspace QUIC libraries +--------------------------------------- + +Userspace QUIC libraries integration would depend on the implementation of the +QUIC protocol. For MVFST library, the control plane is integrated into the +handshake callbacks to properly configure the flows into the socket; and the +data plane is integrated into the methods that perform encryption and send +the packets to the batch scheduler for transmissions to the socket. + +MVFST library can be found at https://github.com/facebookincubator/mvfst. + +Statistics +========== +
Documentation/networking/quic.rst line 186
Quoted 10 lines
Documentation/networking/quic.rst new file +10
-0+QUIC Tx offload to the kernel has counters +(``/proc/net/quic_stat``): + +- ``QuicCurrTxSw`` - + number of currently active kernel offloaded QUIC connections +- ``QuicTxSw`` - + accumulative total number of offloaded QUIC connections +- ``QuicTxSwError`` - + accumulative total number of errors during QUIC Tx offload to kernel +
The documentation looks OK (no new warnings).
Thanks.
Signature · 1 line
An old man doll... just what I always wanted! - Clara
Re: [RFC net-next v2 1/6] Documentation on QUIC kernel Tx crypto.
Updated the commit message, will be visible in v3.
Quoted 214 lines · Bagas Sanjaya
Quoted 3 lines · Adel Abouchaev
Adding
Documentation/networking/quic.rstfile to describe kernel QUIC code.Better say "Add documentation for kernel QUIC code".
Quoted 194 lines
Documentation/networking/index.rst +1
-0@@ -90,6 +90,7 @@ Contents: plip ppp_generic proc_net_tcp + quic radiotap-headers rds regulatoryDocumentation/networking/quic.rst new file +176
-0@@ -0,0 +1,186 @@ +.. _kernel_quic: + +=========== +KERNEL QUIC +=========== + +Overview +======== + +QUIC is a secure general-purpose transport protocol that creates a stateful +interaction between a client and a server. QUIC provides end-to-end integrity +and confidentiality. Refer to RFC 9000 for more information on QUIC. + +The kernel Tx side offload covers the encryption of the application streams +in the kernel rather than in the application. These packets are 1RTT packets +in QUIC connection. Encryption of every other packets is still done by the +QUIC library in user space. + + + +User Interface +============== + +Creating a QUIC connection +-------------------------- + +QUIC connection originates and terminates in the application, using one of many +available QUIC libraries. The code instantiates QUIC client and QUIC server in +some form and configures them to use certain addresses and ports for the +source and destination. The client and server negotiate the set of keys to +protect the communication during different phases of the connection, maintain +the connection and perform congestion control. + +Requesting to add QUIC Tx kernel encryption to the connection +------------------------------------------------------------- + +Each flow that should be encrypted by the kernel needs to be registered with +the kernel using socket API. A setsockopt() call on the socket creates an +association between the QUIC connection ID of the flow with the encryption +parameters for the crypto operations: + +.. code-block:: c + + struct quic_connection_info conn_info; + char conn_id[5] = {0x01, 0x02, 0x03, 0x04, 0x05}; + const size_t conn_id_len = sizeof(conn_id); + char conn_key[16] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f}; + char conn_iv[12] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b}; + char conn_hdr_key[16] = {0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, + 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f + }; + + conn_info.cipher_type = TLS_CIPHER_AES_GCM_128; + + memset(&conn_info.key, 0, sizeof(struct quic_connection_info_key)); + conn_info.key.conn_id_length = 5; + memcpy(&conn_info.key.conn_id[QUIC_MAX_CONNECTION_ID_SIZE + - conn_id_len], + &conn_id, conn_id_len); + + memcpy(&conn_info.payload_key, conn_key, sizeof(conn_key)); + memcpy(&conn_info.payload_iv, conn_iv, sizeof(conn_iv)); + memcpy(&conn_info.header_key, conn_hdr_key, sizeof(conn_hdr_key)); + + setsockopt(fd, SOL_UDP, UDP_QUIC_ADD_TX_CONNECTION, &conn_info, + sizeof(conn_info)); + + +Requesting to remove QUIC Tx kernel crypto offload control messages +------------------------------------------------------------------- + +All flows are removed when the socket is closed. To request an explicit remove +of the offload for the connection during the lifetime of the socket the process +is similar to adding the flow. Only the connection ID and its length are +necessary to supply to remove the connection from the offload: + +.. code-block:: c + + memset(&conn_info.key, 0, sizeof(struct quic_connection_info_key)); + conn_info.key.conn_id_length = 5; + memcpy(&conn_info.key.conn_id[QUIC_MAX_CONNECTION_ID_SIZE + - conn_id_len], + &conn_id, conn_id_len); + setsockopt(fd, SOL_UDP, UDP_QUIC_DEL_TX_CONNECTION, &conn_info, + sizeof(conn_info)); + +Sending QUIC application data +----------------------------- + +For QUIC Tx encryption offload, the application should use sendmsg() socket +call and provide ancillary data with information on connection ID length and +offload flags for the kernel to perform the encryption and GSO support if +requested. + +.. code-block:: c + + size_t cmsg_tx_len = sizeof(struct quic_tx_ancillary_data); + uint8_t cmsg_buf[CMSG_SPACE(cmsg_tx_len)]; + struct quic_tx_ancillary_data * anc_data; + size_t quic_data_len = 4500; + struct cmsghdr * cmsg_hdr; + char quic_data[9000]; + struct iovec iov[2]; + int send_len = 9000; + struct msghdr msg; + int err; + + iov[0].iov_base = quic_data; + iov[0].iov_len = quic_data_len; + iov[1].iov_base = quic_data + 4500; + iov[1].iov_len = quic_data_len; + + if (client.addr.sin_family == AF_INET) { + msg.msg_name = &client.addr; + msg.msg_namelen = sizeof(client.addr); + } else { + msg.msg_name = &client.addr6; + msg.msg_namelen = sizeof(client.addr6); + } + + msg.msg_iov = iov; + msg.msg_iovlen = 2; + msg.msg_control = cmsg_buf; + msg.msg_controllen = sizeof(cmsg_buf); + cmsg_hdr = CMSG_FIRSTHDR(&msg); + cmsg_hdr->cmsg_level = IPPROTO_UDP; + cmsg_hdr->cmsg_type = UDP_QUIC_ENCRYPT; + cmsg_hdr->cmsg_len = CMSG_LEN(cmsg_tx_len); + anc_data = CMSG_DATA(cmsg_hdr); + anc_data->flags = 0; + anc_data->next_pkt_num = 0x0d65c9; + anc_data->conn_id_length = conn_id_len; + err = sendmsg(self->sfd, &msg, 0); + +QUIC Tx offload in kernel will read the data from userspace, encrypt and +copy it to the ciphertext within the same operation. + + +Sending QUIC application data with GSO +-------------------------------------- +When GSO is in use, the kernel will use the GSO fragment size as the target +for ciphertext. The packets from the user space should align on the boundary +of GSO fragment size minus the size of the tag for the chosen cipher. For the +GSO fragment 1200, the plain packets should follow each other at every 1184 +bytes, given the tag size of 16. After the encryption, the rest of the UDP +and IP stacks will follow the defined value of GSO fragment which will include +the trailing tag bytes. + +To set up GSO fragmentation: + +.. code-block:: c + + setsockopt(self->sfd, SOL_UDP, UDP_SEGMENT, &frag_size, + sizeof(frag_size)); + +If the GSO fragment size is provided in ancillary data within the sendmsg() +call, the value in ancillary data will take precedence over the segment size +provided in setsockopt to split the payload into packets. This is consistent +with the UDP stack behavior. + +Integrating to userspace QUIC libraries +--------------------------------------- + +Userspace QUIC libraries integration would depend on the implementation of the +QUIC protocol. For MVFST library, the control plane is integrated into the +handshake callbacks to properly configure the flows into the socket; and the +data plane is integrated into the methods that perform encryption and send +the packets to the batch scheduler for transmissions to the socket. + +MVFST library can be found at https://github.com/facebookincubator/mvfst. + +Statistics +========== +Documentation/networking/quic.rst line 186
Quoted 10 lines
Documentation/networking/quic.rst new file +10
-0+QUIC Tx offload to the kernel has counters +(``/proc/net/quic_stat``): + +- ``QuicCurrTxSw`` - + number of currently active kernel offloaded QUIC connections +- ``QuicTxSw`` - + accumulative total number of offloaded QUIC connections +- ``QuicTxSwError`` - + accumulative total number of errors during QUIC Tx offload to kernel +The documentation looks OK (no new warnings).
Thanks.
RFC v2 2/6 Define QUIC specific constants, control and data plane structures
In reply to Adel AbouchaevDefine control and data plane structures to pass in control plane for flow add/remove and during packet send within ancillary data. Define constants to use within SOL_UDP to program QUIC sockets.
--- include/uapi/linux/quic.h | 61 +++++++++++++++++++++++++++++++++++++++ include/uapi/linux/udp.h | 3 ++ 2 files changed, 64 insertions(+) create mode 100644 include/uapi/linux/quic.h
include/uapi/linux/quic.h new file +61 -0
@@ -0,0 +1,61 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) */ + +#ifndef _UAPI_LINUX_QUIC_H +#define _UAPI_LINUX_QUIC_H + +#include <linux/types.h> +#include <linux/tls.h> + +#define QUIC_MAX_CONNECTION_ID_SIZE 20 + +/* Side by side data for QUIC egress operations */ +#define QUIC_BYPASS_ENCRYPTION 0x01 + +struct quic_tx_ancillary_data { +__aligned_u64 next_pkt_num; +__u8 flags; +__u8 conn_id_length; +}; + +struct quic_connection_info_key { +__u8 conn_id[QUIC_MAX_CONNECTION_ID_SIZE ]; +__u8 conn_id_length; +}; + +struct quic_aes_gcm_128 { +__u8 header_key[TLS_CIPHER_AES_GCM_128_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_GCM_128_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_GCM_128_IV_SIZE ]; +}; + +struct quic_aes_gcm_256 { +__u8 header_key[TLS_CIPHER_AES_GCM_256_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_GCM_256_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_GCM_256_IV_SIZE ]; +}; + +struct quic_aes_ccm_128 { +__u8 header_key[TLS_CIPHER_AES_CCM_128_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_CCM_128_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_CCM_128_IV_SIZE ]; +}; + +struct quic_chacha20_poly1305 { +__u8 header_key[TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_CHACHA20_POLY1305_IV_SIZE ]; +}; + +struct quic_connection_info { +__u16 cipher_type; +struct quic_connection_info_key key; +union { +struct quic_aes_gcm_128 aes_gcm_128; +struct quic_aes_gcm_256 aes_gcm_256; +struct quic_aes_ccm_128 aes_ccm_128; +struct quic_chacha20_poly1305 chacha20_poly1305; + }; +}; + +#endif +
include/uapi/linux/udp.h +3 -0
@@ -34,6 +34,9 @@ struct udphdr {#define UDP_NO_CHECK6_RX 102/* Disable accpeting checksum for UDP6 */ #define UDP_SEGMENT 103/* Set GSO segmentation size */ #define UDP_GRO 104/* This socket can receive UDP GRO packets */ +#define UDP_QUIC_ADD_TX_CONNECTION 106/* Add QUIC Tx crypto offload */ +#define UDP_QUIC_DEL_TX_CONNECTION 107/* Del QUIC Tx crypto offload */ +#define UDP_QUIC_ENCRYPT 108/* QUIC encryption parameters */ /* UDP encapsulation types */ #define UDP_ENCAP_ESPINUDP_NON_IKE 1/* draft-ietf-ipsec-nat-t-ike-00/01 */
Signature · 1 line
2.30.2
RFC v2 3/6 Add UDP ULP operations, initialization and handling prototype functions.
In reply to Adel AbouchaevDefine functions to add UDP ULP handling, registration with UDP protocol and supporting data structures. Create structure for QUIC ULP and add empty prototype functions to support it.
---
v2: Removed reference to net/quic/Kconfig from this patch into the next.
--- include/net/inet_sock.h | 2 + include/net/udp.h | 33 +++++++ include/uapi/linux/udp.h | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 6 ++ net/ipv4/udp_ulp.c | 190 +++++++++++++++++++++++++++++++++++++++ 7 files changed, 235 insertions(+), 1 deletion(-) create mode 100644 net/ipv4/udp_ulp.c
include/net/inet_sock.h +2 -0
@@ -249,6 +249,8 @@ struct inet_sock {__be32 mc_addr;struct ip_mc_socklist __rcu *mc_list;struct inet_cork_full cork; +const struct udp_ulp_ops *udp_ulp_ops; +void __rcu *ulp_data; };#define IPCORK_OPT 1/* ip-options has been held in ipcork.opt */
include/net/udp.h +33 -0
@@ -523,4 +523,37 @@ struct proto *udp_bpf_get_proto(struct sock *sk, struct sk_psock *psock);int udp_bpf_update_proto (struct sock *sk,struct sk_psock *psock,bool restore);#endif +/* +* Interface for adding Upper Level Protocols over UDP +*/ + +#define UDP_ULP_NAME_MAX 16 +#define UDP_ULP_MAX 128 + +struct udp_ulp_ops { +struct list_head list; + +/* initialize ulp */ +int (*init)(struct sock *sk); +/* cleanup ulp */ +void (*release)(struct sock *sk); + +char name[UDP_ULP_NAME_MAX ]; +struct module *owner; +}; + +int udp_register_ulp (struct udp_ulp_ops *type); +void udp_unregister_ulp (struct udp_ulp_ops *type); +int udp_set_ulp (struct sock *sk,const char *name); +void udp_get_available_ulp (char *buf,size_t len); +void udp_cleanup_ulp (struct sock *sk); +int udp_setsockopt_ulp (struct sock *sk,sockptr_t optval, +unsigned int optlen); +int udp_getsockopt_ulp (struct sock *sk,char __user *optval, +int __user *optlen); + +#define MODULE_ALIAS_UDP_ULP (name)\ + __MODULE_INFO(alias, alias_userspace, name);\ + __MODULE_INFO(alias, alias_udp_ulp, "udp-ulp-" name) +#endif /* _UDP_H */
include/uapi/linux/udp.h +1 -0
@@ -34,6 +34,7 @@ struct udphdr {#define UDP_NO_CHECK6_RX 102/* Disable accpeting checksum for UDP6 */ #define UDP_SEGMENT 103/* Set GSO segmentation size */ #define UDP_GRO 104/* This socket can receive UDP GRO packets */ +#define UDP_ULP 105/* Attach ULP to a UDP socket */ #define UDP_QUIC_ADD_TX_CONNECTION 106/* Add QUIC Tx crypto offload */ #define UDP_QUIC_DEL_TX_CONNECTION 107/* Del QUIC Tx crypto offload */ #define UDP_QUIC_ENCRYPT 108/* QUIC encryption parameters */
net/Makefile +1 -0
@@ -16,6 +16,7 @@ obj-y += ethernet/ 802/ sched/ netlink/ bpf/ ethtool/ obj-$(CONFIG_NETFILTER) += netfilter/ obj-$(CONFIG_INET) += ipv4/ obj-$(CONFIG_TLS) += tls/ +obj-$(CONFIG_QUIC) += quic/ obj-$(CONFIG_XFRM) += xfrm/ obj-$(CONFIG_UNIX_SCM) += unix/ obj-y += ipv6/
net/ipv4/Makefile +2 -1
@@ -14,7 +14,8 @@ obj-y := route.o inetpeer.o protocol.o \ udp_offload.o arp.o icmp.o devinet.o af_inet.o igmp.o \ fib_frontend.o fib_semantics.o fib_trie.o fib_notifier.o \ inet_fragment.o ping.o ip_tunnel_core.o gre_offload.o \ - metrics.o netlink.o nexthop.o udp_tunnel_stub.o + metrics.o netlink.o nexthop.o udp_tunnel_stub.o \ + udp_ulp.o obj-$(CONFIG_BPFILTER) += bpfilter/
net/ipv4/udp.c +6 -0
@@ -2779,6 +2779,9 @@ int udp_lib_setsockopt(struct sock *sk, int level, int optname, up->pcflag |=UDPLITE_RECV_CC ;break ; +case UDP_ULP : +return udp_setsockopt_ulp (sk, optval, optlen); +default : err = -ENOPROTOOPT ;break ;
@@ -2847,6 +2850,9 @@ int udp_lib_getsockopt(struct sock *sk, int level, int optname, val = up->pcrlen;break ; +case UDP_ULP : +return udp_getsockopt_ulp (sk, optval, optlen); +default :return -ENOPROTOOPT ; }
net/ipv4/udp_ulp.c new file +190 -0
190 lines not on this page · raw message
Signature · 1 line
2.30.2
RFC v2 4/6 Implement QUIC offload functions
In reply to Adel AbouchaevAdd connection hash to the context do support add, remove operations on QUIC connections for the control plane and lookup for the data plane. Implement setsockopt and add placeholders to add and delete Tx connections.
---
v2: Added net/quic/Kconfig reference to net/Kconfig in this commit.
--- include/net/quic.h | 49 ++ net/Kconfig | 1 + net/ipv4/udp.c | 8 + net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1400 ++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 1482 insertions(+) create mode 100644 include/net/quic.h create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c
include/net/quic.h new file +49 -0
@@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef INCLUDE_NET_QUIC_H +#define INCLUDE_NET_QUIC_H + +#include <linux/mutex.h> +#include <linux/rhashtable.h> +#include <linux/skmsg.h> +#include <uapi/linux/quic.h> + +#define QUIC_MAX_SHORT_HEADER_SIZE 25 +#define QUIC_MAX_CONNECTION_ID_SIZE 20 +#define QUIC_HDR_MASK_SIZE 16 +#define QUIC_MAX_GSO_FRAGS 16 + +// Maximum IV and nonce sizes should be in sync with supported ciphers. +#define QUIC_CIPHER_MAX_IV_SIZE 12 +#define QUIC_CIPHER_MAX_NONCE_SIZE 16 + +/* Side by side data for QUIC egress operations */ +#define QUIC_ANCILLARY_FLAGS (QUIC_BYPASS_ENCRYPTION) + +#define QUIC_MAX_IOVEC_SEGMENTS 8 +#define QUIC_MAX_SG_ALLOC_ELEMENTS 32 +#define QUIC_MAX_PLAIN_PAGES 16 +#define QUIC_MAX_CIPHER_PAGES_ORDER 4 + +struct quic_internal_crypto_context { +struct quic_connection_info conn_info; +struct crypto_skcipher *header_tfm; +struct crypto_aead *packet_aead; +}; + +struct quic_connection_rhash { +struct rhash_head node; +struct quic_internal_crypto_context crypto_ctx; +struct rcu_head rcu; +}; + +struct quic_context { +struct proto *sk_proto; +struct rhashtable tx_connections; +struct scatterlist sg_alloc[QUIC_MAX_SG_ALLOC_ELEMENTS ]; +struct page *cipher_page; +struct mutex sendmsg_mux; +struct rcu_head rcu; +}; + +#endif
net/Kconfig +1 -0
@@ -63,6 +63,7 @@ menu "Networking options" source "net/packet/Kconfig" source "net/unix/Kconfig" source "net/tls/Kconfig" +source "net/quic/Kconfig" source "net/xfrm/Kconfig" source "net/iucv/Kconfig" source "net/smc/Kconfig"
net/ipv4/udp.c +8 -0
@@ -113,6 +113,7 @@#include <net/sock_reuseport.h> #include <net/addrconf.h> #include <net/udp_tunnel.h> +#include <uapi/linux/quic.h> #if IS_ENABLED (CONFIG_IPV6 )#include <net/ipv6_stubs.h> #endif
@@ -1011,6 +1012,13 @@ static int __udp_cmsg_send(struct cmsghdr *cmsg, u16 *gso_size)return -EINVAL ; *gso_size = *(__u16 *)CMSG_DATA (cmsg);return 0 ; +case UDP_QUIC_ENCRYPT : +/* This option is handled in UDP_ULP and is only checked +* here for the bypass bit +*/ +if (cmsg->cmsg_len !=CMSG_LEN (sizeof (struct quic_tx_ancillary_data ))) +return -EINVAL ; +return 0 ;default :return -EINVAL ; }
net/quic/Kconfig new file +16 -0
@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# QUIC configuration +# +config QUIC + tristate "QUIC encryption offload" + depends on INET + select CRYPTO + select CRYPTO_AES + select CRYPTO_GCM + help + Enable kernel support for QUIC crypto offload. Currently only TX + encryption offload is supported. The kernel will perform + copy-during-encryption. + + If unsure, say N.
net/quic/Makefile new file +8 -0
@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Makefile for the QUIC subsystem +# + +obj-$(CONFIG_QUIC) += quic.o + +quic-y := quic_main.o
net/quic/quic_main.c new file +1400 -0
1,400 lines not on this page · raw message
Signature · 1 line
2.30.2
RFC v2 5/6 Add flow counters and Tx processing error counter
In reply to Adel AbouchaevAdded flow counters. Total flow counter is accumulative, the current shows the number of flows currently in flight, the error counters is accumulating the number of errors during Tx processing.
--- include/net/netns/mib.h | 3 +++ include/net/quic.h | 10 +++++++++ include/net/snmp.h | 6 +++++ include/uapi/linux/snmp.h | 11 ++++++++++ net/quic/Makefile | 2 +- net/quic/quic_main.c | 46 +++++++++++++++++++++++++++++++++++++++ net/quic/quic_proc.c | 45 ++++++++++++++++++++++++++++++++++++++ 7 files changed, 122 insertions(+), 1 deletion(-) create mode 100644 net/quic/quic_proc.c
include/net/netns/mib.h +3 -0
@@ -24,6 +24,9 @@ struct netns_mib {#if IS_ENABLED (CONFIG_TLS ) DEFINE_SNMP_STAT(struct linux_tls_mib, tls_statistics);#endif +#if IS_ENABLED (CONFIG_QUIC ) + DEFINE_SNMP_STAT(struct linux_quic_mib, quic_statistics); +#endif #ifdef CONFIG_MPTCP DEFINE_SNMP_STAT(struct mptcp_mib, mptcp_statistics);#endif
include/net/quic.h +10 -0
@@ -25,6 +25,16 @@#define QUIC_MAX_PLAIN_PAGES 16#define QUIC_MAX_CIPHER_PAGES_ORDER 4 +#define __QUIC_INC_STATS (net, field) \ + __SNMP_INC_STATS((net)->mib.quic_statistics, field) +#define QUIC_INC_STATS (net, field) \ + SNMP_INC_STATS((net)->mib.quic_statistics, field) +#define QUIC_DEC_STATS (net, field) \ + SNMP_DEC_STATS((net)->mib.quic_statistics, field) + +int __net_init quic_proc_init (struct net *net); +void __net_exit quic_proc_fini(struct net *net); +struct quic_internal_crypto_context {struct quic_connection_info conn_info;struct crypto_skcipher *header_tfm;
include/net/snmp.h +6 -0
@@ -117,6 +117,12 @@ struct linux_tls_mib {unsigned long mibs[LINUX_MIB_TLSMAX ]; }; +/* Linux QUIC */ +#define LINUX_MIB_QUICMAX __LINUX_MIB_QUICMAX +struct linux_quic_mib { +unsigned long mibs[LINUX_MIB_QUICMAX ]; +}; +#define DEFINE_SNMP_STAT (type, name) \ __typeof__(type) __percpu *name#define DEFINE_SNMP_STAT_ATOMIC (type, name) \
include/uapi/linux/snmp.h +11 -0
@@ -349,4 +349,15 @@ enum__LINUX_MIB_TLSMAX }; +/* linux QUIC mib definitions */ +enum +{ +LINUX_MIB_QUICNUM =0 , +LINUX_MIB_QUICCURRTXSW ,/* QuicCurrTxSw */ +LINUX_MIB_QUICTXSW ,/* QuicTxSw */ +LINUX_MIB_QUICTXSWERROR ,/* QuicTxSwError */ + __LINUX_MIB_QUICMAX +}; + +#endif /* _LINUX_SNMP_H */
net/quic/Makefile +1 -1
@@ -5,4 +5,4 @@ obj-$(CONFIG_QUIC) += quic.o -quic-y := quic_main.o +quic-y := quic_main.o quic_proc.o
net/quic/quic_main.c +46 -0
88 lines not on this page · raw message
net/quic/quic_proc.c new file +45 -0
45 lines not on this page · raw message
Signature · 1 line
2.30.2
RFC v2 6/6 Add self tests for ULP operations, flow setup and crypto tests
In reply to Adel AbouchaevAdd self tests for ULP operations, flow setup and crypto tests.
---
v2: Restored the test build. Changed the QUIC context reference variable names for the keys and iv to match the uAPI.
--- tools/testing/selftests/net/.gitignore | 3 +- tools/testing/selftests/net/Makefile | 2 +- tools/testing/selftests/net/quic.c | 1024 ++++++++++++++++++++++++ tools/testing/selftests/net/quic.sh | 45 ++ 4 files changed, 1072 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
tools/testing/selftests/net/.gitignore +2 -1
@@ -38,4 +38,5 @@ ioam6_parser toeplitz tun cmsg_sender -unix_connect +unix_connect +quic
tools/testing/selftests/net/Makefile +1 -1
@@ -57,7 +57,7 @@ TEST_GEN_FILES += ipsec TEST_GEN_FILES += ioam6_parser TEST_GEN_FILES += gro TEST_GEN_PROGS = reuseport_bpf reuseport_bpf_cpu reuseport_bpf_numa -TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls tun +TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls tun quic TEST_GEN_FILES += toeplitz TEST_GEN_FILES += cmsg_sender TEST_GEN_FILES += stress_reuseport_listen
tools/testing/selftests/net/quic.c new file +1024 -0
1,024 lines not on this page · raw message
tools/testing/selftests/net/quic.sh new file +45 -0
45 lines not on this page · raw message
Signature · 1 line
2.30.2
cover net: support QUIC crypto
In reply to a message not in the archiveQUIC requires end to end encryption of the data. The application usually prepares the data in clear text, encrypts and calls send() which implies multiple copies of the data before the packets hit the networking stack. Similar to kTLS, QUIC kernel offload of cryptography reduces the memory pressure by reducing the number of copies.
The scope of kernel support is limited to the symmetric cryptography, leaving the handshake to the user space library. For QUIC in particular, the application packets that require symmetric cryptography are the 1RTT packets with short headers. Kernel will encrypt the application packets on transmission and decrypt on receive. This series implements Tx only, because in QUIC server applications Tx outweighs Rx by orders of magnitude.
Supporting the combination of QUIC and GSO requires the application to correctly place the data and the kernel to correctly slice it. The encryption process appends an arbitrary number of bytes (tag) to the end of the message to authenticate it. The GSO value should include this overhead, the offload would then subtract the tag size to parse the input on Tx before chunking and encrypting it.
With the kernel cryptography, the buffer copy operation is conjoined with the encryption operation. The memory bandwidth is reduced by 5-8%. When devices supporting QUIC encryption in hardware come to the market, we will be able to free further 7% of CPU utilization which is used today for crypto operations.
*** BLURB HERE ***
Adel Abouchaev (6):
Documentation on QUIC kernel Tx crypto.
Define QUIC specific constants, control and data plane structures Add UDP ULP operations, initialization and handling prototype
functions.
Implement QUIC offload functions
Add flow counters and Tx processing error counter
Add self tests for ULP operations, flow setup and crypto tests
Documentation/networking/index.rst | 1 + Documentation/networking/quic.rst | 186 ++++ include/net/inet_sock.h | 2 + include/net/netns/mib.h | 3 + include/net/quic.h | 63 ++ include/net/snmp.h | 6 + include/net/udp.h | 33 + include/uapi/linux/quic.h | 61 + include/uapi/linux/snmp.h | 9 + include/uapi/linux/udp.h | 4 + net/Kconfig | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 15 + net/ipv4/udp_ulp.c | 192 ++++ net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1417 ++++++++++++++++++++++++ net/quic/quic_proc.c | 45 + tools/testing/selftests/net/.gitignore | 3 +- tools/testing/selftests/net/Makefile | 3 +- tools/testing/selftests/net/quic.c | 1153 +++++++++++++++++++ tools/testing/selftests/net/quic.sh | 46 + 23 files changed, 3268 insertions(+), 3 deletions(-) create mode 100644 Documentation/networking/quic.rst create mode 100644 include/net/quic.h create mode 100644 include/uapi/linux/quic.h create mode 100644 net/ipv4/udp_ulp.c create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c create mode 100644 net/quic/quic_proc.c create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
--
2.30.2
PATCH 1/6 Documentation on QUIC kernel Tx crypto.
Add documentation for kernel QUIC code.
---
Added quic.rst reference to the index.rst file; identation in quic.rst file.
Added SPDX license GPL 2.0.
--- Documentation/networking/index.rst | 1 + Documentation/networking/quic.rst | 186 +++++++++++++++++++++++++++++ 2 files changed, 187 insertions(+) create mode 100644 Documentation/networking/quic.rst
Documentation/networking/index.rst +1 -0
@@ -90,6 +90,7 @@ Contents: plip ppp_generic proc_net_tcp + quic radiotap-headers rds regulatory
Documentation/networking/quic.rst new file +186 -0
186 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH 2/6 Define QUIC specific constants, control and data plane structures
In reply to Adel AbouchaevDefine control and data plane structures to pass in control plane for flow add/remove and during packet send within ancillary data. Define constants to use within SOL_UDP to program QUIC sockets.
--- include/uapi/linux/quic.h | 61 +++++++++++++++++++++++++++++++++++++++ include/uapi/linux/udp.h | 3 ++ 2 files changed, 64 insertions(+) create mode 100644 include/uapi/linux/quic.h
include/uapi/linux/quic.h new file +61 -0
@@ -0,0 +1,61 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) */ + +#ifndef _UAPI_LINUX_QUIC_H +#define _UAPI_LINUX_QUIC_H + +#include <linux/types.h> +#include <linux/tls.h> + +#define QUIC_MAX_CONNECTION_ID_SIZE 20 + +/* Side by side data for QUIC egress operations */ +#define QUIC_BYPASS_ENCRYPTION 0x01 + +struct quic_tx_ancillary_data { +__aligned_u64 next_pkt_num; +__u8 flags; +__u8 conn_id_length; +}; + +struct quic_connection_info_key { +__u8 conn_id[QUIC_MAX_CONNECTION_ID_SIZE ]; +__u8 conn_id_length; +}; + +struct quic_aes_gcm_128 { +__u8 header_key[TLS_CIPHER_AES_GCM_128_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_GCM_128_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_GCM_128_IV_SIZE ]; +}; + +struct quic_aes_gcm_256 { +__u8 header_key[TLS_CIPHER_AES_GCM_256_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_GCM_256_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_GCM_256_IV_SIZE ]; +}; + +struct quic_aes_ccm_128 { +__u8 header_key[TLS_CIPHER_AES_CCM_128_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_CCM_128_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_CCM_128_IV_SIZE ]; +}; + +struct quic_chacha20_poly1305 { +__u8 header_key[TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_CHACHA20_POLY1305_IV_SIZE ]; +}; + +struct quic_connection_info { +__u16 cipher_type; +struct quic_connection_info_key key; +union { +struct quic_aes_gcm_128 aes_gcm_128; +struct quic_aes_gcm_256 aes_gcm_256; +struct quic_aes_ccm_128 aes_ccm_128; +struct quic_chacha20_poly1305 chacha20_poly1305; + }; +}; + +#endif +
include/uapi/linux/udp.h +3 -0
@@ -34,6 +34,9 @@ struct udphdr {#define UDP_NO_CHECK6_RX 102/* Disable accpeting checksum for UDP6 */ #define UDP_SEGMENT 103/* Set GSO segmentation size */ #define UDP_GRO 104/* This socket can receive UDP GRO packets */ +#define UDP_QUIC_ADD_TX_CONNECTION 106/* Add QUIC Tx crypto offload */ +#define UDP_QUIC_DEL_TX_CONNECTION 107/* Del QUIC Tx crypto offload */ +#define UDP_QUIC_ENCRYPT 108/* QUIC encryption parameters */ /* UDP encapsulation types */ #define UDP_ENCAP_ESPINUDP_NON_IKE 1/* draft-ietf-ipsec-nat-t-ike-00/01 */
Signature · 1 line
2.30.2
PATCH 3/6 Add UDP ULP operations, initialization and handling prototype functions.
In reply to Adel AbouchaevDefine functions to add UDP ULP handling, registration with UDP protocol and supporting data structures. Create structure for QUIC ULP and add empty prototype functions to support it.
---
Removed reference to net/quic/Kconfig from this patch into the next.
Fixed formatting around brackets.
--- include/net/inet_sock.h | 2 + include/net/udp.h | 33 +++++++ include/uapi/linux/udp.h | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 6 ++ net/ipv4/udp_ulp.c | 192 +++++++++++++++++++++++++++++++++++++++ 7 files changed, 237 insertions(+), 1 deletion(-) create mode 100644 net/ipv4/udp_ulp.c
include/net/inet_sock.h +2 -0
@@ -249,6 +249,8 @@ struct inet_sock {__be32 mc_addr;struct ip_mc_socklist __rcu *mc_list;struct inet_cork_full cork; +const struct udp_ulp_ops *udp_ulp_ops; +void __rcu *ulp_data; };#define IPCORK_OPT 1/* ip-options has been held in ipcork.opt */
include/net/udp.h +33 -0
@@ -523,4 +523,37 @@ struct proto *udp_bpf_get_proto(struct sock *sk, struct sk_psock *psock);int udp_bpf_update_proto (struct sock *sk,struct sk_psock *psock,bool restore);#endif +/* +* Interface for adding Upper Level Protocols over UDP +*/ + +#define UDP_ULP_NAME_MAX 16 +#define UDP_ULP_MAX 128 + +struct udp_ulp_ops { +struct list_head list; + +/* initialize ulp */ +int (*init)(struct sock *sk); +/* cleanup ulp */ +void (*release)(struct sock *sk); + +char name[UDP_ULP_NAME_MAX ]; +struct module *owner; +}; + +int udp_register_ulp (struct udp_ulp_ops *type); +void udp_unregister_ulp (struct udp_ulp_ops *type); +int udp_set_ulp (struct sock *sk,const char *name); +void udp_get_available_ulp (char *buf,size_t len); +void udp_cleanup_ulp (struct sock *sk); +int udp_setsockopt_ulp (struct sock *sk,sockptr_t optval, +unsigned int optlen); +int udp_getsockopt_ulp (struct sock *sk,char __user *optval, +int __user *optlen); + +#define MODULE_ALIAS_UDP_ULP (name)\ + __MODULE_INFO(alias, alias_userspace, name);\ + __MODULE_INFO(alias, alias_udp_ulp, "udp-ulp-" name) +#endif /* _UDP_H */
include/uapi/linux/udp.h +1 -0
@@ -34,6 +34,7 @@ struct udphdr {#define UDP_NO_CHECK6_RX 102/* Disable accpeting checksum for UDP6 */ #define UDP_SEGMENT 103/* Set GSO segmentation size */ #define UDP_GRO 104/* This socket can receive UDP GRO packets */ +#define UDP_ULP 105/* Attach ULP to a UDP socket */ #define UDP_QUIC_ADD_TX_CONNECTION 106/* Add QUIC Tx crypto offload */ #define UDP_QUIC_DEL_TX_CONNECTION 107/* Del QUIC Tx crypto offload */ #define UDP_QUIC_ENCRYPT 108/* QUIC encryption parameters */
net/Makefile +1 -0
@@ -16,6 +16,7 @@ obj-y += ethernet/ 802/ sched/ netlink/ bpf/ ethtool/ obj-$(CONFIG_NETFILTER) += netfilter/ obj-$(CONFIG_INET) += ipv4/ obj-$(CONFIG_TLS) += tls/ +obj-$(CONFIG_QUIC) += quic/ obj-$(CONFIG_XFRM) += xfrm/ obj-$(CONFIG_UNIX_SCM) += unix/ obj-y += ipv6/
net/ipv4/Makefile +2 -1
@@ -14,7 +14,8 @@ obj-y := route.o inetpeer.o protocol.o \ udp_offload.o arp.o icmp.o devinet.o af_inet.o igmp.o \ fib_frontend.o fib_semantics.o fib_trie.o fib_notifier.o \ inet_fragment.o ping.o ip_tunnel_core.o gre_offload.o \ - metrics.o netlink.o nexthop.o udp_tunnel_stub.o + metrics.o netlink.o nexthop.o udp_tunnel_stub.o \ + udp_ulp.o obj-$(CONFIG_BPFILTER) += bpfilter/
net/ipv4/udp.c +6 -0
@@ -2779,6 +2779,9 @@ int udp_lib_setsockopt(struct sock *sk, int level, int optname, up->pcflag |=UDPLITE_RECV_CC ;break ; +case UDP_ULP : +return udp_setsockopt_ulp (sk, optval, optlen); +default : err = -ENOPROTOOPT ;break ;
@@ -2847,6 +2850,9 @@ int udp_lib_getsockopt(struct sock *sk, int level, int optname, val = up->pcrlen;break ; +case UDP_ULP : +return udp_getsockopt_ulp (sk, optval, optlen); +default :return -ENOPROTOOPT ; }
net/ipv4/udp_ulp.c new file +192 -0
192 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH 4/6 Implement QUIC offload functions
In reply to Adel AbouchaevAdd connection hash to the context do support add, remove operations on QUIC connections for the control plane and lookup for the data plane. Implement setsockopt and add placeholders to add and delete Tx connections.
---
Added net/quic/Kconfig reference to net/Kconfig in this commit.
Initialized pointers with NULL vs 0. Restricted AES counter to __le32 Added address space qualifiers to user space addresses. Removed empty lines. Updated code alignment. Removed inlines.
v3: removed ITER_KVEC flag from iov_iter_kvec call.
v3: fixed Chacha20 encryption bug.
--- include/net/quic.h | 53 ++ net/Kconfig | 1 + net/ipv4/udp.c | 9 + net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1371 ++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 1458 insertions(+) create mode 100644 include/net/quic.h create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c
include/net/quic.h new file +53 -0
@@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef INCLUDE_NET_QUIC_H +#define INCLUDE_NET_QUIC_H + +#include <linux/mutex.h> +#include <linux/rhashtable.h> +#include <linux/skmsg.h> +#include <uapi/linux/quic.h> + +#define QUIC_MAX_SHORT_HEADER_SIZE 25 +#define QUIC_MAX_CONNECTION_ID_SIZE 20 +#define QUIC_HDR_MASK_SIZE 16 +#define QUIC_MAX_GSO_FRAGS 16 + +// Maximum IV and nonce sizes should be in sync with supported ciphers. +#define QUIC_CIPHER_MAX_IV_SIZE 12 +#define QUIC_CIPHER_MAX_NONCE_SIZE 16 + +/* Side by side data for QUIC egress operations */ +#define QUIC_ANCILLARY_FLAGS (QUIC_BYPASS_ENCRYPTION) + +#define QUIC_MAX_IOVEC_SEGMENTS 8 +#define QUIC_MAX_SG_ALLOC_ELEMENTS 32 +#define QUIC_MAX_PLAIN_PAGES 16 +#define QUIC_MAX_CIPHER_PAGES_ORDER 4 + +struct quic_internal_crypto_context { +struct quic_connection_info conn_info; +struct crypto_skcipher *header_tfm; +struct crypto_aead *packet_aead; +}; + +struct quic_connection_rhash { +struct rhash_head node; +struct quic_internal_crypto_context crypto_ctx; +struct rcu_head rcu; +}; + +struct quic_context { +struct proto *sk_proto; +struct rhashtable tx_connections; +struct scatterlist sg_alloc[QUIC_MAX_SG_ALLOC_ELEMENTS ]; +struct page *cipher_page; +/** +* To synchronize concurrent sendmsg() requests through the same socket +* and protect preallocated per-context memory. +**/ +struct mutex sendmsg_mux; +struct rcu_head rcu; +}; + +#endif
net/Kconfig +1 -0
@@ -63,6 +63,7 @@ menu "Networking options" source "net/packet/Kconfig" source "net/unix/Kconfig" source "net/tls/Kconfig" +source "net/quic/Kconfig" source "net/xfrm/Kconfig" source "net/iucv/Kconfig" source "net/smc/Kconfig"
net/ipv4/udp.c +9 -0
@@ -113,6 +113,7 @@#include <net/sock_reuseport.h> #include <net/addrconf.h> #include <net/udp_tunnel.h> +#include <uapi/linux/quic.h> #if IS_ENABLED (CONFIG_IPV6 )#include <net/ipv6_stubs.h> #endif
@@ -1011,6 +1012,14 @@ static int __udp_cmsg_send(struct cmsghdr *cmsg, u16 *gso_size)return -EINVAL ; *gso_size = *(__u16 *)CMSG_DATA (cmsg);return 0 ; +case UDP_QUIC_ENCRYPT : +/* This option is handled in UDP_ULP and is only checked +* here for the bypass bit +*/ +if (cmsg->cmsg_len != +CMSG_LEN (sizeof (struct quic_tx_ancillary_data ))) +return -EINVAL ; +return 0 ;default :return -EINVAL ; }
net/quic/Kconfig new file +16 -0
@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# QUIC configuration +# +config QUIC + tristate "QUIC encryption offload" + depends on INET + select CRYPTO + select CRYPTO_AES + select CRYPTO_GCM + help + Enable kernel support for QUIC crypto offload. Currently only TX + encryption offload is supported. The kernel will perform + copy-during-encryption. + + If unsure, say N.
net/quic/Makefile new file +8 -0
@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Makefile for the QUIC subsystem +# + +obj-$(CONFIG_QUIC) += quic.o + +quic-y := quic_main.o
net/quic/quic_main.c new file +1371 -0
1,371 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH 5/6 Add flow counters and Tx processing error counter
In reply to Adel AbouchaevAdded flow counters. Total flow counter is accumulative, the current shows the number of flows currently in flight, the error counters is accumulating the number of errors during Tx processing.
---
Updated enum bracket to follow enum keyword. Removed extra blank lines.
--- include/net/netns/mib.h | 3 +++ include/net/quic.h | 10 +++++++++ include/net/snmp.h | 6 +++++ include/uapi/linux/snmp.h | 9 ++++++++ net/quic/Makefile | 2 +- net/quic/quic_main.c | 46 +++++++++++++++++++++++++++++++++++++++ net/quic/quic_proc.c | 45 ++++++++++++++++++++++++++++++++++++++ 7 files changed, 120 insertions(+), 1 deletion(-) create mode 100644 net/quic/quic_proc.c
include/net/netns/mib.h +3 -0
@@ -24,6 +24,9 @@ struct netns_mib {#if IS_ENABLED (CONFIG_TLS ) DEFINE_SNMP_STAT(struct linux_tls_mib, tls_statistics);#endif +#if IS_ENABLED (CONFIG_QUIC ) + DEFINE_SNMP_STAT(struct linux_quic_mib, quic_statistics); +#endif #ifdef CONFIG_MPTCP DEFINE_SNMP_STAT(struct mptcp_mib, mptcp_statistics);#endif
include/net/quic.h +10 -0
@@ -25,6 +25,16 @@#define QUIC_MAX_PLAIN_PAGES 16#define QUIC_MAX_CIPHER_PAGES_ORDER 4 +#define __QUIC_INC_STATS (net, field) \ + __SNMP_INC_STATS((net)->mib.quic_statistics, field) +#define QUIC_INC_STATS (net, field) \ + SNMP_INC_STATS((net)->mib.quic_statistics, field) +#define QUIC_DEC_STATS (net, field) \ + SNMP_DEC_STATS((net)->mib.quic_statistics, field) + +int __net_init quic_proc_init (struct net *net); +void __net_exit quic_proc_fini(struct net *net); +struct quic_internal_crypto_context {struct quic_connection_info conn_info;struct crypto_skcipher *header_tfm;
include/net/snmp.h +6 -0
@@ -117,6 +117,12 @@ struct linux_tls_mib {unsigned long mibs[LINUX_MIB_TLSMAX ]; }; +/* Linux QUIC */ +#define LINUX_MIB_QUICMAX __LINUX_MIB_QUICMAX +struct linux_quic_mib { +unsigned long mibs[LINUX_MIB_QUICMAX ]; +}; +#define DEFINE_SNMP_STAT (type, name) \ __typeof__(type) __percpu *name#define DEFINE_SNMP_STAT_ATOMIC (type, name) \
include/uapi/linux/snmp.h +9 -0
@@ -349,4 +349,13 @@ enum__LINUX_MIB_TLSMAX }; +/* linux QUIC mib definitions */ +enum { +LINUX_MIB_QUICNUM =0 , +LINUX_MIB_QUICCURRTXSW ,/* QuicCurrTxSw */ +LINUX_MIB_QUICTXSW ,/* QuicTxSw */ +LINUX_MIB_QUICTXSWERROR ,/* QuicTxSwError */ + __LINUX_MIB_QUICMAX +}; +#endif /* _LINUX_SNMP_H */
net/quic/Makefile +1 -1
@@ -5,4 +5,4 @@ obj-$(CONFIG_QUIC) += quic.o -quic-y := quic_main.o +quic-y := quic_main.o quic_proc.o
net/quic/quic_main.c +46 -0
88 lines not on this page · raw message
net/quic/quic_proc.c new file +45 -0
45 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH 6/6 Add self tests for ULP operations, flow setup and crypto tests
In reply to Adel AbouchaevAdd self tests for ULP operations, flow setup and crypto tests.
---
Restored the test build. Changed the QUIC context reference variable names for the keys and iv to match the uAPI.
Updated alignment, added SPDX license line.
v3: Added Chacha20-Poly1305 test.
--- tools/testing/selftests/net/.gitignore | 3 +- tools/testing/selftests/net/Makefile | 3 +- tools/testing/selftests/net/quic.c | 1153 ++++++++++++++++++++++++ tools/testing/selftests/net/quic.sh | 46 + 4 files changed, 1203 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
tools/testing/selftests/net/.gitignore +2 -1
@@ -38,4 +38,5 @@ ioam6_parser toeplitz tun cmsg_sender -unix_connect +unix_connect +quic
tools/testing/selftests/net/Makefile +2 -1
@@ -42,6 +42,7 @@ TEST_PROGS += arp_ndisc_evict_nocarrier.sh TEST_PROGS += ndisc_unsolicited_na_test.sh TEST_PROGS += arp_ndisc_untracked_subnets.sh TEST_PROGS += stress_reuseport_listen.sh +TEST_PROGS += quic.sh TEST_PROGS_EXTENDED := in_netns.sh setup_loopback.sh setup_veth.sh TEST_PROGS_EXTENDED += toeplitz_client.sh toeplitz.sh TEST_GEN_FILES = socket nettest
@@ -57,7 +58,7 @@ TEST_GEN_FILES += ipsec TEST_GEN_FILES += ioam6_parser TEST_GEN_FILES += gro TEST_GEN_PROGS = reuseport_bpf reuseport_bpf_cpu reuseport_bpf_numa -TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls tun +TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls tun quic TEST_GEN_FILES += toeplitz TEST_GEN_FILES += cmsg_sender TEST_GEN_FILES += stress_reuseport_listen
tools/testing/selftests/net/quic.c new file +1153 -0
1,153 lines not on this page · raw message
tools/testing/selftests/net/quic.sh new file +46 -0
46 lines not on this page · raw message
Signature · 1 line
2.30.2
Quoted 21 lines · Adel Abouchaev
QUIC requires end to end encryption of the data. The application usually prepares the data in clear text, encrypts and calls
send()which implies multiple copies of the data before the packets hit the networking stack. Similar to kTLS, QUIC kernel offload of cryptography reduces the memory pressure by reducing the number of copies.The scope of kernel support is limited to the symmetric cryptography, leaving the handshake to the user space library. For QUIC in particular, the application packets that require symmetric cryptography are the 1RTT packets with short headers. Kernel will encrypt the application packets on transmission and decrypt on receive. This series implements Tx only, because in QUIC server applications Tx outweighs Rx by orders of magnitude.
Supporting the combination of QUIC and GSO requires the application to correctly place the data and the kernel to correctly slice it. The encryption process appends an arbitrary number of bytes (tag) to the end of the message to authenticate it. The GSO value should include this overhead, the offload would then subtract the tag size to parse the input on Tx before chunking and encrypting it.
Quoted 6 lines
With the kernel cryptography, the buffer copy operation is conjoined with the encryption operation. The memory bandwidth is reduced by 5-8%. When devices supporting QUIC encryption in hardware come to the market, we will be able to free further 7% of CPU utilization which is used today for crypto operations.
Hmmm...
I can't cleanly applied this series on top of current net-next. Exactly on what commit this series is based on?
Also, I see two whitespace warnings when applying. Please fixup and resend. When resending, don't forget to pass --base to git-format-patch(1).
Thanks.
Signature · 1 line
An old man doll... just what I always wanted! - Clara
The base commit for the branch I am using here is:
commit f86d1fbbe7858884d6754534a0afbb74fc30bc26 (origin/net-next-upstream, net-next/master, net-next/main, net-next)
Merge: 526942b8134c 7c6327c77d50
Author: Linus Torvalds <[email protected]>
Date: Wed Aug 3 16:29:08 2022 -0700
Will fix the whitespaces and resubmit.
Quoted 38 lines · Bagas Sanjaya
Quoted 21 lines · Adel Abouchaev
QUIC requires end to end encryption of the data. The application usually prepares the data in clear text, encrypts and calls
send()which implies multiple copies of the data before the packets hit the networking stack. Similar to kTLS, QUIC kernel offload of cryptography reduces the memory pressure by reducing the number of copies.The scope of kernel support is limited to the symmetric cryptography, leaving the handshake to the user space library. For QUIC in particular, the application packets that require symmetric cryptography are the 1RTT packets with short headers. Kernel will encrypt the application packets on transmission and decrypt on receive. This series implements Tx only, because in QUIC server applications Tx outweighs Rx by orders of magnitude.
Supporting the combination of QUIC and GSO requires the application to correctly place the data and the kernel to correctly slice it. The encryption process appends an arbitrary number of bytes (tag) to the end of the message to authenticate it. The GSO value should include this overhead, the offload would then subtract the tag size to parse the input on Tx before chunking and encrypting it.
Quoted 6 lines
With the kernel cryptography, the buffer copy operation is conjoined with the encryption operation. The memory bandwidth is reduced by 5-8%. When devices supporting QUIC encryption in hardware come to the market, we will be able to free further 7% of CPU utilization which is used today for crypto operations.
Hmmm...
I can't cleanly applied this series on top of current net-next. Exactly on what commit this series is based on?
Also, I see two whitespace warnings when applying. Please fixup and resend. When resending, don't forget to pass --base to git-format-patch(1).
Thanks.
v2 cover net: support QUIC crypto
In reply to a message not in the archiveQUIC requires end to end encryption of the data. The application usually prepares the data in clear text, encrypts and calls send() which implies multiple copies of the data before the packets hit the networking stack. Similar to kTLS, QUIC kernel offload of cryptography reduces the memory pressure by reducing the number of copies.
The scope of kernel support is limited to the symmetric cryptography, leaving the handshake to the user space library. For QUIC in particular, the application packets that require symmetric cryptography are the 1RTT packets with short headers. Kernel will encrypt the application packets on transmission and decrypt on receive. This series implements Tx only, because in QUIC server applications Tx outweighs Rx by orders of magnitude.
Supporting the combination of QUIC and GSO requires the application to correctly place the data and the kernel to correctly slice it. The encryption process appends an arbitrary number of bytes (tag) to the end of the message to authenticate it. The GSO value should include this overhead, the offload would then subtract the tag size to parse the input on Tx before chunking and encrypting it.
With the kernel cryptography, the buffer copy operation is conjoined with the encryption operation. The memory bandwidth is reduced by 5-8%. When devices supporting QUIC encryption in hardware come to the market, we will be able to free further 7% of CPU utilization which is used today for crypto operations.
Adel Abouchaev (6):
Documentation on QUIC kernel Tx crypto.
Define QUIC specific constants, control and data plane structures Add UDP ULP operations, initialization and handling prototype
functions.
Implement QUIC offload functions
Add flow counters and Tx processing error counter
Add self tests for ULP operations, flow setup and crypto tests
Documentation/networking/index.rst | 1 + Documentation/networking/quic.rst | 185 ++++ include/net/inet_sock.h | 2 + include/net/netns/mib.h | 3 + include/net/quic.h | 63 ++ include/net/snmp.h | 6 + include/net/udp.h | 33 + include/uapi/linux/quic.h | 60 + include/uapi/linux/snmp.h | 9 + include/uapi/linux/udp.h | 4 + net/Kconfig | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 15 + net/ipv4/udp_ulp.c | 192 ++++ net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1417 ++++++++++++++++++++++++ net/quic/quic_proc.c | 45 + tools/testing/selftests/net/.gitignore | 4 +- tools/testing/selftests/net/Makefile | 3 +- tools/testing/selftests/net/quic.c | 1153 +++++++++++++++++++ tools/testing/selftests/net/quic.sh | 46 + 23 files changed, 3267 insertions(+), 3 deletions(-) create mode 100644 Documentation/networking/quic.rst create mode 100644 include/net/quic.h create mode 100644 include/uapi/linux/quic.h create mode 100644 net/ipv4/udp_ulp.c create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c create mode 100644 net/quic/quic_proc.c create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
base-commit: fd78d07c7c35de260eb89f1be4a1e7487b8092ad
Signature · 1 line
2.30.2
PATCH v2 1/6 Documentation on QUIC kernel Tx crypto.
Add documentation for kernel QUIC code.
---
Added quic.rst reference to the index.rst file; identation in quic.rst file.
Added SPDX license GPL 2.0.
v2: Removed whitespace at EOF.
--- Documentation/networking/index.rst | 1 + Documentation/networking/quic.rst | 185 +++++++++++++++++++++++++++++ 2 files changed, 186 insertions(+) create mode 100644 Documentation/networking/quic.rst
Documentation/networking/index.rst +1 -0
@@ -90,6 +90,7 @@ Contents: plip ppp_generic proc_net_tcp + quic radiotap-headers rds regulatory
Documentation/networking/quic.rst new file +185 -0
185 lines not on this page · raw message
Signature · 1 line
2.30.2
Re: [net-next v2 1/6] Documentation on QUIC kernel Tx crypto.
Quoted 2 lines · Adel Abouchaev
Add documentation for kernel QUIC code.
The documentation LGTM.
Signature · 1 line
An old man doll... just what I always wanted! - Clara
PATCH v2 2/6 Define QUIC specific constants, control and data plane structures
In reply to Adel AbouchaevDefine control and data plane structures to pass in control plane for flow add/remove and during packet send within ancillary data. Define constants to use within SOL_UDP to program QUIC sockets.
--- include/uapi/linux/quic.h | 60 +++++++++++++++++++++++++++++++++++++++ include/uapi/linux/udp.h | 3 ++ 2 files changed, 63 insertions(+) create mode 100644 include/uapi/linux/quic.h
include/uapi/linux/quic.h new file +60 -0
@@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) */ + +#ifndef _UAPI_LINUX_QUIC_H +#define _UAPI_LINUX_QUIC_H + +#include <linux/types.h> +#include <linux/tls.h> + +#define QUIC_MAX_CONNECTION_ID_SIZE 20 + +/* Side by side data for QUIC egress operations */ +#define QUIC_BYPASS_ENCRYPTION 0x01 + +struct quic_tx_ancillary_data { +__aligned_u64 next_pkt_num; +__u8 flags; +__u8 conn_id_length; +}; + +struct quic_connection_info_key { +__u8 conn_id[QUIC_MAX_CONNECTION_ID_SIZE ]; +__u8 conn_id_length; +}; + +struct quic_aes_gcm_128 { +__u8 header_key[TLS_CIPHER_AES_GCM_128_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_GCM_128_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_GCM_128_IV_SIZE ]; +}; + +struct quic_aes_gcm_256 { +__u8 header_key[TLS_CIPHER_AES_GCM_256_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_GCM_256_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_GCM_256_IV_SIZE ]; +}; + +struct quic_aes_ccm_128 { +__u8 header_key[TLS_CIPHER_AES_CCM_128_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_CCM_128_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_CCM_128_IV_SIZE ]; +}; + +struct quic_chacha20_poly1305 { +__u8 header_key[TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_CHACHA20_POLY1305_IV_SIZE ]; +}; + +struct quic_connection_info { +__u16 cipher_type; +struct quic_connection_info_key key; +union { +struct quic_aes_gcm_128 aes_gcm_128; +struct quic_aes_gcm_256 aes_gcm_256; +struct quic_aes_ccm_128 aes_ccm_128; +struct quic_chacha20_poly1305 chacha20_poly1305; + }; +}; + +#endif
include/uapi/linux/udp.h +3 -0
@@ -34,6 +34,9 @@ struct udphdr {#define UDP_NO_CHECK6_RX 102/* Disable accpeting checksum for UDP6 */ #define UDP_SEGMENT 103/* Set GSO segmentation size */ #define UDP_GRO 104/* This socket can receive UDP GRO packets */ +#define UDP_QUIC_ADD_TX_CONNECTION 106/* Add QUIC Tx crypto offload */ +#define UDP_QUIC_DEL_TX_CONNECTION 107/* Del QUIC Tx crypto offload */ +#define UDP_QUIC_ENCRYPT 108/* QUIC encryption parameters */ /* UDP encapsulation types */ #define UDP_ENCAP_ESPINUDP_NON_IKE 1/* draft-ietf-ipsec-nat-t-ike-00/01 */
Signature · 1 line
2.30.2
PATCH v2 3/6 Add UDP ULP operations, initialization and handling prototype functions.
In reply to Adel AbouchaevDefine functions to add UDP ULP handling, registration with UDP protocol and supporting data structures. Create structure for QUIC ULP and add empty prototype functions to support it.
---
Removed reference to net/quic/Kconfig from this patch into the next.
Fixed formatting around brackets.
--- include/net/inet_sock.h | 2 + include/net/udp.h | 33 +++++++ include/uapi/linux/udp.h | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 6 ++ net/ipv4/udp_ulp.c | 192 +++++++++++++++++++++++++++++++++++++++ 7 files changed, 237 insertions(+), 1 deletion(-) create mode 100644 net/ipv4/udp_ulp.c
include/net/inet_sock.h +2 -0
@@ -249,6 +249,8 @@ struct inet_sock {__be32 mc_addr;struct ip_mc_socklist __rcu *mc_list;struct inet_cork_full cork; +const struct udp_ulp_ops *udp_ulp_ops; +void __rcu *ulp_data; };#define IPCORK_OPT 1/* ip-options has been held in ipcork.opt */
include/net/udp.h +33 -0
@@ -523,4 +523,37 @@ struct proto *udp_bpf_get_proto(struct sock *sk, struct sk_psock *psock);int udp_bpf_update_proto (struct sock *sk,struct sk_psock *psock,bool restore);#endif +/* +* Interface for adding Upper Level Protocols over UDP +*/ + +#define UDP_ULP_NAME_MAX 16 +#define UDP_ULP_MAX 128 + +struct udp_ulp_ops { +struct list_head list; + +/* initialize ulp */ +int (*init)(struct sock *sk); +/* cleanup ulp */ +void (*release)(struct sock *sk); + +char name[UDP_ULP_NAME_MAX ]; +struct module *owner; +}; + +int udp_register_ulp (struct udp_ulp_ops *type); +void udp_unregister_ulp (struct udp_ulp_ops *type); +int udp_set_ulp (struct sock *sk,const char *name); +void udp_get_available_ulp (char *buf,size_t len); +void udp_cleanup_ulp (struct sock *sk); +int udp_setsockopt_ulp (struct sock *sk,sockptr_t optval, +unsigned int optlen); +int udp_getsockopt_ulp (struct sock *sk,char __user *optval, +int __user *optlen); + +#define MODULE_ALIAS_UDP_ULP (name)\ + __MODULE_INFO(alias, alias_userspace, name);\ + __MODULE_INFO(alias, alias_udp_ulp, "udp-ulp-" name) +#endif /* _UDP_H */
include/uapi/linux/udp.h +1 -0
@@ -34,6 +34,7 @@ struct udphdr {#define UDP_NO_CHECK6_RX 102/* Disable accpeting checksum for UDP6 */ #define UDP_SEGMENT 103/* Set GSO segmentation size */ #define UDP_GRO 104/* This socket can receive UDP GRO packets */ +#define UDP_ULP 105/* Attach ULP to a UDP socket */ #define UDP_QUIC_ADD_TX_CONNECTION 106/* Add QUIC Tx crypto offload */ #define UDP_QUIC_DEL_TX_CONNECTION 107/* Del QUIC Tx crypto offload */ #define UDP_QUIC_ENCRYPT 108/* QUIC encryption parameters */
net/Makefile +1 -0
@@ -16,6 +16,7 @@ obj-y += ethernet/ 802/ sched/ netlink/ bpf/ ethtool/ obj-$(CONFIG_NETFILTER) += netfilter/ obj-$(CONFIG_INET) += ipv4/ obj-$(CONFIG_TLS) += tls/ +obj-$(CONFIG_QUIC) += quic/ obj-$(CONFIG_XFRM) += xfrm/ obj-$(CONFIG_UNIX_SCM) += unix/ obj-y += ipv6/
net/ipv4/Makefile +2 -1
@@ -14,7 +14,8 @@ obj-y := route.o inetpeer.o protocol.o \ udp_offload.o arp.o icmp.o devinet.o af_inet.o igmp.o \ fib_frontend.o fib_semantics.o fib_trie.o fib_notifier.o \ inet_fragment.o ping.o ip_tunnel_core.o gre_offload.o \ - metrics.o netlink.o nexthop.o udp_tunnel_stub.o + metrics.o netlink.o nexthop.o udp_tunnel_stub.o \ + udp_ulp.o obj-$(CONFIG_BPFILTER) += bpfilter/
net/ipv4/udp.c +6 -0
@@ -2779,6 +2779,9 @@ int udp_lib_setsockopt(struct sock *sk, int level, int optname, up->pcflag |=UDPLITE_RECV_CC ;break ; +case UDP_ULP : +return udp_setsockopt_ulp (sk, optval, optlen); +default : err = -ENOPROTOOPT ;break ;
@@ -2847,6 +2850,9 @@ int udp_lib_getsockopt(struct sock *sk, int level, int optname, val = up->pcrlen;break ; +case UDP_ULP : +return udp_getsockopt_ulp (sk, optval, optlen); +default :return -ENOPROTOOPT ; }
net/ipv4/udp_ulp.c new file +192 -0
192 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH v2 4/6 Implement QUIC offload functions
In reply to Adel AbouchaevAdd connection hash to the context do support add, remove operations on QUIC connections for the control plane and lookup for the data plane. Implement setsockopt and add placeholders to add and delete Tx connections.
---
Added net/quic/Kconfig reference to net/Kconfig in this commit.
Initialized pointers with NULL vs 0. Restricted AES counter to __le32 Added address space qualifiers to user space addresses. Removed empty lines. Updated code alignment. Removed inlines.
v3: removed ITER_KVEC flag from iov_iter_kvec call.
v3: fixed Chacha20 encryption bug.
--- include/net/quic.h | 53 ++ net/Kconfig | 1 + net/ipv4/udp.c | 9 + net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1371 ++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 1458 insertions(+) create mode 100644 include/net/quic.h create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c
include/net/quic.h new file +53 -0
@@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef INCLUDE_NET_QUIC_H +#define INCLUDE_NET_QUIC_H + +#include <linux/mutex.h> +#include <linux/rhashtable.h> +#include <linux/skmsg.h> +#include <uapi/linux/quic.h> + +#define QUIC_MAX_SHORT_HEADER_SIZE 25 +#define QUIC_MAX_CONNECTION_ID_SIZE 20 +#define QUIC_HDR_MASK_SIZE 16 +#define QUIC_MAX_GSO_FRAGS 16 + +// Maximum IV and nonce sizes should be in sync with supported ciphers. +#define QUIC_CIPHER_MAX_IV_SIZE 12 +#define QUIC_CIPHER_MAX_NONCE_SIZE 16 + +/* Side by side data for QUIC egress operations */ +#define QUIC_ANCILLARY_FLAGS (QUIC_BYPASS_ENCRYPTION) + +#define QUIC_MAX_IOVEC_SEGMENTS 8 +#define QUIC_MAX_SG_ALLOC_ELEMENTS 32 +#define QUIC_MAX_PLAIN_PAGES 16 +#define QUIC_MAX_CIPHER_PAGES_ORDER 4 + +struct quic_internal_crypto_context { +struct quic_connection_info conn_info; +struct crypto_skcipher *header_tfm; +struct crypto_aead *packet_aead; +}; + +struct quic_connection_rhash { +struct rhash_head node; +struct quic_internal_crypto_context crypto_ctx; +struct rcu_head rcu; +}; + +struct quic_context { +struct proto *sk_proto; +struct rhashtable tx_connections; +struct scatterlist sg_alloc[QUIC_MAX_SG_ALLOC_ELEMENTS ]; +struct page *cipher_page; +/** +* To synchronize concurrent sendmsg() requests through the same socket +* and protect preallocated per-context memory. +**/ +struct mutex sendmsg_mux; +struct rcu_head rcu; +}; + +#endif
net/Kconfig +1 -0
@@ -63,6 +63,7 @@ menu "Networking options" source "net/packet/Kconfig" source "net/unix/Kconfig" source "net/tls/Kconfig" +source "net/quic/Kconfig" source "net/xfrm/Kconfig" source "net/iucv/Kconfig" source "net/smc/Kconfig"
net/ipv4/udp.c +9 -0
@@ -113,6 +113,7 @@#include <net/sock_reuseport.h> #include <net/addrconf.h> #include <net/udp_tunnel.h> +#include <uapi/linux/quic.h> #if IS_ENABLED (CONFIG_IPV6 )#include <net/ipv6_stubs.h> #endif
@@ -1011,6 +1012,14 @@ static int __udp_cmsg_send(struct cmsghdr *cmsg, u16 *gso_size)return -EINVAL ; *gso_size = *(__u16 *)CMSG_DATA (cmsg);return 0 ; +case UDP_QUIC_ENCRYPT : +/* This option is handled in UDP_ULP and is only checked +* here for the bypass bit +*/ +if (cmsg->cmsg_len != +CMSG_LEN (sizeof (struct quic_tx_ancillary_data ))) +return -EINVAL ; +return 0 ;default :return -EINVAL ; }
net/quic/Kconfig new file +16 -0
@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# QUIC configuration +# +config QUIC + tristate "QUIC encryption offload" + depends on INET + select CRYPTO + select CRYPTO_AES + select CRYPTO_GCM + help + Enable kernel support for QUIC crypto offload. Currently only TX + encryption offload is supported. The kernel will perform + copy-during-encryption. + + If unsure, say N.
net/quic/Makefile new file +8 -0
@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Makefile for the QUIC subsystem +# + +obj-$(CONFIG_QUIC) += quic.o + +quic-y := quic_main.o
net/quic/quic_main.c new file +1371 -0
1,371 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH v2 5/6 Add flow counters and Tx processing error counter
In reply to Adel AbouchaevAdded flow counters. Total flow counter is accumulative, the current shows the number of flows currently in flight, the error counters is accumulating the number of errors during Tx processing.
---
Updated enum bracket to follow enum keyword. Removed extra blank lines.
--- include/net/netns/mib.h | 3 +++ include/net/quic.h | 10 +++++++++ include/net/snmp.h | 6 +++++ include/uapi/linux/snmp.h | 9 ++++++++ net/quic/Makefile | 2 +- net/quic/quic_main.c | 46 +++++++++++++++++++++++++++++++++++++++ net/quic/quic_proc.c | 45 ++++++++++++++++++++++++++++++++++++++ 7 files changed, 120 insertions(+), 1 deletion(-) create mode 100644 net/quic/quic_proc.c
include/net/netns/mib.h +3 -0
@@ -24,6 +24,9 @@ struct netns_mib {#if IS_ENABLED (CONFIG_TLS ) DEFINE_SNMP_STAT(struct linux_tls_mib, tls_statistics);#endif +#if IS_ENABLED (CONFIG_QUIC ) + DEFINE_SNMP_STAT(struct linux_quic_mib, quic_statistics); +#endif #ifdef CONFIG_MPTCP DEFINE_SNMP_STAT(struct mptcp_mib, mptcp_statistics);#endif
include/net/quic.h +10 -0
@@ -25,6 +25,16 @@#define QUIC_MAX_PLAIN_PAGES 16#define QUIC_MAX_CIPHER_PAGES_ORDER 4 +#define __QUIC_INC_STATS (net, field) \ + __SNMP_INC_STATS((net)->mib.quic_statistics, field) +#define QUIC_INC_STATS (net, field) \ + SNMP_INC_STATS((net)->mib.quic_statistics, field) +#define QUIC_DEC_STATS (net, field) \ + SNMP_DEC_STATS((net)->mib.quic_statistics, field) + +int __net_init quic_proc_init (struct net *net); +void __net_exit quic_proc_fini(struct net *net); +struct quic_internal_crypto_context {struct quic_connection_info conn_info;struct crypto_skcipher *header_tfm;
include/net/snmp.h +6 -0
@@ -117,6 +117,12 @@ struct linux_tls_mib {unsigned long mibs[LINUX_MIB_TLSMAX ]; }; +/* Linux QUIC */ +#define LINUX_MIB_QUICMAX __LINUX_MIB_QUICMAX +struct linux_quic_mib { +unsigned long mibs[LINUX_MIB_QUICMAX ]; +}; +#define DEFINE_SNMP_STAT (type, name) \ __typeof__(type) __percpu *name#define DEFINE_SNMP_STAT_ATOMIC (type, name) \
include/uapi/linux/snmp.h +9 -0
@@ -349,4 +349,13 @@ enum__LINUX_MIB_TLSMAX }; +/* linux QUIC mib definitions */ +enum { +LINUX_MIB_QUICNUM =0 , +LINUX_MIB_QUICCURRTXSW ,/* QuicCurrTxSw */ +LINUX_MIB_QUICTXSW ,/* QuicTxSw */ +LINUX_MIB_QUICTXSWERROR ,/* QuicTxSwError */ + __LINUX_MIB_QUICMAX +}; +#endif /* _LINUX_SNMP_H */
net/quic/Makefile +1 -1
@@ -5,4 +5,4 @@ obj-$(CONFIG_QUIC) += quic.o -quic-y := quic_main.o +quic-y := quic_main.o quic_proc.o
net/quic/quic_main.c +46 -0
88 lines not on this page · raw message
net/quic/quic_proc.c new file +45 -0
45 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH v2 6/6 Add self tests for ULP operations, flow setup and crypto tests
In reply to Adel AbouchaevAdd self tests for ULP operations, flow setup and crypto tests.
---
Restored the test build. Changed the QUIC context reference variable names for the keys and iv to match the uAPI.
Updated alignment, added SPDX license line.
v3: Added Chacha20-Poly1305 test.
--- tools/testing/selftests/net/.gitignore | 4 +- tools/testing/selftests/net/Makefile | 3 +- tools/testing/selftests/net/quic.c | 1153 ++++++++++++++++++++++++ tools/testing/selftests/net/quic.sh | 46 + 4 files changed, 1204 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
tools/testing/selftests/net/.gitignore +3 -1
@@ -39,4 +39,6 @@ toeplitz tun cmsg_sender unix_connect -tap +tap +unix_connect +quic
tools/testing/selftests/net/Makefile +2 -1
@@ -42,6 +42,7 @@ TEST_PROGS += arp_ndisc_evict_nocarrier.sh TEST_PROGS += ndisc_unsolicited_na_test.sh TEST_PROGS += arp_ndisc_untracked_subnets.sh TEST_PROGS += stress_reuseport_listen.sh +TEST_PROGS += quic.sh TEST_PROGS_EXTENDED := in_netns.sh setup_loopback.sh setup_veth.sh TEST_PROGS_EXTENDED += toeplitz_client.sh toeplitz.sh TEST_GEN_FILES = socket nettest
@@ -57,7 +58,7 @@ TEST_GEN_FILES += ipsec TEST_GEN_FILES += ioam6_parser TEST_GEN_FILES += gro TEST_GEN_PROGS = reuseport_bpf reuseport_bpf_cpu reuseport_bpf_numa -TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls tun tap +TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls tun tap quic TEST_GEN_FILES += toeplitz TEST_GEN_FILES += cmsg_sender TEST_GEN_FILES += stress_reuseport_listen
tools/testing/selftests/net/quic.c new file +1153 -0
1,153 lines not on this page · raw message
tools/testing/selftests/net/quic.sh new file +46 -0
46 lines not on this page · raw message
Signature · 1 line
2.30.2
Bagas Sanjaya
AppliedQuoted 72 lines · Adel Abouchaev
QUIC requires end to end encryption of the data. The application usually prepares the data in clear text, encrypts and calls
send()which implies multiple copies of the data before the packets hit the networking stack. Similar to kTLS, QUIC kernel offload of cryptography reduces the memory pressure by reducing the number of copies.The scope of kernel support is limited to the symmetric cryptography, leaving the handshake to the user space library. For QUIC in particular, the application packets that require symmetric cryptography are the 1RTT packets with short headers. Kernel will encrypt the application packets on transmission and decrypt on receive. This series implements Tx only, because in QUIC server applications Tx outweighs Rx by orders of magnitude.
Supporting the combination of QUIC and GSO requires the application to correctly place the data and the kernel to correctly slice it. The encryption process appends an arbitrary number of bytes (tag) to the end of the message to authenticate it. The GSO value should include this overhead, the offload would then subtract the tag size to parse the input on Tx before chunking and encrypting it.
With the kernel cryptography, the buffer copy operation is conjoined with the encryption operation. The memory bandwidth is reduced by 5-8%. When devices supporting QUIC encryption in hardware come to the market, we will be able to free further 7% of CPU utilization which is used today for crypto operations.
Adel Abouchaev (6):
Documentation on QUIC kernel Tx crypto.
Define QUIC specific constants, control and data plane structures Add UDP ULP operations, initialization and handling prototype
functions.
Implement QUIC offload functions
Add flow counters and Tx processing error counter
Add self tests for ULP operations, flow setup and crypto testsDocumentation/networking/index.rst | 1 + Documentation/networking/quic.rst | 185 ++++ include/net/inet_sock.h | 2 + include/net/netns/mib.h | 3 + include/net/quic.h | 63 ++ include/net/snmp.h | 6 + include/net/udp.h | 33 + include/uapi/linux/quic.h | 60 + include/uapi/linux/snmp.h | 9 + include/uapi/linux/udp.h | 4 + net/Kconfig | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 15 + net/ipv4/udp_ulp.c | 192 ++++ net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1417 ++++++++++++++++++++++++ net/quic/quic_proc.c | 45 + tools/testing/selftests/net/.gitignore | 4 +- tools/testing/selftests/net/Makefile | 3 +- tools/testing/selftests/net/quic.c | 1153 +++++++++++++++++++ tools/testing/selftests/net/quic.sh | 46 + 23 files changed, 3267 insertions(+), 3 deletions(-) create mode 100644 Documentation/networking/quic.rst create mode 100644 include/net/quic.h create mode 100644 include/uapi/linux/quic.h create mode 100644 net/ipv4/udp_ulp.c create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c create mode 100644 net/quic/quic_proc.c create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
Quoted 1 line
base-commit:
fd78d07c7c35de260eb89f1be4a1e7487b8092ad
Applied, but based on f86d1fbbe78588 ("Merge tag 'net-next-6.0' ofgit://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next") instead, since this series fails to apply on the specified base-commit tag.
Thanks.
Signature · 1 line
An old man doll... just what I always wanted! - Clara
Quoted 73 lines · Adel Abouchaev
QUIC requires end to end encryption of the data. The application usually prepares the data in clear text, encrypts and calls
send()which implies multiple copies of the data before the packets hit the networking stack. Similar to kTLS, QUIC kernel offload of cryptography reduces the memory pressure by reducing the number of copies.The scope of kernel support is limited to the symmetric cryptography, leaving the handshake to the user space library. For QUIC in particular, the application packets that require symmetric cryptography are the 1RTT packets with short headers. Kernel will encrypt the application packets on transmission and decrypt on receive. This series implements Tx only, because in QUIC server applications Tx outweighs Rx by orders of magnitude.
Supporting the combination of QUIC and GSO requires the application to correctly place the data and the kernel to correctly slice it. The encryption process appends an arbitrary number of bytes (tag) to the end of the message to authenticate it. The GSO value should include this overhead, the offload would then subtract the tag size to parse the input on Tx before chunking and encrypting it.
With the kernel cryptography, the buffer copy operation is conjoined with the encryption operation. The memory bandwidth is reduced by 5-8%. When devices supporting QUIC encryption in hardware come to the market, we will be able to free further 7% of CPU utilization which is used today for crypto operations.
Adel Abouchaev (6):
Documentation on QUIC kernel Tx crypto.
Define QUIC specific constants, control and data plane structures Add UDP ULP operations, initialization and handling prototype
functions.
Implement QUIC offload functions
Add flow counters and Tx processing error counter
Add self tests for ULP operations, flow setup and crypto testsDocumentation/networking/index.rst | 1 + Documentation/networking/quic.rst | 185 ++++ include/net/inet_sock.h | 2 + include/net/netns/mib.h | 3 + include/net/quic.h | 63 ++ include/net/snmp.h | 6 + include/net/udp.h | 33 + include/uapi/linux/quic.h | 60 + include/uapi/linux/snmp.h | 9 + include/uapi/linux/udp.h | 4 + net/Kconfig | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 15 + net/ipv4/udp_ulp.c | 192 ++++ net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1417 ++++++++++++++++++++++++ net/quic/quic_proc.c | 45 + tools/testing/selftests/net/.gitignore | 4 +- tools/testing/selftests/net/Makefile | 3 +- tools/testing/selftests/net/quic.c | 1153 +++++++++++++++++++ tools/testing/selftests/net/quic.sh | 46 + 23 files changed, 3267 insertions(+), 3 deletions(-) create mode 100644 Documentation/networking/quic.rst create mode 100644 include/net/quic.h create mode 100644 include/uapi/linux/quic.h create mode 100644 net/ipv4/udp_ulp.c create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c create mode 100644 net/quic/quic_proc.c create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
Quoted 4 lines
base-commit:
fd78d07c7c35de260eb89f1be4a1e7487b8092ad--
2.30.2
Hi, Adel,
I don't see how the key update(rfc9001#section-6) is handled on the TX path, which is not using TLS Key update, and "Key Phase" indicates which key will be used after rekeying. Also, I think it is almost impossible to handle the peer rekeying on the RX path either based on your current model in the future.
The patch seems to get the crypto_ctx by doing a connection hash table lookup in the sendmsg(), which is not good from the performance side. One QUIC connection can go over multiple UDP sockets, but I don't think one socket can be used by multiple QUIC connections. So why not save the ctx in the socket instead?
The patch is to reduce the copying operations between user space and the kernel. I might miss something in your user space code, but the msg to send is *already packed* into the Stream Frame in user space, what's the difference if you encrypt it in userspace and then sendmsg(udp_sk) with zero-copy to the kernel.
Didn't really understand the "GSO" you mentioned, as I don't see any code about kernel GSO, I guess it's just "Fragment size", right? BTW, it‘s not common to use "//" for the kernel annotation.
I'm not sure if it's worth adding a ULP layer over UDP for this QUIC TX only. Honestly, I'm more supporting doing a full QUIC stack in the kernel independently with socket APIs to use it:
https://github.com/lxin/tls_hs.
Thanks.
Quoted 81 lines · Xin Long
Quoted 73 lines · Adel Abouchaev
QUIC requires end to end encryption of the data. The application usually prepares the data in clear text, encrypts and calls
send()which implies multiple copies of the data before the packets hit the networking stack. Similar to kTLS, QUIC kernel offload of cryptography reduces the memory pressure by reducing the number of copies.The scope of kernel support is limited to the symmetric cryptography, leaving the handshake to the user space library. For QUIC in particular, the application packets that require symmetric cryptography are the 1RTT packets with short headers. Kernel will encrypt the application packets on transmission and decrypt on receive. This series implements Tx only, because in QUIC server applications Tx outweighs Rx by orders of magnitude.
Supporting the combination of QUIC and GSO requires the application to correctly place the data and the kernel to correctly slice it. The encryption process appends an arbitrary number of bytes (tag) to the end of the message to authenticate it. The GSO value should include this overhead, the offload would then subtract the tag size to parse the input on Tx before chunking and encrypting it.
With the kernel cryptography, the buffer copy operation is conjoined with the encryption operation. The memory bandwidth is reduced by 5-8%. When devices supporting QUIC encryption in hardware come to the market, we will be able to free further 7% of CPU utilization which is used today for crypto operations.
Adel Abouchaev (6):
Documentation on QUIC kernel Tx crypto.
Define QUIC specific constants, control and data plane structures Add UDP ULP operations, initialization and handling prototype
functions.
Implement QUIC offload functions
Add flow counters and Tx processing error counter
Add self tests for ULP operations, flow setup and crypto testsDocumentation/networking/index.rst | 1 + Documentation/networking/quic.rst | 185 ++++ include/net/inet_sock.h | 2 + include/net/netns/mib.h | 3 + include/net/quic.h | 63 ++ include/net/snmp.h | 6 + include/net/udp.h | 33 + include/uapi/linux/quic.h | 60 + include/uapi/linux/snmp.h | 9 + include/uapi/linux/udp.h | 4 + net/Kconfig | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 15 + net/ipv4/udp_ulp.c | 192 ++++ net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1417 ++++++++++++++++++++++++ net/quic/quic_proc.c | 45 + tools/testing/selftests/net/.gitignore | 4 +- tools/testing/selftests/net/Makefile | 3 +- tools/testing/selftests/net/quic.c | 1153 +++++++++++++++++++ tools/testing/selftests/net/quic.sh | 46 + 23 files changed, 3267 insertions(+), 3 deletions(-)create mode 100644
Documentation/networking/quic.rstcreate mode 100644include/net/quic.h
create mode 100644include/uapi/linux/quic.hcreate mode 100644net/ipv4/udp_ulp.c
create mode 100644net/quic/Kconfig
create mode 100644net/quic/Makefile
create mode 100644net/quic/quic_main.ccreate mode 100644net/quic/quic_proc.ccreate mode 100644tools/testing/selftests/net/quic.ccreate mode 100755tools/testing/selftests/net/quic.shQuoted 4 lines
base-commit:
fd78d07c7c35de260eb89f1be4a1e7487b8092ad--
2.30.2
Quoted 7 lines
Hi, Adel,
I don't see how the key update(rfc9001#section-6) is handled on the TX path, which is not using TLS Key update, and "Key Phase" indicates which key will be used after rekeying. Also, I think it is almost impossible to handle the peer rekeying on the RX path either based on your current model in the future.
Key updates are not something that needs to be handled by the kernel in this model. I.e. a key update will be processed as normal by the userspace QUIC code and the sockets will have to be re-associated with the new keying material.
Quoted 6 lines
The patch seems to get the
crypto_ctxby doing a connection hash table lookup in thesendmsg(), which is not good from the performance side. One QUIC connection can go over multiple UDP sockets, but I don't think one socket can be used by multiple QUIC connections. So why not save the ctx in the socket instead?
There’s nothing preventing a single socket or UDP/IP tuple from being used by multiple QUIC connections. This is achievable due to both endpoints having CIDs. Note that it is not uncommon for QUIC deployments to use a single socket for all connections, rather than the TCP listen/accept model. That being said, it would be nice to be able to avoid the lookup cost when using a connected socket.
Quoted 6 lines
The patch is to reduce the copying operations between user space and the kernel. I might miss something in your user space code, but the msg to send is *already packed* into the Stream Frame in user space, what's the difference if you encrypt it in userspace and then
sendmsg(udp_sk)with zero-copy to the kernel.
I would not say that reducing copy operations is the primary goal of this work. There are already ways to achieve minimal copy operations for UDP from userspace.
Quoted 9 lines
Didn't really understand the "GSO" you mentioned, as I don't see any code about kernel GSO, I guess it's just "Fragment size", right? BTW, it‘s not common to use "//" for the kernel annotation.
I'm not sure if it's worth adding a ULP layer over UDP for this QUIC TX only. Honestly, I'm more supporting doing a full QUIC stack in the kernel independently with socket APIs to use it:
https://github.com/lxin/tls_hs.
A full QUIC stack in the kernel with associated socket APIs is solving a different problem than this work. Having an API to offload crypto operations of QUIC allows for the choice of many different QUIC implementations in userspace while potentially taking advantage of offloading the main CPU cost of an encrypted protocol.
Quoted 3 lines
Thanks.
Best,
Matt Joras
Quoted 77 lines · Xin Long
Quoted 72 lines · Adel Abouchaev
QUIC requires end to end encryption of the data. The application usually prepares the data in clear text, encrypts and calls
send()which implies multiple copies of the data before the packets hit the networking stack. Similar to kTLS, QUIC kernel offload of cryptography reduces the memory pressure by reducing the number of copies.The scope of kernel support is limited to the symmetric cryptography, leaving the handshake to the user space library. For QUIC in particular, the application packets that require symmetric cryptography are the 1RTT packets with short headers. Kernel will encrypt the application packets on transmission and decrypt on receive. This series implements Tx only, because in QUIC server applications Tx outweighs Rx by orders of magnitude.
Supporting the combination of QUIC and GSO requires the application to correctly place the data and the kernel to correctly slice it. The encryption process appends an arbitrary number of bytes (tag) to the end of the message to authenticate it. The GSO value should include this overhead, the offload would then subtract the tag size to parse the input on Tx before chunking and encrypting it.
With the kernel cryptography, the buffer copy operation is conjoined with the encryption operation. The memory bandwidth is reduced by 5-8%. When devices supporting QUIC encryption in hardware come to the market, we will be able to free further 7% of CPU utilization which is used today for crypto operations.
Adel Abouchaev (6):
Documentation on QUIC kernel Tx crypto.
Define QUIC specific constants, control and data plane structures Add UDP ULP operations, initialization and handling prototype
functions.
Implement QUIC offload functions
Add flow counters and Tx processing error counter
Add self tests for ULP operations, flow setup and crypto testsDocumentation/networking/index.rst | 1 + Documentation/networking/quic.rst | 185 ++++ include/net/inet_sock.h | 2 + include/net/netns/mib.h | 3 + include/net/quic.h | 63 ++ include/net/snmp.h | 6 + include/net/udp.h | 33 + include/uapi/linux/quic.h | 60 + include/uapi/linux/snmp.h | 9 + include/uapi/linux/udp.h | 4 + net/Kconfig | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 15 + net/ipv4/udp_ulp.c | 192 ++++ net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1417 ++++++++++++++++++++++++ net/quic/quic_proc.c | 45 + tools/testing/selftests/net/.gitignore | 4 +- tools/testing/selftests/net/Makefile | 3 +- tools/testing/selftests/net/quic.c | 1153 +++++++++++++++++++ tools/testing/selftests/net/quic.sh | 46 + 23 files changed, 3267 insertions(+), 3 deletions(-)create mode 100644
Documentation/networking/quic.rstcreate mode 100644include/net/quic.h
create mode 100644include/uapi/linux/quic.hcreate mode 100644net/ipv4/udp_ulp.c
create mode 100644net/quic/Kconfig
create mode 100644net/quic/Makefile
create mode 100644net/quic/quic_main.ccreate mode 100644net/quic/quic_proc.ccreate mode 100644tools/testing/selftests/net/quic.ccreate mode 100755tools/testing/selftests/net/quic.shQuoted 4 lines
base-commit:
fd78d07c7c35de260eb89f1be4a1e7487b8092ad--
2.30.2
Quoted 7 lines
Hi, Adel,
I don't see how the key update(rfc9001#section-6) is handled on the TX path, which is not using TLS Key update, and "Key Phase" indicates which key will be used after rekeying. Also, I think it is almost impossible to handle the peer rekeying on the RX path either based on your current model in the future.
The update is not present in these patches, but it is an important part of the QUIC functionality. As this patch is only storing a single key, you are correct that this approach does not handle the key rotation. To implement re-keying on Tx and on Rx a rolling secret will need to be stored in kernel. In that case, the subsequent 1RTT (Application space) keys will be refreshed by the kernel. After all, when the hardware is mature enough to support QUIC encryption and decryption - the secret will need to be kept in the hardware to react on time on Rx, especially. Tx path could solicit the re-key at any point or by the exhaustion of the counter of GCM (packet number in this case). The RFC expects the implementation to retain 2 keys, at least, while keeping 3 (old, current and next) is not prohibited either. Keeping more is not necessary.
Quoted 6 lines
The patch seems to get the
crypto_ctxby doing a connection hash table lookup in thesendmsg(), which is not good from the performance side. One QUIC connection can go over multiple UDP sockets, but I don't think one socket can be used by multiple QUIC connections. So why not save the ctx in the socket instead?
A single socket could have multiple connections originated from it, having different destinations, if the socket is not connected. An optimization could be made for connected sockets to cache the context and save time on a lookup. The measurement of kernel operations timing did not reveal a significant amount of time spent in this lookup due to a relatively small number of connections per socket in general. A shared table across multiple sockets might experience a different performance grading.
Quoted 6 lines
The patch is to reduce the copying operations between user space and the kernel. I might miss something in your user space code, but the msg to send is *already packed* into the Stream Frame in user space, what's the difference if you encrypt it in userspace and then
sendmsg(udp_sk)with zero-copy to the kernel.
It is possible to do it this way. Zero-copy works best with packet sizes starting at 32K and larger. Anything less than that would consume the improvements of zero-copy by zero-copy pre/post operations and needs to align memory. The other possible obstacle would be that eventual support of QUIC encryption and decryption in hardware would integrate well with this current approach.
Quoted 4 lines
Didn't really understand the "GSO" you mentioned, as I don't see any code about kernel GSO, I guess it's just "Fragment size", right? BTW, it‘s not common to use "//" for the kernel annotation.
Once the payload arrives into the kernel, the GSO on the interface would instruct L3/L4 stack on fragmentation. In this case, the plaintext QUIC packets should be aligned on the GSO marks less the tag size that would be added by encryption. For GSO size 1000, the QUIC packets in the batch for transmission should all be 984 bytes long, except maybe the last one. Once the tag is attached, the new size of 1000 will correctly split the QUIC packets further down the stack for transmission in individual IP/UDP packets. The code is also saving processing time by sending all packets at once to UDP in a single call, when GSO is enabled.
Quoted 7 lines
I'm not sure if it's worth adding a ULP layer over UDP for this QUIC TX only. Honestly, I'm more supporting doing a full QUIC stack in the kernel independently with socket APIs to use it:
https://github.com/lxin/tls_hs.Thanks.
Willem de Bruijn
maintainerQuoted 6 lines
Quoted 6 lines
The patch seems to get the
crypto_ctxby doing a connection hash table lookup in thesendmsg(), which is not good from the performance side. One QUIC connection can go over multiple UDP sockets, but I don't think one socket can be used by multiple QUIC connections. So why not save the ctx in the socket instead?
Quoted 8 lines
A single socket could have multiple connections originated from it, having different destinations, if the socket is not connected. An optimization could be made for connected sockets to cache the context and save time on a lookup. The measurement of kernel operations timing did not reveal a significant amount of time spent in this lookup due to a relatively small number of connections per socket in general. A shared table across multiple sockets might experience a different performance grading.
I'm late to this patch series, sorry. High quality implementation. I have a few design questions similar to Xin.
If multiplexing, instead of looking up a connection by { address, port variable length connection ID }, perhaps return a connection table index on setsockopt and use that in sendmsg.
Quoted 10 lines
Quoted 6 lines
The patch is to reduce the copying operations between user space and the kernel. I might miss something in your user space code, but the msg to send is *already packed* into the Stream Frame in user space, what's the difference if you encrypt it in userspace and then
sendmsg(udp_sk)with zero-copy to the kernel.It is possible to do it this way. Zero-copy works best with packet sizes starting at 32K and larger. Anything less than that would consume the improvements of zero-copy by zero-copy pre/post operations and needs to align memory.
Part of the cost of MSG_ZEROCOPY is in mapping and unmapping user pages. This series re-implements that with its own get_user_pages. That is duplicative non-trivial code. And it will incur the same cost. What this implementation saves is the (indeed non-trivial) asynchronous completion notification over the error queue.
The cover letter gives some performance numbers against a userspace implementation that has to copy from user to kernel. It might be more even to compare against an implementation using MSG_ZEROCOPY and UDP_SEGMENT. A userspace crypto implementation may have other benefits compared to a kernel implementation, such as not having to convert to crypto API scatter-gather arrays and back to network structures.
A few related points
- The implementation support multiplexed connections, but only one crypto sendmsg can be outstanding at any time:
+ /**
+ * To synchronize concurrent sendmsg() requests through the same socket
+ * and protect preallocated per-context memory.
+ **/
+ struct mutex sendmsg_mux;
That is quite limiting for production workloads.
- Crypto operations are also executed synchronously, using crypto_wait_req after each operationn. This limits throughput by using at most one core per UDP socket. And adds sendmsg latency (which may or may not be important to the application). Wireguard shows an example of how to parallelize software crypto across cores.
- The implementation avoids dynamic allocation of cipher text pages by using a single ctx->cipher_page. This is protected by sendmsg_mux (see above). Is that safe when packets leave the protocol stack and are then held in a qdisc or when being processed by the NIC? quic_sendmsg_locked will return, but the cipher page is not free to reuse yet.
- The real benefit of kernel QUIC will come from HW offload. Would it be better to avoid the complexity of an in-kernel software implementation and only focus on HW offload? Basically, pass the plaintext QUIC packets over a standard UDP socket and alongside in a cmsg pass either an index into a HW security association database or the immediate { key, iv } connection_info (for stateless sockets), to be encoded into the descriptor by the device driver.
- With such a simpler path, could we avoid introducing ULP and just have udp [gs]etsockopt CRYPTO_STATE. Where QUIC is the only defined state type yet.
- Small aside: as the series introduces new APIs with non-trivial parsing in the kernel, it's good to run a fuzzer like syzkaller on it (if not having done so yet).
Quoted 7 lines
The other possible obstacle would be that eventual support of QUIC encryption and decryption in hardware would integrate well with this current approach.
Quoted 4 lines
Didn't really understand the "GSO" you mentioned, as I don't see any code about kernel GSO, I guess it's just "Fragment size", right? BTW, it‘s not common to use "//" for the kernel annotation.
minor point: fragment has meaning in IPv4. For GSO, prefer gso_size.
Quoted 16 lines
Once the payload arrives into the kernel, the GSO on the interface would instruct L3/L4 stack on fragmentation. In this case, the plaintext QUIC packets should be aligned on the GSO marks less the tag size that would be added by encryption. For GSO size 1000, the QUIC packets in the batch for transmission should all be 984 bytes long, except maybe the last one. Once the tag is attached, the new size of 1000 will correctly split the QUIC packets further down the stack for transmission in individual IP/UDP packets. The code is also saving processing time by sending all packets at once to UDP in a single call, when GSO is enabled.
Quoted 7 lines
I'm not sure if it's worth adding a ULP layer over UDP for this QUIC TX only. Honestly, I'm more supporting doing a full QUIC stack in the kernel independently with socket APIs to use it:
https://github.com/lxin/tls_hs.Thanks.
Quoted 13 lines · Willem de Bruijn
Quoted 5 lines
Quoted 5 lines
The patch seems to get the
crypto_ctxby doing a connection hash table lookup in thesendmsg(), which is not good from the performance side. One QUIC connection can go over multiple UDP sockets, but I don't think one socket can be used by multiple QUIC connections. So why not save the ctx in the socket instead?Quoted 8 lines
A single socket could have multiple connections originated from it, having different destinations, if the socket is not connected. An optimization could be made for connected sockets to cache the context and save time on a lookup. The measurement of kernel operations timing did not reveal a significant amount of time spent in this lookup due to a relatively small number of connections per socket in general. A shared table across multiple sockets might experience a different performance grading.
Quoted 6 lines
I'm late to this patch series, sorry. High quality implementation. I have a few design questions similar to Xin.
If multiplexing, instead of looking up a connection by { address, port variable length connection ID }, perhaps return a connection table index on setsockopt and use that in sendmsg.
It was deliberate to not to return anything other than 0 from setsockopt() as defined in the spec for the function. Despite that it says "shall", the doc says that 0 is the only value for successful operation. This was the reason not to use setsockopt() for any bidirectional transfers of data and or status. A more sophisticated approach with netlink sockets would be more suitable for it. The second reason is the API asymmetry for Tx and Rx which will be introduced - the Rx will still need to match on the address, port and cid. The third reason is that in current implementations there are no more than a few connections per socket, which does not abuse the rhashtable that does a lookup, although it takes time to hash the key into a hash for a seek. The performance measurement ran against the runtime and did not flag this path as underperforming either, there were other parts that substantially add to the runtime, not the key lookup though.
Quoted 27 lines
Quoted 9 lines
Quoted 5 lines
The patch is to reduce the copying operations between user space and the kernel. I might miss something in your user space code, but the msg to send is *already packed* into the Stream Frame in user space, what's the difference if you encrypt it in userspace and then
sendmsg(udp_sk)with zero-copy to the kernel.It is possible to do it this way. Zero-copy works best with packet sizes starting at 32K and larger. Anything less than that would consume the improvements of zero-copy by zero-copy pre/post operations and needs to align memory.
Part of the cost of
MSG_ZEROCOPYis in mapping and unmapping user pages. This series re-implements that with its ownget_user_pages. That is duplicative non-trivial code. And it will incur the same cost. What this implementation saves is the (indeed non-trivial) asynchronous completion notification over the error queue.The cover letter gives some performance numbers against a userspace implementation that has to copy from user to kernel. It might be more even to compare against an implementation using
MSG_ZEROCOPYandUDP_SEGMENT. A userspace crypto implementation may have other benefits compared to a kernel implementation, such as not having to convert to crypto API scatter-gather arrays and back to network structures.A few related points
- The implementation support multiplexed connections, but only one crypto sendmsg can be outstanding at any time:
Quoted 7 lines
+ /**
+ * To synchronize concurrentsendmsg()requests through the same socket
+ * and protect preallocated per-context memory.
+ **/
+ struct mutexsendmsg_mux;That is quite limiting for production workloads.
The use case that we have with MVFST library currently runs a single worker for a connection and has a single socket attached to it. QUIC allows simultaneous use of multiple connection IDs to swap them in runtime, and implementation would request only a handful of these. The MVFST batches writes into a block of about 8Kb and then uses GSO to send them all at once.
Quoted 6 lines
- Crypto operations are also executed synchronously, using
crypto_wait_reqafter each operationn. This limits throughput by using at most one core per UDP socket. And adds sendmsg latency (which may or may not be important to the application). Wireguard shows an example of how to parallelize software crypto across cores.
Quoted 6 lines
- The implementation avoids dynamic allocation of cipher text pages by using a single
ctx->cipher_page. This is protected bysendmsg_mux(see above). Is that safe when packets leave the protocol stack and are then held in a qdisc or when being processed by the NIC?quic_sendmsg_lockedwill return, but the cipher page is not free to reuse yet.
There is currently no use case that we have in hands that requires parallel transmission of data for the same connection. Multiple connections would have no issue running in parallel as each of them will have it's own preallocated cipher_page in the context.
There is a fragmentation further down the stack with ip_generic_getfrag() that eventually does copy_from_iter() and makea a copy of the data. This is executed as part of __ip_append_data() called from udp_sendmsg() in ipv4/udp.c. The assumption was that this is executed synchronously and the queues and NIC will see a mapping of a different memory area than the ciphertext in the pre-allocated page.
Quoted 8 lines
- The real benefit of kernel QUIC will come from HW offload. Would it be better to avoid the complexity of an in-kernel software implementation and only focus on HW offload? Basically, pass the plaintext QUIC packets over a standard UDP socket and alongside in a cmsg pass either an index into a HW security association database or the immediate { key, iv }
connection_info(for stateless sockets), to be encoded into the descriptor by the device driver.
Hardware usually targets a single ciphersuite such as AES-GCM-128/256, while QUIC also supports Chacha20-Poly1305 and AES-CCM. The generalized support for offload prompted implementation of these ciphers in kernel code. The kernel code could also engage if the future hardware has capacity caps preventing it from handling all requests in the hardware.
Quoted 7 lines
- With such a simpler path, could we avoid introducing ULP and just have udp [gs]etsockopt
CRYPTO_STATE. Where QUIC is the only defined state type yet.- Small aside: as the series introduces new APIs with non-trivial parsing in the kernel, it's good to run a fuzzer like syzkaller on it (if not having done so yet).
Agreed.
Quoted 7 lines
Quoted 6 lines
The other possible obstacle would be that eventual support of QUIC encryption and decryption in hardware would integrate well with this current approach.
Quoted 3 lines
Didn't really understand the "GSO" you mentioned, as I don't see any code about kernel GSO, I guess it's just "Fragment size", right? BTW, it‘s not common to use "//" for the kernel annotation.
minor point: fragment has meaning in IPv4. For GSO, prefer
gso_size.
Sure, will change it to gso_size.
Quoted 16 lines
Quoted 16 lines
Once the payload arrives into the kernel, the GSO on the interface would instruct L3/L4 stack on fragmentation. In this case, the plaintext QUIC packets should be aligned on the GSO marks less the tag size that would be added by encryption. For GSO size 1000, the QUIC packets in the batch for transmission should all be 984 bytes long, except maybe the last one. Once the tag is attached, the new size of 1000 will correctly split the QUIC packets further down the stack for transmission in individual IP/UDP packets. The code is also saving processing time by sending all packets at once to UDP in a single call, when GSO is enabled.
Quoted 6 lines
I'm not sure if it's worth adding a ULP layer over UDP for this QUIC TX only. Honestly, I'm more supporting doing a full QUIC stack in the kernel independently with socket APIs to use it:
https://github.com/lxin/tls_hs.Thanks.
Willem de Bruijn
maintainerQuoted 24 lines · Adel Abouchaev
Quoted 13 lines · Willem de Bruijn
Quoted 5 lines
Quoted 5 lines
The patch seems to get the
crypto_ctxby doing a connection hash table lookup in thesendmsg(), which is not good from the performance side. One QUIC connection can go over multiple UDP sockets, but I don't think one socket can be used by multiple QUIC connections. So why not save the ctx in the socket instead?Quoted 8 lines
A single socket could have multiple connections originated from it, having different destinations, if the socket is not connected. An optimization could be made for connected sockets to cache the context and save time on a lookup. The measurement of kernel operations timing did not reveal a significant amount of time spent in this lookup due to a relatively small number of connections per socket in general. A shared table across multiple sockets might experience a different performance grading.
Quoted 6 lines
I'm late to this patch series, sorry. High quality implementation. I have a few design questions similar to Xin.
If multiplexing, instead of looking up a connection by { address, port variable length connection ID }, perhaps return a connection table index on setsockopt and use that in sendmsg.
Quoted 7 lines
It was deliberate to not to return anything other than 0 from
setsockopt()as defined in the spec for the function. Despite that it says "shall", the doc says that 0 is the only value for successful operation. This was the reason not to usesetsockopt()for any bidirectional transfers of data and or status. A more sophisticated approach with netlink sockets would be more suitable for it. The second reason is the API asymmetry for Tx and Rx which will be introduced - the
I thought the cover letter indicated that due to asymmetry of most QUIC workloads, only Tx offload is implemented. You do intend to add Rx later?
Quoted 3 lines
Rx will still need to match on the address, port and cid. The third reason is that in current implementations there are no more than a few connections per socket,
This is very different from how we do QUIC at Google. There we definitely multiplex many connections across essentially a socket per CPU IIRC.
Quoted 61 lines
which does not abuse the rhashtable that does a
lookup, although it takes time to hash the key into a hash for a seek. The performance measurement ran against the runtime and did not flag this path as underperforming either, there were other parts that substantially add to the runtime, not the key lookup though.Quoted 27 lines
Quoted 9 lines
Quoted 5 lines
The patch is to reduce the copying operations between user space and the kernel. I might miss something in your user space code, but the msg to send is *already packed* into the Stream Frame in user space, what's the difference if you encrypt it in userspace and then
sendmsg(udp_sk)with zero-copy to the kernel.It is possible to do it this way. Zero-copy works best with packet sizes starting at 32K and larger. Anything less than that would consume the improvements of zero-copy by zero-copy pre/post operations and needs to align memory.
Part of the cost of
MSG_ZEROCOPYis in mapping and unmapping user pages. This series re-implements that with its ownget_user_pages. That is duplicative non-trivial code. And it will incur the same cost. What this implementation saves is the (indeed non-trivial) asynchronous completion notification over the error queue.The cover letter gives some performance numbers against a userspace implementation that has to copy from user to kernel. It might be more even to compare against an implementation using
MSG_ZEROCOPYandUDP_SEGMENT. A userspace crypto implementation may have other benefits compared to a kernel implementation, such as not having to convert to crypto API scatter-gather arrays and back to network structures.A few related points
- The implementation support multiplexed connections, but only one crypto sendmsg can be outstanding at any time:
Quoted 7 lines
+ /**
+ * To synchronize concurrentsendmsg()requests through the same socket
+ * and protect preallocated per-context memory.
+ **/
+ struct mutexsendmsg_mux;That is quite limiting for production workloads.
The use case that we have with MVFST library currently runs a single worker for a connection and has a single socket attached to it. QUIC allows simultaneous use of multiple connection IDs to swap them in runtime, and implementation would request only a handful of these. The MVFST batches writes into a block of about 8Kb and then uses GSO to send them all at once.
Quoted 6 lines
- Crypto operations are also executed synchronously, using
crypto_wait_reqafter each operationn. This limits throughput by using at most one core per UDP socket. And adds sendmsg latency (which may or may not be important to the application). Wireguard shows an example of how to parallelize software crypto across cores.Quoted 6 lines
- The implementation avoids dynamic allocation of cipher text pages by using a single
ctx->cipher_page. This is protected bysendmsg_mux(see above). Is that safe when packets leave the protocol stack and are then held in a qdisc or when being processed by the NIC?quic_sendmsg_lockedwill return, but the cipher page is not free to reuse yet.
Quoted 4 lines
There is currently no use case that we have in hands that requires parallel transmission of data for the same connection. Multiple connections would have no issue running in parallel as each of them will have it's own preallocated
cipher_pagein the context.
This still leaves the point that sendmsg may return and the mutex released while the cipher_page is still associated with an skb in the transmit path.
Quoted 15 lines
There is a fragmentation further down the stack with
ip_generic_getfrag()that eventually doescopy_from_iter()and makea a copy of the data. This is executed as part of__ip_append_data()called fromudp_sendmsg()inipv4/udp.c. The assumption was that this is executed synchronously and the queues and NIC will see a mapping of a different memory area than the ciphertext in the pre-allocated page.Quoted 8 lines
- The real benefit of kernel QUIC will come from HW offload. Would it be better to avoid the complexity of an in-kernel software implementation and only focus on HW offload? Basically, pass the plaintext QUIC packets over a standard UDP socket and alongside in a cmsg pass either an index into a HW security association database or the immediate { key, iv }
connection_info(for stateless sockets), to be encoded into the descriptor by the device driver.
Quoted 4 lines
Hardware usually targets a single ciphersuite such as AES-GCM-128/256, while QUIC also supports Chacha20-Poly1305 and AES-CCM. The generalized support for offload prompted implementation of these ciphers in kernel code.
All userspace libraries also support all protocols as fall-back. No need for two fall-backs if HW support is missing?
Quoted 34 lines
The kernel code could also engage if the future hardware has capacity caps preventing it from handling all requests in the hardware.
Quoted 7 lines
- With such a simpler path, could we avoid introducing ULP and just have udp [gs]etsockopt
CRYPTO_STATE. Where QUIC is the only defined state type yet.- Small aside: as the series introduces new APIs with non-trivial parsing in the kernel, it's good to run a fuzzer like syzkaller on it (if not having done so yet).
Agreed.
Quoted 7 lines
Quoted 6 lines
The other possible obstacle would be that eventual support of QUIC encryption and decryption in hardware would integrate well with this current approach.
Quoted 3 lines
Didn't really understand the "GSO" you mentioned, as I don't see any code about kernel GSO, I guess it's just "Fragment size", right? BTW, it‘s not common to use "//" for the kernel annotation.
minor point: fragment has meaning in IPv4. For GSO, prefer
gso_size.Sure, will change it to
gso_size.Quoted 16 lines
Quoted 16 lines
Once the payload arrives into the kernel, the GSO on the interface would instruct L3/L4 stack on fragmentation. In this case, the plaintext QUIC packets should be aligned on the GSO marks less the tag size that would be added by encryption. For GSO size 1000, the QUIC packets in the batch for transmission should all be 984 bytes long, except maybe the last one. Once the tag is attached, the new size of 1000 will correctly split the QUIC packets further down the stack for transmission in individual IP/UDP packets. The code is also saving processing time by sending all packets at once to UDP in a single call, when GSO is enabled.
Quoted 6 lines
I'm not sure if it's worth adding a ULP layer over UDP for this QUIC TX only. Honestly, I'm more supporting doing a full QUIC stack in the kernel independently with socket APIs to use it:
https://github.com/lxin/tls_hs.Thanks.
Quoted 30 lines · Willem de Bruijn
Quoted 22 lines · Adel Abouchaev
Quoted 13 lines · Willem de Bruijn
Quoted 5 lines
Quoted 5 lines
The patch seems to get the
crypto_ctxby doing a connection hash table lookup in thesendmsg(), which is not good from the performance side. One QUIC connection can go over multiple UDP sockets, but I don't think one socket can be used by multiple QUIC connections. So why not save the ctx in the socket instead?Quoted 8 lines
A single socket could have multiple connections originated from it, having different destinations, if the socket is not connected. An optimization could be made for connected sockets to cache the context and save time on a lookup. The measurement of kernel operations timing did not reveal a significant amount of time spent in this lookup due to a relatively small number of connections per socket in general. A shared table across multiple sockets might experience a different performance grading.
Quoted 6 lines
I'm late to this patch series, sorry. High quality implementation. I have a few design questions similar to Xin.
If multiplexing, instead of looking up a connection by { address, port variable length connection ID }, perhaps return a connection table index on setsockopt and use that in sendmsg.
Quoted 7 lines
It was deliberate to not to return anything other than 0 from
setsockopt()as defined in the spec for the function. Despite that it says "shall", the doc says that 0 is the only value for successful operation. This was the reason not to usesetsockopt()for any bidirectional transfers of data and or status. A more sophisticated approach with netlink sockets would be more suitable for it. The second reason is the API asymmetry for Tx and Rx which will be introduced - the
Quoted 3 lines
I thought the cover letter indicated that due to asymmetry of most QUIC workloads, only Tx offload is implemented. You do intend to add Rx later?
We are planning to include Rx later as well, Tx is more compelling from use case perspective as the main application of this would go to an Edge, where Tx is heavy and Rx is much lighter.
Quoted 6 lines
Quoted 3 lines
Rx will still need to match on the address, port and cid. The third reason is that in current implementations there are no more than a few connections per socket,
This is very different from how we do QUIC at Google. There we definitely multiplex many connections across essentially a socket per CPU IIRC.
Ian mentioned that for such a case, while using a zero length QUIC CID, the remote ends are ephemeral ports and are different. In which case each connection will have its own context and yes, it will load the hash table quite a bit for a socket. The limiting factor is still a BSD4.4 interface and returning a value from setsockopt() with the entry ID. It might be more suitable once a standard netlink interface is available for this to easily plug in. However, this will further complicate a userspace library integration which is a not so trivial task today too.
Quoted 61 lines
Quoted 61 lines
which does not abuse the rhashtable that does a
lookup, although it takes time to hash the key into a hash for a seek. The performance measurement ran against the runtime and did not flag this path as underperforming either, there were other parts that substantially add to the runtime, not the key lookup though.Quoted 27 lines
Quoted 9 lines
Quoted 5 lines
The patch is to reduce the copying operations between user space and the kernel. I might miss something in your user space code, but the msg to send is *already packed* into the Stream Frame in user space, what's the difference if you encrypt it in userspace and then
sendmsg(udp_sk)with zero-copy to the kernel.It is possible to do it this way. Zero-copy works best with packet sizes starting at 32K and larger. Anything less than that would consume the improvements of zero-copy by zero-copy pre/post operations and needs to align memory.
Part of the cost of
MSG_ZEROCOPYis in mapping and unmapping user pages. This series re-implements that with its ownget_user_pages. That is duplicative non-trivial code. And it will incur the same cost. What this implementation saves is the (indeed non-trivial) asynchronous completion notification over the error queue.The cover letter gives some performance numbers against a userspace implementation that has to copy from user to kernel. It might be more even to compare against an implementation using
MSG_ZEROCOPYandUDP_SEGMENT. A userspace crypto implementation may have other benefits compared to a kernel implementation, such as not having to convert to crypto API scatter-gather arrays and back to network structures.A few related points
- The implementation support multiplexed connections, but only one crypto sendmsg can be outstanding at any time:
Quoted 7 lines
+ /**
+ * To synchronize concurrentsendmsg()requests through the same socket
+ * and protect preallocated per-context memory.
+ **/
+ struct mutexsendmsg_mux;That is quite limiting for production workloads.
The use case that we have with MVFST library currently runs a single worker for a connection and has a single socket attached to it. QUIC allows simultaneous use of multiple connection IDs to swap them in runtime, and implementation would request only a handful of these. The MVFST batches writes into a block of about 8Kb and then uses GSO to send them all at once.
Quoted 12 lines
- Crypto operations are also executed synchronously, using
crypto_wait_reqafter each operationn. This limits throughput by using at most one core per UDP socket. And adds sendmsg latency (which may or may not be important to the application). Wireguard shows an example of how to parallelize software crypto across cores.- The implementation avoids dynamic allocation of cipher text pages by using a single
ctx->cipher_page. This is protected bysendmsg_mux(see above). Is that safe when packets leave the protocol stack and are then held in a qdisc or when being processed by the NIC?quic_sendmsg_lockedwill return, but the cipher page is not free to reuse yet.
Quoted 7 lines
Quoted 4 lines
There is currently no use case that we have in hands that requires parallel transmission of data for the same connection. Multiple connections would have no issue running in parallel as each of them will have it's own preallocated
cipher_pagein the context.This still leaves the point that sendmsg may return and the mutex released while the
cipher_pageis still associated with an skb in the transmit path.
Correct, there is a further copy from the cipher buffer into fragmented pieces done by ip_generic_getfrag(). Am I reading it wrong that it would not need the cipher text allocated buffer after that is done and all the data is copied from it into further structures using copy_from_iter()? The skb would be built upon a different buffer space than the encrypted data. The udp_sendmsg() assumes that it receives the memory from userspace and does all these ops to move stuff from userspace. While doing that, it would omit tons of it's checks as the memory is already in kernel and still execute quickly.
Quoted 15 lines
Quoted 15 lines
There is a fragmentation further down the stack with
ip_generic_getfrag()that eventually doescopy_from_iter()and makea a copy of the data. This is executed as part of__ip_append_data()called fromudp_sendmsg()inipv4/udp.c. The assumption was that this is executed synchronously and the queues and NIC will see a mapping of a different memory area than the ciphertext in the pre-allocated page.Quoted 7 lines
- The real benefit of kernel QUIC will come from HW offload. Would it be better to avoid the complexity of an in-kernel software implementation and only focus on HW offload? Basically, pass the plaintext QUIC packets over a standard UDP socket and alongside in a cmsg pass either an index into a HW security association database or the immediate { key, iv }
connection_info(for stateless sockets), to be encoded into the descriptor by the device driver.
Quoted 6 lines
Quoted 4 lines
Hardware usually targets a single ciphersuite such as AES-GCM-128/256, while QUIC also supports Chacha20-Poly1305 and AES-CCM. The generalized support for offload prompted implementation of these ciphers in kernel code.
All userspace libraries also support all protocols as fall-back. No need for two fall-backs if HW support is missing?
Could be done that way. Looking at TLS - it has fallback to kernel though in tls_sw, although that is a dual purpose code.
Quoted 33 lines
Quoted 33 lines
The kernel code could also engage if the future hardware has capacity caps preventing it from handling all requests in the hardware.
Quoted 7 lines
- With such a simpler path, could we avoid introducing ULP and just have udp [gs]etsockopt
CRYPTO_STATE. Where QUIC is the only defined state type yet.- Small aside: as the series introduces new APIs with non-trivial parsing in the kernel, it's good to run a fuzzer like syzkaller on it (if not having done so yet).
Agreed.
Quoted 7 lines
Quoted 6 lines
The other possible obstacle would be that eventual support of QUIC encryption and decryption in hardware would integrate well with this current approach.
Quoted 3 lines
Didn't really understand the "GSO" you mentioned, as I don't see any code about kernel GSO, I guess it's just "Fragment size", right? BTW, it‘s not common to use "//" for the kernel annotation.
minor point: fragment has meaning in IPv4. For GSO, prefer
gso_size.Sure, will change it to
gso_size.Quoted 15 lines
Quoted 15 lines
Once the payload arrives into the kernel, the GSO on the interface would instruct L3/L4 stack on fragmentation. In this case, the plaintext QUIC packets should be aligned on the GSO marks less the tag size that would be added by encryption. For GSO size 1000, the QUIC packets in the batch for transmission should all be 984 bytes long, except maybe the last one. Once the tag is attached, the new size of 1000 will correctly split the QUIC packets further down the stack for transmission in individual IP/UDP packets. The code is also saving processing time by sending all packets at once to UDP in a single call, when GSO is enabled.
Quoted 6 lines
I'm not sure if it's worth adding a ULP layer over UDP for this QUIC TX only. Honestly, I'm more supporting doing a full QUIC stack in the kernel independently with socket APIs to use it:
https://github.com/lxin/tls_hs.Thanks.
[net-next] Fix reinitialization of TEST_PROGS in net self tests.
In reply to a message not in the archiveFix reinitialization of TEST_PROGS in net self tests.
--- tools/testing/selftests/net/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/net/Makefile +1 -1
@@ -42,7 +42,7 @@ TEST_PROGS += arp_ndisc_evict_nocarrier.sh TEST_PROGS += ndisc_unsolicited_na_test.sh TEST_PROGS += arp_ndisc_untracked_subnets.sh TEST_PROGS += stress_reuseport_listen.sh -TEST_PROGS := l2_tos_ttl_inherit.sh +TEST_PROGS += l2_tos_ttl_inherit.sh TEST_PROGS_EXTENDED := in_netns.sh setup_loopback.sh setup_veth.sh TEST_PROGS_EXTENDED += toeplitz_client.sh toeplitz.sh TEST_GEN_FILES = socket nettest
Signature · 1 line
2.30.2
Re: [net-next] Fix reinitialization of TEST_PROGS in net self tests.
Quoted 13 lines · Adel Abouchaev
Fix reinitialization of
TEST_PROGSin net self tests.Signed-off-by Adel Abouchaev <[email protected]>--- tools/testing/selftests/net/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/testing/selftests/net/Makefile +0
-0
tools/testing/selftests/net/Makefile TEST_PROGS += arp_ndisc_evict_nocarrier.sh · line 48
Quoted 8 lines
tools/testing/selftests/net/Makefile +1
-1@@ -42,7 +42,7 @@ TEST_PROGS += arp_ndisc_evict_nocarrier.sh TEST_PROGS += ndisc_unsolicited_na_test.sh TEST_PROGS += arp_ndisc_untracked_subnets.sh TEST_PROGS += stress_reuseport_listen.sh -TEST_PROGS := l2_tos_ttl_inherit.sh +TEST_PROGS += l2_tos_ttl_inherit.sh TEST_PROGS_EXTENDED := in_netns.sh setup_loopback.sh setup_veth.sh TEST_PROGS_EXTENDED += toeplitz_client.sh toeplitz.sh TEST_GEN_FILES = socket nettest
Thank you for fixing this. I am seeing these types of bugs recently Have to careful with := vs +=
thanks,
-- Shuah
Re: [net-next] Fix reinitialization of TEST_PROGS in net self tests.
In reply to Adel AbouchaevThis patch was applied to netdev/net-next.git (master) by Jakub Kicinski <[email protected]>:
Hello:
This patch was applied to netdev/net-next.git (master) by Jakub Kicinski <[email protected]>:
Quoted 6 lines · On Wed, 24 Aug 2022 11:43:51 -0700 you wrote:
Fix reinitialization of
TEST_PROGSin net self tests.Signed-off-by Adel Abouchaev <[email protected]>--- tools/testing/selftests/net/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Here is the summary with links:
- [net-next] Fix reinitialization of TEST_PROGS in net self tests.
https://git.kernel.org/netdev/net-next/c/88e500affe72
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html
v3 cover net: support QUIC crypto
In reply to a message not in the archiveQUIC requires end to end encryption of the data. The application usually prepares the data in clear text, encrypts and calls send() which implies multiple copies of the data before the packets hit the networking stack. Similar to kTLS, QUIC kernel offload of cryptography reduces the memory pressure by reducing the number of copies.
The scope of kernel support is limited to the symmetric cryptography, leaving the handshake to the user space library. For QUIC in particular, the application packets that require symmetric cryptography are the 1RTT packets with short headers. Kernel will encrypt the application packets on transmission and decrypt on receive. This series implements Tx only, because in QUIC server applications Tx outweighs Rx by orders of magnitude.
Supporting the combination of QUIC and GSO requires the application to correctly place the data and the kernel to correctly slice it. The encryption process appends an arbitrary number of bytes (tag) to the end of the message to authenticate it. The GSO value should include this overhead, the offload would then subtract the tag size to parse the input on Tx before chunking and encrypting it.
With the kernel cryptography, the buffer copy operation is conjoined with the encryption operation. The memory bandwidth is reduced by 5-8%. When devices supporting QUIC encryption in hardware come to the market, we will be able to free further 7% of CPU utilization which is used today for crypto operations.
Adel Abouchaev (6):
Documentation on QUIC kernel Tx crypto.
Define QUIC specific constants, control and data plane structures Add UDP ULP operations, initialization and handling prototype
functions.
Implement QUIC offload functions
Add flow counters and Tx processing error counter
Add self tests for ULP operations, flow setup and crypto tests
Documentation/networking/index.rst | 1 + Documentation/networking/quic.rst | 211 ++++ include/net/inet_sock.h | 2 + include/net/netns/mib.h | 3 + include/net/quic.h | 63 + include/net/snmp.h | 6 + include/net/udp.h | 33 + include/uapi/linux/quic.h | 66 + include/uapi/linux/snmp.h | 9 + include/uapi/linux/udp.h | 4 + net/Kconfig | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 15 + net/ipv4/udp_ulp.c | 192 +++ net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1533 ++++++++++++++++++++++++ net/quic/quic_proc.c | 45 + tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/Makefile | 3 +- tools/testing/selftests/net/quic.c | 1370 +++++++++++++++++++++ tools/testing/selftests/net/quic.sh | 46 + 23 files changed, 3630 insertions(+), 2 deletions(-) create mode 100644 Documentation/networking/quic.rst create mode 100644 include/net/quic.h create mode 100644 include/uapi/linux/quic.h create mode 100644 net/ipv4/udp_ulp.c create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c create mode 100644 net/quic/quic_proc.c create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
Signature · 1 line
2.30.2
PATCH v3 1/6 net: Documentation on QUIC kernel Tx crypto.
Add documentation for kernel QUIC code.
---
Added quic.rst reference to the index.rst file; identation in quic.rst file.
Added SPDX license GPL 2.0.
v2: Removed whitespace at EOF.
v3: Added explanation of features.
--- Documentation/networking/index.rst | 1 + Documentation/networking/quic.rst | 211 +++++++++++++++++++++++++++++ 2 files changed, 212 insertions(+) create mode 100644 Documentation/networking/quic.rst
Documentation/networking/index.rst +1 -0
@@ -89,6 +89,7 @@ Contents: plip ppp_generic proc_net_tcp + quic radiotap-headers rds regulatory
Documentation/networking/quic.rst new file +211 -0
211 lines not on this page · raw message
Signature · 1 line
2.30.2
Re: [net-next v3 1/6] net: Documentation on QUIC kernel Tx crypto.
Quoted 14 lines · Adel Abouchaev
+===========
(quoted diff) +13
-0+KERNEL QUIC +=========== + +Overview +======== + +QUIC is a secure general-purpose transport protocol that creates a stateful +interaction between a client and a server. QUIC provides end-to-end integrity +and confidentiality. Refer to RFC 9000 for more information on QUIC. + +The kernel Tx side offload covers the encryption of the application streams +in the kernel rather than in the application. These packets are 1RTT packets +in QUIC connection. Encryption of every other packets is still done by the
Review
Quoted 10 lines
(quoted diff) +10
-0+QUIC library in user space. + +The flow match is performed using 5 parameters: source and destination IP +addresses, source and destination UDP ports and destination QUIC connection ID. +Not all 5 parameters are always needed. The Tx direction matches the flow on +the destination IP, port and destination connection ID, while the Rx part would +later match on source IP, port and destination connection ID. This will cover +multiple scenarios where the server is using SO_REUSEADDR and/or empty +destination connection IDs or combination of these. +
Both Tx and Rx direction match destination connection ID. Is it right?
Quoted 175 lines
(quoted diff) +175
-0+The Rx direction is not implemented in this set of patches. + +The connection migration scenario is not handled by the kernel code and will +be handled by the user space portion of QUIC library. On the Tx direction, +the new key would be installed before a packet with an updated destination is +sent. On the Rx direction, the behavior will be to drop a packet if a flow is +missing. + +For the key rotation, the behavior is to drop packets on Tx when the encryption +key with matching key rotation bit is not present. On Rx direction, the packet +will be sent to the userspace library with unencrypted header and encrypted +payload. A separate indication will be added to the ancillary data to indicate +the status of the operation as not matching the current key bit. It is not +possible to use the key rotation bit as part of the key for flow lookup as that +bit is protected by the header protection. A special provision will need to be +done in user mode to still attempt the decryption of the payload to prevent a +timing attack. + + +User Interface +============== + +Creating a QUIC connection +-------------------------- + +QUIC connection originates and terminates in the application, using one of many +available QUIC libraries. The code instantiates QUIC client and QUIC server in +some form and configures them to use certain addresses and ports for the +source and destination. The client and server negotiate the set of keys to +protect the communication during different phases of the connection, maintain +the connection and perform congestion control. + +Requesting to add QUIC Tx kernel encryption to the connection +------------------------------------------------------------- + +Each flow that should be encrypted by the kernel needs to be registered with +the kernel using socket API. A setsockopt() call on the socket creates an +association between the QUIC connection ID of the flow with the encryption +parameters for the crypto operations: + +.. code-block:: c + + struct quic_connection_info conn_info; + char conn_id[5] = {0x01, 0x02, 0x03, 0x04, 0x05}; + const size_t conn_id_len = sizeof(conn_id); + char conn_key[16] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f}; + char conn_iv[12] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b}; + char conn_hdr_key[16] = {0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, + 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f + }; + + conn_info.conn_payload_key_gen = 0; + conn_info.cipher_type = TLS_CIPHER_AES_GCM_128; + + memset(&conn_info.key, 0, sizeof(struct quic_connection_info_key)); + conn_info.key.conn_id_length = 5; + memcpy(&conn_info.key.conn_id[QUIC_MAX_CONNECTION_ID_SIZE + - conn_id_len], + &conn_id, conn_id_len); + + memcpy(&conn_info.payload_key, conn_key, sizeof(conn_key)); + memcpy(&conn_info.payload_iv, conn_iv, sizeof(conn_iv)); + memcpy(&conn_info.header_key, conn_hdr_key, sizeof(conn_hdr_key)); + + setsockopt(fd, SOL_UDP, UDP_QUIC_ADD_TX_CONNECTION, &conn_info, + sizeof(conn_info)); + + +Requesting to remove QUIC Tx kernel crypto offload control messages +------------------------------------------------------------------- + +All flows are removed when the socket is closed. To request an explicit remove +of the offload for the connection during the lifetime of the socket the process +is similar to adding the flow. Only the connection ID and its length are +necessary to supply to remove the connection from the offload: + +.. code-block:: c + + memset(&conn_info.key, 0, sizeof(struct quic_connection_info_key)); + conn_info.key.conn_id_length = 5; + memcpy(&conn_info.key.conn_id[QUIC_MAX_CONNECTION_ID_SIZE + - conn_id_len], + &conn_id, conn_id_len); + setsockopt(fd, SOL_UDP, UDP_QUIC_DEL_TX_CONNECTION, &conn_info, + sizeof(conn_info)); + +Sending QUIC application data +----------------------------- + +For QUIC Tx encryption offload, the application should use sendmsg() socket +call and provide ancillary data with information on connection ID length and +offload flags for the kernel to perform the encryption and GSO support if +requested. + +.. code-block:: c + + size_t cmsg_tx_len = sizeof(struct quic_tx_ancillary_data); + uint8_t cmsg_buf[CMSG_SPACE(cmsg_tx_len)]; + struct quic_tx_ancillary_data * anc_data; + size_t quic_data_len = 4500; + struct cmsghdr * cmsg_hdr; + char quic_data[9000]; + struct iovec iov[2]; + int send_len = 9000; + struct msghdr msg; + int err; + + iov[0].iov_base = quic_data; + iov[0].iov_len = quic_data_len; + iov[1].iov_base = quic_data + 4500; + iov[1].iov_len = quic_data_len; + + if (client.addr.sin_family == AF_INET) { + msg.msg_name = &client.addr; + msg.msg_namelen = sizeof(client.addr); + } else { + msg.msg_name = &client.addr6; + msg.msg_namelen = sizeof(client.addr6); + } + + msg.msg_iov = iov; + msg.msg_iovlen = 2; + msg.msg_control = cmsg_buf; + msg.msg_controllen = sizeof(cmsg_buf); + cmsg_hdr = CMSG_FIRSTHDR(&msg); + cmsg_hdr->cmsg_level = IPPROTO_UDP; + cmsg_hdr->cmsg_type = UDP_QUIC_ENCRYPT; + cmsg_hdr->cmsg_len = CMSG_LEN(cmsg_tx_len); + anc_data = CMSG_DATA(cmsg_hdr); + anc_data->flags = 0; + anc_data->next_pkt_num = 0x0d65c9; + anc_data->conn_id_length = conn_id_len; + err = sendmsg(self->sfd, &msg, 0); + +QUIC Tx offload in kernel will read the data from userspace, encrypt and +copy it to the ciphertext within the same operation. + + +Sending QUIC application data with GSO +-------------------------------------- +When GSO is in use, the kernel will use the GSO fragment size as the target +for ciphertext. The packets from the user space should align on the boundary +of GSO fragment size minus the size of the tag for the chosen cipher. For the +GSO fragment 1200, the plain packets should follow each other at every 1184 +bytes, given the tag size of 16. After the encryption, the rest of the UDP +and IP stacks will follow the defined value of GSO fragment which will include +the trailing tag bytes. + +To set up GSO fragmentation: + +.. code-block:: c + + setsockopt(self->sfd, SOL_UDP, UDP_SEGMENT, &frag_size, + sizeof(frag_size)); + +If the GSO fragment size is provided in ancillary data within the sendmsg() +call, the value in ancillary data will take precedence over the segment size +provided in setsockopt to split the payload into packets. This is consistent +with the UDP stack behavior. + +Integrating to userspace QUIC libraries +--------------------------------------- + +Userspace QUIC libraries integration would depend on the implementation of the +QUIC protocol. For MVFST library, the control plane is integrated into the +handshake callbacks to properly configure the flows into the socket; and the +data plane is integrated into the methods that perform encryption and send +the packets to the batch scheduler for transmissions to the socket. + +MVFST library can be found at https://github.com/facebookincubator/mvfst. + +Statistics +==========
Review
Quoted 10 lines
(quoted diff) +10
-0+ +QUIC Tx offload to the kernel has counters +(``/proc/net/quic_stat``): + +- ``QuicCurrTxSw`` - + number of currently active kernel offloaded QUIC connections +- ``QuicTxSw`` - + accumulative total number of offloaded QUIC connections +- ``QuicTxSwError`` - + accumulative total number of errors during QUIC Tx offload to kernel
The rest of documentation can be improved, like:
---- >8 ----
Documentation/networking/quic.rst +31 -27
@@ -9,22 +9,22 @@ Overview QUIC is a secure general-purpose transport protocol that creates a stateful interaction between a client and a server. QUIC provides end-to-end integrity -and confidentiality. Refer to RFC 9000 for more information on QUIC. +and confidentiality. Refer to RFC 9000 [#rfc9000]_ for the standard document. The kernel Tx side offload covers the encryption of the application streams in the kernel rather than in the application. These packets are 1RTT packets in QUIC connection. Encryption of every other packets is still done by the -QUIC library in user space. +QUIC library in userspace. The flow match is performed using 5 parameters: source and destination IP addresses, source and destination UDP ports and destination QUIC connection ID. -Not all 5 parameters are always needed. The Tx direction matches the flow on -the destination IP, port and destination connection ID, while the Rx part would -later match on source IP, port and destination connection ID. This will cover -multiple scenarios where the server is using SO_REUSEADDR and/or empty -destination connection IDs or combination of these. +Not all these parameters are always needed. The Tx direction matches the flow +on the destination IP, port and destination connection ID; while the Rx +direction would later match on source IP, port and destination connection ID. +This will cover multiple scenarios where the server is using ``SO_REUSEADDR`` +and/or empty destination connection IDs or combination of these. -The Rx direction is not implemented in this set of patches. +The Rx direction is not implemented yet. The connection migration scenario is not handled by the kernel code and will be handled by the user space portion of QUIC library. On the Tx direction,
@@ -39,8 +39,8 @@ payload. A separate indication will be added to the ancillary data to indicate the status of the operation as not matching the current key bit. It is not possible to use the key rotation bit as part of the key for flow lookup as that bit is protected by the header protection. A special provision will need to be -done in user mode to still attempt the decryption of the payload to prevent a -timing attack. +done in user mode to keep attempting the payload decription to prevent timing +attacks. User Interface
@@ -50,7 +50,7 @@ Creating a QUIC connection -------------------------- QUIC connection originates and terminates in the application, using one of many -available QUIC libraries. The code instantiates QUIC client and QUIC server in +available QUIC libraries. The code instantiates the client and server in some form and configures them to use certain addresses and ports for the source and destination. The client and server negotiate the set of keys to protect the communication during different phases of the connection, maintain
@@ -60,7 +60,7 @@ Requesting to add QUIC Tx kernel encryption to the connection ------------------------------------------------------------- Each flow that should be encrypted by the kernel needs to be registered with -the kernel using socket API. A setsockopt() call on the socket creates an +the kernel using socket API. A ``setsockopt()`` call on the socket creates an association between the QUIC connection ID of the flow with the encryption parameters for the crypto operations:
@@ -112,10 +112,10 @@ necessary to supply to remove the connection from the offload: setsockopt(fd, SOL_UDP, UDP_QUIC_DEL_TX_CONNECTION, &conn_info, sizeof(conn_info)); -Sending QUIC application data ------------------------------ +Sending application data +------------------------ -For QUIC Tx encryption offload, the application should use sendmsg() socket +For Tx encryption offload, the application should use ``sendmsg()`` socket call and provide ancillary data with information on connection ID length and offload flags for the kernel to perform the encryption and GSO support if requested.
@@ -168,11 +168,11 @@ Sending QUIC application data with GSO -------------------------------------- When GSO is in use, the kernel will use the GSO fragment size as the target for ciphertext. The packets from the user space should align on the boundary -of GSO fragment size minus the size of the tag for the chosen cipher. For the -GSO fragment 1200, the plain packets should follow each other at every 1184 -bytes, given the tag size of 16. After the encryption, the rest of the UDP -and IP stacks will follow the defined value of GSO fragment which will include -the trailing tag bytes. +of the fragment size minus the tag size for the chosen cipher. For example, +if the fragment size is 1200 bytes and the tag size is 16 bytes, the plain +packets should follow each other at every 1184 bytes. After the encryption, +the rest of UDP and IP stacks will follow the defined value of the fragment, +which includes the trailing tag bytes. To set up GSO fragmentation:
@@ -181,7 +181,7 @@ To set up GSO fragmentation: setsockopt(self->sfd, SOL_UDP, UDP_SEGMENT, &frag_size, sizeof(frag_size)); -If the GSO fragment size is provided in ancillary data within the sendmsg() +If the fragment size is provided in ancillary data within the ``sendmsg()`` call, the value in ancillary data will take precedence over the segment size provided in setsockopt to split the payload into packets. This is consistent with the UDP stack behavior.
@@ -190,12 +190,10 @@ Integrating to userspace QUIC libraries --------------------------------------- Userspace QUIC libraries integration would depend on the implementation of the -QUIC protocol. For MVFST library, the control plane is integrated into the -handshake callbacks to properly configure the flows into the socket; and the -data plane is integrated into the methods that perform encryption and send -the packets to the batch scheduler for transmissions to the socket. - -MVFST library can be found at https://github.com/facebookincubator/mvfst. +QUIC protocol. For MVFST library [#mvfst]_, the control plane is integrated +into the handshake callbacks to properly configure the flows into the socket; +and the data plane is integrated into the methods that perform encryption +and send the packets to the batch scheduler for transmissions to the socket. Statistics ==========
@@ -209,3 +207,9 @@ QUIC Tx offload to the kernel has counters accumulative total number of offloaded QUIC connections - ``QuicTxSwError`` - accumulative total number of errors during QUIC Tx offload to kernel + +References +========== + +.. [#rfc9000] https://datatracker.ietf.org/doc/html/rfc9000 +.. [#mvfst] https://github.com/facebookincubator/mvfst
Thanks.
Signature · 1 line
An old man doll... just what I always wanted! - Clara
Re: [net-next v3 1/6] net: Documentation on QUIC kernel Tx crypto.
Quoted 18 lines · Bagas Sanjaya
Quoted 17 lines · Adel Abouchaev
+===========
(quoted diff) +16
-0+KERNEL QUIC +=========== + +Overview +======== + +QUIC is a secure general-purpose transport protocol that creates a stateful +interaction between a client and a server. QUIC provides end-to-end integrity +and confidentiality. Refer to RFC 9000 for more information on QUIC. + +The kernel Tx side offload covers the encryption of the application streams +in the kernel rather than in the application. These packets are 1RTT packets +in QUIC connection. Encryption of every other packets is still done by the +QUIC library in user space. + +The flow match is performed using 5 parameters: source and destination IP
Quoted 8 lines
Review
Quoted 7 lines
(quoted diff) +7
-0+addresses, source and destination UDP ports and destination QUIC connection ID. +Not all 5 parameters are always needed. The Tx direction matches the flow on +the destination IP, port and destination connection ID, while the Rx part would +later match on source IP, port and destination connection ID. This will cover +multiple scenarios where the server is using SO_REUSEADDR and/or empty +destination connection IDs or combination of these. +Both Tx and Rx direction match destination connection ID. Is it right?
That is correct. The QUIC packet only carries the destination CID in its header.
Although the Tx direction could have an ancillary data carrying the source CID,
it is not required by any viable use case scenario.
Thank you for looking at the doc, I will add the documentation changes into the
v4 update.
Quoted 322 lines
Quoted 176 lines
(quoted diff) +175
-0+The Rx direction is not implemented in this set of patches. + +The connection migration scenario is not handled by the kernel code and will +be handled by the user space portion of QUIC library. On the Tx direction, +the new key would be installed before a packet with an updated destination is +sent. On the Rx direction, the behavior will be to drop a packet if a flow is +missing. + +For the key rotation, the behavior is to drop packets on Tx when the encryption +key with matching key rotation bit is not present. On Rx direction, the packet +will be sent to the userspace library with unencrypted header and encrypted +payload. A separate indication will be added to the ancillary data to indicate +the status of the operation as not matching the current key bit. It is not +possible to use the key rotation bit as part of the key for flow lookup as that +bit is protected by the header protection. A special provision will need to be +done in user mode to still attempt the decryption of the payload to prevent a +timing attack. + + +User Interface +============== + +Creating a QUIC connection +-------------------------- + +QUIC connection originates and terminates in the application, using one of many +available QUIC libraries. The code instantiates QUIC client and QUIC server in +some form and configures them to use certain addresses and ports for the +source and destination. The client and server negotiate the set of keys to +protect the communication during different phases of the connection, maintain +the connection and perform congestion control. + +Requesting to add QUIC Tx kernel encryption to the connection +------------------------------------------------------------- + +Each flow that should be encrypted by the kernel needs to be registered with +the kernel using socket API. A setsockopt() call on the socket creates an +association between the QUIC connection ID of the flow with the encryption +parameters for the crypto operations: + +.. code-block:: c + + struct quic_connection_info conn_info; + char conn_id[5] = {0x01, 0x02, 0x03, 0x04, 0x05}; + const size_t conn_id_len = sizeof(conn_id); + char conn_key[16] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f}; + char conn_iv[12] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b}; + char conn_hdr_key[16] = {0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, + 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f + }; + + conn_info.conn_payload_key_gen = 0; + conn_info.cipher_type = TLS_CIPHER_AES_GCM_128; + + memset(&conn_info.key, 0, sizeof(struct quic_connection_info_key)); + conn_info.key.conn_id_length = 5; + memcpy(&conn_info.key.conn_id[QUIC_MAX_CONNECTION_ID_SIZE + - conn_id_len], + &conn_id, conn_id_len); + + memcpy(&conn_info.payload_key, conn_key, sizeof(conn_key)); + memcpy(&conn_info.payload_iv, conn_iv, sizeof(conn_iv)); + memcpy(&conn_info.header_key, conn_hdr_key, sizeof(conn_hdr_key)); + + setsockopt(fd, SOL_UDP, UDP_QUIC_ADD_TX_CONNECTION, &conn_info, + sizeof(conn_info)); + + +Requesting to remove QUIC Tx kernel crypto offload control messages +------------------------------------------------------------------- + +All flows are removed when the socket is closed. To request an explicit remove +of the offload for the connection during the lifetime of the socket the process +is similar to adding the flow. Only the connection ID and its length are +necessary to supply to remove the connection from the offload: + +.. code-block:: c + + memset(&conn_info.key, 0, sizeof(struct quic_connection_info_key)); + conn_info.key.conn_id_length = 5; + memcpy(&conn_info.key.conn_id[QUIC_MAX_CONNECTION_ID_SIZE + - conn_id_len], + &conn_id, conn_id_len); + setsockopt(fd, SOL_UDP, UDP_QUIC_DEL_TX_CONNECTION, &conn_info, + sizeof(conn_info)); + +Sending QUIC application data +----------------------------- + +For QUIC Tx encryption offload, the application should use sendmsg() socket +call and provide ancillary data with information on connection ID length and +offload flags for the kernel to perform the encryption and GSO support if +requested. + +.. code-block:: c + + size_t cmsg_tx_len = sizeof(struct quic_tx_ancillary_data); + uint8_t cmsg_buf[CMSG_SPACE(cmsg_tx_len)]; + struct quic_tx_ancillary_data * anc_data; + size_t quic_data_len = 4500; + struct cmsghdr * cmsg_hdr; + char quic_data[9000]; + struct iovec iov[2]; + int send_len = 9000; + struct msghdr msg; + int err; + + iov[0].iov_base = quic_data; + iov[0].iov_len = quic_data_len; + iov[1].iov_base = quic_data + 4500; + iov[1].iov_len = quic_data_len; + + if (client.addr.sin_family == AF_INET) { + msg.msg_name = &client.addr; + msg.msg_namelen = sizeof(client.addr); + } else { + msg.msg_name = &client.addr6; + msg.msg_namelen = sizeof(client.addr6); + } + + msg.msg_iov = iov; + msg.msg_iovlen = 2; + msg.msg_control = cmsg_buf; + msg.msg_controllen = sizeof(cmsg_buf); + cmsg_hdr = CMSG_FIRSTHDR(&msg); + cmsg_hdr->cmsg_level = IPPROTO_UDP; + cmsg_hdr->cmsg_type = UDP_QUIC_ENCRYPT; + cmsg_hdr->cmsg_len = CMSG_LEN(cmsg_tx_len); + anc_data = CMSG_DATA(cmsg_hdr); + anc_data->flags = 0; + anc_data->next_pkt_num = 0x0d65c9; + anc_data->conn_id_length = conn_id_len; + err = sendmsg(self->sfd, &msg, 0); + +QUIC Tx offload in kernel will read the data from userspace, encrypt and +copy it to the ciphertext within the same operation. + + +Sending QUIC application data with GSO +-------------------------------------- +When GSO is in use, the kernel will use the GSO fragment size as the target +for ciphertext. The packets from the user space should align on the boundary +of GSO fragment size minus the size of the tag for the chosen cipher. For the +GSO fragment 1200, the plain packets should follow each other at every 1184 +bytes, given the tag size of 16. After the encryption, the rest of the UDP +and IP stacks will follow the defined value of GSO fragment which will include +the trailing tag bytes. + +To set up GSO fragmentation: + +.. code-block:: c + + setsockopt(self->sfd, SOL_UDP, UDP_SEGMENT, &frag_size, + sizeof(frag_size)); + +If the GSO fragment size is provided in ancillary data within the sendmsg() +call, the value in ancillary data will take precedence over the segment size +provided in setsockopt to split the payload into packets. This is consistent +with the UDP stack behavior. + +Integrating to userspace QUIC libraries +--------------------------------------- + +Userspace QUIC libraries integration would depend on the implementation of the +QUIC protocol. For MVFST library, the control plane is integrated into the +handshake callbacks to properly configure the flows into the socket; and the +data plane is integrated into the methods that perform encryption and send +the packets to the batch scheduler for transmissions to the socket. + +MVFST library can be found at https://github.com/facebookincubator/mvfst. + +Statistics +==========Review
Quoted 10 lines
(quoted diff) +10
-0+ +QUIC Tx offload to the kernel has counters +(``/proc/net/quic_stat``): + +- ``QuicCurrTxSw`` - + number of currently active kernel offloaded QUIC connections +- ``QuicTxSw`` - + accumulative total number of offloaded QUIC connections +- ``QuicTxSwError`` - + accumulative total number of errors during QUIC Tx offload to kernelThe rest of documentation can be improved, like:
---- >8 ----
Documentation/networking/quic.rst +31
-27@@ -9,22 +9,22 @@ Overview QUIC is a secure general-purpose transport protocol that creates a stateful interaction between a client and a server. QUIC provides end-to-end integrity -and confidentiality. Refer to RFC 9000 for more information on QUIC. +and confidentiality. Refer to RFC 9000 [#rfc9000]_ for the standard document. The kernel Tx side offload covers the encryption of the application streams in the kernel rather than in the application. These packets are 1RTT packets in QUIC connection. Encryption of every other packets is still done by the -QUIC library in user space. +QUIC library in userspace. The flow match is performed using 5 parameters: source and destination IP addresses, source and destination UDP ports and destination QUIC connection ID. -Not all 5 parameters are always needed. The Tx direction matches the flow on -the destination IP, port and destination connection ID, while the Rx part would -later match on source IP, port and destination connection ID. This will cover -multiple scenarios where the server is using SO_REUSEADDR and/or empty -destination connection IDs or combination of these. +Not all these parameters are always needed. The Tx direction matches the flow +on the destination IP, port and destination connection ID; while the Rx +direction would later match on source IP, port and destination connection ID. +This will cover multiple scenarios where the server is using ``SO_REUSEADDR`` +and/or empty destination connection IDs or combination of these. -The Rx direction is not implemented in this set of patches. +The Rx direction is not implemented yet. The connection migration scenario is not handled by the kernel code and will be handled by the user space portion of QUIC library. On the Tx direction,@@ -39,8 +39,8 @@ payload. A separate indication will be added to the ancillary data to indicate the status of the operation as not matching the current key bit. It is not possible to use the key rotation bit as part of the key for flow lookup as that bit is protected by the header protection. A special provision will need to be -done in user mode to still attempt the decryption of the payload to prevent a -timing attack. +done in user mode to keep attempting the payload decription to prevent timing +attacks. User Interface@@ -50,7 +50,7 @@ Creating a QUIC connection -------------------------- QUIC connection originates and terminates in the application, using one of many -available QUIC libraries. The code instantiates QUIC client and QUIC server in +available QUIC libraries. The code instantiates the client and server in some form and configures them to use certain addresses and ports for the source and destination. The client and server negotiate the set of keys to protect the communication during different phases of the connection, maintain@@ -60,7 +60,7 @@ Requesting to add QUIC Tx kernel encryption to the connection ------------------------------------------------------------- Each flow that should be encrypted by the kernel needs to be registered with -the kernel using socket API. A setsockopt() call on the socket creates an +the kernel using socket API. A ``setsockopt()`` call on the socket creates an association between the QUIC connection ID of the flow with the encryption parameters for the crypto operations:@@ -112,10 +112,10 @@ necessary to supply to remove the connection from the offload: setsockopt(fd, SOL_UDP, UDP_QUIC_DEL_TX_CONNECTION, &conn_info, sizeof(conn_info)); -Sending QUIC application data ------------------------------ +Sending application data +------------------------ -For QUIC Tx encryption offload, the application should use sendmsg() socket +For Tx encryption offload, the application should use ``sendmsg()`` socket call and provide ancillary data with information on connection ID length and offload flags for the kernel to perform the encryption and GSO support if requested.@@ -168,11 +168,11 @@ Sending QUIC application data with GSO -------------------------------------- When GSO is in use, the kernel will use the GSO fragment size as the target for ciphertext. The packets from the user space should align on the boundary -of GSO fragment size minus the size of the tag for the chosen cipher. For the -GSO fragment 1200, the plain packets should follow each other at every 1184 -bytes, given the tag size of 16. After the encryption, the rest of the UDP -and IP stacks will follow the defined value of GSO fragment which will include -the trailing tag bytes. +of the fragment size minus the tag size for the chosen cipher. For example, +if the fragment size is 1200 bytes and the tag size is 16 bytes, the plain +packets should follow each other at every 1184 bytes. After the encryption, +the rest of UDP and IP stacks will follow the defined value of the fragment, +which includes the trailing tag bytes. To set up GSO fragmentation:@@ -181,7 +181,7 @@ To set up GSO fragmentation: setsockopt(self->sfd, SOL_UDP, UDP_SEGMENT, &frag_size, sizeof(frag_size)); -If the GSO fragment size is provided in ancillary data within the sendmsg() +If the fragment size is provided in ancillary data within the ``sendmsg()`` call, the value in ancillary data will take precedence over the segment size provided in setsockopt to split the payload into packets. This is consistent with the UDP stack behavior.@@ -190,12 +190,10 @@ Integrating to userspace QUIC libraries --------------------------------------- Userspace QUIC libraries integration would depend on the implementation of the -QUIC protocol. For MVFST library, the control plane is integrated into the -handshake callbacks to properly configure the flows into the socket; and the -data plane is integrated into the methods that perform encryption and send -the packets to the batch scheduler for transmissions to the socket. - -MVFST library can be found at https://github.com/facebookincubator/mvfst. +QUIC protocol. For MVFST library [#mvfst]_, the control plane is integrated +into the handshake callbacks to properly configure the flows into the socket; +and the data plane is integrated into the methods that perform encryption +and send the packets to the batch scheduler for transmissions to the socket. Statistics ==========@@ -209,3 +207,9 @@ QUIC Tx offload to the kernel has counters accumulative total number of offloaded QUIC connections - ``QuicTxSwError`` - accumulative total number of errors during QUIC Tx offload to kernel + +References +========== + +.. [#rfc9000] https://datatracker.ietf.org/doc/html/rfc9000 +.. [#mvfst] https://github.com/facebookincubator/mvfst
Quoted 2 lines
Thanks.
Cheers,
Adel.
PATCH v3 2/6 net: Define QUIC specific constants, control and data plane structures
In reply to Adel AbouchaevDefine control and data plane structures to pass in control plane for flow add/remove and during packet send within ancillary data. Define constants to use within SOL_UDP to program QUIC sockets.
---
v3: added a 3-tuple to map a flow to a key, added key generation to include into flow context.
--- include/uapi/linux/quic.h | 66 +++++++++++++++++++++++++++++++++++++++ include/uapi/linux/udp.h | 3 ++ 2 files changed, 69 insertions(+) create mode 100644 include/uapi/linux/quic.h
include/uapi/linux/quic.h new file +66 -0
@@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) */ + +#ifndef _UAPI_LINUX_QUIC_H +#define _UAPI_LINUX_QUIC_H + +#include <linux/types.h> +#include <linux/tls.h> + +#define QUIC_MAX_CONNECTION_ID_SIZE 20 + +/* Side by side data for QUIC egress operations */ +#define QUIC_BYPASS_ENCRYPTION 0x01 + +struct quic_tx_ancillary_data { +__aligned_u64 next_pkt_num; +__u8 flags; +__u8 dst_conn_id_length; +}; + +struct quic_connection_info_key { +__u8 dst_conn_id[QUIC_MAX_CONNECTION_ID_SIZE ]; +__u8 dst_conn_id_length; +union { +struct in6_addr ipv6_addr; +struct in_addr ipv4_addr; + } addr; +__be16 udp_port; +}; + +struct quic_aes_gcm_128 { +__u8 header_key[TLS_CIPHER_AES_GCM_128_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_GCM_128_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_GCM_128_IV_SIZE ]; +}; + +struct quic_aes_gcm_256 { +__u8 header_key[TLS_CIPHER_AES_GCM_256_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_GCM_256_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_GCM_256_IV_SIZE ]; +}; + +struct quic_aes_ccm_128 { +__u8 header_key[TLS_CIPHER_AES_CCM_128_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_CCM_128_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_CCM_128_IV_SIZE ]; +}; + +struct quic_chacha20_poly1305 { +__u8 header_key[TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_CHACHA20_POLY1305_IV_SIZE ]; +}; + +struct quic_connection_info { +__u16 cipher_type; +struct quic_connection_info_key key; +__u8 conn_payload_key_gen; +union { +struct quic_aes_gcm_128 aes_gcm_128; +struct quic_aes_gcm_256 aes_gcm_256; +struct quic_aes_ccm_128 aes_ccm_128; +struct quic_chacha20_poly1305 chacha20_poly1305; + }; +}; + +#endif
include/uapi/linux/udp.h +3 -0
@@ -34,6 +34,9 @@ struct udphdr {#define UDP_NO_CHECK6_RX 102/* Disable accpeting checksum for UDP6 */ #define UDP_SEGMENT 103/* Set GSO segmentation size */ #define UDP_GRO 104/* This socket can receive UDP GRO packets */ +#define UDP_QUIC_ADD_TX_CONNECTION 106/* Add QUIC Tx crypto offload */ +#define UDP_QUIC_DEL_TX_CONNECTION 107/* Del QUIC Tx crypto offload */ +#define UDP_QUIC_ENCRYPT 108/* QUIC encryption parameters */ /* UDP encapsulation types */ #define UDP_ENCAP_ESPINUDP_NON_IKE 1/* draft-ietf-ipsec-nat-t-ike-00/01 */
Signature · 1 line
2.30.2
PATCH v3 3/6 net: Add UDP ULP operations, initialization and handling prototype functions.
In reply to Adel AbouchaevDefine functions to add UDP ULP handling, registration with UDP protocol and supporting data structures. Create structure for QUIC ULP and add empty prototype functions to support it.
---
Removed reference to net/quic/Kconfig from this patch into the next.
Fixed formatting around brackets.
--- include/net/inet_sock.h | 2 + include/net/udp.h | 33 +++++++ include/uapi/linux/udp.h | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 6 ++ net/ipv4/udp_ulp.c | 192 +++++++++++++++++++++++++++++++++++++++ 7 files changed, 237 insertions(+), 1 deletion(-) create mode 100644 net/ipv4/udp_ulp.c
include/net/inet_sock.h +2 -0
@@ -249,6 +249,8 @@ struct inet_sock {__be32 mc_addr;struct ip_mc_socklist __rcu *mc_list;struct inet_cork_full cork; +const struct udp_ulp_ops *udp_ulp_ops; +void __rcu *ulp_data; };#define IPCORK_OPT 1/* ip-options has been held in ipcork.opt */
include/net/udp.h +33 -0
@@ -523,4 +523,37 @@ struct proto *udp_bpf_get_proto(struct sock *sk, struct sk_psock *psock);int udp_bpf_update_proto (struct sock *sk,struct sk_psock *psock,bool restore);#endif +/* +* Interface for adding Upper Level Protocols over UDP +*/ + +#define UDP_ULP_NAME_MAX 16 +#define UDP_ULP_MAX 128 + +struct udp_ulp_ops { +struct list_head list; + +/* initialize ulp */ +int (*init)(struct sock *sk); +/* cleanup ulp */ +void (*release)(struct sock *sk); + +char name[UDP_ULP_NAME_MAX ]; +struct module *owner; +}; + +int udp_register_ulp (struct udp_ulp_ops *type); +void udp_unregister_ulp (struct udp_ulp_ops *type); +int udp_set_ulp (struct sock *sk,const char *name); +void udp_get_available_ulp (char *buf,size_t len); +void udp_cleanup_ulp (struct sock *sk); +int udp_setsockopt_ulp (struct sock *sk,sockptr_t optval, +unsigned int optlen); +int udp_getsockopt_ulp (struct sock *sk,char __user *optval, +int __user *optlen); + +#define MODULE_ALIAS_UDP_ULP (name)\ + __MODULE_INFO(alias, alias_userspace, name);\ + __MODULE_INFO(alias, alias_udp_ulp, "udp-ulp-" name) +#endif /* _UDP_H */
include/uapi/linux/udp.h +1 -0
@@ -34,6 +34,7 @@ struct udphdr {#define UDP_NO_CHECK6_RX 102/* Disable accpeting checksum for UDP6 */ #define UDP_SEGMENT 103/* Set GSO segmentation size */ #define UDP_GRO 104/* This socket can receive UDP GRO packets */ +#define UDP_ULP 105/* Attach ULP to a UDP socket */ #define UDP_QUIC_ADD_TX_CONNECTION 106/* Add QUIC Tx crypto offload */ #define UDP_QUIC_DEL_TX_CONNECTION 107/* Del QUIC Tx crypto offload */ #define UDP_QUIC_ENCRYPT 108/* QUIC encryption parameters */
net/Makefile +1 -0
@@ -16,6 +16,7 @@ obj-y += ethernet/ 802/ sched/ netlink/ bpf/ ethtool/ obj-$(CONFIG_NETFILTER) += netfilter/ obj-$(CONFIG_INET) += ipv4/ obj-$(CONFIG_TLS) += tls/ +obj-$(CONFIG_QUIC) += quic/ obj-$(CONFIG_XFRM) += xfrm/ obj-$(CONFIG_UNIX_SCM) += unix/ obj-y += ipv6/
net/ipv4/Makefile +2 -1
@@ -14,7 +14,8 @@ obj-y := route.o inetpeer.o protocol.o \ udp_offload.o arp.o icmp.o devinet.o af_inet.o igmp.o \ fib_frontend.o fib_semantics.o fib_trie.o fib_notifier.o \ inet_fragment.o ping.o ip_tunnel_core.o gre_offload.o \ - metrics.o netlink.o nexthop.o udp_tunnel_stub.o + metrics.o netlink.o nexthop.o udp_tunnel_stub.o \ + udp_ulp.o obj-$(CONFIG_BPFILTER) += bpfilter/
net/ipv4/udp.c +6 -0
@@ -2779,6 +2779,9 @@ int udp_lib_setsockopt(struct sock *sk, int level, int optname, up->pcflag |=UDPLITE_RECV_CC ;break ; +case UDP_ULP : +return udp_setsockopt_ulp (sk, optval, optlen); +default : err = -ENOPROTOOPT ;break ;
@@ -2847,6 +2850,9 @@ int udp_lib_getsockopt(struct sock *sk, int level, int optname, val = up->pcrlen;break ; +case UDP_ULP : +return udp_getsockopt_ulp (sk, optval, optlen); +default :return -ENOPROTOOPT ; }
net/ipv4/udp_ulp.c new file +192 -0
192 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH v3 4/6 net: Implement QUIC offload functions
In reply to Adel AbouchaevAdd connection hash to the context do support add, remove operations on QUIC connections for the control plane and lookup for the data plane. Implement setsockopt and add placeholders to add and delete Tx connections.
---
Added net/quic/Kconfig reference to net/Kconfig in this commit.
Initialized pointers with NULL vs 0. Restricted AES counter to __le32 Added address space qualifiers to user space addresses. Removed empty lines. Updated code alignment. Removed inlines.
v3: removed ITER_KVEC flag from iov_iter_kvec call.
v3: fixed Chacha20 encryption bug.
v3: updated to match the uAPI struct fields
v3: updated Tx flow to match on dst ip, dst port and connection id.
v3: updated to drop packets if key generations do not match.
--- include/net/quic.h | 53 ++ net/Kconfig | 1 + net/ipv4/udp.c | 9 + net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1487 ++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 1574 insertions(+) create mode 100644 include/net/quic.h create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c
include/net/quic.h new file +53 -0
@@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef INCLUDE_NET_QUIC_H +#define INCLUDE_NET_QUIC_H + +#include <linux/mutex.h> +#include <linux/rhashtable.h> +#include <linux/skmsg.h> +#include <uapi/linux/quic.h> + +#define QUIC_MAX_SHORT_HEADER_SIZE 25 +#define QUIC_MAX_CONNECTION_ID_SIZE 20 +#define QUIC_HDR_MASK_SIZE 16 +#define QUIC_MAX_GSO_FRAGS 16 + +// Maximum IV and nonce sizes should be in sync with supported ciphers. +#define QUIC_CIPHER_MAX_IV_SIZE 12 +#define QUIC_CIPHER_MAX_NONCE_SIZE 16 + +/* Side by side data for QUIC egress operations */ +#define QUIC_ANCILLARY_FLAGS (QUIC_BYPASS_ENCRYPTION) + +#define QUIC_MAX_IOVEC_SEGMENTS 8 +#define QUIC_MAX_SG_ALLOC_ELEMENTS 32 +#define QUIC_MAX_PLAIN_PAGES 16 +#define QUIC_MAX_CIPHER_PAGES_ORDER 4 + +struct quic_internal_crypto_context { +struct quic_connection_info conn_info; +struct crypto_skcipher *header_tfm; +struct crypto_aead *packet_aead; +}; + +struct quic_connection_rhash { +struct rhash_head node; +struct quic_internal_crypto_context crypto_ctx; +struct rcu_head rcu; +}; + +struct quic_context { +struct proto *sk_proto; +struct rhashtable tx_connections; +struct scatterlist sg_alloc[QUIC_MAX_SG_ALLOC_ELEMENTS ]; +struct page *cipher_page; +/** +* To synchronize concurrent sendmsg() requests through the same socket +* and protect preallocated per-context memory. +**/ +struct mutex sendmsg_mux; +struct rcu_head rcu; +}; + +#endif
net/Kconfig +1 -0
@@ -63,6 +63,7 @@ menu "Networking options" source "net/packet/Kconfig" source "net/unix/Kconfig" source "net/tls/Kconfig" +source "net/quic/Kconfig" source "net/xfrm/Kconfig" source "net/iucv/Kconfig" source "net/smc/Kconfig"
net/ipv4/udp.c +9 -0
@@ -113,6 +113,7 @@#include <net/sock_reuseport.h> #include <net/addrconf.h> #include <net/udp_tunnel.h> +#include <uapi/linux/quic.h> #if IS_ENABLED (CONFIG_IPV6 )#include <net/ipv6_stubs.h> #endif
@@ -1011,6 +1012,14 @@ static int __udp_cmsg_send(struct cmsghdr *cmsg, u16 *gso_size)return -EINVAL ; *gso_size = *(__u16 *)CMSG_DATA (cmsg);return 0 ; +case UDP_QUIC_ENCRYPT : +/* This option is handled in UDP_ULP and is only checked +* here for the bypass bit +*/ +if (cmsg->cmsg_len != +CMSG_LEN (sizeof (struct quic_tx_ancillary_data ))) +return -EINVAL ; +return 0 ;default :return -EINVAL ; }
net/quic/Kconfig new file +16 -0
@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# QUIC configuration +# +config QUIC + tristate "QUIC encryption offload" + depends on INET + select CRYPTO + select CRYPTO_AES + select CRYPTO_GCM + help + Enable kernel support for QUIC crypto offload. Currently only TX + encryption offload is supported. The kernel will perform + copy-during-encryption. + + If unsure, say N.
net/quic/Makefile new file +8 -0
@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Makefile for the QUIC subsystem +# + +obj-$(CONFIG_QUIC) += quic.o + +quic-y := quic_main.o
net/quic/quic_main.c new file +1487 -0
1,487 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH v3 5/6 net: Add flow counters and Tx processing error counter
In reply to Adel AbouchaevAdded flow counters. Total flow counter is accumulative, the current shows the number of flows currently in flight, the error counters is accumulating the number of errors during Tx processing.
---
Updated enum bracket to follow enum keyword. Removed extra blank lines.
--- include/net/netns/mib.h | 3 +++ include/net/quic.h | 10 +++++++++ include/net/snmp.h | 6 +++++ include/uapi/linux/snmp.h | 9 ++++++++ net/quic/Makefile | 2 +- net/quic/quic_main.c | 46 +++++++++++++++++++++++++++++++++++++++ net/quic/quic_proc.c | 45 ++++++++++++++++++++++++++++++++++++++ 7 files changed, 120 insertions(+), 1 deletion(-) create mode 100644 net/quic/quic_proc.c
include/net/netns/mib.h +3 -0
@@ -24,6 +24,9 @@ struct netns_mib {#if IS_ENABLED (CONFIG_TLS ) DEFINE_SNMP_STAT(struct linux_tls_mib, tls_statistics);#endif +#if IS_ENABLED (CONFIG_QUIC ) + DEFINE_SNMP_STAT(struct linux_quic_mib, quic_statistics); +#endif #ifdef CONFIG_MPTCP DEFINE_SNMP_STAT(struct mptcp_mib, mptcp_statistics);#endif
include/net/quic.h +10 -0
@@ -25,6 +25,16 @@#define QUIC_MAX_PLAIN_PAGES 16#define QUIC_MAX_CIPHER_PAGES_ORDER 4 +#define __QUIC_INC_STATS (net, field) \ + __SNMP_INC_STATS((net)->mib.quic_statistics, field) +#define QUIC_INC_STATS (net, field) \ + SNMP_INC_STATS((net)->mib.quic_statistics, field) +#define QUIC_DEC_STATS (net, field) \ + SNMP_DEC_STATS((net)->mib.quic_statistics, field) + +int __net_init quic_proc_init (struct net *net); +void __net_exit quic_proc_fini(struct net *net); +struct quic_internal_crypto_context {struct quic_connection_info conn_info;struct crypto_skcipher *header_tfm;
include/net/snmp.h +6 -0
@@ -117,6 +117,12 @@ struct linux_tls_mib {unsigned long mibs[LINUX_MIB_TLSMAX ]; }; +/* Linux QUIC */ +#define LINUX_MIB_QUICMAX __LINUX_MIB_QUICMAX +struct linux_quic_mib { +unsigned long mibs[LINUX_MIB_QUICMAX ]; +}; +#define DEFINE_SNMP_STAT (type, name) \ __typeof__(type) __percpu *name#define DEFINE_SNMP_STAT_ATOMIC (type, name) \
include/uapi/linux/snmp.h +9 -0
@@ -349,4 +349,13 @@ enum__LINUX_MIB_TLSMAX }; +/* linux QUIC mib definitions */ +enum { +LINUX_MIB_QUICNUM =0 , +LINUX_MIB_QUICCURRTXSW ,/* QuicCurrTxSw */ +LINUX_MIB_QUICTXSW ,/* QuicTxSw */ +LINUX_MIB_QUICTXSWERROR ,/* QuicTxSwError */ + __LINUX_MIB_QUICMAX +}; +#endif /* _LINUX_SNMP_H */
net/quic/Makefile +1 -1
@@ -5,4 +5,4 @@ obj-$(CONFIG_QUIC) += quic.o -quic-y := quic_main.o +quic-y := quic_main.o quic_proc.o
net/quic/quic_main.c +46 -0
88 lines not on this page · raw message
net/quic/quic_proc.c new file +45 -0
45 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH v3 6/6 net: Add self tests for ULP operations, flow setup and crypto tests
In reply to Adel AbouchaevAdd self tests for ULP operations, flow setup and crypto tests.
---
Restored the test build. Changed the QUIC context reference variable names for the keys and iv to match the uAPI.
Updated alignment, added SPDX license line.
v3: Added Chacha20-Poly1305 test.
v3: Added test to fail sending with wrong key generation bit.
--- tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/Makefile | 3 +- tools/testing/selftests/net/quic.c | 1369 ++++++++++++++++++++++++ tools/testing/selftests/net/quic.sh | 46 + 4 files changed, 1418 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
tools/testing/selftests/net/.gitignore +1 -0
@@ -14,6 +14,7 @@ nettest psock_fanout psock_snd psock_tpacket +quic reuseaddr_conflict reuseaddr_ports_exhausted reuseport_addr_any
tools/testing/selftests/net/Makefile +2 -1
@@ -44,6 +44,7 @@ TEST_PROGS += arp_ndisc_untracked_subnets.sh TEST_PROGS += stress_reuseport_listen.sh TEST_PROGS += l2_tos_ttl_inherit.sh TEST_PROGS += bind_bhash.sh +TEST_PROGS += quic.sh TEST_PROGS_EXTENDED := in_netns.sh setup_loopback.sh setup_veth.sh TEST_PROGS_EXTENDED += toeplitz_client.sh toeplitz.sh TEST_GEN_FILES = socket nettest
@@ -59,7 +60,7 @@ TEST_GEN_FILES += ipsec TEST_GEN_FILES += ioam6_parser TEST_GEN_FILES += gro TEST_GEN_PROGS = reuseport_bpf reuseport_bpf_cpu reuseport_bpf_numa -TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls tun tap +TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls tun tap quic TEST_GEN_FILES += toeplitz TEST_GEN_FILES += cmsg_sender TEST_GEN_FILES += stress_reuseport_listen
tools/testing/selftests/net/quic.c new file +1369 -0
1,369 lines not on this page · raw message
tools/testing/selftests/net/quic.sh new file +46 -0
46 lines not on this page · raw message
Signature · 1 line
2.30.2
v4 cover net: support QUIC crypto
In reply to a message not in the archiveQUIC requires end to end encryption of the data. The application usually prepares the data in clear text, encrypts and calls send() which implies multiple copies of the data before the packets hit the networking stack. Similar to kTLS, QUIC kernel offload of cryptography reduces the memory pressure by reducing the number of copies.
The scope of kernel support is limited to the symmetric cryptography, leaving the handshake to the user space library. For QUIC in particular, the application packets that require symmetric cryptography are the 1RTT packets with short headers. Kernel will encrypt the application packets on transmission and decrypt on receive. This series implements Tx only, because in QUIC server applications Tx outweighs Rx by orders of magnitude.
Supporting the combination of QUIC and GSO requires the application to correctly place the data and the kernel to correctly slice it. The encryption process appends an arbitrary number of bytes (tag) to the end of the message to authenticate it. The GSO value should include this overhead, the offload would then subtract the tag size to parse the input on Tx before chunking and encrypting it.
With the kernel cryptography, the buffer copy operation is conjoined with the encryption operation. The memory bandwidth is reduced by 5-8%. When devices supporting QUIC encryption in hardware come to the market, we will be able to free further 7% of CPU utilization which is used today for crypto operations.
Adel Abouchaev (6):
Documentation on QUIC kernel Tx crypto.
Define QUIC specific constants, control and data plane structures Add UDP ULP operations, initialization and handling prototype
functions.
Implement QUIC offload functions
Add flow counters and Tx processing error counter
Add self tests for ULP operations, flow setup and crypto tests
Documentation/networking/index.rst | 1 + Documentation/networking/quic.rst | 215 ++++ include/net/inet_sock.h | 2 + include/net/netns/mib.h | 3 + include/net/quic.h | 63 + include/net/snmp.h | 6 + include/net/udp.h | 33 + include/uapi/linux/quic.h | 68 ++ include/uapi/linux/snmp.h | 9 + include/uapi/linux/udp.h | 4 + net/Kconfig | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 15 + net/ipv4/udp_ulp.c | 192 +++ net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1533 ++++++++++++++++++++++++ net/quic/quic_proc.c | 45 + security/security.c | 1 + tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/Makefile | 3 +- tools/testing/selftests/net/quic.c | 1369 +++++++++++++++++++++ tools/testing/selftests/net/quic.sh | 46 + 24 files changed, 3636 insertions(+), 2 deletions(-) create mode 100644 Documentation/networking/quic.rst create mode 100644 include/net/quic.h create mode 100644 include/uapi/linux/quic.h create mode 100644 net/ipv4/udp_ulp.c create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c create mode 100644 net/quic/quic_proc.c create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
Signature · 1 line
2.30.2
PATCH v4 1/6 net: Documentation on QUIC kernel Tx crypto.
Add documentation for kernel QUIC code.
---
Added quic.rst reference to the index.rst file; identation in quic.rst file.
Added SPDX license GPL 2.0.
v2: Removed whitespace at EOF.
v3: Added explanation of features.
v4: Updated and formatted the doc for readability.
--- Documentation/networking/index.rst | 1 + Documentation/networking/quic.rst | 215 +++++++++++++++++++++++++++++ 2 files changed, 216 insertions(+) create mode 100644 Documentation/networking/quic.rst
Documentation/networking/index.rst +1 -0
@@ -89,6 +89,7 @@ Contents: plip ppp_generic proc_net_tcp + quic radiotap-headers rds regulatory
Documentation/networking/quic.rst new file +215 -0
215 lines not on this page · raw message
Signature · 1 line
2.30.2
Re: [net-next v4 1/6] net: Documentation on QUIC kernel Tx crypto.
Quoted 2 lines · Adel Abouchaev
Add documentation for kernel QUIC code.
LGTM, thanks.
Signature · 1 line
An old man doll... just what I always wanted! - Clara
PATCH v4 2/6 net: Define QUIC specific constants, control and data plane structures
In reply to Adel AbouchaevDefine control and data plane structures to pass in control plane for flow add/remove and during packet send within ancillary data. Define constants to use within SOL_UDP to program QUIC sockets.
---
v3: added a 3-tuple to map a flow to a key, added key generation to include into flow context.
v4: added missing linux/in.h and linux/in6.h to the quic.h file.
--- include/uapi/linux/quic.h | 68 +++++++++++++++++++++++++++++++++++++++ include/uapi/linux/udp.h | 3 ++ 2 files changed, 71 insertions(+) create mode 100644 include/uapi/linux/quic.h
include/uapi/linux/quic.h new file +68 -0
@@ -0,0 +1,68 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) */ + +#ifndef _UAPI_LINUX_QUIC_H +#define _UAPI_LINUX_QUIC_H + +#include <linux/in.h> +#include <linux/in6.h> +#include <linux/types.h> +#include <linux/tls.h> + +#define QUIC_MAX_CONNECTION_ID_SIZE 20 + +/* Side by side data for QUIC egress operations */ +#define QUIC_BYPASS_ENCRYPTION 0x01 + +struct quic_tx_ancillary_data { +__aligned_u64 next_pkt_num; +__u8 flags; +__u8 dst_conn_id_length; +}; + +struct quic_connection_info_key { +__u8 dst_conn_id[QUIC_MAX_CONNECTION_ID_SIZE ]; +__u8 dst_conn_id_length; +union { +struct in6_addr ipv6_addr; +struct in_addr ipv4_addr; + } addr; +__be16 udp_port; +}; + +struct quic_aes_gcm_128 { +__u8 header_key[TLS_CIPHER_AES_GCM_128_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_GCM_128_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_GCM_128_IV_SIZE ]; +}; + +struct quic_aes_gcm_256 { +__u8 header_key[TLS_CIPHER_AES_GCM_256_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_GCM_256_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_GCM_256_IV_SIZE ]; +}; + +struct quic_aes_ccm_128 { +__u8 header_key[TLS_CIPHER_AES_CCM_128_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_AES_CCM_128_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_AES_CCM_128_IV_SIZE ]; +}; + +struct quic_chacha20_poly1305 { +__u8 header_key[TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE ]; +__u8 payload_key[TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE ]; +__u8 payload_iv[TLS_CIPHER_CHACHA20_POLY1305_IV_SIZE ]; +}; + +struct quic_connection_info { +__u16 cipher_type; +struct quic_connection_info_key key; +__u8 conn_payload_key_gen; +union { +struct quic_aes_gcm_128 aes_gcm_128; +struct quic_aes_gcm_256 aes_gcm_256; +struct quic_aes_ccm_128 aes_ccm_128; +struct quic_chacha20_poly1305 chacha20_poly1305; + }; +}; + +#endif
include/uapi/linux/udp.h +3 -0
@@ -34,6 +34,9 @@ struct udphdr {#define UDP_NO_CHECK6_RX 102/* Disable accpeting checksum for UDP6 */ #define UDP_SEGMENT 103/* Set GSO segmentation size */ #define UDP_GRO 104/* This socket can receive UDP GRO packets */ +#define UDP_QUIC_ADD_TX_CONNECTION 106/* Add QUIC Tx crypto offload */ +#define UDP_QUIC_DEL_TX_CONNECTION 107/* Del QUIC Tx crypto offload */ +#define UDP_QUIC_ENCRYPT 108/* QUIC encryption parameters */ /* UDP encapsulation types */ #define UDP_ENCAP_ESPINUDP_NON_IKE 1/* draft-ietf-ipsec-nat-t-ike-00/01 */
Signature · 1 line
2.30.2
PATCH v4 3/6 net: Add UDP ULP operations, initialization and handling prototype functions.
In reply to Adel AbouchaevDefine functions to add UDP ULP handling, registration with UDP protocol and supporting data structures. Create structure for QUIC ULP and add empty prototype functions to support it.
---
Removed reference to net/quic/Kconfig from this patch into the next.
Fixed formatting around brackets.
--- include/net/inet_sock.h | 2 + include/net/udp.h | 33 +++++++ include/uapi/linux/udp.h | 1 + net/Makefile | 1 + net/ipv4/Makefile | 3 +- net/ipv4/udp.c | 6 ++ net/ipv4/udp_ulp.c | 192 +++++++++++++++++++++++++++++++++++++++ 7 files changed, 237 insertions(+), 1 deletion(-) create mode 100644 net/ipv4/udp_ulp.c
include/net/inet_sock.h +2 -0
@@ -249,6 +249,8 @@ struct inet_sock {__be32 mc_addr;struct ip_mc_socklist __rcu *mc_list;struct inet_cork_full cork; +const struct udp_ulp_ops *udp_ulp_ops; +void __rcu *ulp_data; };#define IPCORK_OPT 1/* ip-options has been held in ipcork.opt */
include/net/udp.h +33 -0
@@ -523,4 +523,37 @@ struct proto *udp_bpf_get_proto(struct sock *sk, struct sk_psock *psock);int udp_bpf_update_proto (struct sock *sk,struct sk_psock *psock,bool restore);#endif +/* +* Interface for adding Upper Level Protocols over UDP +*/ + +#define UDP_ULP_NAME_MAX 16 +#define UDP_ULP_MAX 128 + +struct udp_ulp_ops { +struct list_head list; + +/* initialize ulp */ +int (*init)(struct sock *sk); +/* cleanup ulp */ +void (*release)(struct sock *sk); + +char name[UDP_ULP_NAME_MAX ]; +struct module *owner; +}; + +int udp_register_ulp (struct udp_ulp_ops *type); +void udp_unregister_ulp (struct udp_ulp_ops *type); +int udp_set_ulp (struct sock *sk,const char *name); +void udp_get_available_ulp (char *buf,size_t len); +void udp_cleanup_ulp (struct sock *sk); +int udp_setsockopt_ulp (struct sock *sk,sockptr_t optval, +unsigned int optlen); +int udp_getsockopt_ulp (struct sock *sk,char __user *optval, +int __user *optlen); + +#define MODULE_ALIAS_UDP_ULP (name)\ + __MODULE_INFO(alias, alias_userspace, name);\ + __MODULE_INFO(alias, alias_udp_ulp, "udp-ulp-" name) +#endif /* _UDP_H */
include/uapi/linux/udp.h +1 -0
@@ -34,6 +34,7 @@ struct udphdr {#define UDP_NO_CHECK6_RX 102/* Disable accpeting checksum for UDP6 */ #define UDP_SEGMENT 103/* Set GSO segmentation size */ #define UDP_GRO 104/* This socket can receive UDP GRO packets */ +#define UDP_ULP 105/* Attach ULP to a UDP socket */ #define UDP_QUIC_ADD_TX_CONNECTION 106/* Add QUIC Tx crypto offload */ #define UDP_QUIC_DEL_TX_CONNECTION 107/* Del QUIC Tx crypto offload */ #define UDP_QUIC_ENCRYPT 108/* QUIC encryption parameters */
net/Makefile +1 -0
@@ -16,6 +16,7 @@ obj-y += ethernet/ 802/ sched/ netlink/ bpf/ ethtool/ obj-$(CONFIG_NETFILTER) += netfilter/ obj-$(CONFIG_INET) += ipv4/ obj-$(CONFIG_TLS) += tls/ +obj-$(CONFIG_QUIC) += quic/ obj-$(CONFIG_XFRM) += xfrm/ obj-$(CONFIG_UNIX_SCM) += unix/ obj-y += ipv6/
net/ipv4/Makefile +2 -1
@@ -14,7 +14,8 @@ obj-y := route.o inetpeer.o protocol.o \ udp_offload.o arp.o icmp.o devinet.o af_inet.o igmp.o \ fib_frontend.o fib_semantics.o fib_trie.o fib_notifier.o \ inet_fragment.o ping.o ip_tunnel_core.o gre_offload.o \ - metrics.o netlink.o nexthop.o udp_tunnel_stub.o + metrics.o netlink.o nexthop.o udp_tunnel_stub.o \ + udp_ulp.o obj-$(CONFIG_BPFILTER) += bpfilter/
net/ipv4/udp.c +6 -0
@@ -2781,6 +2781,9 @@ int udp_lib_setsockopt(struct sock *sk, int level, int optname, up->pcflag |=UDPLITE_RECV_CC ;break ; +case UDP_ULP : +return udp_setsockopt_ulp (sk, optval, optlen); +default : err = -ENOPROTOOPT ;break ;
@@ -2849,6 +2852,9 @@ int udp_lib_getsockopt(struct sock *sk, int level, int optname, val = up->pcrlen;break ; +case UDP_ULP : +return udp_getsockopt_ulp (sk, optval, optlen); +default :return -ENOPROTOOPT ; }
net/ipv4/udp_ulp.c new file +192 -0
192 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH v4 4/6 net: Implement QUIC offload functions
In reply to Adel AbouchaevAdd connection hash to the context do support add, remove operations on QUIC connections for the control plane and lookup for the data plane. Implement setsockopt and add placeholders to add and delete Tx connections.
---
Added net/quic/Kconfig reference to net/Kconfig in this commit.
Initialized pointers with NULL vs 0. Restricted AES counter to __le32 Added address space qualifiers to user space addresses. Removed empty lines. Updated code alignment. Removed inlines.
v3: removed ITER_KVEC flag from iov_iter_kvec call.
v3: fixed Chacha20 encryption bug.
v3: updated to match the uAPI struct fields
v3: updated Tx flow to match on dst ip, dst port and connection id.
v3: updated to drop packets if key generations do not match.
v4: made the discovered packet size signed size_t.
v4: exported security_socket_sendmsg from security/security.c.
--- include/net/quic.h | 53 ++ net/Kconfig | 1 + net/ipv4/udp.c | 9 + net/quic/Kconfig | 16 + net/quic/Makefile | 8 + net/quic/quic_main.c | 1487 ++++++++++++++++++++++++++++++++++++++++++ security/security.c | 1 + 7 files changed, 1575 insertions(+) create mode 100644 include/net/quic.h create mode 100644 net/quic/Kconfig create mode 100644 net/quic/Makefile create mode 100644 net/quic/quic_main.c
include/net/quic.h new file +53 -0
@@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef INCLUDE_NET_QUIC_H +#define INCLUDE_NET_QUIC_H + +#include <linux/mutex.h> +#include <linux/rhashtable.h> +#include <linux/skmsg.h> +#include <uapi/linux/quic.h> + +#define QUIC_MAX_SHORT_HEADER_SIZE 25 +#define QUIC_MAX_CONNECTION_ID_SIZE 20 +#define QUIC_HDR_MASK_SIZE 16 +#define QUIC_MAX_GSO_FRAGS 16 + +// Maximum IV and nonce sizes should be in sync with supported ciphers. +#define QUIC_CIPHER_MAX_IV_SIZE 12 +#define QUIC_CIPHER_MAX_NONCE_SIZE 16 + +/* Side by side data for QUIC egress operations */ +#define QUIC_ANCILLARY_FLAGS (QUIC_BYPASS_ENCRYPTION) + +#define QUIC_MAX_IOVEC_SEGMENTS 8 +#define QUIC_MAX_SG_ALLOC_ELEMENTS 32 +#define QUIC_MAX_PLAIN_PAGES 16 +#define QUIC_MAX_CIPHER_PAGES_ORDER 4 + +struct quic_internal_crypto_context { +struct quic_connection_info conn_info; +struct crypto_skcipher *header_tfm; +struct crypto_aead *packet_aead; +}; + +struct quic_connection_rhash { +struct rhash_head node; +struct quic_internal_crypto_context crypto_ctx; +struct rcu_head rcu; +}; + +struct quic_context { +struct proto *sk_proto; +struct rhashtable tx_connections; +struct scatterlist sg_alloc[QUIC_MAX_SG_ALLOC_ELEMENTS ]; +struct page *cipher_page; +/** +* To synchronize concurrent sendmsg() requests through the same socket +* and protect preallocated per-context memory. +**/ +struct mutex sendmsg_mux; +struct rcu_head rcu; +}; + +#endif
net/Kconfig +1 -0
@@ -63,6 +63,7 @@ menu "Networking options" source "net/packet/Kconfig" source "net/unix/Kconfig" source "net/tls/Kconfig" +source "net/quic/Kconfig" source "net/xfrm/Kconfig" source "net/iucv/Kconfig" source "net/smc/Kconfig"
net/ipv4/udp.c +9 -0
@@ -113,6 +113,7 @@#include <net/sock_reuseport.h> #include <net/addrconf.h> #include <net/udp_tunnel.h> +#include <uapi/linux/quic.h> #if IS_ENABLED (CONFIG_IPV6 )#include <net/ipv6_stubs.h> #endif
@@ -1013,6 +1014,14 @@ static int __udp_cmsg_send(struct cmsghdr *cmsg, u16 *gso_size)return -EINVAL ; *gso_size = *(__u16 *)CMSG_DATA (cmsg);return 0 ; +case UDP_QUIC_ENCRYPT : +/* This option is handled in UDP_ULP and is only checked +* here for the bypass bit +*/ +if (cmsg->cmsg_len != +CMSG_LEN (sizeof (struct quic_tx_ancillary_data ))) +return -EINVAL ; +return 0 ;default :return -EINVAL ; }
net/quic/Kconfig new file +16 -0
@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# QUIC configuration +# +config QUIC + tristate "QUIC encryption offload" + depends on INET + select CRYPTO + select CRYPTO_AES + select CRYPTO_GCM + help + Enable kernel support for QUIC crypto offload. Currently only TX + encryption offload is supported. The kernel will perform + copy-during-encryption. + + If unsure, say N.
net/quic/Makefile new file +8 -0
@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Makefile for the QUIC subsystem +# + +obj-$(CONFIG_QUIC) += quic.o + +quic-y := quic_main.o
net/quic/quic_main.c new file +1487 -0
1,487 lines not on this page · raw message
security/security.c +1 -0
@@ -2222,6 +2222,7 @@ int security_socket_sendmsg(struct socket *sock, struct msghdr *msg, int size) {return call_int_hook (socket_sendmsg,0 , sock, msg, size); } +EXPORT_SYMBOL (security_socket_sendmsg);int security_socket_recvmsg (struct socket *sock,struct msghdr *msg,int size,int flags)
Signature · 1 line
2.30.2
PATCH v4 5/6 net: Add flow counters and Tx processing error counter
In reply to Adel AbouchaevAdded flow counters. Total flow counter is accumulative, the current shows the number of flows currently in flight, the error counters is accumulating the number of errors during Tx processing.
---
Updated enum bracket to follow enum keyword. Removed extra blank lines.
--- include/net/netns/mib.h | 3 +++ include/net/quic.h | 10 +++++++++ include/net/snmp.h | 6 +++++ include/uapi/linux/snmp.h | 9 ++++++++ net/quic/Makefile | 2 +- net/quic/quic_main.c | 46 +++++++++++++++++++++++++++++++++++++++ net/quic/quic_proc.c | 45 ++++++++++++++++++++++++++++++++++++++ 7 files changed, 120 insertions(+), 1 deletion(-) create mode 100644 net/quic/quic_proc.c
include/net/netns/mib.h +3 -0
@@ -24,6 +24,9 @@ struct netns_mib {#if IS_ENABLED (CONFIG_TLS ) DEFINE_SNMP_STAT(struct linux_tls_mib, tls_statistics);#endif +#if IS_ENABLED (CONFIG_QUIC ) + DEFINE_SNMP_STAT(struct linux_quic_mib, quic_statistics); +#endif #ifdef CONFIG_MPTCP DEFINE_SNMP_STAT(struct mptcp_mib, mptcp_statistics);#endif
include/net/quic.h +10 -0
@@ -25,6 +25,16 @@#define QUIC_MAX_PLAIN_PAGES 16#define QUIC_MAX_CIPHER_PAGES_ORDER 4 +#define __QUIC_INC_STATS (net, field) \ + __SNMP_INC_STATS((net)->mib.quic_statistics, field) +#define QUIC_INC_STATS (net, field) \ + SNMP_INC_STATS((net)->mib.quic_statistics, field) +#define QUIC_DEC_STATS (net, field) \ + SNMP_DEC_STATS((net)->mib.quic_statistics, field) + +int __net_init quic_proc_init (struct net *net); +void __net_exit quic_proc_fini(struct net *net); +struct quic_internal_crypto_context {struct quic_connection_info conn_info;struct crypto_skcipher *header_tfm;
include/net/snmp.h +6 -0
@@ -117,6 +117,12 @@ struct linux_tls_mib {unsigned long mibs[LINUX_MIB_TLSMAX ]; }; +/* Linux QUIC */ +#define LINUX_MIB_QUICMAX __LINUX_MIB_QUICMAX +struct linux_quic_mib { +unsigned long mibs[LINUX_MIB_QUICMAX ]; +}; +#define DEFINE_SNMP_STAT (type, name) \ __typeof__(type) __percpu *name#define DEFINE_SNMP_STAT_ATOMIC (type, name) \
include/uapi/linux/snmp.h +9 -0
@@ -349,4 +349,13 @@ enum__LINUX_MIB_TLSMAX }; +/* linux QUIC mib definitions */ +enum { +LINUX_MIB_QUICNUM =0 , +LINUX_MIB_QUICCURRTXSW ,/* QuicCurrTxSw */ +LINUX_MIB_QUICTXSW ,/* QuicTxSw */ +LINUX_MIB_QUICTXSWERROR ,/* QuicTxSwError */ + __LINUX_MIB_QUICMAX +}; +#endif /* _LINUX_SNMP_H */
net/quic/Makefile +1 -1
@@ -5,4 +5,4 @@ obj-$(CONFIG_QUIC) += quic.o -quic-y := quic_main.o +quic-y := quic_main.o quic_proc.o
net/quic/quic_main.c +46 -0
88 lines not on this page · raw message
net/quic/quic_proc.c new file +45 -0
45 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH v4 6/6 net: Add self tests for ULP operations, flow setup and crypto tests
In reply to Adel AbouchaevAdd self tests for ULP operations, flow setup and crypto tests.
---
Restored the test build. Changed the QUIC context reference variable names for the keys and iv to match the uAPI.
Updated alignment, added SPDX license line.
v3: Added Chacha20-Poly1305 test.
v3: Added test to fail sending with wrong key generation bit.
--- tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/Makefile | 3 +- tools/testing/selftests/net/quic.c | 1369 ++++++++++++++++++++++++ tools/testing/selftests/net/quic.sh | 46 + 4 files changed, 1418 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/net/quic.c create mode 100755 tools/testing/selftests/net/quic.sh
tools/testing/selftests/net/.gitignore +1 -0
@@ -14,6 +14,7 @@ nettest psock_fanout psock_snd psock_tpacket +quic reuseaddr_conflict reuseaddr_ports_exhausted reuseport_addr_any
tools/testing/selftests/net/Makefile +2 -1
@@ -44,6 +44,7 @@ TEST_PROGS += arp_ndisc_untracked_subnets.sh TEST_PROGS += stress_reuseport_listen.sh TEST_PROGS += l2_tos_ttl_inherit.sh TEST_PROGS += bind_bhash.sh +TEST_PROGS += quic.sh TEST_PROGS_EXTENDED := in_netns.sh setup_loopback.sh setup_veth.sh TEST_PROGS_EXTENDED += toeplitz_client.sh toeplitz.sh TEST_GEN_FILES = socket nettest
@@ -59,7 +60,7 @@ TEST_GEN_FILES += ipsec TEST_GEN_FILES += ioam6_parser TEST_GEN_FILES += gro TEST_GEN_PROGS = reuseport_bpf reuseport_bpf_cpu reuseport_bpf_numa -TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls tun tap +TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls tun tap quic TEST_GEN_FILES += toeplitz TEST_GEN_FILES += cmsg_sender TEST_GEN_FILES += stress_reuseport_listen
tools/testing/selftests/net/quic.c new file +1369 -0
1,369 lines not on this page · raw message
tools/testing/selftests/net/quic.sh new file +46 -0
46 lines not on this page · raw message
Signature · 1 line
2.30.2