Interrupt mode PMD
v5 cover Interrupt mode PMD
v5 changes
- Rebase the patchset onto the HEAD
- Isolate ethdev from EAL for new-added wait-for-rx interrupt function
- Export wait-for-rx interrupt function for shared libraries
- Split-off a new patch file for changed struct rte_intr_handle that other patches depend on, to avoid breaking git bisect
- Change sample applicaiton to accomodate EAL function spec change accordingly
v4 changes
- Export interrupt enable/disable functions for shared libraries
- Adjust position of new-added structure fields and functions to avoid breaking ABI
v3 changes
- Add return value for interrupt enable/disable functions
- Move spinlok from PMD to L3fwd-power
- Remove unnecessary variables in e1000_mac_info
- Fix miscelleous review comments
v2 changes
- Fix compilation issue in Makefile for missed header file.
- Consolidate internal and community review comments of v1 patch set.
The patch series introduce low-latency one-shot rx interrupt into DPDK with polling and interrupt mode switch control example.
DPDK userspace interrupt notification and handling mechanism is based on UIO with below limitation:
1) It is designed to handle LSC interrupt only with inefficient suspended pthread wakeup procedure (e.g. UIO wakes up LSC interrupt handling thread which then wakes up DPDK polling thread). In this way, it introduces non-deterministic wakeup latency for DPDK polling thread as well as packet latency if it is used to handle Rx interrupt.
2) UIO only supports a single interrupt vector which has to been shared by LSC interrupt and interrupts assigned to dedicated rx queues.
This patchset includes below features:
1) Enable one-shot rx queue interrupt in ixgbe PMD(PF & VF) and igb PMD(PF only).
2) Build on top of the VFIO mechanism instead of UIO, so it could support up to 64 interrupt vectors for rx queue interrupts.
3) Have 1 DPDK polling thread handle per Rx queue interrupt with a dedicated VFIO eventfd, which eliminates non-deterministic pthread wakeup latency in user space.
4) Demonstrate interrupts control APIs and userspace NAIP-like polling/interrupt switch algorithms in L3fwd-power example.
Known limitations:
1) It does not work for UIO due to a single interrupt eventfd shared by LSC and rx queue interrupt handlers causes a mess.
2) LSC interrupt is not supported by VF driver, so it is by default disabled in L3fwd-power now. Feel free to turn in on if you want to support both LSC and rx queue interrupts on a PF.
Danny Zhou (6):
ethdev: add rx interrupt enable/disable functions
eal: add rx queue interrupt FDs to intr handle struct
ixgbe: enable rx queue interrupts for both PF and VF
igb: enable rx queue interrupts for PF
eal: add per rx queue interrupt handling based on VFIO
l3fwd-power: enable one-shot rx interrupt and polling/interrupt mode
switch
examples/l3fwd-power/main.c | 155 ++++++--- lib/librte_eal/bsdapp/eal/rte_eal_version.map | 1 + lib/librte_eal/common/include/rte_eal.h | 1 + lib/librte_eal/common/include/rte_interrupts.h | 12 + lib/librte_eal/linuxapp/eal/eal_interrupts.c | 191 ++++++++--- lib/librte_eal/linuxapp/eal/eal_pci_vfio.c | 12 +- .../linuxapp/eal/include/exec-env/rte_interrupts.h | 4 + lib/librte_eal/linuxapp/eal/rte_eal_version.map | 1 + lib/librte_ether/rte_ethdev.c | 43 +++ lib/librte_ether/rte_ethdev.h | 59 ++++ lib/librte_ether/rte_ether_version.map | 2 + lib/librte_pmd_e1000/e1000_ethdev.h | 3 + lib/librte_pmd_e1000/igb_ethdev.c | 228 +++++++++++-- lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 365 ++++++++++++++++++++- lib/librte_pmd_ixgbe/ixgbe_ethdev.h | 7 + 15 files changed, 970 insertions(+), 114 deletions(-)
Signature · 1 line
1.8.1.4
A message that is not in the archive (sent off-list or not yet received): 1424710542-14637-1-git-send-email-danny.zhou-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
PATCH v5 1/6 ethdev: add rx interrupt enable/disable functions
v5 changes
- Rebase the patchset onto the HEAD
v4 changes
- Export interrupt enable/disable functions for shared libraries
- Put new functions at the end of eth_dev_ops to avoid breaking ABI
v3 changes
- Add return value for interrupt enable/disable functions
Add two dev_ops functions to enable and disable rx queue interrupts
--- lib/librte_ether/rte_ethdev.c | 43 +++++++++++++++++++++++++ lib/librte_ether/rte_ethdev.h | 59 ++++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ether_version.map | 2 ++ 3 files changed, 104 insertions(+)
lib/librte_ether/rte_ethdev.c +43 -0
@@ -2830,6 +2830,49 @@ _rte_eth_dev_callback_process(struct rte_eth_dev *dev, }rte_spinlock_unlock (&rte_eth_dev_cb_lock); } + +int +rte_eth_dev_rx_queue_intr_enable (uint8_t port_id, +uint16_t queue_id) +{ +struct rte_eth_dev *dev; + +if (port_id >= nb_ports) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return (-ENODEV ); + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return (-ENODEV ); + } + +FUNC_PTR_OR_ERR_RET (*dev->dev_ops->rx_queue_intr_enable, -ENOTSUP ); +return (*dev->dev_ops->rx_queue_intr_enable)(dev, queue_id); +} + +int +rte_eth_dev_rx_queue_intr_disable (uint8_t port_id, +uint16_t queue_id) +{ +struct rte_eth_dev *dev; + +if (port_id >= nb_ports) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return (-ENODEV ); + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return (-ENODEV ); + } + +FUNC_PTR_OR_ERR_RET (*dev->dev_ops->rx_queue_intr_disable, -ENOTSUP ); +return (*dev->dev_ops->rx_queue_intr_disable)(dev, queue_id); +} +#ifdef RTE_NIC_BYPASS int rte_eth_dev_bypass_init (uint8_t port_id) {
lib/librte_ether/rte_ethdev.h +59 -0
83 lines not on this page · raw message
lib/librte_ether/rte_ether_version.map +2 -0
@@ -42,6 +42,8 @@ DPDK_2.0 { rte_eth_dev_rss_hash_update; rte_eth_dev_rss_reta_query; rte_eth_dev_rss_reta_update; + rte_eth_dev_rx_queue_intr_disable; + rte_eth_dev_rx_queue_intr_enable; rte_eth_dev_rx_queue_start; rte_eth_dev_rx_queue_stop; rte_eth_dev_set_link_down;
Signature · 1 line
1.8.1.4
A message that is not in the archive (sent off-list or not yet received): 1424710542-14637-2-git-send-email-danny.zhou-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH v5 1/6] ethdev: add rx interrupt enable/disable functions
lib/librte_ether/rte_ethdev.c _rte_eth_dev_callback_process(struct rte_eth_dev *dev, · line 2842
Quoted 9 lines · On Tue, 24 Feb 2015 00:55:37 +0800 Zhou Danny…
lib/librte_ether/rte_ethdev.c +9
-0+int +rte_eth_dev_rx_queue_intr_enable (uint8_t port_id, +uint16_t queue_id) +{ +struct rte_eth_dev *dev; + +if (port_id >= nb_ports) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return (-ENODEV );
Please don't use the BSD style of extra useless paren's around returns.
Re: [PATCH v5 1/6] ethdev: add rx interrupt enable/disable functions
Quoted 11 lines
-----Original Message-----
From: Stephen Hemminger [mailto:[email protected]]
Sent: Tuesday, February 24, 2015 12:59 AM
To: Zhou, Danny
Cc: [email protected]
Subject: Re: [dpdk-dev] [PATCH v5 1/6] ethdev: add rx interrupt enable/disable functionsOn Tue, 24 Feb 2015 00:55:37 +0800
Zhou Danny <[email protected]> wrote:
Quoted 12 lines
lib/librte_ether/rte_ethdev.c _rte_eth_dev_callback_process(struct rte_eth_dev *dev, · line 2842
Quoted 9 lines
lib/librte_ether/rte_ethdev.c +9
-0+int +rte_eth_dev_rx_queue_intr_enable (uint8_t port_id, +uint16_t queue_id) +{ +struct rte_eth_dev *dev; + +if (port_id >= nb_ports) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return (-ENODEV );Please don't use the BSD style of extra useless paren's around returns.
This code snippet doing sanity check is copied from other functions defined in the same file lib/librte_ether/rte_ethdev.c, and there are plenty of legacy code in this file doing the similar BSD style thing. I'd suggest somebody to clean all those kinds of BSD style code from DPDK code, in a separated patchset.
PATCH 1/1 lib: syntax cleanup
Remove extra parenthesis from return statements.
--- lib/librte_cmdline/cmdline_parse_etheraddr.c | 10 ++--- lib/librte_cmdline/cmdline_parse_ipaddr.c | 38 ++++++++--------- lib/librte_cmdline/cmdline_parse_num.c | 24 +++++------ lib/librte_cmdline/cmdline_parse_portlist.c | 14 +++---- lib/librte_cmdline/cmdline_socket.c | 2 +- lib/librte_eal/bsdapp/contigmem/contigmem.c | 22 +++++----- lib/librte_eal/bsdapp/eal/eal.c | 4 +- lib/librte_eal/bsdapp/eal/eal_pci.c | 12 +++--- lib/librte_eal/bsdapp/nic_uio/nic_uio.c | 10 ++--- lib/librte_eal/common/eal_common_memzone.c | 2 +- lib/librte_eal/common/include/rte_common.h | 2 +- lib/librte_eal/common/include/rte_pci.h | 6 +-- lib/librte_eal/linuxapp/eal/eal.c | 4 +- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 4 +- lib/librte_eal/linuxapp/eal/eal_memory.c | 4 +- .../linuxapp/kni/ethtool/igb/igb_procfs.c | 2 +- lib/librte_eal/linuxapp/kni/kni_net.c | 8 ++-- lib/librte_ip_frag/ip_frag_internal.c | 12 +++--- lib/librte_ip_frag/rte_ip_frag_common.c | 6 +-- lib/librte_ip_frag/rte_ipv4_reassembly.c | 8 ++-- lib/librte_ip_frag/rte_ipv6_fragmentation.c | 8 ++-- lib/librte_lpm/rte_lpm.c | 2 +- lib/librte_mbuf/rte_mbuf.h | 16 ++++---- lib/librte_mempool/rte_dom0_mempool.c | 2 +- lib/librte_mempool/rte_mempool.c | 18 ++++---- lib/librte_net/rte_ip.h | 2 +- lib/librte_pmd_e1000/em_ethdev.c | 36 ++++++++-------- lib/librte_pmd_e1000/em_rxtx.c | 48 +++++++++++----------- lib/librte_pmd_e1000/igb_ethdev.c | 26 ++++++------ lib/librte_pmd_e1000/igb_rxtx.c | 30 +++++++------- lib/librte_pmd_fm10k/fm10k_ethdev.c | 30 +++++++------- lib/librte_pmd_i40e/i40e_rxtx.c | 14 +++---- lib/librte_pmd_ixgbe/ixgbe_82599_bypass.c | 4 +- lib/librte_pmd_ixgbe/ixgbe_bypass.c | 2 +- lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 42 +++++++++---------- lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 36 ++++++++-------- lib/librte_pmd_virtio/virtio_ethdev.c | 4 +- 37 files changed, 257 insertions(+), 257 deletions(-)
lib/librte_cmdline/cmdline_parse_etheraddr.c +5 -5
@@ -105,32 +105,32 @@ my_ether_aton(const char *a) errno =0 ; o[i] =strtoul (a, &end,16 );if (errno !=0 || end == a || (end[0 ] !=':' && end[0 ] !=0 )) -return (NULL ); +return NULL ; a = end +1 ; }while (++i !=sizeof (o) /sizeof (o[0 ]) && end[0 ] !=0 );/* Junk at the end of line */ if (end[0 ] !=0 ) -return (NULL ); +return NULL ;/* Support the format XX:XX:XX:XX:XX:XX */ if (i ==ETHER_ADDR_LEN ) {while (i-- !=0 ) {if (o[i] >UINT8_MAX ) -return (NULL ); +return NULL ; ether_addr.ea_oct[i] = (uint8_t )o[i]; }/* Support the format XXXX:XXXX:XXXX */ }else if (i ==ETHER_ADDR_LEN /2 ) {while (i-- !=0 ) {if (o[i] >UINT16_MAX ) -return (NULL ); +return NULL ; ether_addr.ea_oct[i *2 ] = (uint8_t )(o[i] >>8 ); ether_addr.ea_oct[i *2 +1 ] = (uint8_t )(o[i] &0xff ); }/* unknown format */ }else -return (NULL ); +return NULL ;return (struct ether_addr *)ðer_addr; }
lib/librte_cmdline/cmdline_parse_ipaddr.c +19 -19
@@ -135,12 +135,12 @@ my_inet_pton(int af, const char *src, void *dst) {switch (af) {case AF_INET : -return (inet_pton4 (src, dst)); +return inet_pton4 (src, dst);case AF_INET6 : -return (inet_pton6 (src, dst)); +return inet_pton6 (src, dst);default : errno =EAFNOSUPPORT ; -return (-1 ); +return -1 ; }/* NOTREACHED */ }
@@ -172,26 +172,26 @@ inet_pton4(const char *src, unsigned char *dst)unsigned int new = *tp *10 + (pch - digits);if (new >255 ) -return (0 ); +return 0 ;if (! saw_digit) {if (++octets >4 ) -return (0 ); +return 0 ; saw_digit =1 ; } *tp = (unsigned char )new; }else if (ch =='.' && saw_digit) {if (octets ==4 ) -return (0 ); +return 0 ; *++tp =0 ; saw_digit =0 ; }else -return (0 ); +return 0 ; }if (octets <4 ) -return (0 ); +return 0 ;memcpy (dst, tmp,INADDRSZ ); -return (1 ); +return 1 ; }/* int
@@ -224,7 +224,7 @@ inet_pton6(const char *src, unsigned char *dst)/* Leading :: requires some special handling. */ if (*src ==':' )if (*++src !=':' ) -return (0 ); +return 0 ; curtok = src; saw_xdigit = count_xdigit =0 ; val =0 ;
@@ -236,11 +236,11 @@ inet_pton6(const char *src, unsigned char *dst) pch =strchr ((xdigits = xdigits_u), ch);if (pch !=NULL ) {if (count_xdigit >=4 ) -return (0 ); +return 0 ; val <<=4 ; val |= (pch - xdigits);if (val >0xffff ) -return (0 ); +return 0 ; saw_xdigit =1 ; count_xdigit++;continue ;
@@ -249,14 +249,14 @@ inet_pton6(const char *src, unsigned char *dst) curtok = src;if (!saw_xdigit) {if (colonp) -return (0 ); +return 0 ; colonp = tp;continue ; }else if (*src =='\0' ) { -return (0 ); +return 0 ; }if (tp +sizeof (int16_t ) > endp) -return (0 ); +return 0 ; *tp++ = (unsigned char ) ((val >>8 ) &0xff ); *tp++ = (unsigned char ) (val &0xff ); saw_xdigit =0 ;
@@ -272,11 +272,11 @@ inet_pton6(const char *src, unsigned char *dst) dbloct_count +=2 ;break ;/* '\0' was seen by inet_pton4(). */ } -return (0 ); +return 0 ; }if (saw_xdigit) {if (tp +sizeof (int16_t ) > endp) -return (0 ); +return 0 ; *tp++ = (unsigned char ) ((val >>8 ) &0xff ); *tp++ = (unsigned char ) (val &0xff ); dbloct_count++;
@@ -300,9 +300,9 @@ inet_pton6(const char *src, unsigned char *dst) tp = endp; }if (tp != endp) -return (0 ); +return 0 ;memcpy (dst, tmp,IN6ADDRSZ ); -return (1 ); +return 1 ; }int
lib/librte_cmdline/cmdline_parse_num.c +12 -12
@@ -315,35 +315,35 @@ cmdline_parse_num(cmdline_parse_token_hdr_t *tk, const char *srcbuf, void *res,case BIN_OK :if ( nd.type ==INT8 && res1 <=INT8_MAX ) {if (res) *(int8_t *)res = (int8_t ) res1; -return (buf-srcbuf); +return buf-srcbuf; }else if ( nd.type ==INT16 && res1 <=INT16_MAX ) {if (res) *(int16_t *)res = (int16_t ) res1; -return (buf-srcbuf); +return buf-srcbuf; }else if ( nd.type ==INT32 && res1 <=INT32_MAX ) {if (res) *(int32_t *)res = (int32_t ) res1; -return (buf-srcbuf); +return buf-srcbuf; }else if ( nd.type ==INT64 && res1 <=INT64_MAX ) {if (res) *(int64_t *)res = (int64_t ) res1; -return (buf-srcbuf); +return buf-srcbuf; }else if ( nd.type ==UINT8 && res1 <=UINT8_MAX ) {if (res) *(uint8_t *)res = (uint8_t ) res1; -return (buf-srcbuf); +return buf-srcbuf; }else if (nd.type ==UINT16 && res1 <=UINT16_MAX ) {if (res) *(uint16_t *)res = (uint16_t ) res1; -return (buf-srcbuf); +return buf-srcbuf; }else if ( nd.type ==UINT32 && res1 <=UINT32_MAX ) {if (res) *(uint32_t *)res = (uint32_t ) res1; -return (buf-srcbuf); +return buf-srcbuf; }else if ( nd.type ==UINT64 ) {if (res) *(uint64_t *)res = res1; -return (buf-srcbuf); +return buf-srcbuf; }else {return -1 ;
@@ -353,19 +353,19 @@ cmdline_parse_num(cmdline_parse_token_hdr_t *tk, const char *srcbuf, void *res,case DEC_NEG_OK :if ( nd.type ==INT8 && res1 <=INT8_MAX +1 ) {if (res) *(int8_t *)res = (int8_t ) (-res1); -return (buf-srcbuf); +return buf-srcbuf; }else if ( nd.type ==INT16 && res1 <= (uint16_t )INT16_MAX +1 ) {if (res) *(int16_t *)res = (int16_t ) (-res1); -return (buf-srcbuf); +return buf-srcbuf; }else if ( nd.type ==INT32 && res1 <= (uint32_t )INT32_MAX +1 ) {if (res) *(int32_t *)res = (int32_t ) (-res1); -return (buf-srcbuf); +return buf-srcbuf; }else if ( nd.type ==INT64 && res1 <= (uint64_t )INT64_MAX +1 ) {if (res) *(int64_t *)res = (int64_t ) (-res1); -return (buf-srcbuf); +return buf-srcbuf; }else {return -1 ;
lib/librte_cmdline/cmdline_parse_portlist.c +7 -7
@@ -102,7 +102,7 @@ parse_ports(cmdline_portlist_t *pl, const char *str) ps =strtoul (first, &end,10 );if (errno !=0 || end == first || (end[0 ] !='-' && end[0 ] !=0 && end != last)) -return (-1 ); +return -1 ;/* Support for N-M portlist format */ if (end[0 ] =='-' ) {
@@ -111,18 +111,18 @@ parse_ports(cmdline_portlist_t *pl, const char *str) pe =strtoul (first, &end,10 );if (errno !=0 || end == first || (end[0 ] !=0 && end != last)) -return (-1 ); +return -1 ; }else { pe = ps; }if (ps > pe || pe >=sizeof (pl->map) *8 ) -return (-1 ); +return -1 ;parse_set_list (pl, ps, pe); } -return (0 ); +return 0 ; }int
@@ -134,7 +134,7 @@ cmdline_parse_portlist(__attribute__((unused)) cmdline_parse_token_hdr_t *tk,cmdline_portlist_t *pl;if (!buf || ! *buf) -return (-1 ); +return -1 ;if (res && ressize <sizeof (cmdline_portlist_t))return -1 ;
@@ -146,7 +146,7 @@ cmdline_parse_portlist(__attribute__((unused)) cmdline_parse_token_hdr_t *tk, token_len++;if (token_len >=PORTLIST_TOKEN_SIZE ) -return (-1 ); +return -1 ;snprintf (portlist_str, token_len+1 ,"%s" , buf);
@@ -155,7 +155,7 @@ cmdline_parse_portlist(__attribute__((unused)) cmdline_parse_token_hdr_t *tk,if (strcmp ("all" , portlist_str) ==0 ) pl->map =UINT32_MAX ;else if (parse_ports (pl, portlist_str) !=0 ) -return (-1 ); +return -1 ; }return token_len;
lib/librte_cmdline/cmdline_socket.c +1 -1
@@ -86,7 +86,7 @@ cmdline_file_new(cmdline_parse_ctx_t *ctx, const char *prompt, const char *path)dprintf ("open() failed\n" );return NULL ; } -return (cmdline_new (ctx, prompt, fd,-1 )); +return cmdline_new (ctx, prompt, fd,-1 ); } struct cmdline *
lib/librte_eal/bsdapp/contigmem/contigmem.c +11 -11
@@ -99,7 +99,7 @@ static int contigmem_modevent(module_t mod, int type, void *arg)break ; } -return (error); +return error; }moduledata_t contigmem_mod = {
@@ -128,14 +128,14 @@ contigmem_load()if (contigmem_num_buffers >RTE_CONTIGMEM_MAX_NUM_BUFS ) {printf ("%d buffers requested is greater than %d allowed\n" , contigmem_num_buffers,RTE_CONTIGMEM_MAX_NUM_BUFS ); -return (EINVAL ); +return EINVAL ; }if (contigmem_buffer_size <PAGE_SIZE || (contigmem_buffer_size & (contigmem_buffer_size -1 )) !=0 ) {printf ("buffer size 0x%lx is not greater than PAGE_SIZE and " "power of two\n" , contigmem_buffer_size); -return (EINVAL ); +return EINVAL ; }for (i =0 ; i < contigmem_num_buffers; i++) {
@@ -145,7 +145,7 @@ contigmem_load()if (contigmem_buffers[i] ==NULL ) {printf ("contigmalloc failed for buffer %d\n" , i); -return (ENOMEM ); +return ENOMEM ; } printf("%2u: virt=%p phys=%p\n", i, contigmem_buffers[i],
@@ -164,7 +164,7 @@ contigmem_load() contigmem_cdev =make_dev_credf (0 , &contigmem_ops,0 ,NULL ,UID_ROOT ,GID_WHEEL ,0600 ,"contigmem" ); -return (0 ); +return 0 ; } static int
@@ -183,7 +183,7 @@ contigmem_unload()contigfree (contigmem_buffers[i], contigmem_buffer_size,M_CONTIGMEM ); -return (0 ); +return 0 ; } static int
@@ -193,14 +193,14 @@ contigmem_physaddr(SYSCTL_HANDLER_ARGS)int index = (int )(uintptr_t )arg1; physaddr = (uint64_t )vtophys (contigmem_buffers[index]); -return (sysctl_handle_64 (oidp, &physaddr,0 , req)); +return sysctl_handle_64 (oidp, &physaddr,0 , req); }static int contigmem_open (struct cdev *cdev,int fflags,int devtype,struct thread *td) { -return (0 ); +return 0 ; } static int
@@ -209,7 +209,7 @@ contigmem_mmap(struct cdev *cdev, vm_ooffset_t offset, vm_paddr_t *paddr, { *paddr = offset; -return (0 ); +return 0 ; } static int
@@ -222,12 +222,12 @@ contigmem_mmap_single(struct cdev *cdev, vm_ooffset_t *offset, vm_size_t size,* app. */ if ((*offset/PAGE_SIZE ) >= contigmem_num_buffers) -return (EINVAL ); +return EINVAL ; *offset = (vm_ooffset_t )vtophys (contigmem_buffers[*offset/PAGE_SIZE ]); *obj =vm_pager_allocate (OBJT_DEVICE , cdev, size, nprot, *offset, curthread->td_ucred); -return (0 ); +return 0 ; }
lib/librte_eal/bsdapp/eal/eal.c +2 -2
@@ -561,12 +561,12 @@ rte_eal_init(int argc, char **argv)enum rte_lcore_role_t rte_eal_lcore_role (unsigned lcore_id) { -return (rte_config.lcore_role[lcore_id]); +return rte_config.lcore_role[lcore_id]; }enum rte_proc_type_t rte_eal_process_type (void ) { -return (rte_config.process_type); +return rte_config.process_type; }
lib/librte_eal/bsdapp/eal/eal_pci.c +6 -6
@@ -179,10 +179,10 @@ pci_uio_map_secondary(struct rte_pci_device *dev) != uio_res->maps[i].addr) {RTE_LOG (ERR ,EAL ,"Cannot mmap device resource\n" ); -return (-1 ); +return -1 ; } } -return (0 ); +return 0 ; }RTE_LOG (ERR ,EAL ,"Cannot find resource for device\n" );
@@ -209,7 +209,7 @@ pci_uio_map_resource(struct rte_pci_device *dev)/* secondary processes - use already recorded details */ if (rte_eal_process_type () !=RTE_PROC_PRIMARY ) -return (pci_uio_map_secondary (dev)); +return pci_uio_map_secondary (dev);snprintf (devname,sizeof (devname),"/dev/uio@pci:%u:%u:%u" , dev->addr.bus, dev->addr.devid, dev->addr.function);
@@ -233,7 +233,7 @@ pci_uio_map_resource(struct rte_pci_device *dev)if ((uio_res =rte_zmalloc ("UIO_RES" ,sizeof (*uio_res),0 )) ==NULL ) {RTE_LOG (ERR ,EAL ,"%s(): cannot store uio mmap details\n" , __func__); -return (-1 ); +return -1 ; }snprintf (uio_res->path,sizeof (uio_res->path),"%s" , devname);
@@ -261,7 +261,7 @@ pci_uio_map_resource(struct rte_pci_device *dev) (size_t)maps[j].size) ) == NULL) {rte_free (uio_res); -return (-1 ); +return -1 ; } maps[j].addr = mapaddr;
@@ -271,7 +271,7 @@ pci_uio_map_resource(struct rte_pci_device *dev)TAILQ_INSERT_TAIL (uio_res_list, uio_res, next); -return (0 ); +return 0 ; }/* Scan one pci sysfs entry, and fill the devices list from it. */
lib/librte_eal/bsdapp/nic_uio/nic_uio.c +5 -5
@@ -131,7 +131,7 @@ nic_uio_mmap(struct cdev *cdev, vm_ooffset_t offset, vm_paddr_t *paddr, int prot, vm_memattr_t *memattr) { *paddr = offset; -return (0 ); +return 0 ; } static int
@@ -197,10 +197,10 @@ nic_uio_probe (device_t dev) pci_get_device(dev) == devices[i].dev) {device_set_desc (dev,"Intel(R) DPDK PCI Device" ); -return (BUS_PROBE_SPECIFIC ); +return BUS_PROBE_SPECIFIC ; } -return (ENXIO ); +return ENXIO ; } static int
@@ -305,7 +305,7 @@ nic_uio_unload(void)static int nic_uio_shutdown (void ) { -return (0 ); +return 0 ; } static int
@@ -326,5 +326,5 @@ nic_uio_modevent(module_t mod, int type, void *arg)break ; } -return (0 ); +return 0 ; }
lib/librte_eal/common/eal_common_memzone.c +1 -1
@@ -120,7 +120,7 @@ align_phys_boundary(const struct rte_memseg *ms, size_t len, size_t align, addr_offset = start - ms->phys_addr; } -return (addr_offset); +return addr_offset; } static const struct rte_memzone *
lib/librte_eal/common/include/rte_common.h +1 -1
@@ -291,7 +291,7 @@ rte_pause(void) {}static inline uint32_t rte_bsf32 (uint32_t v) { -return (__builtin_ctz (v)); +return __builtin_ctz (v); }#ifndef offsetof
lib/librte_eal/common/include/rte_pci.h +3 -3
@@ -227,7 +227,7 @@ do { \ errno =0 ; \ val =strtoul ((in), &end,16 ); \if (errno !=0 || end[0 ] != (dlm) || val > (lim)) \ -return (-EINVAL ); \ +return -EINVAL ; \ (fd) = (typeof (fd ))val; \ (in) = end +1 ; \ }while (0 )
@@ -252,7 +252,7 @@ eal_parse_pci_BDF(const char *input, struct rte_pci_addr *dev_addr)GET_PCIADDR_FIELD (input, dev_addr->bus,UINT8_MAX ,':' );GET_PCIADDR_FIELD (input, dev_addr->devid,UINT8_MAX ,'.' );GET_PCIADDR_FIELD (input, dev_addr->function,UINT8_MAX ,0 ); -return (0 ); +return 0 ; }/**
@@ -274,7 +274,7 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr)GET_PCIADDR_FIELD (input, dev_addr->bus,UINT8_MAX ,':' );GET_PCIADDR_FIELD (input, dev_addr->devid,UINT8_MAX ,'.' );GET_PCIADDR_FIELD (input, dev_addr->function,UINT8_MAX ,0 ); -return (0 ); +return 0 ; }#undef GET_PCIADDR_FIELD
lib/librte_eal/linuxapp/eal/eal.c +2 -2
@@ -852,13 +852,13 @@ rte_eal_init(int argc, char **argv)enum rte_lcore_role_t rte_eal_lcore_role (unsigned lcore_id) { -return (rte_config.lcore_role[lcore_id]); +return rte_config.lcore_role[lcore_id]; }enum rte_proc_type_t rte_eal_process_type (void ) { -return (rte_config.process_type); +return rte_config.process_type; }int rte_eal_has_hugepages (void )
lib/librte_eal/linuxapp/eal/eal_interrupts.c +2 -2
@@ -477,7 +477,7 @@ rte_intr_callback_register(struct rte_intr_handle *intr_handle,if (write (intr_pipe.writefd,"1" ,1 ) <0 )return -EPIPE ; -return (ret); +return ret; }int
@@ -541,7 +541,7 @@ rte_intr_callback_unregister(struct rte_intr_handle *intr_handle, ret = -EPIPE ; } -return (ret); +return ret; }int
lib/librte_eal/linuxapp/eal/eal_memory.c +2 -2
@@ -880,7 +880,7 @@ get_socket_mem_size(int socket) size += hpi->hugepage_sz * hpi->num_pages[socket]; } -return (size); +return size; }/*
@@ -1339,7 +1339,7 @@ rte_eal_hugepage_init(void)"of memory.\n" , i, nr_hugefiles,RTE_STR (CONFIG_RTE_MAX_MEMSEG ), RTE_MAX_MEMSEG); -return (-ENOMEM ); +return -ENOMEM ; }return 0 ;
lib/librte_eal/linuxapp/kni/ethtool/igb/igb_procfs.c +1 -1
@@ -262,7 +262,7 @@ int igb_procfs_topdir_init(void) { igb_top_dir =proc_mkdir ("driver/igb" ,NULL );if (igb_top_dir ==NULL ) -return (-ENOMEM ); +return -ENOMEM ;return 0 ; }
lib/librte_eal/linuxapp/kni/kni_net.c +4 -4
@@ -88,7 +88,7 @@ kni_net_open(struct net_device *dev) req.if_up =1 ; ret =kni_net_process_request (kni, &req); -return (ret ==0 ? req.result : ret); +return (ret ==0 ) ? req.result : ret; } static int
@@ -107,7 +107,7 @@ kni_net_release(struct net_device *dev) req.if_up =0 ; ret =kni_net_process_request (kni, &req); -return (ret ==0 ? req.result : ret); +return (ret ==0 ) ? req.result : ret; }/*
@@ -511,7 +511,7 @@ kni_net_change_mtu(struct net_device *dev, int new_mtu)if (ret ==0 && req.result ==0 ) dev->mtu = new_mtu; -return (ret ==0 ? req.result : ret); +return (ret ==0 ) ? req.result : ret; }/*
@@ -597,7 +597,7 @@ kni_net_header(struct sk_buff *skb, struct net_device *dev,memcpy (eth->h_dest, daddr ? daddr : dev->dev_addr, dev->addr_len); eth->h_proto =htons (type); -return (dev->hard_header_len); +return dev->hard_header_len; }
lib/librte_ip_frag/ip_frag_internal.c +6 -6
@@ -200,7 +200,7 @@ ip_frag_process(struct ip_frag_pkt *fp, struct rte_ip_frag_death_row *dr,ip_frag_key_invalidate (&fp->key);IP_FRAG_MBUF2DR (dr, mb); -return (NULL ); +return NULL ; } fp->frags[idx].ofs = ofs;
@@ -211,7 +211,7 @@ ip_frag_process(struct ip_frag_pkt *fp, struct rte_ip_frag_death_row *dr,/* not all fragments are collected yet. */ if (likely (fp->frag_size < fp->total_size)) { -return (mb); +return mb;/* if we collected all fragments, then try to reassemble. */ } else if (fp->frag_size == fp->total_size &&
@@ -259,7 +259,7 @@ ip_frag_process(struct ip_frag_pkt *fp, struct rte_ip_frag_death_row *dr,/* we are done with that entry, invalidate it. */ ip_frag_key_invalidate (&fp->key); -return (mb); +return mb; }
@@ -327,7 +327,7 @@ ip_frag_find(struct rte_ip_frag_tbl *tbl, struct rte_ip_frag_death_row *dr,IP_FRAG_TBL_STAT_UPDATE (&tbl->stat, fail_total, (pkt ==NULL )); tbl->last = pkt; -return (pkt); +return pkt; } struct ip_frag_pkt *
@@ -347,7 +347,7 @@ ip_frag_lookup(struct rte_ip_frag_tbl *tbl, assoc = tbl->bucket_entries;if (tbl->last !=NULL &&ip_frag_key_cmp (key, &tbl->last->key) ==0 ) -return (tbl->last); +return tbl->last;/* different hashing methods for IPv4 and IPv6 */ if (key->key_len ==IPV4_KEYLEN )
@@ -414,5 +414,5 @@ ip_frag_lookup(struct rte_ip_frag_tbl *tbl, *free = empty; *stale = old; -return (NULL ); +return NULL ; }
lib/librte_ip_frag/rte_ip_frag_common.c +3 -3
@@ -83,7 +83,7 @@ rte_ip_frag_table_create(uint32_t bucket_num, uint32_t bucket_entries, nb_entries > UINT32_MAX || nb_entries == 0 || nb_entries < max_entries) {RTE_LOG (ERR ,USER1 ,"%s: invalid input parameter\n" , __func__); -return (NULL ); +return NULL ; } sz =sizeof (*tbl) + nb_entries *sizeof (tbl->pkt[0 ]);
@@ -92,7 +92,7 @@ rte_ip_frag_table_create(uint32_t bucket_num, uint32_t bucket_entries,RTE_LOG (ERR ,USER1 ,"%s: allocation of %zu bytes at socket %d failed do\n" , __func__, sz, socket_id); -return (NULL ); +return NULL ; } RTE_LOG(INFO, USER1, "%s: allocated of %zu bytes at socket %d\n",
@@ -106,7 +106,7 @@ rte_ip_frag_table_create(uint32_t bucket_num, uint32_t bucket_entries, tbl->entry_mask = (tbl->nb_entries -1 ) & ~(tbl->bucket_entries -1 );TAILQ_INIT (&(tbl->lru)); -return (tbl); +return tbl; }/* dump frag table statistics to file */
lib/librte_ip_frag/rte_ipv4_reassembly.c +4 -4
@@ -73,7 +73,7 @@ ipv4_frag_reassemble(const struct ip_frag_pkt *fp)/* error - hole in the packet. */ if (m == prev) { -return (NULL ); +return NULL ; } }
@@ -94,7 +94,7 @@ ipv4_frag_reassemble(const struct ip_frag_pkt *fp) rte_cpu_to_be_16(IPV4_HDR_DF_FLAG)); ip_hdr->hdr_checksum =0 ; -return (m); +return m; }/*
@@ -151,7 +151,7 @@ rte_ipv4_frag_reassemble_packet(struct rte_ip_frag_tbl *tbl,/* try to find/add entry into the fragment's table. */ if ((fp =ip_frag_find (tbl, dr, &key, tms)) ==NULL ) {IP_FRAG_MBUF2DR (dr, mb); -return (NULL ); +return NULL ; } IP_FRAG_LOG(DEBUG, "%s:%d:\n"
@@ -178,5 +178,5 @@ rte_ipv4_frag_reassemble_packet(struct rte_ip_frag_tbl *tbl, fp, fp->key.src_dst[0 ], fp->key.id, fp->start, fp->total_size, fp->frag_size, fp->last_idx); -return (mb); +return mb; }
lib/librte_ip_frag/rte_ipv6_fragmentation.c +4 -4
@@ -123,7 +123,7 @@ rte_ipv6_fragment_packet(struct rte_mbuf *pkt_in,/* Check that pkts_out is big enough to hold all fragments */ if (unlikely (frag_size * nb_pkts_out < (uint16_t )(pkt_in->pkt_len -sizeof (struct ipv6_hdr )))) -return (-EINVAL ); +return -EINVAL ; in_hdr = rte_pktmbuf_mtod(pkt_in, struct ipv6_hdr *);
@@ -142,7 +142,7 @@ rte_ipv6_fragment_packet(struct rte_mbuf *pkt_in, out_pkt =rte_pktmbuf_alloc (pool_direct);if (unlikely (out_pkt ==NULL )) {__free_fragments (pkts_out, out_pkt_pos); -return (-ENOMEM ); +return -ENOMEM ; }/* Reserve space for the IP header that will be built later */
@@ -160,7 +160,7 @@ rte_ipv6_fragment_packet(struct rte_mbuf *pkt_in,if (unlikely (out_seg ==NULL )) {rte_pktmbuf_free (out_pkt);__free_fragments (pkts_out, out_pkt_pos); -return (-ENOMEM ); +return -ENOMEM ; } out_seg_prev->next = out_seg; out_seg_prev = out_seg;
@@ -211,5 +211,5 @@ rte_ipv6_fragment_packet(struct rte_mbuf *pkt_in, out_pkt_pos ++; } -return (out_pkt_pos); +return out_pkt_pos; }
lib/librte_lpm/rte_lpm.c +1 -1
@@ -371,7 +371,7 @@ rule_find(struct rte_lpm *lpm, uint32_t ip_masked, uint8_t depth)for (rule_index = rule_gindex; rule_index < last_rule; rule_index++) {/* If rule is found return the rule index. */ if (lpm->rules_tbl[rule_index].ip == ip_masked) -return (rule_index); +return rule_index; }/* If rule is not found return -EINVAL. */
lib/librte_mbuf/rte_mbuf.h +8 -8
@@ -530,7 +530,7 @@ static inline struct rte_mbuf *__rte_mbuf_raw_alloc(struct rte_mempool *mp) m = (struct rte_mbuf *)mb;RTE_MBUF_ASSERT (rte_mbuf_refcnt_read (m) ==0 );rte_mbuf_refcnt_set (m,1 ); -return (m); +return m; }/**
@@ -626,7 +626,7 @@ void rte_ctrlmbuf_init(struct rte_mempool *mp, void *opaque_arg,static inline int rte_is_ctrlmbuf (struct rte_mbuf *m) { -return (!!(m->ol_flags &CTRL_MBUF_FLAG )); +return !!(m->ol_flags &CTRL_MBUF_FLAG ); }/* Operations on pkt mbuf */
@@ -797,7 +797,7 @@ static inline struct rte_mbuf *rte_pktmbuf_alloc(struct rte_mempool *mp)struct rte_mbuf *m;if ((m =__rte_mbuf_raw_alloc (mp)) !=NULL )rte_pktmbuf_reset (m); -return (m); +return m; }/**
@@ -910,9 +910,9 @@ __rte_pktmbuf_prefree_seg(struct rte_mbuf *m)if (rte_mbuf_refcnt_update (md,-1 ) ==0 )__rte_mbuf_raw_free (md); } -return (m); +return m; } -return (NULL ); +return NULL ; }/**
@@ -980,7 +980,7 @@ static inline struct rte_mbuf *rte_pktmbuf_clone(struct rte_mbuf *md,uint8_t nseg;if (unlikely ((mc =rte_pktmbuf_alloc (mp)) ==NULL )) -return (NULL ); +return NULL ; mi = mc; prev = &mi->next;
@@ -1002,11 +1002,11 @@ static inline struct rte_mbuf *rte_pktmbuf_clone(struct rte_mbuf *md,/* Allocation of new indirect segment failed */ if (unlikely (mi ==NULL )) {rte_pktmbuf_free (mc); -return (NULL ); +return NULL ; }__rte_mbuf_sanity_check (mc,1 ); -return (mc); +return mc; }/**
lib/librte_mempool/rte_dom0_mempool.c +1 -1
@@ -129,5 +129,5 @@ rte_dom0_mempool_create(const char *name, unsigned elt_num, unsigned elt_size,free (pa); -return (mp); +return mp; }
lib/librte_mempool/rte_mempool.c +9 -9
@@ -200,7 +200,7 @@ rte_mempool_obj_iter(void *vaddr, uint32_t elt_num, size_t elt_sz, size_t align, } } -return (i); +return i; }/*
@@ -309,7 +309,7 @@ rte_mempool_calc_obj_size(uint32_t elt_size, uint32_t flags,/* this is the size of an object, including header and trailer */ sz->total_size = sz->header_size + sz->elt_size + sz->trailer_size; -return (sz->total_size); +return sz->total_size; }
@@ -330,7 +330,7 @@ rte_mempool_xmem_size(uint32_t elt_num, size_t elt_sz, uint32_t pg_shift) sz =RTE_ALIGN_CEIL (elt_sz, pg_sz) * elt_num; } -return (sz); +return sz; }/*
@@ -359,12 +359,12 @@ rte_mempool_xmem_usage(void *vaddr, uint32_t elt_num, size_t elt_sz,if ((n =rte_mempool_obj_iter (vaddr, elt_num, elt_sz,1 , paddr, pg_num, pg_shift, mempool_lelem_iter, &uv)) != elt_num) { -return (-n); +return -n; } uv =RTE_ALIGN_CEIL (uv, pg_sz); usz = uv - va; -return (usz); +return usz; }/* create the mempool */
@@ -376,18 +376,18 @@ rte_mempool_create(const char *name, unsigned n, unsigned elt_size, int socket_id, unsigned flags) {#ifdef RTE_LIBRTE_XEN_DOM0 -return (rte_dom0_mempool_create (name, n, elt_size, +return rte_dom0_mempool_create (name, n, elt_size, cache_size, private_data_size, mp_init, mp_init_arg, obj_init, obj_init_arg, - socket_id, flags)); + socket_id, flags);#else -return (rte_mempool_xmem_create (name, n, elt_size, +return rte_mempool_xmem_create (name, n, elt_size, cache_size, private_data_size, mp_init, mp_init_arg, obj_init, obj_init_arg, socket_id, flags, -NULL ,NULL ,MEMPOOL_PG_NUM_DEFAULT ,MEMPOOL_PG_SHIFT_MAX )); +NULL ,NULL ,MEMPOOL_PG_NUM_DEFAULT ,MEMPOOL_PG_SHIFT_MAX );#endif }
lib/librte_net/rte_ip.h +1 -1
@@ -243,7 +243,7 @@ rte_ipv4_cksum(const struct ipv4_hdr *ipv4_hdr) {uint16_t cksum; cksum =rte_raw_cksum (ipv4_hdr,sizeof (struct ipv4_hdr )); -return ((cksum ==0xffff ) ? cksum : ~cksum); +return (cksum ==0xffff ) ? cksum : ~cksum; }/**
lib/librte_pmd_e1000/em_ethdev.c +18 -18
@@ -277,7 +277,7 @@ eth_em_dev_init(struct rte_eth_dev *eth_dev)rte_intr_callback_register (&(pci_dev->intr_handle), eth_em_interrupt_handler, (void *)eth_dev); -return (0 ); +return 0 ; }static struct eth_driver rte_em_pmd = {
@@ -375,11 +375,11 @@ em_hw_init(struct e1000_hw *hw)PMD_INIT_LOG (ERR ,"PHY reset is blocked due to " "SOL/IDER session" ); } -return (0 ); +return 0 ; error:em_hw_control_release (hw); -return (diag); +return diag; } static int
@@ -392,7 +392,7 @@ eth_em_configure(struct rte_eth_dev *dev) intr->flags |=E1000_FLAG_NEED_LINK_UPDATE ;PMD_INIT_FUNC_TRACE (); -return (0 ); +return 0 ; } static void
@@ -476,7 +476,7 @@ eth_em_start(struct rte_eth_dev *dev)/* Initialize the hardware */ if (em_hardware_init (hw)) {PMD_INIT_LOG (ERR ,"Unable to initialize the hardware" ); -return (-EIO ); +return -EIO ; }E1000_WRITE_REG (hw,E1000_VET ,ETHER_TYPE_VLAN );
@@ -567,14 +567,14 @@ eth_em_start(struct rte_eth_dev *dev)PMD_INIT_LOG (DEBUG ,"<<" ); -return (0 ); +return 0 ; error_invalid_config:PMD_INIT_LOG (ERR ,"Invalid link_speed/link_duplex (%u/%u) for port %u" , dev->data->dev_conf.link_speed, dev->data->dev_conf.link_duplex, dev->data->port_id);em_dev_clear_queues (dev); -return (-EINVAL ); +return -EINVAL ; }/*********************************************************************
@@ -687,9 +687,9 @@ em_hardware_init(struct e1000_hw *hw) diag =e1000_init_hw (hw);if (diag <0 ) -return (diag); +return diag;e1000_check_for_link (hw); -return (0 ); +return 0 ; }/* This function is based on em_update_stats_counters() in e1000/if_em.c */
@@ -843,15 +843,15 @@ em_get_max_pktlen(const struct e1000_hw *hw)case e1000_pch2lan:case e1000_82574:case e1000_80003es2lan:/* 9K Jumbo Frame size */ -return (0x2412 ); +return 0x2412 ;case e1000_pchlan: -return (0x1000 ); +return 0x1000 ;/* Adapters that do not support jumbo frames */ case e1000_82583:case e1000_ich8lan: -return (ETHER_MAX_LEN ); +return ETHER_MAX_LEN ;default : -return (MAX_JUMBO_FRAME_SIZE ); +return MAX_JUMBO_FRAME_SIZE ; } }
@@ -1223,7 +1223,7 @@ eth_em_interrupt_setup(struct rte_eth_dev *dev)E1000_WRITE_REG (hw,E1000_IMS ,E1000_ICR_LSC );rte_intr_enable (&(dev->pci_dev->intr_handle)); -return (0 ); +return 0 ; }/*
@@ -1349,7 +1349,7 @@ eth_em_led_on(struct rte_eth_dev *dev)struct e1000_hw *hw; hw =E1000_DEV_PRIVATE_TO_HW (dev->data->dev_private); -return (e1000_led_on (hw) ==E1000_SUCCESS ?0 : -ENOTSUP ); +return (e1000_led_on (hw) ==E1000_SUCCESS ) ?0 : -ENOTSUP ; } static int
@@ -1358,7 +1358,7 @@ eth_em_led_off(struct rte_eth_dev *dev)struct e1000_hw *hw; hw =E1000_DEV_PRIVATE_TO_HW (dev->data->dev_private); -return (e1000_led_off (hw) ==E1000_SUCCESS ?0 : -ENOTSUP ); +return (e1000_led_off (hw) ==E1000_SUCCESS ) ?0 : -ENOTSUP ; } static int
@@ -1430,7 +1430,7 @@ eth_em_flow_ctrl_set(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf) (fc_conf->high_water < fc_conf->low_water)) {PMD_INIT_LOG (ERR ,"e1000 incorrect high/low water value" );PMD_INIT_LOG (ERR ,"high water must <= 0x%x" , max_high_water); -return (-EINVAL ); +return -EINVAL ; } hw->fc.requested_mode = rte_fcmode_2_e1000_fcmode[fc_conf->mode];
@@ -1460,7 +1460,7 @@ eth_em_flow_ctrl_set(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf) }PMD_INIT_LOG (ERR ,"e1000_setup_link_generic = 0x%x" , err); -return (-EIO ); +return -EIO ; } static void
lib/librte_pmd_e1000/em_rxtx.c +24 -24
182 lines not on this page · raw message
lib/librte_pmd_e1000/igb_ethdev.c +13 -13
@@ -605,7 +605,7 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev) err_late:igb_hw_control_release (hw); -return (error); +return error; }/*
@@ -731,7 +731,7 @@ rte_igbvf_pmd_init(const char *name __rte_unused, const char *params __rte_unusePMD_INIT_FUNC_TRACE ();rte_eth_driver_register (&rte_igbvf_pmd); -return (0 ); +return 0 ; } static int
@@ -744,7 +744,7 @@ eth_igb_configure(struct rte_eth_dev *dev) intr->flags |=E1000_FLAG_NEED_LINK_UPDATE ;PMD_INIT_FUNC_TRACE (); -return (0 ); +return 0 ; } static int
@@ -778,7 +778,7 @@ eth_igb_start(struct rte_eth_dev *dev)/* Initialize the hardware */ if (igb_hardware_init (hw)) {PMD_INIT_LOG (ERR ,"Unable to initialize the hardware" ); -return (-EIO ); +return -EIO ; }E1000_WRITE_REG (hw,E1000_VET ,ETHER_TYPE_VLAN <<16 |ETHER_TYPE_VLAN );
@@ -904,14 +904,14 @@ eth_igb_start(struct rte_eth_dev *dev)PMD_INIT_LOG (DEBUG ,"<<" ); -return (0 ); +return 0 ; error_invalid_config:PMD_INIT_LOG (ERR ,"Invalid link_speed/link_duplex (%u/%u) for port %u" , dev->data->dev_conf.link_speed, dev->data->dev_conf.link_duplex, dev->data->port_id);igb_dev_clear_queues (dev); -return (-EINVAL ); +return -EINVAL ; }/*********************************************************************
@@ -1075,13 +1075,13 @@ igb_hardware_init(struct e1000_hw *hw) diag =e1000_init_hw (hw);if (diag <0 ) -return (diag); +return diag;E1000_WRITE_REG (hw,E1000_VET ,ETHER_TYPE_VLAN <<16 |ETHER_TYPE_VLAN );e1000_get_phy_info (hw);e1000_check_for_link (hw); -return (0 ); +return 0 ; }/* This function is based on igb_update_stats_counters() in igb/if_igb.c */
@@ -1968,7 +1968,7 @@ eth_igb_led_on(struct rte_eth_dev *dev)struct e1000_hw *hw; hw =E1000_DEV_PRIVATE_TO_HW (dev->data->dev_private); -return (e1000_led_on (hw) ==E1000_SUCCESS ?0 : -ENOTSUP ); +return (e1000_led_on (hw) ==E1000_SUCCESS ) ?0 : -ENOTSUP ; } static int
@@ -1977,7 +1977,7 @@ eth_igb_led_off(struct rte_eth_dev *dev)struct e1000_hw *hw; hw =E1000_DEV_PRIVATE_TO_HW (dev->data->dev_private); -return (e1000_led_off (hw) ==E1000_SUCCESS ?0 : -ENOTSUP ); +return (e1000_led_off (hw) ==E1000_SUCCESS ) ?0 : -ENOTSUP ; } static int
@@ -2049,7 +2049,7 @@ eth_igb_flow_ctrl_set(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf) (fc_conf->high_water < fc_conf->low_water)) {PMD_INIT_LOG (ERR ,"e1000 incorrect high/low water value" );PMD_INIT_LOG (ERR ,"high water must <= 0x%x" , max_high_water); -return (-EINVAL ); +return -EINVAL ; } hw->fc.requested_mode = rte_fcmode_2_e1000_fcmode[fc_conf->mode];
@@ -2079,7 +2079,7 @@ eth_igb_flow_ctrl_set(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf) }PMD_INIT_LOG (ERR ,"e1000_setup_link_generic = 0x%x" , err); -return (-EIO ); +return -EIO ; }#define E1000_RAH_POOLSEL_SHIFT (18)
@@ -2284,7 +2284,7 @@ static int igbvf_set_vfta(struct e1000_hw *hw, uint16_t vid, bool on)if (on) msgbuf[0 ] |=E1000_VF_SET_VLAN_ADD ; -return (mbx->ops.write_posted (hw, msgbuf,2 ,0 )); +return mbx->ops.write_posted (hw, msgbuf,2 ,0 ); }static void igbvf_set_vfta_all (struct rte_eth_dev *dev,bool on)
lib/librte_pmd_e1000/igb_rxtx.c +15 -15
@@ -85,7 +85,7 @@ rte_rxmbuf_alloc(struct rte_mempool *mp) m =__rte_mbuf_raw_alloc (mp);__rte_mbuf_sanity_check_raw (m,0 ); -return (m); +return m; }#define RTE_MBUF_DATA_DMA_ADDR (mb) \
@@ -321,7 +321,7 @@ what_advctx_update(struct igb_tx_queue *txq, uint64_t flags, }/* Mismatch, use the previous context */ -return (IGB_CTX_NUM ); +return IGB_CTX_NUM ; } static inline uint32_t
@@ -473,7 +473,7 @@ eth_igb_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,*/ if (! (txr[tx_end].wb.status &E1000_TXD_STAT_DD )) {if (nb_tx ==0 ) -return (0 ); +return 0 ; goto end_of_tx; }
@@ -582,7 +582,7 @@ eth_igb_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, (unsigned) tx_id, (unsigned) nb_tx); txq->tx_tail = tx_id; -return (nb_tx); +return nb_tx; }/*********************************************************************
@@ -821,7 +821,7 @@ eth_igb_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, nb_hold =0 ; } rxq->nb_rx_hold = nb_hold; -return (nb_rx); +return nb_rx; }uint16_t
@@ -1074,7 +1074,7 @@ eth_igb_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, nb_hold =0 ; } rxq->nb_rx_hold = nb_hold; -return (nb_rx); +return nb_rx; }/*
@@ -1244,7 +1244,7 @@ eth_igb_tx_queue_setup(struct rte_eth_dev *dev, txq =rte_zmalloc ("ethdev TX queue" ,sizeof (struct igb_tx_queue ),RTE_CACHE_LINE_SIZE );if (txq ==NULL ) -return (-ENOMEM ); +return -ENOMEM ;/* * Allocate TX ring hardware descriptors. A memzone large enough to
@@ -1256,7 +1256,7 @@ eth_igb_tx_queue_setup(struct rte_eth_dev *dev, size, socket_id);if (tz ==NULL ) {igb_tx_queue_release (txq); -return (-ENOMEM ); +return -ENOMEM ; } txq->nb_tx_desc = nb_desc;
@@ -1283,7 +1283,7 @@ eth_igb_tx_queue_setup(struct rte_eth_dev *dev, RTE_CACHE_LINE_SIZE);if (txq->sw_ring ==NULL ) {igb_tx_queue_release (txq); -return (-ENOMEM ); +return -ENOMEM ; }PMD_INIT_LOG (DEBUG ,"sw_ring=%p hw_ring=%p dma_addr=0x%" PRIx64, txq->sw_ring, txq->tx_ring, txq->tx_ring_phys_addr);
@@ -1292,7 +1292,7 @@ eth_igb_tx_queue_setup(struct rte_eth_dev *dev, dev->tx_pkt_burst = eth_igb_xmit_pkts; dev->data->tx_queues[queue_idx] = txq; -return (0 ); +return 0 ; } static void
@@ -1364,7 +1364,7 @@ eth_igb_rx_queue_setup(struct rte_eth_dev *dev,*/ if (((nb_desc *sizeof (union e1000_adv_rx_desc )) %IGB_ALIGN ) !=0 || (nb_desc >IGB_MAX_RING_DESC ) || (nb_desc <IGB_MIN_RING_DESC )) { -return (-EINVAL ); +return -EINVAL ; }/* Free memory prior to re-allocation if needed */
@@ -1377,7 +1377,7 @@ eth_igb_rx_queue_setup(struct rte_eth_dev *dev, rxq =rte_zmalloc ("ethdev RX queue" ,sizeof (struct igb_rx_queue ),RTE_CACHE_LINE_SIZE );if (rxq ==NULL ) -return (-ENOMEM ); +return -ENOMEM ; rxq->mb_pool = mp; rxq->nb_rx_desc = nb_desc; rxq->pthresh = rx_conf->rx_thresh.pthresh;
@@ -1403,7 +1403,7 @@ eth_igb_rx_queue_setup(struct rte_eth_dev *dev, rz =ring_dma_zone_reserve (dev,"rx_ring" , queue_idx, size, socket_id);if (rz ==NULL ) {igb_rx_queue_release (rxq); -return (-ENOMEM ); +return -ENOMEM ; } rxq->rdt_reg_addr =E1000_PCI_REG_ADDR (hw,E1000_RDT (rxq->reg_idx)); rxq->rdh_reg_addr =E1000_PCI_REG_ADDR (hw,E1000_RDH (rxq->reg_idx));
@@ -1420,7 +1420,7 @@ eth_igb_rx_queue_setup(struct rte_eth_dev *dev, RTE_CACHE_LINE_SIZE);if (rxq->sw_ring ==NULL ) {igb_rx_queue_release (rxq); -return (-ENOMEM ); +return -ENOMEM ; }PMD_INIT_LOG (DEBUG ,"sw_ring=%p hw_ring=%p dma_addr=0x%" PRIx64, rxq->sw_ring, rxq->rx_ring, rxq->rx_ring_phys_addr);
@@ -1863,7 +1863,7 @@ igb_alloc_rx_queue_mbufs(struct igb_rx_queue *rxq)if (mbuf ==NULL ) {PMD_INIT_LOG (ERR ,"RX mbuf alloc failed " "queue_id=%hu" , rxq->queue_id); -return (-ENOMEM ); +return -ENOMEM ; } dma_addr =rte_cpu_to_le_64 (RTE_MBUF_DATA_DMA_ADDR_DEFAULT (mbuf));
lib/librte_pmd_fm10k/fm10k_ethdev.c +15 -15
@@ -876,7 +876,7 @@ handle_rxconf(struct fm10k_rx_queue *q, const struct rte_eth_rxconf *conf) rx_free_thresh,FM10K_RX_FREE_THRESH_MAX (q),FM10K_RX_FREE_THRESH_MIN (q), FM10K_RX_FREE_THRESH_DIV(q)); -return (-EINVAL ); +return -EINVAL ; } q->alloc_thresh = rx_free_thresh;
@@ -936,7 +936,7 @@ fm10k_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id,/* make sure the mempool element size can account for alignment. */ if (!mempool_element_size_valid (mp)) {PMD_INIT_LOG (ERR ,"Error : Mempool element size is too small" ); -return (-EINVAL ); +return -EINVAL ; }/* make sure a valid number of descriptors have been requested */
@@ -948,7 +948,7 @@ fm10k_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id,"and a multiple of %u" , nb_desc, (uint32_t )FM10K_MAX_RX_DESC ,FM10K_MIN_RX_DESC , FM10K_MULT_RX_DESC); -return (-EINVAL ); +return -EINVAL ; }/*
@@ -966,7 +966,7 @@ fm10k_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id, socket_id);if (q ==NULL ) {PMD_INIT_LOG (ERR ,"Cannot allocate queue structure" ); -return (-ENOMEM ); +return -ENOMEM ; }/* setup queue */
@@ -977,7 +977,7 @@ fm10k_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id, q->tail_ptr = (volatile uint32_t *) &((uint32_t *)hw->hw_addr)[FM10K_RDT (queue_id)];if (handle_rxconf (q, conf)) -return (-EINVAL ); +return -EINVAL ;/* allocate memory for the software ring */ q->sw_ring = rte_zmalloc_socket("fm10k sw ring",
@@ -986,7 +986,7 @@ fm10k_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id,if (q->sw_ring ==NULL ) {PMD_INIT_LOG (ERR ,"Cannot allocate software ring" );rte_free (q); -return (-ENOMEM ); +return -ENOMEM ; }/*
@@ -1001,7 +1001,7 @@ fm10k_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id,PMD_INIT_LOG (ERR ,"Cannot allocate hardware ring" );rte_free (q->sw_ring);rte_free (q); -return (-ENOMEM ); +return -ENOMEM ; } q->hw_ring = mz->addr; q->hw_ring_phys_addr = mz->phys_addr;
@@ -1043,7 +1043,7 @@ handle_txconf(struct fm10k_tx_queue *q, const struct rte_eth_txconf *conf) tx_free_thresh,FM10K_TX_FREE_THRESH_MAX (q),FM10K_TX_FREE_THRESH_MIN (q), FM10K_TX_FREE_THRESH_DIV(q)); -return (-EINVAL ); +return -EINVAL ; } q->free_thresh = tx_free_thresh;
@@ -1067,7 +1067,7 @@ handle_txconf(struct fm10k_tx_queue *q, const struct rte_eth_txconf *conf) tx_rs_thresh,FM10K_TX_RS_THRESH_MAX (q),FM10K_TX_RS_THRESH_MIN (q), FM10K_TX_RS_THRESH_DIV(q)); -return (-EINVAL ); +return -EINVAL ; } q->rs_thresh = tx_rs_thresh;
@@ -1095,7 +1095,7 @@ fm10k_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id,"and a multiple of %u" , nb_desc, (uint32_t )FM10K_MAX_TX_DESC ,FM10K_MIN_TX_DESC , FM10K_MULT_TX_DESC); -return (-EINVAL ); +return -EINVAL ; }/*
@@ -1113,7 +1113,7 @@ fm10k_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id, socket_id);if (q ==NULL ) {PMD_INIT_LOG (ERR ,"Cannot allocate queue structure" ); -return (-ENOMEM ); +return -ENOMEM ; }/* setup queue */
@@ -1123,7 +1123,7 @@ fm10k_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id, q->tail_ptr = (volatile uint32_t *) &((uint32_t *)hw->hw_addr)[FM10K_TDT (queue_id)];if (handle_txconf (q, conf)) -return (-EINVAL ); +return -EINVAL ;/* allocate memory for the software ring */ q->sw_ring = rte_zmalloc_socket("fm10k sw ring",
@@ -1132,7 +1132,7 @@ fm10k_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id,if (q->sw_ring ==NULL ) {PMD_INIT_LOG (ERR ,"Cannot allocate software ring" );rte_free (q); -return (-ENOMEM ); +return -ENOMEM ; }/*
@@ -1147,7 +1147,7 @@ fm10k_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id,PMD_INIT_LOG (ERR ,"Cannot allocate hardware ring" );rte_free (q->sw_ring);rte_free (q); -return (-ENOMEM ); +return -ENOMEM ; } q->hw_ring = mz->addr; q->hw_ring_phys_addr = mz->phys_addr;
@@ -1164,7 +1164,7 @@ fm10k_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id,PMD_INIT_LOG (ERR ,"Cannot allocate RS bit tracker" );rte_free (q->sw_ring);rte_free (q); -return (-ENOMEM ); +return -ENOMEM ; } dev->data->tx_queues[queue_id] = q;
lib/librte_pmd_i40e/i40e_rxtx.c +7 -7
@@ -1647,7 +1647,7 @@ i40e_get_queue_offset_by_qindex(struct i40e_pf *pf, uint16_t queue_idx)return queue_idx % pf->vmdq_nb_qps; else {PMD_INIT_LOG (ERR ,"Fail to get queue offset" ); -return (uint16_t )(-1 ); +return (uint16_t )-1 ; } }
@@ -1821,7 +1821,7 @@ i40e_dev_rx_queue_setup(struct rte_eth_dev *dev,if (!rxq) {PMD_DRV_LOG (ERR ,"Failed to allocate memory for " "rx queue data structure" ); -return (-ENOMEM ); +return -ENOMEM ; } rxq->mp = mp; rxq->nb_rx_desc = nb_desc;
@@ -1851,7 +1851,7 @@ i40e_dev_rx_queue_setup(struct rte_eth_dev *dev,if (!rz) {i40e_dev_rx_queue_release (rxq);PMD_DRV_LOG (ERR ,"Failed to reserve DMA memory for RX" ); -return (-ENOMEM ); +return -ENOMEM ; }/* Zero all the descriptors in the ring. */
@@ -1880,7 +1880,7 @@ i40e_dev_rx_queue_setup(struct rte_eth_dev *dev,if (!rxq->sw_ring) {i40e_dev_rx_queue_release (rxq);PMD_DRV_LOG (ERR ,"Failed to allocate memory for SW ring" ); -return (-ENOMEM ); +return -ENOMEM ; }i40e_reset_rx_queue (rxq);
@@ -2105,7 +2105,7 @@ i40e_dev_tx_queue_setup(struct rte_eth_dev *dev,if (!txq) {PMD_DRV_LOG (ERR ,"Failed to allocate memory for " "tx queue structure" ); -return (-ENOMEM ); +return -ENOMEM ; }/* Allocate TX hardware ring descriptors. */
@@ -2119,7 +2119,7 @@ i40e_dev_tx_queue_setup(struct rte_eth_dev *dev,if (!tz) {i40e_dev_tx_queue_release (txq);PMD_DRV_LOG (ERR ,"Failed to reserve DMA memory for TX" ); -return (-ENOMEM ); +return -ENOMEM ; } txq->nb_tx_desc = nb_desc;
@@ -2156,7 +2156,7 @@ i40e_dev_tx_queue_setup(struct rte_eth_dev *dev,if (!txq->sw_ring) {i40e_dev_tx_queue_release (txq);PMD_DRV_LOG (ERR ,"Failed to allocate memory for SW TX ring" ); -return (-ENOMEM ); +return -ENOMEM ; }i40e_reset_tx_queue (txq);
lib/librte_pmd_ixgbe/ixgbe_82599_bypass.c +2 -2
@@ -268,7 +268,7 @@ ixgbe_bypass_get_media_type(struct ixgbe_hw *hw) }else { media_type =ixgbe_get_media_type_82599 (hw); } -return (media_type); +return media_type; }/*
@@ -310,5 +310,5 @@ ixgbe_bypass_init_hw(struct ixgbe_hw *hw) hw->mac.ops.flap_tx_laser =NULL ; } -return (rc); +return rc; }
lib/librte_pmd_ixgbe/ixgbe_bypass.c +1 -1
@@ -153,7 +153,7 @@ ixgbe_bypass_state_show(struct rte_eth_dev *dev, u32 *state)*/ *state = (by_ctl >>BYPASS_STATUS_OFF_SHIFT ) &BYPASS_STATUS_OFF_MASK ; -return (ret_val); +return ret_val; }
lib/librte_pmd_ixgbe/ixgbe_ethdev.c +21 -21
162 lines not on this page · raw message
lib/librte_pmd_ixgbe/ixgbe_rxtx.c +18 -18
@@ -94,7 +94,7 @@ rte_rxmbuf_alloc(struct rte_mempool *mp) m =__rte_mbuf_raw_alloc (mp);__rte_mbuf_sanity_check_raw (m,0 ); -return (m); +return m; }
@@ -461,7 +461,7 @@ what_advctx_update(struct ixgbe_tx_queue *txq, uint64_t flags, }/* Mismatch, use the previous context */ -return (IXGBE_CTX_NUM ); +return IXGBE_CTX_NUM ; } static inline uint32_t
@@ -552,7 +552,7 @@ ixgbe_xmit_cleanup(struct ixgbe_tx_queue *txq) txq->nb_tx_free = (uint16_t )(txq->nb_tx_free + nb_tx_to_clean);/* No Error */ -return (0 ); +return 0 ; }uint16_t
@@ -668,7 +668,7 @@ ixgbe_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,if (ixgbe_xmit_cleanup (txq) !=0 ) {/* Could not clean any descriptors */ if (nb_tx ==0 ) -return (0 ); +return 0 ; goto end_of_tx; }
@@ -697,7 +697,7 @@ ixgbe_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,* descriptors */ if (nb_tx ==0 ) -return (0 ); +return 0 ; goto end_of_tx; } }
@@ -847,7 +847,7 @@ end_of_tx:IXGBE_PCI_REG_WRITE (txq->tdt_reg_addr, tx_id); txq->tx_tail = tx_id; -return (nb_tx); +return nb_tx; }/*********************************************************************
@@ -1037,7 +1037,7 @@ ixgbe_rx_alloc_bufs(struct ixgbe_rx_queue *rxq, bool reset_mbuf) diag =rte_mempool_get_bulk (rxq->mb_pool, (void *)rxep, rxq->rx_free_thresh);if (unlikely (diag !=0 )) -return (-ENOMEM ); +return -ENOMEM ; rxdp = &rxq->rx_ring[alloc_idx];for (i =0 ; i < rxq->rx_free_thresh; ++i) {
@@ -1372,7 +1372,7 @@ ixgbe_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, nb_hold =0 ; } rxq->nb_rx_hold = nb_hold; -return (nb_rx); +return nb_rx; }/**
@@ -2003,7 +2003,7 @@ ixgbe_dev_tx_queue_setup(struct rte_eth_dev *dev, txq =rte_zmalloc_socket ("ethdev TX queue" ,sizeof (struct ixgbe_tx_queue ),RTE_CACHE_LINE_SIZE , socket_id);if (txq ==NULL ) -return (-ENOMEM ); +return -ENOMEM ;/* * Allocate TX ring hardware descriptors. A memzone large enough to
@@ -2015,7 +2015,7 @@ ixgbe_dev_tx_queue_setup(struct rte_eth_dev *dev, socket_id);if (tz ==NULL ) {ixgbe_tx_queue_release (txq); -return (-ENOMEM ); +return -ENOMEM ; } txq->nb_tx_desc = nb_desc;
@@ -2055,7 +2055,7 @@ ixgbe_dev_tx_queue_setup(struct rte_eth_dev *dev, RTE_CACHE_LINE_SIZE, socket_id);if (txq->sw_ring ==NULL ) {ixgbe_tx_queue_release (txq); -return (-ENOMEM ); +return -ENOMEM ; }PMD_INIT_LOG (DEBUG ,"sw_ring=%p hw_ring=%p dma_addr=0x%" PRIx64, txq->sw_ring, txq->tx_ring, txq->tx_ring_phys_addr);
@@ -2068,7 +2068,7 @@ ixgbe_dev_tx_queue_setup(struct rte_eth_dev *dev, dev->data->tx_queues[queue_idx] = txq; -return (0 ); +return 0 ; }/**
@@ -2284,7 +2284,7 @@ ixgbe_dev_rx_queue_setup(struct rte_eth_dev *dev,if (((nb_desc *sizeof (union ixgbe_adv_rx_desc )) %IXGBE_ALIGN ) !=0 || (nb_desc >IXGBE_MAX_RING_DESC ) || (nb_desc <IXGBE_MIN_RING_DESC )) { -return (-EINVAL ); +return -EINVAL ; }/* Free memory prior to re-allocation if needed... */
@@ -2297,7 +2297,7 @@ ixgbe_dev_rx_queue_setup(struct rte_eth_dev *dev, rxq =rte_zmalloc_socket ("ethdev RX queue" ,sizeof (struct ixgbe_rx_queue ),RTE_CACHE_LINE_SIZE , socket_id);if (rxq ==NULL ) -return (-ENOMEM ); +return -ENOMEM ; rxq->mb_pool = mp; rxq->nb_rx_desc = nb_desc; rxq->rx_free_thresh = rx_conf->rx_free_thresh;
@@ -2319,7 +2319,7 @@ ixgbe_dev_rx_queue_setup(struct rte_eth_dev *dev, RX_RING_SZ, socket_id);if (rz ==NULL ) {ixgbe_rx_queue_release (rxq); -return (-ENOMEM ); +return -ENOMEM ; }/*
@@ -2379,7 +2379,7 @@ ixgbe_dev_rx_queue_setup(struct rte_eth_dev *dev, RTE_CACHE_LINE_SIZE, socket_id);if (!rxq->sw_ring) {ixgbe_rx_queue_release (rxq); -return (-ENOMEM ); +return -ENOMEM ; }/*
@@ -2396,7 +2396,7 @@ ixgbe_dev_rx_queue_setup(struct rte_eth_dev *dev, RTE_CACHE_LINE_SIZE, socket_id);if (!rxq->sw_sc_ring) {ixgbe_rx_queue_release (rxq); -return (-ENOMEM ); +return -ENOMEM ; } PMD_INIT_LOG(DEBUG, "sw_ring=%p sw_sc_ring=%p hw_ring=%p "
@@ -3452,7 +3452,7 @@ ixgbe_alloc_rx_queue_mbufs(struct ixgbe_rx_queue *rxq)if (mbuf ==NULL ) {PMD_INIT_LOG (ERR ,"RX mbuf alloc failed queue_id=%u" , (unsigned ) rxq->queue_id); -return (-ENOMEM ); +return -ENOMEM ; }rte_mbuf_refcnt_set (mbuf,1 );
lib/librte_pmd_virtio/virtio_ethdev.c +2 -2
@@ -296,7 +296,7 @@ int virtio_dev_queue_setup(struct rte_eth_dev *dev, }if (vq ==NULL ) {PMD_INIT_LOG (ERR ,"%s: Can not allocate virtqueue" , __func__); -return (-ENOMEM ); +return -ENOMEM ; } vq->hw = hw;
@@ -1293,7 +1293,7 @@ virtio_dev_configure(struct rte_eth_dev *dev)if (rxmode->hw_ip_checksum) {PMD_DRV_LOG (ERR ,"HW IP checksum not supported" ); -return (-EINVAL ); +return -EINVAL ; } hw->vlan_strip = rxmode->hw_vlan_strip;
Signature · 1 line
2.4.0
Thomas Monjalon
AppliedRe: [PATCH] lib: syntax cleanup
2015-05-11 17:10, Ferruh Yigit:
Quoted 34 lines
Remove extra parenthesis from return statements.
Signed-off-by Ferruh Yigit <[email protected]>--- lib/librte_cmdline/cmdline_parse_etheraddr.c | 10 ++--- lib/librte_cmdline/cmdline_parse_ipaddr.c | 38 ++++++++--------- lib/librte_cmdline/cmdline_parse_num.c | 24 +++++------ lib/librte_cmdline/cmdline_parse_portlist.c | 14 +++---- lib/librte_cmdline/cmdline_socket.c | 2 +- lib/librte_eal/bsdapp/contigmem/contigmem.c | 22 +++++----- lib/librte_eal/bsdapp/eal/eal.c | 4 +- lib/librte_eal/bsdapp/eal/eal_pci.c | 12 +++--- lib/librte_eal/bsdapp/nic_uio/nic_uio.c | 10 ++--- lib/librte_eal/common/eal_common_memzone.c | 2 +- lib/librte_eal/common/include/rte_common.h | 2 +- lib/librte_eal/common/include/rte_pci.h | 6 +-- lib/librte_eal/linuxapp/eal/eal.c | 4 +- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 4 +- lib/librte_eal/linuxapp/eal/eal_memory.c | 4 +- .../linuxapp/kni/ethtool/igb/igb_procfs.c | 2 +- lib/librte_eal/linuxapp/kni/kni_net.c | 8 ++-- lib/librte_ip_frag/ip_frag_internal.c | 12 +++--- lib/librte_ip_frag/rte_ip_frag_common.c | 6 +-- lib/librte_ip_frag/rte_ipv4_reassembly.c | 8 ++-- lib/librte_ip_frag/rte_ipv6_fragmentation.c | 8 ++-- lib/librte_lpm/rte_lpm.c | 2 +- lib/librte_mbuf/rte_mbuf.h | 16 ++++---- lib/librte_mempool/rte_dom0_mempool.c | 2 +- lib/librte_mempool/rte_mempool.c | 18 ++++---- lib/librte_net/rte_ip.h | 2 +- lib/librte_pmd_e1000/em_ethdev.c | 36 ++++++++-------- lib/librte_pmd_e1000/em_rxtx.c | 48 +++++++++++----------- lib/librte_pmd_e1000/igb_ethdev.c | 26 ++++++------ lib/librte_pmd_e1000/igb_rxtx.c | 30 +++++++-------
Quoted 8 lines
lib/librte_pmd_fm10k/fm10k_ethdev.c | 30 +++++++------- lib/librte_pmd_i40e/i40e_rxtx.c | 14 +++---- lib/librte_pmd_ixgbe/ixgbe_82599_bypass.c | 4 +- lib/librte_pmd_ixgbe/ixgbe_bypass.c | 2 +- lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 42 +++++++++---------- lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 36 ++++++++-------- lib/librte_pmd_virtio/virtio_ethdev.c | 4 +- 37 files changed, 257 insertions(+), 257 deletions(-)
Applied, thanks
PATCH v5 2/6 eal: add rx queue interrupt FDs to intr handle struct
In reply to a message not in the archivev5 changes:
- Create this new patch file for changed struct rte_intr_handle that other patches depend on, to avoid breaking git bisect.
--- lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h | 4 ++++ 1 file changed, 4 insertions(+)
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +4 -0
@@ -38,6 +38,8 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define VFIO_MAX_QUEUE_ID 32 +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */
@@ -57,6 +59,8 @@ struct rte_intr_handle { };int fd;/**< interrupt event file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +int max_intr;/**< max interrupt requested */ +int queue_fd[VFIO_MAX_QUEUE_ID ];/**< rx and tx queue interrupt file descriptor */ };#endif /* _RTE_LINUXAPP_INTERRUPTS_H_ */
Signature · 1 line
1.8.1.4
PATCH v5 3/6 ixgbe: enable rx queue interrupts for both PF and VF
In reply to a message not in the archivev5 changes
- Rebase the patchset onto the HEAD
v3 changes
- Remove spinlok from PMD
v2 changes
- Consolidate review comments related to coding style
The patch does below things for ixgbe PF and VF:
- Setup NIC to generate MSI-X interrupts
- Set the IVAR register to map interrupt causes to vectors
- Implement interrupt enable/disable functions
--- lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 365 +++++++++++++++++++++++++++++++++++- lib/librte_pmd_ixgbe/ixgbe_ethdev.h | 7 + 2 files changed, 368 insertions(+), 4 deletions(-)
lib/librte_pmd_ixgbe/ixgbe_ethdev.c +361 -4
@@ -83,6 +83,9 @@*/ #define IXGBE_FC_LO 0x40 +/* Default minimum inter-interrupt interval for EITR configuration */ +#define IXGBE_MIN_INTER_INTERRUPT_INTERVAL_DEFAULT 0x79E +/* Timer value included in XOFF frames. */ #define IXGBE_FC_PAUSE 0x680
@@ -173,6 +176,7 @@ static int ixgbe_dev_rss_reta_query(struct rte_eth_dev *dev, uint16_t reta_size);static void ixgbe_dev_link_status_print (struct rte_eth_dev *dev);static int ixgbe_dev_lsc_interrupt_setup (struct rte_eth_dev *dev); +static int ixgbe_dev_rxq_interrupt_setup (struct rte_eth_dev *dev);static int ixgbe_dev_interrupt_get_status (struct rte_eth_dev *dev);static int ixgbe_dev_interrupt_action (struct rte_eth_dev *dev); static void ixgbe_dev_interrupt_handler(struct rte_intr_handle *handle,
@@ -186,11 +190,14 @@ static void ixgbe_dcb_init(struct ixgbe_hw *hw,struct ixgbe_dcb_config *dcb_conf/* For Virtual Function support */ static int eth_ixgbevf_dev_init (struct eth_driver *eth_drv,struct rte_eth_dev *eth_dev); +static int ixgbevf_dev_interrupt_get_status (struct rte_eth_dev *dev); +static int ixgbevf_dev_interrupt_action (struct rte_eth_dev *dev);static int ixgbevf_dev_configure (struct rte_eth_dev *dev);static int ixgbevf_dev_start (struct rte_eth_dev *dev);static void ixgbevf_dev_stop (struct rte_eth_dev *dev);static void ixgbevf_dev_close (struct rte_eth_dev *dev);static void ixgbevf_intr_disable (struct ixgbe_hw *hw); +static void ixgbevf_intr_enable (struct ixgbe_hw *hw);static void ixgbevf_dev_stats_get (struct rte_eth_dev *dev,struct rte_eth_stats *stats);static void ixgbevf_dev_stats_reset (struct rte_eth_dev *dev);
@@ -200,6 +207,15 @@ static void ixgbevf_vlan_strip_queue_set(struct rte_eth_dev *dev, uint16_t queue, int on);static void ixgbevf_vlan_offload_set (struct rte_eth_dev *dev,int mask);static void ixgbevf_set_vfta_all (struct rte_eth_dev *dev,bool on); +static void ixgbevf_dev_interrupt_handler (struct rte_intr_handle *handle, +void *param); +static int ixgbevf_dev_rx_queue_intr_enable (struct rte_eth_dev *dev, +uint16_t queue_id); +static int ixgbevf_dev_rx_queue_intr_disable (struct rte_eth_dev *dev, +uint16_t queue_id); +static void ixgbevf_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector); +static void ixgbevf_configure_msix (struct ixgbe_hw *hw);/* For Eth VMDQ APIs support */ static int ixgbe_uc_hash_table_set(struct rte_eth_dev *dev, struct
@@ -217,6 +233,14 @@ static int ixgbe_mirror_rule_set(struct rte_eth_dev *dev,static int ixgbe_mirror_rule_reset (struct rte_eth_dev *dev,uint8_t rule_id); +static int ixgbe_dev_rx_queue_intr_enable (struct rte_eth_dev *dev, +uint16_t queue_id); +static int ixgbe_dev_rx_queue_intr_disable (struct rte_eth_dev *dev, +uint16_t queue_id); +static void ixgbe_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector); +static void ixgbe_configure_msix (struct ixgbe_hw *hw); +static int ixgbe_set_queue_rate_limit (struct rte_eth_dev *dev,uint16_t queue_idx,uint16_t tx_rate); static int ixgbe_set_vf_rate_limit(struct rte_eth_dev *dev, uint16_t vf,
@@ -265,7 +289,7 @@ static int ixgbevf_dev_set_mtu(struct rte_eth_dev *dev, uint16_t mtu);*/ #define UPDATE_VF_STAT (reg, last, cur) \ { \ - u32 latest = IXGBE_READ_REG(hw, reg); \ + uint32_t latest = IXGBE_READ_REG(hw, reg); \ cur += latest - last; \ last = latest; \ }
@@ -346,6 +370,8 @@ static struct eth_dev_ops ixgbe_eth_dev_ops = { .tx_queue_start = ixgbe_dev_tx_queue_start, .tx_queue_stop = ixgbe_dev_tx_queue_stop, .rx_queue_setup = ixgbe_dev_rx_queue_setup, + .rx_queue_intr_enable = ixgbe_dev_rx_queue_intr_enable, + .rx_queue_intr_disable = ixgbe_dev_rx_queue_intr_disable, .rx_queue_release = ixgbe_dev_rx_queue_release, .rx_queue_count = ixgbe_dev_rx_queue_count, .rx_descriptor_done = ixgbe_dev_rx_descriptor_done,
@@ -406,8 +432,11 @@ static struct eth_dev_ops ixgbevf_eth_dev_ops = { .vlan_offload_set = ixgbevf_vlan_offload_set, .rx_queue_setup = ixgbe_dev_rx_queue_setup, .rx_queue_release = ixgbe_dev_rx_queue_release, + .rx_descriptor_done = ixgbe_dev_rx_descriptor_done, .tx_queue_setup = ixgbe_dev_tx_queue_setup, .tx_queue_release = ixgbe_dev_tx_queue_release, + .rx_queue_intr_enable = ixgbevf_dev_rx_queue_intr_enable, + .rx_queue_intr_disable = ixgbevf_dev_rx_queue_intr_disable, .mac_addr_add = ixgbevf_add_mac_addr, .mac_addr_remove = ixgbevf_remove_mac_addr, };
@@ -904,6 +933,10 @@ eth_ixgbe_dev_init(__attribute__((unused)) struct eth_driver *eth_drv, eth_dev->data->port_id, pci_dev->id.vendor_id, pci_dev->id.device_id); +/* set max interrupt vfio request */ + pci_dev->intr_handle.max_intr = hw->mac.max_rx_queues + +IXGBE_MAX_OTHER_INTR ; +rte_intr_callback_register (&(pci_dev->intr_handle), ixgbe_dev_interrupt_handler, (void *)eth_dev);
@@ -1085,6 +1118,15 @@ eth_ixgbevf_dev_init(__attribute__((unused)) struct eth_driver *eth_drv,return (-EIO ); } +/* set max interrupt vfio request */ + pci_dev->intr_handle.max_intr = hw->mac.max_rx_queues + +IXGBEVF_MAX_OTHER_INTR ; + +rte_intr_callback_register (&(pci_dev->intr_handle), + ixgbevf_dev_interrupt_handler, (void *)eth_dev); + +rte_intr_enable (&(pci_dev->intr_handle)); +PMD_INIT_LOG (DEBUG ,"port %d vendorID=0x%x deviceID=0x%x mac.type=%s" , eth_dev->data->port_id, pci_dev->id.vendor_id, pci_dev->id.device_id,"ixgbe_mac_82599_vf" );
@@ -1486,6 +1528,9 @@ ixgbe_dev_start(struct rte_eth_dev *dev)/* configure PF module if SRIOV enabled */ ixgbe_pf_host_configure (dev); +/* confiugre msix for sleep until rx interrupt */ +ixgbe_configure_msix (hw); +/* initialize transmission unit */ ixgbe_dev_tx_init (dev);
@@ -1561,6 +1606,10 @@ skip_link_setup:if (dev->data->dev_conf.intr_conf.lsc !=0 )ixgbe_dev_lsc_interrupt_setup (dev); +/* check if rxq interrupt is enabled */ +if (dev->data->dev_conf.intr_conf.rxq !=0 ) +ixgbe_dev_rxq_interrupt_setup (dev); +/* resume enabled intr since hw reset */ ixgbe_enable_intr (dev);
@@ -2238,6 +2287,28 @@ ixgbe_dev_lsc_interrupt_setup(struct rte_eth_dev *dev)return 0 ; } +/** +* It clears the interrupt causes and enables the interrupt. +* It will be called once only during nic initialized. +* +* @param dev +* Pointer to struct rte_eth_dev. +* +* @return +* - On success, zero. +* - On failure, a negative value. +*/ +static int +ixgbe_dev_rxq_interrupt_setup (struct rte_eth_dev *dev) +{ +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + + intr->mask |=IXGBE_EICR_RTX_QUEUE ; + +return 0 ; +} +/* * It reads ICR and sets flag (IXGBE_EICR_LSC) for the link_update. *
@@ -2264,10 +2335,10 @@ ixgbe_dev_interrupt_get_status(struct rte_eth_dev *dev)PMD_DRV_LOG (INFO ,"eicr %x" , eicr); intr->flags =0 ; -if (eicr &IXGBE_EICR_LSC ) { -/* set flag for async link update */ + +/* set flag for async link update */ +if (eicr &IXGBE_EICR_LSC ) intr->flags |=IXGBE_FLAG_NEED_LINK_UPDATE ; - }if (eicr &IXGBE_EICR_MAILBOX ) intr->flags |=IXGBE_FLAG_MAILBOX ;
@@ -2275,6 +2346,30 @@ ixgbe_dev_interrupt_get_status(struct rte_eth_dev *dev)return 0 ; } +static int +ixgbevf_dev_interrupt_get_status (struct rte_eth_dev *dev) +{ +uint32_t eicr; +struct ixgbe_hw *hw =IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + +/* clear all cause mask */ +ixgbevf_intr_disable (hw); + +/* read-on-clear nic registers here */ + eicr =IXGBE_READ_REG (hw,IXGBE_VTEICR ); +PMD_DRV_LOG (INFO ,"eicr %x" , eicr); + + intr->flags =0 ; + +/* set flag for async link update */ +if (eicr &IXGBE_EICR_LSC ) + intr->flags |=IXGBE_FLAG_NEED_LINK_UPDATE ; + +return 0 ; +} +/** * It gets and then prints the link status. *
@@ -2370,6 +2465,18 @@ ixgbe_dev_interrupt_action(struct rte_eth_dev *dev)return 0 ; } +static int +ixgbevf_dev_interrupt_action (struct rte_eth_dev *dev) +{ +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); + +PMD_DRV_LOG (DEBUG ,"enable intr immediately" ); +ixgbevf_intr_enable (hw); +rte_intr_enable (&(dev->pci_dev->intr_handle)); +return 0 ; +} +/** * Interrupt handler which shall be registered for alarm callback for delayed * handling specific interrupt to wait for the stable nic state. As the
@@ -2431,6 +2538,15 @@ ixgbe_dev_interrupt_handler(__rte_unused struct rte_intr_handle *handle,ixgbe_dev_interrupt_action (dev); } +static void +ixgbevf_dev_interrupt_handler(__rte_unused struct rte_intr_handle *handle, +void *param) +{ +struct rte_eth_dev *dev = (struct rte_eth_dev *)param; +ixgbevf_dev_interrupt_get_status (dev); +ixgbevf_dev_interrupt_action (dev); +} +static int ixgbe_dev_led_on (struct rte_eth_dev *dev) {
@@ -2929,6 +3045,19 @@ ixgbevf_intr_disable(struct ixgbe_hw *hw)IXGBE_WRITE_FLUSH (hw); } +static void +ixgbevf_intr_enable (struct ixgbe_hw *hw) +{ +PMD_INIT_FUNC_TRACE (); + +/* VF enable interrupt autoclean */ +IXGBE_WRITE_REG (hw,IXGBE_VTEIAM ,IXGBE_VF_IRQ_ENABLE_MASK ); +IXGBE_WRITE_REG (hw,IXGBE_VTEIAC ,IXGBE_VF_IRQ_ENABLE_MASK ); +IXGBE_WRITE_REG (hw,IXGBE_VTEIMS ,IXGBE_VF_IRQ_ENABLE_MASK ); + +IXGBE_WRITE_FLUSH (hw); +} +static int ixgbevf_dev_configure (struct rte_eth_dev *dev) {
@@ -2991,6 +3120,11 @@ ixgbevf_dev_start(struct rte_eth_dev *dev)ixgbevf_dev_rxtx_start (dev); +ixgbevf_configure_msix (hw); + +/* Re-enable interrupt for VF */ +ixgbevf_intr_enable (hw); +return 0 ; }
@@ -3528,6 +3662,229 @@ ixgbe_mirror_rule_reset(struct rte_eth_dev *dev, uint8_t rule_id)return 0 ; } + +static int +ixgbevf_dev_rx_queue_intr_enable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); + + mask =IXGBE_READ_REG (hw,IXGBE_VTEIMS ); + mask |= (1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_VTEIMS , mask); + +return 0 ; +} + +static int +ixgbevf_dev_rx_queue_intr_disable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); + + mask =IXGBE_READ_REG (hw,IXGBE_VTEIMS ); + mask &= ~(1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_VTEIMS , mask); + +return 0 ; +} + +static int +ixgbe_dev_rx_queue_intr_enable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + +if (queue_id <16 ) { +ixgbe_disable_intr (hw); + intr->mask |= (1 << queue_id); +ixgbe_enable_intr (dev); + }else if (queue_id <32 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (0 )); + mask &= (1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (0 ), mask); + }else if (queue_id <64 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (1 )); + mask &= (1 << (queue_id -32 )); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (1 ), mask); + } + +return 0 ; +} + +static int +ixgbe_dev_rx_queue_intr_disable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + +if (queue_id <16 ) { +ixgbe_disable_intr (hw); + intr->mask &= ~(1 << queue_id); +ixgbe_enable_intr (dev); + }else if (queue_id <32 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (0 )); + mask &= ~(1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (0 ), mask); + }else if (queue_id <64 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (1 )); + mask &= ~(1 << (queue_id -32 )); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (1 ), mask); + } + +return 0 ; +} + +static void +ixgbevf_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector) +{ +uint32_t tmp, idx; +if (direction ==-1 ) { +/* other causes */ + msix_vector |=IXGBE_IVAR_ALLOC_VAL ; + tmp =IXGBE_READ_REG (hw,IXGBE_VTIVAR_MISC ); + tmp &= ~0xFF ; + tmp |= msix_vector; +IXGBE_WRITE_REG (hw,IXGBE_VTIVAR_MISC , tmp); + }else { +/* rx or tx cause */ + msix_vector |=IXGBE_IVAR_ALLOC_VAL ; + idx = ((16 * (queue &1 )) + (8 * direction)); + tmp =IXGBE_READ_REG (hw,IXGBE_VTIVAR (queue >>1 )); + tmp &= ~(0xFF << idx); + tmp |= (msix_vector << idx); +IXGBE_WRITE_REG (hw,IXGBE_VTIVAR (queue >>1 ), tmp); + } +} + +/** +* set the IVAR registers, mapping interrupt causes to vectors +* @param hw +* pointer to ixgbe_hw struct +* @direction +* 0 for Rx, 1 for Tx, -1 for other causes +* @queue +* queue to map the corresponding interrupt to +* @msix_vector +* the vector to map to the corresponding queue +*/ +static void +ixgbe_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector) +{ +uint32_t tmp, idx; + + msix_vector |=IXGBE_IVAR_ALLOC_VAL ; +if (hw->mac.type == ixgbe_mac_82598EB) { +if (direction ==-1 ) + direction =0 ; + idx = (((direction *64 ) + queue) >>2 ) &0x1F ; + tmp =IXGBE_READ_REG (hw,IXGBE_IVAR (idx)); + tmp &= ~(0xFF << (8 * (queue &0x3 ))); + tmp |= (msix_vector << (8 * (queue &0x3 ))); +IXGBE_WRITE_REG (hw,IXGBE_IVAR (idx), tmp); + }else if ((hw->mac.type == ixgbe_mac_82599EB) || + (hw->mac.type == ixgbe_mac_X540)) { +if (direction ==-1 ) { +/* other causes */ + idx = ((queue &1 ) *8 ); + tmp =IXGBE_READ_REG (hw,IXGBE_IVAR_MISC ); + tmp &= ~(0xFF << idx); + tmp |= (msix_vector << idx); +IXGBE_WRITE_REG (hw,IXGBE_IVAR_MISC , tmp); + }else { +/* rx or tx causes */ + idx = ((16 * (queue &1 )) + (8 * direction)); + tmp =IXGBE_READ_REG (hw,IXGBE_IVAR (queue >>1 )); + tmp &= ~(0xFF << idx); + tmp |= (msix_vector << idx); +IXGBE_WRITE_REG (hw,IXGBE_IVAR (queue >>1 ), tmp); + } + } +} + +static void +ixgbevf_configure_msix (struct ixgbe_hw *hw) +{ +uint32_t q_idx, vector_idx; +/* Configure all RX queues of VF */ +for (vector_idx =0 ; vector_idx <IXGBE_VF_MAXMSIVECTOR ; vector_idx++) { +for (q_idx =0 ; q_idx < (hw->mac.max_rx_queues -1 ); q_idx++) +ixgbevf_set_ivar_map (hw,0 , q_idx, vector_idx); + } + +/* Configure VF Rx queue ivar */ +ixgbevf_set_ivar_map (hw,-1 ,1 , vector_idx); +} + +/** +* Sets up the hardware to properly generate MSI-X interrupts +* @hw +* board private structure +*/ +static void +ixgbe_configure_msix (struct ixgbe_hw *hw) +{ +int queue_id; +uint32_t mask; +uint32_t gpie; + +/* setup GPIE for MSI-x mode */ + gpie =IXGBE_READ_REG (hw,IXGBE_GPIE ); + gpie |=IXGBE_GPIE_MSIX_MODE |IXGBE_GPIE_PBA_SUPPORT | +IXGBE_GPIE_OCD |IXGBE_GPIE_EIAME ; +/* +* auto clearing and auto setting corresponding bits in EIMS +* when MSI-X interrupt is triggered +*/ +if (hw->mac.type == ixgbe_mac_82598EB) +IXGBE_WRITE_REG (hw,IXGBE_EIAM ,IXGBE_EICS_RTX_QUEUE ); +else { +IXGBE_WRITE_REG (hw,IXGBE_EIAM_EX (0 ),0xFFFFFFFF ); +IXGBE_WRITE_REG (hw,IXGBE_EIAM_EX (1 ),0xFFFFFFFF ); + } +IXGBE_WRITE_REG (hw,IXGBE_GPIE , gpie); + +/* +* Populate the IVAR table and set the ITR values to the +* corresponding register. +*/ +for (queue_id =0 ; queue_id <VFIO_MAX_QUEUE_ID ; queue_id++) +ixgbe_set_ivar_map (hw,0 , queue_id, queue_id); + +switch (hw->mac.type) { +case ixgbe_mac_82598EB: +ixgbe_set_ivar_map (hw,-1 ,IXGBE_IVAR_OTHER_CAUSES_INDEX , +VFIO_MAX_QUEUE_ID ); +break ; +case ixgbe_mac_82599EB: +case ixgbe_mac_X540: +ixgbe_set_ivar_map (hw,-1 ,1 ,VFIO_MAX_QUEUE_ID ); +break ; +default : +break ; + } +IXGBE_WRITE_REG (hw,IXGBE_EITR (queue_id), +IXGBE_MIN_INTER_INTERRUPT_INTERVAL_DEFAULT &0xFFF ); + +/* set up to autoclear timer, and the vectors */ + mask =IXGBE_EIMS_ENABLE_MASK ; + mask &= ~(IXGBE_EIMS_OTHER | +IXGBE_EIMS_MAILBOX | +IXGBE_EIMS_LSC ); + +IXGBE_WRITE_REG (hw,IXGBE_EIAC , mask); +} +static int ixgbe_set_queue_rate_limit (struct rte_eth_dev *dev,uint16_t queue_idx,uint16_t tx_rate) {
lib/librte_pmd_ixgbe/ixgbe_ethdev.h +7 -0
@@ -109,6 +109,12 @@ ETH_RSS_IPV6_TCP_EX | \ ETH_RSS_IPV6_UDP_EX) +#define IXGBE_VF_IRQ_ENABLE_MASK 3/* vf irq enable mask */ +#define IXGBE_VF_MAXMSIVECTOR 1 +/* maximum other interrupts besides rx&tx*/ +#define IXGBE_MAX_OTHER_INTR 1 +#define IXGBEVF_MAX_OTHER_INTR 1 +/* * Information about the fdir mode. */
@@ -317,6 +323,7 @@ uint32_t ixgbe_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id);int ixgbe_dev_rx_descriptor_done (void *rx_queue,uint16_t offset); +int ixgbevf_dev_rx_descriptor_done (void *rx_queue,uint16_t offset);int ixgbe_dev_rx_init (struct rte_eth_dev *dev);
Signature · 1 line
1.8.1.4
PATCH v5 4/6 igb: enable rx queue interrupts for PF
In reply to a message not in the archivev5 changes
- Rebase the patchset onto the HEAD
v3 changes
- Remove unnecessary variables in e1000_mac_info
- Remove spinlok from PMD
v2 changes
- Consolidate review comments related to coding style
The patch does below for igb PF:
- Setup NIC to generate MSI-X interrupts
- Set the IVAR register to map interrupt causes to vectors
- Implement interrupt enable/disable functions
--- lib/librte_pmd_e1000/e1000_ethdev.h | 3 + lib/librte_pmd_e1000/igb_ethdev.c | 228 ++++++++++++++++++++++++++++++++---- 2 files changed, 206 insertions(+), 25 deletions(-)
lib/librte_pmd_e1000/e1000_ethdev.h +3 -0
@@ -108,6 +108,9 @@ ETH_RSS_IPV6_TCP_EX | \ ETH_RSS_IPV6_UDP_EX) +/* maximum number of other interrupts besides Rx & Tx interrupts */ +#define E1000_MAX_OTHER_INTR 1 +/* structure for interrupt relative data */ struct e1000_interrupt {uint32_t flags;
lib/librte_pmd_e1000/igb_ethdev.c +203 -25
296 lines not on this page · raw message
Signature · 1 line
1.8.1.4
PATCH v5 5/6 eal: add per rx queue interrupt handling based on VFIO
In reply to a message not in the archivev5 changes
- Rebase the patchset onto the HEAD
- Isolate ethdev from EAL for new-added wait-for-rx interrupt function
- Export wait-for-rx interrupt function for shared libraries
v4 changes:
- Adjust position of new-added structure fields
v3 changes:
- Fix review comments
v2 changes:
- Fix compilation issue for a missed header file
- Bug fix: free unreleased resources on the exception path before return
- Consolidate coding style related review comments
This patch does below:
- Create multiple VFIO eventfd for rx queues.
- Handle per rx queue interrupt.
- Eliminate unnecessary suspended DPDK polling thread wakeup mechanism for rx interrupt by allowing polling thread epoll_wait rx queue interrupt notification.
--- lib/librte_eal/bsdapp/eal/rte_eal_version.map | 1 + lib/librte_eal/common/include/rte_eal.h | 1 + lib/librte_eal/common/include/rte_interrupts.h | 12 ++ lib/librte_eal/linuxapp/eal/eal_interrupts.c | 191 ++++++++++++++++++------ lib/librte_eal/linuxapp/eal/eal_pci_vfio.c | 12 +- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 1 + 6 files changed, 174 insertions(+), 44 deletions(-)
lib/librte_eal/bsdapp/eal/rte_eal_version.map +1 -0
@@ -57,6 +57,7 @@ DPDK_2.0 { rte_intr_callback_unregister; rte_intr_disable; rte_intr_enable; + rte_intr_wait_rx_pkt; rte_log; rte_log_add_in_history; rte_log_cur_msg_loglevel;
lib/librte_eal/common/include/rte_eal.h +1 -0
@@ -150,6 +150,7 @@ int rte_eal_iopl_init(void);* - On failure, a negative error value. */ int rte_eal_init (int argc,char **argv); +/** * Usage function typedef used by the application usage function. *
lib/librte_eal/common/include/rte_interrupts.h +12 -0
@@ -113,6 +113,18 @@ int rte_intr_enable(struct rte_intr_handle *intr_handle);*/ int rte_intr_disable (struct rte_intr_handle *intr_handle); +/** +* @param intr_handle +* pointer to the interrupt handle. +* @param queue_id +* the queue id +* @return +* - On success, return 0 +* - On failure, returns -1. +*/ +int rte_intr_wait_rx_pkt (struct rte_intr_handle *intr_handle, +uint8_t queue_id); + #ifdef __cplusplus }#endif
lib/librte_eal/linuxapp/eal/eal_interrupts.c +148 -43
255 lines not on this page · raw message
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +11 -1
@@ -283,11 +283,21 @@ pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd) dev->intr_handle.fd = fd; dev->intr_handle.vfio_dev_fd = vfio_dev_fd; -switch (i) {case VFIO_PCI_MSIX_IRQ_INDEX : internal_config.vfio_intr_mode =RTE_INTR_MODE_MSIX ; dev->intr_handle.type =RTE_INTR_HANDLE_VFIO_MSIX ; +for (i =0 ; i <VFIO_MAX_QUEUE_ID ; i++) { + fd =eventfd (0 ,0 ); +if (fd <0 ) { +RTE_LOG (ERR ,EAL , +"cannot setup eventfd," +"error %i (%s)\n" , + errno,strerror (errno)); +return -1 ; + } + dev->intr_handle.queue_fd[i] = fd; + }break ;case VFIO_PCI_MSI_IRQ_INDEX : internal_config.vfio_intr_mode =RTE_INTR_MODE_MSI ;
lib/librte_eal/linuxapp/eal/rte_eal_version.map +1 -0
@@ -57,6 +57,7 @@ DPDK_2.0 { rte_intr_callback_unregister; rte_intr_disable; rte_intr_enable; + rte_intr_wait_rx_pkt; rte_log; rte_log_add_in_history; rte_log_cur_msg_loglevel;
Signature · 1 line
1.8.1.4
A message that is not in the archive (sent off-list or not yet received): 1424710542-14637-6-git-send-email-danny.zhou-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH v5 5/6] eal: add per rx queue interrupt handling based on VFIO
Hello Danny,
On Mon, Feb 23, 2015 at 5:55 PM, Zhou Danny <[email protected]> wrote:
[snip]
+/**
Quoted 13 lines
+ * @param
intr_handle
+ * pointer to the interrupt handle.
+ * @paramqueue_id
+ * the queue id
+ * @return
+ * - On success, return 0
+ * - On failure, returns -1.
+ */(quoted diff) +3
-0+int rte_intr_wait_rx_pkt(struct rte_intr_handle *intr_handle, + uint8_t queue_id); +
Quoted 1 line
From my point of view, the
queue_id(just likeport_id) is something that
should be handled by ethdev, not eal.
(quoted diff) +0 -0
lib/librte_eal/linuxapp/eal/eal_interrupts.c
Quoted 5 lines
b/lib/librte_eal/linuxapp/eal/eal_interrupts.c
index 8c5b834..ee0f019 100644lib/librte_eal/linuxapp/eal/eal_interrupts.c +0
-0
[snip]
+int
Quoted 9 lines
+rte_intr_wait_rx_pkt(struct
rte_intr_handle*intr_handle,uint8_tqueue_id)lib/librte_eal/linuxapp/eal/eal_interrupts.c +5
-0+{ +struct epoll_event ev; +unsigned numfds =0 ; + + if (!intr_handle || intr_handle->fd < 0 || intr_handle->uio_cfg_fd< 0)
lib/librte_eal/linuxapp/eal/eal_interrupts.c
Quoted 10 lines
lib/librte_eal/linuxapp/eal/eal_interrupts.c +10
-0+return -1 ; +if (queue_id >=VFIO_MAX_QUEUE_ID ) +return -1 ; + +/* create epoll fd */ +int pfd =epoll_create (1 ); +if (pfd <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot create epoll instance\n" ); +return -1 ; + }
Why recreate the epoll instance at each call to this function ?
+
Quoted 26 lines
lib/librte_eal/linuxapp/eal/eal_interrupts.c +25
-0+rte_spinlock_lock (&intr_lock); + + ev.events =EPOLLIN |EPOLLPRI ; +switch (intr_handle->type) { +case RTE_INTR_HANDLE_UIO : + ev.data.fd = intr_handle->fd; +break ; +#ifdef VFIO_PRESENT +case RTE_INTR_HANDLE_VFIO_MSIX : +case RTE_INTR_HANDLE_VFIO_MSI : +case RTE_INTR_HANDLE_VFIO_LEGACY : + ev.data.fd = intr_handle->queue_fd[queue_id]; +break ; +#endif +default : +rte_spinlock_unlock (&intr_lock); +close (pfd); +return -1 ; + } + +if (epoll_ctl (pfd,EPOLL_CTL_ADD , ev.data.fd, &ev) <0 ) { +RTE_LOG (ERR ,EAL ,"Error adding fd %d epoll_ctl, %s\n" , + intr_handle->queue_fd[queue_id],strerror (errno)); + }else + numfds++;
lib/librte_eal/linuxapp/eal/eal_interrupts.c
Quoted 10 lines
lib/librte_eal/linuxapp/eal/eal_interrupts.c +10
-0+ +rte_spinlock_unlock (&intr_lock); +/* serve the interrupt */ +eal_intr_handle_rx_interrupts (intr_handle, pfd, numfds); + +/** +* when we return, we need to rebuild the +* list of fds to monitor. +*/ +close (pfd);
Why do we need to rebuild this "list of fds" ?
Afaics, the fds we want to observe are not supposed to change in the meantime.
epoll maintains this list, you don't have to care about this.
Looking at this patchset, I think there is a design issue. eal does not need to know about portid neither queueid.
eal can provide an api to retrieve the interrupt fds, configure an epoll instance, wait on an epoll instance etc...
ethdev is then responsible to setup the mapping between port id / queue id and interrupt fds by asking the eal about those fds.
This would result in an eal api even simpler and we could add other fds in a single epoll fd for other uses.
Signature · 1 line
David Marchand
A message that is not in the archive (sent off-list or not yet received): CALwxeUtc7vJFt1qxpLxAnVmsbwoSu8Sn-NkeQavNN_ReNA2EEw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Re: [PATCH v5 5/6] eal: add per rx queue interrupt handling based on VFIO
Thanks for comments and please see my answers inline.
From: David Marchand [mailto:[email protected]]
Sent: Tuesday, February 24, 2015 6:42 PM
To: Zhou, Danny
Cc: [email protected]
Subject: Re: [dpdk-dev] [PATCH v5 5/6] eal: add per rx queue interrupt handling based on VFIO
Hello Danny,
On Mon, Feb 23, 2015 at 5:55 PM, Zhou Danny <[email protected]<mailto:[email protected]>> wrote:
[snip]
+/**
+ * @param intr_handle
+ * pointer to the interrupt handle.
+ * @param queue_id
+ * the queue id
+ * @return
+ * - On success, return 0
+ * - On failure, returns -1.
+ */
+int rte_intr_wait_rx_pkt(struct rte_intr_handle *intr_handle,
+ uint8_t queue_id);
+
From my point of view, the queue_id (just like port_id) is something that should be handled by ethdev, not eal.
DZ: See comments below.
lib/librte_eal/linuxapp/eal/eal_interrupts.c +0 -0
[snip]
+int
+rte_intr_wait_rx_pkt(struct rte_intr_handle *intr_handle, uint8_t queue_id) +{
+ struct epoll_event ev;
+ unsigned numfds = 0;
+
+ if (!intr_handle || intr_handle->fd < 0 || intr_handle->uio_cfg_fd < 0)
+ return -1;
+ if (queue_id >= VFIO_MAX_QUEUE_ID)
+ return -1;
+
+ /* create epoll fd */
+ int pfd = epoll_create(1);
+ if (pfd < 0) {+ RTE_LOG(ERR, EAL, "Cannot create epoll instance\n");
+ return -1;
+ }
Why recreate the epoll instance at each call to this function ?
DZ: To avoid recreating the epoll instance for each queue, the struct rte_intr_handle(or a new structure added to ethdev) should be extended by adding fields storing per-queue pfd. This way, it could reduce user/kernel context switch overhead when calling epoll_create() each time.
Sounds good?
+
+ rte_spinlock_lock(&intr_lock);
+
+ ev.events = EPOLLIN | EPOLLPRI;
+ switch (intr_handle->type) {
+ case RTE_INTR_HANDLE_UIO:
+ ev.data.fd = intr_handle->fd;
+ break;
+#ifdef VFIO_PRESENT
+ case RTE_INTR_HANDLE_VFIO_MSIX:
+ case RTE_INTR_HANDLE_VFIO_MSI:
+ case RTE_INTR_HANDLE_VFIO_LEGACY:+ ev.data.fd = intr_handle->queue_fd[queue_id];
+ break;
+#endif
+ default:
+ rte_spinlock_unlock(&intr_lock); + close(pfd); + return -1;
+ }
+
+ if (epoll_ctl(pfd, EPOLL_CTL_ADD, ev.data.fd, &ev) < 0) {
+ RTE_LOG(ERR, EAL, "Error adding fd %d epoll_ctl, %s\n",
+ intr_handle->queue_fd[queue_id], strerror(errno));
+ } else
+ numfds++;
+
+ rte_spinlock_unlock(&intr_lock); + /* serve the interrupt */ + eal_intr_handle_rx_interrupts(intr_handle, pfd, numfds); + + /** + * when we return, we need to rebuild the + * list of fds to monitor. + */ + close(pfd);
Why do we need to rebuild this "list of fds" ?
Afaics, the fds we want to observe are not supposed to change in the meantime. epoll maintains this list, you don't have to care about this.
Agreed, it is not needed.
Looking at this patchset, I think there is a design issue. eal does not need to know about portid neither queueid.
eal can provide an api to retrieve the interrupt fds, configure an epoll instance, wait on an epoll instance etc... ethdev is then responsible to setup the mapping between port id / queue id and interrupt fds by asking the eal about those fds.
This would result in an eal api even simpler and we could add other fds in a single epoll fd for other uses.
DZ: The queueid is just an index to the queue related eventfd array stored in EAL. If this array is still in the EAL and ethdev can apply for it and setup mapping for certain queue, there might be issue for multiple-process use case where the fd resources allocated for secondary process are not freed if the secondary process exits unexpectedly.
Probably we can setup the eventfd array inside ethdev, and we just need EAL API to wait for ethdev’fd. So application invokes ethdev API with portid and queueid, and ethdev calls eal API to wait on a ethdev fd which correlates with the specified portid and queueid.
Sounds ok to you?
I am going to travel tomorrow and Steve Liang might follow up on V6 patch submission when I am absent. Thanks Steve!
--
David Marchand
A message that is not in the archive (sent off-list or not yet received): DFDF335405C17848924A094BC35766CF0AABC01C-0J0gbvR4kTg/[email protected]
Re: [PATCH v5 5/6] eal: add per rx queue interrupt handling based on VFIO
Hello Danny,
Quoted 30 lines · Danny
(quoted diff) +2
-0+int +rte_intr_wait_rx_pkt(struct rte_intr_handle *intr_handle, uint8_t
queue_id)(quoted diff) +5
-0+{ + struct epoll_event ev; + unsigned numfds = 0; + + if (!intr_handle || intr_handle->fd < 0 || intr_handle->uio_cfg_fd< 0)
(quoted diff) +10
-0+ return -1; + if (queue_id >= VFIO_MAX_QUEUE_ID) + return -1; + + /* create epoll fd */ + int pfd = epoll_create(1); + if (pfd < 0) { + RTE_LOG(ERR, EAL, "Cannot create epoll instance\n"); + return -1; + }Why recreate the epoll instance at each call to this function ?
DZ: To avoid recreating the epoll instance for each queue, the struct
rte_intr_handle(or a new structure added to ethdev)
Quoted 9 lines
should be extended by adding fields storing per-queue pfd. This way, it could reduce user/kernel context switch overhead
when calling
epoll_create()each time.Sounds good?
You don't need a epfd per queue. And hardcoding epfd == eventfd will give a not very usable api.
Plus, epoll is something linux-specific, so you can't move it out of eal/linux.
I suppose you need an abstraction here (and in the future we could add something for bsd ?).
Quoted 20 lines
Looking at this patchset, I think there is a design issue.
eal does not need to know about portid neither queueid.
eal can provide an api to retrieve the interrupt fds, configure an epoll instance, wait on an epoll instance etc...
ethdev is then responsible to setup the mapping between port id / queue id and interrupt fds by asking the eal about those fds.
This would result in an eal api even simpler and we could add other fds in a single epoll fd for other uses.
Quoted 8 lines
DZ: The queueid is just an index to the queue related eventfd array stored in EAL. If this array is still in the EAL and ethdev can apply for it and setup mapping for certain queue, there
might be issue for multiple-process use case where the fd resources allocated for secondary process are not freed if the secondary process exits unexpectedly.
Not sure I follow you.
If a secondary process exits, the eventfds created in primary process should still be valid and reusable.
Why would you need to free them ? Something to do with vfio ?
Quoted 5 lines
Probably we can setup the eventfd array inside ethdev, and we just need EAL API to wait for ethdev’fd. So application invokes ethdev API with portid and queueid, and ethdev calls eal
Quoted 7 lines
API to wait on a ethdev fd which correlates with the specified portid and queueid.
Sounds ok to you?
eventfds creation can not be handled by ethdev, since it needs infrastructure and informations from within the eal/linux. Again, do we need an abstraction ?
ethdev must be the one that does the mappings between port/queue and eventfds (or any object that represents a way to wake up for a given port/queue).
Signature · 1 line
David Marchand
A message that is not in the archive (sent off-list or not yet received): CALwxeUvwU=SqWwtDWKf7ZTTVFu16XhVVPEmnmz1euHY_8xoYAA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Re: [PATCH v5 5/6] eal: add per rx queue interrupt handling based on VFIO
From: David Marchand [mailto:[email protected]]
Sent: Wednesday, February 25, 2015 6:22 PM
To: Zhou, Danny
Cc: [email protected]; Liang, Cunming
Subject: Re: [dpdk-dev] [PATCH v5 5/6] eal: add per rx queue interrupt handling based on VFIO
Hello Danny,
On Wed, Feb 25, 2015 at 7:58 AM, Zhou, Danny <[email protected]<mailto:[email protected]>> wrote:
+int
+rte_intr_wait_rx_pkt(struct rte_intr_handle *intr_handle, uint8_t queue_id) +{
+ struct epoll_event ev;
+ unsigned numfds = 0;
+
+ if (!intr_handle || intr_handle->fd < 0 || intr_handle->uio_cfg_fd < 0)
+ return -1;
+ if (queue_id >= VFIO_MAX_QUEUE_ID)
+ return -1;
+
+ /* create epoll fd */
+ int pfd = epoll_create(1);
+ if (pfd < 0) {+ RTE_LOG(ERR, EAL, "Cannot create epoll instance\n");
+ return -1;
+ }
Why recreate the epoll instance at each call to this function ?
DZ: To avoid recreating the epoll instance for each queue, the struct rte_intr_handle(or a new structure added to ethdev) should be extended by adding fields storing per-queue pfd. This way, it could reduce user/kernel context switch overhead when calling epoll_create() each time.
Sounds good?
You don't need a epfd per queue. And hardcoding epfd == eventfd will give a not very usable api.
Plus, epoll is something linux-specific, so you can't move it out of eal/linux. I suppose you need an abstraction here (and in the future we could add something for bsd ?).
DZ: libev provides abstraction layer which is a good candidate to integrate, rather than reinventing one I think. The BSD support can be implemented in the files under lib\librte_eal\bsdapp\eal folder by calling BSD specific APIs. Maybe it is a good idea to introduce a separated component like OS Adaption Layer into EAL in the future once DPDK is widely adopted in BSD as well as Windows, then all DPDK components invoke Linux specific APIs could instead calling abstraction APIs.
Adding an abstraction here specifically for epoll does not resolve all the porting/migration problem in my mind.
Looking at this patchset, I think there is a design issue. eal does not need to know about portid neither queueid.
eal can provide an api to retrieve the interrupt fds, configure an epoll instance, wait on an epoll instance etc... ethdev is then responsible to setup the mapping between port id / queue id and interrupt fds by asking the eal about those fds.
This would result in an eal api even simpler and we could add other fds in a single epoll fd for other uses.
DZ: The queueid is just an index to the queue related eventfd array stored in EAL. If this array is still in the EAL and ethdev can apply for it and setup mapping for certain queue, there might be issue for multiple-process use case where the fd resources allocated for secondary process are not freed if the secondary process exits unexpectedly.
Not sure I follow you.
If a secondary process exits, the eventfds created in primary process should still be valid and reusable. Why would you need to free them ? Something to do with vfio ?
DZ: See below.
Probably we can setup the eventfd array inside ethdev, and we just need EAL API to wait for ethdev’fd. So application invokes ethdev API with portid and queueid, and ethdev calls eal API to wait on a ethdev fd which correlates with the specified portid and queueid.
Sounds ok to you?
eventfds creation can not be handled by ethdev, since it needs infrastructure and informations from within the eal/linux. Again, do we need an abstraction ?
ethdev must be the one that does the mappings between port/queue and eventfds (or any object that represents a way to wake up for a given port/queue).
DZ: agreed after revisiting code. Let us follow your direction to create a ethdev API, similar to rte_eth_dev_rx_queue_intr_enable()/rte_eth_dev_rx_queue_intr_disable(), to use portiid and queueid as arguments. Then this ethdev API uses the mapped eventfds to invoke corresponding EAL API, waiting for interrupt event notification from kernel. A V6 patchset will be created for this.
--
David Marchand
A message that is not in the archive (sent off-list or not yet received): DFDF335405C17848924A094BC35766CF0AABCDA3-0J0gbvR4kTg/[email protected]
Re: [PATCH v5 5/6] eal: add per rx queue interrupt handling based on VFIO
Please Danny, click on the button "uninstall Outlook" or configure it to have quote marks.
This email is really hard to read.
2015-02-25 15:29, Zhou, Danny:
Quoted 38 lines
From: David Marchand [mailto:[email protected]]
Sent: Wednesday, February 25, 2015 6:22 PM
To: Zhou, Danny
Cc: [email protected]; Liang, Cunming
Subject: Re: [dpdk-dev] [PATCH v5 5/6] eal: add per rx queue interrupt handling based on VFIOHello Danny,
On Wed, Feb 25, 2015 at 7:58 AM, Zhou, Danny <[email protected]<mailto:[email protected]>> wrote:
(quoted diff) +17
-0+int +rte_intr_wait_rx_pkt(struct rte_intr_handle *intr_handle, uint8_t queue_id) +{ + struct epoll_event ev; + unsigned numfds = 0; + + if (!intr_handle || intr_handle->fd < 0 || intr_handle->uio_cfg_fd < 0) + return -1; + if (queue_id >= VFIO_MAX_QUEUE_ID) + return -1; + + /* create epoll fd */ + int pfd = epoll_create(1); + if (pfd < 0) { + RTE_LOG(ERR, EAL, "Cannot create epoll instance\n"); + return -1; + }Why recreate the epoll instance at each call to this function ?
DZ: To avoid recreating the epoll instance for each queue, the struct
rte_intr_handle(or a new structure added to ethdev)should be extended by adding fields storing per-queue pfd. This way, it could reduce user/kernel context switch overhead when callingepoll_create()each time.Sounds good?
You don't need a epfd per queue. And hardcoding epfd == eventfd will give a not very usable api.
Quoted 8 lines
Plus, epoll is something linux-specific, so you can't move it out of eal/linux. I suppose you need an abstraction here (and in the future we could add something for bsd ?).
DZ: libev provides abstraction layer which is a good candidate to integrate, rather than reinventing one I think. The BSD support can be implemented in the files under lib\librte_eal\bsdapp\eal folder by calling BSD specific APIs. Maybe it is a good idea to introduce a separated component like OS Adaption Layer into EAL in the future once DPDK is widely adopted in BSD as well as Windows, then all DPDK components invoke Linux specific APIs could instead calling abstraction APIs.
EAL means Environment Abstraction Layer.
In my mind, OS is part of the environment.
DPDK components don't invoke Linux specific APIs, they use EAL! What are you thinking about?
Quoted 1 line
Adding an abstraction here specifically for epoll does not resolve all the porting/migration problem in my mind.
Re: [PATCH v5 5/6] eal: add per rx queue interrupt handling based on VFIO
In reply to a message not in the archiveQuoted 54 lines · Danny
*From:* David Marchand [mailto:[email protected]] *Sent:* Wednesday, February 25, 2015 6:22 PM
*To:* Zhou, Danny
*Cc:* [email protected]; Liang, Cunming
*Subject:* Re: [dpdk-dev] [PATCH v5 5/6] eal: add per rx queue interrupt handling based on VFIODZ: To avoid recreating the epoll instance for each queue, the struct
rte_intr_handle(or a new structure added to ethdev)should be extended by adding fields storing per-queue pfd. This way, it could reduce user/kernel context switch overhead
when calling
epoll_create()each time.Sounds good?
You don't need a epfd per queue. And hardcoding epfd == eventfd will give a not very usable api.
Plus, epoll is something linux-specific, so you can't move it out of eal/linux.
I suppose you need an abstraction here (and in the future we could add something for bsd ?).
DZ: libev provides abstraction layer which is a good candidate to integrate, rather than
reinventing one I think. The BSD support can be implemented in the files under
lib\librte_eal\bsdapp\eal folder by calling BSD specific APIs. Maybe it is a good idea to introduce
a separated component like OS Adaption Layer into EAL in the future once DPDK is widely adopted in
Quoted 8 lines
BSD as well as Windows, then all DPDK components invoke Linux specific APIs could instead calling abstraction APIs.
Adding an abstraction here specifically for epoll does not resolve all the porting/migration problem in my mind.
Yes, reusing this kind of library (or libevent) looks like a good idea.
Hum, I would say eal/common is there for the common part and for the different abstractions.
Do you see anything that would not fit in ?
Quoted 13 lines
eventfds creation can not be handled by ethdev, since it needs
infrastructure and informations from within the eal/linux.Again, do we need an abstraction ?
ethdev must be the one that does the mappings between port/queue and eventfds (or any object that represents a way to wake up for a given port/queue).
Quoted 7 lines
DZ: agreed after revisiting code. Let us follow your direction to create a ethdev API, similar to rte_eth_dev_rx_queue_intr_enable()/rte_eth_dev_rx_queue_intr_disable(), to use portiid and queueid as arguments. Then this ethdev API uses the mapped eventfds to invoke corresponding EAL API, waiting for interrupt event notification from kernel. A V6 patchset will be created for this.
Ok, I will look at it when available.
Signature · 1 line
David Marchand
PATCH v5 6/6 l3fwd-power: enable one-shot rx interrupt and polling/interrupt mode switch
In reply to a message not in the archivev5 changes
- Change invoked function name and parameter to accomodate EAL change
v3 changes
- Add spinlock to ensure thread safe when accessing interrupt mask register
v2 changes
- Remove unused function which is for debug purpose
Demonstrate how to handle per rx queue interrupt in a NAPI-like implementation in usersapce. PDK polling thread mainly works in polling mode and switch to interrupt mode only if there is no any packet received in recent polls.
Usersapce interrupt notification generally takes a lot more cycles than kernel, so one-shot interrupt is used here to guarantee minimum overhead and DPDK polling thread returns to polling mode immediately once it receives an interrupt notificaiton for incoming packet.
--- examples/l3fwd-power/main.c | 155 ++++++++++++++++++++++++++++++++------------ 1 file changed, 114 insertions(+), 41 deletions(-)
examples/l3fwd-power/main.c +114 -41
@@ -75,12 +75,14 @@#include <rte_string_fns.h> #include <rte_timer.h> #include <rte_power.h> +#include <rte_eal.h> +#include <rte_spinlock.h> #define RTE_LOGTYPE_L3FWD_POWER RTE_LOGTYPE_USER1#define MAX_PKT_BURST 32 -#define MIN_ZERO_POLL_COUNT 5 +#define MIN_ZERO_POLL_COUNT 10/* around 100ms at 2 Ghz */ #define TIMER_RESOLUTION_CYCLES 200000000ULL
@@ -156,6 +158,9 @@ static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT;/* ethernet addresses of ports */ static struct ether_addr ports_eth_addr[RTE_MAX_ETHPORTS ]; +/* ethernet addresses of ports */ +static rte_spinlock_t locks[RTE_MAX_ETHPORTS ]; +/* mask of enabled ports */ static uint32_t enabled_port_mask =0 ;/* Ports set in promiscuous mode off by default. */
@@ -188,6 +193,9 @@ struct lcore_rx_queue {#define MAX_TX_QUEUE_PER_PORT RTE_MAX_ETHPORTS#define MAX_RX_QUEUE_PER_PORT 128 +#define MAX_RX_QUEUE_INTERRUPT_PER_PORT 16 + +#define MAX_LCORE_PARAMS 1024struct lcore_params {uint8_t port_id;
@@ -214,7 +222,7 @@ static uint16_t nb_lcore_params = sizeof(lcore_params_array_default) /static struct rte_eth_conf port_conf = { .rxmode = { - .mq_mode =ETH_MQ_RX_RSS , + .mq_mode =ETH_MQ_RX_RSS , .max_rx_pkt_len =ETHER_MAX_LEN , .split_hdr_size =0 , .header_split =0 ,/**< Header Split disabled */
@@ -226,11 +234,14 @@ static struct rte_eth_conf port_conf = { .rx_adv_conf = { .rss_conf = { .rss_key = NULL, - .rss_hf = ETH_RSS_IP, + .rss_hf = ETH_RSS_UDP, }, }, .txmode = { - .mq_mode = ETH_DCB_NONE, + .mq_mode = ETH_MQ_TX_NONE, + }, + .intr_conf = { + .rxq = 1, /**< rxq interrupt feature enabled */ }, };
@@ -402,19 +413,22 @@ power_timer_cb(__attribute__((unused)) struct rte_timer *tim,/* accumulate total execution time in us when callback is invoked */ sleep_time_ratio = (float )(stats[lcore_id].sleep_time) / (float )SCALING_PERIOD ; -/** * check whether need to scale down frequency a step if it sleep a lot. */ -if (sleep_time_ratio >=SCALING_DOWN_TIME_RATIO_THRESHOLD ) -rte_power_freq_down (lcore_id); +if (sleep_time_ratio >=SCALING_DOWN_TIME_RATIO_THRESHOLD ) { +if (rte_power_freq_down) +rte_power_freq_down (lcore_id); + }else if ( (unsigned )(stats[lcore_id].nb_rx_processed / - stats[lcore_id].nb_iteration_looped) <MAX_PKT_BURST ) + stats[lcore_id].nb_iteration_looped) <MAX_PKT_BURST ) {/** * scale down a step if average packet per iteration less * than expectation. */ -rte_power_freq_down (lcore_id); +if (rte_power_freq_down) +rte_power_freq_down (lcore_id); + }/** * initialize another timer according to current frequency to ensure
@@ -707,22 +721,20 @@ l3fwd_simple_forward(struct rte_mbuf *m, uint8_t portid, } -#define SLEEP_GEAR1_THRESHOLD 100 -#define SLEEP_GEAR2_THRESHOLD 1000 +#define MINIMUM_SLEEP_TIME 1 +#define SUSPEND_THRESHOLD 300static inline uint32_t power_idle_heuristic (uint32_t zero_rx_packet_count) { -/* If zero count is less than 100, use it as the sleep time in us */ -if (zero_rx_packet_count <SLEEP_GEAR1_THRESHOLD ) -return zero_rx_packet_count; -/* If zero count is less than 1000, sleep time should be 100 us */ -else if ((zero_rx_packet_count >=SLEEP_GEAR1_THRESHOLD ) && - (zero_rx_packet_count <SLEEP_GEAR2_THRESHOLD )) -return SLEEP_GEAR1_THRESHOLD ; -/* If zero count is greater than 1000, sleep time should be 1000 us */ -else if (zero_rx_packet_count >=SLEEP_GEAR2_THRESHOLD ) -return SLEEP_GEAR2_THRESHOLD ; +/* If zero count is less than 100, sleep 1us */ +if (zero_rx_packet_count <SUSPEND_THRESHOLD ) +return MINIMUM_SLEEP_TIME ; +/* If zero count is less than 1000, sleep 100 us which is the +minimum latency switching from C3/C6 to C0 +*/ +else +return SUSPEND_THRESHOLD ;return 0 ; }
@@ -762,6 +774,42 @@ power_freq_scaleup_heuristic(unsigned lcore_id,return FREQ_CURRENT ; } +/** +* force polling thread sleep until one-shot rx interrupt triggers +* @param port_id +* Port id. +* @param queue_id +* Rx queue id. +* @return +* 0 on success +*/ +static int +sleep_until_rx_interrupt (uint8_t port_id,uint8_t queue_id) +{ +struct rte_eth_dev *eth_dev = &rte_eth_devices[port_id]; + +/* Enable one-shot rx interrupt */ +rte_spinlock_lock (&(locks[port_id])); +rte_eth_dev_rx_queue_intr_enable (port_id, queue_id); +rte_spinlock_unlock (&(locks[port_id])); + +RTE_LOG (INFO ,L3FWD_POWER , +"lcore %u sleeps until interrupt on port%d,rxq%d triggers\n" , +rte_lcore_id (), port_id, queue_id); + +rte_intr_wait_rx_pkt (&(eth_dev->pci_dev->intr_handle), queue_id); +RTE_LOG (INFO ,L3FWD_POWER , +"lcore %u is waked up from rx interrupt on port%d,rxq%d\n" , +rte_lcore_id (), port_id, queue_id); + +/* Disable one-shot rx interrupt */ +rte_spinlock_lock (&(locks[port_id])); +rte_eth_dev_rx_queue_intr_disable (port_id, queue_id); +rte_spinlock_unlock (&(locks[port_id])); + +return 0 ; +} +/* main processing loop */ static int main_loop (__attribute__((unused))void *dummy)
@@ -775,7 +823,6 @@ main_loop(__attribute__((unused)) void *dummy)struct lcore_conf *qconf;struct lcore_rx_queue *rx_queue;enum freq_scale_hint_t lcore_scaleup_hint; -uint32_t lcore_rx_idle_count =0 ;uint32_t lcore_idle_hint =0 ;
@@ -835,6 +882,8 @@ main_loop(__attribute__((unused)) void *dummy) prev_tsc_power = cur_tsc_power; } + +start_rx:/* * Read packet from RX queues */
@@ -848,6 +897,7 @@ main_loop(__attribute__((unused)) void *dummy) nb_rx =rte_eth_rx_burst (portid, queueid, pkts_burst,MAX_PKT_BURST ); + stats[lcore_id].nb_rx_processed += nb_rx;if (unlikely (nb_rx ==0 )) {/**
@@ -910,10 +960,13 @@ main_loop(__attribute__((unused)) void *dummy) rx_queue->freq_up_hint; } -if (lcore_scaleup_hint ==FREQ_HIGHEST ) -rte_power_freq_max (lcore_id); -else if (lcore_scaleup_hint ==FREQ_HIGHER ) -rte_power_freq_up (lcore_id); +if (lcore_scaleup_hint ==FREQ_HIGHEST ) { +if (rte_power_freq_max) +rte_power_freq_max (lcore_id); + }else if (lcore_scaleup_hint ==FREQ_HIGHER ) { +if (rte_power_freq_up) +rte_power_freq_up (lcore_id); + } }else {/** * All Rx queues empty in recent consecutive polls,
@@ -928,16 +981,21 @@ main_loop(__attribute__((unused)) void *dummy) lcore_idle_hint = rx_queue->idle_hint; } -if ( lcore_idle_hint <SLEEP_GEAR1_THRESHOLD ) +if (lcore_idle_hint <SUSPEND_THRESHOLD )/** -* execute "pause" instruction to avoid context -* switch for short sleep. -*/ +* execute "pause" instruction to avoid context +* switch which generally take hundres of +* microsecond for short sleep. +*/ rte_delay_us (lcore_idle_hint); -else -/* long sleep force runing thread to suspend */ -usleep (lcore_idle_hint); - +else { +/* suspend untill rx interrupt trigges */ +sleep_until_rx_interrupt ( + qconf->rx_queue_list[0 ].port_id, + qconf->rx_queue_list[0 ].queue_id); +/* start receiving packets immediately */ + goto start_rx; + } stats[lcore_id].sleep_time += lcore_idle_hint; } }
@@ -1270,7 +1328,7 @@ setup_hash(int socketid)char s[64 ];/* create ipv4 hash */ -snprintf (s,sizeof (s),"ipv4_l3fwd_hash_%d" , socketid); +rte_snprintf (s,sizeof (s),"ipv4_l3fwd_hash_%d" , socketid); ipv4_l3fwd_hash_params.name = s; ipv4_l3fwd_hash_params.socket_id = socketid; ipv4_l3fwd_lookup_struct[socketid] =
@@ -1280,7 +1338,7 @@ setup_hash(int socketid) "socket %d\n", socketid);/* create ipv6 hash */ -snprintf (s,sizeof (s),"ipv6_l3fwd_hash_%d" , socketid); +rte_snprintf (s,sizeof (s),"ipv6_l3fwd_hash_%d" , socketid); ipv6_l3fwd_hash_params.name = s; ipv6_l3fwd_hash_params.socket_id = socketid; ipv6_l3fwd_lookup_struct[socketid] =
@@ -1476,6 +1534,7 @@ main(int argc, char **argv)unsigned lcore_id;uint64_t hz;uint32_t n_tx_queue, nb_lcores; +uint32_t dev_rxq_num, dev_txq_num;uint8_t portid, nb_rx_queue, queue, socketid;/* catch SIGINT and restore cpufreq governor to ondemand */
@@ -1525,10 +1584,19 @@ main(int argc, char **argv)printf ("Initializing port %d ... " , portid );fflush (stdout); +rte_eth_dev_info_get (portid, &dev_info); + dev_rxq_num = dev_info.max_rx_queues; + dev_txq_num = dev_info.max_tx_queues; + nb_rx_queue =get_port_n_rx_queues (portid); +if (nb_rx_queue > dev_rxq_num) +rte_exit (EXIT_FAILURE , +"Cannot configure not existed rxq: " +"port=%d\n" , portid); + n_tx_queue = nb_lcores; -if (n_tx_queue >MAX_TX_QUEUE_PER_PORT ) - n_tx_queue =MAX_TX_QUEUE_PER_PORT ; +if (n_tx_queue > dev_txq_num) + n_tx_queue = dev_txq_num;printf ("Creating queues: nb_rxq=%d nb_txq=%u... " , nb_rx_queue, (unsigned )n_tx_queue ); ret = rte_eth_dev_configure(portid, nb_rx_queue,
@@ -1552,6 +1620,9 @@ main(int argc, char **argv)if (rte_lcore_is_enabled (lcore_id) ==0 )continue ; +if (queueid >= dev_txq_num) +continue ; +if (numa_on) socketid = \ (uint8_t )rte_lcore_to_socket_id (lcore_id);
@@ -1586,8 +1657,9 @@ main(int argc, char **argv)/* init power management library */ ret =rte_power_init (lcore_id);if (ret) -rte_exit (EXIT_FAILURE ,"Power management library " -"initialization failed on core%u\n" , lcore_id); +rte_log (RTE_LOG_ERR ,RTE_LOGTYPE_POWER , +"Power management library initialization " +"failed on core%u" , lcore_id);/* init timer structures for each enabled lcore */ rte_timer_init (&power_timers[lcore_id]);
@@ -1635,7 +1707,6 @@ main(int argc, char **argv)if (ret <0 )rte_exit (EXIT_FAILURE ,"rte_eth_dev_start: err=%d, " "port=%d\n" , ret, portid); -/* * If enabled, put device in promiscuous mode. * This allows IO forwarding mode to forward packets
@@ -1644,6 +1715,8 @@ main(int argc, char **argv)*/ if (promiscuous_on)rte_eth_promiscuous_enable (portid); +/* initialize spinlock for each port */ +rte_spinlock_init (&(locks[portid])); }check_all_ports_link_status ((uint8_t )nb_ports, enabled_port_mask);
Signature · 1 line
1.8.1.4
v6 cover Interrupt mode PMD
In reply to a message not in the archivev6 changes
- split rte_intr_wait_rx_pkt into two APIs 'wait' and 'set'.
- rewrite rte_intr_rx_wait/rte_intr_rx_set.
- using vector number instead of queue_id as interrupt API params.
- patch reorder and split.
v5 changes
- Rebase the patchset onto the HEAD
- Isolate ethdev from EAL for new-added wait-for-rx interrupt function
- Export wait-for-rx interrupt function for shared libraries
- Split-off a new patch file for changed struct rte_intr_handle that other patches depend on, to avoid breaking git bisect
- Change sample applicaiton to accomodate EAL function spec change accordingly
v4 changes
- Export interrupt enable/disable functions for shared libraries
- Adjust position of new-added structure fields and functions to avoid breaking ABI
v3 changes
- Add return value for interrupt enable/disable functions
- Move spinlok from PMD to L3fwd-power
- Remove unnecessary variables in e1000_mac_info
- Fix miscelleous review comments
v2 changes
- Fix compilation issue in Makefile for missed header file.
- Consolidate internal and community review comments of v1 patch set.
The patch series introduce low-latency one-shot rx interrupt into DPDK with polling and interrupt mode switch control example.
DPDK userspace interrupt notification and handling mechanism is based on UIO with below limitation:
1) It is designed to handle LSC interrupt only with inefficient suspended pthread wakeup procedure (e.g. UIO wakes up LSC interrupt handling thread which then wakes up DPDK polling thread). In this way, it introduces non-deterministic wakeup latency for DPDK polling thread as well as packet latency if it is used to handle Rx interrupt.
2) UIO only supports a single interrupt vector which has to been shared by LSC interrupt and interrupts assigned to dedicated rx queues.
This patchset includes below features:
1) Enable one-shot rx queue interrupt in ixgbe PMD(PF & VF) and igb PMD(PF only).
2) Build on top of the VFIO mechanism instead of UIO, so it could support up to 64 interrupt vectors for rx queue interrupts.
3) Have 1 DPDK polling thread handle per Rx queue interrupt with a dedicated VFIO eventfd, which eliminates non-deterministic pthread wakeup latency in user space.
4) Demonstrate interrupts control APIs and userspace NAIP-like polling/interrupt switch algorithms in L3fwd-power example.
Known limitations:
1) It does not work for UIO due to a single interrupt eventfd shared by LSC and rx queue interrupt handlers causes a mess.
2) LSC interrupt is not supported by VF driver, so it is by default disabled in L3fwd-power now. Feel free to turn in on if you want to support both LSC and rx queue interrupts on a PF.
Cunming Liang (5):
eal: declare new interrupt api
eal/linux: add rx queue interrupt FDs to intr handle struct eal/bsd: dummy for new intr definition
eal/linux: add per rx queue interrupt handling based on VFIO
ethdev: add rx interrupt enable/disable functions
Zhou, Danny (3):
ixgbe: enable rx queue interrupts for both PF and VF
igb: enable rx queue interrupts for PF
l3fwd-power: enable one-shot rx interrupt and polling/interrupt mode
switch
examples/l3fwd-power/main.c | 194 ++++++++--- lib/librte_eal/bsdapp/eal/eal_interrupts.c | 15 + .../bsdapp/eal/include/exec-env/rte_interrupts.h | 4 + lib/librte_eal/bsdapp/eal/rte_eal_version.map | 2 + lib/librte_eal/common/include/rte_interrupts.h | 38 +++ lib/librte_eal/linuxapp/eal/eal_interrupts.c | 224 +++++++++--- lib/librte_eal/linuxapp/eal/eal_pci_vfio.c | 23 +- .../linuxapp/eal/include/exec-env/rte_interrupts.h | 9 + lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + lib/librte_ether/rte_ethdev.c | 66 ++++ lib/librte_ether/rte_ethdev.h | 77 +++++ lib/librte_ether/rte_ether_version.map | 3 + lib/librte_pmd_e1000/e1000_ethdev.h | 3 + lib/librte_pmd_e1000/igb_ethdev.c | 231 +++++++++++-- lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 377 ++++++++++++++++++++- lib/librte_pmd_ixgbe/ixgbe_ethdev.h | 7 + 16 files changed, 1156 insertions(+), 119 deletions(-)
Signature · 1 line
1.8.1.4
A message that is not in the archive (sent off-list or not yet received): 1425012976-10173-1-git-send-email-cunming.liang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
PATCH v6 1/8 eal: declare new interrupt api
Add two API *rte_intr_rx_wait* and *rte_intr_rx_set* for RX interrupt. They're only available in VFIO_MSIX.
--- lib/librte_eal/common/include/rte_interrupts.h | 38 ++++++++++++++++++++++++++ 1 file changed, 38 insertions(+)
lib/librte_eal/common/include/rte_interrupts.h +38 -0
@@ -45,6 +45,8 @@extern "C" {#endif +#include <stdint.h> +/** Interrupt handle */ struct rte_intr_handle ;
@@ -54,6 +56,8 @@ typedef void (*rte_intr_callback_fn)(struct rte_intr_handle *intr_handle,#include <exec-env/rte_interrupts.h> +#define RTE_EPOLL_FD_ANY -1/**< to hint using per thread epfd */ +/** * It registers the callback for the specific interrupt. Multiple * callbacks cal be registered at the same time.
@@ -113,6 +117,40 @@ int rte_intr_enable(struct rte_intr_handle *intr_handle);*/ int rte_intr_disable (struct rte_intr_handle *intr_handle); +/** +* @param intr_handle +* pointer to the interrupt handle. +* @param fd +* event fd the thread wait for. +* @param vec +* array for all the ready vectors. +* @param num +* the max number of vector array. +* @return +* - On success, returns the number of available vectors. +* - On failure, a negative value. +*/ +int +rte_intr_rx_wait (struct rte_intr_handle *intr_handle, +int fd,uint32_t *vec,uint16_t num); + +/** +* @param intr_handle +* pointer to the interrupt handle. +* @param fd +* event fd which the intr vector associated to. +* @param op +* operation type of {ADD, DEL}. +* @param vec +* vector number added to the event wait list. +* @return +* - On success, zero. +* - On failure, a negative value. +*/ +int +rte_intr_rx_set (struct rte_intr_handle *intr_handle, +int fd,int op,uint32_t vec); + #ifdef __cplusplus }#endif
Signature · 1 line
1.8.1.4
PATCH v6 2/8 eal/linux: add rx queue interrupt FDs to intr handle struct
In reply to a message not in the archivePer vector event fd will store in rte_intr_handle during init. Device drivers take responsibility to fill queue-vec mapping table(vec_num[]).
---
v6 changes:
- add mapping table between irq vector number and queue id.
v5 changes:
- Create this new patch file for changed struct rte_intr_handle that other patches depend on, to avoid breaking git bisect.
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h | 9 +++++++++ 1 file changed, 9 insertions(+)
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +9 -0
@@ -38,6 +38,9 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define VFIO_MAX_RXTX_INTR_ID 32 +#define VFIO_MAX_QUEUE_ID VFIO_MAX_RXTX_INTR_ID +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */
@@ -48,6 +51,9 @@ enum rte_intr_handle_type {RTE_INTR_HANDLE_MAX }; +#define RTE_INTR_EVENT_ADD 1UL +#define RTE_INTR_EVENT_DEL 2UL +/** Handle for interrupts. */ struct rte_intr_handle { union {
@@ -57,6 +63,9 @@ struct rte_intr_handle { };int fd;/**< interrupt event file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +int max_intr;/**< max interrupt requested */ +int efds[VFIO_MAX_RXTX_INTR_ID ];/**< rxtx intr event fd for vfio */ +uint32_t vec_num[VFIO_MAX_QUEUE_ID ];/**< rxtx intr vector number */ };#endif /* _RTE_LINUXAPP_INTERRUPTS_H_ */
Signature · 1 line
1.8.1.4
A message that is not in the archive (sent off-list or not yet received): 1425012976-10173-3-git-send-email-cunming.liang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH v6 2/8] eal/linux: add rx queue interrupt FDs to intr handle struct
Hello,
Quoted 18 lines · Cunming Liang
Per vector event fd will store in
rte_intr_handleduring init. Device drivers take responsibility to fill queue-vec mappingtable(vec_num[]).Signed-off-by Danny Zhou <[email protected]>Signed-off-by Cunming Liang <[email protected]>---
v6 changes:
- add mapping table between irq vector number and queue id.lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +0
-0
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h line 44
Quoted 7 lines
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +3
-0@@ -38,6 +38,9 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define VFIO_MAX_RXTX_INTR_ID 32 +#define VFIO_MAX_QUEUE_ID VFIO_MAX_RXTX_INTR_ID +
This is a little weird to talk about vfio here.
This file is "generic".
Ok, you will store vfio eventfds here, but vfio is an implementation, not the abstraction.
Signature · 1 line
David Marchand
A message that is not in the archive (sent off-list or not yet received): CALwxeUtr8iUCjroCf0ix0PPU2ers3EgTeTT6XUDAgwmwsOBS6Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Re: [PATCH v6 2/8] eal/linux: add rx queue interrupt FDs to intr handle struct
From: David Marchand [mailto:[email protected]]
Sent: Friday, February 27, 2015 6:33 PM
To: Liang, Cunming
Cc: [email protected]; Stephen Hemminger; Thomas Monjalon; Zhou, Danny
Subject: Re: [PATCH v6 2/8] eal/linux: add rx queue interrupt FDs to intr handle struct
Hello,
On Fri, Feb 27, 2015 at 5:56 AM, Cunming Liang <[email protected]<mailto:[email protected]>> wrote:
Per vector event fd will store in rte_intr_handle during init.
Device drivers take responsibility to fill queue-vec mapping table(vec_num[]).
---
v6 changes:
- add mapping table between irq vector number and queue id.
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +3 -0
@@ -38,6 +38,9 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define VFIO_MAX_RXTX_INTR_ID 32 +#define VFIO_MAX_QUEUE_ID VFIO_MAX_RXTX_INTR_ID +
This is a little weird to talk about vfio here.
This file is "generic".
Ok, you will store vfio eventfds here, but vfio is an implementation, not the abstraction.
[Liang, Cunming] If looking at the rte_intr_hanle_type, it includes UIO/VFIO_LEGACY/VFIO_MSI/VFIO_MSIX. I agree, VFIO is an implementation, but the different type combination is a kind of ‘abstraction’. So in rte_intr_handle (like a multiplexing), some specified field for vfio interrupter mapping, I feel it’s reasonable.
--
David Marchand
A message that is not in the archive (sent off-list or not yet received): D0158A423229094DA7ABF71CF2FA0DA3118DFBF1-E2R4CRU6q/[email protected]
Re: [PATCH v6 2/8] eal/linux: add rx queue interrupt FDs to intr handle struct
2015-02-27 11:28, Liang, Cunming:
Quoted 24 lines
From: David Marchand [mailto:[email protected]]
Sent: Friday, February 27, 2015 6:33 PMlib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h line 43
Quoted 9 lines · On Fri, Feb 27, 2015 at 5:56 AM, Cunming Liang…
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +2
-0@@ -38,6 +38,9 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define VFIO_MAX_RXTX_INTR_ID 32 +#define VFIO_MAX_QUEUE_ID VFIO_MAX_RXTX_INTR_ID>This is a little weird to talk about vfio here.
This file is "generic".Ok, you will store vfio eventfds here, but vfio is an implementation, not the abstraction.
[Liang, Cunming] If looking at therte_intr_hanle_type, it includes UIO/VFIO_LEGACY/VFIO_MSI/VFIO_MSIX. I agree, VFIO is an implementation, but the different type combination is a kind of ‘abstraction’. So inrte_intr_handle(like a multiplexing), some specified field for vfio interrupter mapping, I feel it’s reasonable.--
David Marchand
Re: [PATCH v6 2/8] eal/linux: add rx queue interrupt FDs to intr handle struct
In reply to a message not in the archive2015-02-27 11:28, Liang, Cunming:
Quoted 16 lines
From: David Marchand [mailto:[email protected]]
Sent: Friday, February 27, 2015 6:33 PMQuoted 14 lines · Cunming Liang
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h line 43
Quoted 9 lines
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +2
-0@@ -38,6 +38,9 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define VFIO_MAX_RXTX_INTR_ID 32 +#define VFIO_MAX_QUEUE_ID VFIO_MAX_RXTX_INTR_IDThis is a little weird to talk about vfio here.
This file is "generic".
Quoted 6 lines
Quoted 2 lines
Ok, you will store vfio eventfds here, but vfio is an implementation, not the abstraction.
[Liang, Cunming] If looking at the
rte_intr_hanle_type, it includes UIO/VFIO_LEGACY/VFIO_MSI/VFIO_MSIX. I agree, VFIO is an implementation, but the different type combination is a kind of ‘abstraction’. So inrte_intr_handle(like a multiplexing), some specified field for vfio interrupter mapping, I feel it’s reasonable.
Not sure to understand. Are we trying to mask the different kernel drivers from an application point of view, and provide a generic interrupt mechanism? If yes, why some VFIO constants are needed?
I'm not saying that the current implementation is perfect, but we should try to improve it.
Thanks
Re: [PATCH v6 2/8] eal/linux: add rx queue interrupt FDs to intr handle struct
Quoted 34 lines
-----Original Message-----
From: Thomas Monjalon [mailto:[email protected]]
Sent: Friday, February 27, 2015 10:52 PM
To: Liang, Cunming
Cc: David Marchand; [email protected]; Stephen Hemminger; Zhou, Danny
Subject: Re: [PATCH v6 2/8] eal/linux: add rx queue interrupt FDs to intr handle struct2015-02-27 11:28, Liang, Cunming:
Quoted 16 lines
From: David Marchand [mailto:[email protected]]
Sent: Friday, February 27, 2015 6:33 PMQuoted 14 lines · Cunming Liang
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h line 43
Quoted 9 lines
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +2
-0@@ -38,6 +38,9 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define VFIO_MAX_RXTX_INTR_ID 32 +#define VFIO_MAX_QUEUE_ID VFIO_MAX_RXTX_INTR_IDThis is a little weird to talk about vfio here.
This file is "generic".Quoted 4 lines
Quoted 2 lines
Ok, you will store vfio eventfds here, but vfio is an implementation, not the abstraction.
[Liang, Cunming] If looking at the
rte_intr_hanle_type, it includesUIO/VFIO_LEGACY/VFIO_MSI/VFIO_MSIX.
Quoted 1 line
I agree, VFIO is an implementation, but the different type combination is a kind
of ‘abstraction’.
Quoted 8 lines
Quoted 1 line
So in
rte_intr_handle(like a multiplexing), some specified field for vfiointerrupter mapping, I feel it’s reasonable.
Not sure to understand. Are we trying to mask the different kernel drivers from an application point of view, and provide a generic interrupt mechanism? If yes, why some VFIO constants are needed?
I'm not saying that the current implementation is perfect, but we should try to improve it.
[LCM] VFIO_MAX_RXTX_INTR_ID is easy to fix, it can move to a private interrupt header file, as only be used inside EAL. VFIO_MAX_QUEUE_ID can be removed, so vec_num[] dynamic creation by the device driver. Sounds good ?
Quoted 2 lines
Thanks
PATCH v6 3/8 eal/bsd: dummy for new intr definition
In reply to a message not in the archiveTo make bsd compiling happy with new intr changes.
--- lib/librte_eal/bsdapp/eal/eal_interrupts.c | 15 +++++++++++++++ .../bsdapp/eal/include/exec-env/rte_interrupts.h | 4 ++++ lib/librte_eal/bsdapp/eal/rte_eal_version.map | 2 ++ 3 files changed, 21 insertions(+)
lib/librte_eal/bsdapp/eal/eal_interrupts.c +15 -0
@@ -69,3 +69,18 @@ rte_eal_intr_init(void)return 0 ; } +int +rte_intr_rx_wait(struct rte_intr_handle *intr_handle __rte_unused, + int fd __rte_unused, uint32_t *vec __rte_unused, + uint16_t num __rte_unused) +{ +return -ENOTSUP ; +} + +int +rte_intr_rx_set(struct rte_intr_handle *intr_handle __rte_unused, + int fd __rte_unused, int op __rte_unused, + uint32_t vec __rte_unused) +{ +return -ENOTSUP ; +}
lib/librte_eal/bsdapp/eal/include/exec-env/rte_interrupts.h +4 -0
@@ -38,6 +38,8 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define VFIO_MAX_RXTX_INTR_ID 32 +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */
@@ -49,6 +51,8 @@ enum rte_intr_handle_type {struct rte_intr_handle {int fd;/**< file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +int max_intr;/**< max interrupt requested */ +uint32_t vec_num[VFIO_MAX_QUEUE_ID ];/**< rxtx intr vector number */ };#endif /* _RTE_LINUXAPP_INTERRUPTS_H_ */
lib/librte_eal/bsdapp/eal/rte_eal_version.map +2 -0
@@ -58,6 +58,8 @@ DPDK_2.0 { rte_intr_callback_unregister; rte_intr_disable; rte_intr_enable; + rte_intr_rx_set; + rte_intr_rx_wait; rte_log; rte_log_add_in_history; rte_log_cur_msg_loglevel;
Signature · 1 line
1.8.1.4
A message that is not in the archive (sent off-list or not yet received): 1425012976-10173-4-git-send-email-cunming.liang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH v6 3/8] eal/bsd: dummy for new intr definition
Hello,
Quoted 16 lines · Cunming Liang
lib/librte_eal/bsdapp/eal/include/exec-env/rte_interrupts.h +2
-0@@ -38,6 +38,8 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define VFIO_MAX_RXTX_INTR_ID 32 +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */
lib/librte_eal/bsdapp/eal/include/exec-env/rte_interrupts.h enum rte_intr_handle_type { · line 58
Quoted 8 lines
lib/librte_eal/bsdapp/eal/include/exec-env/rte_interrupts.h +2
-0@@ -49,6 +51,8 @@ enum rte_intr_handle_type {struct rte_intr_handle {int fd;/**< file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +int max_intr;/**< max interrupt requested */ +uint32_t vec_num[VFIO_MAX_QUEUE_ID ];/**< rxtx intr vector number */ };
No need to add those since this is not supported for bsd.
Signature · 1 line
David Marchand
A message that is not in the archive (sent off-list or not yet received): CALwxeUudTApYsd3qsAYeqKmAKK6ztbWQJ-oe5M817jqNXaswdA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Re: [PATCH v6 3/8] eal/bsd: dummy for new intr definition
From: David Marchand [mailto:[email protected]]
Sent: Friday, February 27, 2015 6:00 PM
To: Liang, Cunming
Cc: [email protected]; Stephen Hemminger; Thomas Monjalon
Subject: Re: [PATCH v6 3/8] eal/bsd: dummy for new intr definition
Hello,
On Fri, Feb 27, 2015 at 5:56 AM, Cunming Liang <[email protected]<mailto:[email protected]>> wrote:
lib/librte_eal/bsdapp/eal/include/exec-env/rte_interrupts.h +4 -0
@@ -38,6 +38,8 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define VFIO_MAX_RXTX_INTR_ID 32 +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */
@@ -49,6 +51,8 @@ enum rte_intr_handle_type {struct rte_intr_handle {int fd;/**< file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +int max_intr;/**< max interrupt requested */ +uint32_t vec_num[VFIO_MAX_QUEUE_ID ];/**< rxtx intr vector number */ };
No need to add those since this is not supported for bsd.
[Liang, Cunming] max_intr is used in dev_init for pci_dev->intr_handle init.Vec_num is used in ethdev API rx_intr_vec_get. Without it, BSD macro will used for each of the reference place. As they’re quite generic, even bsd will require either max_intr or vec mapping table.
--
David Marchand
A message that is not in the archive (sent off-list or not yet received): D0158A423229094DA7ABF71CF2FA0DA3118DFBCE-E2R4CRU6q/[email protected]
Re: [PATCH v6 3/8] eal/bsd: dummy for new intr definition
2015-02-27 11:21, Liang, Cunming:
Quoted 15 lines
From: David Marchand [mailto:[email protected]]
Quoted 5 lines · Cunming Liang
Quoted 4 lines
lib/librte_eal/bsdapp/eal/include/exec-env/rte_interrupts.h +0
-0@@ -49,6 +51,8 @@ enum rte_intr_handle_type {struct rte_intr_handle {Quoted 8 lines
Quoted 6 lines
lib/librte_eal/bsdapp/eal/include/exec-env/rte_interrupts.h +2
-0int fd;/**< file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +int max_intr;/**< max interrupt requested */ +uint32_t vec_num[VFIO_MAX_QUEUE_ID ];/**< rxtx intr vector number */ };
No need to add those since this is not supported for bsd.
Quoted 5 lines
[Liang, Cunming]
max_intris used indev_initforpci_dev->intr_handleinit.Vec_numis used in ethdev APIrx_intr_vec_get. Without it, BSD macro will used for each of the reference place.
As they’re quite generic, even bsd will require eithermax_intror vec mapping table.
Is it needed to build and run DPDK on FreeBSD?
Re: [PATCH v6 3/8] eal/bsd: dummy for new intr definition
Quoted 26 lines
-----Original Message-----
From: Thomas Monjalon [mailto:[email protected]]
Sent: Friday, February 27, 2015 10:22 PM
To: Liang, Cunming
Cc: David Marchand; [email protected]; Stephen Hemminger
Subject: Re: [PATCH v6 3/8] eal/bsd: dummy for new intr definition2015-02-27 11:21, Liang, Cunming:
Quoted 11 lines
From: David Marchand [mailto:[email protected]]
Quoted 10 lines · Cunming Liang
lib/librte_eal/bsdapp/eal/include/exec-env/rte_interrupts.h enum rte_intr_handle_type { · line 58
Quoted 9 lines
lib/librte_eal/bsdapp/eal/include/exec-env/rte_interrupts.h +2
-0@@ -49,6 +51,8 @@ enum rte_intr_handle_type {struct rte_intr_handle {int fd;/**< file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +int max_intr;/**< max interrupt requested */ +uint32_t vec_num[VFIO_MAX_QUEUE_ID ];/**< rxtx intr vector number */
Quoted 4 lines
Quoted 3 lines
Quoted 1 line
};
No need to add those since this is not supported for bsd.
Quoted 7 lines
Quoted 5 lines
[Liang, Cunming]
max_intris used indev_initforpci_dev->intr_handleinit.Vec_numis used in ethdev APIrx_intr_vec_get. Without it, BSD macro will used for each of the reference place.
As they’re quite generic, even bsd will require eithermax_intror vec mapping table.Is it needed to build and run DPDK on FreeBSD?
[LCM] As it's the EAL change, so I try to make sure FreeBSD can build and run as normal.
PATCH v6 4/8 eal/linux: add per rx queue interrupt handling based on VFIO
In reply to a message not in the archiveThis patch does below:
- Create multiple VFIO eventfd for rx queues.
- Handle per rx queue interrupt.
- Eliminate unnecessary suspended DPDK polling thread wakeup mechanism for rx interrupt by allowing polling thread epoll_wait rx queue interrupt notification.
---
v6 changes
- split rte_intr_wait_rx_pkt into two function, wait and set.
- rewrite rte_intr_rx_wait/rte_intr_rx_set to remove queue visibility on eal.
- rte_intr_rx_wait to support multiplexing.
- allow epfd as input to support flexible event fd combination.
v5 changes
- Rebase the patchset onto the HEAD
- Isolate ethdev from EAL for new-added wait-for-rx interrupt function
- Export wait-for-rx interrupt function for shared libraries
v4 changes:
- Adjust position of new-added structure fields
v3 changes:
- Fix review comments
v2 changes:
- Fix compilation issue for a missed header file
- Bug fix: free unreleased resources on the exception path before return
- Consolidate coding style related review comments
lib/librte_eal/linuxapp/eal/eal_interrupts.c | 224 +++++++++++++++++++----- lib/librte_eal/linuxapp/eal/eal_pci_vfio.c | 23 ++- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + 3 files changed, 201 insertions(+), 48 deletions(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +181 -43
@@ -44,6 +44,7 @@#include <sys/epoll.h> #include <sys/signalfd.h> #include <sys/ioctl.h> +#include <assert.h> #include <rte_common.h> #include <rte_interrupts.h>
@@ -70,6 +71,8 @@#define EAL_INTR_EPOLL_WAIT_FOREVER (-1) +static RTE_DEFINE_PER_LCORE(int, _epfd) = -1; /**< epoll fd per thread */ +/** * union for pipe fds. */
@@ -127,6 +130,9 @@ static pthread_t intr_thread;#ifdef VFIO_PRESENT #define IRQ_SET_BUF_LEN (sizeof(struct vfio_irq_set) + sizeof(int)) +/* irq set buffer length for queue interrupts and LSC interrupt */ +#define MSIX_IRQ_SET_BUF_LEN (sizeof(struct vfio_irq_set) + \ + sizeof(int) * (VFIO_MAX_RXTX_INTR_ID + 1))/* enable legacy (INTx) interrupts */ static int
@@ -218,10 +224,10 @@ vfio_disable_intx(struct rte_intr_handle *intr_handle) {return 0 ; } -/* enable MSI-X interrupts */ +/* enable MSI interrupts */ static int vfio_enable_msi (struct rte_intr_handle *intr_handle) { -int len, ret; +int len, ret, max_intr;char irq_set_buf[IRQ_SET_BUF_LEN ];struct vfio_irq_set *irq_set;int *fd_ptr;
@@ -230,12 +236,19 @@ vfio_enable_msi(struct rte_intr_handle *intr_handle) { irq_set = (struct vfio_irq_set *) irq_set_buf; irq_set->argsz = len; - irq_set->count =1 ; +if ((!intr_handle->max_intr) || + (intr_handle->max_intr >VFIO_MAX_RXTX_INTR_ID )) + max_intr =VFIO_MAX_RXTX_INTR_ID +1 ; +else + max_intr = intr_handle->max_intr; + + irq_set->count = max_intr; irq_set->flags =VFIO_IRQ_SET_DATA_EVENTFD |VFIO_IRQ_SET_ACTION_TRIGGER ; irq_set->index =VFIO_PCI_MSI_IRQ_INDEX ; irq_set->start =0 ; fd_ptr = (int *) &irq_set->data; - *fd_ptr = intr_handle->fd; +memcpy (fd_ptr, intr_handle->efds,sizeof (intr_handle->efds)); + fd_ptr[max_intr -1 ] = intr_handle->fd; ret =ioctl (intr_handle->vfio_dev_fd,VFIO_DEVICE_SET_IRQS , irq_set);
@@ -244,27 +257,10 @@ vfio_enable_msi(struct rte_intr_handle *intr_handle) { intr_handle->fd);return -1 ; } - -/* manually trigger interrupt to enable it */ -memset (irq_set,0 , len); - len =sizeof (struct vfio_irq_set ); - irq_set->argsz = len; - irq_set->count =1 ; - irq_set->flags =VFIO_IRQ_SET_DATA_NONE |VFIO_IRQ_SET_ACTION_TRIGGER ; - irq_set->index =VFIO_PCI_MSI_IRQ_INDEX ; - irq_set->start =0 ; - - ret =ioctl (intr_handle->vfio_dev_fd,VFIO_DEVICE_SET_IRQS , irq_set); - -if (ret) { -RTE_LOG (ERR ,EAL ,"Error triggering MSI interrupts for fd %d\n" , - intr_handle->fd); -return -1 ; - }return 0 ; } -/* disable MSI-X interrupts */ +/* disable MSI interrupts */ static int vfio_disable_msi (struct rte_intr_handle *intr_handle) {struct vfio_irq_set *irq_set;
@@ -292,8 +288,8 @@ vfio_disable_msi(struct rte_intr_handle *intr_handle) {/* enable MSI-X interrupts */ static int vfio_enable_msix (struct rte_intr_handle *intr_handle) { -int len, ret; -char irq_set_buf[IRQ_SET_BUF_LEN ]; +int len, ret, max_intr; +char irq_set_buf[MSIX_IRQ_SET_BUF_LEN ];struct vfio_irq_set *irq_set;int *fd_ptr;
@@ -301,12 +297,19 @@ vfio_enable_msix(struct rte_intr_handle *intr_handle) { irq_set = (struct vfio_irq_set *) irq_set_buf; irq_set->argsz = len; - irq_set->count =1 ; +if ((!intr_handle->max_intr) || + (intr_handle->max_intr >VFIO_MAX_RXTX_INTR_ID )) + max_intr =VFIO_MAX_RXTX_INTR_ID +1 ; +else + max_intr = intr_handle->max_intr; + + irq_set->count = max_intr; irq_set->flags =VFIO_IRQ_SET_DATA_EVENTFD |VFIO_IRQ_SET_ACTION_TRIGGER ; irq_set->index =VFIO_PCI_MSIX_IRQ_INDEX ; irq_set->start =0 ; fd_ptr = (int *) &irq_set->data; - *fd_ptr = intr_handle->fd; +memcpy (fd_ptr, intr_handle->efds,sizeof (intr_handle->efds)); + fd_ptr[max_intr -1 ] = intr_handle->fd; ret =ioctl (intr_handle->vfio_dev_fd,VFIO_DEVICE_SET_IRQS , irq_set);
@@ -316,22 +319,6 @@ vfio_enable_msix(struct rte_intr_handle *intr_handle) {return -1 ; } -/* manually trigger interrupt to enable it */ -memset (irq_set,0 , len); - len =sizeof (struct vfio_irq_set ); - irq_set->argsz = len; - irq_set->count =1 ; - irq_set->flags =VFIO_IRQ_SET_DATA_NONE |VFIO_IRQ_SET_ACTION_TRIGGER ; - irq_set->index =VFIO_PCI_MSIX_IRQ_INDEX ; - irq_set->start =0 ; - - ret =ioctl (intr_handle->vfio_dev_fd,VFIO_DEVICE_SET_IRQS , irq_set); - -if (ret) { -RTE_LOG (ERR ,EAL ,"Error triggering MSI-X interrupts for fd %d\n" , - intr_handle->fd); -return -1 ; - }return 0 ; }
@@ -339,7 +326,7 @@ vfio_enable_msix(struct rte_intr_handle *intr_handle) {static int vfio_disable_msix (struct rte_intr_handle *intr_handle) {struct vfio_irq_set *irq_set; -char irq_set_buf[IRQ_SET_BUF_LEN ]; +char irq_set_buf[MSIX_IRQ_SET_BUF_LEN ];int len, ret; len =sizeof (struct vfio_irq_set );
@@ -860,3 +847,154 @@ rte_eal_intr_init(void)return -ret; } +static void +eal_intr_process_rxtx_interrupts (struct rte_intr_handle *intr_handle, +struct epoll_event *events, +uint32_t *vec,int nfds) +{ +int i, bytes_read; +union rte_intr_read_buffer buf; +int fd; + +for (i =0 ; i < nfds; i++) { +/* set the length to be read for different handle type */ +switch (intr_handle->type) { +case RTE_INTR_HANDLE_UIO : + bytes_read =sizeof (buf.uio_intr_count); +break ; +case RTE_INTR_HANDLE_ALARM : + bytes_read =sizeof (buf.timerfd_num); +break ; +#ifdef VFIO_PRESENT +case RTE_INTR_HANDLE_VFIO_MSIX : +case RTE_INTR_HANDLE_VFIO_MSI : +case RTE_INTR_HANDLE_VFIO_LEGACY : + bytes_read =sizeof (buf.vfio_intr_count); +break ; +#endif +default : + bytes_read =1 ; +break ; + } + +/** +* read out to clear the ready-to-be-read flag +* for epoll_wait. +*/ + vec[i] = events[i].data.u32; +assert (vec[i] <VFIO_MAX_RXTX_INTR_ID ); + + fd = intr_handle->efds[vec[i]]; + bytes_read =read (fd, &buf, bytes_read); +if (bytes_read <0 ) +RTE_LOG (ERR ,EAL ,"Error reading from file " +"descriptor %d: %s\n" , fd,strerror (errno)); +else if (bytes_read ==0 ) +RTE_LOG (ERR ,EAL ,"Read nothing from file " +"descriptor %d\n" , fd); + } +} + +static int init_tls_epfd (void ) +{ +int pfd =epoll_create (1 ); +if (pfd <0 ) { +RTE_LOG (ERR ,EAL , +"Cannot create epoll instance\n" ); +return -1 ; + } +return pfd; +} + +int +rte_intr_rx_wait (struct rte_intr_handle *intr_handle,int epfd, +uint32_t *vec,uint16_t num) +{ +#define MAX_EVENTS 8 +struct epoll_event events[MAX_EVENTS ]; +int ret, nfds =0 ; + +if (!intr_handle || !vec) { +RTE_LOG (ERR ,EAL ,"invalid input parameter\n" ); +return -1 ; + } + +if (intr_handle->type !=RTE_INTR_HANDLE_VFIO_MSIX ) { +RTE_LOG (ERR ,EAL ,"intr type should be VFIO_MSIX\n" ); +return -1 ; + } + +if (epfd ==RTE_EPOLL_FD_ANY ) { +/* using per thread epoll fd */ +if (unlikely (RTE_PER_LCORE (_epfd) ==-1 )) +RTE_PER_LCORE (_epfd) =init_tls_epfd (); + epfd =RTE_PER_LCORE (_epfd); + } + +do { + ret =epoll_wait (epfd, events, +RTE_MIN (num,MAX_EVENTS ), +EAL_INTR_EPOLL_WAIT_FOREVER ); +if (unlikely (ret <0 )) { +/* epoll_wait fail */ +RTE_LOG (ERR ,EAL ,"epoll_wait returns with fail\n" ); +return -1 ; + }else if (ret >0 ) { +/* epoll_wait has at least one fd ready to read */ +eal_intr_process_rxtx_interrupts (intr_handle, events, + vec, ret); + num -= ret; + vec += ret; + nfds += ret; + }else if (nfds >0 ) +break ; + }while (num >0 ); + +return nfds; +} + +int +rte_intr_rx_set (struct rte_intr_handle *intr_handle,int epfd, +int op,uint32_t vec) +{ +struct epoll_event ev; + +if (!intr_handle || vec >=VFIO_MAX_RXTX_INTR_ID ) { +RTE_LOG (ERR ,EAL ,"invalid input parameter\n" ); +return -1 ; + } + +if (intr_handle->type !=RTE_INTR_HANDLE_VFIO_MSIX ) { +RTE_LOG (ERR ,EAL ,"intr type should be VFIO_MSIX\n" ); +return -1 ; + } + +switch (op) { +case RTE_INTR_EVENT_ADD : + op =EPOLL_CTL_ADD ; +break ; +case RTE_INTR_EVENT_DEL : + op =EPOLL_CTL_DEL ; +break ; +default : +RTE_LOG (ERR ,EAL ,"event op type mismatch\n" ); +return -1 ; + } + +if (epfd ==RTE_EPOLL_FD_ANY ) { +/* using per thread epoll fd */ +if (RTE_PER_LCORE (_epfd) ==-1 ) +RTE_PER_LCORE (_epfd) =init_tls_epfd (); + epfd =RTE_PER_LCORE (_epfd); + } + + ev.data.u32 = vec; + ev.events =EPOLLIN |EPOLLPRI ; +if (epoll_ctl (epfd, op, intr_handle->efds[vec], &ev) <0 ) { +RTE_LOG (ERR ,EAL ,"Error op %d fd %d epoll_ctl, %s\n" , + op, intr_handle->efds[vec],strerror (errno)); +return -1 ; + } + +return 0 ; +}
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +18 -5
@@ -38,6 +38,7 @@#include <sys/socket.h> #include <sys/ioctl.h> #include <sys/mman.h> +#include <sys/epoll.h> #include <rte_log.h> #include <rte_pci.h>
@@ -274,16 +275,18 @@ pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd) ret =ioctl (vfio_dev_fd,VFIO_DEVICE_GET_IRQ_INFO , &irq);if (ret <0 ) {RTE_LOG (ERR ,EAL ," cannot get IRQ info, " -"error %i (%s)\n" , errno,strerror (errno)); +"error %i (%s)\n" , errno,strerror (errno));return -1 ; }/* if this vector cannot be used with eventfd, fail if we explicitly * specified interrupt type, otherwise continue */ if ((irq.flags &VFIO_IRQ_INFO_EVENTFD ) ==0 ) { -if (internal_config.vfio_intr_mode !=RTE_INTR_MODE_NONE ) { +if (internal_config.vfio_intr_mode != +RTE_INTR_MODE_NONE ) {RTE_LOG (ERR ,EAL , -" interrupt vector does not support eventfd!\n" ); +" interrupt vector " +"does not support eventfd!\n" );return -1 ; }else continue ;
@@ -293,17 +296,27 @@ pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd) fd =eventfd (0 ,0 );if (fd <0 ) {RTE_LOG (ERR ,EAL ," cannot set up eventfd, " -"error %i (%s)\n" , errno,strerror (errno)); +"error %i (%s)\n" , errno,strerror (errno));return -1 ; } dev->intr_handle.fd = fd; dev->intr_handle.vfio_dev_fd = vfio_dev_fd; -switch (i) {case VFIO_PCI_MSIX_IRQ_INDEX : internal_config.vfio_intr_mode =RTE_INTR_MODE_MSIX ; dev->intr_handle.type =RTE_INTR_HANDLE_VFIO_MSIX ; +for (i =0 ; i <VFIO_MAX_RXTX_INTR_ID ; i++) { + fd =eventfd (0 ,0 ); +if (fd <0 ) { +RTE_LOG (ERR ,EAL , +"cannot setup eventfd," +"error %i (%s)\n" , + errno,strerror (errno)); +return -1 ; + } + dev->intr_handle.efds[i] = fd; + }break ;case VFIO_PCI_MSI_IRQ_INDEX : internal_config.vfio_intr_mode =RTE_INTR_MODE_MSI ;
lib/librte_eal/linuxapp/eal/rte_eal_version.map +2 -0
@@ -64,6 +64,8 @@ DPDK_2.0 { rte_intr_callback_unregister; rte_intr_disable; rte_intr_enable; + rte_intr_rx_set; + rte_intr_rx_wait; rte_log; rte_log_add_in_history; rte_log_cur_msg_loglevel;
Signature · 1 line
1.8.1.4
A message that is not in the archive (sent off-list or not yet received): 1425012976-10173-5-git-send-email-cunming.liang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH v6 4/8] eal/linux: add per rx queue interrupt handling based on VFIO
I am not really comfortable with this api.
This is just creating something on top of the standard epoll api with limitations.
In the end, we could just use an external lib that does this already.
So ok, this will work for your limited use case, but this will not be really useful for anything else.
Not sure it has its place in eal, this is more an example to me.
Quoted 31 lines · Cunming Liang
This patch does below:
- Create multiple VFIO eventfd for rx queues.
- Handle per rx queue interrupt.
- Eliminate unnecessary suspended DPDK polling thread wakeup mechanism for rx interrupt by allowing polling threadepoll_waitrx queue interrupt notification.Signed-off-by Danny Zhou <[email protected]>Signed-off-by Cunming Liang <[email protected]>---
v6 changes
- splitrte_intr_wait_rx_pktinto two function, wait and set.
- rewrite rte_intr_rx_wait/rte_intr_rx_set to remove queue visibility on
eal.
-rte_intr_rx_waitto support multiplexing.
- allow epfd as input to support flexible event fd combination.lib/librte_eal/linuxapp/eal/eal_interrupts.c | 224 +++++++++++++++++++----- lib/librte_eal/linuxapp/eal/eal_pci_vfio.c | 23 ++- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + 3 files changed, 201 insertions(+), 48 deletions(-)lib/librte_eal/linuxapp/eal/eal_interrupts.c +0
-0
[snip]
Quoted 45 lines
lib/librte_eal/linuxapp/eal/eal_interrupts.c +42
-0+static void +eal_intr_process_rxtx_interrupts (struct rte_intr_handle *intr_handle, +struct epoll_event *events, +uint32_t *vec,int nfds) +{ +int i, bytes_read; +union rte_intr_read_buffer buf; +int fd; + +for (i =0 ; i < nfds; i++) { +/* set the length to be read for different handle type */ +switch (intr_handle->type) { +case RTE_INTR_HANDLE_UIO : + bytes_read =sizeof (buf.uio_intr_count); +break ; +case RTE_INTR_HANDLE_ALARM : + bytes_read =sizeof (buf.timerfd_num); +break ; +#ifdef VFIO_PRESENT +case RTE_INTR_HANDLE_VFIO_MSIX : +case RTE_INTR_HANDLE_VFIO_MSI : +case RTE_INTR_HANDLE_VFIO_LEGACY : + bytes_read =sizeof (buf.vfio_intr_count); +break ; +#endif +default : + bytes_read =1 ; +break ; + } + +/** +* read out to clear the ready-to-be-read flag +* for epoll_wait. +*/ + vec[i] = events[i].data.u32; +assert (vec[i] <VFIO_MAX_RXTX_INTR_ID ); + + fd = intr_handle->efds[vec[i]]; + bytes_read =read (fd, &buf, bytes_read); +if (bytes_read <0 ) + RTE_LOG(ERR, EAL, "Error reading from file " + "descriptor %d: %s\n", fd,strerror(errno));
lib/librte_eal/linuxapp/eal/eal_interrupts.c rte_eal_intr_init(void) · line 896
Quoted 5 lines
lib/librte_eal/linuxapp/eal/eal_interrupts.c +5
-0+else if (bytes_read ==0 ) +RTE_LOG (ERR ,EAL ,"Read nothing from file " +"descriptor %d\n" , fd); + } +}
Why unconditionnally read ?
You are absorbing events from the application if the application gave you an external epfd and populated it with its own fds.
Quoted 7 lines
lib/librte_eal/linuxapp/eal/eal_interrupts.c +6
-0+ +static int init_tls_epfd (void ) +{ +int pfd =epoll_create (1 ); +if (pfd <0 ) { + RTE_LOG(ERR, EAL,
lib/librte_eal/linuxapp/eal/eal_interrupts.c rte_eal_intr_init(void) · line 912
Quoted 10 lines
lib/librte_eal/linuxapp/eal/eal_interrupts.c +10
-0+ "Cannot create epoll instance\n"); +return -1 ; + } +return pfd; +} + +int +rte_intr_rx_wait (struct rte_intr_handle *intr_handle,int epfd, +uint32_t *vec,uint16_t num) +{
In the end, this "rx" does not mean anything to eal.
+#define MAX_EVENTS 8
Quoted 10 lines
lib/librte_eal/linuxapp/eal/eal_interrupts.c +9
-0+struct epoll_event events[MAX_EVENTS ]; +int ret, nfds =0 ; + +if (!intr_handle || !vec) { +RTE_LOG (ERR ,EAL ,"invalid input parameter\n" ); +return -1 ; + } + +if (intr_handle->type !=RTE_INTR_HANDLE_VFIO_MSIX ) {
lib/librte_eal/linuxapp/eal/eal_interrupts.c rte_eal_intr_init(void) · line 932
Quoted 10 lines
lib/librte_eal/linuxapp/eal/eal_interrupts.c +10
-0+RTE_LOG (ERR ,EAL ,"intr type should be VFIO_MSIX\n" ); +return -1 ; + } + +if (epfd ==RTE_EPOLL_FD_ANY ) { +/* using per thread epoll fd */ +if (unlikely (RTE_PER_LCORE (_epfd) ==-1 )) +RTE_PER_LCORE (_epfd) =init_tls_epfd (); + epfd =RTE_PER_LCORE (_epfd); + }
Rather than testing every time, this should be set by the caller, i.e. epfd is always valid.
If application does not want to create a epfd, then it calls
rte_intr_rx_wait with RTE_EPOLL_FD_ANY (this name is not well chosen) that
is a macro wrapped to RTE_PER_LCORE(_epfd).
init_tls_epfd() should be called only once at init time. No need to check every time.
+
Quoted 14 lines
lib/librte_eal/linuxapp/eal/eal_interrupts.c +7
-0+ do { + ret = epoll_wait(epfd, events, + RTE_MIN(num, MAX_EVENTS), + EAL_INTR_EPOLL_WAIT_FOREVER); + if (unlikely(ret < 0)) { + /* epoll_wait fail */ + RTE_LOG(ERR, EAL, "epoll_wait returns withfail\n");
lib/librte_eal/linuxapp/eal/eal_interrupts.c +4
-0+return -1 ; + }else if (ret >0 ) { +/* epoll_wait has at least one fd ready to read */ + eal_intr_process_rxtx_interrupts(intr_handle,events,
lib/librte_eal/linuxapp/eal/eal_interrupts.c rte_eal_intr_init(void) · line 954
Quoted 10 lines
lib/librte_eal/linuxapp/eal/eal_interrupts.c +10
-0+ vec, ret); + num -= ret; + vec += ret; + nfds += ret; + }else if (nfds >0 ) +break ; + }while (num >0 ); + +return nfds; +}
You are blocking unless all fds have been set, so you are serialising all events.
+
Quoted 63 lines
lib/librte_eal/linuxapp/eal/eal_interrupts.c +45
-0+int +rte_intr_rx_set (struct rte_intr_handle *intr_handle,int epfd, +int op,uint32_t vec) +{ +struct epoll_event ev; + +if (!intr_handle || vec >=VFIO_MAX_RXTX_INTR_ID ) { +RTE_LOG (ERR ,EAL ,"invalid input parameter\n" ); +return -1 ; + } + +if (intr_handle->type !=RTE_INTR_HANDLE_VFIO_MSIX ) { +RTE_LOG (ERR ,EAL ,"intr type should be VFIO_MSIX\n" ); +return -1 ; + } + +switch (op) { +case RTE_INTR_EVENT_ADD : + op =EPOLL_CTL_ADD ; +break ; +case RTE_INTR_EVENT_DEL : + op =EPOLL_CTL_DEL ; +break ; +default : +RTE_LOG (ERR ,EAL ,"event op type mismatch\n" ); +return -1 ; + } + +if (epfd ==RTE_EPOLL_FD_ANY ) { +/* using per thread epoll fd */ +if (RTE_PER_LCORE (_epfd) ==-1 ) +RTE_PER_LCORE (_epfd) =init_tls_epfd (); + epfd =RTE_PER_LCORE (_epfd); + } + + ev.data.u32 = vec; + ev.events =EPOLLIN |EPOLLPRI ; +if (epoll_ctl (epfd, op, intr_handle->efds[vec], &ev) <0 ) { +RTE_LOG (ERR ,EAL ,"Error op %d fd %d epoll_ctl, %s\n" , + op, intr_handle->efds[vec],strerror (errno)); +return -1 ; + } + +return 0 ; +}lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +1
-0@@ -38,6 +38,7 @@#include <sys/socket.h> #include <sys/ioctl.h> #include <sys/mman.h> +#include <sys/epoll.h> #include <rte_log.h> #include <rte_pci.h>
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c pci_vfio_setup_interrupts(struct rte_pci_device · line 283
Quoted 10 lines
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +1
-1@@ -274,16 +275,18 @@ pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd) ret =ioctl (vfio_dev_fd,VFIO_DEVICE_GET_IRQ_INFO , &irq);if (ret <0 ) {RTE_LOG (ERR ,EAL ," cannot get IRQ info, " -"error %i (%s)\n" , errno, strerror(errno)); + "error %i (%s)\n", errno, strerror(errno));return -1 ; }
Garbage, this has nothing to do with the patch.
Quoted 13 lines
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +0
-0/* if this vector cannot be used with eventfd, fail if we explicitly
* specified interrupt type, otherwise continue */
if ((irq.flags &VFIO_IRQ_INFO_EVENTFD) == 0) {
- if (internal_config.vfio_intr_mode !=RTE_INTR_MODE_NONE) {lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +2
-1+if (internal_config.vfio_intr_mode != +RTE_INTR_MODE_NONE ) { RTE_LOG(ERR, EAL, - " interrupt vector doesnot support eventfd!\n");
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd) · line 292
Quoted 5 lines
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +2
-0+" interrupt vector " + "does not support eventfd!\n");return -1 ; }else continue ;
Idem.
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c pci_vfio_setup_interrupts(struct rte_pci_device · line 302
Quoted 9 lines
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +1
-1@@ -293,17 +296,27 @@ pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd) fd =eventfd (0 ,0 );if (fd <0 ) {RTE_LOG (ERR ,EAL ," cannot set up eventfd, " -"error %i (%s)\n" , errno, strerror(errno)); + "error %i (%s)\n", errno, strerror(errno));
Idem.
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd)
Quoted 7 lines
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +0
-1return -1 ; } dev->intr_handle.fd = fd; dev->intr_handle.vfio_dev_fd = vfio_dev_fd; -
Idem.
Quoted 36 lines
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +0
-0switch (i) {case VFIO_PCI_MSIX_IRQ_INDEX : internal_config.vfio_intr_mode =RTE_INTR_MODE_MSIX; dev->intr_handle.type = RTE_INTR_HANDLE_VFIO_MSIX;lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +11
-0+for (i =0 ; i <VFIO_MAX_RXTX_INTR_ID ; i++) { + fd =eventfd (0 ,0 ); +if (fd <0 ) { +RTE_LOG (ERR ,EAL , +"cannot setup eventfd," +"error %i (%s)\n" , + errno,strerror (errno)); +return -1 ; + } + dev->intr_handle.efds[i] = fd; + }break ;case VFIO_PCI_MSI_IRQ_INDEX : internal_config.vfio_intr_mode =RTE_INTR_MODE_MSI ;lib/librte_eal/linuxapp/eal/rte_eal_version.map +2
-1@@ -64,6 +64,8 @@ DPDK_2.0 { rte_intr_callback_unregister; rte_intr_disable; rte_intr_enable; + rte_intr_rx_set; + rte_intr_rx_wait; rte_log; rte_log_add_in_history; rte_log_cur_msg_loglevel; --1.8.1.4
Signature · 1 line
David Marchand
A message that is not in the archive (sent off-list or not yet received): CALwxeUu2GCzqR+RSgpp1GX=bCDpZTyPcqD4RQzsCx5d48RvBdQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Re: [PATCH v6 4/8] eal/linux: add per rx queue interrupt handling based on VFIO
Hi,
From: David Marchand [mailto:[email protected]]
Sent: Friday, February 27, 2015 6:34 PM
To: Liang, Cunming
Cc: [email protected]; Stephen Hemminger; Thomas Monjalon; Zhou, Danny
Subject: Re: [PATCH v6 4/8] eal/linux: add per rx queue interrupt handling based on VFIO
I am not really comfortable with this api.
This is just creating something on top of the standard epoll api with limitations.
In the end, we could just use an external lib that does this already.
[Liang, Cunming] Not really, I think. We try to protect the data inside ‘rte_intr_handle’, it doesn’t expect user to understand the things defined inside ‘rte_intr_handle’. It’s better typedef ‘rte_intr_handle’ as a raw integer ID, having a function to get it from a ethdev. Then all the interrupt api is around it. It provides the common pci NIC devices rxtx interrupt processing approach. For the limitations, we can fix it step by step.
So ok, this will work for your limited use case, but this will not be really useful for anything else.
Not sure it has its place in eal, this is more an example to me.
[Liang, Cunming] ‘limited use case’ do you means only for rxtx ? It don’t expect to provide a generic event mechanism (like libev/libevent does), but a simple way to allow PMD work with DMA interrupt. It mainly abstract for rx interrupt purpose. I appreciate if you could help to list more useful cases.
On Fri, Feb 27, 2015 at 5:56 AM, Cunming Liang <[email protected]<mailto:[email protected]>> wrote:
This patch does below:
- Create multiple VFIO eventfd for rx queues.
- Handle per rx queue interrupt.
- Eliminate unnecessary suspended DPDK polling thread wakeup mechanism
for rx interrupt by allowing polling thread epoll_wait rx queue
interrupt notification.
---
v6 changes
- split rte_intr_wait_rx_pkt into two function, wait and set.
- rewrite rte_intr_rx_wait/rte_intr_rx_set to remove queue visibility on eal.
- rte_intr_rx_wait to support multiplexing.
- allow epfd as input to support flexible event fd combination.
lib/librte_eal/linuxapp/eal/eal_interrupts.c | 224 +++++++++++++++++++----- lib/librte_eal/linuxapp/eal/eal_pci_vfio.c | 23 ++- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + 3 files changed, 201 insertions(+), 48 deletions(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +0 -0
[snip]
+static void
+eal_intr_process_rxtx_interrupts(struct rte_intr_handle *intr_handle,
+ struct epoll_event *events,
+ uint32_t *vec, int nfds)
+{
+ int i, bytes_read;
+ union rte_intr_read_buffer buf;
+ int fd;
+
+ for (i = 0; i < nfds; i++) {
+ /* set the length to be read for different handle type */
+ switch (intr_handle->type) {
+ case RTE_INTR_HANDLE_UIO:+ bytes_read = sizeof(buf.uio_intr_count);
+ break;
+ case RTE_INTR_HANDLE_ALARM:
+ bytes_read = sizeof(buf.timerfd_num);
+ break;
+#ifdef VFIO_PRESENT
+ case RTE_INTR_HANDLE_VFIO_MSIX: + case RTE_INTR_HANDLE_VFIO_MSI: + case RTE_INTR_HANDLE_VFIO_LEGACY:
+ bytes_read = sizeof(buf.vfio_intr_count);
+ break;
+#endif
+ default:
+ bytes_read = 1;
+ break;
+ }
+
+ /** + * read out to clear the ready-to-be-read flag + * for epoll_wait. + */ + vec[i] = events[i].data.u32; + assert(vec[i] < VFIO_MAX_RXTX_INTR_ID); + + fd = intr_handle->efds[vec[i]]; + bytes_read = read(fd, &buf, bytes_read); + if (bytes_read < 0)
+ RTE_LOG(ERR, EAL, "Error reading from file "
+ "descriptor %d: %s\n", fd, strerror(errno));
+ else if (bytes_read == 0)
+ RTE_LOG(ERR, EAL, "Read nothing from file "
+ "descriptor %d\n", fd);
+ }
+}
Why unconditionnally read ?
You are absorbing events from the application if the application gave you an external epfd and populated it with its own fds.
[Liang, Cunming] The vector number was checked. If an external epfd populated some event carry fd rather than a data.u32 but the value inside the valid range, it considers as a valid vector number. No matter the read success or not, it always notify the event. Do you have any suggestion used here to check the condition ?
+
+static int init_tls_epfd(void)
+{
+ int pfd = epoll_create(1);
+ if (pfd < 0) {
+ RTE_LOG(ERR, EAL,
+ "Cannot create epoll instance\n");
+ return -1;
+ }
+ return pfd;
+}
+
+int
+rte_intr_rx_wait(struct rte_intr_handle *intr_handle, int epfd,
+ uint32_t *vec, uint16_t num)
+{
In the end, this "rx" does not mean anything to eal.
[Liang, Cunming] That’s a good point. I tried to remove ‘rx’ and use a generic word here. ‘rte_intr_wait’ looks like too generic, ‘rte_intr_epfd_wait’ looks not abstract with bsd. As the function only serves for rxtx vector, so using the rx prefix. Which name do you prefer ?
+#define MAX_EVENTS 8
+ struct epoll_event events[MAX_EVENTS];
+ int ret, nfds = 0;
+
+ if (!intr_handle || !vec) {
+ RTE_LOG(ERR, EAL, "invalid input parameter\n");
+ return -1;
+ }
+
+ if (intr_handle->type != RTE_INTR_HANDLE_VFIO_MSIX) {
+ RTE_LOG(ERR, EAL, "intr type should be VFIO_MSIX\n");
+ return -1;
+ }
+
+ if (epfd == RTE_EPOLL_FD_ANY) {
+ /* using per thread epoll fd */
+ if (unlikely(RTE_PER_LCORE(_epfd) == -1))
+ RTE_PER_LCORE(_epfd) = init_tls_epfd();
+ epfd = RTE_PER_LCORE(_epfd);
+ }
Rather than testing every time, this should be set by the caller, i.e. epfd is always valid.
If application does not want to create a epfd, then it calls rte_intr_rx_wait with RTE_EPOLL_FD_ANY (this name is not well chosen) that is a macro wrapped to RTE_PER_LCORE(_epfd). [Liang, Cunming] It sounds good to me. As we don’t expect to expose *rte_per_lcore__epfd* as an public symbol, so will define rte_epfd() instread. Within rte_epfd(), if RTE_PER_LCORE(_epfd) not assigned, then init_tls_epfd() once.
init_tls_epfd() should be called only once at init time.
No need to check every time.
[Liang, Cunming] As it probably not need per thread epfd at all. So I prefer to create it when it real needed as above I mentioned.
+
+ do {
+ ret = epoll_wait(epfd, events,
+ RTE_MIN(num, MAX_EVENTS),
+ EAL_INTR_EPOLL_WAIT_FOREVER);
+ if (unlikely(ret < 0)) {
+ /* epoll_wait fail */ + RTE_LOG(ERR, EAL, "epoll_wait returns with fail\n"); + return -1;
+ } else if (ret > 0) {
+ /* epoll_wait has at least one fd ready to read */
+ eal_intr_process_rxtx_interrupts(intr_handle, events,
+ vec, ret);
+ num -= ret; + vec += ret; + nfds += ret;
+ } else if (nfds > 0)
+ break;
+ } while (num > 0);
+
+ return nfds;
+}
You are blocking unless all fds have been set, so you are serialising all events.
[Liang, Cunming] I’m not sure fully got your point. If any event arrives, it gets back. Do you means if no fds added in, it’s always blocking. You expect to have a timeout return ?
+
+int
+rte_intr_rx_set(struct rte_intr_handle *intr_handle, int epfd,
+ int op, uint32_t vec)
+{
+ struct epoll_event ev;
+
+ if (!intr_handle || vec >= VFIO_MAX_RXTX_INTR_ID) {
+ RTE_LOG(ERR, EAL, "invalid input parameter\n");
+ return -1;
+ }
+
+ if (intr_handle->type != RTE_INTR_HANDLE_VFIO_MSIX) {
+ RTE_LOG(ERR, EAL, "intr type should be VFIO_MSIX\n");
+ return -1;
+ }
+
+ switch (op) {
+ case RTE_INTR_EVENT_ADD:
+ op = EPOLL_CTL_ADD;
+ break;
+ case RTE_INTR_EVENT_DEL:
+ op = EPOLL_CTL_DEL;
+ break;
+ default:
+ RTE_LOG(ERR, EAL, "event op type mismatch\n");
+ return -1;
+ }
+
+ if (epfd == RTE_EPOLL_FD_ANY) {
+ /* using per thread epoll fd */
+ if (RTE_PER_LCORE(_epfd) == -1)
+ RTE_PER_LCORE(_epfd) = init_tls_epfd();
+ epfd = RTE_PER_LCORE(_epfd);
+ }
+
+ ev.data.u32 = vec;
+ ev.events = EPOLLIN | EPOLLPRI;
+ if (epoll_ctl(epfd, op, intr_handle->efds[vec], &ev) < 0) {+ RTE_LOG(ERR, EAL, "Error op %d fd %d epoll_ctl, %s\n",
+ op, intr_handle->efds[vec], strerror(errno));
+ return -1;
+ }
+
+ return 0;
+}
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +2 -1
@@ -38,6 +38,7 @@#include <sys/socket.h> #include <sys/ioctl.h> #include <sys/mman.h> +#include <sys/epoll.h> #include <rte_log.h> #include <rte_pci.h>
@@ -274,16 +275,18 @@ pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd) ret =ioctl (vfio_dev_fd,VFIO_DEVICE_GET_IRQ_INFO , &irq);if (ret <0 ) {RTE_LOG (ERR ,EAL ," cannot get IRQ info, " -"error %i (%s)\n" , errno,strerror (errno)); +"error %i (%s)\n" , errno,strerror (errno));return -1 ; }
Garbage, this has nothing to do with the patch. [Liang, Cunming] It’s for line number exceed 80 margin complain.
/* if this vector cannot be used with eventfd, fail if we explicitly
* specified interrupt type, otherwise continue */
if ((irq.flags & VFIO_IRQ_INFO_EVENTFD) == 0) {- if (internal_config.vfio_intr_mode != RTE_INTR_MODE_NONE) {
+ if (internal_config.vfio_intr_mode !=
+ RTE_INTR_MODE_NONE) {
RTE_LOG(ERR, EAL,
- " interrupt vector does not support eventfd!\n");
+ " interrupt vector "
+ "does not support eventfd!\n");
return -1;
} else
continue;Idem.
[Liang, Cunming] The same.
@@ -293,17 +296,27 @@ pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd)
fd = eventfd(0, 0);
if (fd < 0) {
RTE_LOG(ERR, EAL, " cannot set up eventfd, "- "error %i (%s)\n", errno, strerror(errno));
+ "error %i (%s)\n", errno, strerror(errno));
Idem.
[Liang, Cunming] The same.
return -1;
}
dev->intr_handle.fd = fd;
dev->intr_handle.vfio_dev_fd = vfio_dev_fd;-
Idem.
[Liang, Cunming] Accept.
switch (i) {
case VFIO_PCI_MSIX_IRQ_INDEX:
internal_config.vfio_intr_mode = RTE_INTR_MODE_MSIX;
dev->intr_handle.type = RTE_INTR_HANDLE_VFIO_MSIX;+ for (i = 0; i < VFIO_MAX_RXTX_INTR_ID; i++) {
+ fd = eventfd(0, 0);
+ if (fd < 0) {
+ RTE_LOG(ERR, EAL,
+ "cannot setup eventfd," + "error %i (%s)\n", + errno, strerror(errno));
+ return -1;
+ }
+ dev->intr_handle.efds[i] = fd;
+ }
break;
case VFIO_PCI_MSI_IRQ_INDEX:
internal_config.vfio_intr_mode = RTE_INTR_MODE_MSI;lib/librte_eal/linuxapp/eal/rte_eal_version.map +2 -0
@@ -64,6 +64,8 @@ DPDK_2.0 { rte_intr_callback_unregister; rte_intr_disable; rte_intr_enable; + rte_intr_rx_set; + rte_intr_rx_wait; rte_log; rte_log_add_in_history; rte_log_cur_msg_loglevel;
--
1.8.1.4
--
David Marchand
A message that is not in the archive (sent off-list or not yet received): D0158A423229094DA7ABF71CF2FA0DA3118DFC6E-E2R4CRU6q/[email protected]
Re: [PATCH v6 4/8] eal/linux: add per rx queue interrupt handling based on VFIO
Hi Cunming,
First, sorry to have to say that, but it is not easy to read discussions where quote marks are not used. I re-insert them for clarity.
Comments below.
2015-02-27 12:22, Liang, Cunming:
Quoted 21 lines
From: David Marchand [mailto:[email protected]]
Sent: Friday, February 27, 2015 6:34 PMQuoted 5 lines
I am not really comfortable with this api.
This is just creating something on top of the standard epoll api with limitations. In the end, we could just use an external lib that does this already.
[Liang, Cunming] Not really, I think. We try to protect the data inside ‘rte_intr_handle’, it doesn’t expect user to understand the things defined inside ‘rte_intr_handle’.
It’s better typedef ‘rte_intr_handle’ as a raw integer ID, having a function to get it from a ethdev. Then all the interrupt api is around it. It provides the common pci NIC devices rxtx interrupt processing approach. For the limitations, we can fix it step by step.Quoted 3 lines
So ok, this will work for your limited use case, but this will not be really useful for anything else.
Not sure it has its place in eal, this is more an example to me.
Quoted 5 lines
[Liang, Cunming] ‘limited use case’ do you means only for rxtx ? It don’t expect to provide a generic event mechanism (like libev/libevent does), but a simple way to allow PMD work with DMA interrupt. It mainly abstract for rx interrupt purpose. I appreciate if you could help to list more useful cases.
You don't expect to provide a generic event mechanism but application developpers could need to wait for many events at once, not only Rx ones. That's why it's better to provide only the needed parts to use something generic like libevent.
And we should avoid reinventing the wheel.
Quoted 76 lines
Quoted 43 lines
Quoted 43 lines
(quoted diff) +43
-0+static void +eal_intr_process_rxtx_interrupts(struct rte_intr_handle *intr_handle, + struct epoll_event *events, + uint32_t *vec, int nfds) +{ + int i, bytes_read; + union rte_intr_read_buffer buf; + int fd; + + for (i = 0; i < nfds; i++) { + /* set the length to be read for different handle type */ + switch (intr_handle->type) { + case RTE_INTR_HANDLE_UIO: + bytes_read = sizeof(buf.uio_intr_count); + break; + case RTE_INTR_HANDLE_ALARM: + bytes_read = sizeof(buf.timerfd_num); + break; +#ifdef VFIO_PRESENT + case RTE_INTR_HANDLE_VFIO_MSIX: + case RTE_INTR_HANDLE_VFIO_MSI: + case RTE_INTR_HANDLE_VFIO_LEGACY: + bytes_read = sizeof(buf.vfio_intr_count); + break; +#endif + default: + bytes_read = 1; + break; + } + + /** + * read out to clear the ready-to-be-read flag + * for epoll_wait. + */ + vec[i] = events[i].data.u32; + assert(vec[i] < VFIO_MAX_RXTX_INTR_ID); + + fd = intr_handle->efds[vec[i]]; + bytes_read = read(fd, &buf, bytes_read); + if (bytes_read < 0) + RTE_LOG(ERR, EAL, "Error reading from file " + "descriptor %d: %s\n", fd, strerror(errno)); + else if (bytes_read == 0)Quoted 8 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ RTE_LOG(ERR, EAL, "Read nothing from file " + "descriptor %d\n", fd); + } +}Why unconditionnally read ?
You are absorbing events from the application if the application gave you an external epfd and populated it with its own fds.[Liang, Cunming] The vector number was checked. If an external epfd populated some event carry fd rather than a data.u32 but the value inside the valid range, it considers as a valid vector number. No matter the read success or not, it always notify the event. Do you have any suggestion used here to check the condition ?
Quoted 9 lines
Quoted 9 lines
(quoted diff) +9
-0+static int init_tls_epfd(void) +{ + int pfd = epoll_create(1); + if (pfd < 0) { + RTE_LOG(ERR, EAL, + "Cannot create epoll instance\n"); + return -1; + } + return pfd;Quoted 8 lines
Review
Quoted 6 lines
(quoted diff) +6
-0+} + +int +rte_intr_rx_wait(struct rte_intr_handle *intr_handle, int epfd, + uint32_t *vec, uint16_t num) +{In the end, this "rx" does not mean anything to eal.
Quoted 5 lines
[Liang, Cunming] That’s a good point. I tried to remove ‘rx’ and use a generic word here. ‘rte_intr_wait’ looks like too generic, ‘rte_intr_epfd_wait’ looks not abstract with bsd.
As the function only serves for rxtx vector, so using the rx prefix. Which name do you prefer ?
You should understand that you are trying to wrongly replace a generic lib. The best name is probably /dev/null.
Quoted 68 lines
Quoted 18 lines
Quoted 18 lines
(quoted diff) +18
-0+#define MAX_EVENTS 8 + struct epoll_event events[MAX_EVENTS]; + int ret, nfds = 0; + + if (!intr_handle || !vec) { + RTE_LOG(ERR, EAL, "invalid input parameter\n"); + return -1; + } + + if (intr_handle->type != RTE_INTR_HANDLE_VFIO_MSIX) { + RTE_LOG(ERR, EAL, "intr type should be VFIO_MSIX\n"); + return -1; + } + + if (epfd == RTE_EPOLL_FD_ANY) { + /* using per thread epoll fd */ + if (unlikely(RTE_PER_LCORE(_epfd) == -1)) + RTE_PER_LCORE(_epfd) = init_tls_epfd();Quoted 8 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ epfd = RTE_PER_LCORE(_epfd); + }Rather than testing every time, this should be set by the caller, i.e. epfd is always valid.
If application does not want to create a epfd, then it callsrte_intr_rx_waitwithRTE_EPOLL_FD_ANY(this name is not well chosen) that is a macro wrapped toRTE_PER_LCORE(_epfd).[Liang, Cunming] It sounds good to me. As we don’t expect to expose *rte_per_lcore__epfd* as an public symbol, so will define
rte_epfd()instread.
Withinrte_epfd(), ifRTE_PER_LCORE(_epfd)not assigned, theninit_tls_epfd()once.Quoted 2 lines
init_tls_epfd()should be called only once at init time. No need to check every time.[Liang, Cunming] As it probably not need per thread epfd at all. So I prefer to create it when it real needed as above I mentioned.
Quoted 16 lines
Quoted 16 lines
(quoted diff) +16
-0+ do { + ret = epoll_wait(epfd, events, + RTE_MIN(num, MAX_EVENTS), + EAL_INTR_EPOLL_WAIT_FOREVER); + if (unlikely(ret < 0)) { + /* epoll_wait fail */ + RTE_LOG(ERR, EAL, "epoll_wait returns with fail\n"); + return -1; + } else if (ret > 0) { + /* epoll_wait has at least one fd ready to read */ + eal_intr_process_rxtx_interrupts(intr_handle, events, + vec, ret); + num -= ret; + vec += ret; + nfds += ret; + } else if (nfds > 0)Quoted 8 lines
Review
Quoted 5 lines
(quoted diff) +5
-0+ break; + } while (num > 0); + + return nfds; +}You are blocking unless all fds have been set, so you are serialising all events.
[Liang, Cunming] I’m not sure fully got your point. If any event arrives, it gets back. Do you means if no fds added in, it’s always blocking. You expect to have a timeout return ?
Quoted 8 lines
Quoted 6 lines
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd) · line 278
Quoted 4 lines
RTE_LOG(ERR, EAL, " cannot get IRQ info, "
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +1
-1- "error %i (%s)\n", errno, strerror(errno)); + "error %i (%s)\n", errno, strerror(errno));Garbage, this has nothing to do with the patch.
[Liang, Cunming] It’s for line number exceed 80 margin complain.
The title of the patch is "add per rx queue interrupt handling based on VFIO". So this kind of modification is a garbage.
Sorry, I won't play the game "idem / the same" below ;)
Quoted 25 lines
Quoted 9 lines
Review
Quoted 7 lines
(quoted diff) +4
-2- if (internal_config.vfio_intr_mode != RTE_INTR_MODE_NONE) { + if (internal_config.vfio_intr_mode != + RTE_INTR_MODE_NONE) { RTE_LOG(ERR, EAL, - " interrupt vector does not support eventfd!\n"); + " interrupt vector " + "does not support eventfd!\n");> >Idem.
[Liang, Cunming] The same.
Quoted 4 lines
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd) · line 278
Quoted 2 lines
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +1
-1- "error %i (%s)\n", errno, strerror(errno)); + "error %i (%s)\n", errno, strerror(errno));Idem.
[Liang, Cunming] The same.
Quoted 4 lines
Quoted 2 lines
dev->intr_handle.vfio_dev_fd = vfio_dev_fd;-
Idem.
[Liang, Cunming] Accept.
Re: [PATCH v6 4/8] eal/linux: add per rx queue interrupt handling based on VFIO
Thanks Thomas.
It's my fault that directly reply David's mail, haven't notice his mail isn't in a plain text mode.
Quoted 43 lines
-----Original Message-----
From: Thomas Monjalon [mailto:[email protected]]
Sent: Friday, February 27, 2015 10:13 PM
To: Liang, Cunming
Cc: David Marchand; [email protected]; Stephen Hemminger; Zhou, Danny
Subject: Re: [PATCH v6 4/8] eal/linux: add per rx queue interrupt handling based on VFIOHi Cunming,
First, sorry to have to say that, but it is not easy to read discussions where quote marks are not used. I re-insert them for clarity.
Comments below.
2015-02-27 12:22, Liang, Cunming:
Quoted 21 lines
From: David Marchand [mailto:[email protected]]
Sent: Friday, February 27, 2015 6:34 PMQuoted 5 lines
I am not really comfortable with this api.
This is just creating something on top of the standard epoll api with limitations. In the end, we could just use an external lib that does this already.
[Liang, Cunming] Not really, I think. We try to protect the data inside ‘rte_intr_handle’, it doesn’t expect user to understand the things defined inside ‘rte_intr_handle’.
It’s better typedef ‘rte_intr_handle’ as a raw integer ID, having a function to get it from a ethdev. Then all the interrupt api is around it. It provides the common pci NIC devices rxtx interrupt processing approach. For the limitations, we can fix it step by step.Quoted 3 lines
So ok, this will work for your limited use case, but this will not be really useful for anything else.
Not sure it has its place in eal, this is more an example to me.Quoted 5 lines
[Liang, Cunming] ‘limited use case’ do you means only for rxtx ? It don’t expect to provide a generic event mechanism (like libev/libevent does), but a simple way to allow PMD work with DMA interrupt. It mainly abstract for rx interrupt purpose. I appreciate if you could help to list more useful cases.
Quoted 5 lines
You don't expect to provide a generic event mechanism but application developpers could need to wait for many events at once, not only Rx ones. That's why it's better to provide only the needed parts to use something generic like libevent.
And we should avoid reinventing the wheel.
[LCM] Ok, I get you. I have a simple proposal to allow either RX event or other events can be handled in rte_intr_wait(). For the input data 'epoll_data', instead of using 'u32', let's keep use 'int fd'.
If the most significant bit is 0, event[n] stands for a fd. If it's 1, event[0]&0xFFFF stands for a vector number. So during 'rte_intr_set', it get 16bit vector number and encode it as a 32bit int with the most significant bit 1. Then on 'rte_intr_wait', only process the data.fd with the most significant bit 1. And bypass the user fd. 'rte_intr_wait(struct rte_intr_handle *intr_handle, int epfd, int *event, uint16_t num)'.
As user already can assign an epfd, so they can add any normal event fd into the epfd. Make sense ?
Quoted 77 lines
Quoted 77 lines
Quoted 44 lines
Quoted 44 lines
(quoted diff) +43
-0+static void +eal_intr_process_rxtx_interrupts(struct rte_intr_handle *intr_handle, + struct epoll_event *events, + uint32_t *vec, int nfds) +{ + int i, bytes_read; + union rte_intr_read_buffer buf; + int fd; + + for (i = 0; i < nfds; i++) { + /* set the length to be read for different handle type */ + switch (intr_handle->type) { + case RTE_INTR_HANDLE_UIO: + bytes_read = sizeof(buf.uio_intr_count); + break; + case RTE_INTR_HANDLE_ALARM: + bytes_read = sizeof(buf.timerfd_num); + break; +#ifdef VFIO_PRESENT + case RTE_INTR_HANDLE_VFIO_MSIX: + case RTE_INTR_HANDLE_VFIO_MSI: + case RTE_INTR_HANDLE_VFIO_LEGACY: + bytes_read = sizeof(buf.vfio_intr_count); + break; +#endif + default: + bytes_read = 1; + break; + } + + /** + * read out to clear the ready-to-be-read flag + * for epoll_wait. + */ + vec[i] = events[i].data.u32; + assert(vec[i] < VFIO_MAX_RXTX_INTR_ID); + + fd = intr_handle->efds[vec[i]]; + bytes_read = read(fd, &buf, bytes_read); + if (bytes_read < 0) + RTE_LOG(ERR, EAL, "Error reading from file " + "descriptor %d: %s\n", fd, strerror(errno)); + else if (bytes_read == 0)Quoted 8 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ RTE_LOG(ERR, EAL, "Read nothing from file " + "descriptor %d\n", fd); + } +}Why unconditionnally read ?
You are absorbing events from the application if the application gave you an external epfd and populated it with its own fds.[Liang, Cunming] The vector number was checked. If an external epfd populated some event carry fd rather than a data.u32 but the value inside the valid range, it considers as a valid vector number. No matter the read success or not, it always notify the event. Do you have any suggestion used here to check the condition ?
Quoted 17 lines
Quoted 5 lines
(quoted diff) +5
-0+static int init_tls_epfd(void) +{ + int pfd = epoll_create(1); + if (pfd < 0) { + RTE_LOG(ERR, EAL,Review
Quoted 10 lines
(quoted diff) +10
-0+ "Cannot create epoll instance\n"); + return -1; + } + return pfd; +} + +int +rte_intr_rx_wait(struct rte_intr_handle *intr_handle, int epfd, + uint32_t *vec, uint16_t num) +{In the end, this "rx" does not mean anything to eal.
Quoted 8 lines
Quoted 5 lines
[Liang, Cunming] That’s a good point. I tried to remove ‘rx’ and use a generic word here. ‘rte_intr_wait’ looks like too generic, ‘rte_intr_epfd_wait’ looks not abstract with bsd.
As the function only serves for rxtx vector, so using the rx prefix. Which name do you prefer ?You should understand that you are trying to wrongly replace a generic lib. The best name is probably
/dev/null.
[Liang, Cunming] If allowing other user fd added into the epfd instance, I feel ok to rename it to rte_intr_wait().
Quoted 74 lines
Quoted 74 lines
Quoted 19 lines
Quoted 19 lines
(quoted diff) +18
-0+#define MAX_EVENTS 8 + struct epoll_event events[MAX_EVENTS]; + int ret, nfds = 0; + + if (!intr_handle || !vec) { + RTE_LOG(ERR, EAL, "invalid input parameter\n"); + return -1; + } + + if (intr_handle->type != RTE_INTR_HANDLE_VFIO_MSIX) { + RTE_LOG(ERR, EAL, "intr type should be VFIO_MSIX\n"); + return -1; + } + + if (epfd == RTE_EPOLL_FD_ANY) { + /* using per thread epoll fd */ + if (unlikely(RTE_PER_LCORE(_epfd) == -1)) + RTE_PER_LCORE(_epfd) = init_tls_epfd();Quoted 8 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ epfd = RTE_PER_LCORE(_epfd); + }Rather than testing every time, this should be set by the caller, i.e. epfd is always valid.
If application does not want to create a epfd, then it callsrte_intr_rx_waitwithRTE_EPOLL_FD_ANY(this name is not well chosen) that is a macro wrapped toRTE_PER_LCORE(_epfd).[Liang, Cunming] It sounds good to me. As we don’t expect to expose *rte_per_lcore__epfd* as an public symbol, so will define
rte_epfd()instread.
Withinrte_epfd(), ifRTE_PER_LCORE(_epfd)not assigned, theninit_tls_epfd()once.Quoted 2 lines
init_tls_epfd()should be called only once at init time. No need to check every time.[Liang, Cunming] As it probably not need per thread epfd at all. So I prefer to create it when it real needed as above I mentioned.
Quoted 16 lines
Quoted 16 lines
(quoted diff) +16
-0+ do { + ret = epoll_wait(epfd, events, + RTE_MIN(num, MAX_EVENTS), + EAL_INTR_EPOLL_WAIT_FOREVER); + if (unlikely(ret < 0)) { + /* epoll_wait fail */ + RTE_LOG(ERR, EAL, "epoll_wait returns with fail\n"); + return -1; + } else if (ret > 0) { + /* epoll_wait has at least one fd ready to read */ + eal_intr_process_rxtx_interrupts(intr_handle, events, + vec, ret); + num -= ret; + vec += ret; + nfds += ret; + } else if (nfds > 0)Quoted 8 lines
Review
Quoted 5 lines
(quoted diff) +5
-0+ break; + } while (num > 0); + + return nfds; +}You are blocking unless all fds have been set, so you are serialising all events.
[Liang, Cunming] I’m not sure fully got your point. If any event arrives, it gets back. Do you means if no fds added in, it’s always blocking. You expect to have a timeout return ?
Quoted 5 lines
Quoted 4 lines
RTE_LOG(ERR, EAL, " cannot get IRQ info, "
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +1
-1- "error %i (%s)\n", errno, strerror(errno)); + "error %i (%s)\n", errno, strerror(errno));
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Garbage, this has nothing to do with the patch.
[Liang, Cunming] It’s for line number exceed 80 margin complain.
The title of the patch is "add per rx queue interrupt handling based on VFIO". So this kind of modification is a garbage.
Sorry, I won't play the game "idem / the same" below ;)
[Liang, Cunming] That's actually a question in my mind.
For example, there's one line change in diff file. It usually has several line code around it. If these line code can't pass code checkpatch rule, what's the correct way to handle it?
Do you suggest having a patch first to correct the format, and then follows the patch to submit the formal content.
Quoted 27 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
- if (internal_config.vfio_intr_mode !=
RTE_INTR_MODE_NONE){
Quoted 24 lines
Quoted 8 lines
Review
Quoted 6 lines
(quoted diff) +4
-1+ if (internal_config.vfio_intr_mode != + RTE_INTR_MODE_NONE) { RTE_LOG(ERR, EAL, - " interrupt vector does not support eventfd!\n"); + " interrupt vector " + "does not support eventfd!\n");> >Idem.
[Liang, Cunming] The same.
Quoted 4 lines
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd) · line 278
Quoted 2 lines
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +1
-1- "error %i (%s)\n", errno, strerror(errno)); + "error %i (%s)\n", errno, strerror(errno));Idem.
[Liang, Cunming] The same.
Quoted 4 lines
Quoted 2 lines
dev->intr_handle.vfio_dev_fd = vfio_dev_fd;-
Idem.
[Liang, Cunming] Accept.
PATCH v6 5/8 ethdev: add rx interrupt enable/disable functions
In reply to a message not in the archiveAdd three dev_ops functions to enable and disable rx queue interrupts; and to retrieve the vector num which the specified queue assosiated with.
---
v6 changes
- add rx_intr_vec_get to retrieve the vector num of the queue.
v5 changes
- Rebase the patchset onto the HEAD
v4 changes
- Export interrupt enable/disable functions for shared libraries
- Put new functions at the end of eth_dev_ops to avoid breaking ABI
v3 changes
- Add return value for interrupt enable/disable functions
lib/librte_ether/rte_ethdev.c | 66 +++++++++++++++++++++++++++++ lib/librte_ether/rte_ethdev.h | 77 ++++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ether_version.map | 3 ++ 3 files changed, 146 insertions(+)
lib/librte_ether/rte_ethdev.c +66 -0
@@ -3320,6 +3320,72 @@ _rte_eth_dev_callback_process(struct rte_eth_dev *dev, }rte_spinlock_unlock (&rte_eth_dev_cb_lock); } + +int +rte_eth_dev_rx_intr_vec_get (uint8_t port_id,uint16_t queue_id, +uint32_t *vec) +{ +struct rte_eth_dev *dev; +struct rte_intr_handle *intr_handle; + +if (port_id >= nb_ports) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return -ENODEV ; + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return -ENODEV ; + } + + intr_handle = &dev->pci_dev->intr_handle; + *vec = intr_handle->vec_num[queue_id]; +return 0 ; +} + +int +rte_eth_dev_rx_intr_enable (uint8_t port_id, +uint16_t queue_id) +{ +struct rte_eth_dev *dev; + +if (port_id >= nb_ports) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return -ENODEV ; + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return -ENODEV ; + } + +FUNC_PTR_OR_ERR_RET (*dev->dev_ops->rx_queue_intr_enable, -ENOTSUP ); +return (*dev->dev_ops->rx_queue_intr_enable)(dev, queue_id); +} + +int +rte_eth_dev_rx_intr_disable (uint8_t port_id, +uint16_t queue_id) +{ +struct rte_eth_dev *dev; + +if (port_id >= nb_ports) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return -ENODEV ; + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return -ENODEV ; + } + +FUNC_PTR_OR_ERR_RET (*dev->dev_ops->rx_queue_intr_disable, -ENOTSUP ); +return (*dev->dev_ops->rx_queue_intr_disable)(dev, queue_id); +} +#ifdef RTE_NIC_BYPASS int rte_eth_dev_bypass_init (uint8_t port_id) {
lib/librte_ether/rte_ethdev.h +77 -0
101 lines not on this page · raw message
lib/librte_ether/rte_ether_version.map +3 -0
@@ -47,6 +47,9 @@ DPDK_2.0 { rte_eth_dev_rss_hash_update; rte_eth_dev_rss_reta_query; rte_eth_dev_rss_reta_update; + rte_eth_dev_rx_intr_disable; + rte_eth_dev_rx_intr_enable; + rte_eth_dev_rx_intr_vec_get; rte_eth_dev_rx_queue_start; rte_eth_dev_rx_queue_stop; rte_eth_dev_set_link_down;
Signature · 1 line
1.8.1.4
PATCH v6 6/8 ixgbe: enable rx queue interrupts for both PF and VF
In reply to a message not in the archiveFrom: "Zhou, Danny" <[email protected]>
The patch does below things for ixgbe PF and VF:
- Setup NIC to generate MSI-X interrupts
- Set the IVAR register to map interrupt causes to vectors
- Implement interrupt enable/disable functions
---
v6 changes
- fill queue-vector mapping table
v5 changes
- Rebase the patchset onto the HEAD
v3 changes
- Remove spinlok from PMD
v2 changes
- Consolidate review comments related to coding style
lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 377 +++++++++++++++++++++++++++++++++++- lib/librte_pmd_ixgbe/ixgbe_ethdev.h | 7 + 2 files changed, 380 insertions(+), 4 deletions(-)
lib/librte_pmd_ixgbe/ixgbe_ethdev.c +373 -4
@@ -83,6 +83,9 @@*/ #define IXGBE_FC_LO 0x40 +/* Default minimum inter-interrupt interval for EITR configuration */ +#define IXGBE_MIN_INTER_INTERRUPT_INTERVAL_DEFAULT 0x79E +/* Timer value included in XOFF frames. */ #define IXGBE_FC_PAUSE 0x680
@@ -173,6 +176,7 @@ static int ixgbe_dev_rss_reta_query(struct rte_eth_dev *dev, uint16_t reta_size);static void ixgbe_dev_link_status_print (struct rte_eth_dev *dev);static int ixgbe_dev_lsc_interrupt_setup (struct rte_eth_dev *dev); +static int ixgbe_dev_rxq_interrupt_setup (struct rte_eth_dev *dev);static int ixgbe_dev_interrupt_get_status (struct rte_eth_dev *dev);static int ixgbe_dev_interrupt_action (struct rte_eth_dev *dev); static void ixgbe_dev_interrupt_handler(struct rte_intr_handle *handle,
@@ -186,11 +190,14 @@ static void ixgbe_dcb_init(struct ixgbe_hw *hw,struct ixgbe_dcb_config *dcb_conf/* For Virtual Function support */ static int eth_ixgbevf_dev_init (struct eth_driver *eth_drv,struct rte_eth_dev *eth_dev); +static int ixgbevf_dev_interrupt_get_status (struct rte_eth_dev *dev); +static int ixgbevf_dev_interrupt_action (struct rte_eth_dev *dev);static int ixgbevf_dev_configure (struct rte_eth_dev *dev);static int ixgbevf_dev_start (struct rte_eth_dev *dev);static void ixgbevf_dev_stop (struct rte_eth_dev *dev);static void ixgbevf_dev_close (struct rte_eth_dev *dev);static void ixgbevf_intr_disable (struct ixgbe_hw *hw); +static void ixgbevf_intr_enable (struct ixgbe_hw *hw);static void ixgbevf_dev_stats_get (struct rte_eth_dev *dev,struct rte_eth_stats *stats);static void ixgbevf_dev_stats_reset (struct rte_eth_dev *dev);
@@ -200,6 +207,15 @@ static void ixgbevf_vlan_strip_queue_set(struct rte_eth_dev *dev, uint16_t queue, int on);static void ixgbevf_vlan_offload_set (struct rte_eth_dev *dev,int mask);static void ixgbevf_set_vfta_all (struct rte_eth_dev *dev,bool on); +static void ixgbevf_dev_interrupt_handler (struct rte_intr_handle *handle, +void *param); +static int ixgbevf_dev_rx_queue_intr_enable (struct rte_eth_dev *dev, +uint16_t queue_id); +static int ixgbevf_dev_rx_queue_intr_disable (struct rte_eth_dev *dev, +uint16_t queue_id); +static void ixgbevf_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector); +static void ixgbevf_configure_msix (struct rte_eth_dev *dev);/* For Eth VMDQ APIs support */ static int ixgbe_uc_hash_table_set(struct rte_eth_dev *dev, struct
@@ -217,6 +233,14 @@ static int ixgbe_mirror_rule_set(struct rte_eth_dev *dev,static int ixgbe_mirror_rule_reset (struct rte_eth_dev *dev,uint8_t rule_id); +static int ixgbe_dev_rx_queue_intr_enable (struct rte_eth_dev *dev, +uint16_t queue_id); +static int ixgbe_dev_rx_queue_intr_disable (struct rte_eth_dev *dev, +uint16_t queue_id); +static void ixgbe_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector); +static void ixgbe_configure_msix (struct rte_eth_dev *dev); +static int ixgbe_set_queue_rate_limit (struct rte_eth_dev *dev,uint16_t queue_idx,uint16_t tx_rate); static int ixgbe_set_vf_rate_limit(struct rte_eth_dev *dev, uint16_t vf,
@@ -265,7 +289,7 @@ static int ixgbevf_dev_set_mtu(struct rte_eth_dev *dev, uint16_t mtu);*/ #define UPDATE_VF_STAT (reg, last, cur) \ { \ - u32 latest = IXGBE_READ_REG(hw, reg); \ + uint32_t latest = IXGBE_READ_REG(hw, reg); \ cur += latest - last; \ last = latest; \ }
@@ -346,6 +370,8 @@ static struct eth_dev_ops ixgbe_eth_dev_ops = { .tx_queue_start = ixgbe_dev_tx_queue_start, .tx_queue_stop = ixgbe_dev_tx_queue_stop, .rx_queue_setup = ixgbe_dev_rx_queue_setup, + .rx_queue_intr_enable = ixgbe_dev_rx_queue_intr_enable, + .rx_queue_intr_disable = ixgbe_dev_rx_queue_intr_disable, .rx_queue_release = ixgbe_dev_rx_queue_release, .rx_queue_count = ixgbe_dev_rx_queue_count, .rx_descriptor_done = ixgbe_dev_rx_descriptor_done,
@@ -406,8 +432,11 @@ static struct eth_dev_ops ixgbevf_eth_dev_ops = { .vlan_offload_set = ixgbevf_vlan_offload_set, .rx_queue_setup = ixgbe_dev_rx_queue_setup, .rx_queue_release = ixgbe_dev_rx_queue_release, + .rx_descriptor_done = ixgbe_dev_rx_descriptor_done, .tx_queue_setup = ixgbe_dev_tx_queue_setup, .tx_queue_release = ixgbe_dev_tx_queue_release, + .rx_queue_intr_enable = ixgbevf_dev_rx_queue_intr_enable, + .rx_queue_intr_disable = ixgbevf_dev_rx_queue_intr_disable, .mac_addr_add = ixgbevf_add_mac_addr, .mac_addr_remove = ixgbevf_remove_mac_addr, };
@@ -904,6 +933,10 @@ eth_ixgbe_dev_init(__attribute__((unused)) struct eth_driver *eth_drv, eth_dev->data->port_id, pci_dev->id.vendor_id, pci_dev->id.device_id); +/* set max interrupt vfio request */ + pci_dev->intr_handle.max_intr = hw->mac.max_rx_queues + +IXGBE_MAX_OTHER_INTR ; +rte_intr_callback_register (&(pci_dev->intr_handle), ixgbe_dev_interrupt_handler, (void *)eth_dev);
@@ -1085,6 +1118,15 @@ eth_ixgbevf_dev_init(__attribute__((unused)) struct eth_driver *eth_drv,return (-EIO ); } +/* set max interrupt vfio request */ + pci_dev->intr_handle.max_intr = hw->mac.max_rx_queues + +IXGBEVF_MAX_OTHER_INTR ; + +rte_intr_callback_register (&(pci_dev->intr_handle), + ixgbevf_dev_interrupt_handler, (void *)eth_dev); + +rte_intr_enable (&(pci_dev->intr_handle)); +PMD_INIT_LOG (DEBUG ,"port %d vendorID=0x%x deviceID=0x%x mac.type=%s" , eth_dev->data->port_id, pci_dev->id.vendor_id, pci_dev->id.device_id,"ixgbe_mac_82599_vf" );
@@ -1486,6 +1528,9 @@ ixgbe_dev_start(struct rte_eth_dev *dev)/* configure PF module if SRIOV enabled */ ixgbe_pf_host_configure (dev); +/* confiugre msix for sleep until rx interrupt */ +ixgbe_configure_msix (dev); +/* initialize transmission unit */ ixgbe_dev_tx_init (dev);
@@ -1561,6 +1606,10 @@ skip_link_setup:if (dev->data->dev_conf.intr_conf.lsc !=0 )ixgbe_dev_lsc_interrupt_setup (dev); +/* check if rxq interrupt is enabled */ +if (dev->data->dev_conf.intr_conf.rxq !=0 ) +ixgbe_dev_rxq_interrupt_setup (dev); +/* resume enabled intr since hw reset */ ixgbe_enable_intr (dev);
@@ -2238,6 +2287,28 @@ ixgbe_dev_lsc_interrupt_setup(struct rte_eth_dev *dev)return 0 ; } +/** +* It clears the interrupt causes and enables the interrupt. +* It will be called once only during nic initialized. +* +* @param dev +* Pointer to struct rte_eth_dev. +* +* @return +* - On success, zero. +* - On failure, a negative value. +*/ +static int +ixgbe_dev_rxq_interrupt_setup (struct rte_eth_dev *dev) +{ +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + + intr->mask |=IXGBE_EICR_RTX_QUEUE ; + +return 0 ; +} +/* * It reads ICR and sets flag (IXGBE_EICR_LSC) for the link_update. *
@@ -2264,10 +2335,10 @@ ixgbe_dev_interrupt_get_status(struct rte_eth_dev *dev)PMD_DRV_LOG (INFO ,"eicr %x" , eicr); intr->flags =0 ; -if (eicr &IXGBE_EICR_LSC ) { -/* set flag for async link update */ + +/* set flag for async link update */ +if (eicr &IXGBE_EICR_LSC ) intr->flags |=IXGBE_FLAG_NEED_LINK_UPDATE ; - }if (eicr &IXGBE_EICR_MAILBOX ) intr->flags |=IXGBE_FLAG_MAILBOX ;
@@ -2275,6 +2346,30 @@ ixgbe_dev_interrupt_get_status(struct rte_eth_dev *dev)return 0 ; } +static int +ixgbevf_dev_interrupt_get_status (struct rte_eth_dev *dev) +{ +uint32_t eicr; +struct ixgbe_hw *hw =IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + +/* clear all cause mask */ +ixgbevf_intr_disable (hw); + +/* read-on-clear nic registers here */ + eicr =IXGBE_READ_REG (hw,IXGBE_VTEICR ); +PMD_DRV_LOG (INFO ,"eicr %x" , eicr); + + intr->flags =0 ; + +/* set flag for async link update */ +if (eicr &IXGBE_EICR_LSC ) + intr->flags |=IXGBE_FLAG_NEED_LINK_UPDATE ; + +return 0 ; +} +/** * It gets and then prints the link status. *
@@ -2370,6 +2465,18 @@ ixgbe_dev_interrupt_action(struct rte_eth_dev *dev)return 0 ; } +static int +ixgbevf_dev_interrupt_action (struct rte_eth_dev *dev) +{ +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); + +PMD_DRV_LOG (DEBUG ,"enable intr immediately" ); +ixgbevf_intr_enable (hw); +rte_intr_enable (&(dev->pci_dev->intr_handle)); +return 0 ; +} +/** * Interrupt handler which shall be registered for alarm callback for delayed * handling specific interrupt to wait for the stable nic state. As the
@@ -2431,6 +2538,15 @@ ixgbe_dev_interrupt_handler(__rte_unused struct rte_intr_handle *handle,ixgbe_dev_interrupt_action (dev); } +static void +ixgbevf_dev_interrupt_handler(__rte_unused struct rte_intr_handle *handle, +void *param) +{ +struct rte_eth_dev *dev = (struct rte_eth_dev *)param; +ixgbevf_dev_interrupt_get_status (dev); +ixgbevf_dev_interrupt_action (dev); +} +static int ixgbe_dev_led_on (struct rte_eth_dev *dev) {
@@ -2929,6 +3045,19 @@ ixgbevf_intr_disable(struct ixgbe_hw *hw)IXGBE_WRITE_FLUSH (hw); } +static void +ixgbevf_intr_enable (struct ixgbe_hw *hw) +{ +PMD_INIT_FUNC_TRACE (); + +/* VF enable interrupt autoclean */ +IXGBE_WRITE_REG (hw,IXGBE_VTEIAM ,IXGBE_VF_IRQ_ENABLE_MASK ); +IXGBE_WRITE_REG (hw,IXGBE_VTEIAC ,IXGBE_VF_IRQ_ENABLE_MASK ); +IXGBE_WRITE_REG (hw,IXGBE_VTEIMS ,IXGBE_VF_IRQ_ENABLE_MASK ); + +IXGBE_WRITE_FLUSH (hw); +} +static int ixgbevf_dev_configure (struct rte_eth_dev *dev) {
@@ -2991,6 +3120,11 @@ ixgbevf_dev_start(struct rte_eth_dev *dev)ixgbevf_dev_rxtx_start (dev); +ixgbevf_configure_msix (dev); + +/* Re-enable interrupt for VF */ +ixgbevf_intr_enable (hw); +return 0 ; }
@@ -3528,6 +3662,241 @@ ixgbe_mirror_rule_reset(struct rte_eth_dev *dev, uint8_t rule_id)return 0 ; } + +static int +ixgbevf_dev_rx_queue_intr_enable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); + + mask =IXGBE_READ_REG (hw,IXGBE_VTEIMS ); + mask |= (1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_VTEIMS , mask); + +return 0 ; +} + +static int +ixgbevf_dev_rx_queue_intr_disable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); + + mask =IXGBE_READ_REG (hw,IXGBE_VTEIMS ); + mask &= ~(1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_VTEIMS , mask); + +return 0 ; +} + +static int +ixgbe_dev_rx_queue_intr_enable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + +if (queue_id <16 ) { +ixgbe_disable_intr (hw); + intr->mask |= (1 << queue_id); +ixgbe_enable_intr (dev); + }else if (queue_id <32 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (0 )); + mask &= (1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (0 ), mask); + }else if (queue_id <64 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (1 )); + mask &= (1 << (queue_id -32 )); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (1 ), mask); + } + +return 0 ; +} + +static int +ixgbe_dev_rx_queue_intr_disable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + +if (queue_id <16 ) { +ixgbe_disable_intr (hw); + intr->mask &= ~(1 << queue_id); +ixgbe_enable_intr (dev); + }else if (queue_id <32 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (0 )); + mask &= ~(1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (0 ), mask); + }else if (queue_id <64 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (1 )); + mask &= ~(1 << (queue_id -32 )); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (1 ), mask); + } + +return 0 ; +} + +static void +ixgbevf_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector) +{ +uint32_t tmp, idx; +if (direction ==-1 ) { +/* other causes */ + msix_vector |=IXGBE_IVAR_ALLOC_VAL ; + tmp =IXGBE_READ_REG (hw,IXGBE_VTIVAR_MISC ); + tmp &= ~0xFF ; + tmp |= msix_vector; +IXGBE_WRITE_REG (hw,IXGBE_VTIVAR_MISC , tmp); + }else { +/* rx or tx cause */ + msix_vector |=IXGBE_IVAR_ALLOC_VAL ; + idx = ((16 * (queue &1 )) + (8 * direction)); + tmp =IXGBE_READ_REG (hw,IXGBE_VTIVAR (queue >>1 )); + tmp &= ~(0xFF << idx); + tmp |= (msix_vector << idx); +IXGBE_WRITE_REG (hw,IXGBE_VTIVAR (queue >>1 ), tmp); + } +} + +/** +* set the IVAR registers, mapping interrupt causes to vectors +* @param hw +* pointer to ixgbe_hw struct +* @direction +* 0 for Rx, 1 for Tx, -1 for other causes +* @queue +* queue to map the corresponding interrupt to +* @msix_vector +* the vector to map to the corresponding queue +*/ +static void +ixgbe_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector) +{ +uint32_t tmp, idx; + + msix_vector |=IXGBE_IVAR_ALLOC_VAL ; +if (hw->mac.type == ixgbe_mac_82598EB) { +if (direction ==-1 ) + direction =0 ; + idx = (((direction *64 ) + queue) >>2 ) &0x1F ; + tmp =IXGBE_READ_REG (hw,IXGBE_IVAR (idx)); + tmp &= ~(0xFF << (8 * (queue &0x3 ))); + tmp |= (msix_vector << (8 * (queue &0x3 ))); +IXGBE_WRITE_REG (hw,IXGBE_IVAR (idx), tmp); + }else if ((hw->mac.type == ixgbe_mac_82599EB) || + (hw->mac.type == ixgbe_mac_X540)) { +if (direction ==-1 ) { +/* other causes */ + idx = ((queue &1 ) *8 ); + tmp =IXGBE_READ_REG (hw,IXGBE_IVAR_MISC ); + tmp &= ~(0xFF << idx); + tmp |= (msix_vector << idx); +IXGBE_WRITE_REG (hw,IXGBE_IVAR_MISC , tmp); + }else { +/* rx or tx causes */ + idx = ((16 * (queue &1 )) + (8 * direction)); + tmp =IXGBE_READ_REG (hw,IXGBE_IVAR (queue >>1 )); + tmp &= ~(0xFF << idx); + tmp |= (msix_vector << idx); +IXGBE_WRITE_REG (hw,IXGBE_IVAR (queue >>1 ), tmp); + } + } +} + +static void +ixgbevf_configure_msix (struct rte_eth_dev *dev) +{ +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle; +uint32_t q_idx; +uint32_t vector_idx =0 ; + +/* Configure all RX queues of VF */ +for (q_idx =0 ; q_idx < (hw->mac.max_rx_queues -1 ); q_idx++) { +/* Force all queue use vector 0, +* as IXGBE_VF_MAXMSIVECOTR = 1 */ +ixgbevf_set_ivar_map (hw,0 , q_idx, vector_idx); + intr_handle->vec_num[q_idx] = vector_idx; + } + +/* Configure VF Rx queue ivar */ +ixgbevf_set_ivar_map (hw,-1 ,1 , vector_idx); +} + +/** +* Sets up the hardware to properly generate MSI-X interrupts +* @hw +* board private structure +*/ +static void +ixgbe_configure_msix (struct rte_eth_dev *dev) +{ +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle; +int queue_id; +uint32_t mask; +uint32_t gpie; + +/* setup GPIE for MSI-x mode */ + gpie =IXGBE_READ_REG (hw,IXGBE_GPIE ); + gpie |=IXGBE_GPIE_MSIX_MODE |IXGBE_GPIE_PBA_SUPPORT | +IXGBE_GPIE_OCD |IXGBE_GPIE_EIAME ; +/* +* auto clearing and auto setting corresponding bits in EIMS +* when MSI-X interrupt is triggered +*/ +if (hw->mac.type == ixgbe_mac_82598EB) +IXGBE_WRITE_REG (hw,IXGBE_EIAM ,IXGBE_EICS_RTX_QUEUE ); +else { +IXGBE_WRITE_REG (hw,IXGBE_EIAM_EX (0 ),0xFFFFFFFF ); +IXGBE_WRITE_REG (hw,IXGBE_EIAM_EX (1 ),0xFFFFFFFF ); + } +IXGBE_WRITE_REG (hw,IXGBE_GPIE , gpie); + +/* +* Populate the IVAR table and set the ITR values to the +* corresponding register. +*/ +for (queue_id =0 ; queue_id <VFIO_MAX_QUEUE_ID ; queue_id++) { +ixgbe_set_ivar_map (hw,0 , queue_id, queue_id); + intr_handle->vec_num[queue_id] = queue_id; + } + +switch (hw->mac.type) { +case ixgbe_mac_82598EB: +ixgbe_set_ivar_map (hw,-1 ,IXGBE_IVAR_OTHER_CAUSES_INDEX , +VFIO_MAX_QUEUE_ID ); +break ; +case ixgbe_mac_82599EB: +case ixgbe_mac_X540: +ixgbe_set_ivar_map (hw,-1 ,1 ,VFIO_MAX_QUEUE_ID ); +break ; +default : +break ; + } +IXGBE_WRITE_REG (hw,IXGBE_EITR (queue_id), +IXGBE_MIN_INTER_INTERRUPT_INTERVAL_DEFAULT &0xFFF ); + +/* set up to autoclear timer, and the vectors */ + mask =IXGBE_EIMS_ENABLE_MASK ; + mask &= ~(IXGBE_EIMS_OTHER | +IXGBE_EIMS_MAILBOX | +IXGBE_EIMS_LSC ); + +IXGBE_WRITE_REG (hw,IXGBE_EIAC , mask); +} +static int ixgbe_set_queue_rate_limit (struct rte_eth_dev *dev,uint16_t queue_idx,uint16_t tx_rate) {
lib/librte_pmd_ixgbe/ixgbe_ethdev.h +7 -0
@@ -109,6 +109,12 @@ ETH_RSS_IPV6_TCP_EX | \ ETH_RSS_IPV6_UDP_EX) +#define IXGBE_VF_IRQ_ENABLE_MASK 3/* vf irq enable mask */ +#define IXGBE_VF_MAXMSIVECTOR 1 +/* maximum other interrupts besides rx&tx*/ +#define IXGBE_MAX_OTHER_INTR 1 +#define IXGBEVF_MAX_OTHER_INTR 1 +/* * Information about the fdir mode. */
@@ -317,6 +323,7 @@ uint32_t ixgbe_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id);int ixgbe_dev_rx_descriptor_done (void *rx_queue,uint16_t offset); +int ixgbevf_dev_rx_descriptor_done (void *rx_queue,uint16_t offset);int ixgbe_dev_rx_init (struct rte_eth_dev *dev);
Signature · 1 line
1.8.1.4
PATCH v6 7/8 igb: enable rx queue interrupts for PF
In reply to a message not in the archiveFrom: "Zhou, Danny" <[email protected]>
The patch does below for igb PF:
- Setup NIC to generate MSI-X interrupts
- Set the IVAR register to map interrupt causes to vectors
- Implement interrupt enable/disable functions
---
v6 changes
- fill queue-vector mapping table
v5 changes
- Rebase the patchset onto the HEAD
v3 changes
- Remove unnecessary variables in e1000_mac_info
- Remove spinlok from PMD
v2 changes
- Consolidate review comments related to coding style
lib/librte_pmd_e1000/e1000_ethdev.h | 3 + lib/librte_pmd_e1000/igb_ethdev.c | 231 ++++++++++++++++++++++++++++++++---- 2 files changed, 209 insertions(+), 25 deletions(-)
lib/librte_pmd_e1000/e1000_ethdev.h +3 -0
@@ -108,6 +108,9 @@ ETH_RSS_IPV6_TCP_EX | \ ETH_RSS_IPV6_UDP_EX) +/* maximum number of other interrupts besides Rx & Tx interrupts */ +#define E1000_MAX_OTHER_INTR 1 +/* structure for interrupt relative data */ struct e1000_interrupt {uint32_t flags;
lib/librte_pmd_e1000/igb_ethdev.c +206 -25
299 lines not on this page · raw message
Signature · 1 line
1.8.1.4
A message that is not in the archive (sent off-list or not yet received): 1425012976-10173-8-git-send-email-cunming.liang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH v6 7/8] igb: enable rx queue interrupts for PF
Quoted 12 lines · On Fri, 27 Feb 2015 12:56:15 +0800 Cunming Liang…
/*
+ * It clears the interrupt causes and enables the interrupt.
+ * It will be called once only during nic initialized.
+ *
+ * @param dev
+ * Pointer to structrte_eth_dev.
+ *
+ * @return
+ * - On success, zero.
+ * - On failure, a negative value.
+ */
lib/librte_pmd_e1000/igb_ethdev.c eth_igb_lsc_interrupt_setup(struct rte_eth_dev *dev) · line 1851
Quoted 3 lines
lib/librte_pmd_e1000/igb_ethdev.c +3
-0+static int eth_igb_rxq_interrupt_setup (struct rte_eth_dev *dev) +{ +
This function should be void
It always succeeds and the caller just not check the return value.
If you did this in one driver, I bet other drivers have same problem.
Re: [PATCH v6 7/8] igb: enable rx queue interrupts for PF
Quoted 21 lines · Stephen Hemminger
Quoted 12 lines · On Fri, 27 Feb 2015 12:56:15 +0800 Cunming Liang <[email protected]>…
/*
+ * It clears the interrupt causes and enables the interrupt.
+ * It will be called once only during nic initialized.
+ *
+ * @param dev
+ * Pointer to structrte_eth_dev.
+ *
+ * @return
+ * - On success, zero.
+ * - On failure, a negative value.
+ */lib/librte_pmd_e1000/igb_ethdev.c eth_igb_lsc_interrupt_setup(struct rte_eth_dev *dev) · line 1851
Quoted 3 lines
lib/librte_pmd_e1000/igb_ethdev.c +3
-0+static int eth_igb_rxq_interrupt_setup (struct rte_eth_dev *dev) +{ +This function should be void
It always succeeds and the caller just not check the return value.
Quoted 1 line
If you did this in one driver, I bet other drivers have same problem.
[LCM] The previous reason probably to keep consistent with lsc_interrupt_setup. But I think it's reasonable to change to void. I'm considering another thing is that does is necessary to have condition rxq_interrupt_setup by intr_conf.rxq. As even without it, we can manually turn on rxq interrupt by API rte_eth_dev_rx_intr_enable.
PATCH v6 8/8 l3fwd-power: enable one-shot rx interrupt and polling/interrupt mode switch
In reply to a message not in the archiveFrom: "Zhou, Danny" <[email protected]>
Demonstrate how to handle per rx queue interrupt in a NAPI-like implementation in usersapce. PDK polling thread mainly works in polling mode and switch to interrupt mode only if there is no any packet received in recent polls.
Usersapce interrupt notification generally takes a lot more cycles than kernel, so one-shot interrupt is used here to guarantee minimum overhead and DPDK polling thread returns to polling mode immediately once it receives an interrupt notificaiton for incoming packet.
---
v6 changes
- Split event fd add and wait
v5 changes
- Change invoked function name and parameter to accomodate EAL change
v3 changes
- Add spinlock to ensure thread safe when accessing interrupt mask register
v2 changes
- Remove unused function which is for debug purpose
examples/l3fwd-power/main.c | 194 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 152 insertions(+), 42 deletions(-)
examples/l3fwd-power/main.c +152 -42
@@ -75,12 +75,14 @@#include <rte_string_fns.h> #include <rte_timer.h> #include <rte_power.h> +#include <rte_eal.h> +#include <rte_spinlock.h> #define RTE_LOGTYPE_L3FWD_POWER RTE_LOGTYPE_USER1#define MAX_PKT_BURST 32 -#define MIN_ZERO_POLL_COUNT 5 +#define MIN_ZERO_POLL_COUNT 10/* around 100ms at 2 Ghz */ #define TIMER_RESOLUTION_CYCLES 200000000ULL
@@ -156,6 +158,9 @@ static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT;/* ethernet addresses of ports */ static struct ether_addr ports_eth_addr[RTE_MAX_ETHPORTS ]; +/* ethernet addresses of ports */ +static rte_spinlock_t locks[RTE_MAX_ETHPORTS ]; +/* mask of enabled ports */ static uint32_t enabled_port_mask =0 ;/* Ports set in promiscuous mode off by default. */
@@ -188,6 +193,9 @@ struct lcore_rx_queue {#define MAX_TX_QUEUE_PER_PORT RTE_MAX_ETHPORTS#define MAX_RX_QUEUE_PER_PORT 128 +#define MAX_RX_QUEUE_INTERRUPT_PER_PORT 16 + +#define MAX_LCORE_PARAMS 1024struct lcore_params {uint8_t port_id;
@@ -214,7 +222,7 @@ static uint16_t nb_lcore_params = sizeof(lcore_params_array_default) /static struct rte_eth_conf port_conf = { .rxmode = { - .mq_mode =ETH_MQ_RX_RSS , + .mq_mode =ETH_MQ_RX_RSS , .max_rx_pkt_len =ETHER_MAX_LEN , .split_hdr_size =0 , .header_split =0 ,/**< Header Split disabled */
@@ -226,11 +234,14 @@ static struct rte_eth_conf port_conf = { .rx_adv_conf = { .rss_conf = { .rss_key = NULL, - .rss_hf = ETH_RSS_IP, + .rss_hf = ETH_RSS_UDP, }, }, .txmode = { - .mq_mode = ETH_DCB_NONE, + .mq_mode = ETH_MQ_TX_NONE, + }, + .intr_conf = { + .rxq = 1, /**< rxq interrupt feature enabled */ }, };
@@ -402,19 +413,22 @@ power_timer_cb(__attribute__((unused)) struct rte_timer *tim,/* accumulate total execution time in us when callback is invoked */ sleep_time_ratio = (float )(stats[lcore_id].sleep_time) / (float )SCALING_PERIOD ; -/** * check whether need to scale down frequency a step if it sleep a lot. */ -if (sleep_time_ratio >=SCALING_DOWN_TIME_RATIO_THRESHOLD ) -rte_power_freq_down (lcore_id); +if (sleep_time_ratio >=SCALING_DOWN_TIME_RATIO_THRESHOLD ) { +if (rte_power_freq_down) +rte_power_freq_down (lcore_id); + }else if ( (unsigned )(stats[lcore_id].nb_rx_processed / - stats[lcore_id].nb_iteration_looped) <MAX_PKT_BURST ) + stats[lcore_id].nb_iteration_looped) <MAX_PKT_BURST ) {/** * scale down a step if average packet per iteration less * than expectation. */ -rte_power_freq_down (lcore_id); +if (rte_power_freq_down) +rte_power_freq_down (lcore_id); + }/** * initialize another timer according to current frequency to ensure
@@ -707,22 +721,20 @@ l3fwd_simple_forward(struct rte_mbuf *m, uint8_t portid, } -#define SLEEP_GEAR1_THRESHOLD 100 -#define SLEEP_GEAR2_THRESHOLD 1000 +#define MINIMUM_SLEEP_TIME 1 +#define SUSPEND_THRESHOLD 300static inline uint32_t power_idle_heuristic (uint32_t zero_rx_packet_count) { -/* If zero count is less than 100, use it as the sleep time in us */ -if (zero_rx_packet_count <SLEEP_GEAR1_THRESHOLD ) -return zero_rx_packet_count; -/* If zero count is less than 1000, sleep time should be 100 us */ -else if ((zero_rx_packet_count >=SLEEP_GEAR1_THRESHOLD ) && - (zero_rx_packet_count <SLEEP_GEAR2_THRESHOLD )) -return SLEEP_GEAR1_THRESHOLD ; -/* If zero count is greater than 1000, sleep time should be 1000 us */ -else if (zero_rx_packet_count >=SLEEP_GEAR2_THRESHOLD ) -return SLEEP_GEAR2_THRESHOLD ; +/* If zero count is less than 100, sleep 1us */ +if (zero_rx_packet_count <SUSPEND_THRESHOLD ) +return MINIMUM_SLEEP_TIME ; +/* If zero count is less than 1000, sleep 100 us which is the +minimum latency switching from C3/C6 to C0 +*/ +else +return SUSPEND_THRESHOLD ;return 0 ; }
@@ -762,6 +774,72 @@ power_freq_scaleup_heuristic(unsigned lcore_id,return FREQ_CURRENT ; } +/** +* force polling thread sleep until one-shot rx interrupt triggers +* @param port_id +* Port id. +* @param queue_id +* Rx queue id. +* @return +* 0 on success +*/ +static int +sleep_until_rx_interrupt (uint8_t port_id,uint8_t queue_id) +{ +struct rte_eth_dev *eth_dev = &rte_eth_devices[port_id]; +struct rte_intr_handle *intr_handle = &(eth_dev->pci_dev->intr_handle); +uint32_t vec; + +/* Enable one-shot rx interrupt */ +rte_spinlock_lock (&(locks[port_id])); +rte_eth_dev_rx_intr_enable (port_id, queue_id); +rte_spinlock_unlock (&(locks[port_id])); + +RTE_LOG (INFO ,L3FWD_POWER , +"lcore %u sleeps until interrupt on port%d,rxq%d triggers\n" , +rte_lcore_id (), port_id, queue_id); + +rte_intr_rx_wait (intr_handle,RTE_EPOLL_FD_ANY , &vec,1 ); +RTE_LOG (INFO ,L3FWD_POWER , +"lcore %u is waked up from rx interrupt on port%d,rxq%d\n" , +rte_lcore_id (), port_id, queue_id); + +/* Disable one-shot rx interrupt */ +rte_spinlock_lock (&(locks[port_id])); +rte_eth_dev_rx_intr_disable (port_id, queue_id); +rte_spinlock_unlock (&(locks[port_id])); + +return 0 ; +} + +static int event_register (struct lcore_conf *qconf) +{ +struct rte_eth_dev *eth_dev; +struct rte_intr_handle *intr_handle; +struct lcore_rx_queue *rx_queue; +uint8_t portid, queueid; +uint32_t vec; +int ret; +int i; + +for (i =0 ; i < qconf->n_rx_queue; ++i) { + rx_queue = &(qconf->rx_queue_list[i]); + portid = rx_queue->port_id; + queueid = rx_queue->queue_id; + + eth_dev = &rte_eth_devices[portid]; + intr_handle = &(eth_dev->pci_dev->intr_handle); + +rte_eth_dev_rx_intr_vec_get (portid, queueid, &vec); + ret =rte_intr_rx_set (intr_handle,RTE_EPOLL_FD_ANY , +RTE_INTR_EVENT_ADD , vec); +if (ret <0 ) +return -1 ; + } + +return 0 ; +} +/* main processing loop */ static int main_loop (__attribute__((unused))void *dummy)
@@ -775,9 +853,9 @@ main_loop(__attribute__((unused)) void *dummy)struct lcore_conf *qconf;struct lcore_rx_queue *rx_queue;enum freq_scale_hint_t lcore_scaleup_hint; -uint32_t lcore_rx_idle_count =0 ;uint32_t lcore_idle_hint =0 ; +int intr_en =0 ;const uint64_t drain_tsc = (rte_get_tsc_hz () +US_PER_S -1 ) /US_PER_S *BURST_TX_DRAIN_US ;
@@ -794,13 +872,18 @@ main_loop(__attribute__((unused)) void *dummy)RTE_LOG (INFO ,L3FWD_POWER ,"entering main loop on lcore %u\n" , lcore_id);for (i =0 ; i < qconf->n_rx_queue; i++) { - portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id;RTE_LOG (INFO ,L3FWD_POWER ," -- lcoreid=%u portid=%hhu " "rxqueueid=%hhu\n" , lcore_id, portid, queueid); } +/* add into event wait list */ +if (event_register (qconf) ==0 ) + intr_en =1 ; +else +RTE_LOG (INFO ,L3FWD_POWER ,"RX interrupt won't enable.\n" ); +while (1 ) { stats[lcore_id].nb_iteration_looped++;
@@ -835,6 +918,7 @@ main_loop(__attribute__((unused)) void *dummy) prev_tsc_power = cur_tsc_power; } +start_rx:/* * Read packet from RX queues */
@@ -848,6 +932,7 @@ main_loop(__attribute__((unused)) void *dummy) nb_rx =rte_eth_rx_burst (portid, queueid, pkts_burst,MAX_PKT_BURST ); + stats[lcore_id].nb_rx_processed += nb_rx;if (unlikely (nb_rx ==0 )) {/**
@@ -910,10 +995,13 @@ main_loop(__attribute__((unused)) void *dummy) rx_queue->freq_up_hint; } -if (lcore_scaleup_hint ==FREQ_HIGHEST ) -rte_power_freq_max (lcore_id); -else if (lcore_scaleup_hint ==FREQ_HIGHER ) -rte_power_freq_up (lcore_id); +if (lcore_scaleup_hint ==FREQ_HIGHEST ) { +if (rte_power_freq_max) +rte_power_freq_max (lcore_id); + }else if (lcore_scaleup_hint ==FREQ_HIGHER ) { +if (rte_power_freq_up) +rte_power_freq_up (lcore_id); + } }else {/** * All Rx queues empty in recent consecutive polls,
@@ -928,16 +1016,23 @@ main_loop(__attribute__((unused)) void *dummy) lcore_idle_hint = rx_queue->idle_hint; } -if ( lcore_idle_hint <SLEEP_GEAR1_THRESHOLD ) +if (lcore_idle_hint <SUSPEND_THRESHOLD )/** -* execute "pause" instruction to avoid context -* switch for short sleep. -*/ +* execute "pause" instruction to avoid context +* switch which generally take hundres of +* microsecond for short sleep. +*/ rte_delay_us (lcore_idle_hint); -else -/* long sleep force runing thread to suspend */ -usleep (lcore_idle_hint); - +else { +/* suspend untill rx interrupt trigges */ +if (intr_en) +sleep_until_rx_interrupt ( + qconf->rx_queue_list[0 ].port_id, + qconf->rx_queue_list[0 ].queue_id + ); +/* start receiving packets immediately */ + goto start_rx; + } stats[lcore_id].sleep_time += lcore_idle_hint; } }
@@ -1270,7 +1365,7 @@ setup_hash(int socketid)char s[64 ];/* create ipv4 hash */ -snprintf (s,sizeof (s),"ipv4_l3fwd_hash_%d" , socketid); +rte_snprintf (s,sizeof (s),"ipv4_l3fwd_hash_%d" , socketid); ipv4_l3fwd_hash_params.name = s; ipv4_l3fwd_hash_params.socket_id = socketid; ipv4_l3fwd_lookup_struct[socketid] =
@@ -1280,7 +1375,7 @@ setup_hash(int socketid) "socket %d\n", socketid);/* create ipv6 hash */ -snprintf (s,sizeof (s),"ipv6_l3fwd_hash_%d" , socketid); +rte_snprintf (s,sizeof (s),"ipv6_l3fwd_hash_%d" , socketid); ipv6_l3fwd_hash_params.name = s; ipv6_l3fwd_hash_params.socket_id = socketid; ipv6_l3fwd_lookup_struct[socketid] =
@@ -1476,6 +1571,7 @@ main(int argc, char **argv)unsigned lcore_id;uint64_t hz;uint32_t n_tx_queue, nb_lcores; +uint32_t dev_rxq_num, dev_txq_num;uint8_t portid, nb_rx_queue, queue, socketid;/* catch SIGINT and restore cpufreq governor to ondemand */
@@ -1525,10 +1621,19 @@ main(int argc, char **argv)printf ("Initializing port %d ... " , portid );fflush (stdout); +rte_eth_dev_info_get (portid, &dev_info); + dev_rxq_num = dev_info.max_rx_queues; + dev_txq_num = dev_info.max_tx_queues; + nb_rx_queue =get_port_n_rx_queues (portid); +if (nb_rx_queue > dev_rxq_num) +rte_exit (EXIT_FAILURE , +"Cannot configure not existed rxq: " +"port=%d\n" , portid); + n_tx_queue = nb_lcores; -if (n_tx_queue >MAX_TX_QUEUE_PER_PORT ) - n_tx_queue =MAX_TX_QUEUE_PER_PORT ; +if (n_tx_queue > dev_txq_num) + n_tx_queue = dev_txq_num;printf ("Creating queues: nb_rxq=%d nb_txq=%u... " , nb_rx_queue, (unsigned )n_tx_queue ); ret = rte_eth_dev_configure(portid, nb_rx_queue,
@@ -1552,6 +1657,9 @@ main(int argc, char **argv)if (rte_lcore_is_enabled (lcore_id) ==0 )continue ; +if (queueid >= dev_txq_num) +continue ; +if (numa_on) socketid = \ (uint8_t )rte_lcore_to_socket_id (lcore_id);
@@ -1586,8 +1694,9 @@ main(int argc, char **argv)/* init power management library */ ret =rte_power_init (lcore_id);if (ret) -rte_exit (EXIT_FAILURE ,"Power management library " -"initialization failed on core%u\n" , lcore_id); +rte_log (RTE_LOG_ERR ,RTE_LOGTYPE_POWER , +"Power management library initialization " +"failed on core%u" , lcore_id);/* init timer structures for each enabled lcore */ rte_timer_init (&power_timers[lcore_id]);
@@ -1635,7 +1744,6 @@ main(int argc, char **argv)if (ret <0 )rte_exit (EXIT_FAILURE ,"rte_eth_dev_start: err=%d, " "port=%d\n" , ret, portid); -/* * If enabled, put device in promiscuous mode. * This allows IO forwarding mode to forward packets
@@ -1644,6 +1752,8 @@ main(int argc, char **argv)*/ if (promiscuous_on)rte_eth_promiscuous_enable (portid); +/* initialize spinlock for each port */ +rte_spinlock_init (&(locks[portid])); }check_all_ports_link_status ((uint8_t )nb_ports, enabled_port_mask);
Signature · 1 line
1.8.1.4
A message that is not in the archive (sent off-list or not yet received): 1425012976-10173-9-git-send-email-cunming.liang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH v6 8/8] l3fwd-power: enable one-shot rx interrupt and polling/interrupt mode switch
examples/l3fwd-power/main.c power_freq_scaleup_heuristic(unsigned lcore_id, · line 797
Quoted 5 lines · On Fri, 27 Feb 2015 12:56:16 +0800 Cunming Liang…
examples/l3fwd-power/main.c +5
-0+/* Enable one-shot rx interrupt */ +rte_spinlock_lock (&(locks[port_id])); +rte_eth_dev_rx_intr_enable (port_id, queue_id); +rte_spinlock_unlock (&(locks[port_id])); +
If always requires locks like this, then the API should do the locking internally.
Re: [PATCH v6 8/8] l3fwd-power: enable one-shot rx interrupt and polling/interrupt mode switch
In reply to a message not in the archiveexamples/l3fwd-power/main.c static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; · line 162
Quoted 2 lines · On Fri, 27 Feb 2015 12:56:16 +0800 Cunming Liang…
examples/l3fwd-power/main.c +2
-0+/* ethernet addresses of ports */ +static rte_spinlock_t locks[RTE_MAX_ETHPORTS ];
Comment is incorrect this is a lock array not an address array.
examples/l3fwd-power/main.c static uint16_t nb_lcore_params = sizeof(lcore_params_array_default) / · line 225
Quoted 4 lines
static struct
rte_eth_confport_conf= {examples/l3fwd-power/main.c +1
-1.rxmode = { - .mq_mode = ETH_MQ_RX_RSS, + .mq_mode = ETH_MQ_RX_RSS,
Please don't mix white space changes with code changes
Liu, Yong
Tested-by- Tested Commit: 00c685634b8a43e4594e26949a6c4f1cf5b67047
- OS: Fedora20 3.15.8-200.fc20.x86_64
- GCC: gcc version 4.8.3 20140911 (Red Hat 4.8.3-7) (GCC)
- CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
- NIC: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection
- Default x86_64-native-linuxapp-gcc configuration
- Total 4 cases, 4 passed, 0 failed
- Case: interrupt pmd on PF with single queue
Description: Check interrupt pmd work with single queue Command / instruction:
Bind ports to vfio-pci.
modprobe vfio
modprobe vfio-pci
./tools/dpdk_nic_bind.py --bind=vfio-pci 0000:08:00.0 0000:08:00.1
Start l3fwd-power with one queue per port.
l3fwd-power -c 7 -n 4 -- -p 0x3 -P --config="(0,0,1),(1,0,2)"
Send one packet to Port0 and Port1, check that thread on core1 and core2 waked up.
L3FWD_POWER: lcore 1 is waked up from rx interrupt on port1,rxq0
L3FWD_POWER: lcore 2 is waked up from rx interrupt on port1,rxq0
Expected test result:
l3fwd-power can forward packets normally and thread on core1 and core2 will sleep when there's no packet received.
- Case: interrupt pmd on PF with multi queue
Description: Check interrupt pmd work with multiple queues Command / instruction:
Start l3fwd-power with two queues per port.
l3fwd-power -c 1f -n 4 -- -p 0x3 \
--config="(0,0,1),(0,1,2)(1,0,3),(1,1,4)"
Send packet with increased dest IP to Port0 and Port1, check that thread on core1,core2,core3,core4 waked up.
L3FWD_POWER: lcore 1 is waked up from rx interrupt on port1,rxq0
L3FWD_POWER: lcore 2 is waked up from rx interrupt on port1,rxq1
L3FWD_POWER: lcore 3 is waked up from rx interrupt on port1,rxq0
L3FWD_POWER: lcore 4 is waked up from rx interrupt on port1,rxq1
Expected test result:
l3fwd-power can forward packets normally and thread on core1-core4 will
sleep when there's no packet received.
- Case: interrupt pmd on PF with max Rx queues
Description: Check interrupt pmd work with maximum queues Command / instruction:
Start l3fwd-power with 32 queues per port.
l3fwd-power -c ffffffff -n 4 -- -p 0x3 -P --config="(0,0,0),(0,1,1),\
(0,2,2),(0,3,3),(0,4,4),(0,5,5),(0,6,6),(0,7,7),(0,8,8),
(0,9,9),(0,10,10),(0,11,11),(0,12,12),(0,13,13),(0,14,14),\
(0,15,15),\
(1,0,16),(1,1,17),(1,2,18),(1,3,19),(1,4,20),(1,5,21),(1,6,22),\
(1,7,23),(1,8,24),(1,9,25),(1,10,26),(1,11,27),(1,12,28),\
(1,13,29),(1,14,30),\(1,15,31)"
Send packet with increased dest IP to Port0 and Port1, check that all
threads waked up. Expected test result:
l3fwd-power can forward packets normally and thread on core1-core31 will sleep when there's no packet received.
- Case: interrupt pmd on VF with single queue
Description: Check interrupt pmd work on VF device
Command / instruction:
Bind ports to back to ixgbe driver.
./tools/dpdk_nic_bind.py --bind=ixgbe 0000:08:00.0 0000:08:00.1
Create one VF per Port in host and make sure PF interface up
echo 1 > /sys/bus/pci/devices/0000\:08\:00.0/sriov_numvfs
echo 1 > /sys/bus/pci/devices/0000\:08\:00.1/sriov_numvfs ifconfig p786p1 up
ifconfig p786p2 up
Bind VF device to vfio-pci.
./tools/dpdk_nic_bind.py --bind=vfio-pci 0000:08:10.0 0000:08:10.1
Start l3fwd-power on host with one queue per port.
l3fwd-power -c 1f -n 4 -- -p 0x3 -P --config="(0,0,1),(1,0,2)"
Send one packet to Port0 and Port1, check that thread on core1 and core2 waked up.
L3FWD_POWER: lcore 1 is waked up from rx interrupt on port1,rxq0
L3FWD_POWER: lcore 2 is waked up from rx interrupt on port1,rxq0
Expected test result:
l3fwd-power can forward packets normally on VF and thread on core1 and core2 will sleep when there's no packet received.
Quoted 111 lines
-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Cunming Liang
Sent: Friday, February 27, 2015 12:56 PM
To: [email protected]
Subject: [dpdk-dev] [PATCH v6 0/8] Interrupt mode PMDv6 changes
- splitrte_intr_wait_rx_pktinto two APIs 'wait' and 'set'.
- rewrite rte_intr_rx_wait/rte_intr_rx_set.
- using vector number instead ofqueue_idas interrupt API params.
- patch reorder and split.v5 changes
- Rebase the patchset onto the HEAD
- Isolate ethdev from EAL for new-added wait-for-rx interrupt function
- Export wait-for-rx interrupt function for shared libraries
- Split-off a new patch file for changed structrte_intr_handlethat other patches depend on, to avoid breaking git bisect
- Change sample applicaiton to accomodate EAL function spec change accordinglyv4 changes
- Export interrupt enable/disable functions for shared libraries
- Adjust position of new-added structure fields and functions to avoid breaking ABIv3 changes
- Add return value for interrupt enable/disable functions
- Move spinlok from PMD to L3fwd-power
- Remove unnecessary variables ine1000_mac_info
- Fix miscelleous review commentsv2 changes
- Fix compilation issue inMakefilefor missed header file.
- Consolidate internal and community review comments of v1 patch set.The patch series introduce low-latency one-shot rx interrupt into DPDK with
polling and interrupt mode switch control example.DPDK userspace interrupt notification and handling mechanism is based on UIO
with below limitation:
1) It is designed to handle LSC interrupt only with inefficient suspended pthread wakeup procedure (e.g. UIO wakes up LSC interrupt handling
thread
which then wakes up DPDK polling thread). In this way, it introduces non-deterministic wakeup latency for DPDK polling thread as well as
packet
latency if it is used to handle Rx interrupt.
2) UIO only supports a single interrupt vector which has to been shared by LSC interrupt and interrupts assigned to dedicated rx queues.This patchset includes below features:
1) Enable one-shot rx queue interrupt in ixgbePMD(PF & VF)and igb PMD(PF only).
2) Build on top of the VFIO mechanism instead of UIO, so it could support up to 64 interrupt vectors for rx queue interrupts.
3) Have 1 DPDK polling thread handle per Rx queue interrupt with a dedicated
VFIO eventfd, which eliminates non-deterministic pthread wakeup latency
in
user space.
4) Demonstrate interrupts control APIs and userspace NAIP-like polling/interrupt
switch algorithms in L3fwd-power example.Known limitations:
1) It does not work for UIO due to a single interrupt eventfd shared by LSC
and rx queue interrupt handlers causes a mess.
2) LSC interrupt is not supported by VF driver, so it is by default disabled
in L3fwd-power now. Feel free to turn in on if you want to support both
LSC
and rx queue interrupts on a PF.Cunming Liang (5):
eal: declare new interrupt api
eal/linux: add rx queue interrupt FDs to intr handle struct eal/bsd: dummy for new intr definition
eal/linux: add per rx queue interrupt handling based on VFIO
ethdev: add rx interrupt enable/disable functionsZhou, Danny (3):
ixgbe: enable rx queue interrupts for both PF and VF
igb: enable rx queue interrupts for PF
l3fwd-power: enable one-shot rx interrupt and polling/interrupt mode
switchexamples/l3fwd-power/main.c | 194 ++++++++--- lib/librte_eal/bsdapp/eal/eal_interrupts.c | 15 + .../bsdapp/eal/include/exec-env/rte_interrupts.h | 4 + lib/librte_eal/bsdapp/eal/rte_eal_version.map | 2 + lib/librte_eal/common/include/rte_interrupts.h | 38 +++ lib/librte_eal/linuxapp/eal/eal_interrupts.c | 224 +++++++++--- lib/librte_eal/linuxapp/eal/eal_pci_vfio.c | 23 +- .../linuxapp/eal/include/exec-env/rte_interrupts.h | 9 + lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + lib/librte_ether/rte_ethdev.c | 66 ++++ lib/librte_ether/rte_ethdev.h | 77 +++++ lib/librte_ether/rte_ether_version.map | 3 + lib/librte_pmd_e1000/e1000_ethdev.h | 3 + lib/librte_pmd_e1000/igb_ethdev.c | 231 +++++++++++-- lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 377 ++++++++++++++++++++- lib/librte_pmd_ixgbe/ixgbe_ethdev.h | 7 + 16 files changed, 1156 insertions(+), 119 deletions(-)--
1.8.1.4
Quoted 7 lines · Cunming Liang
v6 changes
- splitrte_intr_wait_rx_pktinto two APIs 'wait' and 'set'.
- rewrite rte_intr_rx_wait/rte_intr_rx_set.
- using vector number instead ofqueue_idas interrupt API params.
- patch reorder and split.
Ok, so after looking at this patchset, I would say this is the right direction, but still this is too limited.
The ethdev part and the vfio eventfds part look acceptable to me. But thinking about it, I could just reuse a standard event library with the eventfds I would get from ethdev without a need for a new eal api.
Signature · 1 line
David Marchand
A message that is not in the archive (sent off-list or not yet received): CALwxeUuryZ6AvvC7PDHVVGUm9bOoR74jTf9P5H1UJy87GTADkQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Quoted 11 lines · On Fri, 27 Feb 2015 11:38:25 +0100 David Marchand…
On Fri, Feb 27, 2015 at 5:56 AM, Cunming Liang <[email protected]> wrote:
v6 changes
- splitrte_intr_wait_rx_pktinto two APIs 'wait' and 'set'.
- rewrite rte_intr_rx_wait/rte_intr_rx_set.
- using vector number instead ofqueue_idas interrupt API params.
- patch reorder and split.Ok, so after looking at this patchset, I would say this is the right direction, but still this is too limited.
The ethdev part and the vfio eventfds part look acceptable to me.
But thinking about it, I could just reuse a standard event library with the eventfds I would get from ethdev without a need for a new eal api.
Also, you need to introduce a flag (in pci drv_flags?) so that application can know if poll mode interrupt will work or not on the given device before configuring it.
Quoted 3 lines · On Fri, 27 Feb 2015 11:38:25 +0100 David Marchand…
Ok, so after looking at this patchset, I would say this is the right direction, but still this is too limited.
The ethdev part and the vfio eventfds part look acceptable to me.
But thinking about it, I could just reuse a standard event library with the eventfds I would get from ethdev without a need for a new eal api.
I would prefer that there was just an fd and a callback.
An application should be able to use what ever event model or library it wants.
IMHO the existing interrupt thread model is incorrectly designed and creates lots of opportunities for races because of that. Look at the effort it has to use to pass the event back to link state code.
Hi Stephen,
Quoted 8 lines · Stephen Hemminger
Quoted 3 lines · On Fri, 27 Feb 2015 11:38:25 +0100 David Marchand…
Ok, so after looking at this patchset, I would say this is the right direction, but still this is too limited.
The ethdev part and the vfio eventfds part look acceptable to me.
But thinking about it, I could just reuse a standard event library with the eventfds I would get from ethdev without a need for a new eal api.I would prefer that there was just an fd and a callback.
An application should be able to use what ever event model or library it wants.
[LCM] I agree, on application perspective it is.
As it's easy to get RX/TX interrupt fd, there's no limit for application to do all the things with the 3rd party event library. The improvement probably be 1) a rte_intr_vec_to_fd() API; 2) expose eal_intr_process_rxtx_interrupts() as a public API for RX/TX interrupt callback. However, it should allow to use the packet interrupt feature in case application don't choose any 3rd party event library. That's the motivation to give a very lightweight 'wait' EAL API. Sounds reasonable ?
Quoted 4 lines
IMHO the existing interrupt thread model is incorrectly designed and creates lots of opportunities for races because of that. Look at the effort it has to use to pass the event back to link state code.
From: Cunming Liang <[email protected]>
In reply to a message not in the archivev7 changes
- decouple epoll event and intr operation
- add condition check in the case intr vector is disabled
- renaming some APIs
v6 changes
- split rte_intr_wait_rx_pkt into two APIs 'wait' and 'set'.
- rewrite rte_intr_rx_wait/rte_intr_rx_set.
- using vector number instead of queue_id as interrupt API params.
- patch reorder and split.
v5 changes
- Rebase the patchset onto the HEAD
- Isolate ethdev from EAL for new-added wait-for-rx interrupt function
- Export wait-for-rx interrupt function for shared libraries
- Split-off a new patch file for changed struct rte_intr_handle that other patches depend on, to avoid breaking git bisect
- Change sample applicaiton to accomodate EAL function spec change accordingly
v4 changes
- Export interrupt enable/disable functions for shared libraries
- Adjust position of new-added structure fields and functions to avoid breaking ABI
v3 changes
- Add return value for interrupt enable/disable functions
- Move spinlok from PMD to L3fwd-power
- Remove unnecessary variables in e1000_mac_info
- Fix miscelleous review comments
v2 changes
- Fix compilation issue in Makefile for missed header file.
- Consolidate internal and community review comments of v1 patch set.
The patch series introduce low-latency one-shot rx interrupt into DPDK with polling and interrupt mode switch control example.
DPDK userspace interrupt notification and handling mechanism is based on UIO with below limitation:
1) It is designed to handle LSC interrupt only with inefficient suspended pthread wakeup procedure (e.g. UIO wakes up LSC interrupt handling thread which then wakes up DPDK polling thread). In this way, it introduces non-deterministic wakeup latency for DPDK polling thread as well as packet latency if it is used to handle Rx interrupt.
2) UIO only supports a single interrupt vector which has to been shared by LSC interrupt and interrupts assigned to dedicated rx queues.
This patchset includes below features:
1) Enable one-shot rx queue interrupt in ixgbe PMD(PF & VF) and igb PMD(PF only).
2) Build on top of the VFIO mechanism instead of UIO, so it could support up to 64 interrupt vectors for rx queue interrupts.
3) Have 1 DPDK polling thread handle per Rx queue interrupt with a dedicated VFIO eventfd, which eliminates non-deterministic pthread wakeup latency in user space.
4) Demonstrate interrupts control APIs and userspace NAIP-like polling/interrupt switch algorithms in L3fwd-power example.
Known limitations:
1) It does not work for UIO due to a single interrupt eventfd shared by LSC and rx queue interrupt handlers causes a mess.
2) LSC interrupt is not supported by VF driver, so it is by default disabled in L3fwd-power now. Feel free to turn in on if you want to support both LSC and rx queue interrupts on a PF.
Cunming Liang (10):
eal/linux: add interrupt vectors support in intr_handle eal/linux: add rte_epoll_wait/ctl support
eal/linux: add API to set rx interrupt event monitor eal/bsd: dummy for new intr definition
eal/linux: fix comments typo on vfio msi
eal/linux: add interrupt vectors handling on VFIO
ethdev: add rx intr enable, disable and ctl functions
ixgbe: enable rx queue interrupts for both PF and VF
igb: enable rx queue interrupts for PF
l3fwd-power: enable one-shot rx interrupt and polling/interrupt mode
switch
examples/l3fwd-power/main.c | 206 ++++++++-- .../bsdapp/eal/include/exec-env/rte_interrupts.h | 6 + lib/librte_eal/linuxapp/eal/eal_interrupts.c | 232 +++++++++-- lib/librte_eal/linuxapp/eal/eal_pci_vfio.c | 12 + .../linuxapp/eal/include/exec-env/rte_interrupts.h | 97 +++++ lib/librte_eal/linuxapp/eal/rte_eal_version.map | 4 + lib/librte_ether/rte_ethdev.c | 132 +++++++ lib/librte_ether/rte_ethdev.h | 104 +++++ lib/librte_ether/rte_ether_version.map | 4 + lib/librte_pmd_e1000/e1000_ethdev.h | 3 + lib/librte_pmd_e1000/igb_ethdev.c | 256 +++++++++++-- lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 425 ++++++++++++++++++++- lib/librte_pmd_ixgbe/ixgbe_ethdev.h | 7 + 13 files changed, 1394 insertions(+), 94 deletions(-)
Signature · 1 line
1.8.1.4
A message that is not in the archive (sent off-list or not yet received): 1430804386-28949-1-git-send-email-cunming.liang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
PATCH v7 1/10 eal/linux: add interrupt vectors support in intr_handle
The patch adds interrupt vectors support in rte_intr_handle.
'vec_en' is set when interrupt vectors are detected and associated event fds are set. Those event fds are stored in efds[].
'intr_vec' is reserved for device driver to initialize the vector mapping table.
When the event fds add to a specified epoll instance, 'eptrs' will hold the rte_epoll_event object pointer.
---
v7 changes:
- add eptrs[], it's used to store the register rte_epoll_event instances.
- add vec_en, to log the vector capability status.
v6 changes:
- add mapping table between irq vector number and queue id.
v5 changes:
- Create this new patch file for changed struct rte_intr_handle that other patches depend on, to avoid breaking git bisect.
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h | 10 ++++++++++ 1 file changed, 10 insertions(+)
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +10 -0
@@ -38,6 +38,8 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define RTE_MAX_RXTX_INTR_VEC_ID 32 +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */
@@ -48,6 +50,8 @@ enum rte_intr_handle_type {RTE_INTR_HANDLE_MAX }; +struct rte_epoll_event ; +/** Handle for interrupts. */ struct rte_intr_handle { union {
@@ -57,6 +61,12 @@ struct rte_intr_handle { };int fd;/**< interrupt event file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +int max_intr;/**< max interrupt requested */ +int vec_en;/**< intr vectors enabled */ +int efds[RTE_MAX_RXTX_INTR_VEC_ID ];/**< intr vectors/efds mapping */ +struct rte_epoll_event *eptrs[RTE_MAX_RXTX_INTR_VEC_ID ]; +/**< intr vector epoll event ptr */ +int *intr_vec;/**< intr vector number array */ };#endif /* _RTE_LINUXAPP_INTERRUPTS_H_ */
Signature · 1 line
1.8.1.4
PATCH v7 2/10 eal/linux: add rte_epoll_wait/ctl support
In reply to a message not in the archiveThe patch adds 'rte_epoll_wait' and 'rte_epoll_ctl' for async event wakeup.
It defines 'struct rte_epoll_event' as the event param.
The 'op' uses the same enum as epoll_wait/ctl does.
The epoll event support to carry a raw user data and to register a callback which is exectuted during wakeup.
---
v7 changes
- split v6[4/8] into two patches, one for epoll event(this one) another for rx intr(next patch)
- introduce rte_epoll_event definition
- rte_epoll_wait/ctl for more generic RTE epoll API
v6 changes
- split rte_intr_wait_rx_pkt into two function, wait and set.
- rewrite rte_intr_rx_wait/rte_intr_rx_set to remove queue visibility on eal.
- rte_intr_rx_wait to support multiplexing.
- allow epfd as input to support flexible event fd combination.
lib/librte_eal/linuxapp/eal/eal_interrupts.c | 97 ++++++++++++++++++++++ .../linuxapp/eal/include/exec-env/rte_interrupts.h | 66 ++++++++++++++- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 3 + 3 files changed, 165 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +97 -0
@@ -69,6 +69,8 @@#define EAL_INTR_EPOLL_WAIT_FOREVER (-1) +static RTE_DEFINE_PER_LCORE(int, _epfd) = -1; /**< epoll fd per thread */ +/** * union for pipe fds. */
@@ -859,3 +861,98 @@ rte_eal_intr_init(void)return -ret; } +static void +eal_epoll_process_event (struct epoll_event *evs,int n, +struct rte_epoll_event *events) +{ +int i; +struct rte_epoll_event *rev; +for (i =0 ; i < n; i++) { + rev = (struct rte_epoll_event *)evs[i].data.ptr; +if (rev) { + events[i].fd = rev->fd; + events[i].event = rev->event; + events[i].data = rev->data; +if (rev->cb_fun) + rev->cb_fun (rev->fd, rev->cb_arg); + } + } +} + +static inline int +eal_init_tls_epfd (void ) +{ +int pfd =epoll_create (255 ); +if (pfd <0 ) { +RTE_LOG (ERR ,EAL , +"Cannot create epoll instance\n" ); +return -1 ; + } +return pfd; +} + +int +rte_intr_tls_epfd (void ) +{ +if (RTE_PER_LCORE (_epfd) ==-1 ) +RTE_PER_LCORE (_epfd) =eal_init_tls_epfd (); + +return RTE_PER_LCORE (_epfd); +} + +int +rte_epoll_wait (int epfd,struct rte_epoll_event *events, +int maxevents,int timeout) +{ +struct epoll_event evs[maxevents]; +int rc; + +if (!events) { +RTE_LOG (ERR ,EAL ,"rte_epoll_event can't be NULL\n" ); +return -1 ; + } + +/* using per thread epoll fd */ +if (epfd ==RTE_EPOLL_PER_THREAD ) + epfd =rte_intr_tls_epfd (); + + rc =epoll_wait (epfd, evs, maxevents, timeout); +if (likely (rc >0 )) +/* epoll_wait has at least one fd ready to read */ +eal_epoll_process_event (evs, rc, events); +else if (rc <0 ) { +/* epoll_wait fail */ +RTE_LOG (ERR ,EAL ,"epoll_wait returns with fail %s\n" , +strerror (errno)); + rc =-1 ; + } + +return rc; +} + +int +rte_epoll_ctl (int epfd,int op,int fd, +struct rte_epoll_event *event) +{ +struct epoll_event ev; + +if (!event) { +RTE_LOG (ERR ,EAL ,"rte_epoll_event can't be NULL\n" ); +return -1 ; + } + +/* using per thread epoll fd */ +if (epfd ==RTE_EPOLL_PER_THREAD ) + epfd =rte_intr_tls_epfd (); + + event->fd = fd;/* ignore fd in rev */ + ev.data.ptr = (void *)event; + ev.events = event->event; +if (epoll_ctl (epfd, op, fd, &ev) <0 ) { +RTE_LOG (ERR ,EAL ,"Error op %d fd %d epoll_ctl, %s\n" , + op, fd,strerror (errno)); +return -1 ; + } + +return 0 ; +}
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +65 -1
76 lines not on this page · raw message
lib/librte_eal/linuxapp/eal/rte_eal_version.map +3 -0
@@ -52,6 +52,8 @@ DPDK_2.0 { rte_eal_vdev_init; rte_eal_vdev_uninit; rte_eal_wait_lcore; + rte_epoll_ctl; + rte_epoll_wait; rte_exit; rte_get_hpet_cycles; rte_get_hpet_hz;
@@ -61,6 +63,7 @@ DPDK_2.0 { rte_intr_callback_unregister; rte_intr_disable; rte_intr_enable; + rte_intr_tls_epfd; rte_log; rte_log_add_in_history; rte_log_cur_msg_loglevel;
Signature · 1 line
1.8.1.4
Re: [PATCH v7 02/10] eal/linux: add rte_epoll_wait/ctl support
lib/librte_eal/linuxapp/eal/eal_interrupts.c rte_eal_intr_init(void) · line 926
Quoted 4 lines · On Tue, 5 May 2015 13:39:38 +0800 Cunming Liang <[email protected]>…
lib/librte_eal/linuxapp/eal/eal_interrupts.c +4
-0+else if (rc <0 ) { +/* epoll_wait fail */ +RTE_LOG (ERR ,EAL ,"epoll_wait returns with fail %s\n" , +strerror (errno));
In real application there maybe other random signals.
Therefore the code should ignore and return for case of EWOULDBLOCK and EINTR
Re: [PATCH v7 02/10] eal/linux: add rte_epoll_wait/ctl support
Quoted 9 lines · Stephen Hemminger
lib/librte_eal/linuxapp/eal/eal_interrupts.c rte_eal_intr_init(void) · line 926
Quoted 4 lines · On Tue, 5 May 2015 13:39:38 +0800 Cunming Liang <[email protected]>…
lib/librte_eal/linuxapp/eal/eal_interrupts.c +4
-0+else if (rc <0 ) { +/* epoll_wait fail */ +RTE_LOG (ERR ,EAL ,"epoll_wait returns with fail %s\n" , +strerror (errno));In real application there maybe other random signals.
Therefore the code should ignore and return for case of EWOULDBLOCK and EINTR
[LCM] Thanks, you're right, when EINTR happens, shall continue epoll_wait instead of return. Per EWOULDBLOCK, seems epoll_wait won't return it, so I assume your mention is about epoll event read.
PATCH v7 3/10 eal/linux: add API to set rx interrupt event monitor
In reply to a message not in the archiveThe patch adds 'rte_intr_rx_ctl' to add or delete interrupt vector events monitor on specified epoll instance.
---
v7 changes
- rename rte_intr_rx_set to rte_intr_rx_ctl.
- rte_intr_rx_ctl uses rte_epoll_ctl to register epoll event instance.
- the intr rx event instance includes a intr process callback.
v6 changes
- split rte_intr_wait_rx_pkt into two function, wait and set.
- rewrite rte_intr_rx_wait/rte_intr_rx_set to remove queue visibility on eal.
- rte_intr_rx_wait to support multiplexing.
- allow epfd as input to support flexible event fd combination.
lib/librte_eal/linuxapp/eal/eal_interrupts.c | 95 ++++++++++++++++++++++ .../linuxapp/eal/include/exec-env/rte_interrupts.h | 23 ++++++ lib/librte_eal/linuxapp/eal/rte_eal_version.map | 1 + 3 files changed, 119 insertions(+)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +95 -0
@@ -862,6 +862,35 @@ rte_eal_intr_init(void) }static void +eal_intr_proc_rxtx_intr (int fd,struct rte_intr_handle *intr_handle) +{ +union rte_intr_read_buffer buf; +int bytes_read =1 ; + +if (intr_handle->type !=RTE_INTR_HANDLE_VFIO_MSIX ) { +RTE_LOG (ERR ,EAL ,"intr type should be VFIO_MSIX\n" ); +return ; + } + +#ifdef VFIO_PRESENT + bytes_read =sizeof (buf.vfio_intr_count); +#endif + +/** +* read out to clear the ready-to-be-read flag +* for epoll_wait. +*/ + bytes_read =read (fd, &buf, bytes_read); +if (bytes_read <0 ) +RTE_LOG (ERR ,EAL ,"Error reading from file " +"descriptor %d: %s\n" , fd, +strerror (errno)); +else if (bytes_read ==0 ) +RTE_LOG (ERR ,EAL ,"Read nothing from file " +"descriptor %d\n" , fd); +} + +static void eal_epoll_process_event (struct epoll_event *evs,int n,struct rte_epoll_event *events) {
@@ -956,3 +985,69 @@ rte_epoll_ctl(int epfd, int op, int fd,return 0 ; } + +int +rte_intr_rx_ctl (struct rte_intr_handle *intr_handle,int epfd, +int op,unsigned int vec,void *data,int socket) +{ +struct rte_epoll_event *rev; +int epfd_op; +int rc =0 ; + +if (!intr_handle || vec >=RTE_MAX_RXTX_INTR_VEC_ID || + !intr_handle->vec_en) { +RTE_LOG (ERR ,EAL ,"Wrong intr vector number.\n" ); +return -1 ; + } + +if (socket ==SOCKET_ID_ANY ) + socket =rte_socket_id (); + +switch (op) { +case RTE_INTR_EVENT_ADD : + epfd_op =EPOLL_CTL_ADD ; +if (intr_handle->eptrs[vec] !=NULL ) { +RTE_LOG (ERR ,EAL ,"Event already been added.\n" ); +return -1 ; + } + +/* new event */ + rev =rte_zmalloc_socket ("eptrs" ,sizeof (*rev), +RTE_CACHE_LINE_SIZE , socket); +if (rev ==NULL ) { +RTE_LOG (ERR ,EAL ,"event obj alloc fail\n" ); +return -1 ; + } + +/* attach to intr vector fd */ + rev->fd = intr_handle->efds[vec]; + rev->event =EPOLLIN |EPOLLPRI |EPOLLET ; + rev->data = data; + rev->cb_fun = (rte_intr_event_cb_t )eal_intr_proc_rxtx_intr; + rev->cb_arg = (void *)intr_handle; + + rc =rte_epoll_ctl (epfd, epfd_op, rev->fd, rev); +if (!rc) + intr_handle->eptrs[vec] = rev; +else +rte_free (rev); + +break ; +case RTE_INTR_EVENT_DEL : + epfd_op =EPOLL_CTL_DEL ; +if (intr_handle->eptrs[vec] !=NULL ) { + rev = intr_handle->eptrs[vec]; + rc =rte_epoll_ctl (epfd, epfd_op, rev->fd, rev); +if (!rc) { +rte_free (rev); + intr_handle->eptrs[vec] =NULL ; + } + } +break ; +default : +RTE_LOG (ERR ,EAL ,"event op type mismatch\n" ); + rc =-1 ; + } + +return rc; +}
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +23 -0
@@ -133,4 +133,27 @@ rte_epoll_ctl(int epfd, int op, int fd,int rte_intr_tls_epfd (void ); +/** +* @param intr_handle +* Pointer to the interrupt handle. +* @param epfd +* Epoll instance fd which the intr vector associated to. +* @param op +* The operation be performed for the vector. +* Operation type of {ADD, DEL}. +* @param vec +* RX intr vector number added to the epoll instance wait list. +* @param data +* User raw data. +* @param socket +* Specifying the socket id. +* @return +* - On success, zero. +* - On failure, a negative value. +*/ +int +rte_intr_rx_ctl (struct rte_intr_handle *intr_handle, +int epfd,int op,unsigned int vec, +void *data,int socket); +#endif /* _RTE_LINUXAPP_INTERRUPTS_H_ */
lib/librte_eal/linuxapp/eal/rte_eal_version.map +1 -0
@@ -63,6 +63,7 @@ DPDK_2.0 { rte_intr_callback_unregister; rte_intr_disable; rte_intr_enable; + rte_intr_rx_ctl; rte_intr_tls_epfd; rte_log; rte_log_add_in_history;
Signature · 1 line
1.8.1.4
A message that is not in the archive (sent off-list or not yet received): 1430804386-28949-4-git-send-email-cunming.liang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH v7 03/10] eal/linux: add API to set rx interrupt event monitor
lib/librte_eal/linuxapp/eal/eal_interrupts.c rte_eal_intr_init(void) · line 866
Quoted 3 lines · On Tue, 5 May 2015 13:39:39 +0800 Cunming Liang…
static void
lib/librte_eal/linuxapp/eal/eal_interrupts.c +2
-0+eal_intr_proc_rxtx_intr(int fd, struct rte_intr_handle *intr_handle) +{
Should be const intr_handle is not modified
Re: [PATCH v7 03/10] eal/linux: add API to set rx interrupt event monitor
Quoted 7 lines · Stephen Hemminger
lib/librte_eal/linuxapp/eal/eal_interrupts.c rte_eal_intr_init(void) · line 866
Quoted 3 lines · On Tue, 5 May 2015 13:39:39 +0800 Cunming Liang…
static void
lib/librte_eal/linuxapp/eal/eal_interrupts.c +2
-0+eal_intr_proc_rxtx_intr(int fd, struct rte_intr_handle *intr_handle) +{Should be const
intr_handleis not modified
[LCM] accept.
Re: [PATCH v7 03/10] eal/linux: add API to set rx interrupt event monitor
In reply to Cunming Lianglib/librte_eal/linuxapp/eal/eal_interrupts.c rte_eal_intr_init(void)
Quoted 5 lines · On Tue, 5 May 2015 13:39:39 +0800 Cunming Liang <[email protected]>…
lib/librte_eal/linuxapp/eal/eal_interrupts.c +5
-0+ bytes_read =read (fd, &buf, bytes_read); +if (bytes_read <0 ) +RTE_LOG (ERR ,EAL ,"Error reading from file " +"descriptor %d: %s\n" , fd, + strerror(errno)
The read could be interrupted (EINTR) or there could be a race (EWOULDBLOCK). In those cases the code should not log anything.
PATCH v7 4/10 eal/bsd: dummy for new intr definition
In reply to a message not in the archiveTo make bsd compiling happy with new intr changes.
---
v7 changes
- remove stub 'linux only' function from source file
lib/librte_eal/bsdapp/eal/include/exec-env/rte_interrupts.h | 6 ++++++ 1 file changed, 6 insertions(+)
lib/librte_eal/bsdapp/eal/include/exec-env/rte_interrupts.h +6 -0
@@ -38,6 +38,8 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define RTE_MAX_RXTX_INTR_VEC_ID 32 +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */
@@ -49,6 +51,10 @@ enum rte_intr_handle_type {struct rte_intr_handle {int fd;/**< file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +int max_intr;/**< max interrupt requested */ +int vec_en;/**< intr vectors enabled */ +int efds[RTE_MAX_RXTX_INTR_VEC_ID ];/**< intr vectors/efds mapping */ +uint16_t *intr_vec;/**< intr vector number array */ };#endif /* _RTE_LINUXAPP_INTERRUPTS_H_ */
Signature · 1 line
1.8.1.4
PATCH v7 5/10 eal/linux: fix comments typo on vfio msi
In reply to a message not in the archive--- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +2 -2
@@ -219,7 +219,7 @@ vfio_disable_intx(struct rte_intr_handle *intr_handle) {return 0 ; } -/* enable MSI-X interrupts */ +/* enable MSI interrupts */ static int vfio_enable_msi (struct rte_intr_handle *intr_handle) {int len, ret;
@@ -265,7 +265,7 @@ vfio_enable_msi(struct rte_intr_handle *intr_handle) {return 0 ; } -/* disable MSI-X interrupts */ +/* disable MSI interrupts */ static int vfio_disable_msi (struct rte_intr_handle *intr_handle) {struct vfio_irq_set *irq_set;
Signature · 1 line
1.8.1.4
PATCH v7 6/10 eal/linux: add interrupt vectors handling on VFIO
In reply to a message not in the archiveThis patch does below:
- Create VFIO eventfds for each interrupt vector
- Assign per interrupt vector's eventfd to VFIO by ioctl
---
v7 changes
- cleanup unnecessary code change
- split event and intr operation to other patches
lib/librte_eal/linuxapp/eal/eal_interrupts.c | 36 ++++++++++++---------------- lib/librte_eal/linuxapp/eal/eal_pci_vfio.c | 12 ++++++++++ 2 files changed, 27 insertions(+), 21 deletions(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +15 -21
@@ -128,6 +128,9 @@ static pthread_t intr_thread;#ifdef VFIO_PRESENT #define IRQ_SET_BUF_LEN (sizeof(struct vfio_irq_set) + sizeof(int)) +/* irq set buffer length for queue interrupts and LSC interrupt */ +#define MSIX_IRQ_SET_BUF_LEN (sizeof(struct vfio_irq_set) + \ + sizeof(int) * (RTE_MAX_RXTX_INTR_VEC_ID + 1))/* enable legacy (INTx) interrupts */ static int
@@ -293,8 +296,8 @@ vfio_disable_msi(struct rte_intr_handle *intr_handle) {/* enable MSI-X interrupts */ static int vfio_enable_msix (struct rte_intr_handle *intr_handle) { -int len, ret; -char irq_set_buf[IRQ_SET_BUF_LEN ]; +int len, ret, max_intr; +char irq_set_buf[MSIX_IRQ_SET_BUF_LEN ];struct vfio_irq_set *irq_set;int *fd_ptr;
@@ -302,12 +305,19 @@ vfio_enable_msix(struct rte_intr_handle *intr_handle) { irq_set = (struct vfio_irq_set *) irq_set_buf; irq_set->argsz = len; - irq_set->count =1 ; +if ((!intr_handle->max_intr) || + (intr_handle->max_intr >RTE_MAX_RXTX_INTR_VEC_ID )) + max_intr =RTE_MAX_RXTX_INTR_VEC_ID +1 ; +else + max_intr = intr_handle->max_intr; + + irq_set->count = max_intr; irq_set->flags =VFIO_IRQ_SET_DATA_EVENTFD |VFIO_IRQ_SET_ACTION_TRIGGER ; irq_set->index =VFIO_PCI_MSIX_IRQ_INDEX ; irq_set->start =0 ; fd_ptr = (int *) &irq_set->data; - *fd_ptr = intr_handle->fd; +memcpy (fd_ptr, intr_handle->efds,sizeof (intr_handle->efds)); + fd_ptr[max_intr -1 ] = intr_handle->fd; ret =ioctl (intr_handle->vfio_dev_fd,VFIO_DEVICE_SET_IRQS , irq_set);
@@ -317,22 +327,6 @@ vfio_enable_msix(struct rte_intr_handle *intr_handle) {return -1 ; } -/* manually trigger interrupt to enable it */ -memset (irq_set,0 , len); - len =sizeof (struct vfio_irq_set ); - irq_set->argsz = len; - irq_set->count =1 ; - irq_set->flags =VFIO_IRQ_SET_DATA_NONE |VFIO_IRQ_SET_ACTION_TRIGGER ; - irq_set->index =VFIO_PCI_MSIX_IRQ_INDEX ; - irq_set->start =0 ; - - ret =ioctl (intr_handle->vfio_dev_fd,VFIO_DEVICE_SET_IRQS , irq_set); - -if (ret) { -RTE_LOG (ERR ,EAL ,"Error triggering MSI-X interrupts for fd %d\n" , - intr_handle->fd); -return -1 ; - }return 0 ; }
@@ -340,7 +334,7 @@ vfio_enable_msix(struct rte_intr_handle *intr_handle) {static int vfio_disable_msix (struct rte_intr_handle *intr_handle) {struct vfio_irq_set *irq_set; -char irq_set_buf[IRQ_SET_BUF_LEN ]; +char irq_set_buf[MSIX_IRQ_SET_BUF_LEN ];int len, ret; len =sizeof (struct vfio_irq_set );
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +12 -0
@@ -308,6 +308,18 @@ pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd)case VFIO_PCI_MSIX_IRQ_INDEX : internal_config.vfio_intr_mode =RTE_INTR_MODE_MSIX ; dev->intr_handle.type =RTE_INTR_HANDLE_VFIO_MSIX ; +for (i =0 ; i <RTE_MAX_RXTX_INTR_VEC_ID ; i++) { + fd =eventfd (0 ,0 ); +if (fd <0 ) { +RTE_LOG (ERR ,EAL , +"cannot setup eventfd," +"error %i (%s)\n" , + errno,strerror (errno)); +return -1 ; + } + dev->intr_handle.efds[i] = fd; + } + dev->intr_handle.vec_en =1 ;break ;case VFIO_PCI_MSI_IRQ_INDEX : internal_config.vfio_intr_mode =RTE_INTR_MODE_MSI ;
Signature · 1 line
1.8.1.4
A message that is not in the archive (sent off-list or not yet received): 1430804386-28949-7-git-send-email-cunming.liang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH v7 06/10] eal/linux: add interrupt vectors handling on VFIO
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd) · line 314
Quoted 12 lines · On Tue, 5 May 2015 13:39:42 +0800 Cunming Liang…
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +4
-0@@ -308,6 +308,18 @@ pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd)case VFIO_PCI_MSIX_IRQ_INDEX : internal_config.vfio_intr_mode =RTE_INTR_MODE_MSIX ; dev->intr_handle.type =RTE_INTR_HANDLE_VFIO_MSIX ; +for (i =0 ; i <RTE_MAX_RXTX_INTR_VEC_ID ; i++) { + fd =eventfd (0 ,0 ); +if (fd <0 ) { +
You should pass EFD_NONBLOCK and EFD_CLOEXEC as flags to any eventfd's created internally.
Re: [PATCH v7 06/10] eal/linux: add interrupt vectors handling on VFIO
Quoted 17 lines · Stephen Hemminger
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd) · line 314
Quoted 12 lines · On Tue, 5 May 2015 13:39:42 +0800 Cunming Liang…
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +4
-0@@ -308,6 +308,18 @@ pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd)case VFIO_PCI_MSIX_IRQ_INDEX : internal_config.vfio_intr_mode =RTE_INTR_MODE_MSIX ; dev->intr_handle.type =RTE_INTR_HANDLE_VFIO_MSIX ; +for (i =0 ; i <RTE_MAX_RXTX_INTR_VEC_ID ; i++) { + fd =eventfd (0 ,0 ); +if (fd <0 ) { +You should pass
EFD_NONBLOCKandEFD_CLOEXECas flags to any eventfd's created internally.
[LCM] Agree, make sense.
PATCH v7 7/10 ethdev: add rx intr enable, disable and ctl functions
In reply to a message not in the archiveThe patch adds two dev_ops functions to enable and disable rx queue interrupts.
In addtion, it adds rte_eth_dev_rx_intr_ctl/rx_intr_q to support per port or per queue rx intr event set.
---
v7 changes
- remove rx_intr_vec_get
- add rx_intr_ctl and rx_intr_ctl_q
v6 changes
- add rx_intr_vec_get to retrieve the vector num of the queue.
v5 changes
- Rebase the patchset onto the HEAD
v4 changes
- Export interrupt enable/disable functions for shared libraries
- Put new functions at the end of eth_dev_ops to avoid breaking ABI
v3 changes
- Add return value for interrupt enable/disable functions
lib/librte_ether/rte_ethdev.c | 132 +++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ethdev.h | 104 ++++++++++++++++++++++++++ lib/librte_ether/rte_ether_version.map | 4 + 3 files changed, 240 insertions(+)
lib/librte_ether/rte_ethdev.c +132 -0
@@ -3281,6 +3281,138 @@ _rte_eth_dev_callback_process(struct rte_eth_dev *dev, }rte_spinlock_unlock (&rte_eth_dev_cb_lock); } + +int +rte_eth_dev_rx_intr_ctl (uint8_t port_id,int epfd,int op,void *data) +{ +uint32_t vec; +struct rte_eth_dev *dev; +struct rte_intr_handle *intr_handle; +uint16_t qid; +int rc; + +if (!rte_eth_dev_is_valid_port (port_id)) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return -ENODEV ; + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return -ENODEV ; + } + + intr_handle = &dev->pci_dev->intr_handle; +if (!intr_handle->intr_vec) { +PMD_DEBUG_TRACE ("RX Intr vector unset\n" ); +return -EPERM ; + } + +for (qid =0 ; qid < dev->data->nb_rx_queues; qid++) { +if (intr_handle->intr_vec[qid] <0 ) { +PMD_DEBUG_TRACE ("RX Intr vector invalid on %d\n" , qid); +continue ; + } + + vec = intr_handle->intr_vec[qid]; + rc =rte_intr_rx_ctl (intr_handle, epfd, op, vec, + data,rte_eth_dev_socket_id (port_id)); +if (rc) { +PMD_DEBUG_TRACE ("p %d q %d rx ctl error" +" op %d epfd %d vec %u\n" , + port_id, qid, op, epfd, vec); + } + } + +return 0 ; +} + +int +rte_eth_dev_rx_intr_ctl_q (uint8_t port_id,uint16_t queue_id, +int epfd,int op,void *data) +{ +uint32_t vec; +struct rte_eth_dev *dev; +struct rte_intr_handle *intr_handle; +int rc; + +if (!rte_eth_dev_is_valid_port (port_id)) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return -ENODEV ; + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return -ENODEV ; + } + +if (queue_id >= dev->data->nb_rx_queues) { +PMD_DEBUG_TRACE ("Invalid RX queue_id=%d\n" , rx_queue_id); +return -EINVAL ; + } + + intr_handle = &dev->pci_dev->intr_handle; +if (!intr_handle->intr_vec || intr_handle->intr_vec[queue_id] <0 ) { +PMD_DEBUG_TRACE ("RX Intr vector unset on %d\n" , rx_queue_id); +return -EPERM ; + } + + vec = intr_handle->intr_vec[queue_id]; + rc =rte_intr_rx_ctl (intr_handle, epfd, op, vec, + data,rte_eth_dev_socket_id (port_id)); +if (rc) { +PMD_DEBUG_TRACE ("p %d q %d rx ctl error" +" op %d epfd %d vec %u\n" , + port_id, queue_id, op, epfd, vec); +return rc; + } + +return 0 ; +} + +int +rte_eth_dev_rx_intr_enable (uint8_t port_id, +uint16_t queue_id) +{ +struct rte_eth_dev *dev; + +if (!rte_eth_dev_is_valid_port (port_id)) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return -ENODEV ; + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return -ENODEV ; + } + +FUNC_PTR_OR_ERR_RET (*dev->dev_ops->rx_queue_intr_enable, -ENOTSUP ); +return (*dev->dev_ops->rx_queue_intr_enable)(dev, queue_id); +} + +int +rte_eth_dev_rx_intr_disable (uint8_t port_id, +uint16_t queue_id) +{ +struct rte_eth_dev *dev; + +if (!rte_eth_dev_is_valid_port (port_id)) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return -ENODEV ; + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return -ENODEV ; + } + +FUNC_PTR_OR_ERR_RET (*dev->dev_ops->rx_queue_intr_disable, -ENOTSUP ); +return (*dev->dev_ops->rx_queue_intr_disable)(dev, queue_id); +} +#ifdef RTE_NIC_BYPASS int rte_eth_dev_bypass_init (uint8_t port_id) {
lib/librte_ether/rte_ethdev.h +104 -0
128 lines not on this page · raw message
lib/librte_ether/rte_ether_version.map +4 -0
@@ -48,6 +48,10 @@ DPDK_2.0 { rte_eth_dev_rss_hash_update; rte_eth_dev_rss_reta_query; rte_eth_dev_rss_reta_update; + rte_eth_dev_rx_intr_ctl; + rte_eth_dev_rx_intr_ctl_q; + rte_eth_dev_rx_intr_disable; + rte_eth_dev_rx_intr_enable; rte_eth_dev_rx_queue_start; rte_eth_dev_rx_queue_stop; rte_eth_dev_set_link_down;
Signature · 1 line
1.8.1.4
PATCH v7 8/10 ixgbe: enable rx queue interrupts for both PF and VF
In reply to a message not in the archiveThe patch does below things for ixgbe PF and VF:
- Setup NIC to generate MSI-X interrupts
- Set the IVAR register to map interrupt causes to vectors
- Implement interrupt enable/disable functions
---
v7 changes
- add condition check when intr vector is not enabled
v6 changes
- fill queue-vector mapping table
v5 changes
- Rebase the patchset onto the HEAD
v3 changes
- Remove spinlok from PMD
v2 changes
- Consolidate review comments related to coding style
lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 425 +++++++++++++++++++++++++++++++++++- lib/librte_pmd_ixgbe/ixgbe_ethdev.h | 7 + 2 files changed, 428 insertions(+), 4 deletions(-)
lib/librte_pmd_ixgbe/ixgbe_ethdev.c +421 -4
@@ -82,6 +82,9 @@*/ #define IXGBE_FC_LO 0x40 +/* Default minimum inter-interrupt interval for EITR configuration */ +#define IXGBE_MIN_INTER_INTERRUPT_INTERVAL_DEFAULT 0x79E +/* Timer value included in XOFF frames. */ #define IXGBE_FC_PAUSE 0x680
@@ -171,6 +174,7 @@ static int ixgbe_dev_rss_reta_query(struct rte_eth_dev *dev, uint16_t reta_size);static void ixgbe_dev_link_status_print (struct rte_eth_dev *dev);static int ixgbe_dev_lsc_interrupt_setup (struct rte_eth_dev *dev); +static int ixgbe_dev_rxq_interrupt_setup (struct rte_eth_dev *dev);static int ixgbe_dev_interrupt_get_status (struct rte_eth_dev *dev);static int ixgbe_dev_interrupt_action (struct rte_eth_dev *dev); static void ixgbe_dev_interrupt_handler(struct rte_intr_handle *handle,
@@ -183,11 +187,14 @@ static void ixgbe_dcb_init(struct ixgbe_hw *hw,struct ixgbe_dcb_config *dcb_conf/* For Virtual Function support */ static int eth_ixgbevf_dev_init (struct rte_eth_dev *eth_dev); +static int ixgbevf_dev_interrupt_get_status (struct rte_eth_dev *dev); +static int ixgbevf_dev_interrupt_action (struct rte_eth_dev *dev);static int ixgbevf_dev_configure (struct rte_eth_dev *dev);static int ixgbevf_dev_start (struct rte_eth_dev *dev);static void ixgbevf_dev_stop (struct rte_eth_dev *dev);static void ixgbevf_dev_close (struct rte_eth_dev *dev);static void ixgbevf_intr_disable (struct ixgbe_hw *hw); +static void ixgbevf_intr_enable (struct ixgbe_hw *hw);static void ixgbevf_dev_stats_get (struct rte_eth_dev *dev,struct rte_eth_stats *stats);static void ixgbevf_dev_stats_reset (struct rte_eth_dev *dev);
@@ -197,6 +204,15 @@ static void ixgbevf_vlan_strip_queue_set(struct rte_eth_dev *dev, uint16_t queue, int on);static void ixgbevf_vlan_offload_set (struct rte_eth_dev *dev,int mask);static void ixgbevf_set_vfta_all (struct rte_eth_dev *dev,bool on); +static void ixgbevf_dev_interrupt_handler (struct rte_intr_handle *handle, +void *param); +static int ixgbevf_dev_rx_queue_intr_enable (struct rte_eth_dev *dev, +uint16_t queue_id); +static int ixgbevf_dev_rx_queue_intr_disable (struct rte_eth_dev *dev, +uint16_t queue_id); +static void ixgbevf_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector); +static void ixgbevf_configure_msix (struct rte_eth_dev *dev);/* For Eth VMDQ APIs support */ static int ixgbe_uc_hash_table_set(struct rte_eth_dev *dev, struct
@@ -214,6 +230,14 @@ static int ixgbe_mirror_rule_set(struct rte_eth_dev *dev,static int ixgbe_mirror_rule_reset (struct rte_eth_dev *dev,uint8_t rule_id); +static int ixgbe_dev_rx_queue_intr_enable (struct rte_eth_dev *dev, +uint16_t queue_id); +static int ixgbe_dev_rx_queue_intr_disable (struct rte_eth_dev *dev, +uint16_t queue_id); +static void ixgbe_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector); +static void ixgbe_configure_msix (struct rte_eth_dev *dev); +static int ixgbe_set_queue_rate_limit (struct rte_eth_dev *dev,uint16_t queue_idx,uint16_t tx_rate); static int ixgbe_set_vf_rate_limit(struct rte_eth_dev *dev, uint16_t vf,
@@ -262,7 +286,7 @@ static int ixgbevf_dev_set_mtu(struct rte_eth_dev *dev, uint16_t mtu);*/ #define UPDATE_VF_STAT (reg, last, cur) \ { \ - u32 latest = IXGBE_READ_REG(hw, reg); \ + uint32_t latest = IXGBE_READ_REG(hw, reg); \ cur += latest - last; \ last = latest; \ }
@@ -343,6 +367,8 @@ static const struct eth_dev_ops ixgbe_eth_dev_ops = { .tx_queue_start = ixgbe_dev_tx_queue_start, .tx_queue_stop = ixgbe_dev_tx_queue_stop, .rx_queue_setup = ixgbe_dev_rx_queue_setup, + .rx_queue_intr_enable = ixgbe_dev_rx_queue_intr_enable, + .rx_queue_intr_disable = ixgbe_dev_rx_queue_intr_disable, .rx_queue_release = ixgbe_dev_rx_queue_release, .rx_queue_count = ixgbe_dev_rx_queue_count, .rx_descriptor_done = ixgbe_dev_rx_descriptor_done,
@@ -402,8 +428,11 @@ static const struct eth_dev_ops ixgbevf_eth_dev_ops = { .vlan_offload_set = ixgbevf_vlan_offload_set, .rx_queue_setup = ixgbe_dev_rx_queue_setup, .rx_queue_release = ixgbe_dev_rx_queue_release, + .rx_descriptor_done = ixgbe_dev_rx_descriptor_done, .tx_queue_setup = ixgbe_dev_tx_queue_setup, .tx_queue_release = ixgbe_dev_tx_queue_release, + .rx_queue_intr_enable = ixgbevf_dev_rx_queue_intr_enable, + .rx_queue_intr_disable = ixgbevf_dev_rx_queue_intr_disable, .mac_addr_add = ixgbevf_add_mac_addr, .mac_addr_remove = ixgbevf_remove_mac_addr, };
@@ -899,9 +928,24 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev) eth_dev->data->port_id, pci_dev->id.vendor_id, pci_dev->id.device_id); +/* set max interrupt vfio request */ +if (pci_dev->intr_handle.vec_en) { + pci_dev->intr_handle.max_intr = hw->mac.max_rx_queues + +IXGBE_MAX_OTHER_INTR ; + pci_dev->intr_handle.intr_vec = +rte_zmalloc ("intr_vec" , + hw->mac.max_rx_queues *sizeof (int ),0 ); +if (pci_dev->intr_handle.intr_vec ==NULL ) { +PMD_INIT_LOG (ERR ,"Failed to allocate %d rx_queues" +"intr_vec\n" , hw->mac.max_rx_queues); +return -ENOMEM ; + } + } +rte_intr_callback_register (&(pci_dev->intr_handle), ixgbe_dev_interrupt_handler, (void *)eth_dev); +/* enable uio intr after callback register */ rte_intr_enable (&(pci_dev->intr_handle));
@@ -1079,6 +1123,25 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)return (-EIO ); } +/* set max interrupt vfio request */ +if (pci_dev->intr_handle.vec_en) { + pci_dev->intr_handle.max_intr = hw->mac.max_rx_queues + +IXGBEVF_MAX_OTHER_INTR ; + pci_dev->intr_handle.intr_vec = +rte_zmalloc ("intr_vec" , + hw->mac.max_rx_queues *sizeof (int ),0 ); +if (pci_dev->intr_handle.intr_vec ==NULL ) { +PMD_INIT_LOG (ERR ,"Failed to allocate %d rx_queues" +" intr_vec\n" , hw->mac.max_rx_queues); +return -ENOMEM ; + } + } + +rte_intr_callback_register (&(pci_dev->intr_handle), + ixgbevf_dev_interrupt_handler, (void *)eth_dev); + +rte_intr_enable (&(pci_dev->intr_handle)); +PMD_INIT_LOG (DEBUG ,"port %d vendorID=0x%x deviceID=0x%x mac.type=%s" , eth_dev->data->port_id, pci_dev->id.vendor_id, pci_dev->id.device_id,"ixgbe_mac_82599_vf" );
@@ -1489,6 +1552,9 @@ ixgbe_dev_start(struct rte_eth_dev *dev)/* configure PF module if SRIOV enabled */ ixgbe_pf_host_configure (dev); +/* confiugre msix for sleep until rx interrupt */ +ixgbe_configure_msix (dev); +/* initialize transmission unit */ ixgbe_dev_tx_init (dev);
@@ -1564,6 +1630,10 @@ skip_link_setup:if (dev->data->dev_conf.intr_conf.lsc !=0 )ixgbe_dev_lsc_interrupt_setup (dev); +/* check if rxq interrupt is enabled */ +if (dev->data->dev_conf.intr_conf.rxq !=0 ) +ixgbe_dev_rxq_interrupt_setup (dev); +/* resume enabled intr since hw reset */ ixgbe_enable_intr (dev);
@@ -1725,6 +1795,7 @@ ixgbe_dev_close(struct rte_eth_dev *dev) {struct ixgbe_hw *hw =IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct rte_pci_device *pci_dev;PMD_INIT_FUNC_TRACE ();
@@ -1737,6 +1808,12 @@ ixgbe_dev_close(struct rte_eth_dev *dev)/* reprogram the RAR[0] in case user changed it. */ ixgbe_set_rar (hw,0 , hw->mac.addr,0 ,IXGBE_RAH_AV ); + + pci_dev = dev->pci_dev; +if (pci_dev->intr_handle.intr_vec) { +rte_free (pci_dev->intr_handle.intr_vec); + pci_dev->intr_handle.intr_vec =NULL ; + } }/*
@@ -2252,6 +2329,28 @@ ixgbe_dev_lsc_interrupt_setup(struct rte_eth_dev *dev)return 0 ; } +/** +* It clears the interrupt causes and enables the interrupt. +* It will be called once only during nic initialized. +* +* @param dev +* Pointer to struct rte_eth_dev. +* +* @return +* - On success, zero. +* - On failure, a negative value. +*/ +static int +ixgbe_dev_rxq_interrupt_setup (struct rte_eth_dev *dev) +{ +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + + intr->mask |=IXGBE_EICR_RTX_QUEUE ; + +return 0 ; +} +/* * It reads ICR and sets flag (IXGBE_EICR_LSC) for the link_update. *
@@ -2278,10 +2377,10 @@ ixgbe_dev_interrupt_get_status(struct rte_eth_dev *dev)PMD_DRV_LOG (INFO ,"eicr %x" , eicr); intr->flags =0 ; -if (eicr &IXGBE_EICR_LSC ) { -/* set flag for async link update */ + +/* set flag for async link update */ +if (eicr &IXGBE_EICR_LSC ) intr->flags |=IXGBE_FLAG_NEED_LINK_UPDATE ; - }if (eicr &IXGBE_EICR_MAILBOX ) intr->flags |=IXGBE_FLAG_MAILBOX ;
@@ -2289,6 +2388,30 @@ ixgbe_dev_interrupt_get_status(struct rte_eth_dev *dev)return 0 ; } +static int +ixgbevf_dev_interrupt_get_status (struct rte_eth_dev *dev) +{ +uint32_t eicr; +struct ixgbe_hw *hw =IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + +/* clear all cause mask */ +ixgbevf_intr_disable (hw); + +/* read-on-clear nic registers here */ + eicr =IXGBE_READ_REG (hw,IXGBE_VTEICR ); +PMD_DRV_LOG (INFO ,"eicr %x" , eicr); + + intr->flags =0 ; + +/* set flag for async link update */ +if (eicr &IXGBE_EICR_LSC ) + intr->flags |=IXGBE_FLAG_NEED_LINK_UPDATE ; + +return 0 ; +} +/** * It gets and then prints the link status. *
@@ -2384,6 +2507,18 @@ ixgbe_dev_interrupt_action(struct rte_eth_dev *dev)return 0 ; } +static int +ixgbevf_dev_interrupt_action (struct rte_eth_dev *dev) +{ +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); + +PMD_DRV_LOG (DEBUG ,"enable intr immediately" ); +ixgbevf_intr_enable (hw); +rte_intr_enable (&(dev->pci_dev->intr_handle)); +return 0 ; +} +/** * Interrupt handler which shall be registered for alarm callback for delayed * handling specific interrupt to wait for the stable nic state. As the
@@ -2445,6 +2580,15 @@ ixgbe_dev_interrupt_handler(__rte_unused struct rte_intr_handle *handle,ixgbe_dev_interrupt_action (dev); } +static void +ixgbevf_dev_interrupt_handler(__rte_unused struct rte_intr_handle *handle, +void *param) +{ +struct rte_eth_dev *dev = (struct rte_eth_dev *)param; +ixgbevf_dev_interrupt_get_status (dev); +ixgbevf_dev_interrupt_action (dev); +} +static int ixgbe_dev_led_on (struct rte_eth_dev *dev) {
@@ -2943,6 +3087,19 @@ ixgbevf_intr_disable(struct ixgbe_hw *hw)IXGBE_WRITE_FLUSH (hw); } +static void +ixgbevf_intr_enable (struct ixgbe_hw *hw) +{ +PMD_INIT_FUNC_TRACE (); + +/* VF enable interrupt autoclean */ +IXGBE_WRITE_REG (hw,IXGBE_VTEIAM ,IXGBE_VF_IRQ_ENABLE_MASK ); +IXGBE_WRITE_REG (hw,IXGBE_VTEIAC ,IXGBE_VF_IRQ_ENABLE_MASK ); +IXGBE_WRITE_REG (hw,IXGBE_VTEIMS ,IXGBE_VF_IRQ_ENABLE_MASK ); + +IXGBE_WRITE_FLUSH (hw); +} +static int ixgbevf_dev_configure (struct rte_eth_dev *dev) {
@@ -3005,6 +3162,11 @@ ixgbevf_dev_start(struct rte_eth_dev *dev)ixgbevf_dev_rxtx_start (dev); +ixgbevf_configure_msix (dev); + +/* Re-enable interrupt for VF */ +ixgbevf_intr_enable (hw); +return 0 ; }
@@ -3034,6 +3196,7 @@ static void ixgbevf_dev_close(struct rte_eth_dev *dev) {struct ixgbe_hw *hw =IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct rte_pci_device *pci_dev;PMD_INIT_FUNC_TRACE ();
@@ -3043,6 +3206,12 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)/* reprogram the RAR[0] in case user changed it. */ ixgbe_set_rar (hw,0 , hw->mac.addr,0 ,IXGBE_RAH_AV ); + + pci_dev = dev->pci_dev; +if (pci_dev->intr_handle.intr_vec) { +rte_free (pci_dev->intr_handle.intr_vec); + pci_dev->intr_handle.intr_vec =NULL ; + } }static void ixgbevf_set_vfta_all (struct rte_eth_dev *dev,bool on)
@@ -3542,6 +3711,254 @@ ixgbe_mirror_rule_reset(struct rte_eth_dev *dev, uint8_t rule_id)return 0 ; } + +static int +ixgbevf_dev_rx_queue_intr_enable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); + + mask =IXGBE_READ_REG (hw,IXGBE_VTEIMS ); + mask |= (1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_VTEIMS , mask); + +return 0 ; +} + +static int +ixgbevf_dev_rx_queue_intr_disable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); + + mask =IXGBE_READ_REG (hw,IXGBE_VTEIMS ); + mask &= ~(1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_VTEIMS , mask); + +return 0 ; +} + +static int +ixgbe_dev_rx_queue_intr_enable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + +if (queue_id <16 ) { +ixgbe_disable_intr (hw); + intr->mask |= (1 << queue_id); +ixgbe_enable_intr (dev); + }else if (queue_id <32 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (0 )); + mask &= (1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (0 ), mask); + }else if (queue_id <64 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (1 )); + mask &= (1 << (queue_id -32 )); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (1 ), mask); + } + +return 0 ; +} + +static int +ixgbe_dev_rx_queue_intr_disable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + +if (queue_id <16 ) { +ixgbe_disable_intr (hw); + intr->mask &= ~(1 << queue_id); +ixgbe_enable_intr (dev); + }else if (queue_id <32 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (0 )); + mask &= ~(1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (0 ), mask); + }else if (queue_id <64 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (1 )); + mask &= ~(1 << (queue_id -32 )); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (1 ), mask); + } + +return 0 ; +} + +static void +ixgbevf_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector) +{ +uint32_t tmp, idx; +if (direction ==-1 ) { +/* other causes */ + msix_vector |=IXGBE_IVAR_ALLOC_VAL ; + tmp =IXGBE_READ_REG (hw,IXGBE_VTIVAR_MISC ); + tmp &= ~0xFF ; + tmp |= msix_vector; +IXGBE_WRITE_REG (hw,IXGBE_VTIVAR_MISC , tmp); + }else { +/* rx or tx cause */ + msix_vector |=IXGBE_IVAR_ALLOC_VAL ; + idx = ((16 * (queue &1 )) + (8 * direction)); + tmp =IXGBE_READ_REG (hw,IXGBE_VTIVAR (queue >>1 )); + tmp &= ~(0xFF << idx); + tmp |= (msix_vector << idx); +IXGBE_WRITE_REG (hw,IXGBE_VTIVAR (queue >>1 ), tmp); + } +} + +/** +* set the IVAR registers, mapping interrupt causes to vectors +* @param hw +* pointer to ixgbe_hw struct +* @direction +* 0 for Rx, 1 for Tx, -1 for other causes +* @queue +* queue to map the corresponding interrupt to +* @msix_vector +* the vector to map to the corresponding queue +*/ +static void +ixgbe_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector) +{ +uint32_t tmp, idx; + + msix_vector |=IXGBE_IVAR_ALLOC_VAL ; +if (hw->mac.type == ixgbe_mac_82598EB) { +if (direction ==-1 ) + direction =0 ; + idx = (((direction *64 ) + queue) >>2 ) &0x1F ; + tmp =IXGBE_READ_REG (hw,IXGBE_IVAR (idx)); + tmp &= ~(0xFF << (8 * (queue &0x3 ))); + tmp |= (msix_vector << (8 * (queue &0x3 ))); +IXGBE_WRITE_REG (hw,IXGBE_IVAR (idx), tmp); + }else if ((hw->mac.type == ixgbe_mac_82599EB) || + (hw->mac.type == ixgbe_mac_X540)) { +if (direction ==-1 ) { +/* other causes */ + idx = ((queue &1 ) *8 ); + tmp =IXGBE_READ_REG (hw,IXGBE_IVAR_MISC ); + tmp &= ~(0xFF << idx); + tmp |= (msix_vector << idx); +IXGBE_WRITE_REG (hw,IXGBE_IVAR_MISC , tmp); + }else { +/* rx or tx causes */ + idx = ((16 * (queue &1 )) + (8 * direction)); + tmp =IXGBE_READ_REG (hw,IXGBE_IVAR (queue >>1 )); + tmp &= ~(0xFF << idx); + tmp |= (msix_vector << idx); +IXGBE_WRITE_REG (hw,IXGBE_IVAR (queue >>1 ), tmp); + } + } +} + +static void +ixgbevf_configure_msix (struct rte_eth_dev *dev) +{ +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle; +uint32_t q_idx; +uint32_t vector_idx =0 ; + +/* won't configure msix register if no mapping is done +* between intr vector and event fd */ +if (!intr_handle->vec_en) +return ; + +/* Configure all RX queues of VF */ +for (q_idx =0 ; q_idx < (hw->mac.max_rx_queues -1 ); q_idx++) { +/* Force all queue use vector 0, +* as IXGBE_VF_MAXMSIVECOTR = 1 */ +ixgbevf_set_ivar_map (hw,0 , q_idx, vector_idx); + intr_handle->intr_vec[q_idx] = vector_idx; + } + +/* Configure VF Rx queue ivar */ +ixgbevf_set_ivar_map (hw,-1 ,1 , vector_idx); +} + +/** +* Sets up the hardware to properly generate MSI-X interrupts +* @hw +* board private structure +*/ +static void +ixgbe_configure_msix (struct rte_eth_dev *dev) +{ +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle; +int queue_id, vec =0 ; +uint32_t mask; +uint32_t gpie; + +/* won't configure msix register if no mapping is done +* between intr vector and event fd */ +if (!intr_handle->vec_en) +return ; + +/* setup GPIE for MSI-x mode */ + gpie =IXGBE_READ_REG (hw,IXGBE_GPIE ); + gpie |=IXGBE_GPIE_MSIX_MODE |IXGBE_GPIE_PBA_SUPPORT | +IXGBE_GPIE_OCD |IXGBE_GPIE_EIAME ; +/* +* auto clearing and auto setting corresponding bits in EIMS +* when MSI-X interrupt is triggered +*/ +if (hw->mac.type == ixgbe_mac_82598EB) +IXGBE_WRITE_REG (hw,IXGBE_EIAM ,IXGBE_EICS_RTX_QUEUE ); +else { +IXGBE_WRITE_REG (hw,IXGBE_EIAM_EX (0 ),0xFFFFFFFF ); +IXGBE_WRITE_REG (hw,IXGBE_EIAM_EX (1 ),0xFFFFFFFF ); + } +IXGBE_WRITE_REG (hw,IXGBE_GPIE , gpie); + +/* +* Populate the IVAR table and set the ITR values to the +* corresponding register. +*/ +for (queue_id =0 ; queue_id < dev->data->nb_rx_queues; queue_id++) { +/* by default, 1:1 mapping */ +ixgbe_set_ivar_map (hw,0 , queue_id, vec); + intr_handle->intr_vec[queue_id] = vec; +if (vec <RTE_MAX_RXTX_INTR_VEC_ID ) + vec++; + } + +switch (hw->mac.type) { +case ixgbe_mac_82598EB: +ixgbe_set_ivar_map (hw,-1 ,IXGBE_IVAR_OTHER_CAUSES_INDEX , +RTE_MAX_RXTX_INTR_VEC_ID ); +break ; +case ixgbe_mac_82599EB: +case ixgbe_mac_X540: +ixgbe_set_ivar_map (hw,-1 ,1 ,RTE_MAX_RXTX_INTR_VEC_ID ); +break ; +default : +break ; + } +IXGBE_WRITE_REG (hw,IXGBE_EITR (queue_id), +IXGBE_MIN_INTER_INTERRUPT_INTERVAL_DEFAULT &0xFFF ); + +/* set up to autoclear timer, and the vectors */ + mask =IXGBE_EIMS_ENABLE_MASK ; + mask &= ~(IXGBE_EIMS_OTHER | +IXGBE_EIMS_MAILBOX | +IXGBE_EIMS_LSC ); + +IXGBE_WRITE_REG (hw,IXGBE_EIAC , mask); +} +static int ixgbe_set_queue_rate_limit (struct rte_eth_dev *dev,uint16_t queue_idx,uint16_t tx_rate) {
lib/librte_pmd_ixgbe/ixgbe_ethdev.h +7 -0
@@ -117,6 +117,12 @@ ETH_RSS_IPV6_TCP_EX | \ ETH_RSS_IPV6_UDP_EX) +#define IXGBE_VF_IRQ_ENABLE_MASK 3/* vf irq enable mask */ +#define IXGBE_VF_MAXMSIVECTOR 1 +/* maximum other interrupts besides rx&tx*/ +#define IXGBE_MAX_OTHER_INTR 1 +#define IXGBEVF_MAX_OTHER_INTR 1 +/* * Information about the fdir mode. */
@@ -325,6 +331,7 @@ uint32_t ixgbe_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id);int ixgbe_dev_rx_descriptor_done (void *rx_queue,uint16_t offset); +int ixgbevf_dev_rx_descriptor_done (void *rx_queue,uint16_t offset);int ixgbe_dev_rx_init (struct rte_eth_dev *dev);
Signature · 1 line
1.8.1.4
A message that is not in the archive (sent off-list or not yet received): 1430804386-28949-9-git-send-email-cunming.liang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH v7 08/10] ixgbe: enable rx queue interrupts for both PF and VF
lib/librte_pmd_ixgbe/ixgbe_ethdev.c eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev) · line 1139
Quoted 9 lines · On Tue, 5 May 2015 13:39:44 +0800 Cunming Liang…
lib/librte_pmd_ixgbe/ixgbe_ethdev.c +8
-0+/* set max interrupt vfio request */ +if (pci_dev->intr_handle.vec_en) { + pci_dev->intr_handle.max_intr = hw->mac.max_rx_queues + +IXGBEVF_MAX_OTHER_INTR ; + pci_dev->intr_handle.intr_vec = +rte_zmalloc ("intr_vec" , + hw->mac.max_rx_queues *sizeof (int ),0 ); +
Since MSI-X vectors are limited on many hardware platforms, this whole API should be changed so that max_intr is based on number of rx_queues actually used by the application. That means the setup needs to move from init to configure.
Re: [PATCH v7 08/10] ixgbe: enable rx queue interrupts for both PF and VF
Quoted 4 lines · Stephen Hemminger
Quoted 1 line · On Tue, 5 May 2015 13:39:44 +0800 Cunming Liang <[email protected]>…
Quoted 11 lines
lib/librte_pmd_ixgbe/ixgbe_ethdev.c eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev) · line 1139
Quoted 8 lines
lib/librte_pmd_ixgbe/ixgbe_ethdev.c +8
-0+/* set max interrupt vfio request */ +if (pci_dev->intr_handle.vec_en) { + pci_dev->intr_handle.max_intr = hw->mac.max_rx_queues + +IXGBEVF_MAX_OTHER_INTR ; + pci_dev->intr_handle.intr_vec = +rte_zmalloc ("intr_vec" , + hw->mac.max_rx_queues *sizeof (int ),0 ); +Since MSI-X vectors are limited on many hardware platforms, this whole API should be changed so that
max_intris based on number ofrx_queuesactually used by the application. That means the setup needs to move from init to configure.
[LCM] When MSI-X is not used, intr_vec and set max_intr are useless. It doesn't matter to non MSI-X mode.
As it allows the sequence "dev_stop->dev_reconfig->dev_start", the real used number of queue may change.
So allocation only on dev_init and release only on dev_close, just make it simple. During configure_msix, it do use the real useful queue number to set queue/vector mapping, refer xxx_configure_msix().
Re: [PATCH v7 08/10] ixgbe: enable rx queue interrupts for both PF and VF
Quoted 10 lines · Cunming
Quoted 3 lines
Since MSI-X vectors are limited on many hardware platforms, this whole API should be changed so that
max_intris based on number ofrx_queuesactually used by the application. That means the setup needs to move from init to configure.[LCM] When MSI-X is not used,
intr_vecand setmax_intrare useless. It doesn't matter to non MSI-X mode.
As it allows the sequence "dev_stop->dev_reconfig->dev_start", the real used number of queue may change.
So allocation only ondev_initand release only ondev_close, just make it simple. Duringconfigure_msix, it do use the real useful queue number to set queue/vector mapping, referxxx_configure_msix().
The problem is that if a customer has 16 NIC's with 32 MSI vectors per NIC, it maybe that the MSI table in south bridge gets full. That is why the ixgbe driver for Linux limits itself to num_online_cpu() + 1 MSI interrrupts.
Re: [PATCH v7 08/10] ixgbe: enable rx queue interrupts for both PF and VF
Quoted 13 lines · Stephen Hemminger
Quoted 10 lines · Cunming
Quoted 3 lines
Since MSI-X vectors are limited on many hardware platforms, this whole API should be changed so that
max_intris based on number ofrx_queuesactually used by the application. That means the setup needs to move from init to configure.[LCM] When MSI-X is not used,
intr_vecand setmax_intrare useless. It doesn't matter to non MSI-X mode.
As it allows the sequence "dev_stop->dev_reconfig->dev_start", the real used number of queue may change.
So allocation only ondev_initand release only ondev_close, just make it simple. Duringconfigure_msix, it do use the real useful queue number to set queue/vector mapping, referxxx_configure_msix().
Quoted 4 lines
The problem is that if a customer has 16 NIC's with 32 MSI vectors per NIC, it maybe that the MSI table in south bridge gets full. That is why the ixgbe driver for Linux limits itself to
num_online_cpu()+ 1 MSI interrrupts.
[LCM] So your concern actual is not about when to allocate the queue/vec mapping table, but the number of vectors enabled per NIC. But even using num_online_cpu()+1, the number of online cpu on a 2U system is easy to exceed 32. So probably using a MSI_VECTOR_NB_MAX to define it in config.
PATCH v7 9/10 igb: enable rx queue interrupts for PF
In reply to a message not in the archiveThe patch does below for igb PF:
- Setup NIC to generate MSI-X interrupts
- Set the IVAR register to map interrupt causes to vectors
- Implement interrupt enable/disable functions
---
v7 changes
- add condition check when intr vector is not enabled
v6 changes
- fill queue-vector mapping table
v5 changes
- Rebase the patchset onto the HEAD
v3 changes
- Remove unnecessary variables in e1000_mac_info
- Remove spinlok from PMD
v2 changes
- Consolidate review comments related to coding style
lib/librte_pmd_e1000/e1000_ethdev.h | 3 + lib/librte_pmd_e1000/igb_ethdev.c | 256 ++++++++++++++++++++++++++++++++---- 2 files changed, 234 insertions(+), 25 deletions(-)
lib/librte_pmd_e1000/e1000_ethdev.h +3 -0
@@ -108,6 +108,9 @@ ETH_RSS_IPV6_TCP_EX | \ ETH_RSS_IPV6_UDP_EX) +/* maximum number of other interrupts besides Rx & Tx interrupts */ +#define E1000_MAX_OTHER_INTR 1 +/* structure for interrupt relative data */ struct e1000_interrupt {uint32_t flags;
lib/librte_pmd_e1000/igb_ethdev.c +231 -25
336 lines not on this page · raw message
Signature · 1 line
1.8.1.4
A message that is not in the archive (sent off-list or not yet received): 1430804386-28949-10-git-send-email-cunming.liang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH v7 09/10] igb: enable rx queue interrupts for PF
Quoted 7 lines · On Tue, 5 May 2015 13:39:45 +0800 Cunming Liang…
The patch does below for igb PF:
- Setup NIC to generate MSI-X interrupts
- Set the IVAR register to map interrupt causes to vectors
- Implement interrupt enable/disable functionsSigned-off-by Danny Zhou <[email protected]>Signed-off-by Cunming Liang <[email protected]>
What about E1000?
This only usable if it works on all devices.
Re: [PATCH v7 09/10] igb: enable rx queue interrupts for PF
Quoted 8 lines · Stephen Hemminger
Quoted 5 lines · On Tue, 5 May 2015 13:39:45 +0800 Cunming Liang <[email protected]>…
The patch does below for igb PF:
- Setup NIC to generate MSI-X interrupts
- Set the IVAR register to map interrupt causes to vectors
- Implement interrupt enable/disable functions
Quoted 5 lines
Quoted 2 lines
Signed-off-by Danny Zhou <[email protected]>Signed-off-by Cunming Liang <[email protected]>What about E1000?
This only usable if it works on all devices.
[LCM] Agree with you, will send separate patch for e1000 after the patch series close.
Re: [PATCH v7 09/10] igb: enable rx queue interrupts for PF
In reply to Cunming Lianglib/librte_pmd_e1000/igb_ethdev.c eth_igb_dev_init(struct rte_eth_dev *eth_dev) · line 617
Quoted 4 lines · On Tue, 5 May 2015 13:39:45 +0800 Cunming Liang <[email protected]>…
lib/librte_pmd_e1000/igb_ethdev.c +4
-0+ pci_dev->intr_handle.intr_vec = +rte_zmalloc ("intr_vec" , + dev_info.max_rx_queues *sizeof (int ),0 ); +
This and other drivers should be using rte_zmalloc_socket to ensure that the intr_vec table is allocated on the same NUMA node as the hardware.
PATCH v7 10/10 l3fwd-power: enable one-shot rx interrupt and polling/interrupt mode switch
In reply to a message not in the archiveDemonstrate how to handle per rx queue interrupt in a NAPI-like implementation in usersapce. PDK polling thread mainly works in polling mode and switch to interrupt mode only if there is no any packet received in recent polls.
Usersapce interrupt notification generally takes a lot more cycles than kernel, so one-shot interrupt is used here to guarantee minimum overhead and DPDK polling thread returns to polling mode immediately once it receives an interrupt notificaiton for incoming packet.
---
v7 changes
- using new APIs
- demo multiple port/queue pair wait on the same epoll instance
v6 changes
- Split event fd add and wait
v5 changes
- Change invoked function name and parameter to accomodate EAL change
v3 changes
- Add spinlock to ensure thread safe when accessing interrupt mask register
v2 changes
- Remove unused function which is for debug purpose
examples/l3fwd-power/main.c | 206 +++++++++++++++++++++++++++++++++++--------- 1 file changed, 164 insertions(+), 42 deletions(-)
examples/l3fwd-power/main.c +164 -42
@@ -74,12 +74,14 @@#include <rte_string_fns.h> #include <rte_timer.h> #include <rte_power.h> +#include <rte_eal.h> +#include <rte_spinlock.h> #define RTE_LOGTYPE_L3FWD_POWER RTE_LOGTYPE_USER1#define MAX_PKT_BURST 32 -#define MIN_ZERO_POLL_COUNT 5 +#define MIN_ZERO_POLL_COUNT 10/* around 100ms at 2 Ghz */ #define TIMER_RESOLUTION_CYCLES 200000000ULL
@@ -155,6 +157,9 @@ static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT;/* ethernet addresses of ports */ static struct ether_addr ports_eth_addr[RTE_MAX_ETHPORTS ]; +/* ethernet addresses of ports */ +static rte_spinlock_t locks[RTE_MAX_ETHPORTS ]; +/* mask of enabled ports */ static uint32_t enabled_port_mask =0 ;/* Ports set in promiscuous mode off by default. */
@@ -187,6 +192,9 @@ struct lcore_rx_queue {#define MAX_TX_QUEUE_PER_PORT RTE_MAX_ETHPORTS#define MAX_RX_QUEUE_PER_PORT 128 +#define MAX_RX_QUEUE_INTERRUPT_PER_PORT 16 + +#define MAX_LCORE_PARAMS 1024struct lcore_params {uint8_t port_id;
@@ -213,7 +221,7 @@ static uint16_t nb_lcore_params = sizeof(lcore_params_array_default) /static struct rte_eth_conf port_conf = { .rxmode = { - .mq_mode =ETH_MQ_RX_RSS , + .mq_mode =ETH_MQ_RX_RSS , .max_rx_pkt_len =ETHER_MAX_LEN , .split_hdr_size =0 , .header_split =0 ,/**< Header Split disabled */
@@ -225,11 +233,14 @@ static struct rte_eth_conf port_conf = { .rx_adv_conf = { .rss_conf = { .rss_key = NULL, - .rss_hf = ETH_RSS_IP, + .rss_hf = ETH_RSS_UDP, }, }, .txmode = { - .mq_mode = ETH_DCB_NONE, + .mq_mode = ETH_MQ_TX_NONE, + }, + .intr_conf = { + .rxq = 1, /**< rxq interrupt feature enabled */ }, };
@@ -401,19 +412,22 @@ power_timer_cb(__attribute__((unused)) struct rte_timer *tim,/* accumulate total execution time in us when callback is invoked */ sleep_time_ratio = (float )(stats[lcore_id].sleep_time) / (float )SCALING_PERIOD ; -/** * check whether need to scale down frequency a step if it sleep a lot. */ -if (sleep_time_ratio >=SCALING_DOWN_TIME_RATIO_THRESHOLD ) -rte_power_freq_down (lcore_id); +if (sleep_time_ratio >=SCALING_DOWN_TIME_RATIO_THRESHOLD ) { +if (rte_power_freq_down) +rte_power_freq_down (lcore_id); + }else if ( (unsigned )(stats[lcore_id].nb_rx_processed / - stats[lcore_id].nb_iteration_looped) <MAX_PKT_BURST ) + stats[lcore_id].nb_iteration_looped) <MAX_PKT_BURST ) {/** * scale down a step if average packet per iteration less * than expectation. */ -rte_power_freq_down (lcore_id); +if (rte_power_freq_down) +rte_power_freq_down (lcore_id); + }/** * initialize another timer according to current frequency to ensure
@@ -706,22 +720,20 @@ l3fwd_simple_forward(struct rte_mbuf *m, uint8_t portid, } -#define SLEEP_GEAR1_THRESHOLD 100 -#define SLEEP_GEAR2_THRESHOLD 1000 +#define MINIMUM_SLEEP_TIME 1 +#define SUSPEND_THRESHOLD 300static inline uint32_t power_idle_heuristic (uint32_t zero_rx_packet_count) { -/* If zero count is less than 100, use it as the sleep time in us */ -if (zero_rx_packet_count <SLEEP_GEAR1_THRESHOLD ) -return zero_rx_packet_count; -/* If zero count is less than 1000, sleep time should be 100 us */ -else if ((zero_rx_packet_count >=SLEEP_GEAR1_THRESHOLD ) && - (zero_rx_packet_count <SLEEP_GEAR2_THRESHOLD )) -return SLEEP_GEAR1_THRESHOLD ; -/* If zero count is greater than 1000, sleep time should be 1000 us */ -else if (zero_rx_packet_count >=SLEEP_GEAR2_THRESHOLD ) -return SLEEP_GEAR2_THRESHOLD ; +/* If zero count is less than 100, sleep 1us */ +if (zero_rx_packet_count <SUSPEND_THRESHOLD ) +return MINIMUM_SLEEP_TIME ; +/* If zero count is less than 1000, sleep 100 us which is the +minimum latency switching from C3/C6 to C0 +*/ +else +return SUSPEND_THRESHOLD ;return 0 ; }
@@ -761,6 +773,84 @@ power_freq_scaleup_heuristic(unsigned lcore_id,return FREQ_CURRENT ; } +/** +* force polling thread sleep until one-shot rx interrupt triggers +* @param port_id +* Port id. +* @param queue_id +* Rx queue id. +* @return +* 0 on success +*/ +static int +sleep_until_rx_interrupt (int num) +{ +struct rte_epoll_event event[num]; +int n, i; +uint8_t port_id, queue_id; +void *data; + +RTE_LOG (INFO ,L3FWD_POWER , +"lcore %u sleeps until interrupt triggers\n" , +rte_lcore_id ()); + + n =rte_epoll_wait (RTE_EPOLL_PER_THREAD , event, num,-1 ); +for (i =0 ; i < n; i++) { + data = event[i].data; + port_id = ((uintptr_t )data) >>CHAR_BIT ; + queue_id = ((uintptr_t )data) & +RTE_LEN2MASK (CHAR_BIT , uint8_t); +RTE_LOG (INFO ,L3FWD_POWER , +"lcore %u is waked up from rx interrupt on" +" port %d queue %d\n" , +rte_lcore_id (), port_id, queue_id); + } + +return 0 ; +} + +static int turn_on_intr (struct lcore_conf *qconf) +{ +int i; +struct lcore_rx_queue *rx_queue; +uint8_t port_id, queue_id; + +for (i =0 ; i < qconf->n_rx_queue; ++i) { + rx_queue = &(qconf->rx_queue_list[i]); + port_id = rx_queue->port_id; + queue_id = rx_queue->queue_id; + +rte_spinlock_lock (&(locks[port_id])); +rte_eth_dev_rx_intr_enable (port_id, queue_id); +rte_spinlock_unlock (&(locks[port_id])); + } +} + +static int event_register (struct lcore_conf *qconf) +{ +struct lcore_rx_queue *rx_queue; +uint8_t portid, queueid; +uint32_t data; +int ret; +int i; + +for (i =0 ; i < qconf->n_rx_queue; ++i) { + rx_queue = &(qconf->rx_queue_list[i]); + portid = rx_queue->port_id; + queueid = rx_queue->queue_id; + data = portid <<CHAR_BIT | queueid; + + ret =rte_eth_dev_rx_intr_ctl_q (portid, queueid, +RTE_EPOLL_PER_THREAD , +RTE_INTR_EVENT_ADD , + (void *)((uintptr_t )data)); +if (ret) +return ret; + } + +return 0 ; +} +/* main processing loop */ static int main_loop (__attribute__((unused))void *dummy)
@@ -774,9 +864,9 @@ main_loop(__attribute__((unused)) void *dummy)struct lcore_conf *qconf;struct lcore_rx_queue *rx_queue;enum freq_scale_hint_t lcore_scaleup_hint; -uint32_t lcore_rx_idle_count =0 ;uint32_t lcore_idle_hint =0 ; +int intr_en =0 ;const uint64_t drain_tsc = (rte_get_tsc_hz () +US_PER_S -1 ) /US_PER_S *BURST_TX_DRAIN_US ;
@@ -793,13 +883,18 @@ main_loop(__attribute__((unused)) void *dummy)RTE_LOG (INFO ,L3FWD_POWER ,"entering main loop on lcore %u\n" , lcore_id);for (i =0 ; i < qconf->n_rx_queue; i++) { - portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id;RTE_LOG (INFO ,L3FWD_POWER ," -- lcoreid=%u portid=%hhu " "rxqueueid=%hhu\n" , lcore_id, portid, queueid); } +/* add into event wait list */ +if (event_register (qconf) ==0 ) + intr_en =1 ; +else +RTE_LOG (INFO ,L3FWD_POWER ,"RX interrupt won't enable.\n" ); +while (1 ) { stats[lcore_id].nb_iteration_looped++;
@@ -834,6 +929,7 @@ main_loop(__attribute__((unused)) void *dummy) prev_tsc_power = cur_tsc_power; } +start_rx:/* * Read packet from RX queues */
@@ -847,6 +943,7 @@ main_loop(__attribute__((unused)) void *dummy) nb_rx =rte_eth_rx_burst (portid, queueid, pkts_burst,MAX_PKT_BURST ); + stats[lcore_id].nb_rx_processed += nb_rx;if (unlikely (nb_rx ==0 )) {/**
@@ -909,10 +1006,13 @@ main_loop(__attribute__((unused)) void *dummy) rx_queue->freq_up_hint; } -if (lcore_scaleup_hint ==FREQ_HIGHEST ) -rte_power_freq_max (lcore_id); -else if (lcore_scaleup_hint ==FREQ_HIGHER ) -rte_power_freq_up (lcore_id); +if (lcore_scaleup_hint ==FREQ_HIGHEST ) { +if (rte_power_freq_max) +rte_power_freq_max (lcore_id); + }else if (lcore_scaleup_hint ==FREQ_HIGHER ) { +if (rte_power_freq_up) +rte_power_freq_up (lcore_id); + } }else {/** * All Rx queues empty in recent consecutive polls,
@@ -927,16 +1027,23 @@ main_loop(__attribute__((unused)) void *dummy) lcore_idle_hint = rx_queue->idle_hint; } -if ( lcore_idle_hint <SLEEP_GEAR1_THRESHOLD ) +if (lcore_idle_hint <SUSPEND_THRESHOLD )/** -* execute "pause" instruction to avoid context -* switch for short sleep. -*/ +* execute "pause" instruction to avoid context +* switch which generally take hundres of +* microsecond for short sleep. +*/ rte_delay_us (lcore_idle_hint); -else -/* long sleep force runing thread to suspend */ -usleep (lcore_idle_hint); - +else { +/* suspend untill rx interrupt trigges */ +if (intr_en) { +turn_on_intr (qconf); +sleep_until_rx_interrupt ( + qconf->n_rx_queue); + } +/* start receiving packets immediately */ + goto start_rx; + } stats[lcore_id].sleep_time += lcore_idle_hint; } }
@@ -1269,7 +1376,7 @@ setup_hash(int socketid)char s[64 ];/* create ipv4 hash */ -snprintf (s,sizeof (s),"ipv4_l3fwd_hash_%d" , socketid); +rte_snprintf (s,sizeof (s),"ipv4_l3fwd_hash_%d" , socketid); ipv4_l3fwd_hash_params.name = s; ipv4_l3fwd_hash_params.socket_id = socketid; ipv4_l3fwd_lookup_struct[socketid] =
@@ -1279,7 +1386,7 @@ setup_hash(int socketid) "socket %d\n", socketid);/* create ipv6 hash */ -snprintf (s,sizeof (s),"ipv6_l3fwd_hash_%d" , socketid); +rte_snprintf (s,sizeof (s),"ipv6_l3fwd_hash_%d" , socketid); ipv6_l3fwd_hash_params.name = s; ipv6_l3fwd_hash_params.socket_id = socketid; ipv6_l3fwd_lookup_struct[socketid] =
@@ -1472,6 +1579,7 @@ main(int argc, char **argv)unsigned lcore_id;uint64_t hz;uint32_t n_tx_queue, nb_lcores; +uint32_t dev_rxq_num, dev_txq_num;uint8_t portid, nb_rx_queue, queue, socketid;/* catch SIGINT and restore cpufreq governor to ondemand */
@@ -1521,10 +1629,19 @@ main(int argc, char **argv)printf ("Initializing port %d ... " , portid );fflush (stdout); +rte_eth_dev_info_get (portid, &dev_info); + dev_rxq_num = dev_info.max_rx_queues; + dev_txq_num = dev_info.max_tx_queues; + nb_rx_queue =get_port_n_rx_queues (portid); +if (nb_rx_queue > dev_rxq_num) +rte_exit (EXIT_FAILURE , +"Cannot configure not existed rxq: " +"port=%d\n" , portid); + n_tx_queue = nb_lcores; -if (n_tx_queue >MAX_TX_QUEUE_PER_PORT ) - n_tx_queue =MAX_TX_QUEUE_PER_PORT ; +if (n_tx_queue > dev_txq_num) + n_tx_queue = dev_txq_num;printf ("Creating queues: nb_rxq=%d nb_txq=%u... " , nb_rx_queue, (unsigned )n_tx_queue ); ret = rte_eth_dev_configure(portid, nb_rx_queue,
@@ -1548,6 +1665,9 @@ main(int argc, char **argv)if (rte_lcore_is_enabled (lcore_id) ==0 )continue ; +if (queueid >= dev_txq_num) +continue ; +if (numa_on) socketid = \ (uint8_t )rte_lcore_to_socket_id (lcore_id);
@@ -1582,8 +1702,9 @@ main(int argc, char **argv)/* init power management library */ ret =rte_power_init (lcore_id);if (ret) -rte_exit (EXIT_FAILURE ,"Power management library " -"initialization failed on core%u\n" , lcore_id); +rte_log (RTE_LOG_ERR ,RTE_LOGTYPE_POWER , +"Power management library initialization " +"failed on core%u" , lcore_id);/* init timer structures for each enabled lcore */ rte_timer_init (&power_timers[lcore_id]);
@@ -1631,7 +1752,6 @@ main(int argc, char **argv)if (ret <0 )rte_exit (EXIT_FAILURE ,"rte_eth_dev_start: err=%d, " "port=%d\n" , ret, portid); -/* * If enabled, put device in promiscuous mode. * This allows IO forwarding mode to forward packets
@@ -1640,6 +1760,8 @@ main(int argc, char **argv)*/ if (promiscuous_on)rte_eth_promiscuous_enable (portid); +/* initialize spinlock for each port */ +rte_spinlock_init (&(locks[portid])); }check_all_ports_link_status ((uint8_t )nb_ports, enabled_port_mask);
Signature · 1 line
1.8.1.4
v8 cover Interrupt mode PMD
In reply to Cunming Liangv8 changes
- remove condition check for only vfio-msix
- add multiplex intr support when only one intr vector allowed
- lsc and rxq interrupt runtime enable decision
- add safe event delete while the event wakeup execution happens
v7 changes
- decouple epoll event and intr operation
- add condition check in the case intr vector is disabled
- renaming some APIs
v6 changes
- split rte_intr_wait_rx_pkt into two APIs 'wait' and 'set'.
- rewrite rte_intr_rx_wait/rte_intr_rx_set.
- using vector number instead of queue_id as interrupt API params.
- patch reorder and split.
v5 changes
- Rebase the patchset onto the HEAD
- Isolate ethdev from EAL for new-added wait-for-rx interrupt function
- Export wait-for-rx interrupt function for shared libraries
- Split-off a new patch file for changed struct rte_intr_handle that other patches depend on, to avoid breaking git bisect
- Change sample applicaiton to accomodate EAL function spec change accordingly
v4 changes
- Export interrupt enable/disable functions for shared libraries
- Adjust position of new-added structure fields and functions to avoid breaking ABI
v3 changes
- Add return value for interrupt enable/disable functions
- Move spinlok from PMD to L3fwd-power
- Remove unnecessary variables in e1000_mac_info
- Fix miscelleous review comments
v2 changes
- Fix compilation issue in Makefile for missed header file.
- Consolidate internal and community review comments of v1 patch set.
The patch series introduce low-latency one-shot rx interrupt into DPDK with polling and interrupt mode switch control example.
DPDK userspace interrupt notification and handling mechanism is based on UIO with below limitation:
1) It is designed to handle LSC interrupt only with inefficient suspended pthread wakeup procedure (e.g. UIO wakes up LSC interrupt handling thread which then wakes up DPDK polling thread). In this way, it introduces non-deterministic wakeup latency for DPDK polling thread as well as packet latency if it is used to handle Rx interrupt.
2) UIO only supports a single interrupt vector which has to been shared by LSC interrupt and interrupts assigned to dedicated rx queues.
This patchset includes below features:
1) Enable one-shot rx queue interrupt in ixgbe PMD(PF & VF) and igb PMD(PF only).
2) Build on top of the VFIO mechanism instead of UIO, so it could support up to 64 interrupt vectors for rx queue interrupts.
3) Have 1 DPDK polling thread handle per Rx queue interrupt with a dedicated VFIO eventfd, which eliminates non-deterministic pthread wakeup latency in user space.
4) Demonstrate interrupts control APIs and userspace NAIP-like polling/interrupt switch algorithms in L3fwd-power example.
Known limitations:
1) It does not work for UIO due to a single interrupt eventfd shared by LSC and rx queue interrupt handlers causes a mess.
2) LSC interrupt is not supported by VF driver, so it is by default disabled in L3fwd-power now. Feel free to turn in on if you want to support both LSC and rx queue interrupts on a PF.
Cunming Liang (11):
eal/linux: add interrupt vectors support in intr_handle eal/linux: add rte_epoll_wait/ctl support
eal/linux: add API to set rx interrupt event monitor eal/linux: fix comments typo on vfio msi
eal/linux: add interrupt vectors handling on VFIO eal/linux: standalone intr event fd create support eal/bsd: dummy for new intr definition
ethdev: add rx intr enable, disable and ctl functions
ixgbe: enable rx queue interrupts for both PF and VF
igb: enable rx queue interrupts for PF
l3fwd-power: enable one-shot rx interrupt and polling/interrupt mode
switch
examples/l3fwd-power/main.c | 207 +++++++-- lib/librte_eal/bsdapp/eal/eal_interrupts.c | 20 + .../bsdapp/eal/include/exec-env/rte_interrupts.h | 77 ++++ lib/librte_eal/bsdapp/eal/rte_eal_version.map | 5 + lib/librte_eal/linuxapp/eal/eal_interrupts.c | 351 +++++++++++++-- .../linuxapp/eal/include/exec-env/rte_interrupts.h | 160 +++++++ lib/librte_eal/linuxapp/eal/rte_eal_version.map | 8 + lib/librte_ether/rte_ethdev.c | 127 ++++++ lib/librte_ether/rte_ethdev.h | 104 +++++ lib/librte_ether/rte_ether_version.map | 4 + lib/librte_pmd_e1000/igb_ethdev.c | 292 +++++++++++-- lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 482 ++++++++++++++++++++- lib/librte_pmd_ixgbe/ixgbe_ethdev.h | 4 + 13 files changed, 1715 insertions(+), 126 deletions(-)
Signature · 1 line
1.8.1.4
PATCH v8 1/11 eal/linux: add interrupt vectors support in intr_handle
The patch adds interrupt vectors support in rte_intr_handle.
'vec_en' is set when interrupt vectors are detected and associated event fds are set. Those event fds are stored in efds[].
'intr_vec' is reserved for device driver to initialize the vector mapping table.
When the event fds add to a specified epoll instance, 'elist' will hold the rte_epoll_event object pointer.
---
v7 changes:
- add eptrs[], it's used to store the register rte_epoll_event instances.
- add vec_en, to log the vector capability status.
v6 changes:
- add mapping table between irq vector number and queue id.
v5 changes:
- Create this new patch file for changed struct rte_intr_handle that other patches depend on, to avoid breaking git bisect.
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h | 10 ++++++++++ 1 file changed, 10 insertions(+)
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +10 -0
@@ -38,6 +38,8 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define RTE_MAX_RXTX_INTR_VEC_ID 32 +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */
@@ -48,6 +50,8 @@ enum rte_intr_handle_type {RTE_INTR_HANDLE_MAX }; +struct rte_epoll_event ; +/** Handle for interrupts. */ struct rte_intr_handle { union {
@@ -57,6 +61,12 @@ struct rte_intr_handle { };int fd;/**< interrupt event file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +uint32_t max_intr;/**< max interrupt requested */ +uint32_t nb_efd;/**< number of available efds */ +int efds[RTE_MAX_RXTX_INTR_VEC_ID ];/**< intr vectors/efds mapping */ +struct rte_epoll_event *elist[RTE_MAX_RXTX_INTR_VEC_ID ]; +/**< intr vector epoll event ptr */ +int *intr_vec;/**< intr vector number array */ };#endif /* _RTE_LINUXAPP_INTERRUPTS_H_ */
Signature · 1 line
1.8.1.4
Re: [PATCH v8 01/11] eal/linux: add interrupt vectors support in intr_handle
Quoted 47 lines · Cunming Liang
The patch adds interrupt vectors support in
rte_intr_handle.
'vec_en' is set when interrupt vectors are detected and associated event fds are set. Those event fds are stored in efds[].
'intr_vec' is reserved for device driver to initialize the vector mapping table.
When the event fds add to a specified epoll instance, 'elist' will hold therte_epoll_eventobject pointer.Signed-off-by Danny Zhou <[email protected]>Signed-off-by Cunming Liang <[email protected]>---
v7 changes:
- add eptrs[], it's used to store the registerrte_epoll_eventinstances.
- addvec_en, to log the vector capability status.v6 changes:
- add mapping table between irq vector number and queue id.v5 changes:
- Create this new patch file for changed structrte_intr_handlethat other patches depend on, to avoid breaking git bisect.lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h | 10 ++++++++++ 1 file changed, 10 insertions(+)lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +4
-0@@ -38,6 +38,8 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define RTE_MAX_RXTX_INTR_VEC_ID 32 +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */@@ -48,6 +50,8 @@ enum rte_intr_handle_type {RTE_INTR_HANDLE_MAX }; +struct rte_epoll_event ; +/** Handle for interrupts. */ struct rte_intr_handle { union {@@ -57,6 +61,12 @@ struct rte_intr_handle { };
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h struct rte_intr_handle { · line 71
Quoted 10 lines
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +6
-0int fd;/**< interrupt event file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +uint32_t max_intr;/**< max interrupt requested */ +uint32_t nb_efd;/**< number of available efds */ +int efds[RTE_MAX_RXTX_INTR_VEC_ID ];/**< intr vectors/efds mapping */ +struct rte_epoll_event *elist[RTE_MAX_RXTX_INTR_VEC_ID ]; +/**< intr vector epoll event ptr */ +int *intr_vec;/**< intr vector number array */ };
This is going to be ABI breaking if this from test_interrupts.c:
static struct rte_intr_handle intr_handles[TEST_INTERRUPT_HANDLE_MAX];
is a plausible way of using this structure. Even putting the data at the end of the structure won't help, as the array indicies are off
Neil
Quoted 5 lines
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +0
-0#endif /* _RTE_LINUXAPP_INTERRUPTS_H_ */ Signature · 1 line
1.8.1.4
A message that is not in the archive (sent off-list or not yet received): 20150521104300.00757b4e@urahara
Re: [PATCH v8 01/11] eal/linux: add interrupt vectors support in intr_handle
Quoted 65 lines · Stephen Hemminger
Quoted 62 lines · On Thu, 21 May 2015 06:32:02 -0400 Neil Horman <[email protected]> wrote:
Quoted 47 lines · Cunming Liang
The patch adds interrupt vectors support in
rte_intr_handle.
'vec_en' is set when interrupt vectors are detected and associated event fds are set. Those event fds are stored in efds[].
'intr_vec' is reserved for device driver to initialize the vector mapping table.
When the event fds add to a specified epoll instance, 'elist' will hold therte_epoll_eventobject pointer.Signed-off-by Danny Zhou <[email protected]>Signed-off-by Cunming Liang <[email protected]>---
v7 changes:
- add eptrs[], it's used to store the registerrte_epoll_eventinstances.
- addvec_en, to log the vector capability status.v6 changes:
- add mapping table between irq vector number and queue id.v5 changes:
- Create this new patch file for changed structrte_intr_handlethat other patches depend on, to avoid breaking git bisect.lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h | 10 ++++++++++ 1 file changed, 10 insertions(+)lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +4
-0@@ -38,6 +38,8 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define RTE_MAX_RXTX_INTR_VEC_ID 32 +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */@@ -48,6 +50,8 @@ enum rte_intr_handle_type {RTE_INTR_HANDLE_MAX }; +struct rte_epoll_event ; +/** Handle for interrupts. */ struct rte_intr_handle { union {@@ -57,6 +61,12 @@ struct rte_intr_handle { };lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h struct rte_intr_handle { · line 71
Quoted 10 lines
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +6
-0int fd;/**< interrupt event file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +uint32_t max_intr;/**< max interrupt requested */ +uint32_t nb_efd;/**< number of available efds */ +int efds[RTE_MAX_RXTX_INTR_VEC_ID ];/**< intr vectors/efds mapping */ +struct rte_epoll_event *elist[RTE_MAX_RXTX_INTR_VEC_ID ]; +/**< intr vector epoll event ptr */ +int *intr_vec;/**< intr vector number array */ };This is going to be ABI breaking if this from
test_interrupts.c:
static structrte_intr_handleintr_handles[TEST_INTERRUPT_HANDLE_MAX];
Quoted 5 lines
Quoted 2 lines
is a plausible way of using this structure. Even putting the data at the end of the structure won't help, as the array indicies are off
This needs to go in 2.0 and 2.0 has to have new ABI anyway.
We've already released 2.0, I think you mean 2.1, but 2.1 can't have a new ABI because we didn't announce it in 1.8. The earliest we can update the ABI (according to the ABI docs) at this point is 2.2, since we need to announce the change in 2.1, then make it in 2.2
Neil
Re: [PATCH v8 01/11] eal/linux: add interrupt vectors support in intr_handle
Quoted 69 lines · On Thu, 21 May 2015 13:58:46 -0400 Neil Horman <[email protected]> wrote:
Quoted 68 lines · Stephen Hemminger
Quoted 64 lines · On Thu, 21 May 2015 06:32:02 -0400 Neil Horman <[email protected]> wrote:
Quoted 47 lines · Cunming Liang
The patch adds interrupt vectors support in
rte_intr_handle.
'vec_en' is set when interrupt vectors are detected and associated event fds are set. Those event fds are stored in efds[].
'intr_vec' is reserved for device driver to initialize the vector mapping table.
When the event fds add to a specified epoll instance, 'elist' will hold therte_epoll_eventobject pointer.Signed-off-by Danny Zhou <[email protected]>Signed-off-by Cunming Liang <[email protected]>---
v7 changes:
- add eptrs[], it's used to store the registerrte_epoll_eventinstances.
- addvec_en, to log the vector capability status.v6 changes:
- add mapping table between irq vector number and queue id.v5 changes:
- Create this new patch file for changed structrte_intr_handlethat other patches depend on, to avoid breaking git bisect.lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h | 10 ++++++++++ 1 file changed, 10 insertions(+)lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +4
-0@@ -38,6 +38,8 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define RTE_MAX_RXTX_INTR_VEC_ID 32 +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */@@ -48,6 +50,8 @@ enum rte_intr_handle_type {RTE_INTR_HANDLE_MAX }; +struct rte_epoll_event ; +/** Handle for interrupts. */ struct rte_intr_handle { union {@@ -57,6 +61,12 @@ struct rte_intr_handle { };lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h struct rte_intr_handle { · line 71
Quoted 10 lines
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +6
-0int fd;/**< interrupt event file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +uint32_t max_intr;/**< max interrupt requested */ +uint32_t nb_efd;/**< number of available efds */ +int efds[RTE_MAX_RXTX_INTR_VEC_ID ];/**< intr vectors/efds mapping */ +struct rte_epoll_event *elist[RTE_MAX_RXTX_INTR_VEC_ID ]; +/**< intr vector epoll event ptr */ +int *intr_vec;/**< intr vector number array */ };This is going to be ABI breaking if this from
test_interrupts.c:
static structrte_intr_handleintr_handles[TEST_INTERRUPT_HANDLE_MAX];is a plausible way of using this structure. Even putting the data at the end of the structure won't help, as the array indicies are off
Quoted 9 lines
Quoted 2 lines
This needs to go in 2.0 and 2.0 has to have new ABI anyway.
We've already released 2.0, I think you mean 2.1, but 2.1 can't have a new ABI because we didn't announce it in 1.8. The earliest we can update the ABI (according to the ABI docs) at this point is 2.2, since we need to announce the change in 2.1, then make it in 2.2
Neil
Then just skip 2.1 (or make it a trivial doc change only dummy release), and call it 2.2.
I guess we need to proactively say every .x release will have new ABI. Sorry, this is a project under development.
A message that is not in the archive (sent off-list or not yet received): 20150521111400.2a04a196@urahara
Re: [PATCH v8 01/11] eal/linux: add interrupt vectors support in intr_handle
Quoted 79 lines · Stephen Hemminger
Quoted 76 lines · On Thu, 21 May 2015 13:58:46 -0400 Neil Horman <[email protected]> wrote:
Quoted 65 lines · Stephen Hemminger
Quoted 62 lines · On Thu, 21 May 2015 06:32:02 -0400 Neil Horman <[email protected]> wrote:
Quoted 47 lines · Cunming Liang
The patch adds interrupt vectors support in
rte_intr_handle.
'vec_en' is set when interrupt vectors are detected and associated event fds are set. Those event fds are stored in efds[].
'intr_vec' is reserved for device driver to initialize the vector mapping table.
When the event fds add to a specified epoll instance, 'elist' will hold therte_epoll_eventobject pointer.Signed-off-by Danny Zhou <[email protected]>Signed-off-by Cunming Liang <[email protected]>---
v7 changes:
- add eptrs[], it's used to store the registerrte_epoll_eventinstances.
- addvec_en, to log the vector capability status.v6 changes:
- add mapping table between irq vector number and queue id.v5 changes:
- Create this new patch file for changed structrte_intr_handlethat other patches depend on, to avoid breaking git bisect.lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h | 10 ++++++++++ 1 file changed, 10 insertions(+)lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +4
-0@@ -38,6 +38,8 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define RTE_MAX_RXTX_INTR_VEC_ID 32 +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */@@ -48,6 +50,8 @@ enum rte_intr_handle_type {RTE_INTR_HANDLE_MAX }; +struct rte_epoll_event ; +/** Handle for interrupts. */ struct rte_intr_handle { union {@@ -57,6 +61,12 @@ struct rte_intr_handle { };lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h struct rte_intr_handle { · line 71
Quoted 10 lines
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +6
-0int fd;/**< interrupt event file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +uint32_t max_intr;/**< max interrupt requested */ +uint32_t nb_efd;/**< number of available efds */ +int efds[RTE_MAX_RXTX_INTR_VEC_ID ];/**< intr vectors/efds mapping */ +struct rte_epoll_event *elist[RTE_MAX_RXTX_INTR_VEC_ID ]; +/**< intr vector epoll event ptr */ +int *intr_vec;/**< intr vector number array */ };This is going to be ABI breaking if this from
test_interrupts.c:
static structrte_intr_handleintr_handles[TEST_INTERRUPT_HANDLE_MAX];Quoted 5 lines
Quoted 2 lines
is a plausible way of using this structure. Even putting the data at the end of the structure won't help, as the array indicies are off
This needs to go in 2.0 and 2.0 has to have new ABI anyway.
We've already released 2.0, I think you mean 2.1, but 2.1 can't have a new ABI because we didn't announce it in 1.8. The earliest we can update the ABI (according to the ABI docs) at this point is 2.2, since we need to announce the change in 2.1, then make it in 2.2
Quoted 9 lines
Quoted 2 lines
Neil
Then just skip 2.1 (or make it a trivial doc change only dummy release), and call it 2.2.
I guess we need to proactively say every .x release will have new ABI. Sorry, this is a project under development.
Sorry, NAK. I didn't go through all the trouble of creating an ABI infrastructure just to throw it out the window on some rubber stamp. We decided on the rules, we need to stick to them. We have large projects that rely on DPDK now (OVS primarily), and we owe it to them to not just go completely throw out the ABI every release. We have a process for doing it, lets follow it.
Neil
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH v8 01/11] eal/linux: add interrupt vectors support in intr_handle
Quoted 89 lines · On Fri, 22 May 2015 00:05:36 +0000 Neil Horman <[email protected]> wrote:
Quoted 79 lines · Stephen Hemminger
Quoted 76 lines · On Thu, 21 May 2015 13:58:46 -0400 Neil Horman <[email protected]> wrote:
Quoted 65 lines · Stephen Hemminger
Quoted 62 lines · On Thu, 21 May 2015 06:32:02 -0400 Neil Horman <[email protected]> wrote:
Quoted 47 lines · Cunming Liang
The patch adds interrupt vectors support in
rte_intr_handle.
'vec_en' is set when interrupt vectors are detected and associated event fds are set. Those event fds are stored in efds[].
'intr_vec' is reserved for device driver to initialize the vector mapping table.
When the event fds add to a specified epoll instance, 'elist' will hold therte_epoll_eventobject pointer.Signed-off-by Danny Zhou <[email protected]>Signed-off-by Cunming Liang <[email protected]>---
v7 changes:
- add eptrs[], it's used to store the registerrte_epoll_eventinstances.
- addvec_en, to log the vector capability status.v6 changes:
- add mapping table between irq vector number and queue id.v5 changes:
- Create this new patch file for changed structrte_intr_handlethat other patches depend on, to avoid breaking git bisect.lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h | 10 ++++++++++ 1 file changed, 10 insertions(+)lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +4
-0@@ -38,6 +38,8 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define RTE_MAX_RXTX_INTR_VEC_ID 32 +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */@@ -48,6 +50,8 @@ enum rte_intr_handle_type {RTE_INTR_HANDLE_MAX }; +struct rte_epoll_event ; +/** Handle for interrupts. */ struct rte_intr_handle { union {@@ -57,6 +61,12 @@ struct rte_intr_handle { };lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h struct rte_intr_handle { · line 71
Quoted 10 lines
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +6
-0int fd;/**< interrupt event file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +uint32_t max_intr;/**< max interrupt requested */ +uint32_t nb_efd;/**< number of available efds */ +int efds[RTE_MAX_RXTX_INTR_VEC_ID ];/**< intr vectors/efds mapping */ +struct rte_epoll_event *elist[RTE_MAX_RXTX_INTR_VEC_ID ]; +/**< intr vector epoll event ptr */ +int *intr_vec;/**< intr vector number array */ };This is going to be ABI breaking if this from
test_interrupts.c:
static structrte_intr_handleintr_handles[TEST_INTERRUPT_HANDLE_MAX];Quoted 5 lines
Quoted 2 lines
is a plausible way of using this structure. Even putting the data at the end of the structure won't help, as the array indicies are off
This needs to go in 2.0 and 2.0 has to have new ABI anyway.
We've already released 2.0, I think you mean 2.1, but 2.1 can't have a new ABI because we didn't announce it in 1.8. The earliest we can update the ABI (according to the ABI docs) at this point is 2.2, since we need to announce the change in 2.1, then make it in 2.2
Quoted 9 lines
Quoted 2 lines
Neil
Then just skip 2.1 (or make it a trivial doc change only dummy release), and call it 2.2.
I guess we need to proactively say every .x release will have new ABI. Sorry, this is a project under development.
Quoted 8 lines
Sorry, NAK. I didn't go through all the trouble of creating an ABI infrastructure just to throw it out the window on some rubber stamp. We decided on the rules, we need to stick to them. We have large projects that rely on DPDK now (OVS primarily), and we owe it to them to not just go completely throw out the ABI every release. We have a process for doing it, lets follow it.
Neil
I meant, that close and ship existing 2.1 code base early and open 2.2 early to keep things rolling. But in general this project needs x.x.y releases with ABI stability, and just admit that x.x releases will not have stable ABI. That is reality now.
A lot of the ABI problem is that the code does not do a good job of hiding. And also does not sepearte driver ABI from user ABI. There are things like structure of PCI and interrupt handles that the user from library point of view should not care about, but drivers will need to.
Re: [PATCH v8 01/11] eal/linux: add interrupt vectors support in intr_handle
Quoted 98 lines · Stephen Hemminger
Quoted 95 lines · On Fri, 22 May 2015 00:05:36 +0000 Neil Horman <[email protected]> wrote:
Quoted 79 lines · Stephen Hemminger
Quoted 76 lines · On Thu, 21 May 2015 13:58:46 -0400 Neil Horman <[email protected]> wrote:
Quoted 65 lines · Stephen Hemminger
Quoted 62 lines · On Thu, 21 May 2015 06:32:02 -0400 Neil Horman <[email protected]> wrote:
Quoted 47 lines · Cunming Liang
The patch adds interrupt vectors support in
rte_intr_handle.
'vec_en' is set when interrupt vectors are detected and associated event fds are set. Those event fds are stored in efds[].
'intr_vec' is reserved for device driver to initialize the vector mapping table.
When the event fds add to a specified epoll instance, 'elist' will hold therte_epoll_eventobject pointer.Signed-off-by Danny Zhou <[email protected]>Signed-off-by Cunming Liang <[email protected]>---
v7 changes:
- add eptrs[], it's used to store the registerrte_epoll_eventinstances.
- addvec_en, to log the vector capability status.v6 changes:
- add mapping table between irq vector number and queue id.v5 changes:
- Create this new patch file for changed structrte_intr_handlethat other patches depend on, to avoid breaking git bisect.lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h | 10 ++++++++++ 1 file changed, 10 insertions(+)lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +4
-0@@ -38,6 +38,8 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define RTE_MAX_RXTX_INTR_VEC_ID 32 +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */@@ -48,6 +50,8 @@ enum rte_intr_handle_type {RTE_INTR_HANDLE_MAX }; +struct rte_epoll_event ; +/** Handle for interrupts. */ struct rte_intr_handle { union {@@ -57,6 +61,12 @@ struct rte_intr_handle { };lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h struct rte_intr_handle { · line 71
Quoted 10 lines
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +6
-0int fd;/**< interrupt event file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +uint32_t max_intr;/**< max interrupt requested */ +uint32_t nb_efd;/**< number of available efds */ +int efds[RTE_MAX_RXTX_INTR_VEC_ID ];/**< intr vectors/efds mapping */ +struct rte_epoll_event *elist[RTE_MAX_RXTX_INTR_VEC_ID ]; +/**< intr vector epoll event ptr */ +int *intr_vec;/**< intr vector number array */ };This is going to be ABI breaking if this from
test_interrupts.c:
static structrte_intr_handleintr_handles[TEST_INTERRUPT_HANDLE_MAX];Quoted 5 lines
Quoted 2 lines
is a plausible way of using this structure. Even putting the data at the end of the structure won't help, as the array indicies are off
This needs to go in 2.0 and 2.0 has to have new ABI anyway.
We've already released 2.0, I think you mean 2.1, but 2.1 can't have a new ABI because we didn't announce it in 1.8. The earliest we can update the ABI (according to the ABI docs) at this point is 2.2, since we need to announce the change in 2.1, then make it in 2.2
Quoted 9 lines
Quoted 2 lines
Neil
Then just skip 2.1 (or make it a trivial doc change only dummy release), and call it 2.2.
I guess we need to proactively say every .x release will have new ABI. Sorry, this is a project under development.
Sorry, NAK. I didn't go through all the trouble of creating an ABI infrastructure just to throw it out the window on some rubber stamp. We decided on the rules, we need to stick to them. We have large projects that rely on DPDK now (OVS primarily), and we owe it to them to not just go completely throw out the ABI every release. We have a process for doing it, lets follow it.
Quoted 8 lines
Quoted 2 lines
Neil
I meant, that close and ship existing 2.1 code base early and open 2.2 early to keep things rolling. But in general this project needs x.x.y releases with ABI stability, and just admit that x.x releases will not have stable ABI. That is reality now.
I'm not opposed to doing that, though the purpose of the proposed cadence was to give sufficient notice to downstream consumers of DPDK that ABI changes were comming. As long as the time delta beweeen 2.X and 2.X+1 is sufficient for consumers to have time to react and update their applications I'm ok with it (which I know is subjective, but I'm willing to experiment there).
Quoted 5 lines
A lot of the ABI problem is that the code does not do a good job of hiding. And also does not sepearte driver ABI from user ABI. There are things like structure of PCI and interrupt handles that the user from library point of view should not care about, but drivers will need to.
I agree. I had hoped that implementing an ABI process would help drive improvements in this area, but it hasn't seemed to yet.
Neil
Quoted 2 lines
Re: [PATCH v8 01/11] eal/linux: add interrupt vectors support in intr_handle
In reply to Neil HormanHi Neil,
Quoted 67 lines · Neil Horman
Quoted 66 lines · Stephen Hemminger
Quoted 63 lines · On Thu, 21 May 2015 06:32:02 -0400 Neil Horman <[email protected]> wrote:
Quoted 47 lines · Cunming Liang
The patch adds interrupt vectors support in
rte_intr_handle.
'vec_en' is set when interrupt vectors are detected and associated event fds are set. Those event fds are stored in efds[].
'intr_vec' is reserved for device driver to initialize the vector mapping table.
When the event fds add to a specified epoll instance, 'elist' will hold therte_epoll_eventobject pointer.Signed-off-by Danny Zhou <[email protected]>Signed-off-by Cunming Liang <[email protected]>---
v7 changes:
- add eptrs[], it's used to store the registerrte_epoll_eventinstances.
- addvec_en, to log the vector capability status.v6 changes:
- add mapping table between irq vector number and queue id.v5 changes:
- Create this new patch file for changed structrte_intr_handlethat other patches depend on, to avoid breaking git bisect.lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h | 10 ++++++++++ 1 file changed, 10 insertions(+)lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +4
-0@@ -38,6 +38,8 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define RTE_MAX_RXTX_INTR_VEC_ID 32 +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */@@ -48,6 +50,8 @@ enum rte_intr_handle_type {RTE_INTR_HANDLE_MAX }; +struct rte_epoll_event ; +/** Handle for interrupts. */ struct rte_intr_handle { union {@@ -57,6 +61,12 @@ struct rte_intr_handle { };lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h struct rte_intr_handle { · line 71
Quoted 10 lines
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +6
-0int fd;/**< interrupt event file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +uint32_t max_intr;/**< max interrupt requested */ +uint32_t nb_efd;/**< number of available efds */ +int efds[RTE_MAX_RXTX_INTR_VEC_ID ];/**< intr vectors/efds mapping */ +struct rte_epoll_event *elist[RTE_MAX_RXTX_INTR_VEC_ID ]; +/**< intr vector epoll event ptr */ +int *intr_vec;/**< intr vector number array */ };This is going to be ABI breaking if this from
test_interrupts.c:
static structrte_intr_handleintr_handles[TEST_INTERRUPT_HANDLE_MAX];is a plausible way of using this structure. Even putting the data at the end of the structure won't help, as the array indicies are off
Quoted 9 lines
Quoted 2 lines
This needs to go in 2.0 and 2.0 has to have new ABI anyway.
We've already released 2.0, I think you mean 2.1, but 2.1 can't have a new ABI because we didn't announce it in 1.8. The earliest we can update the ABI (according to the ABI docs) at this point is 2.2, since we need to announce the change in 2.1, then make it in 2.2
Neil
I'll follow your guidance to send a separate patch to announce the ABI changes in this release.
For this code patch series, I propose to turn off the whole feature by default so as to avoid ABI broken in this release.
On next release v2.2, I'll send another cleanup patch to remove the feature macro.
In this way, we either won't block the feature code review or won't break ABI.
For users who are still going to use the feature in v2.1, they shall make sure aware of the impact of ABI changes and take risk to turn on the feature manually. The v9 patch will include this part. Does it sound good to you?
Thanks,
Steve
PATCH v8 2/11 eal/linux: add rte_epoll_wait/ctl support
In reply to Cunming LiangThe patch adds 'rte_epoll_wait' and 'rte_epoll_ctl' for async event wakeup.
It defines 'struct rte_epoll_event' as the event param.
The 'op' uses the same enum as epoll_wait/ctl does.
The epoll event support to carry a raw user data and to register a callback which is exectuted during wakeup.
---
v8 changes
- support delete event in safety during the wakeup execution
- add EINTR process during epoll_wait
v7 changes
- split v6[4/8] into two patches, one for epoll event(this one) another for rx intr(next patch)
- introduce rte_epoll_event definition
- rte_epoll_wait/ctl for more generic RTE epoll API
v6 changes
- split rte_intr_wait_rx_pkt into two function, wait and set.
- rewrite rte_intr_rx_wait/rte_intr_rx_set to remove queue visibility on eal.
- rte_intr_rx_wait to support multiplexing.
- allow epfd as input to support flexible event fd combination.
lib/librte_eal/linuxapp/eal/eal_interrupts.c | 137 +++++++++++++++++++++ .../linuxapp/eal/include/exec-env/rte_interrupts.h | 82 +++++++++++- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 3 + 3 files changed, 219 insertions(+), 3 deletions(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +137 -0
@@ -69,6 +69,8 @@#define EAL_INTR_EPOLL_WAIT_FOREVER (-1) +static RTE_DEFINE_PER_LCORE(int, _epfd) = -1; /**< epoll fd per thread */ +/** * union for pipe fds. */
@@ -859,3 +861,138 @@ rte_eal_intr_init(void)return -ret; } +static int +eal_epoll_process_event (struct epoll_event *evs,int n, +struct rte_epoll_event *events) +{ +int i; +int count =0 ; +struct rte_epoll_event *rev; +for (i =0 ; i < n; i++) { + rev = (struct rte_epoll_event *)evs[i].data.ptr; +if (!rev || !rte_atomic32_cmpset (&rev->status,RTE_EPOLL_VALID , +RTE_EPOLL_EXEC )) +continue ; + + events[count].status =RTE_EPOLL_VALID ; + events[count].fd = rev->fd; + events[count].epfd = rev->epfd; + events[count].epdata.event = rev->epdata.event; + events[count].epdata.data = rev->epdata.data; +if (rev->epdata.cb_fun) + rev->epdata.cb_fun (rev->fd, + rev->epdata.cb_arg); + +rte_compiler_barrier (); + rev->status =RTE_EPOLL_VALID ; + count++; + } +return count; +} + +static inline int +eal_init_tls_epfd (void ) +{ +int pfd =epoll_create (255 ); +if (pfd <0 ) { +RTE_LOG (ERR ,EAL , +"Cannot create epoll instance\n" ); +return -1 ; + } +return pfd; +} + +int +rte_intr_tls_epfd (void ) +{ +if (RTE_PER_LCORE (_epfd) ==-1 ) +RTE_PER_LCORE (_epfd) =eal_init_tls_epfd (); + +return RTE_PER_LCORE (_epfd); +} + +int +rte_epoll_wait (int epfd,struct rte_epoll_event *events, +int maxevents,int timeout) +{ +struct epoll_event evs[maxevents]; +int rc; + +if (!events) { +RTE_LOG (ERR ,EAL ,"rte_epoll_event can't be NULL\n" ); +return -1 ; + } + +/* using per thread epoll fd */ +if (epfd ==RTE_EPOLL_PER_THREAD ) + epfd =rte_intr_tls_epfd (); + +while (1 ) { + rc =epoll_wait (epfd, evs, maxevents, timeout); +if (likely (rc >0 )) { +/* epoll_wait has at least one fd ready to read */ + rc =eal_epoll_process_event (evs, rc, events); +break ; + }else if (rc <0 ) { +if (errno ==EINTR ) +continue ; +/* epoll_wait fail */ +RTE_LOG (ERR ,EAL ,"epoll_wait returns with fail %s\n" , +strerror (errno)); + rc =-1 ; +break ; + } + } + +return rc; +} + +static inline void +eal_epoll_data_safe_free (struct rte_epoll_event *ev) +{ +while (!rte_atomic32_cmpset (&ev->status,RTE_EPOLL_VALID , +RTE_EPOLL_INVALID )) +while (ev->status !=RTE_EPOLL_VALID ) +rte_pause (); +memset (&ev->epdata,0 ,sizeof (ev->epdata)); + ev->fd =-1 ; + ev->epfd =-1 ; +} + +int +rte_epoll_ctl (int epfd,int op,int fd, +struct rte_epoll_event *event) +{ +struct epoll_event ev; + +if (!event) { +RTE_LOG (ERR ,EAL ,"rte_epoll_event can't be NULL\n" ); +return -1 ; + } + +/* using per thread epoll fd */ +if (epfd ==RTE_EPOLL_PER_THREAD ) + epfd =rte_intr_tls_epfd (); + +if (op ==EPOLL_CTL_ADD ) { + event->status =RTE_EPOLL_VALID ; + event->fd = fd;/* ignore fd in event */ + event->epfd = epfd; + ev.data.ptr = (void *)event; + } + + ev.events = event->epdata.event; +if (epoll_ctl (epfd, op, fd, &ev) <0 ) { +RTE_LOG (ERR ,EAL ,"Error op %d fd %d epoll_ctl, %s\n" , + op, fd,strerror (errno)); +if (op ==EPOLL_CTL_ADD ) +/* rollback status when CTL_ADD fail */ + event->status =RTE_EPOLL_INVALID ; +return -1 ; + } + +if (op ==EPOLL_CTL_DEL && event->status !=RTE_EPOLL_INVALID ) +eal_epoll_data_safe_free (event); + +return 0 ; +}
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +79 -3
95 lines not on this page · raw message
lib/librte_eal/linuxapp/eal/rte_eal_version.map +3 -0
@@ -52,6 +52,8 @@ DPDK_2.0 { rte_eal_vdev_init; rte_eal_vdev_uninit; rte_eal_wait_lcore; + rte_epoll_ctl; + rte_epoll_wait; rte_exit; rte_get_hpet_cycles; rte_get_hpet_hz;
@@ -61,6 +63,7 @@ DPDK_2.0 { rte_intr_callback_unregister; rte_intr_disable; rte_intr_enable; + rte_intr_tls_epfd; rte_log; rte_log_add_in_history; rte_log_cur_msg_loglevel;
Signature · 1 line
1.8.1.4
Re: [PATCH v8 02/11] eal/linux: add rte_epoll_wait/ctl support
Quoted 8 lines · On Thu, 21 May 2015 16:55:54 +0800 Cunming Liang <[email protected]>…
lib/librte_eal/linuxapp/eal/eal_interrupts.c +8
-0+static int +eal_epoll_process_event (struct epoll_event *evs,int n, +struct rte_epoll_event *events) +{ +int i; +int count =0 ; +struct rte_epoll_event *rev; +for (i =0 ; i < n; i++) {
lib/librte_eal/linuxapp/eal/eal_interrupts.c rte_eal_intr_init(void) · line 881
Quoted 10 lines
lib/librte_eal/linuxapp/eal/eal_interrupts.c +10
-0+ rev = (struct rte_epoll_event *)evs[i].data.ptr; +if (!rev || !rte_atomic32_cmpset (&rev->status,RTE_EPOLL_VALID , +RTE_EPOLL_EXEC )) +continue ; + + events[count].status =RTE_EPOLL_VALID ; + events[count].fd = rev->fd; + events[count].epfd = rev->epfd; + events[count].epdata.event = rev->epdata.event; + events[count].epdata.data = rev->epdata.data;
This code has several style issues:
1. Always put blank line after declarations
2. Use unsigned where ever it makes sense as a matter of habit.
unsigned int i, count = 0;
3. Don't add casts where not necessary, it reduces compiler type checking and is a bad habit. In this case evs[i].data.ptr is void * and therefore no cast is needed.
A message that is not in the archive (sent off-list or not yet received): 20150521111704.727cf3a1@urahara
Re: [PATCH v8 02/11] eal/linux: add rte_epoll_wait/ctl support
Quoted 24 lines · Stephen Hemminger
Quoted 8 lines · On Thu, 21 May 2015 16:55:54 +0800 Cunming Liang <[email protected]>…
lib/librte_eal/linuxapp/eal/eal_interrupts.c +8
-0+static int +eal_epoll_process_event (struct epoll_event *evs,int n, +struct rte_epoll_event *events) +{ +int i; +int count =0 ; +struct rte_epoll_event *rev; +for (i =0 ; i < n; i++) {lib/librte_eal/linuxapp/eal/eal_interrupts.c rte_eal_intr_init(void) · line 881
Quoted 10 lines
lib/librte_eal/linuxapp/eal/eal_interrupts.c +10
-0+ rev = (struct rte_epoll_event *)evs[i].data.ptr; +if (!rev || !rte_atomic32_cmpset (&rev->status,RTE_EPOLL_VALID , +RTE_EPOLL_EXEC )) +continue ; + + events[count].status =RTE_EPOLL_VALID ; + events[count].fd = rev->fd; + events[count].epfd = rev->epfd; + events[count].epdata.event = rev->epdata.event; + events[count].epdata.data = rev->epdata.data;This code has several style issues:
1. Always put blank line after declarations
Quoted 6 lines
2. Use unsigned where ever it makes sense as a matter of habit.
unsigned int i, count = 0;3. Don't add casts where not necessary, it reduces compiler type checking and is a bad habit. In this case evs[i].data.ptr is void * and therefore no cast is needed.
Fully agree, thanks for the comment.
PATCH v8 3/11 eal/linux: add API to set rx interrupt event monitor
In reply to Cunming LiangThe patch adds 'rte_intr_rx_ctl' to add or delete interrupt vector events monitor on specified epoll instance.
---
v8 changes
- fix EWOULDBLOCK and EINTR processing
- add event status check
v7 changes
- rename rte_intr_rx_set to rte_intr_rx_ctl.
- rte_intr_rx_ctl uses rte_epoll_ctl to register epoll event instance.
- the intr rx event instance includes a intr process callback.
v6 changes
- split rte_intr_wait_rx_pkt into two function, wait and set.
- rewrite rte_intr_rx_wait/rte_intr_rx_set to remove queue visibility on eal.
- rte_intr_rx_wait to support multiplexing.
- allow epfd as input to support flexible event fd combination.
lib/librte_eal/linuxapp/eal/eal_interrupts.c | 103 +++++++++++++++++++++ .../linuxapp/eal/include/exec-env/rte_interrupts.h | 23 +++++ lib/librte_eal/linuxapp/eal/rte_eal_version.map | 1 + 3 files changed, 127 insertions(+)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +103 -0
@@ -861,6 +861,49 @@ rte_eal_intr_init(void)return -ret; } +static void +eal_intr_proc_rxtx_intr (int fd,const struct rte_intr_handle *intr_handle) +{ +union rte_intr_read_buffer buf; +int bytes_read =1 ; + +switch (intr_handle->type) { +case RTE_INTR_HANDLE_UIO : + bytes_read =sizeof (buf.uio_intr_count); +break ; +#ifdef VFIO_PRESENT +case RTE_INTR_HANDLE_VFIO_MSIX : +case RTE_INTR_HANDLE_VFIO_MSI : +case RTE_INTR_HANDLE_VFIO_LEGACY : + bytes_read =sizeof (buf.vfio_intr_count); +break ; +#endif +default : + bytes_read =1 ; +RTE_LOG (INFO ,EAL ,"unexpected intr type\n" ); +break ; + } + +/** +* read out to clear the ready-to-be-read flag +* for epoll_wait. +*/ +do { + bytes_read =read (fd, &buf, bytes_read); +if (bytes_read <0 ) { +if (errno ==EINTR || errno ==EWOULDBLOCK || + errno ==EAGAIN ) +continue ; +RTE_LOG (ERR ,EAL ,"Error reading from file " +"descriptor %d: %s\n" , fd, +strerror (errno)); + }else if (bytes_read ==0 ) +RTE_LOG (ERR ,EAL ,"Read nothing from file " +"descriptor %d\n" , fd); +return ; + }while (1 ); +} +static int eal_epoll_process_event (struct epoll_event *evs,int n,struct rte_epoll_event *events)
@@ -996,3 +1039,63 @@ rte_epoll_ctl(int epfd, int op, int fd,return 0 ; } + +int +rte_intr_rx_ctl (struct rte_intr_handle *intr_handle,int epfd, +int op,unsigned int vec,void *data,int socket) +{ +struct rte_epoll_event *rev; +struct rte_epoll_data *epdata; +int epfd_op; +int rc =0 ; + +if (!intr_handle || intr_handle->nb_efd ==0 || + vec >= intr_handle->nb_efd) { +RTE_LOG (ERR ,EAL ,"Wrong intr vector number.\n" ); +return -EPERM ; + } + +if (socket ==SOCKET_ID_ANY ) + socket =rte_socket_id (); + +switch (op) { +case RTE_INTR_EVENT_ADD : + epfd_op =EPOLL_CTL_ADD ; + rev = &intr_handle->elist[vec]; +if (rev->status !=RTE_EPOLL_INVALID ) { +RTE_LOG (INFO ,EAL ,"Event already been added.\n" ); +return -EEXIST ; + } + +/* attach to intr vector fd */ + epdata = &rev->epdata; + epdata->event =EPOLLIN |EPOLLPRI |EPOLLET ; + epdata->data = data; + epdata->cb_fun = (rte_intr_event_cb_t )eal_intr_proc_rxtx_intr; + epdata->cb_arg = (void *)intr_handle; + rc =rte_epoll_ctl (epfd, epfd_op, intr_handle->efds[vec], rev); +if (!rc) +RTE_LOG (DEBUG ,EAL ,"eventfd %d associated with vec %d" +" is added on epfd %d\n" , rev->fd, vec, epfd); +else + rc = -EPERM ; +break ; +case RTE_INTR_EVENT_DEL : + epfd_op =EPOLL_CTL_DEL ; + rev = &intr_handle->elist[vec]; +if (rev->status ==RTE_EPOLL_INVALID ) { +RTE_LOG (INFO ,EAL ,"Event does not exist.\n" ); +return -EPERM ; + } + + rc =rte_epoll_ctl (rev->epfd, epfd_op, rev->fd, rev); +if (rc) + rc = -EPERM ; +break ; +default : +RTE_LOG (ERR ,EAL ,"event op type mismatch\n" ); + rc = -EPERM ; + } + +return rc; +}
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +23 -0
@@ -145,4 +145,27 @@ rte_epoll_ctl(int epfd, int op, int fd,int rte_intr_tls_epfd (void ); +/** +* @param intr_handle +* Pointer to the interrupt handle. +* @param epfd +* Epoll instance fd which the intr vector associated to. +* @param op +* The operation be performed for the vector. +* Operation type of {ADD, DEL}. +* @param vec +* RX intr vector number added to the epoll instance wait list. +* @param data +* User raw data. +* @param socket +* Specifying the socket id. +* @return +* - On success, zero. +* - On failure, a negative value. +*/ +int +rte_intr_rx_ctl (struct rte_intr_handle *intr_handle, +int epfd,int op,unsigned int vec, +void *data,int socket); +#endif /* _RTE_LINUXAPP_INTERRUPTS_H_ */
lib/librte_eal/linuxapp/eal/rte_eal_version.map +1 -0
@@ -63,6 +63,7 @@ DPDK_2.0 { rte_intr_callback_unregister; rte_intr_disable; rte_intr_enable; + rte_intr_rx_ctl; rte_intr_tls_epfd; rte_log; rte_log_add_in_history;
Signature · 1 line
1.8.1.4
PATCH v8 4/11 eal/linux: fix comments typo on vfio msi
In reply to Cunming Liang--- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +2 -2
@@ -219,7 +219,7 @@ vfio_disable_intx(struct rte_intr_handle *intr_handle) {return 0 ; } -/* enable MSI-X interrupts */ +/* enable MSI interrupts */ static int vfio_enable_msi (struct rte_intr_handle *intr_handle) {int len, ret;
@@ -265,7 +265,7 @@ vfio_enable_msi(struct rte_intr_handle *intr_handle) {return 0 ; } -/* disable MSI-X interrupts */ +/* disable MSI interrupts */ static int vfio_disable_msi (struct rte_intr_handle *intr_handle) {struct vfio_irq_set *irq_set;
Signature · 1 line
1.8.1.4
PATCH v8 5/11 eal/linux: add interrupt vectors handling on VFIO
In reply to Cunming LiangThis patch does below:
- Create VFIO eventfds for each interrupt vector (move to next)
- Assign per interrupt vector's eventfd to VFIO by ioctl
---
v8 changes
- move eventfd creation out of the setup_interrupts to a standalone function
v7 changes
- cleanup unnecessary code change
- split event and intr operation to other patches
lib/librte_eal/linuxapp/eal/eal_interrupts.c | 50 ++++++++-------------------- 1 file changed, 13 insertions(+), 37 deletions(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +13 -37
@@ -128,6 +128,9 @@ static pthread_t intr_thread;#ifdef VFIO_PRESENT #define IRQ_SET_BUF_LEN (sizeof(struct vfio_irq_set) + sizeof(int)) +/* irq set buffer length for queue interrupts and LSC interrupt */ +#define MSIX_IRQ_SET_BUF_LEN (sizeof(struct vfio_irq_set) + \ + sizeof(int) * (RTE_MAX_RXTX_INTR_VEC_ID + 1))/* enable legacy (INTx) interrupts */ static int
@@ -245,23 +248,6 @@ vfio_enable_msi(struct rte_intr_handle *intr_handle) { intr_handle->fd);return -1 ; } - -/* manually trigger interrupt to enable it */ -memset (irq_set,0 , len); - len =sizeof (struct vfio_irq_set ); - irq_set->argsz = len; - irq_set->count =1 ; - irq_set->flags =VFIO_IRQ_SET_DATA_NONE |VFIO_IRQ_SET_ACTION_TRIGGER ; - irq_set->index =VFIO_PCI_MSI_IRQ_INDEX ; - irq_set->start =0 ; - - ret =ioctl (intr_handle->vfio_dev_fd,VFIO_DEVICE_SET_IRQS , irq_set); - -if (ret) { -RTE_LOG (ERR ,EAL ,"Error triggering MSI interrupts for fd %d\n" , - intr_handle->fd); -return -1 ; - }return 0 ; }
@@ -294,7 +280,7 @@ vfio_disable_msi(struct rte_intr_handle *intr_handle) {static int vfio_enable_msix (struct rte_intr_handle *intr_handle) {int len, ret; -char irq_set_buf[IRQ_SET_BUF_LEN ]; +char irq_set_buf[MSIX_IRQ_SET_BUF_LEN ];struct vfio_irq_set *irq_set;int *fd_ptr;
@@ -302,12 +288,18 @@ vfio_enable_msix(struct rte_intr_handle *intr_handle) { irq_set = (struct vfio_irq_set *) irq_set_buf; irq_set->argsz = len; - irq_set->count =1 ; +if (!intr_handle->max_intr) + intr_handle->max_intr =1 ; +else if (intr_handle->max_intr >RTE_MAX_RXTX_INTR_VEC_ID ) + intr_handle->max_intr =RTE_MAX_RXTX_INTR_VEC_ID +1 ; + + irq_set->count = intr_handle->max_intr; irq_set->flags =VFIO_IRQ_SET_DATA_EVENTFD |VFIO_IRQ_SET_ACTION_TRIGGER ; irq_set->index =VFIO_PCI_MSIX_IRQ_INDEX ; irq_set->start =0 ; fd_ptr = (int *) &irq_set->data; - *fd_ptr = intr_handle->fd; +memcpy (fd_ptr, intr_handle->efds,sizeof (intr_handle->efds)); + fd_ptr[intr_handle->max_intr -1 ] = intr_handle->fd; ret =ioctl (intr_handle->vfio_dev_fd,VFIO_DEVICE_SET_IRQS , irq_set);
@@ -317,22 +309,6 @@ vfio_enable_msix(struct rte_intr_handle *intr_handle) {return -1 ; } -/* manually trigger interrupt to enable it */ -memset (irq_set,0 , len); - len =sizeof (struct vfio_irq_set ); - irq_set->argsz = len; - irq_set->count =1 ; - irq_set->flags =VFIO_IRQ_SET_DATA_NONE |VFIO_IRQ_SET_ACTION_TRIGGER ; - irq_set->index =VFIO_PCI_MSIX_IRQ_INDEX ; - irq_set->start =0 ; - - ret =ioctl (intr_handle->vfio_dev_fd,VFIO_DEVICE_SET_IRQS , irq_set); - -if (ret) { -RTE_LOG (ERR ,EAL ,"Error triggering MSI-X interrupts for fd %d\n" , - intr_handle->fd); -return -1 ; - }return 0 ; }
@@ -340,7 +316,7 @@ vfio_enable_msix(struct rte_intr_handle *intr_handle) {static int vfio_disable_msix (struct rte_intr_handle *intr_handle) {struct vfio_irq_set *irq_set; -char irq_set_buf[IRQ_SET_BUF_LEN ]; +char irq_set_buf[MSIX_IRQ_SET_BUF_LEN ];int len, ret; len =sizeof (struct vfio_irq_set );
Signature · 1 line
1.8.1.4
Re: [PATCH v8 05/11] eal/linux: add interrupt vectors handling on VFIO
Quoted 6 lines · On Thu, 21 May 2015 16:55:57 +0800 Cunming Liang <[email protected]>…
This patch does below:
- Create VFIO eventfds for each interrupt vector (move to next)
- Assign per interrupt vector's eventfd to VFIO by ioctlSigned-off-by Danny Zhou <[email protected]>Signed-off-by Cunming Liang <[email protected]>
One non-trivial performance related change here would be to set SMP affinity of the receive IRQ to the CPU that is handling that receive queue. Not sure the full API to do this, but ideally you should not have the receive interrupt occurring on one CPU then having to cause scheduler to wakeup receive thread on another CPU.
Re: [PATCH v8 05/11] eal/linux: add interrupt vectors handling on VFIO
Quoted 7 lines · Stephen Hemminger
Quoted 4 lines · On Thu, 21 May 2015 16:55:57 +0800 Cunming Liang <[email protected]>…
This patch does below:
- Create VFIO eventfds for each interrupt vector (move to next)
- Assign per interrupt vector's eventfd to VFIO by ioctl
Quoted 8 lines
Quoted 2 lines
Signed-off-by Danny Zhou <[email protected]>Signed-off-by Cunming Liang <[email protected]>One non-trivial performance related change here would be to set SMP affinity of the receive IRQ to the CPU that is handling that receive queue. Not sure the full API to do this, but ideally you should not have the receive interrupt occurring on one CPU then having to cause scheduler to wakeup receive thread on another CPU.
That's a good point. The previous thought was to configure irq affinity by script from outside. I haven't found some API to do that well, the well known way is by sysfs.
PATCH v8 6/11 eal/linux: standalone intr event fd create support
In reply to Cunming LiangThe patch exposes intr event fd create and release for PMD.
The device driver can assign the number of event associated with interrupt vector. It also provides misc funtions to check 1) allows other slowpath intr(e.g. lsc);
2) intr event on fastpath is enabled or not.
--- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 57 ++++++++++++++++++++++ .../linuxapp/eal/include/exec-env/rte_interrupts.h | 51 +++++++++++++++++++ lib/librte_eal/linuxapp/eal/rte_eal_version.map | 4 ++ 3 files changed, 112 insertions(+)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +57 -0
@@ -44,6 +44,7 @@#include <sys/epoll.h> #include <sys/signalfd.h> #include <sys/ioctl.h> +#include <sys/eventfd.h> #include <rte_common.h> #include <rte_interrupts.h>
@@ -68,6 +69,7 @@#include "eal_vfio.h" #define EAL_INTR_EPOLL_WAIT_FOREVER (-1) +#define NB_OTHER_INTR 1 static RTE_DEFINE_PER_LCORE(int, _epfd) = -1; /**< epoll fd per thread */
@@ -1075,3 +1077,58 @@ rte_intr_rx_ctl(struct rte_intr_handle *intr_handle, int epfd,return rc; } + +int +rte_intr_efd_enable (struct rte_intr_handle *intr_handle,uint32_t nb_efd) +{ +uint32_t i; +int fd; +uint32_t n =RTE_MIN (nb_efd, (uint32_t )RTE_MAX_RXTX_INTR_VEC_ID ); + +if (intr_handle->type ==RTE_INTR_HANDLE_VFIO_MSIX ) { +for (i =0 ; i < n; i++) { + fd =eventfd (0 ,EFD_NONBLOCK |EFD_CLOEXEC ); +if (fd <0 ) { +RTE_LOG (ERR ,EAL , +"cannot setup eventfd," +"error %i (%s)\n" , + errno,strerror (errno)); +return -1 ; + } + intr_handle->efds[i] = fd; + } + intr_handle->nb_efd = n; + intr_handle->max_intr =NB_OTHER_INTR + n; + }else { + intr_handle->efds[0 ] = intr_handle->fd; + intr_handle->nb_efd =RTE_MIN (nb_efd,1U ); + intr_handle->max_intr =NB_OTHER_INTR ; + } + +return 0 ; +} + +void +rte_intr_efd_disable (struct rte_intr_handle *intr_handle) +{ +uint32_t i; +struct rte_epoll_event *rev; + +for (i =0 ; i < intr_handle->nb_efd; i++) { + rev = &intr_handle->elist[i]; +if (rev->status ==RTE_EPOLL_INVALID ) +continue ; +if (rte_epoll_ctl (rev->epfd,EPOLL_CTL_DEL , rev->fd, rev)) { +/* force free if the entry valid */ +eal_epoll_data_safe_free (rev); + rev->status =RTE_EPOLL_INVALID ; + } + } + +if (intr_handle->max_intr > intr_handle->nb_efd) { +for (i =0 ; i < intr_handle->nb_efd; i++) +close (intr_handle->efds[i]); + } + intr_handle->nb_efd =0 ; + intr_handle->max_intr =0 ; +}
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +51 -0
55 lines not on this page · raw message
lib/librte_eal/linuxapp/eal/rte_eal_version.map +4 -0
@@ -61,7 +61,11 @@ DPDK_2.0 { rte_hexdump; rte_intr_callback_register; rte_intr_callback_unregister; + rte_intr_allow_others; rte_intr_disable; + rte_intr_dp_is_en; + rte_intr_efd_enable; + rte_intr_efd_disable; rte_intr_enable; rte_intr_rx_ctl; rte_intr_tls_epfd;
Signature · 1 line
1.8.1.4
PATCH v8 7/11 eal/bsd: dummy for new intr definition
In reply to Cunming LiangTo make bsd compiling happy with new intr changes.
---
v8 changes
- add stub for new function
v7 changes
- remove stub 'linux only' function from source file
lib/librte_eal/bsdapp/eal/eal_interrupts.c | 20 ++++++ .../bsdapp/eal/include/exec-env/rte_interrupts.h | 77 ++++++++++++++++++++++ lib/librte_eal/bsdapp/eal/rte_eal_version.map | 5 ++ 3 files changed, 102 insertions(+)
lib/librte_eal/bsdapp/eal/eal_interrupts.c +20 -0
@@ -69,3 +69,23 @@ rte_eal_intr_init(void)return 0 ; } +int +rte_intr_rx_ctl (struct rte_intr_handle *intr_handle, +int epfd,int op,unsigned int vec, +void *data,int socket) +{ +return -ENOTSUP ; +} + +int +rte_intr_efd_enable (struct rte_intr_handle *intr_handle,uint32_t nb_efd) +{ +return 0 ; +} + +void +rte_intr_efd_disable (struct rte_intr_handle *intr_handle) +{ +return ; +} +
lib/librte_eal/bsdapp/eal/include/exec-env/rte_interrupts.h +77 -0
83 lines not on this page · raw message
lib/librte_eal/bsdapp/eal/rte_eal_version.map +5 -0
@@ -53,8 +53,13 @@ DPDK_2.0 { rte_hexdump; rte_intr_callback_register; rte_intr_callback_unregister; + rte_intr_allow_others; rte_intr_disable; + rte_intr_dp_is_en; + rte_intr_efd_enable; + rte_intr_efd_disable; rte_intr_enable; + rte_intr_rx_ctl; rte_log; rte_log_add_in_history; rte_log_cur_msg_loglevel;
Signature · 1 line
1.8.1.4
PATCH v8 8/11 ethdev: add rx intr enable, disable and ctl functions
In reply to Cunming LiangThe patch adds two dev_ops functions to enable and disable rx queue interrupts.
In addtion, it adds rte_eth_dev_rx_intr_ctl/rx_intr_q to support per port or per queue rx intr event set.
---
v8 changes
- add addtion check for EEXIT
v7 changes
- remove rx_intr_vec_get
- add rx_intr_ctl and rx_intr_ctl_q
v6 changes
- add rx_intr_vec_get to retrieve the vector num of the queue.
v5 changes
- Rebase the patchset onto the HEAD
v4 changes
- Export interrupt enable/disable functions for shared libraries
- Put new functions at the end of eth_dev_ops to avoid breaking ABI
v3 changes
- Add return value for interrupt enable/disable functions
lib/librte_ether/rte_ethdev.c | 127 +++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ethdev.h | 104 +++++++++++++++++++++++++++ lib/librte_ether/rte_ether_version.map | 4 ++ 3 files changed, 235 insertions(+)
lib/librte_ether/rte_ethdev.c +127 -0
@@ -3281,6 +3281,133 @@ _rte_eth_dev_callback_process(struct rte_eth_dev *dev, }rte_spinlock_unlock (&rte_eth_dev_cb_lock); } + +int +rte_eth_dev_rx_intr_ctl (uint8_t port_id,int epfd,int op,void *data) +{ +uint32_t vec; +struct rte_eth_dev *dev; +struct rte_intr_handle *intr_handle; +uint16_t qid; +int rc; + +if (!rte_eth_dev_is_valid_port (port_id)) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return -ENODEV ; + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return -ENODEV ; + } + + intr_handle = &dev->pci_dev->intr_handle; +if (!intr_handle->intr_vec) { +PMD_DEBUG_TRACE ("RX Intr vector unset\n" ); +return -EPERM ; + } + +for (qid =0 ; qid < dev->data->nb_rx_queues; qid++) { + vec = intr_handle->intr_vec[qid]; + rc =rte_intr_rx_ctl (intr_handle, epfd, op, vec, + data,rte_eth_dev_socket_id (port_id)); +if (rc && rc != -EEXIST ) { +PMD_DEBUG_TRACE ("p %d q %d rx ctl error" +" op %d epfd %d vec %u\n" , + port_id, qid, op, epfd, vec); + } + } + +return 0 ; +} + +int +rte_eth_dev_rx_intr_ctl_q (uint8_t port_id,uint16_t queue_id, +int epfd,int op,void *data) +{ +uint32_t vec; +struct rte_eth_dev *dev; +struct rte_intr_handle *intr_handle; +int rc; + +if (!rte_eth_dev_is_valid_port (port_id)) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return -ENODEV ; + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return -ENODEV ; + } + +if (queue_id >= dev->data->nb_rx_queues) { +PMD_DEBUG_TRACE ("Invalid RX queue_id=%d\n" , rx_queue_id); +return -EINVAL ; + } + + intr_handle = &dev->pci_dev->intr_handle; +if (!intr_handle->intr_vec) { +PMD_DEBUG_TRACE ("RX Intr vector unset\n" ); +return -EPERM ; + } + + vec = intr_handle->intr_vec[queue_id]; + rc =rte_intr_rx_ctl (intr_handle, epfd, op, vec, + data,rte_eth_dev_socket_id (port_id)); +if (rc && rc != -EEXIST ) { +PMD_DEBUG_TRACE ("p %d q %d rx ctl error" +" op %d epfd %d vec %u\n" , + port_id, queue_id, op, epfd, vec); +return rc; + } + +return 0 ; +} + +int +rte_eth_dev_rx_intr_enable (uint8_t port_id, +uint16_t queue_id) +{ +struct rte_eth_dev *dev; + +if (!rte_eth_dev_is_valid_port (port_id)) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return -ENODEV ; + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return -ENODEV ; + } + +FUNC_PTR_OR_ERR_RET (*dev->dev_ops->rx_queue_intr_enable, -ENOTSUP ); +return (*dev->dev_ops->rx_queue_intr_enable)(dev, queue_id); +} + +int +rte_eth_dev_rx_intr_disable (uint8_t port_id, +uint16_t queue_id) +{ +struct rte_eth_dev *dev; + +if (!rte_eth_dev_is_valid_port (port_id)) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return -ENODEV ; + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return -ENODEV ; + } + +FUNC_PTR_OR_ERR_RET (*dev->dev_ops->rx_queue_intr_disable, -ENOTSUP ); +return (*dev->dev_ops->rx_queue_intr_disable)(dev, queue_id); +} +#ifdef RTE_NIC_BYPASS int rte_eth_dev_bypass_init (uint8_t port_id) {
lib/librte_ether/rte_ethdev.h +104 -0
128 lines not on this page · raw message
lib/librte_ether/rte_ether_version.map +4 -0
@@ -48,6 +48,10 @@ DPDK_2.0 { rte_eth_dev_rss_hash_update; rte_eth_dev_rss_reta_query; rte_eth_dev_rss_reta_update; + rte_eth_dev_rx_intr_ctl; + rte_eth_dev_rx_intr_ctl_q; + rte_eth_dev_rx_intr_disable; + rte_eth_dev_rx_intr_enable; rte_eth_dev_rx_queue_start; rte_eth_dev_rx_queue_stop; rte_eth_dev_set_link_down;
Signature · 1 line
1.8.1.4
Re: [PATCH v8 08/11] ethdev: add rx intr enable, disable and ctl functions
lib/librte_ether/rte_ethdev.c _rte_eth_dev_callback_process(struct rte_eth_dev *dev, · line 3303
Quoted 10 lines · On Thu, 21 May 2015 16:56:00 +0800 Cunming Liang <[email protected]>…
lib/librte_ether/rte_ethdev.c +10
-0+if (!rte_eth_dev_is_valid_port (port_id)) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id); +return -ENODEV ; + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return -ENODEV ; + }
This check is not needed, rte_eth_dev_is_valid_port already checked that.
Re: [PATCH v8 08/11] ethdev: add rx intr enable, disable and ctl functions
In reply to Cunming Lianglib/librte_ether/rte_ethdev.c _rte_eth_dev_callback_process(struct rte_eth_dev *dev, · line 3335
Quoted 11 lines · On Thu, 21 May 2015 16:56:00 +0800 Cunming Liang <[email protected]>…
lib/librte_ether/rte_ethdev.c +11
-0+int +rte_eth_dev_rx_intr_ctl_q (uint8_t port_id,uint16_t queue_id, +int epfd,int op,void *data) +{ +uint32_t vec; +struct rte_eth_dev *dev; +struct rte_intr_handle *intr_handle; +int rc; + +if (!rte_eth_dev_is_valid_port (port_id)) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id);
Use %u when printing unsigned value
lib/librte_ether/rte_ethdev.c _rte_eth_dev_callback_process(struct rte_eth_dev *dev, · line 3303
Quoted 8 lines
lib/librte_ether/rte_ethdev.c +8
-0+return -ENODEV ; + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return -ENODEV ; + }
Another unnecessary conditional check, already done in rte_eth_dev_is_valid_port
lib/librte_ether/rte_ethdev.c _rte_eth_dev_callback_process(struct rte_eth_dev *dev, · line 3346
Quoted 3 lines
lib/librte_ether/rte_ethdev.c +3
-0+ +if (queue_id >= dev->data->nb_rx_queues) { +PMD_DEBUG_TRACE ("Invalid RX queue_id=%d\n" , rx_queue_id);
This is wrong, won't build with debug enabled, You meant to use queue_id here and use %u
Quoted 9 lines
lib/librte_ether/rte_ethdev.c +9
-0+return -EINVAL ; + } + + intr_handle = &dev->pci_dev->intr_handle; +if (!intr_handle->intr_vec) { +PMD_DEBUG_TRACE ("RX Intr vector unset\n" ); +return -EPERM ; + } +
A message that is not in the archive (sent off-list or not yet received): 20150521112030.4d31a0e4@urahara
Re: [PATCH v8 08/11] ethdev: add rx intr enable, disable and ctl functions
Quoted 15 lines · Stephen Hemminger
lib/librte_ether/rte_ethdev.c _rte_eth_dev_callback_process(struct rte_eth_dev *dev, · line 3335
Quoted 11 lines · On Thu, 21 May 2015 16:56:00 +0800 Cunming Liang <[email protected]>…
lib/librte_ether/rte_ethdev.c +11
-0+int +rte_eth_dev_rx_intr_ctl_q (uint8_t port_id,uint16_t queue_id, +int epfd,int op,void *data) +{ +uint32_t vec; +struct rte_eth_dev *dev; +struct rte_intr_handle *intr_handle; +int rc; + +if (!rte_eth_dev_is_valid_port (port_id)) { +PMD_DEBUG_TRACE ("Invalid port_id=%d\n" , port_id);Use %u when printing unsigned value
Agree.
Quoted 10 lines
lib/librte_ether/rte_ethdev.c _rte_eth_dev_callback_process(struct rte_eth_dev *dev, · line 3303
Quoted 9 lines
lib/librte_ether/rte_ethdev.c +8
-0+return -ENODEV ; + } + + dev = &rte_eth_devices[port_id]; +if (dev ==NULL ) { +PMD_DEBUG_TRACE ("Invalid port device\n" ); +return -ENODEV ; + }Another unnecessary conditional check, already done in
rte_eth_dev_is_valid_port
Yes, it's not necessary anymore.
Quoted 5 lines
lib/librte_ether/rte_ethdev.c _rte_eth_dev_callback_process(struct rte_eth_dev *dev, · line 3346
Quoted 3 lines
lib/librte_ether/rte_ethdev.c +3
-0+ +if (queue_id >= dev->data->nb_rx_queues) { +PMD_DEBUG_TRACE ("Invalid RX queue_id=%d\n" , rx_queue_id);This is wrong, won't build with debug enabled, You meant to use
queue_idhere and use %u
I'm sorry, that's a mistake, thanks.
Quoted 9 lines
Quoted 9 lines
lib/librte_ether/rte_ethdev.c +9
-0+return -EINVAL ; + } + + intr_handle = &dev->pci_dev->intr_handle; +if (!intr_handle->intr_vec) { +PMD_DEBUG_TRACE ("RX Intr vector unset\n" ); +return -EPERM ; + } +
PATCH v8 9/11 ixgbe: enable rx queue interrupts for both PF and VF
In reply to Cunming LiangThe patch does below things for ixgbe PF and VF:
- Setup NIC to generate MSI-X interrupts
- Set the IVAR register to map interrupt causes to vectors
- Implement interrupt enable/disable functions
---
v8 changes
- add vfio-msi/vfio-legacy and uio-legacy support
v7 changes
- add condition check when intr vector is not enabled
v6 changes
- fill queue-vector mapping table
v5 changes
- Rebase the patchset onto the HEAD
v3 changes
- Remove spinlok from PMD
v2 changes
- Consolidate review comments related to coding style
lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 482 +++++++++++++++++++++++++++++++++++- lib/librte_pmd_ixgbe/ixgbe_ethdev.h | 4 + 2 files changed, 474 insertions(+), 12 deletions(-)
lib/librte_pmd_ixgbe/ixgbe_ethdev.c +470 -12
@@ -82,6 +82,9 @@*/ #define IXGBE_FC_LO 0x40 +/* Default minimum inter-interrupt interval for EITR configuration */ +#define IXGBE_MIN_INTER_INTERRUPT_INTERVAL_DEFAULT 0x79E +/* Timer value included in XOFF frames. */ #define IXGBE_FC_PAUSE 0x680
@@ -171,6 +174,7 @@ static int ixgbe_dev_rss_reta_query(struct rte_eth_dev *dev, uint16_t reta_size);static void ixgbe_dev_link_status_print (struct rte_eth_dev *dev);static int ixgbe_dev_lsc_interrupt_setup (struct rte_eth_dev *dev); +static int ixgbe_dev_rxq_interrupt_setup (struct rte_eth_dev *dev);static int ixgbe_dev_interrupt_get_status (struct rte_eth_dev *dev);static int ixgbe_dev_interrupt_action (struct rte_eth_dev *dev); static void ixgbe_dev_interrupt_handler(struct rte_intr_handle *handle,
@@ -183,11 +187,14 @@ static void ixgbe_dcb_init(struct ixgbe_hw *hw,struct ixgbe_dcb_config *dcb_conf/* For Virtual Function support */ static int eth_ixgbevf_dev_init (struct rte_eth_dev *eth_dev); +static int ixgbevf_dev_interrupt_get_status (struct rte_eth_dev *dev); +static int ixgbevf_dev_interrupt_action (struct rte_eth_dev *dev);static int ixgbevf_dev_configure (struct rte_eth_dev *dev);static int ixgbevf_dev_start (struct rte_eth_dev *dev);static void ixgbevf_dev_stop (struct rte_eth_dev *dev);static void ixgbevf_dev_close (struct rte_eth_dev *dev);static void ixgbevf_intr_disable (struct ixgbe_hw *hw); +static void ixgbevf_intr_enable (struct ixgbe_hw *hw);static void ixgbevf_dev_stats_get (struct rte_eth_dev *dev,struct rte_eth_stats *stats);static void ixgbevf_dev_stats_reset (struct rte_eth_dev *dev);
@@ -197,6 +204,15 @@ static void ixgbevf_vlan_strip_queue_set(struct rte_eth_dev *dev, uint16_t queue, int on);static void ixgbevf_vlan_offload_set (struct rte_eth_dev *dev,int mask);static void ixgbevf_set_vfta_all (struct rte_eth_dev *dev,bool on); +static void ixgbevf_dev_interrupt_handler (struct rte_intr_handle *handle, +void *param); +static int ixgbevf_dev_rx_queue_intr_enable (struct rte_eth_dev *dev, +uint16_t queue_id); +static int ixgbevf_dev_rx_queue_intr_disable (struct rte_eth_dev *dev, +uint16_t queue_id); +static void ixgbevf_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector); +static void ixgbevf_configure_msix (struct rte_eth_dev *dev);/* For Eth VMDQ APIs support */ static int ixgbe_uc_hash_table_set(struct rte_eth_dev *dev, struct
@@ -214,6 +230,14 @@ static int ixgbe_mirror_rule_set(struct rte_eth_dev *dev,static int ixgbe_mirror_rule_reset (struct rte_eth_dev *dev,uint8_t rule_id); +static int ixgbe_dev_rx_queue_intr_enable (struct rte_eth_dev *dev, +uint16_t queue_id); +static int ixgbe_dev_rx_queue_intr_disable (struct rte_eth_dev *dev, +uint16_t queue_id); +static void ixgbe_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector); +static void ixgbe_configure_msix (struct rte_eth_dev *dev); +static int ixgbe_set_queue_rate_limit (struct rte_eth_dev *dev,uint16_t queue_idx,uint16_t tx_rate); static int ixgbe_set_vf_rate_limit(struct rte_eth_dev *dev, uint16_t vf,
@@ -262,7 +286,7 @@ static int ixgbevf_dev_set_mtu(struct rte_eth_dev *dev, uint16_t mtu);*/ #define UPDATE_VF_STAT (reg, last, cur) \ { \ - u32 latest = IXGBE_READ_REG(hw, reg); \ + uint32_t latest = IXGBE_READ_REG(hw, reg); \ cur += latest - last; \ last = latest; \ }
@@ -343,6 +367,8 @@ static const struct eth_dev_ops ixgbe_eth_dev_ops = { .tx_queue_start = ixgbe_dev_tx_queue_start, .tx_queue_stop = ixgbe_dev_tx_queue_stop, .rx_queue_setup = ixgbe_dev_rx_queue_setup, + .rx_queue_intr_enable = ixgbe_dev_rx_queue_intr_enable, + .rx_queue_intr_disable = ixgbe_dev_rx_queue_intr_disable, .rx_queue_release = ixgbe_dev_rx_queue_release, .rx_queue_count = ixgbe_dev_rx_queue_count, .rx_descriptor_done = ixgbe_dev_rx_descriptor_done,
@@ -402,8 +428,11 @@ static const struct eth_dev_ops ixgbevf_eth_dev_ops = { .vlan_offload_set = ixgbevf_vlan_offload_set, .rx_queue_setup = ixgbe_dev_rx_queue_setup, .rx_queue_release = ixgbe_dev_rx_queue_release, + .rx_descriptor_done = ixgbe_dev_rx_descriptor_done, .tx_queue_setup = ixgbe_dev_tx_queue_setup, .tx_queue_release = ixgbe_dev_tx_queue_release, + .rx_queue_intr_enable = ixgbevf_dev_rx_queue_intr_enable, + .rx_queue_intr_disable = ixgbevf_dev_rx_queue_intr_disable, .mac_addr_add = ixgbevf_add_mac_addr, .mac_addr_remove = ixgbevf_remove_mac_addr, };
@@ -899,12 +928,6 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev) eth_dev->data->port_id, pci_dev->id.vendor_id, pci_dev->id.device_id); -rte_intr_callback_register (&(pci_dev->intr_handle), - ixgbe_dev_interrupt_handler, (void *)eth_dev); - -/* enable uio intr after callback register */ -rte_intr_enable (&(pci_dev->intr_handle)); -/* enable support intr */ ixgbe_enable_intr (eth_dev);
@@ -1430,6 +1453,8 @@ ixgbe_dev_configure(struct rte_eth_dev *dev)IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private);struct ixgbe_hw *hw =IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle; +uint32_t intr_vector =0 ;PMD_INIT_FUNC_TRACE ();
@@ -1443,6 +1468,24 @@ ixgbe_dev_configure(struct rte_eth_dev *dev) hw->rx_bulk_alloc_allowed = true; hw->rx_vec_allowed = true; +if (dev->data->dev_conf.intr_conf.rxq !=0 ) + intr_vector = dev->data->nb_rx_queues; + +if (rte_intr_efd_enable (intr_handle, intr_vector)) +return -1 ; + +if (rte_intr_dp_is_en (intr_handle)) { + intr_handle->intr_vec = +rte_zmalloc ("intr_vec" , + dev->data->nb_rx_queues *sizeof (int ), +0 ); +if (intr_handle->intr_vec ==NULL ) { +PMD_INIT_LOG (ERR ,"Failed to allocate %d rx_queues" +"intr_vec\n" , dev->data->nb_rx_queues); +return -1 ; + } + } +return 0 ; }
@@ -1489,6 +1532,9 @@ ixgbe_dev_start(struct rte_eth_dev *dev)/* configure PF module if SRIOV enabled */ ixgbe_pf_host_configure (dev); +/* confiugre msix for sleep until rx interrupt */ +ixgbe_configure_msix (dev); +/* initialize transmission unit */ ixgbe_dev_tx_init (dev);
@@ -1561,8 +1607,23 @@ ixgbe_dev_start(struct rte_eth_dev *dev) skip_link_setup:/* check if lsc interrupt is enabled */ -if (dev->data->dev_conf.intr_conf.lsc !=0 ) -ixgbe_dev_lsc_interrupt_setup (dev); +if (dev->data->dev_conf.intr_conf.lsc !=0 ) { +if (rte_intr_allow_others (&dev->pci_dev->intr_handle)) { +rte_intr_callback_register (&(dev->pci_dev->intr_handle), + ixgbe_dev_interrupt_handler, + (void *)dev); +ixgbe_dev_lsc_interrupt_setup (dev); + }else +PMD_INIT_LOG (INFO ,"lsc won't enable because of" +" no intr multiplex\n" ); + } + +/* check if rxq interrupt is enabled */ +if (dev->data->dev_conf.intr_conf.rxq !=0 ) +ixgbe_dev_rxq_interrupt_setup (dev); + +/* enable uio/vfio intr/eventfd mapping */ +rte_intr_enable (&(dev->pci_dev->intr_handle));/* resume enabled intr since hw reset */ ixgbe_enable_intr (dev);
@@ -1619,6 +1680,7 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)struct ixgbe_filter_info *filter_info =IXGBE_DEV_PRIVATE_TO_FILTER_INFO (dev->data->dev_private);struct ixgbe_5tuple_filter *p_5tuple, *p_5tuple_next; +struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle;int vf;PMD_INIT_FUNC_TRACE ();
@@ -1626,6 +1688,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)/* disable interrupts */ ixgbe_disable_intr (hw); +/* disable intr eventfd mapping */ +rte_intr_disable (intr_handle); +/* reset the NIC */ ixgbe_pf_reset_hw (hw); hw->adapter_stopped = FALSE;
@@ -1661,6 +1726,12 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)memset (filter_info->fivetuple_mask,0 ,sizeof (uint32_t ) *IXGBE_5TUPLE_ARRAY_SIZE ); +/* Clean datapath event and queue/vec mapping */ +rte_intr_efd_disable (intr_handle); +if (intr_handle->intr_vec !=NULL ) { +rte_free (intr_handle->intr_vec); + intr_handle->intr_vec =NULL ; + } }/*
@@ -2252,6 +2323,28 @@ ixgbe_dev_lsc_interrupt_setup(struct rte_eth_dev *dev)return 0 ; } +/** +* It clears the interrupt causes and enables the interrupt. +* It will be called once only during nic initialized. +* +* @param dev +* Pointer to struct rte_eth_dev. +* +* @return +* - On success, zero. +* - On failure, a negative value. +*/ +static int +ixgbe_dev_rxq_interrupt_setup (struct rte_eth_dev *dev) +{ +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + + intr->mask |=IXGBE_EICR_RTX_QUEUE ; + +return 0 ; +} +/* * It reads ICR and sets flag (IXGBE_EICR_LSC) for the link_update. *
@@ -2278,10 +2371,10 @@ ixgbe_dev_interrupt_get_status(struct rte_eth_dev *dev)PMD_DRV_LOG (INFO ,"eicr %x" , eicr); intr->flags =0 ; -if (eicr &IXGBE_EICR_LSC ) { -/* set flag for async link update */ + +/* set flag for async link update */ +if (eicr &IXGBE_EICR_LSC ) intr->flags |=IXGBE_FLAG_NEED_LINK_UPDATE ; - }if (eicr &IXGBE_EICR_MAILBOX ) intr->flags |=IXGBE_FLAG_MAILBOX ;
@@ -2289,6 +2382,30 @@ ixgbe_dev_interrupt_get_status(struct rte_eth_dev *dev)return 0 ; } +static int +ixgbevf_dev_interrupt_get_status (struct rte_eth_dev *dev) +{ +uint32_t eicr; +struct ixgbe_hw *hw =IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + +/* clear all cause mask */ +ixgbevf_intr_disable (hw); + +/* read-on-clear nic registers here */ + eicr =IXGBE_READ_REG (hw,IXGBE_VTEICR ); +PMD_DRV_LOG (INFO ,"eicr %x" , eicr); + + intr->flags =0 ; + +/* set flag for async link update */ +if (eicr &IXGBE_EICR_LSC ) + intr->flags |=IXGBE_FLAG_NEED_LINK_UPDATE ; + +return 0 ; +} +/** * It gets and then prints the link status. *
@@ -2384,6 +2501,18 @@ ixgbe_dev_interrupt_action(struct rte_eth_dev *dev)return 0 ; } +static int +ixgbevf_dev_interrupt_action (struct rte_eth_dev *dev) +{ +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); + +PMD_DRV_LOG (DEBUG ,"enable intr immediately" ); +ixgbevf_intr_enable (hw); +rte_intr_enable (&(dev->pci_dev->intr_handle)); +return 0 ; +} +/** * Interrupt handler which shall be registered for alarm callback for delayed * handling specific interrupt to wait for the stable nic state. As the
@@ -2445,6 +2574,15 @@ ixgbe_dev_interrupt_handler(__rte_unused struct rte_intr_handle *handle,ixgbe_dev_interrupt_action (dev); } +static void +ixgbevf_dev_interrupt_handler(__rte_unused struct rte_intr_handle *handle, +void *param) +{ +struct rte_eth_dev *dev = (struct rte_eth_dev *)param; +ixgbevf_dev_interrupt_get_status (dev); +ixgbevf_dev_interrupt_action (dev); +} +static int ixgbe_dev_led_on (struct rte_eth_dev *dev) {
@@ -2943,10 +3081,25 @@ ixgbevf_intr_disable(struct ixgbe_hw *hw)IXGBE_WRITE_FLUSH (hw); } +static void +ixgbevf_intr_enable (struct ixgbe_hw *hw) +{ +PMD_INIT_FUNC_TRACE (); + +/* VF enable interrupt autoclean */ +IXGBE_WRITE_REG (hw,IXGBE_VTEIAM ,IXGBE_VF_IRQ_ENABLE_MASK ); +IXGBE_WRITE_REG (hw,IXGBE_VTEIAC ,IXGBE_VF_IRQ_ENABLE_MASK ); +IXGBE_WRITE_REG (hw,IXGBE_VTEIMS ,IXGBE_VF_IRQ_ENABLE_MASK ); + +IXGBE_WRITE_FLUSH (hw); +} +static int ixgbevf_dev_configure (struct rte_eth_dev *dev) {struct rte_eth_conf * conf = &dev->data->dev_conf; +uint32_t intr_vector =0 ; +struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle;PMD_INIT_LOG (DEBUG ,"Configured Virtual Function port id: %d" , dev->data->port_id);
@@ -2967,6 +3120,24 @@ ixgbevf_dev_configure(struct rte_eth_dev *dev) }#endif +if (dev->data->dev_conf.intr_conf.rxq !=0 ) + intr_vector = dev->data->nb_rx_queues; + +if (rte_intr_efd_enable (intr_handle, intr_vector)) +return -1 ; + +/* set max interrupt vfio request */ +if (rte_intr_dp_is_en (intr_handle)) { + intr_handle->intr_vec = +rte_zmalloc ("intr_vec" , + dev->data->nb_rx_queues *sizeof (int ),0 ); +if (intr_handle->intr_vec ==NULL ) { +PMD_INIT_LOG (ERR ,"Failed to allocate %d rx_queues" +" intr_vec\n" , dev->data->nb_rx_queues); +return -ENOMEM ; + } + } +return 0 ; }
@@ -3005,6 +3176,23 @@ ixgbevf_dev_start(struct rte_eth_dev *dev)ixgbevf_dev_rxtx_start (dev); +ixgbevf_configure_msix (dev); + +if (dev->data->dev_conf.intr_conf.lsc !=0 ) { +if (rte_intr_allow_others (&dev->pci_dev->intr_handle)) +rte_intr_callback_register (&dev->pci_dev->intr_handle, + ixgbevf_dev_interrupt_handler, + (void *)dev); +else +PMD_INIT_LOG (INFO ,"lsc won't enable because of" +" no intr multiplex\n" ); + } + +rte_intr_enable (&dev->pci_dev->intr_handle); + +/* Re-enable interrupt for VF */ +ixgbevf_intr_enable (hw); +return 0 ; }
@@ -3012,6 +3200,7 @@ static void ixgbevf_dev_stop(struct rte_eth_dev *dev) {struct ixgbe_hw *hw =IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle;PMD_INIT_FUNC_TRACE ();
@@ -3028,12 +3217,23 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev) dev->data->scattered_rx =0 ;ixgbe_dev_clear_queues (dev); + +/* disable intr eventfd mapping */ +rte_intr_disable (intr_handle); + +/* Clean datapath event and queue/vec mapping */ +rte_intr_efd_disable (intr_handle); +if (intr_handle->intr_vec !=NULL ) { +rte_free (intr_handle->intr_vec); + intr_handle->intr_vec =NULL ; + } }static void ixgbevf_dev_close (struct rte_eth_dev *dev) {struct ixgbe_hw *hw =IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct rte_pci_device *pci_dev;PMD_INIT_FUNC_TRACE ();
@@ -3043,6 +3243,12 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)/* reprogram the RAR[0] in case user changed it. */ ixgbe_set_rar (hw,0 , hw->mac.addr,0 ,IXGBE_RAH_AV ); + + pci_dev = dev->pci_dev; +if (pci_dev->intr_handle.intr_vec) { +rte_free (pci_dev->intr_handle.intr_vec); + pci_dev->intr_handle.intr_vec =NULL ; + } }static void ixgbevf_set_vfta_all (struct rte_eth_dev *dev,bool on)
@@ -3542,6 +3748,258 @@ ixgbe_mirror_rule_reset(struct rte_eth_dev *dev, uint8_t rule_id)return 0 ; } + +static int +ixgbevf_dev_rx_queue_intr_enable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); + + mask =IXGBE_READ_REG (hw,IXGBE_VTEIMS ); + mask |= (1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_VTEIMS , mask); + +rte_intr_enable (&(dev->pci_dev->intr_handle)); + +return 0 ; +} + +static int +ixgbevf_dev_rx_queue_intr_disable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); + + mask =IXGBE_READ_REG (hw,IXGBE_VTEIMS ); + mask &= ~(1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_VTEIMS , mask); + +return 0 ; +} + +static int +ixgbe_dev_rx_queue_intr_enable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + +if (queue_id <16 ) { +ixgbe_disable_intr (hw); + intr->mask |= (1 << queue_id); +ixgbe_enable_intr (dev); + }else if (queue_id <32 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (0 )); + mask &= (1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (0 ), mask); + }else if (queue_id <64 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (1 )); + mask &= (1 << (queue_id -32 )); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (1 ), mask); + } +rte_intr_enable (&(dev->pci_dev->intr_handle)); + +return 0 ; +} + +static int +ixgbe_dev_rx_queue_intr_disable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +uint32_t mask; +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct ixgbe_interrupt *intr = +IXGBE_DEV_PRIVATE_TO_INTR (dev->data->dev_private); + +if (queue_id <16 ) { +ixgbe_disable_intr (hw); + intr->mask &= ~(1 << queue_id); +ixgbe_enable_intr (dev); + }else if (queue_id <32 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (0 )); + mask &= ~(1 << queue_id); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (0 ), mask); + }else if (queue_id <64 ) { + mask =IXGBE_READ_REG (hw,IXGBE_EIMS_EX (1 )); + mask &= ~(1 << (queue_id -32 )); +IXGBE_WRITE_REG (hw,IXGBE_EIMS_EX (1 ), mask); + } + +return 0 ; +} + +static void +ixgbevf_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector) +{ +uint32_t tmp, idx; +if (direction ==-1 ) { +/* other causes */ + msix_vector |=IXGBE_IVAR_ALLOC_VAL ; + tmp =IXGBE_READ_REG (hw,IXGBE_VTIVAR_MISC ); + tmp &= ~0xFF ; + tmp |= msix_vector; +IXGBE_WRITE_REG (hw,IXGBE_VTIVAR_MISC , tmp); + }else { +/* rx or tx cause */ + msix_vector |=IXGBE_IVAR_ALLOC_VAL ; + idx = ((16 * (queue &1 )) + (8 * direction)); + tmp =IXGBE_READ_REG (hw,IXGBE_VTIVAR (queue >>1 )); + tmp &= ~(0xFF << idx); + tmp |= (msix_vector << idx); +IXGBE_WRITE_REG (hw,IXGBE_VTIVAR (queue >>1 ), tmp); + } +} + +/** +* set the IVAR registers, mapping interrupt causes to vectors +* @param hw +* pointer to ixgbe_hw struct +* @direction +* 0 for Rx, 1 for Tx, -1 for other causes +* @queue +* queue to map the corresponding interrupt to +* @msix_vector +* the vector to map to the corresponding queue +*/ +static void +ixgbe_set_ivar_map (struct ixgbe_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector) +{ +uint32_t tmp, idx; + + msix_vector |=IXGBE_IVAR_ALLOC_VAL ; +if (hw->mac.type == ixgbe_mac_82598EB) { +if (direction ==-1 ) + direction =0 ; + idx = (((direction *64 ) + queue) >>2 ) &0x1F ; + tmp =IXGBE_READ_REG (hw,IXGBE_IVAR (idx)); + tmp &= ~(0xFF << (8 * (queue &0x3 ))); + tmp |= (msix_vector << (8 * (queue &0x3 ))); +IXGBE_WRITE_REG (hw,IXGBE_IVAR (idx), tmp); + }else if ((hw->mac.type == ixgbe_mac_82599EB) || + (hw->mac.type == ixgbe_mac_X540)) { +if (direction ==-1 ) { +/* other causes */ + idx = ((queue &1 ) *8 ); + tmp =IXGBE_READ_REG (hw,IXGBE_IVAR_MISC ); + tmp &= ~(0xFF << idx); + tmp |= (msix_vector << idx); +IXGBE_WRITE_REG (hw,IXGBE_IVAR_MISC , tmp); + }else { +/* rx or tx causes */ + idx = ((16 * (queue &1 )) + (8 * direction)); + tmp =IXGBE_READ_REG (hw,IXGBE_IVAR (queue >>1 )); + tmp &= ~(0xFF << idx); + tmp |= (msix_vector << idx); +IXGBE_WRITE_REG (hw,IXGBE_IVAR (queue >>1 ), tmp); + } + } +} + +static void +ixgbevf_configure_msix (struct rte_eth_dev *dev) +{ +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle; +uint32_t q_idx; +uint32_t vector_idx =0 ; + +/* won't configure msix register if no mapping is done +* between intr vector and event fd */ +if (!rte_intr_dp_is_en (intr_handle)) +return ; + +/* Configure all RX queues of VF */ +for (q_idx =0 ; q_idx < dev->data->nb_rx_queues; q_idx++) { +/* Force all queue use vector 0, +* as IXGBE_VF_MAXMSIVECOTR = 1 */ +ixgbevf_set_ivar_map (hw,0 , q_idx, vector_idx); + intr_handle->intr_vec[q_idx] = vector_idx; + } + +/* Configure VF Rx queue ivar */ +ixgbevf_set_ivar_map (hw,-1 ,1 , vector_idx); +} + +/** +* Sets up the hardware to properly generate MSI-X interrupts +* @hw +* board private structure +*/ +static void +ixgbe_configure_msix (struct rte_eth_dev *dev) +{ +struct ixgbe_hw *hw = +IXGBE_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle; +uint32_t queue_id, vec =0 ; +uint32_t mask; +uint32_t gpie; + +/* won't configure msix register if no mapping is done +* between intr vector and event fd */ +if (!rte_intr_dp_is_en (intr_handle)) +return ; + +/* setup GPIE for MSI-x mode */ + gpie =IXGBE_READ_REG (hw,IXGBE_GPIE ); + gpie |=IXGBE_GPIE_MSIX_MODE |IXGBE_GPIE_PBA_SUPPORT | +IXGBE_GPIE_OCD |IXGBE_GPIE_EIAME ; +/* +* auto clearing and auto setting corresponding bits in EIMS +* when MSI-X interrupt is triggered +*/ +if (hw->mac.type == ixgbe_mac_82598EB) +IXGBE_WRITE_REG (hw,IXGBE_EIAM ,IXGBE_EICS_RTX_QUEUE ); +else { +IXGBE_WRITE_REG (hw,IXGBE_EIAM_EX (0 ),0xFFFFFFFF ); +IXGBE_WRITE_REG (hw,IXGBE_EIAM_EX (1 ),0xFFFFFFFF ); + } +IXGBE_WRITE_REG (hw,IXGBE_GPIE , gpie); + +/* +* Populate the IVAR table and set the ITR values to the +* corresponding register. +*/ +for (queue_id =0 ; queue_id < dev->data->nb_rx_queues; + queue_id++) { +/* by default, 1:1 mapping */ +ixgbe_set_ivar_map (hw,0 , queue_id, vec); + intr_handle->intr_vec[queue_id] = vec; +if (vec < intr_handle->nb_efd -1 ) + vec++; + } + +switch (hw->mac.type) { +case ixgbe_mac_82598EB: +ixgbe_set_ivar_map (hw,-1 ,IXGBE_IVAR_OTHER_CAUSES_INDEX , + intr_handle->max_intr -1 ); +break ; +case ixgbe_mac_82599EB: +case ixgbe_mac_X540: +ixgbe_set_ivar_map (hw,-1 ,1 , intr_handle->max_intr -1 ); +break ; +default : +break ; + } +IXGBE_WRITE_REG (hw,IXGBE_EITR (queue_id), +IXGBE_MIN_INTER_INTERRUPT_INTERVAL_DEFAULT &0xFFF ); + +/* set up to autoclear timer, and the vectors */ + mask =IXGBE_EIMS_ENABLE_MASK ; + mask &= ~(IXGBE_EIMS_OTHER | +IXGBE_EIMS_MAILBOX | +IXGBE_EIMS_LSC ); + +IXGBE_WRITE_REG (hw,IXGBE_EIAC , mask); +} +static int ixgbe_set_queue_rate_limit (struct rte_eth_dev *dev,uint16_t queue_idx,uint16_t tx_rate) {
lib/librte_pmd_ixgbe/ixgbe_ethdev.h +4 -0
@@ -117,6 +117,9 @@ ETH_RSS_IPV6_TCP_EX | \ ETH_RSS_IPV6_UDP_EX) +#define IXGBE_VF_IRQ_ENABLE_MASK 3/* vf irq enable mask */ +#define IXGBE_VF_MAXMSIVECTOR 1 +/* * Information about the fdir mode. */
@@ -325,6 +328,7 @@ uint32_t ixgbe_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id);int ixgbe_dev_rx_descriptor_done (void *rx_queue,uint16_t offset); +int ixgbevf_dev_rx_descriptor_done (void *rx_queue,uint16_t offset);int ixgbe_dev_rx_init (struct rte_eth_dev *dev);
Signature · 1 line
1.8.1.4
PATCH v8 10/11 igb: enable rx queue interrupts for PF
In reply to Cunming LiangThe patch does below for igb PF:
- Setup NIC to generate MSI-X interrupts
- Set the IVAR register to map interrupt causes to vectors
- Implement interrupt enable/disable functions
---
v8 changes
- add vfio-msi/vfio-legacy and uio-legacy support
v7 changes
- add condition check when intr vector is not enabled
v6 changes
- fill queue-vector mapping table
v5 changes
- Rebase the patchset onto the HEAD
v3 changes
- Remove unnecessary variables in e1000_mac_info
- Remove spinlok from PMD
v2 changes
- Consolidate review comments related to coding style
lib/librte_pmd_e1000/igb_ethdev.c | 292 +++++++++++++++++++++++++++++++++----- 1 file changed, 259 insertions(+), 33 deletions(-)
lib/librte_pmd_e1000/igb_ethdev.c +259 -33
@@ -96,6 +96,7 @@ static int eth_igb_flow_ctrl_get(struct rte_eth_dev *dev,static int eth_igb_flow_ctrl_set (struct rte_eth_dev *dev,struct rte_eth_fc_conf *fc_conf);static int eth_igb_lsc_interrupt_setup (struct rte_eth_dev *dev); +static int eth_igb_rxq_interrupt_setup (struct rte_eth_dev *dev);static int eth_igb_interrupt_get_status (struct rte_eth_dev *dev);static int eth_igb_interrupt_action (struct rte_eth_dev *dev); static void eth_igb_interrupt_handler(struct rte_intr_handle *handle,
@@ -194,6 +195,16 @@ static int eth_igb_filter_ctrl(struct rte_eth_dev *dev,enum rte_filter_op filter_op, void *arg); +static int eth_igb_rx_queue_intr_enable (struct rte_eth_dev *dev, +uint16_t queue_id); +static int eth_igb_rx_queue_intr_disable (struct rte_eth_dev *dev, +uint16_t queue_id); +static void eth_igb_assign_msix_vector (struct e1000_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector); +static void eth_igb_configure_msix_intr (struct rte_eth_dev *dev); +static void eth_igb_write_ivar (struct e1000_hw *hw,uint8_t msix_vector, +uint8_t index,uint8_t offset); +/* * Define VF Stats MACRO for Non "cleared on read" register */
@@ -253,6 +264,8 @@ static const struct eth_dev_ops eth_igb_ops = { .vlan_tpid_set = eth_igb_vlan_tpid_set, .vlan_offload_set = eth_igb_vlan_offload_set, .rx_queue_setup = eth_igb_rx_queue_setup, + .rx_queue_intr_enable = eth_igb_rx_queue_intr_enable, + .rx_queue_intr_disable = eth_igb_rx_queue_intr_disable, .rx_queue_release = eth_igb_rx_queue_release, .rx_queue_count = eth_igb_rx_queue_count, .rx_descriptor_done = eth_igb_rx_descriptor_done,
@@ -584,12 +597,6 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev) eth_dev->data->port_id, pci_dev->id.vendor_id, pci_dev->id.device_id); -rte_intr_callback_register (&(pci_dev->intr_handle), - eth_igb_interrupt_handler, (void *)eth_dev); - -/* enable uio intr after callback register */ -rte_intr_enable (&(pci_dev->intr_handle)); -/* enable support intr */ igb_intr_enable (eth_dev);
@@ -739,11 +746,30 @@ eth_igb_configure(struct rte_eth_dev *dev) {struct e1000_interrupt *intr =E1000_DEV_PRIVATE_TO_INTR (dev->data->dev_private); +struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle; +uint32_t intr_vector =0 ;PMD_INIT_FUNC_TRACE (); intr->flags |=E1000_FLAG_NEED_LINK_UPDATE ;PMD_INIT_FUNC_TRACE (); +if (dev->data->dev_conf.intr_conf.rxq !=0 ) + intr_vector = dev->data->nb_rx_queues; + +if (rte_intr_efd_enable (intr_handle, intr_vector)) +return -1 ; + +if (rte_intr_dp_is_en (intr_handle)) { + intr_handle->intr_vec = +rte_zmalloc ("intr_vec" , + dev->data->nb_rx_queues *sizeof (int ),0 ); +if (intr_handle->intr_vec ==NULL ) { +PMD_INIT_LOG (ERR ,"Failed to allocate %d rx_queues" +" intr_vec\n" , dev->data->nb_rx_queues); +return -ENOMEM ; + } + } +return (0 ); }
@@ -752,7 +778,7 @@ eth_igb_start(struct rte_eth_dev *dev) {struct e1000_hw *hw =E1000_DEV_PRIVATE_TO_HW (dev->data->dev_private); -int ret, i, mask; +int ret, mask;uint32_t ctrl_ext;PMD_INIT_FUNC_TRACE ();
@@ -792,6 +818,9 @@ eth_igb_start(struct rte_eth_dev *dev)/* configure PF module if SRIOV enabled */ igb_pf_host_configure (dev); +/* confiugre msix for rx interrupt */ +eth_igb_configure_msix_intr (dev); +/* Configure for OS presence */ igb_init_manageability (hw);
@@ -819,33 +848,9 @@ eth_igb_start(struct rte_eth_dev *dev)igb_vmdq_vlan_hw_filter_enable (dev); } -/* -* Configure the Interrupt Moderation register (EITR) with the maximum -* possible value (0xFFFF) to minimize "System Partial Write" issued by -* spurious [DMA] memory updates of RX and TX ring descriptors. -* -* With a EITR granularity of 2 microseconds in the 82576, only 7/8 -* spurious memory updates per second should be expected. -* ((65535 * 2) / 1000.1000 ~= 0.131 second). -* -* Because interrupts are not used at all, the MSI-X is not activated -* and interrupt moderation is controlled by EITR[0]. -* -* Note that having [almost] disabled memory updates of RX and TX ring -* descriptors through the Interrupt Moderation mechanism, memory -* updates of ring descriptors are now moderated by the configurable -* value of Write-Back Threshold registers. -*/ if ((hw->mac.type == e1000_82576) || (hw->mac.type == e1000_82580) || (hw->mac.type == e1000_i350) || (hw->mac.type == e1000_i210) || (hw->mac.type == e1000_i211)) { -uint32_t ivar; - -/* Enable all RX & TX queues in the IVAR registers */ - ivar = (uint32_t ) ((E1000_IVAR_VALID <<16 ) |E1000_IVAR_VALID ); -for (i =0 ; i <8 ; i++) -E1000_WRITE_REG_ARRAY (hw,E1000_IVAR0 , i, ivar); -/* Configure EITR with the maximum possible value (0xFFFF) */ E1000_WRITE_REG (hw,E1000_EITR (0 ),0xFFFF ); }
@@ -896,8 +901,23 @@ eth_igb_start(struct rte_eth_dev *dev)e1000_setup_link (hw);/* check if lsc interrupt feature is enabled */ -if (dev->data->dev_conf.intr_conf.lsc !=0 ) - ret =eth_igb_lsc_interrupt_setup (dev); +if (dev->data->dev_conf.intr_conf.lsc !=0 ) { +if (rte_intr_allow_others (&dev->pci_dev->intr_handle)) { +rte_intr_callback_register (&(dev->pci_dev->intr_handle), + eth_igb_interrupt_handler, + (void *)dev); +eth_igb_lsc_interrupt_setup (dev); + }else +PMD_INIT_LOG (INFO ,"lsc won't enable because of" +" no intr multiplex\n" ); + } + +/* check if rxq interrupt is enabled */ +if (dev->data->dev_conf.intr_conf.rxq !=0 ) +eth_igb_rxq_interrupt_setup (dev); + +/* enable uio/vfio intr/eventfd mapping */ +rte_intr_enable (&(dev->pci_dev->intr_handle));/* resume enabled intr since hw reset */ igb_intr_enable (dev);
@@ -930,8 +950,13 @@ eth_igb_stop(struct rte_eth_dev *dev)struct e1000_flex_filter *p_flex;struct e1000_5tuple_filter *p_5tuple, *p_5tuple_next;struct e1000_2tuple_filter *p_2tuple, *p_2tuple_next; +struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle;igb_intr_disable (hw); + +/* disable intr eventfd mapping */ +rte_intr_disable (intr_handle); +igb_pf_reset_hw (hw);E1000_WRITE_REG (hw,E1000_WUC ,0 );
@@ -980,6 +1005,13 @@ eth_igb_stop(struct rte_eth_dev *dev)rte_free (p_2tuple); } filter_info->twotuple_mask =0 ; + +/* Clean datapath event and queue/vec mapping */ +rte_intr_efd_disable (intr_handle); +if (intr_handle->intr_vec !=NULL ) { +rte_free (intr_handle->intr_vec); + intr_handle->intr_vec =NULL ; + } } static void
@@ -987,6 +1019,7 @@ eth_igb_close(struct rte_eth_dev *dev) {struct e1000_hw *hw =E1000_DEV_PRIVATE_TO_HW (dev->data->dev_private);struct rte_eth_link link; +struct rte_pci_device *pci_dev;eth_igb_stop (dev);e1000_phy_hw_reset (hw);
@@ -1004,6 +1037,12 @@ eth_igb_close(struct rte_eth_dev *dev)igb_dev_clear_queues (dev); + pci_dev = dev->pci_dev; +if (pci_dev->intr_handle.intr_vec) { +rte_free (pci_dev->intr_handle.intr_vec); + pci_dev->intr_handle.intr_vec =NULL ; + } +memset (&link,0 ,sizeof (link));rte_igb_dev_atomic_write_link_status (dev, &link); }
@@ -1828,6 +1867,34 @@ eth_igb_lsc_interrupt_setup(struct rte_eth_dev *dev) }/* +* It clears the interrupt causes and enables the interrupt. +* It will be called once only during nic initialized. +* +* @param dev +* Pointer to struct rte_eth_dev. +* +* @return +* - On success, zero. +* - On failure, a negative value. +*/ +static int eth_igb_rxq_interrupt_setup (struct rte_eth_dev *dev) +{ +uint32_t mask, regval; +struct e1000_hw *hw = +E1000_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct rte_eth_dev_info dev_info; + +memset (&dev_info,0 ,sizeof (dev_info)); +eth_igb_infos_get (dev, &dev_info); + + mask =0xFFFFFFFF >> (32 - dev_info.max_rx_queues); + regval =E1000_READ_REG (hw,E1000_EIMS ); +E1000_WRITE_REG (hw,E1000_EIMS , regval | mask); + +return 0 ; +} + +/* * It reads ICR and gets interrupt causes, check it and set a bit flag * to update link status. *
@@ -3652,5 +3719,164 @@ static struct rte_driver pmd_igbvf_drv = { .init = rte_igbvf_pmd_init, }; +static int +eth_igb_rx_queue_intr_disable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +struct e1000_hw *hw = +E1000_DEV_PRIVATE_TO_HW (dev->data->dev_private); +uint32_t mask =1 << queue_id; + +E1000_WRITE_REG (hw,E1000_EIMC , mask); +E1000_WRITE_FLUSH (hw); + +return 0 ; +} + +static int +eth_igb_rx_queue_intr_enable (struct rte_eth_dev *dev,uint16_t queue_id) +{ +struct e1000_hw *hw = +E1000_DEV_PRIVATE_TO_HW (dev->data->dev_private); +uint32_t mask =1 << queue_id; +uint32_t regval; + + regval =E1000_READ_REG (hw,E1000_EIMS ); +E1000_WRITE_REG (hw,E1000_EIMS , regval | mask); +E1000_WRITE_FLUSH (hw); + +rte_intr_enable (&(dev->pci_dev->intr_handle)); + +return 0 ; +} + +static void +eth_igb_write_ivar (struct e1000_hw *hw,uint8_t msix_vector, +uint8_t index,uint8_t offset) +{ +uint32_t val =E1000_READ_REG_ARRAY (hw,E1000_IVAR0 , index); + +/* clear bits */ + val &= ~((uint32_t )0xFF << offset); + +/* write vector and valid bit */ + val |= (msix_vector |E1000_IVAR_VALID ) << offset; + +E1000_WRITE_REG_ARRAY (hw,E1000_IVAR0 , index, val); +} + +static void +eth_igb_assign_msix_vector (struct e1000_hw *hw,int8_t direction, +uint8_t queue,uint8_t msix_vector) +{ +uint32_t tmp =0 ; +if (hw->mac.type == e1000_82575) { +if (direction ==0 ) + tmp =E1000_EICR_RX_QUEUE0 << queue; +else if (direction ==1 ) + tmp =E1000_EICR_TX_QUEUE0 << queue; +E1000_WRITE_REG (hw,E1000_MSIXBM (msix_vector), tmp); + }else if (hw->mac.type == e1000_82576) { +if ((direction ==0 ) || (direction ==1 )) +eth_igb_write_ivar (hw, msix_vector, queue &0x7 , + ((queue &0x8 ) <<1 ) +8 * direction); + }else if ((hw->mac.type == e1000_82580) || + (hw->mac.type == e1000_i350) || + (hw->mac.type == e1000_i354) || + (hw->mac.type == e1000_i210) || + (hw->mac.type == e1000_i211)) { +if ((direction ==0 ) || (direction ==1 )) +eth_igb_write_ivar (hw, msix_vector, + queue >>1 , + ((queue &0x1 ) <<4 ) +8 * direction); + } +} + +/* +* Sets up the hardware to generate MSI-X interrupts properly +* @hw +* board private structure +*/ +static void +eth_igb_configure_msix_intr (struct rte_eth_dev *dev) +{ +int queue_id; +uint32_t tmpval, regval, intr_mask; +uint32_t max_rx_queues; +struct rte_eth_dev_info dev_info; +struct e1000_hw *hw = +E1000_DEV_PRIVATE_TO_HW (dev->data->dev_private); +struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle; +uint32_t vec =0 ; + +/* won't configure msix register if no mapping is done +* between intr vector and event fd */ +if (!rte_intr_dp_is_en (intr_handle)) +return ; + +memset (&dev_info,0 ,sizeof (dev_info)); +eth_igb_infos_get (dev, &dev_info); + max_rx_queues = dev_info.max_rx_queues; + +/* set interrupt vector for other causes */ +if (hw->mac.type == e1000_82575) { + tmpval =E1000_READ_REG (hw,E1000_CTRL_EXT ); +/* enable MSI-X PBA support */ + tmpval |=E1000_CTRL_EXT_PBA_CLR ; + +/* Auto-Mask interrupts upon ICR read */ + tmpval |=E1000_CTRL_EXT_EIAME ; + tmpval |=E1000_CTRL_EXT_IRCA ; + +E1000_WRITE_REG (hw,E1000_CTRL_EXT , tmpval); + +/* enable msix_other interrupt */ +E1000_WRITE_REG_ARRAY (hw,E1000_MSIXBM (0 ),0 ,E1000_EIMS_OTHER ); + regval =E1000_READ_REG (hw,E1000_EIAC ); +E1000_WRITE_REG (hw,E1000_EIAC , regval |E1000_EIMS_OTHER ); + regval =E1000_READ_REG (hw,E1000_EIAM ); +E1000_WRITE_REG (hw,E1000_EIMS , regval |E1000_EIMS_OTHER ); + }else if ((hw->mac.type == e1000_82576) || + (hw->mac.type == e1000_82580) || + (hw->mac.type == e1000_i350) || + (hw->mac.type == e1000_i354) || + (hw->mac.type == e1000_i210) || + (hw->mac.type == e1000_i211)) { +/* turn on MSI-X capability first */ +E1000_WRITE_REG (hw,E1000_GPIE ,E1000_GPIE_MSIX_MODE | +E1000_GPIE_PBA |E1000_GPIE_EIAME | +E1000_GPIE_NSICR ); + +/* enable msix_other interrupt */ + intr_mask =1 << max_rx_queues; + regval =E1000_READ_REG (hw,E1000_EIAC ); +E1000_WRITE_REG (hw,E1000_EIAC , regval | intr_mask); + regval =E1000_READ_REG (hw,E1000_EIMS ); +E1000_WRITE_REG (hw,E1000_EIMS , regval | intr_mask); + tmpval = (max_rx_queues |E1000_IVAR_VALID ) <<8 ; + +E1000_WRITE_REG (hw,E1000_IVAR_MISC , tmpval); + } + +/* +* use EIAM and EIAC to auto-mask and auto-clear when MSI-X interrupt +* is asserted, this saves a register write for every interrupt +*/ + intr_mask =0xFFFFFFFF >> (32 - max_rx_queues); + regval =E1000_READ_REG (hw,E1000_EIAC ); +E1000_WRITE_REG (hw,E1000_EIAC , regval | intr_mask); + regval =E1000_READ_REG (hw,E1000_EIAM ); +E1000_WRITE_REG (hw,E1000_EIAM , regval | intr_mask); + +for (queue_id =0 ; queue_id < dev->data->nb_rx_queues; queue_id++) { +eth_igb_assign_msix_vector (hw,0 , queue_id, vec); + intr_handle->intr_vec[queue_id] = vec; +if (vec < intr_handle->nb_efd -1 ) + vec++; + } + +E1000_WRITE_FLUSH (hw); +} + +PMD_REGISTER_DRIVER (pmd_igb_drv);PMD_REGISTER_DRIVER (pmd_igbvf_drv);
Signature · 1 line
1.8.1.4
PATCH v8 11/11 l3fwd-power: enable one-shot rx interrupt and polling/interrupt mode switch
In reply to Cunming LiangDemonstrate how to handle per rx queue interrupt in a NAPI-like implementation in usersapce. PDK polling thread mainly works in polling mode and switch to interrupt mode only if there is no any packet received in recent polls.
Usersapce interrupt notification generally takes a lot more cycles than kernel, so one-shot interrupt is used here to guarantee minimum overhead and DPDK polling thread returns to polling mode immediately once it receives an interrupt notificaiton for incoming packet.
---
v7 changes
- using new APIs
- demo multiple port/queue pair wait on the same epoll instance
v6 changes
- Split event fd add and wait
v5 changes
- Change invoked function name and parameter to accomodate EAL change
v3 changes
- Add spinlock to ensure thread safe when accessing interrupt mask register
v2 changes
- Remove unused function which is for debug purpose
examples/l3fwd-power/main.c | 207 +++++++++++++++++++++++++++++++++++--------- 1 file changed, 165 insertions(+), 42 deletions(-)
examples/l3fwd-power/main.c +165 -42
@@ -74,12 +74,14 @@#include <rte_string_fns.h> #include <rte_timer.h> #include <rte_power.h> +#include <rte_eal.h> +#include <rte_spinlock.h> #define RTE_LOGTYPE_L3FWD_POWER RTE_LOGTYPE_USER1#define MAX_PKT_BURST 32 -#define MIN_ZERO_POLL_COUNT 5 +#define MIN_ZERO_POLL_COUNT 10/* around 100ms at 2 Ghz */ #define TIMER_RESOLUTION_CYCLES 200000000ULL
@@ -155,6 +157,9 @@ static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT;/* ethernet addresses of ports */ static struct ether_addr ports_eth_addr[RTE_MAX_ETHPORTS ]; +/* ethernet addresses of ports */ +static rte_spinlock_t locks[RTE_MAX_ETHPORTS ]; +/* mask of enabled ports */ static uint32_t enabled_port_mask =0 ;/* Ports set in promiscuous mode off by default. */
@@ -187,6 +192,9 @@ struct lcore_rx_queue {#define MAX_TX_QUEUE_PER_PORT RTE_MAX_ETHPORTS#define MAX_RX_QUEUE_PER_PORT 128 +#define MAX_RX_QUEUE_INTERRUPT_PER_PORT 16 + +#define MAX_LCORE_PARAMS 1024struct lcore_params {uint8_t port_id;
@@ -213,7 +221,7 @@ static uint16_t nb_lcore_params = sizeof(lcore_params_array_default) /static struct rte_eth_conf port_conf = { .rxmode = { - .mq_mode =ETH_MQ_RX_RSS , + .mq_mode =ETH_MQ_RX_RSS , .max_rx_pkt_len =ETHER_MAX_LEN , .split_hdr_size =0 , .header_split =0 ,/**< Header Split disabled */
@@ -225,11 +233,15 @@ static struct rte_eth_conf port_conf = { .rx_adv_conf = { .rss_conf = { .rss_key = NULL, - .rss_hf = ETH_RSS_IP, + .rss_hf = ETH_RSS_UDP, }, }, .txmode = { - .mq_mode = ETH_DCB_NONE, + .mq_mode = ETH_MQ_TX_NONE, + }, + .intr_conf = { + .lsc = 1, + .rxq = 1, /**< rxq interrupt feature enabled */ }, };
@@ -401,19 +413,22 @@ power_timer_cb(__attribute__((unused)) struct rte_timer *tim,/* accumulate total execution time in us when callback is invoked */ sleep_time_ratio = (float )(stats[lcore_id].sleep_time) / (float )SCALING_PERIOD ; -/** * check whether need to scale down frequency a step if it sleep a lot. */ -if (sleep_time_ratio >=SCALING_DOWN_TIME_RATIO_THRESHOLD ) -rte_power_freq_down (lcore_id); +if (sleep_time_ratio >=SCALING_DOWN_TIME_RATIO_THRESHOLD ) { +if (rte_power_freq_down) +rte_power_freq_down (lcore_id); + }else if ( (unsigned )(stats[lcore_id].nb_rx_processed / - stats[lcore_id].nb_iteration_looped) <MAX_PKT_BURST ) + stats[lcore_id].nb_iteration_looped) <MAX_PKT_BURST ) {/** * scale down a step if average packet per iteration less * than expectation. */ -rte_power_freq_down (lcore_id); +if (rte_power_freq_down) +rte_power_freq_down (lcore_id); + }/** * initialize another timer according to current frequency to ensure
@@ -706,22 +721,20 @@ l3fwd_simple_forward(struct rte_mbuf *m, uint8_t portid, } -#define SLEEP_GEAR1_THRESHOLD 100 -#define SLEEP_GEAR2_THRESHOLD 1000 +#define MINIMUM_SLEEP_TIME 1 +#define SUSPEND_THRESHOLD 300static inline uint32_t power_idle_heuristic (uint32_t zero_rx_packet_count) { -/* If zero count is less than 100, use it as the sleep time in us */ -if (zero_rx_packet_count <SLEEP_GEAR1_THRESHOLD ) -return zero_rx_packet_count; -/* If zero count is less than 1000, sleep time should be 100 us */ -else if ((zero_rx_packet_count >=SLEEP_GEAR1_THRESHOLD ) && - (zero_rx_packet_count <SLEEP_GEAR2_THRESHOLD )) -return SLEEP_GEAR1_THRESHOLD ; -/* If zero count is greater than 1000, sleep time should be 1000 us */ -else if (zero_rx_packet_count >=SLEEP_GEAR2_THRESHOLD ) -return SLEEP_GEAR2_THRESHOLD ; +/* If zero count is less than 100, sleep 1us */ +if (zero_rx_packet_count <SUSPEND_THRESHOLD ) +return MINIMUM_SLEEP_TIME ; +/* If zero count is less than 1000, sleep 100 us which is the +minimum latency switching from C3/C6 to C0 +*/ +else +return SUSPEND_THRESHOLD ;return 0 ; }
@@ -761,6 +774,84 @@ power_freq_scaleup_heuristic(unsigned lcore_id,return FREQ_CURRENT ; } +/** +* force polling thread sleep until one-shot rx interrupt triggers +* @param port_id +* Port id. +* @param queue_id +* Rx queue id. +* @return +* 0 on success +*/ +static int +sleep_until_rx_interrupt (int num) +{ +struct rte_epoll_event event[num]; +int n, i; +uint8_t port_id, queue_id; +void *data; + +RTE_LOG (INFO ,L3FWD_POWER , +"lcore %u sleeps until interrupt triggers\n" , +rte_lcore_id ()); + + n =rte_epoll_wait (RTE_EPOLL_PER_THREAD , event, num,-1 ); +for (i =0 ; i < n; i++) { + data = event[i].epdata.data; + port_id = ((uintptr_t )data) >>CHAR_BIT ; + queue_id = ((uintptr_t )data) & +RTE_LEN2MASK (CHAR_BIT , uint8_t); +RTE_LOG (INFO ,L3FWD_POWER , +"lcore %u is waked up from rx interrupt on" +" port %d queue %d\n" , +rte_lcore_id (), port_id, queue_id); + } + +return 0 ; +} + +static int turn_on_intr (struct lcore_conf *qconf) +{ +int i; +struct lcore_rx_queue *rx_queue; +uint8_t port_id, queue_id; + +for (i =0 ; i < qconf->n_rx_queue; ++i) { + rx_queue = &(qconf->rx_queue_list[i]); + port_id = rx_queue->port_id; + queue_id = rx_queue->queue_id; + +rte_spinlock_lock (&(locks[port_id])); +rte_eth_dev_rx_intr_enable (port_id, queue_id); +rte_spinlock_unlock (&(locks[port_id])); + } +} + +static int event_register (struct lcore_conf *qconf) +{ +struct lcore_rx_queue *rx_queue; +uint8_t portid, queueid; +uint32_t data; +int ret; +int i; + +for (i =0 ; i < qconf->n_rx_queue; ++i) { + rx_queue = &(qconf->rx_queue_list[i]); + portid = rx_queue->port_id; + queueid = rx_queue->queue_id; + data = portid <<CHAR_BIT | queueid; + + ret =rte_eth_dev_rx_intr_ctl_q (portid, queueid, +RTE_EPOLL_PER_THREAD , +RTE_INTR_EVENT_ADD , + (void *)((uintptr_t )data)); +if (ret) +return ret; + } + +return 0 ; +} +/* main processing loop */ static int main_loop (__attribute__((unused))void *dummy)
@@ -774,9 +865,9 @@ main_loop(__attribute__((unused)) void *dummy)struct lcore_conf *qconf;struct lcore_rx_queue *rx_queue;enum freq_scale_hint_t lcore_scaleup_hint; -uint32_t lcore_rx_idle_count =0 ;uint32_t lcore_idle_hint =0 ; +int intr_en =0 ;const uint64_t drain_tsc = (rte_get_tsc_hz () +US_PER_S -1 ) /US_PER_S *BURST_TX_DRAIN_US ;
@@ -793,13 +884,18 @@ main_loop(__attribute__((unused)) void *dummy)RTE_LOG (INFO ,L3FWD_POWER ,"entering main loop on lcore %u\n" , lcore_id);for (i =0 ; i < qconf->n_rx_queue; i++) { - portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id;RTE_LOG (INFO ,L3FWD_POWER ," -- lcoreid=%u portid=%hhu " "rxqueueid=%hhu\n" , lcore_id, portid, queueid); } +/* add into event wait list */ +if (port_conf.intr_conf.rxq &&event_register (qconf) ==0 ) + intr_en =1 ; +else +RTE_LOG (INFO ,L3FWD_POWER ,"RX interrupt won't enable.\n" ); +while (1 ) { stats[lcore_id].nb_iteration_looped++;
@@ -834,6 +930,7 @@ main_loop(__attribute__((unused)) void *dummy) prev_tsc_power = cur_tsc_power; } +start_rx:/* * Read packet from RX queues */
@@ -847,6 +944,7 @@ main_loop(__attribute__((unused)) void *dummy) nb_rx =rte_eth_rx_burst (portid, queueid, pkts_burst,MAX_PKT_BURST ); + stats[lcore_id].nb_rx_processed += nb_rx;if (unlikely (nb_rx ==0 )) {/**
@@ -909,10 +1007,13 @@ main_loop(__attribute__((unused)) void *dummy) rx_queue->freq_up_hint; } -if (lcore_scaleup_hint ==FREQ_HIGHEST ) -rte_power_freq_max (lcore_id); -else if (lcore_scaleup_hint ==FREQ_HIGHER ) -rte_power_freq_up (lcore_id); +if (lcore_scaleup_hint ==FREQ_HIGHEST ) { +if (rte_power_freq_max) +rte_power_freq_max (lcore_id); + }else if (lcore_scaleup_hint ==FREQ_HIGHER ) { +if (rte_power_freq_up) +rte_power_freq_up (lcore_id); + } }else {/** * All Rx queues empty in recent consecutive polls,
@@ -927,16 +1028,23 @@ main_loop(__attribute__((unused)) void *dummy) lcore_idle_hint = rx_queue->idle_hint; } -if ( lcore_idle_hint <SLEEP_GEAR1_THRESHOLD ) +if (lcore_idle_hint <SUSPEND_THRESHOLD )/** -* execute "pause" instruction to avoid context -* switch for short sleep. -*/ +* execute "pause" instruction to avoid context +* switch which generally take hundres of +* microsecond for short sleep. +*/ rte_delay_us (lcore_idle_hint); -else -/* long sleep force runing thread to suspend */ -usleep (lcore_idle_hint); - +else { +/* suspend untill rx interrupt trigges */ +if (intr_en) { +turn_on_intr (qconf); +sleep_until_rx_interrupt ( + qconf->n_rx_queue); + } +/* start receiving packets immediately */ + goto start_rx; + } stats[lcore_id].sleep_time += lcore_idle_hint; } }
@@ -1269,7 +1377,7 @@ setup_hash(int socketid)char s[64 ];/* create ipv4 hash */ -snprintf (s,sizeof (s),"ipv4_l3fwd_hash_%d" , socketid); +rte_snprintf (s,sizeof (s),"ipv4_l3fwd_hash_%d" , socketid); ipv4_l3fwd_hash_params.name = s; ipv4_l3fwd_hash_params.socket_id = socketid; ipv4_l3fwd_lookup_struct[socketid] =
@@ -1279,7 +1387,7 @@ setup_hash(int socketid) "socket %d\n", socketid);/* create ipv6 hash */ -snprintf (s,sizeof (s),"ipv6_l3fwd_hash_%d" , socketid); +rte_snprintf (s,sizeof (s),"ipv6_l3fwd_hash_%d" , socketid); ipv6_l3fwd_hash_params.name = s; ipv6_l3fwd_hash_params.socket_id = socketid; ipv6_l3fwd_lookup_struct[socketid] =
@@ -1472,6 +1580,7 @@ main(int argc, char **argv)unsigned lcore_id;uint64_t hz;uint32_t n_tx_queue, nb_lcores; +uint32_t dev_rxq_num, dev_txq_num;uint8_t portid, nb_rx_queue, queue, socketid;/* catch SIGINT and restore cpufreq governor to ondemand */
@@ -1521,10 +1630,19 @@ main(int argc, char **argv)printf ("Initializing port %d ... " , portid );fflush (stdout); +rte_eth_dev_info_get (portid, &dev_info); + dev_rxq_num = dev_info.max_rx_queues; + dev_txq_num = dev_info.max_tx_queues; + nb_rx_queue =get_port_n_rx_queues (portid); +if (nb_rx_queue > dev_rxq_num) +rte_exit (EXIT_FAILURE , +"Cannot configure not existed rxq: " +"port=%d\n" , portid); + n_tx_queue = nb_lcores; -if (n_tx_queue >MAX_TX_QUEUE_PER_PORT ) - n_tx_queue =MAX_TX_QUEUE_PER_PORT ; +if (n_tx_queue > dev_txq_num) + n_tx_queue = dev_txq_num;printf ("Creating queues: nb_rxq=%d nb_txq=%u... " , nb_rx_queue, (unsigned )n_tx_queue ); ret = rte_eth_dev_configure(portid, nb_rx_queue,
@@ -1548,6 +1666,9 @@ main(int argc, char **argv)if (rte_lcore_is_enabled (lcore_id) ==0 )continue ; +if (queueid >= dev_txq_num) +continue ; +if (numa_on) socketid = \ (uint8_t )rte_lcore_to_socket_id (lcore_id);
@@ -1582,8 +1703,9 @@ main(int argc, char **argv)/* init power management library */ ret =rte_power_init (lcore_id);if (ret) -rte_exit (EXIT_FAILURE ,"Power management library " -"initialization failed on core%u\n" , lcore_id); +rte_log (RTE_LOG_ERR ,RTE_LOGTYPE_POWER , +"Power management library initialization " +"failed on core%u" , lcore_id);/* init timer structures for each enabled lcore */ rte_timer_init (&power_timers[lcore_id]);
@@ -1631,7 +1753,6 @@ main(int argc, char **argv)if (ret <0 )rte_exit (EXIT_FAILURE ,"rte_eth_dev_start: err=%d, " "port=%d\n" , ret, portid); -/* * If enabled, put device in promiscuous mode. * This allows IO forwarding mode to forward packets
@@ -1640,6 +1761,8 @@ main(int argc, char **argv)*/ if (promiscuous_on)rte_eth_promiscuous_enable (portid); +/* initialize spinlock for each port */ +rte_spinlock_init (&(locks[portid])); }check_all_ports_link_status ((uint8_t )nb_ports, enabled_port_mask);
Signature · 1 line
1.8.1.4
v9 cover Interrupt mode PMD
In reply to Cunming Liangv9 changes
- code rework to fix open comment
- bug fix for igb lsc when both lsc and rxq are enabled in vfio-msix
- new patch to turn off the feature by defalut so as to avoid v2.1 abi broken
v8 changes
- remove condition check for only vfio-msix
- add multiplex intr support when only one intr vector allowed
- lsc and rxq interrupt runtime enable decision
- add safe event delete while the event wakeup execution happens
v7 changes
- decouple epoll event and intr operation
- add condition check in the case intr vector is disabled
- renaming some APIs
v6 changes
- split rte_intr_wait_rx_pkt into two APIs 'wait' and 'set'.
- rewrite rte_intr_rx_wait/rte_intr_rx_set.
- using vector number instead of queue_id as interrupt API params.
- patch reorder and split.
v5 changes
- Rebase the patchset onto the HEAD
- Isolate ethdev from EAL for new-added wait-for-rx interrupt function
- Export wait-for-rx interrupt function for shared libraries
- Split-off a new patch file for changed struct rte_intr_handle that other patches depend on, to avoid breaking git bisect
- Change sample applicaiton to accomodate EAL function spec change accordingly
v4 changes
- Export interrupt enable/disable functions for shared libraries
- Adjust position of new-added structure fields and functions to avoid breaking ABI
v3 changes
- Add return value for interrupt enable/disable functions
- Move spinlok from PMD to L3fwd-power
- Remove unnecessary variables in e1000_mac_info
- Fix miscelleous review comments
v2 changes
- Fix compilation issue in Makefile for missed header file.
- Consolidate internal and community review comments of v1 patch set.
The patch series introduce low-latency one-shot rx interrupt into DPDK with polling and interrupt mode switch control example.
DPDK userspace interrupt notification and handling mechanism is based on UIO with below limitation:
1) It is designed to handle LSC interrupt only with inefficient suspended pthread wakeup procedure (e.g. UIO wakes up LSC interrupt handling thread which then wakes up DPDK polling thread). In this way, it introduces non-deterministic wakeup latency for DPDK polling thread as well as packet latency if it is used to handle Rx interrupt.
2) UIO only supports a single interrupt vector which has to been shared by LSC interrupt and interrupts assigned to dedicated rx queues.
This patchset includes below features:
1) Enable one-shot rx queue interrupt in ixgbe PMD(PF & VF) and igb PMD(PF only).
2) Build on top of the VFIO mechanism instead of UIO, so it could support up to 64 interrupt vectors for rx queue interrupts.
3) Have 1 DPDK polling thread handle per Rx queue interrupt with a dedicated VFIO eventfd, which eliminates non-deterministic pthread wakeup latency in user space.
4) Demonstrate interrupts control APIs and userspace NAIP-like polling/interrupt switch algorithms in L3fwd-power example.
Known limitations:
1) It does not work for UIO due to a single interrupt eventfd shared by LSC and rx queue interrupt handlers causes a mess. [FIXED]
2) LSC interrupt is not supported by VF driver, so it is by default disabled in L3fwd-power now. Feel free to turn in on if you want to support both LSC and rx queue interrupts on a PF.
Cunming Liang (12):
eal/linux: add interrupt vectors support in intr_handle eal/linux: add rte_epoll_wait/ctl support
eal/linux: add API to set rx interrupt event monitor eal/linux: fix comments typo on vfio msi
eal/linux: add interrupt vectors handling on VFIO eal/linux: standalone intr event fd create support eal/bsd: dummy for new intr definition
ethdev: add rx intr enable, disable and ctl functions
ixgbe: enable rx queue interrupts for both PF and VF
igb: enable rx queue interrupts for PF
l3fwd-power: enable one-shot rx interrupt and polling/interrupt mode
switch
abi: fix v2.1 abi broken issue
drivers/net/e1000/igb_ethdev.c | 311 ++++++++++-- drivers/net/ixgbe/ixgbe_ethdev.c | 519 ++++++++++++++++++++- drivers/net/ixgbe/ixgbe_ethdev.h | 4 + examples/l3fwd-power/main.c | 207 ++++++-- lib/librte_eal/bsdapp/eal/eal_interrupts.c | 19 + .../bsdapp/eal/include/exec-env/rte_interrupts.h | 81 ++++ lib/librte_eal/bsdapp/eal/rte_eal_version.map | 5 + lib/librte_eal/linuxapp/eal/eal_interrupts.c | 358 ++++++++++++-- .../linuxapp/eal/include/exec-env/rte_interrupts.h | 219 +++++++++ lib/librte_eal/linuxapp/eal/rte_eal_version.map | 8 + lib/librte_ether/rte_ethdev.c | 109 +++++ lib/librte_ether/rte_ethdev.h | 132 ++++++ lib/librte_ether/rte_ether_version.map | 4 + 13 files changed, 1851 insertions(+), 125 deletions(-)
Signature · 1 line
1.8.1.4
PATCH v9 1/12 eal/linux: add interrupt vectors support in intr_handle
The patch adds interrupt vectors support in rte_intr_handle.
'vec_en' is set when interrupt vectors are detected and associated event fds are set. Those event fds are stored in efds[].
'intr_vec' is reserved for device driver to initialize the vector mapping table.
When the event fds add to a specified epoll instance, 'eptrs' will hold the rte_epoll_event object pointer.
---
v7 changes:
- add eptrs[], it's used to store the register rte_epoll_event instances.
- add vec_en, to log the vector capability status.
v6 changes:
- add mapping table between irq vector number and queue id.
v5 changes:
- Create this new patch file for changed struct rte_intr_handle that other patches depend on, to avoid breaking git bisect.
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h | 10 ++++++++++ 1 file changed, 10 insertions(+)
lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h +10 -0
@@ -38,6 +38,8 @@#ifndef _RTE_LINUXAPP_INTERRUPTS_H_#define _RTE_LINUXAPP_INTERRUPTS_H_ +#define RTE_MAX_RXTX_INTR_VEC_ID 32 +enum rte_intr_handle_type {RTE_INTR_HANDLE_UNKNOWN =0 , RTE_INTR_HANDLE_UIO, /**< uio device handle */
@@ -49,6 +51,8 @@ enum rte_intr_handle_type {RTE_INTR_HANDLE_MAX }; +struct rte_epoll_event ; +/** Handle for interrupts. */ struct rte_intr_handle { union {
@@ -58,6 +62,12 @@ struct rte_intr_handle { };int fd;/**< interrupt event file descriptor */ enum rte_intr_handle_type type;/**< handle type */ +uint32_t max_intr;/**< max interrupt requested */ +uint32_t nb_efd;/**< number of available efds */ +int efds[RTE_MAX_RXTX_INTR_VEC_ID ];/**< intr vectors/efds mapping */ +struct rte_epoll_event *elist[RTE_MAX_RXTX_INTR_VEC_ID ]; +/**< intr vector epoll event ptr */ +int *intr_vec;/**< intr vector number array */ };#endif /* _RTE_LINUXAPP_INTERRUPTS_H_ */
Signature · 1 line
1.8.1.4