Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44F21C433E0 for ; Mon, 1 Feb 2021 11:51:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E87CE64E06 for ; Mon, 1 Feb 2021 11:51:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230432AbhBALv3 convert rfc822-to-8bit (ORCPT ); Mon, 1 Feb 2021 06:51:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51090 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230437AbhBALvT (ORCPT ); Mon, 1 Feb 2021 06:51:19 -0500 Received: from Chamillionaire.breakpoint.cc (Chamillionaire.breakpoint.cc [IPv6:2a0a:51c0:0:12e:520::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D15AAC061573 for ; Mon, 1 Feb 2021 03:50:38 -0800 (PST) Received: from fw by Chamillionaire.breakpoint.cc with local (Exim 4.92) (envelope-from ) id 1l6Xj2-0001Fv-Vc; Mon, 01 Feb 2021 12:50:37 +0100 Date: Mon, 1 Feb 2021 12:50:36 +0100 From: Florian Westphal To: Roi Dayan Cc: Pablo Neira Ayuso , netdev@vger.kernel.org, Paul Blakey , Oz Shlomo , fw@strlen.de Subject: Re: [PATCH net 1/1] netfilter: conntrack: Check offload bit on table dump Message-ID: <20210201115036.GB12443@breakpoint.cc> References: <20210128074052.777999-1-roid@nvidia.com> <20210130120114.GA7846@salvia> <3a29e9b5-7bf8-5c00-3ede-738f9b4725bf@nvidia.com> <997cbda4-acd1-a000-1408-269bc5c3abf3@nvidia.com> <20210201030853.GA19878@salvia> <1229b966-7772-44bd-6e91-fbde213ceb2d@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: 8BIT In-Reply-To: <1229b966-7772-44bd-6e91-fbde213ceb2d@nvidia.com> User-Agent: Mutt/1.10.1 (2018-07-13) Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Roi Dayan wrote: > > > There is a 3rd caller nf_ct_gc_expired() which being called by 3 > > > other callers: > > > ____nf_conntrack_find() > > > nf_conntrack_tuple_taken() > > > early_drop_list() > > > > Hm. I'm not sure yet what path is triggering this bug. > > > > Florian came up with the idea of setting a very large timeout for > > offloaded flows (that are refreshed by the garbage collector) to avoid > > the extra check from the packet path, so those 3 functions above never > > hit the garbage collection path. This also applies for the ctnetlink > > (conntrack -L) and the /proc/net/nf_conntrack sysctl paths that the > > patch describes, those should not ever see an offloaded flow with a > > small timeout. > > > > nf_ct_offload_timeout() is called from: > > > > #1 flow_offload_add() to set a very large timer. > > #2 the garbage collector path, to refresh the timeout the very large > > offload timer. > > > > Probably there is a race between setting the IPS_OFFLOAD and when > > flow_offload_add() is called? Garbage collector gets in between and > > zaps the connection. Is a newly offloaded connection that you observed > > that is being removed? > > > > yes. the flows being removed are newly offloaded connections. If they are new, how can they be timed out already? TCP initial timeout is one minute, UDP 30 seconds. That should surely be enough to do flow_offload_add (which extends the timeout)? Maybe something is doing flow_offload_add() for unconfirmed conntrack? In unconfirmed conntrack case, ct->timeout is absolute timeout value, e.g. for tcp it will be set to 60 * HZ. conntrack confirmation adds jiffies32 to it to make it relative to current time (this is before insertion into the conntrack table, so GC isn't supposed to happen before this). In any case adding test for the offload bit seems to be papering over invalid/broken ct->timeout value.