Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3CD9BCCA480 for ; Tue, 12 Jul 2022 11:05:39 +0000 (UTC) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 4559584187; Tue, 12 Jul 2022 13:05:37 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=konsulko.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=konsulko.com header.i=@konsulko.com header.b="hcn3BGFw"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id EA8718414E; Tue, 12 Jul 2022 13:05:34 +0200 (CEST) Received: from mail-qt1-x831.google.com (mail-qt1-x831.google.com [IPv6:2607:f8b0:4864:20::831]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id AC6428417B for ; Tue, 12 Jul 2022 13:05:29 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=konsulko.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=trini@konsulko.com Received: by mail-qt1-x831.google.com with SMTP id g6so9082287qtu.2 for ; Tue, 12 Jul 2022 04:05:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=konsulko.com; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=oaiYHPGKJt14UkzSK4HhkWSDgraCEd5Vs9f7ldLm4Fs=; b=hcn3BGFwaScoDCsB7WaF/GumiN2n8enXeDOyQoMCgY3wMVGcEjbhtGWo8FD+fL+ZGA Z4YkBr8SkxVz3hpJdCkvVIcjF8/7QTy3QJDGNocEL8YsiRQn/63yatgjoXcCg1J6VRGP 6JNcRsGOmRxwMkTQz5NEN9ZvvvDvClvBAEdWk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=oaiYHPGKJt14UkzSK4HhkWSDgraCEd5Vs9f7ldLm4Fs=; b=sWQIJLP4yqWOzg1NlZV/Lb1aD+oKK27a+decbDpKmFozmjdL501Q606f5TBRJx4wE+ DPyVXdZ9bBMUxl2ljAtoccnB6EtXR6L1KHmF1Tab3NDXD6VYcmAMyk16u9Z9yEEaFJfy Q8N47rOfnmxfirzrOT1N3mtWWUCsaSVm9JL8AaWCofzd9XCYjD/KCEy0kJPD8M4z1NmK p/OZctz3BC3qAqL9BNSva14WYpQI7Gc7IoDGIXf2ork2zt+Y89zRN6QaW/KO1xYHSqxD 1hUGMOnFT/kfeIaFOmkJcKy9av2P/Qy4BEdpULQv0bOIqGn1+aE5Z7jaP6gB2/PGl6jZ 3k2g== X-Gm-Message-State: AJIora80R8flt3lDGJPsihxHZ8EQDsxDFZOziEJIGxFMwcw/fZ030ldV u9iIE51b0/SCo1RvylkV7uav4Q== X-Google-Smtp-Source: AGRyM1vR7hEtsSYj5fsR1sb8D7PfkG1HMzKCpap3LZhLj7CvrnPQcavQ5yVM3Zl9AYQ6a9gU4qmrHA== X-Received: by 2002:a05:622a:2d3:b0:31e:b00e:ab31 with SMTP id a19-20020a05622a02d300b0031eb00eab31mr11262272qtx.460.1657623928343; Tue, 12 Jul 2022 04:05:28 -0700 (PDT) Received: from bill-the-cat (cpe-65-184-195-139.ec.res.rr.com. [65.184.195.139]) by smtp.gmail.com with ESMTPSA id q16-20020a37a710000000b006b548f67f8csm8617479qke.49.2022.07.12.04.05.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Jul 2022 04:05:27 -0700 (PDT) Date: Tue, 12 Jul 2022 07:05:25 -0400 From: Tom Rini To: Simon Glass Cc: U-Boot Mailing List , Wolfgang Denk Subject: Re: [gitdm PATCH 2/2] logparser.py: Try and be more robust with unicode handling Message-ID: <20220712110525.GG1146598@bill-the-cat> References: <20220707192215.509444-1-trini@konsulko.com> <20220707192215.509444-2-trini@konsulko.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="TFvfn7K37npmEsCd" Content-Disposition: inline In-Reply-To: X-Clacks-Overhead: GNU Terry Pratchett X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean --TFvfn7K37npmEsCd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 12, 2022 at 04:58:46AM -0600, Simon Glass wrote: > On Thu, 7 Jul 2022 at 13:22, Tom Rini wrote: > > > > Given the sometimes oddly formatted data that can come through when > > removing code, we need to be as flexible as possible when handling it. > > Set our encoding to unicode_escape and if we still run in to a problem, > > it's likely going to be OK to ignore it. > > > > Signed-off-by: Tom Rini > > --- > > I've emailed this to Jonathan Corbet as well as he's the upstream for > > the project, and this does work for me. But I'm not a python guru by > > any means. But trying to run the stats for v2022.04..v2022.07-rc6 blows > > up in places otherwise. > > > > logparser.py | 1 + > > 1 file changed, 1 insertion(+) >=20 > Reviewed-by: Simon Glass >=20 > BTW I have found that using binary is helpful in many places, the > convert to UTF-8 when displaying things. >=20 >=20 > > > > diff --git a/logparser.py b/logparser.py > > index efbc72f868eb..d5906e97689d 100644 > > --- a/logparser.py > > +++ b/logparser.py > > @@ -37,6 +37,7 @@ class LogPatchSplitter: > > self.fd =3D fd > > self.buffer =3D None > > self.patch =3D [] > > + sys.stdin.reconfigure(encoding=3D'unicode_escape', errors=3D'i= gnore') > > > > def __iter__(self): > > return self So, I followed up with Jonathan, but hadn't yet for the list. unicode_escape works, but then the results don't read right. It turned out utf-8 was the right encoding, but the first time I tried testing it I had some other problem locally. --=20 Tom --TFvfn7K37npmEsCd Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEGjx/cOCPqxcHgJu/FHw5/5Y0tywFAmLNVXUACgkQFHw5/5Y0 tyxkwgwAq8zV0y6MVZE1PyDRgGKyGBNuI1Nu3Vo+acW/N8dc9H0zjdgVq2cQiVod IkhnGE3+5UqDbtlSLYBN8BkPII8cqyyoPSwQ7iVY6Z1m/7IYVNThRDstQxWLat9v Mu4SKjAapuX4hZiexsHhgt/AhBDHG0+cZBPGt54K2LJ5aNJH/npuwXwpkm+04DYV 30ZybeTVeKLj4RCLGFVdTAiijQsR6ClNGZgt+fAIe5r3WqEiiCInGg0W8cRFcJKN en8Ud9ksovEXDTn3OvTwgyyxuF8ZoMCYK1BTXy7nahhMIbrdBAtUj3QUeBQFggWQ KDKOQ77Rd4icW04SJC9xxYZzRIzsQ0M6vk6cFrofRQkI0uulcmVnv547XRDTDQJ8 DygmbT/nN+/ay+1uzGYui7Mll/TR/iFv+ZLslK7okJs7Ansa6U5Ni9kPvaaZU15C k9PB9gqa1T3CJnxJhu2fMA4qe/oEBJw8bRdwJLtTLlsf1qZlRTAC0COd8QJ/ZfMF znmjofG/ =mUjs -----END PGP SIGNATURE----- --TFvfn7K37npmEsCd--