Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A8C69C433F5 for ; Sat, 29 Jan 2022 21:17:02 +0000 (UTC) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id E59888386B; Sat, 29 Jan 2022 22:15:35 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="O8Gkeoos"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 7A3D7836C1; Sat, 29 Jan 2022 22:14:56 +0100 (CET) Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 4FE70836D2 for ; Sat, 29 Jan 2022 22:14:47 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2f.google.com with SMTP id q204so12036216iod.8 for ; Sat, 29 Jan 2022 13:14:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+UDCLkeyUFk12RKyZNud5vhes7n0Zpg07RLG04Nf+hA=; b=O8GkeoosZNq7gDEsEUDA/bo3SEFB81sHTauPh1vaTG8UF+KPDteJCPvbX2TEAG584H YiqL59elKFxg57xA+O6E9BcBIRwdwGV3gJKHYuMjUT/u7hRTry6uwCMtO+LrMTeZOl7D cJcPjZSdd0rOIQDUT3vzby68Fh6t8mlOxHXFo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+UDCLkeyUFk12RKyZNud5vhes7n0Zpg07RLG04Nf+hA=; b=Jqk1ltsfI8357Hok01EmH7l0/Tbmv0ejg0TS7gobsgLRuU3Up7/Lbo8VfB3n5Bjib0 rYcZl/9YhwubxlyBEYBOBMZd5rO06vVaMPrFph7b60YCBh2P8sKkJj0oHloCyg/iWH71 fDOnZkhlIAHJWrHiwFZVV0ewzg2LE5aKYVBzavtCOt5Yx9YFFnws/tah+C05URq5hRMT b6zt2dOyXFOBbNgc2ARfr/PMk/mk0OgFO5YSSgauJeLXv87CjVxpMjij+RFA8NauLLAo u0SiqG6sCN3VvV/ZoYUX75BLPSsLcdLhC3gf4YNZbW9zRZfnvAbrgWBYBO3nx6w2i6GD ryXw== X-Gm-Message-State: AOAM530fhkTvbnl/Gp+8cQ6OfPVtuYXwe4ihOZind9Pc+0zwzoFT3zaP CsLpfJeFsPo3QL6mfGGFrmcObLyNEiiL4w== X-Google-Smtp-Source: ABdhPJwsIsovNRvdD0VZ5O5gcJMmxduafQ14D+CLVYSsoRXViw94Gt5cPhMOVv3t5F5jpNERwjP7Yw== X-Received: by 2002:a5d:8341:: with SMTP id q1mr8097855ior.185.1643490885989; Sat, 29 Jan 2022 13:14:45 -0800 (PST) Received: from kiwi.bld.corp.google.com (c-67-190-101-114.hsd1.co.comcast.net. [67.190.101.114]) by smtp.gmail.com with ESMTPSA id c3sm17218110iow.28.2022.01.29.13.14.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 29 Jan 2022 13:14:45 -0800 (PST) From: Simon Glass To: U-Boot Mailing List Cc: Heinrich Schuchardt , Tom Rini , Simon Glass Subject: [PATCH 17/17] RFC: tools: Add a camel-case conversion script Date: Sat, 29 Jan 2022 14:14:20 -0700 Message-Id: <20220129211420.528117-18-sjg@chromium.org> X-Mailer: git-send-email 2.35.0.rc2.247.g8bbb082509-goog In-Reply-To: <20220129211420.528117-1-sjg@chromium.org> References: <20220129211420.528117-1-sjg@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.5 at phobos.denx.de X-Virus-Status: Clean This is only for posterity, since once the conversion is done, the script is of no use. Signed-off-by: Simon Glass --- scripts/style.py | 180 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 180 insertions(+) create mode 100755 scripts/style.py diff --git a/scripts/style.py b/scripts/style.py new file mode 100755 index 00000000000..7b73b007dea --- /dev/null +++ b/scripts/style.py @@ -0,0 +1,180 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0+ +# +# Copyright 2021 Google LLC +# + +"""Changes the functions and class methods in a file to use snake case, updating +other tools which use them""" + +from argparse import ArgumentParser +import glob +import os +import re +import subprocess + +import camel_case + +# Exclude functions with these names +EXCLUDE_NAMES = set(['setUp', 'tearDown', 'setUpClass', 'tearDownClass']) + +# Find function definitions in a file +RE_FUNC = re.compile(r' *def (\w+)\(') + +# Where to find files that might call the file being converted +FILES_GLOB = 'tools/**/*.py' + +def collect_funcs(fname): + """Collect a list of functions in a file + + Args: + fname (str): Filename to read + + Returns: + tuple: + str: contents of file + list of str: List of function names + """ + with open(fname, encoding='utf-8') as inf: + data = inf.read() + funcs = RE_FUNC.findall(data) + return data, funcs + +def get_module_name(fname): + """Convert a filename to a module name + + Args: + fname (str): Filename to convert, e.g. 'tools/patman/command.py' + + Returns: + tuple: + str: Full module name, e.g. 'patman.command' + str: Leaf module name, e.g. 'command' + str: Program name, e.g. 'patman' + """ + parts = os.path.splitext(fname)[0].split('/')[1:] + module_name = '.'.join(parts) + return module_name, parts[-1], parts[0] + +def process_caller(data, conv, module_name, leaf): + """Process a file that might call another module + + This converts all the camel-case references in the provided file contents + with the corresponding snake-case references. + + Args: + data (str): Contents of file to convert + conv (dict): Identifies to convert + key: Current name in camel case, e.g. 'DoIt' + value: New name in snake case, e.g. 'do_it' + module_name: Name of module as referenced by the file, e.g. + 'patman.command' + leaf: Leaf module name, e.g. 'command' + + Returns: + str: New file contents, or None if it was not modified + """ + total = 0 + + # Update any simple functions calls into the module + for name, new_name in conv.items(): + newdata, count = re.subn(fr'{leaf}.{name}\(', + f'{leaf}.{new_name}(', data) + total += count + data = newdata + + # Deal with files that import symbols individually + imports = re.findall(fr'from {module_name} import (.*)\n', data) + for item in imports: + #print('item', item) + names = [n.strip() for n in item.split(',')] + new_names = [conv.get(n) or n for n in names] + new_line = f"from {module_name} import {', '.join(new_names)}\n" + data = re.sub(fr'from {module_name} import (.*)\n', new_line, data) + for name in names: + new_name = conv.get(name) + if new_name: + newdata = re.sub(fr'\b{name}\(', f'{new_name}(', data) + data = newdata + + # Deal with mocks like: + # unittest.mock.patch.object(module, 'Function', ... + for name, new_name in conv.items(): + newdata, count = re.subn(fr"{leaf}, '{name}'", + f"{leaf}, '{new_name}'", data) + total += count + data = newdata + + if total or imports: + return data + return None + +def process_file(srcfile, do_write, commit): + """Process a file to rename its camel-case functions + + This renames the class methods and functions in a file so that they use + snake case. Then it updates other modules that call those functions. + + Args: + srcfile (str): Filename to process + do_write (bool): True to write back to files, False to do a dry run + commit (bool): True to create a commit with the changes + """ + data, funcs = collect_funcs(srcfile) + module_name, leaf, prog = get_module_name(srcfile) + #print('module_name', module_name) + #print(len(funcs)) + #print(funcs[0]) + conv = {} + for name in funcs: + if name not in EXCLUDE_NAMES: + conv[name] = camel_case.to_snake(name) + + # Convert name to new_name in the file + for name, new_name in conv.items(): + #print(name, new_name) + # Don't match if it is preceded by a '.', since that indicates that + # it is calling this same function name but in a different module + newdata = re.sub(fr'(?