documentation audit script
cover documentation audit script
Hello,
I've been using this script to perform audits of which packages are producing documentation, and feel it's long overdue for this to be added to the git tree. So here it is.
The following changes since commit f586aaa8d00361a9597a546d665077c75cf4d520:
libxml-parser-perl, libxml-simple-perl, expat, sgmlspl-native, git: bump PR to rebuild after perl upgrade (2011-10-25 08:36:01 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib sgarman/docaudit-script
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=sgarman/docaudit-script
Scott Garman (1):
documentation-audit.sh: script for auditing documentation build
status
scripts/contrib/documentation-audit.sh | 92 ++++++++++++++++++++++++++++++++ 1 files changed, 92 insertions(+), 0 deletions(-) create mode 100755 scripts/contrib/documentation-audit.sh
Signature · 1 line
1.7.5.4
PATCH 1/1 documentation-audit.sh: script for auditing documentation build status
This script is used to enumerate which recipes are building documentation. It does this by checking that a -doc package gets generated and contains files.
The script works by building each recipe using the output from bitbake -s.
It will generate several report files, listing which recipes include documentation, which are missing documentation, and which did not successfully build at all.
--- scripts/contrib/documentation-audit.sh | 92 ++++++++++++++++++++++++++++++++ 1 files changed, 92 insertions(+), 0 deletions(-) create mode 100755 scripts/contrib/documentation-audit.sh
scripts/contrib/documentation-audit.sh new file +92 -0
@@ -0,0 +1,92 @@ +#!/bin/bash +# +# Perform an audit of which packages provide documentation and which +# are missing -doc packages. +# +# Setup requirements: be sure to be building for MACHINE=qemux86. Run +# this script after source'ing the build environment script, so you're +# running it from build/ directory. +# +# Maintainer: Scott Garman <[email protected]> + +REPORT_DOC_SIMPLE="documentation_exists.txt" +REPORT_DOC_DETAIL="documentation_exists_detail.txt" +REPORT_MISSING_SIMPLE="documentation_missing.txt" +REPORT_MISSING_DETAIL="documentation_missing_detail.txt" +REPORT_BUILD_ERRORS="build_errors.txt" + +rm -rf $REPORT_DOC_SIMPLE $REPORT_DOC_DETAIL $REPORT_MISSING_SIMPLE $REPORT_MISSING_DETAIL + +BITBAKE=`which bitbake` +if [ -z "$BITBAKE" ]; then + echo "Error: bitbake command not found." + echo "Did you forget to source the build environment script?" + exit 1 +fi + +echo "REMINDER: you need to build for MACHINE=qemux86 or you won't get useful results" +echo "REMINDER: you need to have COMMERCIAL_LICENSE = \"\" in local.conf or you'll get false positives" + +for pkg in `bitbake -s | awk '{ print \$1 }'`; do + if [[ "$pkg" == "Loading" || "$pkg" == "Loaded" || + "$pkg" == "Parsing" || "$pkg" == "Package" || + "$pkg" == "NOTE:" || "$pkg" == "WARNING:" || + "$pkg" == "done." || "$pkg" == "============" ]] + then + # Skip initial bitbake output + continue + fi + if [[ "$pkg" =~ -native$ || "$pkg" =~ -nativesdk$ || + "$pkg" =~ -cross-canadian ]]; then + # Skip native/nativesdk/cross-canadian recipes + continue + fi + if [[ "$pkg" =~ ^meta- || "$pkg" =~ ^task- || "$pkg" =~ -image ]]; then + # Skip meta, task and image recipes + continue + fi + if [[ "$pkg" =~ ^glibc- || "$pkg" =~ ^libiconv$ || + "$pkg" =~ -toolchain$ || "$pkg" =~ ^package-index$ || + "$pkg" =~ ^linux- || "$pkg" =~ ^adt-installer$ || + "$pkg" =~ ^eds-tools$ || "$pkg" =~ ^external-python-tarball$ || + "$pkg" =~ ^qt4-embedded$ || "$pkg" =~ ^qt-mobility ]]; then + # Skip glibc, libiconv, -toolchain, and other recipes known + # to cause build conflicts or trigger false positives. + continue + fi + + echo "Building package $pkg..." + bitbake $pkg > /dev/null + if [ $? -ne 0 ]; then + echo "There was an error building package $pkg" >> "$REPORT_MISSING_DETAIL" + echo "$pkg" >> $REPORT_BUILD_ERRORS + + # Do not skip the remaining tests, as sometimes the + # exit status is 1 due to QA errors, and we can still + # perform the -doc checks. + fi + + echo "$pkg built successfully, checking for a documentation package..." + WORKDIR=`bitbake -e $pkg | grep ^WORKDIR | awk -F '=' '{ print \$2 }' | awk -F '"' '{ print \$2 }'` + FIND_DOC_PKG=`find $WORKDIR/packages-split/*-doc -maxdepth 0 -type d` + if [ -z "$FIND_DOC_PKG" ]; then + # No -doc package was generated: + echo "No -doc package: $pkg" >> "$REPORT_MISSING_DETAIL" + echo "$pkg" >> $REPORT_MISSING_SIMPLE + continue + fi + + FIND_DOC_FILES=`find $FIND_DOC_PKG -type f` + if [ -z "$FIND_DOC_FILES" ]; then + # No files shipped with the -doc package: + echo "No files shipped with the -doc package: $pkg" >> "$REPORT_MISSING_DETAIL" + echo "$pkg" >> $REPORT_MISSING_SIMPLE + continue + fi + + echo "Documentation shipped with $pkg:" >> "$REPORT_DOC_DETAIL" + echo "$FIND_DOC_FILES" >> "$REPORT_DOC_DETAIL" + echo "" >> "$REPORT_DOC_DETAIL" + + echo "$pkg" >> "$REPORT_DOC_SIMPLE" +done
Signature · 1 line
1.7.5.4
Quoted 14 lines · Scott Garman
Hello,
I've been using this script to perform audits of which packages are producing documentation, and feel it's long overdue for this to be added to the git tree. So here it is.
The following changes since commit
f586aaa8d00361a9597a546d665077c75cf4d520:libxml-parser-perl, libxml-simple-perl, expat, sgmlspl-native, git: bump PR to rebuild after perl upgrade (2011-10-25 08:36:01 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contribsgarman/docaudit-script
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=sgarman/docaudit-script
Quoted 8 lines
Scott Garman (1):
documentation-audit.sh: script for auditing documentation build
statusscripts/contrib/documentation-audit.sh | 92 ++++++++++++++++++++++++++++++++ 1 files changed, 92 insertions(+), 0 deletions(-)create mode 100755
scripts/contrib/documentation-audit.sh
Can you please run this and generate a report.
Thanks
Sau!
Quoted 22 lines · Saul Wold
Quoted 21 lines · Scott Garman
Hello,
I've been using this script to perform audits of which packages are producing documentation, and feel it's long overdue for this to be added to the git tree. So here it is.
The following changes since commit
f586aaa8d00361a9597a546d665077c75cf4d520:libxml-parser-perl, libxml-simple-perl, expat, sgmlspl-native, git:
bump PR to rebuild after perl upgrade (2011-10-25 08:36:01 +0100)are available in the git repository at:
git://git.yoctoproject.org/poky-contribsgarman/docaudit-script
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=sgarman/docaudit-scriptScott Garman (1):
documentation-audit.sh: script for auditing documentation build status
Quoted 7 lines
Quoted 5 lines
scripts/contrib/documentation-audit.sh| 92
++++++++++++++++++++++++++++++++1 files changed, 92 insertions(+), 0 deletions(-)create mode 100755
scripts/contrib/documentation-audit.shCan you please run this and generate a report.
I did this and sent the report to the yocto ML on 10/26.
Scott
Signature · 3 lines
Scott Garman Embedded Linux Engineer - Yocto Project Intel Open Source Technology Center
Saul Wold
AppliedQuoted 14 lines · Scott Garman
Hello,
I've been using this script to perform audits of which packages are producing documentation, and feel it's long overdue for this to be added to the git tree. So here it is.
The following changes since commit
f586aaa8d00361a9597a546d665077c75cf4d520:libxml-parser-perl, libxml-simple-perl, expat, sgmlspl-native, git: bump PR to rebuild after perl upgrade (2011-10-25 08:36:01 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contribsgarman/docaudit-script
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=sgarman/docaudit-script
Quoted 8 lines
Scott Garman (1):
documentation-audit.sh: script for auditing documentation build
statusscripts/contrib/documentation-audit.sh | 92 ++++++++++++++++++++++++++++++++ 1 files changed, 92 insertions(+), 0 deletions(-)create mode 100755
scripts/contrib/documentation-audit.sh
Merged into OE-Core
Thanks
Sau!