knotty: Fold knotty2 into knotty and make it the default
PATCH 1/1 knotty: Fold knotty2 into knotty and make it the default
There is no good reason knotty2 shouldn't be the default now. If you need the old behaviour, just pipe the output through cat as non-interactive terminals get the old output.
---
bitbake/lib/bb/ui/knotty.py +101 -13
@@ -27,6 +27,9 @@ import logging import progressbar import signal import bb.msg +import fcntl +import struct +import copy from bb.ui import uihelper logger = logging.getLogger("BitBake")
@@ -84,39 +87,124 @@ def pluralise(singular, plural, qty): else: return plural % qty + +class InteractConsoleLogFilter(logging.Filter): + def __init__(self, tf, format): + self.tf = tf + self.format = format + + def filter(self, record): + if record.levelno == self.format.NOTE and (record.msg.startswith("Running") or record.msg.startswith("package ")): + return False + self.tf.clearFooter() + return True + class TerminalFilter(object): + columns = 80 + + def sigwinch_handle(self, signum, frame): + self.columns = self.getTerminalColumns() + if self._sigwinch_default: + self._sigwinch_default(signum, frame) + + def getTerminalColumns(self): + def ioctl_GWINSZ(fd): + try: + cr = struct.unpack('hh', fcntl.ioctl(fd, self.termios.TIOCGWINSZ, '1234')) + except: + return None + return cr + cr = ioctl_GWINSZ(sys.stdout.fileno()) + if not cr: + try: + fd = os.open(os.ctermid(), os.O_RDONLY) + cr = ioctl_GWINSZ(fd) + os.close(fd) + except: + pass + if not cr: + try: + cr = (env['LINES'], env['COLUMNS']) + except: + cr = (25, 80) + return cr[1] + def __init__(self, main, helper, console, format): self.main = main self.helper = helper + self.cuu = None + self.stdinbackup = None + self.interactive = sys.stdout.isatty() + self.footer_present = False + self.lastpids = [] + + if not self.interactive: + return + + import curses + import termios + self.curses = curses + self.termios = termios + try: + fd = sys.stdin.fileno() + self.stdinbackup = termios.tcgetattr(fd) + new = copy.deepcopy(self.stdinbackup) + new[3] = new[3] & ~termios.ECHO + termios.tcsetattr(fd, termios.TCSADRAIN, new) + curses.setupterm() + self.ed = curses.tigetstr("ed") + if self.ed: + self.cuu = curses.tigetstr("cuu") + try: + self._sigwinch_default = signal.getsignal(signal.SIGWINCH) + signal.signal(signal.SIGWINCH, self.sigwinch_handle) + except: + pass + self.columns = self.getTerminalColumns() + except: + self.cuu = None + console.addFilter(InteractConsoleLogFilter(self, format)) def clearFooter(self): - return + if self.footer_present: + lines = self.footer_present + sys.stdout.write(self.curses.tparm(self.cuu, lines)) + sys.stdout.write(self.curses.tparm(self.ed)) + self.footer_present = False def updateFooter(self): - if not main.shutdown or not self.helper.needUpdate: + if not self.cuu: return - activetasks = self.helper.running_tasks + failedtasks = self.helper.failed_tasks runningpids = self.helper.running_pids - - if len(runningpids) == 0: + if self.footer_present and (self.lastpids == runningpids): + return + if self.footer_present: + self.clearFooter() + if not activetasks: return - - self.helper.getTasks() - tasks = [] for t in runningpids: tasks.append("%s (pid %s)" % (activetasks[t]["title"], t)) - if main.shutdown: - print("Waiting for %s running tasks to finish:" % len(activetasks)) + if self.main.shutdown: + content = "Waiting for %s running tasks to finish:" % len(activetasks) else: - print("Currently %s running tasks (%s of %s):" % (len(activetasks), self.helper.tasknumber_current, self.helper.tasknumber_total)) + content = "Currently %s running tasks (%s of %s):" % (len(activetasks), self.helper.tasknumber_current, self.helper.tasknumber_total) + print content + lines = 1 + int(len(content) / (self.columns + 1)) for tasknum, task in enumerate(tasks): - print("%s: %s" % (tasknum, task)) + content = "%s: %s" % (tasknum, task) + print content + lines = lines + 1 + int(len(content) / (self.columns + 1)) + self.footer_present = lines + self.lastpids = runningpids[:] def finish(self): - return + if self.stdinbackup: + fd = sys.stdin.fileno() + self.termios.tcsetattr(fd, self.termios.TCSADRAIN, self.stdinbackup) def main(server, eventHandler, tf = TerminalFilter):
dev/null renamed from bitbake/lib/bb/ui/knotty2.py +0 -149
149 lines not on this page · raw message
Quoted 3 lines · Richard Purdie
There is no good reason knotty2 shouldn't be the default now. If you need the old behaviour, just pipe the output through cat as non-interactive terminals get the old output.
I didn't look at this in much detail to see if it is was a direct fold or if you made changes along the way.
I have a whole series of patches outstanding against knotty2 for fixing its behavior for the very annoying line wrapping problems and the like. Certainly I can rebase the work done to this end, but these problems need to be addressed, preferably before you make this the default.
Jason.
Quoted 6 lines · Jason Wessel
Quoted 3 lines · Richard Purdie
There is no good reason knotty2 shouldn't be the default now. If you need the old behaviour, just pipe the output through cat as non-interactive terminals get the old output.
Quoted 7 lines
I didn't look at this in much detail to see if it is was a direct fold or if you made changes along the way.
I have a whole series of patches outstanding against knotty2 for fixing its behavior for the very annoying line wrapping problems and the like. Certainly I can rebase the work done to this end, but these problems need to be addressed, preferably before you make this the default.
Have a look at the preceding patches ;-)
I've fixed the line wrapping and window resizing issues first. It was based on some of your changes but I fixed various things (like having progressbar chain the WINCH signal handlers instead of hacking around it).
This doesn't add the interactive logging as I'm still not sure that makes sense in master. If we do that I want to think more carefully about the UI to it.
The interactive terminal stuff is also pending as I was pretty horrified when I realised how you implemented it. What we need to do is issue an event and have bitbake respond to that event where the event contains a "command" parameter.
So I'm working through the patches but I think with the changes I just posted, knotty2 should be ready for wider usage.
Cheers,
Richard
Quoted 5 lines · Richard Purdie
There is no good reason knotty2 shouldn't be the default now. If you need the old behaviour, just pipe the output through cat as non-interactive terminals get the old output.
Signed-off-by Richard Purdie <[email protected]>
I'm 100% in favor of this series, for what it's worth. I've been using knotty2 as my default BITBAKE_UI for months, just ignoring the minor bugs. The reduction in unnecessary output makes it *much* easier to spot *real* problems.
That said, I think we should look into prefixing some of our log messages — some of them don't make it clear what task/recipe emitted them, making it difficult to isolate their cause. But of course, that was an issue with knotty as well, but it was slightly easier to narrow down the cause based on where it stood between the task started/completed messages :). Would you be open to potentially injecting a log message filter which adds task context informastion, in the code which forks the tasks? Then the formatter would have to do a hasattr() to check for that and add it to the format, but it'd mean we *always* know what task log messages are coming from.
Signature · 1 line
Christopher Larson
Quoted 6 lines · Chris Larson
Quoted 4 lines · Richard Purdie
There is no good reason knotty2 shouldn't be the default now. If you need the old behaviour, just pipe the output through cat as non-interactive terminals get the old output.
Quoted 6 lines
Quoted 1 line
Signed-off-by Richard Purdie <[email protected]>I'm 100% in favor of this series, for what it's worth. I've been using knotty2 as my default
BITBAKE_UIfor months, just ignoring the minor bugs. The reduction in unnecessary output makes it *much* easier to spot *real* problems.
Agreed, that is one of the intents.
Quoted 10 lines
That said, I think we should look into prefixing some of our log messages — some of them don't make it clear what task/recipe emitted them, making it difficult to isolate their cause. But of course, that was an issue with knotty as well, but it was slightly easier to narrow down the cause based on where it stood between the task started/completed messages :). Would you be open to potentially injecting a log message filter which adds task context informastion, in the code which forks the tasks? Then the formatter would have to do a
hasattr()to check for that and add it to the format, but it'd mean we *always* know what task log messages are coming from.
Obviously such a change needs to be carefully considered but yes, I think knowing where a message has come from is important and I'm in favour of doing something. I'm slightly surprised there isn't something in the message already.
Thinking about this more, I think we did add in the pid or at least looked at doing so, with the assumption that the UI code could have information about what that pid represents and be able to supplement the data. I think the graphical UIs might use that to put the messages into different "buckets" in the graphical task display.
The immediate reaction is "lets just add all the data to the event" which I can certainly understand. My big worry is the pushing up the size of each event to something that damages performance, particularly if we so start using remote UIs. This is why the uihelper code exists, effectively to maintain a UI side cache of the data rather than pickle/unpickling the same thing many times.
As a side note, I do want to get handlers only receiving the events they care about with server side filtering too. That is starting to show up on profiles of some workloads.
Also, we currently use the word "package" in the old knotty output, I want to change that to "recipe" as its confusing and we should be consistent about our terminology. I guess I should propose a patch :)
Cheers,
Richard