Skip to contentSkip to conversation

sequencer: refactor functions working on a todo_list

PATCH Superseded Alban Gruin 6 participants 190 messages Oct 7, 2018 – Mar 13, 2019 Page 1 of 2

lore
Share & tools
lore
#0 lore raw

cover sequencer: refactor functions working on a todo_list

At the center of the "interactive" part of the interactive rebase lies the todo list. When the user starts an interactive rebase, a todo list is generated, presented to the user (who then edits it using a text editor), read back, and then is checked and processed before the actual rebase takes place.

Some of this processing includes adding execs commands, reordering fixup! and squash! commits, and checking if no commits were accidentally dropped by the user.

Before I converted the interactive rebase in C, these functions were called by git-rebase--interactive.sh through git-rebase--helper. Since the only way to pass around a large amount of data between a shell script and a C program is to use a file (or any declination of a file), the functions that checked and processed the todo list were directly working on a file, the same file that the user edited.

During the conversion, I did not address this issue, which lead to a complete_action() that reads the todo list file, does some computation based on its content, and writes it back to the disk, several times in the same function.

As it is not an efficient way to handle a data structure, this patch series refactor the functions that processes the todo list to work on a todo_list structure instead of reading it from the disk.

Some commits consists in modifying edit_todo_list() (initially used by --edit-todo) to handle the initial edition of the todo list, to increase code sharing.

Alban Gruin (15):
sequencer: clear the number of items of a todo_list before parsing
sequencer: make the todo_list structure public
sequencer: refactor check_todo_list() to work on a todo_list
sequencer: refactor sequencer_add_exec_commands() to work on a
todo_list
sequencer: refactor rearrange_squash() to work on a todo_list
sequencer: refactor transform_todos() to work on a todo_list
sequencer: make sequencer_make_script() write its script to a strbuf
sequencer: change complete_action() to use the refactored functions
sequencer: refactor skip_unnecessary_picks() to work on a todo_list
rebase-interactive: use todo_list_transform() in edit_todo_list()
rebase-interactive: append_todo_help() changes
rebase-interactive: rewrite edit_todo_list() to handle the initial
edit
sequencer: use edit_todo_list() in complete_action()
sequencer: fix a call to error() in transform_todo_file()
rebase--interactive: move transform_todo_file() to
rebase--interactive.c

 builtin/rebase--interactive.c |  65 +++--
 rebase-interactive.c          | 161 ++++++++++--
 rebase-interactive.h          |   8 +-
 sequencer.c                   | 479 ++++++++++++----------------------
 sequencer.h                   |  66 ++++-
 5 files changed, 406 insertions(+), 373 deletions(-)
Signature · 1 line
2.19.1
#1 lore raw

PATCH 1/15 sequencer: clear the number of items of a todo_list before parsing

This clears the number of items of a todo_list before parsing it to allow to parse the same list multiple times without issues.

As its items are not dynamically allocated, or don’t need to allocate memory, no additionnal memory management is required here.

---
 sequencer.c | 2 ++
 1 file changed, 2 insertions(+)
sequencer.c +2 -0
index 0a3292d5c4..ed798b95d1 100644
@@ -2063,6 +2063,8 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 	char *p = buf, *next_p;
 	int i, res = 0, fixup_okay = file_exists(rebase_path_done());
 
+	todo_list->current = todo_list->nr = 0;
+
 	for (i = 1; *p; i++, p = next_p) {
 		char *eol = strchrnul(p, '\n');
 
Signature · 1 line
2.19.1
#2 lore raw

PATCH 2/15 sequencer: make the todo_list structure public

In reply to Alban Gruin

This makes the structures todo_list and todo_item, and the functions todo_list_release() and parse_insn_buffer(), accessible outside of sequencer.c.

---
 sequencer.c | 66 +++++++++--------------------------------------------
 sequencer.h | 48 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 59 insertions(+), 55 deletions(-)
sequencer.c +11 -55
index ed798b95d1..bb8ca2477f 100644
@@ -1443,31 +1443,6 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 		return 1;
 }
 
-/*
- * Note that ordering matters in this enum. Not only must it match the mapping
- * below, it is also divided into several sections that matter.  When adding
- * new commands, make sure you add it in the right section.
- */
-enum todo_command {
-	/* commands that handle commits */
-	TODO_PICK = 0,
-	TODO_REVERT,
-	TODO_EDIT,
-	TODO_REWORD,
-	TODO_FIXUP,
-	TODO_SQUASH,
-	/* commands that do something else than handling a single commit */
-	TODO_EXEC,
-	TODO_LABEL,
-	TODO_RESET,
-	TODO_MERGE,
-	/* commands that do nothing but are counted for reporting progress */
-	TODO_NOOP,
-	TODO_DROP,
-	/* comments (not counted for reporting progress) */
-	TODO_COMMENT
-};
-
 static struct {
 	char c;
 	const char *str;
@@ -1937,26 +1912,7 @@ enum todo_item_flags {
 	TODO_EDIT_MERGE_MSG = 1
 };
 
-struct todo_item {
-	enum todo_command command;
-	struct commit *commit;
-	unsigned int flags;
-	const char *arg;
-	int arg_len;
-	size_t offset_in_buf;
-};
-
-struct todo_list {
-	struct strbuf buf;
-	struct todo_item *items;
-	int nr, alloc, current;
-	int done_nr, total_nr;
-	struct stat_data stat;
-};
-
-#define TODO_LIST_INIT { STRBUF_INIT }
-
-static void todo_list_release(struct todo_list *todo_list)
+void todo_list_release(struct todo_list *todo_list)
 {
 	strbuf_release(&todo_list->buf);
 	FREE_AND_NULL(todo_list->items);
@@ -2057,7 +2013,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	return !item->commit;
 }
 
-static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
+int todo_list_parse_insn_buffer(char *buf, struct todo_list *todo_list)
 {
 	struct todo_item *item;
 	char *p = buf, *next_p;
@@ -2152,7 +2108,7 @@ static int read_populate_todo(struct todo_list *todo_list,
 		return error(_("could not stat '%s'"), todo_file);
 	fill_stat_data(&todo_list->stat, &st);
 
-	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
+	res = todo_list_parse_insn_buffer(todo_list->buf.buf, todo_list);
 	if (res) {
 		if (is_rebase_i(opts))
 			return error(_("please fix this using "
@@ -2183,7 +2139,7 @@ static int read_populate_todo(struct todo_list *todo_list,
 		FILE *f = fopen_or_warn(rebase_path_msgtotal(), "w");
 
 		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
-				!parse_insn_buffer(done.buf.buf, &done))
+				!todo_list_parse_insn_buffer(done.buf.buf, &done))
 			todo_list->done_nr = count_commands(&done);
 		else
 			todo_list->done_nr = 0;
@@ -4429,7 +4385,7 @@ int sequencer_add_exec_commands(const char *commands)
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4485,7 +4441,7 @@ int transform_todos(unsigned flags)
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4571,7 +4527,7 @@ int check_todo_list(void)
 		goto leave_check;
 	}
 	advise_to_edit_todo = res =
-		parse_insn_buffer(todo_list.buf.buf, &todo_list);
+		todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list);
 
 	if (res || check_level == MISSING_COMMIT_CHECK_IGNORE)
 		goto leave_check;
@@ -4590,7 +4546,7 @@ int check_todo_list(void)
 		goto leave_check;
 	}
 	strbuf_release(&todo_file);
-	res = !!parse_insn_buffer(todo_list.buf.buf, &todo_list);
+	res = !!todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list);
 
 	/* Find commits in git-rebase-todo.backup yet unseen */
 	for (i = todo_list.nr - 1; i >= 0; i--) {
@@ -4672,7 +4628,7 @@ static int skip_unnecessary_picks(struct object_id *output_oid)
 
 	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
 		return -1;
-	if (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {
 		todo_list_release(&todo_list);
 		return -1;
 	}
@@ -4760,7 +4716,7 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 	if (strbuf_read_file(buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(buf->buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(buf->buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4868,7 +4824,7 @@ int rearrange_squash(void)
 
 	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
 		return -1;
-	if (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {
 		todo_list_release(&todo_list);
 		return -1;
 	}
sequencer.h +48 -0

54 lines not on this page · raw message

Signature · 1 line
2.19.1
#3 lore raw

PATCH 3/15 sequencer: refactor check_todo_list() to work on a todo_list

In reply to Alban Gruin

This refactors check_todo_list() to work on a todo_list to avoid redundant reads and writes to the disk. The function is renamed todo_list_check().

As rebase -p still need to check the todo list from the disk, a new function is introduced, check_todo_list_from_file(). It reads the file from the disk, parses it, pass the todo_list to todo_list_check(), and writes it back to the disk.

As get_missing_commit_check_level() and the enum missing_commit_check_level are no longer needed inside of sequencer.c, they are moved to rebase-interactive.c, and made static again.

---
 builtin/rebase--interactive.c |   2 +-
 rebase-interactive.c          | 106 ++++++++++++++++++++++++++++++++-
 rebase-interactive.h          |   1 +
 sequencer.c                   | 109 ++++------------------------------
 sequencer.h                   |   9 +--
 5 files changed, 120 insertions(+), 107 deletions(-)
builtin/rebase--interactive.c +1 -1
index a2ab68ed06..ea1f93ccb6 100644
@@ -255,7 +255,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = transform_todos(flags);
 		break;
 	case CHECK_TODO_LIST:
-		ret = check_todo_list();
+		ret = check_todo_list_from_file();
 		break;
 	case REARRANGE_SQUASH:
 		ret = rearrange_squash();
rebase-interactive.c +105 -1

116 lines not on this page · raw message

rebase-interactive.h +1 -0
index 971da03776..6bc7bc315d 100644
@@ -4,5 +4,6 @@
 void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 		      struct strbuf *buf);
 int edit_todo_list(unsigned flags);
+int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
 
 #endif
sequencer.c +12 -97

130 lines not on this page · raw message

sequencer.h +1 -8
index c786dee543..fb8b85bf9e 100644
@@ -63,12 +63,6 @@ struct replay_opts {
 };
 #define REPLAY_OPTS_INIT { .action = -1, .current_fixups = STRBUF_INIT }
 
-enum missing_commit_check_level {
-	MISSING_COMMIT_CHECK_IGNORE = 0,
-	MISSING_COMMIT_CHECK_WARN,
-	MISSING_COMMIT_CHECK_ERROR
-};
-
 int write_message(const void *buf, size_t len, const char *filename,
 		  int append_eol);
 
@@ -142,8 +136,7 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 
 int sequencer_add_exec_commands(const char *command);
 int transform_todos(unsigned flags);
-enum missing_commit_check_level get_missing_commit_check_level(void);
-int check_todo_list(void);
+int check_todo_list_from_file(void);
 int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, const char *cmd,
Signature · 1 line
2.19.1
#4 lore raw

Re: [PATCH 03/15] sequencer: refactor check_todo_list() to work on a todo_list

Hi Alban

I like the direction that this series is going

Quoted 8 lines · Alban Gruin

This refactors check_todo_list() to work on a todo_list to avoid redundant reads and writes to the disk. The function is renamed todo_list_check().

As rebase -p still need to check the todo list from the disk, a new function is introduced, check_todo_list_from_file(). It reads the file from the disk, parses it, pass the todo_list to todo_list_check(), and writes it back to the disk.

After this commit we still use check_todo_list_from_file() even without '-p', it would be clearer if this (and the following commits) mentioned that the new function will be wired up later.

Quoted 76 lines

As get_missing_commit_check_level() and the enum missing_commit_check_level are no longer needed inside of sequencer.c, they are moved to rebase-interactive.c, and made static again.

---
  builtin/rebase--interactive.c |   2 +-
  rebase-interactive.c          | 106 ++++++++++++++++++++++++++++++++-
  rebase-interactive.h          |   1 +
  sequencer.c                   | 109 ++++------------------------------
  sequencer.h                   |   9 +--
  5 files changed, 120 insertions(+), 107 deletions(-)
builtin/rebase--interactive.c +1 -1
index a2ab68ed06..ea1f93ccb6 100644
@@ -255,7 +255,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
  		ret = transform_todos(flags);
  		break;
  	case CHECK_TODO_LIST:
-		ret = check_todo_list();
+		ret = check_todo_list_from_file();
  		break;
  	case REARRANGE_SQUASH:
  		ret = rearrange_squash();
rebase-interactive.c +33 -1
index 0f4119cbae..ef8540245d 100644
@@ -1,8 +1,32 @@
  #include "cache.h"
  #include "commit.h"
-#include "rebase-interactive.h"
  #include "sequencer.h"
+#include "rebase-interactive.h"
  #include "strbuf.h"
+#include "commit-slab.h"
+#include "config.h"
+
+enum missing_commit_check_level {
+	MISSING_COMMIT_CHECK_IGNORE = 0,
+	MISSING_COMMIT_CHECK_WARN,
+	MISSING_COMMIT_CHECK_ERROR
+};
+
+static enum missing_commit_check_level get_missing_commit_check_level(void)
+{
+	const char *value;
+
+	if (git_config_get_value("rebase.missingcommitscheck", &value) ||
+			!strcasecmp("ignore", value))
+		return MISSING_COMMIT_CHECK_IGNORE;
+	if (!strcasecmp("warn", value))
+		return MISSING_COMMIT_CHECK_WARN;
+	if (!strcasecmp("error", value))
+		return MISSING_COMMIT_CHECK_ERROR;
+	warning(_("unrecognized setting %s for option "
+		  "rebase.missingCommitsCheck. Ignoring."), value);
+	return MISSING_COMMIT_CHECK_IGNORE;
+}
 
  void append_todo_help(unsigned edit_todo, unsigned keep_empty,
  		      struct strbuf *buf)
@@ -88,3 +112,83 @@ int edit_todo_list(unsigned flags)
 
  	return 0;
  }
+
+define_commit_slab(commit_seen, unsigned char);
+/*
+ * Check if the user dropped some commits by mistake
+ * Behaviour determined by rebase.missingCommitsCheck.
+ * Check if there is an unrecognized command or a
+ * bad SHA-1 in a command.
+ */

rebase-interactive.c int edit_todo_list(unsigned flags) · line 132

Quoted 10 lines
rebase-interactive.c +10 -0
+int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo)
+{
+	enum missing_commit_check_level check_level = get_missing_commit_check_level();
+	struct strbuf missing = STRBUF_INIT;
+	int advise_to_edit_todo = 0, res = 0, i;
+	struct commit_seen commit_seen;
+
+	init_commit_seen(&commit_seen);
+
+	res = todo_list_parse_insn_buffer(old_todo->buf.buf, old_todo);

While it made sense to parse the old and new todo lists here when they were being loaded from a file, I think it is confusing to keep that behavior now it is being passed struct todo_lists. When we get to patch 8 the newly read commit list is passed to this function without being parsed which confused me until I came back and check what was going on here.

Best Wishes

Phillip

Quoted 237 lines
rebase-interactive.c +62 -0
+	if (!res)
+		res = todo_list_parse_insn_buffer(new_todo->buf.buf, new_todo);
+	if (res) {
+		advise_to_edit_todo = res;
+		goto leave_check;
+	}
+
+	if (check_level == MISSING_COMMIT_CHECK_IGNORE)
+		goto leave_check;
+
+	/* Mark the commits in git-rebase-todo as seen */
+	for (i = 0; i < new_todo->nr; i++) {
+		struct commit *commit = new_todo->items[i].commit;
+		if (commit)
+			*commit_seen_at(&commit_seen, commit) = 1;
+	}
+
+	/* Find commits in git-rebase-todo.backup yet unseen */
+	for (i = old_todo->nr - 1; i >= 0; i--) {
+		struct todo_item *item = old_todo->items + i;
+		struct commit *commit = item->commit;
+		if (commit && !*commit_seen_at(&commit_seen, commit)) {
+			strbuf_addf(&missing, " - %s %.*s\n",
+				    find_unique_abbrev(&commit->object.oid, DEFAULT_ABBREV),
+				    item->arg_len, item->arg);
+			*commit_seen_at(&commit_seen, commit) = 1;
+		}
+	}
+
+	/* Warn about missing commits */
+	if (!missing.len)
+		goto leave_check;
+
+	if (check_level == MISSING_COMMIT_CHECK_ERROR)
+		advise_to_edit_todo = res = 1;
+
+	fprintf(stderr,
+		_("Warning: some commits may have been dropped accidentally.\n"
+		"Dropped commits (newer to older):\n"));
+
+	/* Make the list user-friendly and display */
+	fputs(missing.buf, stderr);
+	strbuf_release(&missing);
+
+	fprintf(stderr, _("To avoid this message, use \"drop\" to "
+		"explicitly remove a commit.\n\n"
+		"Use 'git config rebase.missingCommitsCheck' to change "
+		"the level of warnings.\n"
+		"The possible behaviours are: ignore, warn, error.\n\n"));
+
+leave_check:
+	clear_commit_seen(&commit_seen);
+
+	if (advise_to_edit_todo)
+		fprintf(stderr,
+			_("You can fix this with 'git rebase --edit-todo' "
+			  "and then run 'git rebase --continue'.\n"
+			  "Or you can abort the rebase with 'git rebase"
+			  " --abort'.\n"));
+
+	return res;
+}
rebase-interactive.h +1 -0
index 971da03776..6bc7bc315d 100644
@@ -4,5 +4,6 @@
  void append_todo_help(unsigned edit_todo, unsigned keep_empty,
  		      struct strbuf *buf);
  int edit_todo_list(unsigned flags);
+int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
 
  #endif
sequencer.c +12 -97
index bb8ca2477f..8dda61927c 100644
@@ -4487,111 +4487,26 @@ int transform_todos(unsigned flags)
  	return i;
  }
 
-enum missing_commit_check_level get_missing_commit_check_level(void)
+int check_todo_list_from_file(void)
  {
-	const char *value;
-
-	if (git_config_get_value("rebase.missingcommitscheck", &value) ||
-			!strcasecmp("ignore", value))
-		return MISSING_COMMIT_CHECK_IGNORE;
-	if (!strcasecmp("warn", value))
-		return MISSING_COMMIT_CHECK_WARN;
-	if (!strcasecmp("error", value))
-		return MISSING_COMMIT_CHECK_ERROR;
-	warning(_("unrecognized setting %s for option "
-		  "rebase.missingCommitsCheck. Ignoring."), value);
-	return MISSING_COMMIT_CHECK_IGNORE;
-}
-
-define_commit_slab(commit_seen, unsigned char);
-/*
- * Check if the user dropped some commits by mistake
- * Behaviour determined by rebase.missingCommitsCheck.
- * Check if there is an unrecognized command or a
- * bad SHA-1 in a command.
- */
-int check_todo_list(void)
-{
-	enum missing_commit_check_level check_level = get_missing_commit_check_level();
-	struct strbuf todo_file = STRBUF_INIT;
-	struct todo_list todo_list = TODO_LIST_INIT;
-	struct strbuf missing = STRBUF_INIT;
-	int advise_to_edit_todo = 0, res = 0, i;
-	struct commit_seen commit_seen;
-
-	init_commit_seen(&commit_seen);
+	struct todo_list old_todo = TODO_LIST_INIT, new_todo = TODO_LIST_INIT;
+	int res = 0;
 
-	strbuf_addstr(&todo_file, rebase_path_todo());
-	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file.buf) < 0) {
+	if (strbuf_read_file_or_whine(&new_todo.buf, rebase_path_todo()) < 0) {
  		res = -1;
-		goto leave_check;
-	}
-	advise_to_edit_todo = res =
-		todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list);
-
-	if (res || check_level == MISSING_COMMIT_CHECK_IGNORE)
-		goto leave_check;
-
-	/* Mark the commits in git-rebase-todo as seen */
-	for (i = 0; i < todo_list.nr; i++) {
-		struct commit *commit = todo_list.items[i].commit;
-		if (commit)
-			*commit_seen_at(&commit_seen, commit) = 1;
+		goto out;
  	}
 
-	todo_list_release(&todo_list);
-	strbuf_addstr(&todo_file, ".backup");
-	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file.buf) < 0) {
+	if (strbuf_read_file_or_whine(&old_todo.buf, rebase_path_todo_backup()) < 0) {
  		res = -1;
-		goto leave_check;
-	}
-	strbuf_release(&todo_file);
-	res = !!todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list);
-
-	/* Find commits in git-rebase-todo.backup yet unseen */
-	for (i = todo_list.nr - 1; i >= 0; i--) {
-		struct todo_item *item = todo_list.items + i;
-		struct commit *commit = item->commit;
-		if (commit && !*commit_seen_at(&commit_seen, commit)) {
-			strbuf_addf(&missing, " - %s %.*s\n",
-				    short_commit_name(commit),
-				    item->arg_len, item->arg);
-			*commit_seen_at(&commit_seen, commit) = 1;
-		}
+		goto out;
  	}
 
-	/* Warn about missing commits */
-	if (!missing.len)
-		goto leave_check;
-
-	if (check_level == MISSING_COMMIT_CHECK_ERROR)
-		advise_to_edit_todo = res = 1;
+	res = todo_list_check(&old_todo, &new_todo);
 
-	fprintf(stderr,
-		_("Warning: some commits may have been dropped accidentally.\n"
-		"Dropped commits (newer to older):\n"));
-
-	/* Make the list user-friendly and display */
-	fputs(missing.buf, stderr);
-	strbuf_release(&missing);
-
-	fprintf(stderr, _("To avoid this message, use \"drop\" to "
-		"explicitly remove a commit.\n\n"
-		"Use 'git config rebase.missingCommitsCheck' to change "
-		"the level of warnings.\n"
-		"The possible behaviours are: ignore, warn, error.\n\n"));
-
-leave_check:
-	clear_commit_seen(&commit_seen);
-	strbuf_release(&todo_file);
-	todo_list_release(&todo_list);
-
-	if (advise_to_edit_todo)
-		fprintf(stderr,
-			_("You can fix this with 'git rebase --edit-todo' "
-			  "and then run 'git rebase --continue'.\n"
-			  "Or you can abort the rebase with 'git rebase"
-			  " --abort'.\n"));
+out:
+	todo_list_release(&old_todo);
+	todo_list_release(&new_todo);
 
  	return res;
  }
@@ -4769,7 +4684,7 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 
  	todo_list_release(&todo_list);
 
-	if (check_todo_list()) {
+	if (check_todo_list_from_file()) {
  		checkout_onto(opts, onto_name, onto, orig_head);
  		return -1;
  	}
sequencer.h +1 -8
index c786dee543..fb8b85bf9e 100644
@@ -63,12 +63,6 @@ struct replay_opts {
  };
  #define REPLAY_OPTS_INIT { .action = -1, .current_fixups = STRBUF_INIT }
 
-enum missing_commit_check_level {
-	MISSING_COMMIT_CHECK_IGNORE = 0,
-	MISSING_COMMIT_CHECK_WARN,
-	MISSING_COMMIT_CHECK_ERROR
-};
-
  int write_message(const void *buf, size_t len, const char *filename,
  		  int append_eol);
 
@@ -142,8 +136,7 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 
  int sequencer_add_exec_commands(const char *command);
  int transform_todos(unsigned flags);
-enum missing_commit_check_level get_missing_commit_check_level(void);
-int check_todo_list(void);
+int check_todo_list_from_file(void);
  int complete_action(struct replay_opts *opts, unsigned flags,
  		    const char *shortrevisions, const char *onto_name,
  		    const char *onto, const char *orig_head, const char *cmd,
#5 lore raw

PATCH 4/15 sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

In reply to Alban Gruin

This refactors sequencer_add_exec_commands() to work on a todo_list to avoid redundant reads and writes to the disk.

sequencer_add_exec_commands() still reads the todo list from the disk, as it is needed by rebase -p. todo_list_add_exec_commands() works on a todo_list structure, and reparses it at the end.

---
 sequencer.c | 56 +++++++++++++++++++++++++++++++----------------------
 1 file changed, 33 insertions(+), 23 deletions(-)
sequencer.c +33 -23
index 8dda61927c..6d998f21a4 100644
@@ -4370,34 +4370,21 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	return 0;
 }
 
-/*
- * Add commands after pick and (series of) squash/fixup commands
- * in the todo list.
- */
-int sequencer_add_exec_commands(const char *commands)
+static void todo_list_add_exec_commands(struct todo_list *todo_list,
+					const char *commands)
 {
-	const char *todo_file = rebase_path_todo();
-	struct todo_list todo_list = TODO_LIST_INIT;
-	struct strbuf *buf = &todo_list.buf;
+	struct strbuf *buf = &todo_list->buf;
 	size_t offset = 0, commands_len = strlen(commands);
 	int i, insert;
 
-	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
-		return error(_("could not read '%s'."), todo_file);
-
-	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
-		todo_list_release(&todo_list);
-		return error(_("unusable todo list: '%s'"), todo_file);
-	}
-
 	/*
 	 * Insert <commands> after every pick. Here, fixup/squash chains
 	 * are considered part of the pick, so we insert the commands *after*
 	 * those chains if there are any.
 	 */
 	insert = -1;
-	for (i = 0; i < todo_list.nr; i++) {
-		enum todo_command command = todo_list.items[i].command;
+	for (i = 0; i < todo_list->nr; i++) {
+		enum todo_command command = todo_list->items[i].command;
 
 		if (insert >= 0) {
 			/* skip fixup/squash chains */
@@ -4408,7 +4395,7 @@ int sequencer_add_exec_commands(const char *commands)
 				continue;
 			}
 			strbuf_insert(buf,
-				      todo_list.items[insert].offset_in_buf +
+				      todo_list->items[insert].offset_in_buf +
 				      offset, commands, commands_len);
 			offset += commands_len;
 			insert = -1;
@@ -4419,15 +4406,38 @@ int sequencer_add_exec_commands(const char *commands)
 	}
 
 	/* insert or append final <commands> */
-	if (insert >= 0 && insert < todo_list.nr)
-		strbuf_insert(buf, todo_list.items[insert].offset_in_buf +
+	if (insert >= 0 && insert < todo_list->nr)
+		strbuf_insert(buf, todo_list->items[insert].offset_in_buf +
 			      offset, commands, commands_len);
 	else if (insert >= 0 || !offset)
 		strbuf_add(buf, commands, commands_len);
 
-	i = write_message(buf->buf, buf->len, todo_file, 0);
+	if (todo_list_parse_insn_buffer(buf->buf, todo_list))
+		BUG("unusable todo list");}
+
+/*
+ * Add commands after pick and (series of) squash/fixup commands
+ * in the todo list.
+ */
+int sequencer_add_exec_commands(const char *commands)
+{
+	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res;
+
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
+		return error(_("could not read '%s'."), todo_file);
+
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
+		todo_list_release(&todo_list);
+		return error(_("unusable todo list: '%s'"), todo_file);
+	}
+
+	todo_list_add_exec_commands(&todo_list, commands);
+	res = write_message(todo_list.buf.buf, todo_list.buf.len, todo_file, 0);
 	todo_list_release(&todo_list);
-	return i;
+
+	return res;
 }
 
 int transform_todos(unsigned flags)
Signature · 1 line
2.19.1
#6 lore raw

Re: [PATCH 04/15] sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

Quoted 70 lines · Alban Gruin

This refactors sequencer_add_exec_commands() to work on a todo_list to avoid redundant reads and writes to the disk.

sequencer_add_exec_commands() still reads the todo list from the disk, as it is needed by rebase -p. todo_list_add_exec_commands() works on a todo_list structure, and reparses it at the end.

---
  sequencer.c | 56 +++++++++++++++++++++++++++++++----------------------
  1 file changed, 33 insertions(+), 23 deletions(-)
sequencer.c +6 -20
index 8dda61927c..6d998f21a4 100644
@@ -4370,34 +4370,21 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
  	return 0;
  }
 
-/*
- * Add commands after pick and (series of) squash/fixup commands
- * in the todo list.
- */
-int sequencer_add_exec_commands(const char *commands)
+static void todo_list_add_exec_commands(struct todo_list *todo_list,
+					const char *commands)
  {
-	const char *todo_file = rebase_path_todo();
-	struct todo_list todo_list = TODO_LIST_INIT;
-	struct strbuf *buf = &todo_list.buf;
+	struct strbuf *buf = &todo_list->buf;
  	size_t offset = 0, commands_len = strlen(commands);
  	int i, insert;
 
-	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
-		return error(_("could not read '%s'."), todo_file);
-
-	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
-		todo_list_release(&todo_list);
-		return error(_("unusable todo list: '%s'"), todo_file);
-	}
-
  	/*
  	 * Insert <commands> after every pick. Here, fixup/squash chains
  	 * are considered part of the pick, so we insert the commands *after*
  	 * those chains if there are any.
  	 */
  	insert = -1;
-	for (i = 0; i < todo_list.nr; i++) {
-		enum todo_command command = todo_list.items[i].command;
+	for (i = 0; i < todo_list->nr; i++) {
+		enum todo_command command = todo_list->items[i].command;
 
  		if (insert >= 0) {
  			/* skip fixup/squash chains */
@@ -4408,7 +4395,7 @@ int sequencer_add_exec_commands(const char *commands)
  				continue;
  			}
  			strbuf_insert(buf,
-				      todo_list.items[insert].offset_in_buf +
+				      todo_list->items[insert].offset_in_buf +
  				      offset, commands, commands_len);
  			offset += commands_len;
  			insert = -1;
@@ -4419,15 +4406,38 @@ int sequencer_add_exec_commands(const char *commands)
  	}
 
  	/* insert or append final <commands> */
-	if (insert >= 0 && insert < todo_list.nr)

sequencer.c int sequencer_add_exec_commands(const char *commands) · line 4416

Quoted 10 lines
sequencer.c +4 -2
-		strbuf_insert(buf, todo_list.items[insert].offset_in_buf +
+	if (insert >= 0 && insert < todo_list->nr)
+		strbuf_insert(buf, todo_list->items[insert].offset_in_buf +
  			      offset, commands, commands_len);
  	else if (insert >= 0 || !offset)
  		strbuf_add(buf, commands, commands_len);
 
-	i = write_message(buf->buf, buf->len, todo_file, 0);
+	if (todo_list_parse_insn_buffer(buf->buf, todo_list))
+		BUG("unusable todo list");}

It is a shame to have to re-parse the todo list, I wonder how difficult it would be to adjust the todo_list item array as the exec commands are inserted. The same applies to the next couple of patches

Best Wishes

Phillip

Quoted 29 lines
sequencer.c +23 -1
+
+/*
+ * Add commands after pick and (series of) squash/fixup commands
+ * in the todo list.
+ */
+int sequencer_add_exec_commands(const char *commands)
+{
+	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res;
+
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
+		return error(_("could not read '%s'."), todo_file);
+
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
+		todo_list_release(&todo_list);
+		return error(_("unusable todo list: '%s'"), todo_file);
+	}
+
+	todo_list_add_exec_commands(&todo_list, commands);
+	res = write_message(todo_list.buf.buf, todo_list.buf.len, todo_file, 0);
  	todo_list_release(&todo_list);
-	return i;
+
+	return res;
  }
 
  int transform_todos(unsigned flags)
#7 lore raw

Re: [PATCH 04/15] sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

Hi Phillip,

thanks for taking the time to review my patches.

Le 11/10/2018 à 13:25, Phillip Wood a écrit :

Quoted 11 lines · Alban Gruin
Quoted 10 lines
sequencer.c +2 -2
@@ -4419,15 +4406,38 @@ int sequencer_add_exec_commands(const char
 *commands)
       }
         /* insert or append final <commands> */
-    if (insert >= 0 && insert < todo_list.nr)
-        strbuf_insert(buf, todo_list.items[insert].offset_in_buf +
+    if (insert >= 0 && insert < todo_list->nr)
+        strbuf_insert(buf, todo_list->items[insert].offset_in_buf +
                     offset, commands, commands_len);
       else if (insert >= 0 || !offset)
Quoted 9 lines

sequencer.c int sequencer_add_exec_commands(const char *commands) · line 4416

Quoted 4 lines
sequencer.c +2 -0
           strbuf_add(buf, commands, commands_len);
   -    i = write_message(buf->buf, buf->len, todo_file, 0);
+    if (todo_list_parse_insn_buffer(buf->buf, todo_list))
+        BUG("unusable todo list");}

It is a shame to have to re-parse the todo list, I wonder how difficult it would be to adjust the todo_list item array as the exec commands are inserted. The same applies to the next couple of patches

Good question.

This function inserts an exec' command after every pick' command. These commands are stored in a dynamically allocated list, grew with ALLOW_GROW().

If we want to keep the current structure, we would have to grow the size of the list by 1 and move several element to the end every time we want to add an `exec' command. It would not be very effective. Perhaps I should use a linked list here, instead. It may also work well with rearrange_squash() and skip_unnecessary_picks().

Maybe we could even get rid of the strbuf at some point.

Quoted 4 lines

Best Wishes

Phillip

Cheers,
Alban

#8 lore raw

Re: [PATCH 04/15] sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

Quoted 32 lines · Alban Gruin

Hi Phillip,

thanks for taking the time to review my patches.

Le 11/10/2018 à 13:25, Phillip Wood a écrit :

Quoted 11 lines · Alban Gruin
Quoted 10 lines
sequencer.c +2 -2
@@ -4419,15 +4406,38 @@ int sequencer_add_exec_commands(const char
 *commands)
       }
         /* insert or append final <commands> */
-    if (insert >= 0 && insert < todo_list.nr)
-        strbuf_insert(buf, todo_list.items[insert].offset_in_buf +
+    if (insert >= 0 && insert < todo_list->nr)
+        strbuf_insert(buf, todo_list->items[insert].offset_in_buf +
                     offset, commands, commands_len);
       else if (insert >= 0 || !offset)
Quoted 9 lines

sequencer.c int sequencer_add_exec_commands(const char *commands) · line 4416

Quoted 4 lines
sequencer.c +2 -0
           strbuf_add(buf, commands, commands_len);
   -    i = write_message(buf->buf, buf->len, todo_file, 0);
+    if (todo_list_parse_insn_buffer(buf->buf, todo_list))
+        BUG("unusable todo list");}

It is a shame to have to re-parse the todo list, I wonder how difficult it would be to adjust the todo_list item array as the exec commands are inserted. The same applies to the next couple of patches

Good question.

This function inserts an exec' command after every pick' command. These commands are stored in a dynamically allocated list, grew with ALLOW_GROW().

Quoted 7 lines

If we want to keep the current structure, we would have to grow the size of the list by 1 and move several element to the end every time we want to add an `exec' command. It would not be very effective. Perhaps I should use a linked list here, instead. It may also work well with rearrange_squash() and skip_unnecessary_picks().

Maybe we could even get rid of the strbuf at some point.

Another way would be to use the strbuf as a string pool rather than a copy of the text of the file. There could be a write_todo_list() function that takes a todo list and some flags, iterates over the items in the todo list and writes the file. The flags would specify whether to append the todo help and whether to abbreviate the object ids (so there is no need for a separate call to transform_todos()). Then add_exec_commands() could allocate a new array of todo items which it builds up as it works through the original list and replaces the original list with the new one at the end. The text of the exec items can be added to the end of the strbuf (we only need one copy of the exec text with this scheme). rearrange_squash() can use a temporary array to build a new list as well or just memmove() things but that might be slower if there is a lot of rearrangement to do. skip_unecessary_picks() could just set the current item to the one we want to start with.

Best Wishes

Phillip

Quoted 9 lines
Quoted 5 lines

Best Wishes

Phillip

Cheers,
Alban

#9 lore raw

Re: [PATCH 04/15] sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

Quoted 40 lines · Phillip Wood
Quoted 31 lines · Alban Gruin

Hi Phillip,

thanks for taking the time to review my patches.

Quoted 11 lines · Phillip Wood
Quoted 10 lines · Alban Gruin
sequencer.c +2 -2
@@ -4419,15 +4406,38 @@ int sequencer_add_exec_commands(const char
 *commands)
      }
        /* insert or append final <commands> */
-    if (insert >= 0 && insert < todo_list.nr)
-        strbuf_insert(buf, todo_list.items[insert].offset_in_buf +
+    if (insert >= 0 && insert < todo_list->nr)
+        strbuf_insert(buf, todo_list->items[insert].offset_in_buf +
                    offset, commands, commands_len);
      else if (insert >= 0 || !offset)
Quoted 8 lines

sequencer.c int sequencer_add_exec_commands(const char *commands) · line 4416

Quoted 4 lines
sequencer.c +2 -0
          strbuf_add(buf, commands, commands_len);
  -    i = write_message(buf->buf, buf->len, todo_file, 0);
+    if (todo_list_parse_insn_buffer(buf->buf, todo_list))
+        BUG("unusable todo list");}

It is a shame to have to re-parse the todo list, I wonder how difficult it would be to adjust the todo_list item array as the exec commands are inserted. The same applies to the next couple of patches

Good question.

This function inserts an exec' command after every pick' command. These commands are stored in a dynamically allocated list, grew with ALLOW_GROW().

Quoted 7 lines

If we want to keep the current structure, we would have to grow the size of the list by 1 and move several element to the end every time we want to add an `exec' command. It would not be very effective. Perhaps I should use a linked list here, instead. It may also work well with rearrange_squash() and skip_unnecessary_picks().

Maybe we could even get rid of the strbuf at some point.

Quoted 15 lines

Another way would be to use the strbuf as a string pool rather than a copy of the text of the file. There could be a write_todo_list() function that takes a todo list and some flags, iterates over the items in the todo list and writes the file. The flags would specify whether to append the todo help and whether to abbreviate the object ids (so there is no need for a separate call to transform_todos()). Then add_exec_commands() could allocate a new array of todo items which it builds up as it works through the original list and replaces the original list with the new one at the end. The text of the exec items can be added to the end of the strbuf (we only need one copy of the exec text with this scheme). rearrange_squash() can use a temporary array to build a new list as well or just memmove() things but that might be slower if there is a lot of rearrangement to do. skip_unecessary_picks() could just set the current item to the one we want to start with.

This sounds good, and it looks like the solution dscho proposed on IRC a few hours ago[0]. I will try to do this.

[0] http://colabti.org/irclogger/irclogger_log/git-devel?date=2018-10-12#l46

Cheers,
Alban

#10 lore raw

PATCH 5/15 sequencer: refactor rearrange_squash() to work on a todo_list

In reply to Alban Gruin

This refactors rearrange_squash() to work on a todo_list to avoid redundant reads and writes. The function is renamed todo_list_rearrange_squash().

As rebase -p still need to check the todo list from the disk, a new function is introduced, rearrange_squash_in_todo_file().

---
 builtin/rebase--interactive.c |  2 +-
 sequencer.c                   | 73 +++++++++++++++++++++--------------
 sequencer.h                   |  2 +-
 3 files changed, 46 insertions(+), 31 deletions(-)
builtin/rebase--interactive.c +1 -1
index ea1f93ccb6..8deef126d1 100644
@@ -258,7 +258,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = check_todo_list_from_file();
 		break;
 	case REARRANGE_SQUASH:
-		ret = rearrange_squash();
+		ret = rearrange_squash_in_todo_file();
 		break;
 	case ADD_EXEC:
 		ret = sequencer_add_exec_commands(cmd);
sequencer.c +44 -29

145 lines not on this page · raw message

sequencer.h +1 -1
index fb8b85bf9e..7f5668500f 100644
@@ -141,7 +141,7 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, const char *cmd,
 		    unsigned autosquash);
-int rearrange_squash(void);
+int rearrange_squash_in_todo_file(void);
 
 extern const char sign_off_header[];
 
Signature · 1 line
2.19.1
#11 lore raw

PATCH 6/15 sequencer: refactor transform_todos() to work on a todo_list

In reply to Alban Gruin

This refactors transform_todos() to work on a todo_list. The function is renamed todo_list_transform().

As rebase -p still need to check the todo list from the disk, a new function is introduced, transform_todo_file().

todo_list_transform() is not a static function, because it will be used by edit_todo_list() from rebase-interactive.c in a future commit.

---
 builtin/rebase--interactive.c |  2 +-
 rebase-interactive.c          |  4 +--
 sequencer.c                   | 46 +++++++++++++++++++++++------------
 sequencer.h                   |  3 ++-
 4 files changed, 35 insertions(+), 20 deletions(-)
builtin/rebase--interactive.c +1 -1
index 8deef126d1..f827e53f05 100644
@@ -252,7 +252,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 	}
 	case SHORTEN_OIDS:
 	case EXPAND_OIDS:
-		ret = transform_todos(flags);
+		ret = transform_todo_file(flags);
 		break;
 	case CHECK_TODO_LIST:
 		ret = check_todo_list_from_file();
rebase-interactive.c +2 -2
index ef8540245d..7c7f720a3d 100644
@@ -92,7 +92,7 @@ int edit_todo_list(unsigned flags)
 
 	strbuf_release(&buf);
 
-	transform_todos(flags | TODO_LIST_SHORTEN_IDS);
+	transform_todo_file(flags | TODO_LIST_SHORTEN_IDS);
 
 	if (strbuf_read_file(&buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
@@ -108,7 +108,7 @@ int edit_todo_list(unsigned flags)
 	if (launch_sequence_editor(todo_file, NULL, NULL))
 		return -1;
 
-	transform_todos(flags & ~(TODO_LIST_SHORTEN_IDS));
+	transform_todo_file(flags & ~(TODO_LIST_SHORTEN_IDS));
 
 	return 0;
 }
sequencer.c +30 -16

76 lines not on this page · raw message

sequencer.h +2 -1
index 7f5668500f..e1faca7884 100644
@@ -112,6 +112,7 @@ struct todo_list {
 #define TODO_LIST_INIT { STRBUF_INIT }
 
 int todo_list_parse_insn_buffer(char *buf, struct todo_list *todo_list);
+void todo_list_transform(struct todo_list *todo_list, unsigned flags);
 void todo_list_release(struct todo_list *todo_list);
 
 /* Call this to setup defaults before parsing command line options */
@@ -135,7 +136,7 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
 int sequencer_add_exec_commands(const char *command);
-int transform_todos(unsigned flags);
+int transform_todo_file(unsigned flags);
 int check_todo_list_from_file(void);
 int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
Signature · 1 line
2.19.1
#12 lore raw

PATCH 7/15 sequencer: make sequencer_make_script() write its script to a strbuf

In reply to Alban Gruin

This makes sequencer_make_script() write its script to a strbuf (ie. the buffer of a todo_list) instead of a FILE. This reduce the amount of read/write made by rebase interactive.

---
 builtin/rebase--interactive.c | 13 +++++++-----
 sequencer.c                   | 38 ++++++++++++++++-------------------
 sequencer.h                   |  2 +-
 3 files changed, 26 insertions(+), 27 deletions(-)
builtin/rebase--interactive.c +8 -5
index f827e53f05..eef1ff2e83 100644
@@ -71,7 +71,8 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	const char *head_hash = NULL;
 	char *revisions = NULL, *shortrevisions = NULL;
 	struct argv_array make_script_args = ARGV_ARRAY_INIT;
-	FILE *todo_list;
+	FILE *todo_list_file;
+	struct todo_list todo_list = TODO_LIST_INIT;
 
 	if (prepare_branch_to_be_rebased(opts, switch_to))
 		return -1;
@@ -93,8 +94,8 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	if (!upstream && squash_onto)
 		write_file(path_squash_onto(), "%s\n", squash_onto);
 
-	todo_list = fopen(rebase_path_todo(), "w");
-	if (!todo_list) {
+	todo_list_file = fopen(rebase_path_todo(), "w");
+	if (!todo_list_file) {
 		free(revisions);
 		free(shortrevisions);
 
@@ -105,10 +106,11 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	if (restrict_revision)
 		argv_array_push(&make_script_args, restrict_revision);
 
-	ret = sequencer_make_script(todo_list,
+	ret = sequencer_make_script(&todo_list.buf,
 				    make_script_args.argc, make_script_args.argv,
 				    flags);
-	fclose(todo_list);
+	fputs(todo_list.buf.buf, todo_list_file);
+	fclose(todo_list_file);
 
 	if (ret)
 		error(_("could not generate todo list"));
@@ -120,6 +122,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 
 	free(revisions);
 	free(shortrevisions);
+	todo_list_release(&todo_list);
 	argv_array_clear(&make_script_args);
 
 	return ret;
sequencer.c +17 -21
index 30a7fe3958..dfb8d1c974 100644
@@ -4083,7 +4083,7 @@ static const char *label_oid(struct object_id *oid, const char *label,
 }
 
 static int make_script_with_merges(struct pretty_print_context *pp,
-				   struct rev_info *revs, FILE *out,
+				   struct rev_info *revs, struct strbuf *out,
 				   unsigned flags)
 {
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
@@ -4230,7 +4230,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 	 * gathering commits not yet shown, reversing the list on the fly,
 	 * then outputting that list (labeling revisions as needed).
 	 */
-	fprintf(out, "%s onto\n", cmd_label);
+	strbuf_addf(out, "%s onto\n", cmd_label);
 	for (iter = tips; iter; iter = iter->next) {
 		struct commit_list *list = NULL, *iter2;
 
@@ -4240,9 +4240,9 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 		entry = oidmap_get(&state.commit2label, &commit->object.oid);
 
 		if (entry)
-			fprintf(out, "\n%c Branch %s\n", comment_line_char, entry->string);
+			strbuf_addf(out, "\n%c Branch %s\n", comment_line_char, entry->string);
 		else
-			fprintf(out, "\n");
+			strbuf_addf(out, "\n");
 
 		while (oidset_contains(&interesting, &commit->object.oid) &&
 		       !oidset_contains(&shown, &commit->object.oid)) {
@@ -4255,8 +4255,8 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 		}
 
 		if (!commit)
-			fprintf(out, "%s %s\n", cmd_reset,
-				rebase_cousins ? "onto" : "[new root]");
+			strbuf_addf(out, "%s %s\n", cmd_reset,
+				    rebase_cousins ? "onto" : "[new root]");
 		else {
 			const char *to = NULL;
 
@@ -4269,12 +4269,12 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 					       &state);
 
 			if (!to || !strcmp(to, "onto"))
-				fprintf(out, "%s onto\n", cmd_reset);
+				strbuf_addf(out, "%s onto\n", cmd_reset);
 			else {
 				strbuf_reset(&oneline);
 				pretty_print_commit(pp, commit, &oneline);
-				fprintf(out, "%s %s # %s\n",
-					cmd_reset, to, oneline.buf);
+				strbuf_addf(out, "%s %s # %s\n",
+					    cmd_reset, to, oneline.buf);
 			}
 		}
 
@@ -4283,11 +4283,11 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 			entry = oidmap_get(&commit2todo, oid);
 			/* only show if not already upstream */
 			if (entry)
-				fprintf(out, "%s\n", entry->string);
+				strbuf_addf(out, "%s\n", entry->string);
 			entry = oidmap_get(&state.commit2label, oid);
 			if (entry)
-				fprintf(out, "%s %s\n",
-					cmd_label, entry->string);
+				strbuf_addf(out, "%s %s\n",
+					    cmd_label, entry->string);
 			oidset_insert(&shown, oid);
 		}
 
@@ -4309,12 +4309,11 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 	return 0;
 }
 
-int sequencer_make_script(FILE *out, int argc, const char **argv,
+int sequencer_make_script(struct strbuf *out, int argc, const char **argv,
 			  unsigned flags)
 {
 	char *format = NULL;
 	struct pretty_print_context pp = {0};
-	struct strbuf buf = STRBUF_INIT;
 	struct rev_info revs;
 	struct commit *commit;
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
@@ -4357,16 +4356,13 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 
 		if (!is_empty && (commit->object.flags & PATCHSAME))
 			continue;
-		strbuf_reset(&buf);
 		if (!keep_empty && is_empty)
-			strbuf_addf(&buf, "%c ", comment_line_char);
-		strbuf_addf(&buf, "%s %s ", insn,
+			strbuf_addf(out, "%c ", comment_line_char);
+		strbuf_addf(out, "%s %s ", insn,
 			    oid_to_hex(&commit->object.oid));
-		pretty_print_commit(&pp, commit, &buf);
-		strbuf_addch(&buf, '\n');
-		fputs(buf.buf, out);
+		pretty_print_commit(&pp, commit, out);
+		strbuf_addch(out, '\n');
 	}
-	strbuf_release(&buf);
 	return 0;
 }
 
sequencer.h +1 -1
index e1faca7884..21d9ba09ab 100644
@@ -132,7 +132,7 @@ int sequencer_remove_state(struct replay_opts *opts);
  * commits should be rebased onto the new base, this flag needs to be passed.
  */
 #define TODO_LIST_REBASE_COUSINS (1U << 4)
-int sequencer_make_script(FILE *out, int argc, const char **argv,
+int sequencer_make_script(struct strbuf *out, int argc, const char **argv,
 			  unsigned flags);
 
 int sequencer_add_exec_commands(const char *command);
Signature · 1 line
2.19.1
#13 lore raw

Re: [PATCH 07/15] sequencer: make sequencer_make_script() write its script to a strbuf

Quoted 23 lines · Alban Gruin
sequencer.c +2 -2
index 30a7fe3958..dfb8d1c974 100644
@@ -4083,7 +4083,7 @@ static const char *label_oid(struct object_id *oid, const char *label,
 }
 
 static int make_script_with_merges(struct pretty_print_context *pp,
-				   struct rev_info *revs, FILE *out,
+				   struct rev_info *revs, struct strbuf *out,
 				   unsigned flags)
 {
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
@@ -4230,7 +4230,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 	 * gathering commits not yet shown, reversing the list on the fly,
 	 * then outputting that list (labeling revisions as needed).
 	 */
-	fprintf(out, "%s onto\n", cmd_label);
+	strbuf_addf(out, "%s onto\n", cmd_label);
 	for (iter = tips; iter; iter = iter->next) {
 		struct commit_list *list = NULL, *iter2;
 

sequencer.c static int make_script_with_merges(struct pretty_print_context *pp, · line 4245

Quoted 8 lines
sequencer.c +2 -2
@@ -4240,9 +4240,9 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 		entry = oidmap_get(&state.commit2label, &commit->object.oid);
 
 		if (entry)
-			fprintf(out, "\n%c Branch %s\n", comment_line_char, entry->string);
+			strbuf_addf(out, "\n%c Branch %s\n", comment_line_char, entry->string);
 		else
-			fprintf(out, "\n");
+			strbuf_addf(out, "\n");

Please use plain strbuf_add() here.

Or strbuf_complete_line()? Dunno, as seen in the previous hunk, 'out' won't be empty at this point.

#14 lore raw

Re: [PATCH 07/15] sequencer: make sequencer_make_script() write its script to a strbuf

Quoted 8 lines · SZEDER Gábor

Review

Quoted 6 lines
(quoted diff) +2 -2
 		if (entry)
-			fprintf(out, "\n%c Branch %s\n", comment_line_char, entry->string);
+			strbuf_addf(out, "\n%c Branch %s\n", comment_line_char, entry->string);
 		else
-			fprintf(out, "\n");
+			strbuf_addf(out, "\n");

Please use plain strbuf_add() here.

FWIW, contrib/coccinelle/strbuf.cocci.patch gave us this:

sequencer.c +1 -1
diff -u -p a/sequencer.c b/sequencer.c
@@ -4311,7 +4311,7 @@ static int make_script_with_merges(struc
 		if (entry)
 			strbuf_addf(out, "\n%c Branch %s\n", comment_line_char, entry->string);
 		else
-			strbuf_addf(out, "\n");
+			strbuf_addstr(out, "\n");
 
 		while (oidset_contains(&interesting, &commit->object.oid) &&
 		       !oidset_contains(&shown, &commit->object.oid)) {
#15 lore raw

Re: [PATCH 07/15] sequencer: make sequencer_make_script() write its script to a strbuf

Quoted 17 lines · Junio C Hamano
Quoted 8 lines · SZEDER Gábor

Review

Quoted 6 lines
(quoted diff) +2 -2
 		if (entry)
-			fprintf(out, "\n%c Branch %s\n", comment_line_char, entry->string);
+			strbuf_addf(out, "\n%c Branch %s\n", comment_line_char, entry->string);
 		else
-			fprintf(out, "\n");
+			strbuf_addf(out, "\n");

Please use plain strbuf_add() here.

FWIW, contrib/coccinelle/strbuf.cocci.patch gave us this:

diff -u -p a/sequencer.c b/sequencer.c

sequencer.c static int make_script_with_merges(struc · line 4317

Quoted 8 lines
sequencer.c +1 -1
@@ -4311,7 +4311,7 @@ static int make_script_with_merges(struc
 		if (entry)
 			strbuf_addf(out, "\n%c Branch %s\n", comment_line_char, entry->string);
 		else
-			strbuf_addf(out, "\n");
+			strbuf_addstr(out, "\n");
 
 		while (oidset_contains(&interesting, &commit->object.oid) &&
 		       !oidset_contains(&shown, &commit->object.oid)) {

Uh, right. I didn't want to copy-paste a patch with too long lines into my mailer, as it usually doesn't end too good, so I just typed the function name. Evidently I couldn't quite manage.

#16 lore raw

PATCH 8/15 sequencer: change complete_action() to use the refactored functions

In reply to Alban Gruin

complete_action() used functions that read the todo-list file, made some changes to it, and wrote it back to the disk.

The previous commits were dedicated to separate the part that deals with the file from the actual logic of these functions. Now that this is done, we can call directly the "logic" functions to avoid useless file access.

---
 builtin/rebase--interactive.c | 13 +-----
 sequencer.c                   | 76 +++++++++++++++++------------------
 sequencer.h                   |  2 +-
 3 files changed, 38 insertions(+), 53 deletions(-)
builtin/rebase--interactive.c +1 -12
index eef1ff2e83..0700339f90 100644
@@ -71,7 +71,6 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	const char *head_hash = NULL;
 	char *revisions = NULL, *shortrevisions = NULL;
 	struct argv_array make_script_args = ARGV_ARRAY_INIT;
-	FILE *todo_list_file;
 	struct todo_list todo_list = TODO_LIST_INIT;
 
 	if (prepare_branch_to_be_rebased(opts, switch_to))
@@ -94,14 +93,6 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	if (!upstream && squash_onto)
 		write_file(path_squash_onto(), "%s\n", squash_onto);
 
-	todo_list_file = fopen(rebase_path_todo(), "w");
-	if (!todo_list_file) {
-		free(revisions);
-		free(shortrevisions);
-
-		return error_errno(_("could not open %s"), rebase_path_todo());
-	}
-
 	argv_array_pushl(&make_script_args, "", revisions, NULL);
 	if (restrict_revision)
 		argv_array_push(&make_script_args, restrict_revision);
@@ -109,15 +100,13 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	ret = sequencer_make_script(&todo_list.buf,
 				    make_script_args.argc, make_script_args.argv,
 				    flags);
-	fputs(todo_list.buf.buf, todo_list_file);
-	fclose(todo_list_file);
 
 	if (ret)
 		error(_("could not generate todo list"));
 	else {
 		discard_cache();
 		ret = complete_action(opts, flags, shortrevisions, onto_name, onto,
-				      head_hash, cmd, autosquash);
+				      head_hash, cmd, autosquash, &todo_list);
 	}
 
 	free(revisions);
sequencer.c +36 -40

129 lines not on this page · raw message

sequencer.h +1 -1
index 21d9ba09ab..5bd3b79282 100644
@@ -141,7 +141,7 @@ int check_todo_list_from_file(void);
 int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, const char *cmd,
-		    unsigned autosquash);
+		    unsigned autosquash, struct todo_list *todo_list);
 int rearrange_squash_in_todo_file(void);
 
 extern const char sign_off_header[];
Signature · 1 line
2.19.1
#17 lore raw

Re: [PATCH 08/15] sequencer: change complete_action() to use the refactored functions

Quoted 43 lines · Alban Gruin

complete_action() used functions that read the todo-list file, made some changes to it, and wrote it back to the disk.

The previous commits were dedicated to separate the part that deals with the file from the actual logic of these functions. Now that this is done, we can call directly the "logic" functions to avoid useless file access.

---
  builtin/rebase--interactive.c | 13 +-----
  sequencer.c                   | 76 +++++++++++++++++------------------
  sequencer.h                   |  2 +-
  3 files changed, 38 insertions(+), 53 deletions(-)
builtin/rebase--interactive.c +0 -9
index eef1ff2e83..0700339f90 100644
@@ -71,7 +71,6 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
  	const char *head_hash = NULL;
  	char *revisions = NULL, *shortrevisions = NULL;
  	struct argv_array make_script_args = ARGV_ARRAY_INIT;
-	FILE *todo_list_file;
  	struct todo_list todo_list = TODO_LIST_INIT;
 
  	if (prepare_branch_to_be_rebased(opts, switch_to))
@@ -94,14 +93,6 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
  	if (!upstream && squash_onto)
  		write_file(path_squash_onto(), "%s\n", squash_onto);
 
-	todo_list_file = fopen(rebase_path_todo(), "w");
-	if (!todo_list_file) {
-		free(revisions);
-		free(shortrevisions);
-
-		return error_errno(_("could not open %s"), rebase_path_todo());
-	}
-
  	argv_array_pushl(&make_script_args, "", revisions, NULL);
  	if (restrict_revision)
  		argv_array_push(&make_script_args, restrict_revision);

builtin/rebase--interactive.c static int do_interactive_rebase(struct replay_opts *opts, unsigned flags, · line 102

Quoted 3 lines
builtin/rebase--interactive.c +0 -0
@@ -109,15 +100,13 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
  	ret = sequencer_make_script(&todo_list.buf,
  				    make_script_args.argc, make_script_args.argv,
  				    flags);

I think it would be clearer to parse the todo list here explicitly rather than doing it implicitly in complete_action()

Quoted 131 lines
builtin/rebase--interactive.c +1 -3
-	fputs(todo_list.buf.buf, todo_list_file);
-	fclose(todo_list_file);
 
  	if (ret)
  		error(_("could not generate todo list"));
  	else {
  		discard_cache() >   		ret = complete_action(opts, flags, shortrevisions, onto_name, onto,
-				      head_hash, cmd, autosquash);
+				      head_hash, cmd, autosquash, &todo_list);
  	}
 
  	free(revisions);
sequencer.c +28 -38
index dfb8d1c974..b37935e5ab 100644
@@ -4624,93 +4624,89 @@ static int skip_unnecessary_picks(struct object_id *output_oid)
  	return 0;
  }
 
+static int todo_list_rearrange_squash(struct todo_list *todo_list);
+
  int complete_action(struct replay_opts *opts, unsigned flags,
  		    const char *shortrevisions, const char *onto_name,
  		    const char *onto, const char *orig_head, const char *cmd,
-		    unsigned autosquash)
+		    unsigned autosquash, struct todo_list *todo_list)
  {
  	const char *shortonto, *todo_file = rebase_path_todo();
-	struct todo_list todo_list = TODO_LIST_INIT;
-	struct strbuf *buf = &(todo_list.buf);
+	struct todo_list new_todo = TODO_LIST_INIT;
+	struct strbuf *buf = &todo_list->buf;
  	struct object_id oid;
-	struct stat st;
+	int command_count;
 
  	get_oid(onto, &oid);
  	shortonto = find_unique_abbrev(&oid, DEFAULT_ABBREV);
 
-	if (!lstat(todo_file, &st) && st.st_size == 0 &&
-	    write_message("noop\n", 5, todo_file, 0))
-		return -1;
+	if (buf->len == 0)
+		strbuf_add(buf, "noop\n", 5);
+
+	if (todo_list_parse_insn_buffer(buf->buf, todo_list))
+		BUG("unusable todo list");
 
-	if (autosquash && rearrange_squash_in_todo_file())
+	if (autosquash && todo_list_rearrange_squash(todo_list))
  		return -1;
 
  	if (cmd && *cmd)
-		sequencer_add_exec_commands(cmd);
+		todo_list_add_exec_commands(todo_list, cmd);
 
-	if (strbuf_read_file(buf, todo_file, 0) < 0)
-		return error_errno(_("could not read '%s'."), todo_file);
-
-	if (todo_list_parse_insn_buffer(buf->buf, &todo_list)) {
-		todo_list_release(&todo_list);
-		return error(_("unusable todo list: '%s'"), todo_file);
-	}
-
-	if (count_commands(&todo_list) == 0) {
+	command_count = count_commands(todo_list);
+	if (command_count == 0) {
  		apply_autostash(opts);
  		sequencer_remove_state(opts);
-		todo_list_release(&todo_list);
 
  		return error(_("nothing to do"));
  	}
 
+	todo_list_transform(todo_list, flags | TODO_LIST_SHORTEN_IDS);
+
  	strbuf_addch(buf, '\n');
  	strbuf_commented_addf(buf, Q_("Rebase %s onto %s (%d command)",
  				      "Rebase %s onto %s (%d commands)",
-				      count_commands(&todo_list)),
-			      shortrevisions, shortonto, count_commands(&todo_list));
+				      command_count),
+			      shortrevisions, shortonto, command_count);
  	append_todo_help(0, flags & TODO_LIST_KEEP_EMPTY, buf);
 
-	if (write_message(buf->buf, buf->len, todo_file, 0)) {
-		todo_list_release(&todo_list);
-		return -1;
-	}
+	if (write_message(buf->buf, buf->len, todo_file, 0))
+		return error_errno(_("could not write '%s'"), todo_file);
 
  	if (copy_file(rebase_path_todo_backup(), todo_file, 0666))
  		return error(_("could not copy '%s' to '%s'."), todo_file,
  			     rebase_path_todo_backup());
 
-	if (transform_todo_file(flags | TODO_LIST_SHORTEN_IDS))
-		return error(_("could not transform the todo list"));
-
-	strbuf_reset(buf);
-
-	if (launch_sequence_editor(todo_file, buf, NULL)) {
+	if (launch_sequence_editor(todo_file, &new_todo.buf, NULL)) {
  		apply_autostash(opts);
  		sequencer_remove_state(opts);
-		todo_list_release(&todo_list);
 
  		return -1;
  	}
 
-	strbuf_stripspace(buf, 1);
-	if (buf->len == 0) {
+	strbuf_stripspace(&new_todo.buf, 1);
+	if (new_todo.buf.len == 0) {
  		apply_autostash(opts);
  		sequencer_remove_state(opts);
-		todo_list_release(&todo_list);
+		todo_list_release(&new_todo);
 
  		return error(_("nothing to do"));
  	}
 
-	todo_list_release(&todo_list);
-
-	if (check_todo_list_from_file()) {
+	if (todo_list_check(todo_list, &new_todo)) {
  		checkout_onto(opts, onto_name, onto, orig_head);
+		todo_list_release(&new_todo);
+
  		return -1;

sequencer.c static int skip_unnecessary_picks(struct object_id *output_oid) · line 4707

Quoted 10 lines
sequencer.c +6 -2
  	}
 
-	if (transform_todo_file(flags & ~(TODO_LIST_SHORTEN_IDS)))
-		return error(_("could not transform the todo list"));
+	todo_list_transform(&new_todo, flags & ~(TODO_LIST_SHORTEN_IDS));
+
+	if (rewrite_file(todo_file, new_todo.buf.buf, new_todo.buf.len) < 0) {
+		todo_list_release(&new_todo);
+		return error_errno(_("could not write '%s'"), todo_file);
+	}

rewrite_file() can truncate the old version of the file if there is an error when writing the new version, I think it would be better to use write_message() instead as that atomically updates the file. The same applies to patch 5 (refactor rearrange_squash()) after which I think there will be no callers to rewrite_file() so it can be deleted.

Best Wishes

Phillip

Quoted 19 lines
sequencer.c +2 -0
+
+	todo_list_release(&new_todo);
 
  	if (opts->allow_ff && skip_unnecessary_picks(&oid))
  		return error(_("could not skip unnecessary pick commands"));
sequencer.h +1 -1
index 21d9ba09ab..5bd3b79282 100644
@@ -141,7 +141,7 @@ int check_todo_list_from_file(void);
  int complete_action(struct replay_opts *opts, unsigned flags,
  		    const char *shortrevisions, const char *onto_name,
  		    const char *onto, const char *orig_head, const char *cmd,
-		    unsigned autosquash);
+		    unsigned autosquash, struct todo_list *todo_list);
  int rearrange_squash_in_todo_file(void);
 
  extern const char sign_off_header[];
#18 lore raw

Re: [PATCH 08/15] sequencer: change complete_action() to use the refactored functions

Le 11/10/2018 à 15:51, Phillip Wood a écrit :

Quoted 7 lines · Alban Gruin
Quoted 5 lines

+    if (rewrite_file(todo_file, new_todo.buf.buf, new_todo.buf.len) <
0) {
+        todo_list_release(&new_todo);
+        return error_errno(_("could not write '%s'"), todo_file); +    }

Quoted 5 lines

rewrite_file() can truncate the old version of the file if there is an error when writing the new version, I think it would be better to use write_message() instead as that atomically updates the file. The same applies to patch 5 (refactor rearrange_squash()) after which I think there will be no callers to rewrite_file() so it can be deleted.

You’re right, I didn’t notice that.

Quoted 5 lines

Best Wishes

Phillip

Cheers,
Alban

#19 lore raw

PATCH 9/15 sequencer: refactor skip_unnecessary_picks() to work on a todo_list

In reply to Alban Gruin

This refactors skip_unnecessary_picks() to work on a todo_list. The file-handling logic is completely dropped here, as its only usage is made by complete_action().

---
 sequencer.c | 56 +++++++++++++++--------------------------------------
 1 file changed, 16 insertions(+), 40 deletions(-)
sequencer.c +16 -40
index b37935e5ab..a432b64048 100644
@@ -4545,38 +4545,20 @@ static int rewrite_file(const char *path, const char *buf, size_t len)
 }
 
 /* skip picking commits whose parents are unchanged */
-static int skip_unnecessary_picks(struct object_id *output_oid)
+static int skip_unnecessary_picks(struct todo_list *todo_list,
+				  struct object_id *output_oid)
 {
-	const char *todo_file = rebase_path_todo();
-	struct strbuf buf = STRBUF_INIT;
-	struct todo_list todo_list = TODO_LIST_INIT;
 	struct object_id *parent_oid;
 	int fd, i;
 
-	if (!read_oneliner(&buf, rebase_path_onto(), 0))
-		return error(_("could not read 'onto'"));
-	if (get_oid(buf.buf, output_oid)) {
-		strbuf_release(&buf);
-		return error(_("need a HEAD to fixup"));
-	}
-	strbuf_release(&buf);
-
-	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
-		return -1;
-	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {
-		todo_list_release(&todo_list);
-		return -1;
-	}
-
-	for (i = 0; i < todo_list.nr; i++) {
-		struct todo_item *item = todo_list.items + i;
+	for (i = 0; i < todo_list->nr; i++) {
+		struct todo_item *item = todo_list->items + i;
 
 		if (item->command >= TODO_NOOP)
 			continue;
 		if (item->command != TODO_PICK)
 			break;
 		if (parse_commit(item->commit)) {
-			todo_list_release(&todo_list);
 			return error(_("could not parse commit '%s'"),
 				oid_to_hex(&item->commit->object.oid));
 		}
@@ -4590,37 +4572,29 @@ static int skip_unnecessary_picks(struct object_id *output_oid)
 		oidcpy(output_oid, &item->commit->object.oid);
 	}
 	if (i > 0) {
-		int offset = get_item_line_offset(&todo_list, i);
+		int offset = get_item_line_offset(todo_list, i);
 		const char *done_path = rebase_path_done();
 
 		fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
 		if (fd < 0) {
 			error_errno(_("could not open '%s' for writing"),
 				    done_path);
-			todo_list_release(&todo_list);
 			return -1;
 		}
-		if (write_in_full(fd, todo_list.buf.buf, offset) < 0) {
+		if (write_in_full(fd, todo_list->buf.buf, offset) < 0) {
 			error_errno(_("could not write to '%s'"), done_path);
-			todo_list_release(&todo_list);
 			close(fd);
 			return -1;
 		}
 		close(fd);
 
-		if (rewrite_file(rebase_path_todo(), todo_list.buf.buf + offset,
-				 todo_list.buf.len - offset) < 0) {
-			todo_list_release(&todo_list);
-			return -1;
-		}
+		strbuf_splice(&todo_list->buf, 0, offset, NULL, 0);
 
-		todo_list.current = i;
-		if (is_fixup(peek_command(&todo_list, 0)))
-			record_in_rewritten(output_oid, peek_command(&todo_list, 0));
+		todo_list->current = i;
+		if (is_fixup(peek_command(todo_list, 0)))
+			record_in_rewritten(output_oid, peek_command(todo_list, 0));
 	}
 
-	todo_list_release(&todo_list);
-
 	return 0;
 }
 
@@ -4701,6 +4675,11 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 
 	todo_list_transform(&new_todo, flags & ~(TODO_LIST_SHORTEN_IDS));
 
+	if (opts->allow_ff && skip_unnecessary_picks(&new_todo, &oid)) {
+		todo_list_release(&new_todo);
+		return error(_("could not skip unnecessary pick commands"));
+	}
+
 	if (rewrite_file(todo_file, new_todo.buf.buf, new_todo.buf.len) < 0) {
 		todo_list_release(&new_todo);
 		return error_errno(_("could not write '%s'"), todo_file);
@@ -4708,12 +4687,9 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 
 	todo_list_release(&new_todo);
 
-	if (opts->allow_ff && skip_unnecessary_picks(&oid))
-		return error(_("could not skip unnecessary pick commands"));
-
 	if (checkout_onto(opts, onto_name, oid_to_hex(&oid), orig_head))
 		return -1;
-;
+
 	if (require_clean_work_tree("rebase", "", 1, 1))
 		return -1;
 
Signature · 1 line
2.19.1
#20 lore raw

PATCH 10/15 rebase-interactive: use todo_list_transform() in edit_todo_list()

In reply to Alban Gruin

Just like complete_action(), edit_todo_list() used a function (transform_todo_file()) that read the todo-list from the disk and wrote it back, resulting in useless disk accesses.

This changes edit_todo_list() to call directly todo_list_transform() instead.

---
 rebase-interactive.c | 40 +++++++++++++++++++---------------------
 1 file changed, 19 insertions(+), 21 deletions(-)
rebase-interactive.c +19 -21
index 7c7f720a3d..f42d48e192 100644
@@ -78,39 +78,37 @@ void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 
 int edit_todo_list(unsigned flags)
 {
-	struct strbuf buf = STRBUF_INIT;
 	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res = 0;
 
-	if (strbuf_read_file(&buf, todo_file, 0) < 0)
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
 
-	strbuf_stripspace(&buf, 1);
-	if (write_message(buf.buf, buf.len, todo_file, 0)) {
-		strbuf_release(&buf);
-		return -1;
-	}
-
-	strbuf_release(&buf);
+	strbuf_stripspace(&todo_list.buf, 1);
+	if (!todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list))
+		todo_list_transform(&todo_list, flags | TODO_LIST_SHORTEN_IDS);
 
-	transform_todo_file(flags | TODO_LIST_SHORTEN_IDS);
-
-	if (strbuf_read_file(&buf, todo_file, 0) < 0)
-		return error_errno(_("could not read '%s'."), todo_file);
+	append_todo_help(1, 0, &todo_list.buf);
 
-	append_todo_help(1, 0, &buf);
-	if (write_message(buf.buf, buf.len, todo_file, 0)) {
-		strbuf_release(&buf);
+	if (write_message(todo_list.buf.buf, todo_list.buf.len, todo_file, 0)) {
+		todo_list_release(&todo_list);
 		return -1;
 	}
 
-	strbuf_release(&buf);
-
-	if (launch_sequence_editor(todo_file, NULL, NULL))
+	strbuf_reset(&todo_list.buf);
+	if (launch_sequence_editor(todo_file, &todo_list.buf, NULL)) {
+		todo_list_release(&todo_list);
 		return -1;
+	}
 
-	transform_todo_file(flags & ~(TODO_LIST_SHORTEN_IDS));
+	if (!todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
+		todo_list_transform(&todo_list, flags & ~(TODO_LIST_SHORTEN_IDS));
+		res = write_message(todo_list.buf.buf, todo_list.buf.len, todo_file, 0);
+	}
 
-	return 0;
+	todo_list_release(&todo_list);
+	return res;
 }
 
 define_commit_slab(commit_seen, unsigned char);
Signature · 1 line
2.19.1
#21 lore raw

Re: [PATCH 10/15] rebase-interactive: use todo_list_transform() in edit_todo_list()

Quoted 37 lines · Alban Gruin

Just like complete_action(), edit_todo_list() used a function (transform_todo_file()) that read the todo-list from the disk and wrote it back, resulting in useless disk accesses.

This changes edit_todo_list() to call directly todo_list_transform() instead.

---
  rebase-interactive.c | 40 +++++++++++++++++++---------------------
  1 file changed, 19 insertions(+), 21 deletions(-)
rebase-interactive.c +4 -9
index 7c7f720a3d..f42d48e192 100644
@@ -78,39 +78,37 @@ void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 
  int edit_todo_list(unsigned flags)
  {
-	struct strbuf buf = STRBUF_INIT;
  	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res = 0;
 
-	if (strbuf_read_file(&buf, todo_file, 0) < 0)
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
  		return error_errno(_("could not read '%s'."), todo_file);
 
-	strbuf_stripspace(&buf, 1);
-	if (write_message(buf.buf, buf.len, todo_file, 0)) {
-		strbuf_release(&buf);
-		return -1;
-	}
-
-	strbuf_release(&buf);
+	strbuf_stripspace(&todo_list.buf, 1);

rebase-interactive.c void append_todo_help(unsigned edit_todo, unsigned keep_empty, · line 100

Quoted 10 lines
rebase-interactive.c +3 -5
+	if (!todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list))
+		todo_list_transform(&todo_list, flags | TODO_LIST_SHORTEN_IDS);
 
-	transform_todo_file(flags | TODO_LIST_SHORTEN_IDS);
-
-	if (strbuf_read_file(&buf, todo_file, 0) < 0)
-		return error_errno(_("could not read '%s'."), todo_file);
+	append_todo_help(1, 0, &todo_list.buf);
 
-	append_todo_help(1, 0, &buf);

I think this patch is fine, I was just wondering if you meant to move the call to append_todo_help() above the blank line?

Best Wishes

Phillip

Quoted 29 lines
rebase-interactive.c +12 -7
-	if (write_message(buf.buf, buf.len, todo_file, 0)) {
-		strbuf_release(&buf);
+	if (write_message(todo_list.buf.buf, todo_list.buf.len, todo_file, 0)) {
+		todo_list_release(&todo_list);
  		return -1;
  	}
 
-	strbuf_release(&buf);
-
-	if (launch_sequence_editor(todo_file, NULL, NULL))
+	strbuf_reset(&todo_list.buf);
+	if (launch_sequence_editor(todo_file, &todo_list.buf, NULL)) {
+		todo_list_release(&todo_list);
  		return -1;
+	}
 
-	transform_todo_file(flags & ~(TODO_LIST_SHORTEN_IDS));
+	if (!todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
+		todo_list_transform(&todo_list, flags & ~(TODO_LIST_SHORTEN_IDS));
+		res = write_message(todo_list.buf.buf, todo_list.buf.len, todo_file, 0);
+	}
 
-	return 0;
+	todo_list_release(&todo_list);
+	return res;
  }
 
  define_commit_slab(commit_seen, unsigned char);
#22 lore raw

Re: [PATCH 10/15] rebase-interactive: use todo_list_transform() in edit_todo_list()

Le 11/10/2018 à 17:16, Phillip Wood a écrit :

Quoted 39 lines · Alban Gruin
Quoted 38 lines

Just like complete_action(), edit_todo_list() used a function (transform_todo_file()) that read the todo-list from the disk and wrote it back, resulting in useless disk accesses.

This changes edit_todo_list() to call directly todo_list_transform() instead.

---
  rebase-interactive.c | 40 +++++++++++++++++++---------------------

  1 file changed, 19 insertions(+), 21 deletions(-)
rebase-interactive.c +6 -7
index 7c7f720a3d..f42d48e192 100644
@@ -78,39 +78,37 @@ void append_todo_help(unsigned edit_todo, unsigned
 keep_empty,
     int edit_todo_list(unsigned flags)
   {
-    struct strbuf buf = STRBUF_INIT;
       const char *todo_file = rebase_path_todo();
+    struct todo_list todo_list = TODO_LIST_INIT;
+    int res = 0;
   -    if (strbuf_read_file(&buf, todo_file, 0) < 0)
+    if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
           return error_errno(_("could not read '%s'."), todo_file);
   -    strbuf_stripspace(&buf, 1);
-    if (write_message(buf.buf, buf.len, todo_file, 0)) {
-        strbuf_release(&buf);
-        return -1;
-    }
-
-    strbuf_release(&buf);
+    strbuf_stripspace(&todo_list.buf, 1);
+    if (!todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list))
+        todo_list_transform(&todo_list, flags | TODO_LIST_SHORTEN_IDS);
   -    transform_todo_file(flags | TODO_LIST_SHORTEN_IDS);
Quoted 9 lines

rebase-interactive.c void append_todo_help(unsigned edit_todo, unsigned keep_empty,

Quoted 5 lines
rebase-interactive.c +1 -3
-
-    if (strbuf_read_file(&buf, todo_file, 0) < 0)
-        return error_errno(_("could not read '%s'."), todo_file);
+    append_todo_help(1, 0, &todo_list.buf);
   -    append_todo_help(1, 0, &buf);

I think this patch is fine, I was just wondering if you meant to move the call to append_todo_help() above the blank line?

No

Quoted 4 lines

Best Wishes

Phillip

Cheers,
Alban

#23 lore raw

PATCH 11/15 rebase-interactive: append_todo_help() changes

In reply to Alban Gruin

This moves the writing of the comment "Rebase $shortrevisions onto $shortonto ($command_count commands)" from complete_action() to append_todo_help().

shortrevisions, shortonto, and command_count are passed as parameters to append_todo_help().

During the initial edit of the todo list, shortrevisions and shortonto are not NULL. Therefore, if shortrevisions or shortonto is NULL, then edit_todo would be true, otherwise it would be false. Thus, edit_todo is removed from the parameters of append_todo_help().

edit_todo_list() and complete_action() are modified to fit these changes.

---
 rebase-interactive.c | 14 ++++++++++++--
 rebase-interactive.h |  3 ++-
 sequencer.c          |  8 ++------
 3 files changed, 16 insertions(+), 9 deletions(-)
rebase-interactive.c +12 -2
index f42d48e192..7168d56d17 100644
@@ -28,7 +28,8 @@ static enum missing_commit_check_level get_missing_commit_check_level(void)
 	return MISSING_COMMIT_CHECK_IGNORE;
 }
 
-void append_todo_help(unsigned edit_todo, unsigned keep_empty,
+void append_todo_help(unsigned keep_empty, int command_count,
+		      const char *shortrevisions, const char *shortonto,
 		      struct strbuf *buf)
 {
 	const char *msg = _("\nCommands:\n"
@@ -47,6 +48,15 @@ void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 ".       specified). Use -c <commit> to reword the commit message.\n"
 "\n"
 "These lines can be re-ordered; they are executed from top to bottom.\n");
+	unsigned edit_todo = !(shortrevisions && shortonto);
+
+	if (!edit_todo) {
+		strbuf_addch(buf, '\n');
+		strbuf_commented_addf(buf, Q_("Rebase %s onto %s (%d command)",
+					      "Rebase %s onto %s (%d commands)",
+					      command_count),
+				      shortrevisions, shortonto, command_count);
+	}
 
 	strbuf_add_commented_lines(buf, msg, strlen(msg));
 
@@ -89,7 +99,7 @@ int edit_todo_list(unsigned flags)
 	if (!todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list))
 		todo_list_transform(&todo_list, flags | TODO_LIST_SHORTEN_IDS);
 
-	append_todo_help(1, 0, &todo_list.buf);
+	append_todo_help(flags, 0, NULL, NULL, &todo_list.buf);
 
 	if (write_message(todo_list.buf.buf, todo_list.buf.len, todo_file, 0)) {
 		todo_list_release(&todo_list);
rebase-interactive.h +2 -1
index 6bc7bc315d..61858f3a02 100644
@@ -1,7 +1,8 @@
 #ifndef REBASE_INTERACTIVE_H
 #define REBASE_INTERACTIVE_H
 
-void append_todo_help(unsigned edit_todo, unsigned keep_empty,
+void append_todo_help(unsigned keep_empty, int command_count,
+		      const char *shortrevisions, const char *shortonto,
 		      struct strbuf *buf);
 int edit_todo_list(unsigned flags);
 int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
sequencer.c +2 -6
index a432b64048..94d3058359 100644
@@ -4636,12 +4636,8 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 
 	todo_list_transform(todo_list, flags | TODO_LIST_SHORTEN_IDS);
 
-	strbuf_addch(buf, '\n');
-	strbuf_commented_addf(buf, Q_("Rebase %s onto %s (%d command)",
-				      "Rebase %s onto %s (%d commands)",
-				      command_count),
-			      shortrevisions, shortonto, command_count);
-	append_todo_help(0, flags & TODO_LIST_KEEP_EMPTY, buf);
+	append_todo_help(flags & TODO_LIST_KEEP_EMPTY, command_count,
+			 shortrevisions, shortonto, buf);
 
 	if (write_message(buf->buf, buf->len, todo_file, 0))
 		return error_errno(_("could not write '%s'"), todo_file);
Signature · 1 line
2.19.1
#24 lore raw

PATCH 12/15 rebase-interactive: rewrite edit_todo_list() to handle the initial edit

In reply to Alban Gruin

edit_todo_list() is changed to work on a todo_list, and to handle the initial edition of the todo list (ie. making a backup of the todo list).

It does not check for dropped commits yet, as todo_list_check() does not work if the old todo list has invalid commands.

---
 builtin/rebase--interactive.c | 22 +++++++++++++++-
 rebase-interactive.c          | 49 ++++++++++++++++++-----------------
 rebase-interactive.h          |  4 ++-
 sequencer.c                   |  3 +--
 sequencer.h                   |  1 +
 5 files changed, 51 insertions(+), 28 deletions(-)
builtin/rebase--interactive.c +21 -1
index 0700339f90..264e940b47 100644
@@ -13,6 +13,26 @@ static GIT_PATH_FUNC(path_state_dir, "rebase-merge/")
 static GIT_PATH_FUNC(path_squash_onto, "rebase-merge/squash-onto")
 static GIT_PATH_FUNC(path_interactive, "rebase-merge/interactive")
 
+static int edit_todo_file(unsigned flags)
+{
+	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT,
+		new_todo = TODO_LIST_INIT;
+
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
+		return error_errno(_("could not read '%s'."), todo_file);
+
+	strbuf_stripspace(&todo_list.buf, 1);
+	if (!edit_todo_list(&todo_list, &new_todo, flags, 0, NULL, NULL) &&
+	    write_message(new_todo.buf.buf, new_todo.buf.len, todo_file, 0) < 0)
+		return error_errno(_("could not write '%s'"), todo_file);
+
+	todo_list_release(&todo_list);
+	todo_list_release(&new_todo);
+
+	return 0;
+}
+
 static int get_revision_ranges(const char *upstream, const char *onto,
 			       const char **head_hash,
 			       char **revisions, char **shortrevisions)
@@ -231,7 +251,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		break;
 	}
 	case EDIT_TODO:
-		ret = edit_todo_list(flags);
+		ret = edit_todo_file(flags);
 		break;
 	case SHOW_CURRENT_PATCH: {
 		struct child_process cmd = CHILD_PROCESS_INIT;
rebase-interactive.c +25 -24

64 lines not on this page · raw message

rebase-interactive.h +3 -1
index 61858f3a02..83cde455e6 100644
@@ -4,7 +4,9 @@
 void append_todo_help(unsigned keep_empty, int command_count,
 		      const char *shortrevisions, const char *shortonto,
 		      struct strbuf *buf);
-int edit_todo_list(unsigned flags);
+int edit_todo_list(struct todo_list *todo_list, struct todo_list *new_todo,
+		   unsigned flags, int command_count,
+		   const char *shortrevisions, const char *shortonto);
 int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
 
 #endif
sequencer.c +1 -2
index 94d3058359..bfcbe8239b 100644
@@ -55,8 +55,7 @@ static GIT_PATH_FUNC(rebase_path, "rebase-merge")
  * file and written to the tail of 'done'.
  */
 GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
-static GIT_PATH_FUNC(rebase_path_todo_backup,
-		     "rebase-merge/git-rebase-todo.backup")
+GIT_PATH_FUNC(rebase_path_todo_backup, "rebase-merge/git-rebase-todo.backup")
 
 /*
  * The rebase command lines that have already been processed. A line
sequencer.h +1 -0
index 5bd3b79282..fa84918c55 100644
@@ -9,6 +9,7 @@ struct commit;
 const char *git_path_commit_editmsg(void);
 const char *git_path_seq_dir(void);
 const char *rebase_path_todo(void);
+const char *rebase_path_todo_backup(void);
 
 #define APPEND_SIGNOFF_DEDUP (1u << 0)
 
Signature · 1 line
2.19.1
#25 lore raw

PATCH 13/15 sequencer: use edit_todo_list() in complete_action()

In reply to Alban Gruin

This changes complete_action() to use edit_todo_list(), now that it can handle the initial edit of the todo list.

---
 sequencer.c | 27 +++++++--------------------
 1 file changed, 7 insertions(+), 20 deletions(-)
sequencer.c +7 -20
index bfcbe8239b..93b9b40f66 100644
@@ -4608,7 +4608,7 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 	struct todo_list new_todo = TODO_LIST_INIT;
 	struct strbuf *buf = &todo_list->buf;
 	struct object_id oid;
-	int command_count;
+	int command_count, res;
 
 	get_oid(onto, &oid);
 	shortonto = find_unique_abbrev(&oid, DEFAULT_ABBREV);
@@ -4633,27 +4633,16 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 		return error(_("nothing to do"));
 	}
 
-	todo_list_transform(todo_list, flags | TODO_LIST_SHORTEN_IDS);
-
-	append_todo_help(flags & TODO_LIST_KEEP_EMPTY, command_count,
-			 shortrevisions, shortonto, buf);
-
-	if (write_message(buf->buf, buf->len, todo_file, 0))
-		return error_errno(_("could not write '%s'"), todo_file);
-
-	if (copy_file(rebase_path_todo_backup(), todo_file, 0666))
-		return error(_("could not copy '%s' to '%s'."), todo_file,
-			     rebase_path_todo_backup());
-
-	if (launch_sequence_editor(todo_file, &new_todo.buf, NULL)) {
+	res = edit_todo_list(todo_list, &new_todo, flags,
+			     command_count, shortrevisions, shortonto);
+	if (res == -1)
+		return -1;
+	else if (res == -2) {
 		apply_autostash(opts);
 		sequencer_remove_state(opts);
 
 		return -1;
-	}
-
-	strbuf_stripspace(&new_todo.buf, 1);
-	if (new_todo.buf.len == 0) {
+	} else if (res == -3) {
 		apply_autostash(opts);
 		sequencer_remove_state(opts);
 		todo_list_release(&new_todo);
@@ -4668,8 +4657,6 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 		return -1;
 	}
 
-	todo_list_transform(&new_todo, flags & ~(TODO_LIST_SHORTEN_IDS));
-
 	if (opts->allow_ff && skip_unnecessary_picks(&new_todo, &oid)) {
 		todo_list_release(&new_todo);
 		return error(_("could not skip unnecessary pick commands"));
Signature · 1 line
2.19.1
#26 lore raw

PATCH 14/15 sequencer: fix a call to error() in transform_todo_file()

In reply to Alban Gruin

This replaces a call to error() by a call to error_errno() after writing the content of the todo list to the disk in transform_todo_file().

---
 sequencer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
sequencer.c +1 -1
index 93b9b40f66..65bf251ba5 100644
@@ -4421,7 +4421,7 @@ int sequencer_add_exec_commands(const char *commands)
 	int res;
 
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
-		return error(_("could not read '%s'."), todo_file);
+		return error_errno(_("could not read '%s'."), todo_file);
 
 	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
 		todo_list_release(&todo_list);
Signature · 1 line
2.19.1
#27 lore raw

PATCH 15/15 rebase--interactive: move transform_todo_file() to rebase--interactive.c

In reply to Alban Gruin

As transform_todo_file() is only needed inside of rebase--interactive.c, it is moved there from sequencer.c.

---
 builtin/rebase--interactive.c | 21 +++++++++++++++++++++
 sequencer.c                   | 21 ---------------------
 sequencer.h                   |  1 -
 3 files changed, 21 insertions(+), 22 deletions(-)
builtin/rebase--interactive.c +21 -0
index 264e940b47..50b5c25402 100644
@@ -33,6 +33,27 @@ static int edit_todo_file(unsigned flags)
 	return 0;
 }
 
+static int transform_todo_file(unsigned flags)
+{
+	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res;
+
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
+		return error_errno(_("could not read '%s'."), todo_file);
+
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
+		todo_list_release(&todo_list);
+		return error(_("unusable todo list: '%s'"), todo_file);
+	}
+
+	todo_list_transform(&todo_list, flags);
+
+	res = write_message(todo_list.buf.buf, todo_list.buf.len, todo_file, 0);
+	todo_list_release(&todo_list);
+	return res;
+}
+
 static int get_revision_ranges(const char *upstream, const char *onto,
 			       const char **head_hash,
 			       char **revisions, char **shortrevisions)
sequencer.c +0 -21
index 65bf251ba5..e837e52b64 100644
@@ -4485,27 +4485,6 @@ void todo_list_transform(struct todo_list *todo_list, unsigned flags)
 		BUG("unusable todo list");
 }
 
-int transform_todo_file(unsigned flags)
-{
-	const char *todo_file = rebase_path_todo();
-	struct todo_list todo_list = TODO_LIST_INIT;
-	int res;
-
-	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
-		return error(_("could not read '%s'."), todo_file);
-
-	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
-		todo_list_release(&todo_list);
-		return error(_("unusable todo list: '%s'"), todo_file);
-	}
-
-	todo_list_transform(&todo_list, flags);
-
-	res = write_message(todo_list.buf.buf, todo_list.buf.len, todo_file, 0);
-	todo_list_release(&todo_list);
-	return res;
-}
-
 int check_todo_list_from_file(void)
 {
 	struct todo_list old_todo = TODO_LIST_INIT, new_todo = TODO_LIST_INIT;
sequencer.h +0 -1
index fa84918c55..a4b0113206 100644
@@ -137,7 +137,6 @@ int sequencer_make_script(struct strbuf *out, int argc, const char **argv,
 			  unsigned flags);
 
 int sequencer_add_exec_commands(const char *command);
-int transform_todo_file(unsigned flags);
 int check_todo_list_from_file(void);
 int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
Signature · 1 line
2.19.1
#28 lore raw
In reply to Alban Gruin

Le 07/10/2018 à 21:54, Alban Gruin a écrit :

Quoted 22 lines

At the center of the "interactive" part of the interactive rebase lies the todo list. When the user starts an interactive rebase, a todo list is generated, presented to the user (who then edits it using a text editor), read back, and then is checked and processed before the actual rebase takes place.

Some of this processing includes adding execs commands, reordering fixup! and squash! commits, and checking if no commits were accidentally dropped by the user.

Before I converted the interactive rebase in C, these functions were called by git-rebase--interactive.sh through git-rebase--helper. Since the only way to pass around a large amount of data between a shell script and a C program is to use a file (or any declination of a file), the functions that checked and processed the todo list were directly working on a file, the same file that the user edited.

During the conversion, I did not address this issue, which lead to a complete_action() that reads the todo list file, does some computation based on its content, and writes it back to the disk, several times in the same function.

Quoted 8 lines

As it is not an efficient way to handle a data structure, this patch series refactor the functions that processes the todo list to work on a todo_list structure instead of reading it from the disk.

Some commits consists in modifying edit_todo_list() (initially used by --edit-todo) to handle the initial edition of the todo list, to increase code sharing.

And it’s based on the 8th version of my patch series “rebase -i: rewrite in C”.

#29 lore raw

v2 cover sequencer: refactor functions working on a todo_list

In reply to Alban Gruin

At the center of the "interactive" part of the interactive rebase lies the todo list. When the user starts an interactive rebase, a todo list is generated, presented to the user (who then edits it using a text editor), read back, and then is checked and processed before the actual rebase takes place.

Some of this processing includes adding execs commands, reordering fixup! and squash! commits, and checking if no commits were accidentally dropped by the user.

Before I converted the interactive rebase in C, these functions were called by git-rebase--interactive.sh through git-rebase--helper. Since the only way to pass around a large amount of data between a shell script and a C program is to use a file (or any declination of a file), the functions that checked and processed the todo list were directly working on a file, the same file that the user edited.

During the conversion, I did not address this issue, which lead to a complete_action() that reads the todo list file, does some computation based on its content, and writes it back to the disk, several times in the same function.

As it is not an efficient way to handle a data structure, this patch series refactor the functions that processes the todo list to work on a todo_list structure instead of reading it from the disk.

Some commits consists in modifying edit_todo_list() (initially used by --edit-todo) to handle the initial edition of the todo list, to increase code sharing.

It is based onto ag/rebase-i-in-c (34b4731, "rebase -i: move rebase--helper modes to rebase--interactive").

Changes since v1:

- When a line is invalid, parse_insn_buffer() sets the type of the corresponding command to a garbage value instead of `noop', and its argument is defined properly.

- todo_list_add_exec_commands(), todo_list_rearrange_squash(), skip_unnecessary_picks() don’t reparse the todo list after processing them. Instead, they recreate a new item list.

- Due to the previous change, a todo list buffer can’t directly be written to the disk. A new function, todo_list_write_to_disk(), is introduced to take care of this task.

- rewrite_file() has been deleted.

- A call to strbuf_addf(&buf, "\n"); has been replaced by strbuf_addch(…).

- complete_action() and todo_list_check() expect that their input todo list have already been parsed.

- complete_action() no longer writes "noop\n" to the todo list buffer if it is empty. Instead, it appends a `noop' command to the item list.

Alban Gruin (16):
sequencer: changes in parse_insn_buffer()
sequencer: make the todo_list structure public
sequencer: refactor transform_todos() to work on a todo_list
sequencer: introduce todo_list_write_to_file()
sequencer: refactor check_todo_list() to work on a todo_list
sequencer: refactor sequencer_add_exec_commands() to work on a
todo_list
sequencer: refactor rearrange_squash() to work on a todo_list
sequencer: make sequencer_make_script() write its script to a strbuf
sequencer: change complete_action() to use the refactored functions
sequencer: refactor skip_unnecessary_picks() to work on a todo_list
rebase-interactive: use todo_list_write_to_file() in edit_todo_list()
rebase-interactive: append_todo_help() changes
rebase-interactive: rewrite edit_todo_list() to handle the initial
edit
sequencer: use edit_todo_list() in complete_action()
sequencer: fix a call to error() in transform_todo_file()
rebase--interactive: move transform_todo_file() to
rebase--interactive.c

 builtin/rebase--interactive.c |  68 +++-
 rebase-interactive.c          | 142 ++++++--
 rebase-interactive.h          |   8 +-
 sequencer.c                   | 589 +++++++++++++---------------------
 sequencer.h                   |  68 +++-
 5 files changed, 455 insertions(+), 420 deletions(-)
Signature · 1 line
2.19.1
#30 lore raw

PATCH v2 1/16 sequencer: changes in parse_insn_buffer()

This clears the number of items of a todo_list before parsing it to allow to parse the same list multiple times without issues. As its items are not dynamically allocated, or don’t need to allocate memory, no additionnal memory management is required here.

Furthermore, if a line is invalid, the type of the corresponding command is set to a garbage value, and its argument is defined properly. This will allow to recreate the text of a todo list from its commands, even if one of them is incorrect.

---
 sequencer.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
sequencer.c +6 -1
index 8dd6db5a01..9c8bd3f632 100644
@@ -2023,6 +2023,8 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 	char *p = buf, *next_p;
 	int i, res = 0, fixup_okay = file_exists(rebase_path_done());
 
+	todo_list->current = todo_list->nr = 0;
+
 	for (i = 1; *p; i++, p = next_p) {
 		char *eol = strchrnul(p, '\n');
 
@@ -2036,7 +2038,10 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		if (parse_insn_line(item, p, eol)) {
 			res = error(_("invalid line %d: %.*s"),
 				i, (int)(eol - p), p);
-			item->command = TODO_NOOP;
+			item->command = TODO_COMMENT + 1;
+			item->arg = p;
+			item->arg_len = (int)(eol - p);
+			item->commit = NULL;
 		}
 
 		if (fixup_okay)
Signature · 1 line
2.19.1
#31 lore raw

PATCH v2 2/16 sequencer: make the todo_list structure public

In reply to Alban Gruin

This makes the structures todo_list and todo_item, and the functions todo_list_release() and parse_insn_buffer(), accessible outside of sequencer.c.

---
No changes since v1.

 sequencer.c | 66 +++++++++--------------------------------------------
 sequencer.h | 48 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 59 insertions(+), 55 deletions(-)
sequencer.c +11 -55
index 9c8bd3f632..f791729271 100644
@@ -1401,31 +1401,6 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 		return 1;
 }
 
-/*
- * Note that ordering matters in this enum. Not only must it match the mapping
- * below, it is also divided into several sections that matter.  When adding
- * new commands, make sure you add it in the right section.
- */
-enum todo_command {
-	/* commands that handle commits */
-	TODO_PICK = 0,
-	TODO_REVERT,
-	TODO_EDIT,
-	TODO_REWORD,
-	TODO_FIXUP,
-	TODO_SQUASH,
-	/* commands that do something else than handling a single commit */
-	TODO_EXEC,
-	TODO_LABEL,
-	TODO_RESET,
-	TODO_MERGE,
-	/* commands that do nothing but are counted for reporting progress */
-	TODO_NOOP,
-	TODO_DROP,
-	/* comments (not counted for reporting progress) */
-	TODO_COMMENT
-};
-
 static struct {
 	char c;
 	const char *str;
@@ -1897,26 +1872,7 @@ enum todo_item_flags {
 	TODO_EDIT_MERGE_MSG = 1
 };
 
-struct todo_item {
-	enum todo_command command;
-	struct commit *commit;
-	unsigned int flags;
-	const char *arg;
-	int arg_len;
-	size_t offset_in_buf;
-};
-
-struct todo_list {
-	struct strbuf buf;
-	struct todo_item *items;
-	int nr, alloc, current;
-	int done_nr, total_nr;
-	struct stat_data stat;
-};
-
-#define TODO_LIST_INIT { STRBUF_INIT }
-
-static void todo_list_release(struct todo_list *todo_list)
+void todo_list_release(struct todo_list *todo_list)
 {
 	strbuf_release(&todo_list->buf);
 	FREE_AND_NULL(todo_list->items);
@@ -2017,7 +1973,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	return !item->commit;
 }
 
-static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
+int todo_list_parse_insn_buffer(char *buf, struct todo_list *todo_list)
 {
 	struct todo_item *item;
 	char *p = buf, *next_p;
@@ -2115,7 +2071,7 @@ static int read_populate_todo(struct todo_list *todo_list,
 		return error(_("could not stat '%s'"), todo_file);
 	fill_stat_data(&todo_list->stat, &st);
 
-	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
+	res = todo_list_parse_insn_buffer(todo_list->buf.buf, todo_list);
 	if (res) {
 		if (is_rebase_i(opts))
 			return error(_("please fix this using "
@@ -2146,7 +2102,7 @@ static int read_populate_todo(struct todo_list *todo_list,
 		FILE *f = fopen_or_warn(rebase_path_msgtotal(), "w");
 
 		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
-				!parse_insn_buffer(done.buf.buf, &done))
+				!todo_list_parse_insn_buffer(done.buf.buf, &done))
 			todo_list->done_nr = count_commands(&done);
 		else
 			todo_list->done_nr = 0;
@@ -4276,7 +4232,7 @@ int sequencer_add_exec_commands(const char *commands)
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4311,7 +4267,7 @@ int transform_todos(unsigned flags)
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4397,7 +4353,7 @@ int check_todo_list(void)
 		goto leave_check;
 	}
 	advise_to_edit_todo = res =
-		parse_insn_buffer(todo_list.buf.buf, &todo_list);
+		todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list);
 
 	if (res || check_level == MISSING_COMMIT_CHECK_IGNORE)
 		goto leave_check;
@@ -4416,7 +4372,7 @@ int check_todo_list(void)
 		goto leave_check;
 	}
 	strbuf_release(&todo_file);
-	res = !!parse_insn_buffer(todo_list.buf.buf, &todo_list);
+	res = !!todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list);
 
 	/* Find commits in git-rebase-todo.backup yet unseen */
 	for (i = todo_list.nr - 1; i >= 0; i--) {
@@ -4498,7 +4454,7 @@ static int skip_unnecessary_picks(struct object_id *output_oid)
 
 	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
 		return -1;
-	if (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {
 		todo_list_release(&todo_list);
 		return -1;
 	}
@@ -4586,7 +4542,7 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 	if (strbuf_read_file(buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(buf->buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(buf->buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4694,7 +4650,7 @@ int rearrange_squash(void)
 
 	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
 		return -1;
-	if (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {
 		todo_list_release(&todo_list);
 		return -1;
 	}
sequencer.h +48 -0

54 lines not on this page · raw message

Signature · 1 line
2.19.1
#32 lore raw

PATCH v2 3/16 sequencer: refactor transform_todos() to work on a todo_list

In reply to Alban Gruin

This refactors transform_todos() to work on a todo_list. The function is renamed todo_list_transform().

As rebase -p still need to check the todo list from the disk, a new function is introduced, transform_todo_file(). It is still used by complete_action() and edit_todo_list() for now, but they will be replaced in a future commit.

todo_list_transform() is not a static function, because it will be used by edit_todo_list() from rebase-interactive.c in a future commit.

---
 builtin/rebase--interactive.c |  2 +-
 rebase-interactive.c          |  4 +--
 sequencer.c                   | 46 +++++++++++++++++++++++------------
 sequencer.h                   |  3 ++-
 4 files changed, 35 insertions(+), 20 deletions(-)
builtin/rebase--interactive.c +1 -1
index a2ab68ed06..abdf6126df 100644
@@ -252,7 +252,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 	}
 	case SHORTEN_OIDS:
 	case EXPAND_OIDS:
-		ret = transform_todos(flags);
+		ret = transform_todo_file(flags);
 		break;
 	case CHECK_TODO_LIST:
 		ret = check_todo_list();
rebase-interactive.c +2 -2
index 0f4119cbae..49f2f549e1 100644
@@ -68,7 +68,7 @@ int edit_todo_list(unsigned flags)
 
 	strbuf_release(&buf);
 
-	transform_todos(flags | TODO_LIST_SHORTEN_IDS);
+	transform_todo_file(flags | TODO_LIST_SHORTEN_IDS);
 
 	if (strbuf_read_file(&buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
@@ -84,7 +84,7 @@ int edit_todo_list(unsigned flags)
 	if (launch_sequence_editor(todo_file, NULL, NULL))
 		return -1;
 
-	transform_todos(flags & ~(TODO_LIST_SHORTEN_IDS));
+	transform_todo_file(flags & ~(TODO_LIST_SHORTEN_IDS));
 
 	return 0;
 }
sequencer.c +30 -16

76 lines not on this page · raw message

sequencer.h +2 -1
index cc69473bc2..029d842ac5 100644
@@ -113,6 +113,7 @@ struct todo_list {
 #define TODO_LIST_INIT { STRBUF_INIT }
 
 int todo_list_parse_insn_buffer(char *buf, struct todo_list *todo_list);
+void todo_list_transform(struct todo_list *todo_list, unsigned flags);
 void todo_list_release(struct todo_list *todo_list);
 
 /* Call this to setup defaults before parsing command line options */
@@ -136,7 +137,7 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
 int sequencer_add_exec_commands(const char *command);
-int transform_todos(unsigned flags);
+int transform_todo_file(unsigned flags);
 enum missing_commit_check_level get_missing_commit_check_level(void);
 int check_todo_list(void);
 int complete_action(struct replay_opts *opts, unsigned flags,
Signature · 1 line
2.19.1
#33 lore raw

PATCH v2 4/16 sequencer: introduce todo_list_write_to_file()

In reply to Alban Gruin

This introduce a new function to recreate the text of a todo list from its commands, and then to write it to the disk. This will be useful in the future, the buffer of a todo list won’t be treated as a strict mirror of the todo file by some of its functions once they will be refactored.

This functionnality can already be found in todo_list_transform(), but it is specifically made to replace the buffer of a todo list, which is not the desired behaviour. Thus, the part of todo_list_transform() that actually creates the buffer is moved to a new function, todo_list_to_strbuf(). The rest is unused, and so is dropped.

todo_list_write_to_file() can also take care to append the help text to the buffer before writing it to the disk, or to write only the first n items of the list.

---
 sequencer.c | 59 ++++++++++++++++++++++++++++++++++++-----------------
 sequencer.h |  4 +++-
 2 files changed, 43 insertions(+), 20 deletions(-)
sequencer.c +40 -19
index 07296f1f57..0dd45677b1 100644
@@ -4256,24 +4256,27 @@ int sequencer_add_exec_commands(const char *commands)
 	return i;
 }
 
-void todo_list_transform(struct todo_list *todo_list, unsigned flags)
+static void todo_list_to_strbuf(struct todo_list *todo_list, struct strbuf *buf,
+				int num, unsigned flags)
 {
-	struct strbuf buf = STRBUF_INIT;
 	struct todo_item *item;
-	int i;
+	int i, max = todo_list->nr;
 
-	for (item = todo_list->items, i = 0; i < todo_list->nr; i++, item++) {
+	if (num > 0 && num < max)
+		max = num;
+
+	for (item = todo_list->items, i = 0; i < max; i++, item++) {
 		/* if the item is not a command write it and continue */
 		if (item->command >= TODO_COMMENT) {
-			strbuf_addf(&buf, "%.*s\n", item->arg_len, item->arg);
+			strbuf_addf(buf, "%.*s\n", item->arg_len, item->arg);
 			continue;
 		}
 
 		/* add command to the buffer */
 		if (flags & TODO_LIST_ABBREVIATE_CMDS)
-			strbuf_addch(&buf, command_to_char(item->command));
+			strbuf_addch(buf, command_to_char(item->command));
 		else
-			strbuf_addstr(&buf, command_to_string(item->command));
+			strbuf_addstr(buf, command_to_string(item->command));
 
 		/* add commit id */
 		if (item->commit) {
@@ -4283,27 +4286,46 @@ void todo_list_transform(struct todo_list *todo_list, unsigned flags)
 
 			if (item->command == TODO_MERGE) {
 				if (item->flags & TODO_EDIT_MERGE_MSG)
-					strbuf_addstr(&buf, " -c");
+					strbuf_addstr(buf, " -c");
 				else
-					strbuf_addstr(&buf, " -C");
+					strbuf_addstr(buf, " -C");
 			}
 
-			strbuf_addf(&buf, " %s", oid);
+			strbuf_addf(buf, " %s", oid);
 		}
 
 		/* add all the rest */
 		if (!item->arg_len)
-			strbuf_addch(&buf, '\n');
+			strbuf_addch(buf, '\n');
 		else
-			strbuf_addf(&buf, " %.*s\n", item->arg_len, item->arg);
+			strbuf_addf(buf, " %.*s\n", item->arg_len, item->arg);
 	}
+}
 
-	strbuf_reset(&todo_list->buf);
-	strbuf_add(&todo_list->buf, buf.buf, buf.len);
+int todo_list_write_to_file(struct todo_list *todo_list, const char *file,
+			    const char *shortrevisions, const char *shortonto,
+			    int command_count, int append_help, int num, unsigned flags)
+{
+	int edit_todo = !(shortrevisions && shortonto), res;
+	struct strbuf buf = STRBUF_INIT;
+
+	todo_list_to_strbuf(todo_list, &buf, num, flags);
+
+	if (append_help) {
+		if (!edit_todo) {
+			strbuf_addch(&buf, '\n');
+			strbuf_commented_addf(&buf, Q_("Rebase %s onto %s (%d command)",
+						       "Rebase %s onto %s (%d commands)",
+						       command_count),
+					      shortrevisions, shortonto, command_count);
+		}
+		append_todo_help(edit_todo, flags & TODO_LIST_KEEP_EMPTY, &buf);
+	}
+
+	res = write_message(buf.buf, buf.len, file, 0);
 	strbuf_release(&buf);
 
-	if (todo_list_parse_insn_buffer(todo_list->buf.buf, todo_list))
-		BUG("unusable todo list");
+	return res;
 }
 
 int transform_todo_file(unsigned flags)
@@ -4320,9 +4342,8 @@ int transform_todo_file(unsigned flags)
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
 
-	todo_list_transform(&todo_list, flags);
-
-	res = write_message(todo_list.buf.buf, todo_list.buf.len, todo_file, 0);
+	res = todo_list_write_to_file(&todo_list, todo_file,
+				      NULL, NULL, 0, 0, -1, flags);
 	todo_list_release(&todo_list);
 	return res;
 }
sequencer.h +3 -1
index 029d842ac5..a299c977fe 100644
@@ -113,7 +113,9 @@ struct todo_list {
 #define TODO_LIST_INIT { STRBUF_INIT }
 
 int todo_list_parse_insn_buffer(char *buf, struct todo_list *todo_list);
-void todo_list_transform(struct todo_list *todo_list, unsigned flags);
+int todo_list_write_to_file(struct todo_list *todo_list, const char *file,
+			    const char *shortrevisions, const char *shortonto,
+			    int command_count, int append_help, int num, unsigned flags);
 void todo_list_release(struct todo_list *todo_list);
 
 /* Call this to setup defaults before parsing command line options */
Signature · 1 line
2.19.1
#34 lore raw

Re: [PATCH v2 04/16] sequencer: introduce todo_list_write_to_file()

Hi Alban

I like the direction this is going, it is an improvement on re-scanning the list at the end of each function.

Quoted 5 lines · Alban Gruin

This introduce a new function to recreate the text of a todo list from its commands, and then to write it to the disk. This will be useful in the future, the buffer of a todo list won’t be treated as a strict mirror of the todo file by some of its functions once they will be refactored.

I'd suggest rewording this slightly, maybe something like

This introduces a new function to recreate the text of a todo list from
its commands and write it to a file. This will be useful as the next few commits will change the use of the buffer in struct todo_list so it will no-longer be a mirror of the file on disk.

Quoted 1 line

This functionnality can already be found in todo_list_transform(), but

s/functionnality/functionality/

Quoted 6 lines

it is specifically made to replace the buffer of a todo list, which is not the desired behaviour. Thus, the part of todo_list_transform() that actually creates the buffer is moved to a new function, todo_list_to_strbuf(). The rest is unused, and so is dropped.

todo_list_write_to_file() can also take care to append the help text to

s/care to append/care of appending/

Quoted 2 lines

the buffer before writing it to the disk, or to write only the first n items of the list.

Why/when do we only want to write a subset of the items?

Quoted 65 lines
---
  sequencer.c | 59 ++++++++++++++++++++++++++++++++++++-----------------
  sequencer.h |  4 +++-
  2 files changed, 43 insertions(+), 20 deletions(-)
sequencer.c +14 -11
index 07296f1f57..0dd45677b1 100644
@@ -4256,24 +4256,27 @@ int sequencer_add_exec_commands(const char *commands)
  	return i;
  }
 
-void todo_list_transform(struct todo_list *todo_list, unsigned flags)
+static void todo_list_to_strbuf(struct todo_list *todo_list, struct strbuf *buf,
+				int num, unsigned flags)
  {
-	struct strbuf buf = STRBUF_INIT;
  	struct todo_item *item;
-	int i;
+	int i, max = todo_list->nr;
 
-	for (item = todo_list->items, i = 0; i < todo_list->nr; i++, item++) {
+	if (num > 0 && num < max)
+		max = num;
+
+	for (item = todo_list->items, i = 0; i < max; i++, item++) {
  		/* if the item is not a command write it and continue */
  		if (item->command >= TODO_COMMENT) {
-			strbuf_addf(&buf, "%.*s\n", item->arg_len, item->arg);
+			strbuf_addf(buf, "%.*s\n", item->arg_len, item->arg);
  			continue;
  		}
 
  		/* add command to the buffer */
  		if (flags & TODO_LIST_ABBREVIATE_CMDS)
-			strbuf_addch(&buf, command_to_char(item->command));
+			strbuf_addch(buf, command_to_char(item->command));
  		else
-			strbuf_addstr(&buf, command_to_string(item->command));
+			strbuf_addstr(buf, command_to_string(item->command));
 
  		/* add commit id */
  		if (item->commit) {
@@ -4283,27 +4286,46 @@ void todo_list_transform(struct todo_list *todo_list, unsigned flags)
 
  			if (item->command == TODO_MERGE) {
  				if (item->flags & TODO_EDIT_MERGE_MSG)
-					strbuf_addstr(&buf, " -c");
+					strbuf_addstr(buf, " -c");
  				else
-					strbuf_addstr(&buf, " -C");
+					strbuf_addstr(buf, " -C");
  			}
 
-			strbuf_addf(&buf, " %s", oid);
+			strbuf_addf(buf, " %s", oid);
  		}
 
  		/* add all the rest */
  		if (!item->arg_len)
-			strbuf_addch(&buf, '\n');
+			strbuf_addch(buf, '\n');
  		else

sequencer.c void todo_list_transform(struct todo_list *todo_list, unsigned flags) · line 4307

Quoted 10 lines
sequencer.c +5 -3
-			strbuf_addf(&buf, " %.*s\n", item->arg_len, item->arg);
+			strbuf_addf(buf, " %.*s\n", item->arg_len, item->arg);
  	}
+}
 
-	strbuf_reset(&todo_list->buf);
-	strbuf_add(&todo_list->buf, buf.buf, buf.len);
+int todo_list_write_to_file(struct todo_list *todo_list, const char *file,
+			    const char *shortrevisions, const char *shortonto,
+			    int command_count, int append_help, int num, unsigned flags)

This is a really long argument list which makes it easy for callers to get the parameters in the wrong order. I think append_help could probably be folded into the flags, I'm not sure what the command_count is used for but I've only read the first few patches. Maybe it would be better to pass a struct so we have named fields.

Best Wishes

Phillip

Quoted 54 lines
sequencer.c +21 -5
+{
+	int edit_todo = !(shortrevisions && shortonto), res;
+	struct strbuf buf = STRBUF_INIT;
+
+	todo_list_to_strbuf(todo_list, &buf, num, flags);
+
+	if (append_help) {
+		if (!edit_todo) {
+			strbuf_addch(&buf, '\n');
+			strbuf_commented_addf(&buf, Q_("Rebase %s onto %s (%d command)",
+						       "Rebase %s onto %s (%d commands)",
+						       command_count),
+					      shortrevisions, shortonto, command_count);
+		}
+		append_todo_help(edit_todo, flags & TODO_LIST_KEEP_EMPTY, &buf);
+	}
+
+	res = write_message(buf.buf, buf.len, file, 0);
  	strbuf_release(&buf);
 
-	if (todo_list_parse_insn_buffer(todo_list->buf.buf, todo_list))
-		BUG("unusable todo list");
+	return res;
  }
 
  int transform_todo_file(unsigned flags)
@@ -4320,9 +4342,8 @@ int transform_todo_file(unsigned flags)
  		return error(_("unusable todo list: '%s'"), todo_file);
  	}
 
-	todo_list_transform(&todo_list, flags);
-
-	res = write_message(todo_list.buf.buf, todo_list.buf.len, todo_file, 0);
+	res = todo_list_write_to_file(&todo_list, todo_file,
+				      NULL, NULL, 0, 0, -1, flags);
  	todo_list_release(&todo_list);
  	return res;
  }
sequencer.h +3 -1
index 029d842ac5..a299c977fe 100644
@@ -113,7 +113,9 @@ struct todo_list {
  #define TODO_LIST_INIT { STRBUF_INIT }
 
  int todo_list_parse_insn_buffer(char *buf, struct todo_list *todo_list);
-void todo_list_transform(struct todo_list *todo_list, unsigned flags);
+int todo_list_write_to_file(struct todo_list *todo_list, const char *file,
+			    const char *shortrevisions, const char *shortonto,
+			    int command_count, int append_help, int num, unsigned flags);
  void todo_list_release(struct todo_list *todo_list);
 
  /* Call this to setup defaults before parsing command line options */
#35 lore raw

Re: [PATCH v2 04/16] sequencer: introduce todo_list_write_to_file()

Hi Phillip,

Le 30/10/2018 à 17:28, Phillip Wood a écrit :

Quoted 28 lines

Hi Alban

I like the direction this is going, it is an improvement on re-scanning the list at the end of each function.

Quoted 5 lines · Alban Gruin

This introduce a new function to recreate the text of a todo list from its commands, and then to write it to the disk.  This will be useful in the future, the buffer of a todo list won’t be treated as a strict mirror of the todo file by some of its functions once they will be refactored.

I'd suggest rewording this slightly, maybe something like

This introduces a new function to recreate the text of a todo list from its commands and write it to a file. This will be useful as the next few commits will change the use of the buffer in struct todo_list so it will no-longer be a mirror of the file on disk.

Quoted 1 line

This functionnality can already be found in todo_list_transform(), but

s/functionnality/functionality/

Quoted 5 lines

it is specifically made to replace the buffer of a todo list, which is not the desired behaviour.  Thus, the part of todo_list_transform() that actually creates the buffer is moved to a new function, todo_list_to_strbuf().  The rest is unused, and so is dropped.

Quoted 9 lines
Quoted 1 line

todo_list_write_to_file() can also take care to append the help text to

s/care to append/care of appending/

Quoted 2 lines

the buffer before writing it to the disk, or to write only the first n items of the list.

Why/when do we only want to write a subset of the items?

In skip_unnecessary_picks(), in patch [10/16]. It needs to write the elements of the todo list that were already done in the `done' file.

Quoted 7 lines

[…]

Quoted 5 lines

+int todo_list_write_to_file(struct todo_list *todo_list, const char *file,
+                const char *shortrevisions, const char *shortonto, +                int command_count, int append_help, int num, unsigned flags)

Quoted 6 lines

This is a really long argument list which makes it easy for callers to get the parameters in the wrong order. I think append_help could probably be folded into the flags, I'm not sure what the command_count is used for but I've only read the first few patches. Maybe it would be better to pass a struct so we have named fields.

You’re right, command_count is not really needed since we pass the complete todo list.

The only bit that irks me is that, if I stop passing command_count, I would have to call count_commands() twice in complete_action(): once to check if there are any commands in the todo list, and again inside of todo_list_write_to_file() (see [09/16].)

Perhaps I could move this check before calling todo_list_rearrange_squash()?

As a sidenote, this is not why I added command_count to the parameters of todo_list_write_to_file(). It was a confusion of my part.

Quoted 4 lines

Best Wishes

Phillip

Cheers,
Alban

#36 lore raw

PATCH v2 5/16 sequencer: refactor check_todo_list() to work on a todo_list

In reply to Alban Gruin

This refactors check_todo_list() to work on a todo_list to avoid redundant reads and writes to the disk. The function is renamed todo_list_check(). The parsing of the two todo lists is left to the caller.

As rebase -p still need to check the todo list from the disk, a new function is introduced, check_todo_list_from_file(). It reads the file from the disk, parses it, pass the todo_list to todo_list_check(), and writes it back to the disk.

As get_missing_commit_check_level() and the enum missing_commit_check_level are no longer needed inside of sequencer.c, they are moved to rebase-interactive.c, and made static again.

---
 builtin/rebase--interactive.c |   2 +-
 rebase-interactive.c          |  90 ++++++++++++++++++++++++-
 rebase-interactive.h          |   1 +
 sequencer.c                   | 120 +++++++---------------------------
 sequencer.h                   |   9 +--
 5 files changed, 115 insertions(+), 107 deletions(-)
builtin/rebase--interactive.c +1 -1
index abdf6126df..c1d87c0fe6 100644
@@ -255,7 +255,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = transform_todo_file(flags);
 		break;
 	case CHECK_TODO_LIST:
-		ret = check_todo_list();
+		ret = check_todo_list_from_file();
 		break;
 	case REARRANGE_SQUASH:
 		ret = rearrange_squash();
rebase-interactive.c +89 -1

100 lines not on this page · raw message

rebase-interactive.h +1 -0
index 971da03776..6bc7bc315d 100644
@@ -4,5 +4,6 @@
 void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 		      struct strbuf *buf);
 int edit_todo_list(unsigned flags);
+int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
 
 #endif
sequencer.c +23 -97

141 lines not on this page · raw message

sequencer.h +1 -8
index a299c977fe..e203930084 100644
@@ -58,12 +58,6 @@ struct replay_opts {
 };
 #define REPLAY_OPTS_INIT { .action = -1, .current_fixups = STRBUF_INIT }
 
-enum missing_commit_check_level {
-	MISSING_COMMIT_CHECK_IGNORE = 0,
-	MISSING_COMMIT_CHECK_WARN,
-	MISSING_COMMIT_CHECK_ERROR
-};
-
 int write_message(const void *buf, size_t len, const char *filename,
 		  int append_eol);
 
@@ -140,8 +134,7 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 
 int sequencer_add_exec_commands(const char *command);
 int transform_todo_file(unsigned flags);
-enum missing_commit_check_level get_missing_commit_check_level(void);
-int check_todo_list(void);
+int check_todo_list_from_file(void);
 int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, const char *cmd,
Signature · 1 line
2.19.1
#37 lore raw

PATCH v2 6/16 sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

In reply to Alban Gruin

This refactors sequencer_add_exec_commands() to work on a todo_list to avoid redundant reads and writes to the disk.

An obvious way to do this would be to insert the `exec' command between the other commands, and reparse it once this is done. This is not what is done here. Instead, the command is appended to the buffer once, and a new list of items is created. Items from the old list are copied to it, and new `exec' items are appended when necessary. This eliminates the need to reparse the todo list, but this also means its buffer cannot be directly written to the disk, hence todo_list_write_to_disk().

sequencer_add_exec_commands() still reads the todo list from the disk, as it is needed by rebase -p. todo_list_add_exec_commands() works on a todo_list structure, and reparses it at the end.

complete_action() still uses sequencer_add_exec_commands() for now. This will be changed in a future commit.

---
 sequencer.c | 69 +++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 49 insertions(+), 20 deletions(-)
sequencer.c +49 -20
index e12860c047..12a3efeca8 100644
@@ -4216,6 +4216,50 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	return 0;
 }
 
+static void todo_list_add_exec_commands(struct todo_list *todo_list,
+					const char *commands)
+{
+	struct strbuf *buf = &todo_list->buf;
+	const char *old_buf = buf->buf;
+	size_t commands_len = strlen(commands + strlen("exec ")) - 1;
+	int i, first = 1, nr = 0, alloc = 0;
+	struct todo_item *items = NULL,
+		base_item = {TODO_EXEC, NULL, 0, 0, commands_len, 0};
+
+	strbuf_addstr(buf, commands);
+	base_item.offset_in_buf = buf->len - commands_len - 1;
+	base_item.arg = buf->buf + base_item.offset_in_buf;
+
+	/*
+	 * Insert <commands> after every pick. Here, fixup/squash chains
+	 * are considered part of the pick, so we insert the commands *after*
+	 * those chains if there are any.
+	 */
+	for (i = 0; i < todo_list->nr; i++) {
+		enum todo_command command = todo_list->items[i].command;
+		if (todo_list->items[i].arg)
+			todo_list->items[i].arg = todo_list->items[i].arg - old_buf + buf->buf;
+
+		if (command == TODO_PICK && !first) {
+			ALLOC_GROW(items, nr + 1, alloc);
+			memcpy(items + nr++, &base_item, sizeof(struct todo_item));
+		}
+
+		ALLOC_GROW(items, nr + 1, alloc);
+		memcpy(items + nr++, todo_list->items + i, sizeof(struct todo_item));
+		first = 0;
+	}
+
+	/* insert or append final <commands> */
+	ALLOC_GROW(items, nr + 1, alloc);
+	memcpy(items + nr++, &base_item, sizeof(struct todo_item));
+
+	FREE_AND_NULL(todo_list->items);
+	todo_list->items = items;
+	todo_list->nr = nr;
+	todo_list->alloc = alloc;
+}
+
 /*
  * Add commands after pick and (series of) squash/fixup commands
  * in the todo list.
@@ -4224,10 +4268,7 @@ int sequencer_add_exec_commands(const char *commands)
 {
 	const char *todo_file = rebase_path_todo();
 	struct todo_list todo_list = TODO_LIST_INIT;
-	struct todo_item *item;
-	struct strbuf *buf = &todo_list.buf;
-	size_t offset = 0, commands_len = strlen(commands);
-	int i, first;
+	int res;
 
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error(_("could not read '%s'."), todo_file);
@@ -4237,23 +4278,11 @@ int sequencer_add_exec_commands(const char *commands)
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
 
-	first = 1;
-	/* insert <commands> before every pick except the first one */
-	for (item = todo_list.items, i = 0; i < todo_list.nr; i++, item++) {
-		if (item->command == TODO_PICK && !first) {
-			strbuf_insert(buf, item->offset_in_buf + offset,
-				      commands, commands_len);
-			offset += commands_len;
-		}
-		first = 0;
-	}
-
-	/* append final <commands> */
-	strbuf_add(buf, commands, commands_len);
-
-	i = write_message(buf->buf, buf->len, todo_file, 0);
+	todo_list_add_exec_commands(&todo_list, commands);
+	res = todo_list_write_to_file(&todo_list, todo_file, NULL, NULL, 0, 0, -1, 0);
 	todo_list_release(&todo_list);
-	return i;
+
+	return res;
 }
 
 static void todo_list_to_strbuf(struct todo_list *todo_list, struct strbuf *buf,
Signature · 1 line
2.19.1
#38 lore raw

Re: [PATCH v2 06/16] sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

Quoted 10 lines · Alban Gruin

This refactors sequencer_add_exec_commands() to work on a todo_list to avoid redundant reads and writes to the disk.

An obvious way to do this would be to insert the `exec' command between the other commands, and reparse it once this is done. This is not what is done here. Instead, the command is appended to the buffer once, and a new list of items is created. Items from the old list are copied to it, and new `exec' items are appended when necessary. This eliminates the need to reparse the todo list, but this also means its buffer cannot be directly written to the disk, hence todo_list_write_to_disk().

I'd reword this slightly, maybe

Instead of just inserting the `exec' command between the other commands, and re-parsing the buffer at the end the exec command is appended to the buffer once, and a new list of items is created. Items from the old list are copied across and new `exec' items are appended when necessary.
This eliminates the need to reparse the buffer, but this also means we
have to use todo_list_write_to_disk() to write the file.

Quoted 3 lines

sequencer_add_exec_commands() still reads the todo list from the disk, as it is needed by rebase -p. todo_list_add_exec_commands() works on a todo_list structure, and reparses it at the end.

I think the saying 'reparses' is confusing as that is what we're trying to avoid.

Quoted 13 lines

complete_action() still uses sequencer_add_exec_commands() for now. This will be changed in a future commit.

---
  sequencer.c | 69 +++++++++++++++++++++++++++++++++++++----------------
  1 file changed, 49 insertions(+), 20 deletions(-)
sequencer.c +0 -0
index e12860c047..12a3efeca8 100644

sequencer.c int sequencer_make_script(FILE *out, int argc, const char **argv, · line 4225

Quoted 10 lines
sequencer.c +7 -0
@@ -4216,6 +4216,50 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
  	return 0;
  }
 
+static void todo_list_add_exec_commands(struct todo_list *todo_list,
+					const char *commands)
+{
+	struct strbuf *buf = &todo_list->buf;
+	const char *old_buf = buf->buf;
+	size_t commands_len = strlen(commands + strlen("exec ")) - 1;
+	int i, first = 1, nr = 0, alloc = 0;

Minor nit pick, I think it is clearer if first is initialized just before the loop as it is in the deleted code below.

sequencer.c int sequencer_make_script(FILE *out, int argc, const char **argv, · line 4231

Quoted 6 lines
sequencer.c +6 -0
+	struct todo_item *items = NULL,
+		base_item = {TODO_EXEC, NULL, 0, 0, commands_len, 0};
+
+	strbuf_addstr(buf, commands);
+	base_item.offset_in_buf = buf->len - commands_len - 1;
+	base_item.arg = buf->buf + base_item.offset_in_buf;

I think if the user gives --exec more than once on the command line then commands will contain more than one exec command so this needs to parse commands and create one todo_item for each command.

Quoted 4 lines
sequencer.c +4 -0
+
+	/*
+	 * Insert <commands> after every pick. Here, fixup/squash chains
+	 * are considered part of the pick, so we insert the commands *after*

sequencer.c int sequencer_make_script(FILE *out, int argc, const char **argv, · line 4245

Quoted 10 lines
sequencer.c +10 -0
+	 * those chains if there are any.
+	 */
+	for (i = 0; i < todo_list->nr; i++) {
+		enum todo_command command = todo_list->items[i].command;
+		if (todo_list->items[i].arg)
+			todo_list->items[i].arg = todo_list->items[i].arg - old_buf + buf->buf;
+
+		if (command == TODO_PICK && !first) {
+			ALLOC_GROW(items, nr + 1, alloc);
+			memcpy(items + nr++, &base_item, sizeof(struct todo_item));

I think it would be clearer to say

	items[nr++] = base_item;

rather than using memcpy. This applies below and to some of the other patches as well. Also this needs to loop over all the base_items if the user gave --exec more than once on the command line.

Best Wishes

Phillip

Quoted 61 lines
sequencer.c +22 -20
+		}
+
+		ALLOC_GROW(items, nr + 1, alloc);
+		memcpy(items + nr++, todo_list->items + i, sizeof(struct todo_item));
+		first = 0;
+	}
+
+	/* insert or append final <commands> */
+	ALLOC_GROW(items, nr + 1, alloc);
+	memcpy(items + nr++, &base_item, sizeof(struct todo_item));
+
+	FREE_AND_NULL(todo_list->items);
+	todo_list->items = items;
+	todo_list->nr = nr;
+	todo_list->alloc = alloc;
+}
+
  /*
   * Add commands after pick and (series of) squash/fixup commands
   * in the todo list.
@@ -4224,10 +4268,7 @@ int sequencer_add_exec_commands(const char *commands)
  {
  	const char *todo_file = rebase_path_todo();
  	struct todo_list todo_list = TODO_LIST_INIT;
-	struct todo_item *item;
-	struct strbuf *buf = &todo_list.buf;
-	size_t offset = 0, commands_len = strlen(commands);
-	int i, first;
+	int res;
 
  	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
  		return error(_("could not read '%s'."), todo_file);
@@ -4237,23 +4278,11 @@ int sequencer_add_exec_commands(const char *commands)
  		return error(_("unusable todo list: '%s'"), todo_file);
  	}
 
-	first = 1;
-	/* insert <commands> before every pick except the first one */
-	for (item = todo_list.items, i = 0; i < todo_list.nr; i++, item++) {
-		if (item->command == TODO_PICK && !first) {
-			strbuf_insert(buf, item->offset_in_buf + offset,
-				      commands, commands_len);
-			offset += commands_len;
-		}
-		first = 0;
-	}
-
-	/* append final <commands> */
-	strbuf_add(buf, commands, commands_len);
-
-	i = write_message(buf->buf, buf->len, todo_file, 0);
+	todo_list_add_exec_commands(&todo_list, commands);
+	res = todo_list_write_to_file(&todo_list, todo_file, NULL, NULL, 0, 0, -1, 0);
  	todo_list_release(&todo_list);
-	return i;
+
+	return res;
  }
 
  static void todo_list_to_strbuf(struct todo_list *todo_list, struct strbuf *buf,
#39 lore raw

Re: [PATCH v2 06/16] sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

Le 30/10/2018 à 17:47, Phillip Wood a écrit :

Quoted 62 lines · Alban Gruin
Quoted 10 lines

This refactors sequencer_add_exec_commands() to work on a todo_list to avoid redundant reads and writes to the disk.

An obvious way to do this would be to insert the `exec' command between the other commands, and reparse it once this is done.  This is not what is done here.  Instead, the command is appended to the buffer once, and a new list of items is created.  Items from the old list are copied to it, and new `exec' items are appended when necessary.  This eliminates the need to reparse the todo list, but this also means its buffer cannot be directly written to the disk, hence todo_list_write_to_disk().

I'd reword this slightly, maybe

Instead of just inserting the `exec' command between the other commands, and re-parsing the buffer at the end the exec command is appended to the buffer once, and a new list of items is created.  Items from the old list are copied across and new `exec' items are appended when necessary. This eliminates the need to reparse the buffer, but this also means we have to use todo_list_write_to_disk() to write the file.

Quoted 3 lines

sequencer_add_exec_commands() still reads the todo list from the disk, as it is needed by rebase -p.  todo_list_add_exec_commands() works on a todo_list structure, and reparses it at the end.

I think the saying 'reparses' is confusing as that is what we're trying to avoid.

Quoted 13 lines

complete_action() still uses sequencer_add_exec_commands() for now. This will be changed in a future commit.

---
  sequencer.c | 69 +++++++++++++++++++++++++++++++++++++----------------

  1 file changed, 49 insertions(+), 20 deletions(-)
sequencer.c +0 -0
index e12860c047..12a3efeca8 100644

sequencer.c int sequencer_make_script(FILE *out, int argc, · line 4225

Quoted 10 lines
sequencer.c +6 -0
@@ -4216,6 +4216,50 @@ int sequencer_make_script(FILE *out, int argc,
 const char **argv,
       return 0;
   }
   +static void todo_list_add_exec_commands(struct todo_list *todo_list,
+                    const char *commands)
+{
+    struct strbuf *buf = &todo_list->buf;
+    const char *old_buf = buf->buf;
+    size_t commands_len = strlen(commands + strlen("exec ")) - 1;
+    int i, first = 1, nr = 0, alloc = 0;

Minor nit pick, I think it is clearer if first is initialized just before the loop as it is in the deleted code below.

sequencer.c int sequencer_make_script(FILE *out, int argc, const char **argv, · line 4231

Quoted 6 lines
sequencer.c +6 -0
+    struct todo_item *items = NULL,
+        base_item = {TODO_EXEC, NULL, 0, 0, commands_len, 0};
+
+    strbuf_addstr(buf, commands);
+    base_item.offset_in_buf = buf->len - commands_len - 1;
+    base_item.arg = buf->buf + base_item.offset_in_buf;
Quoted 4 lines

I think if the user gives --exec more than once on the command line then commands will contain more than one exec command so this needs to parse commands and create one todo_item for each command.

Ouch, you’re right. Thanks for the heads up.

Quoted 14 lines
Quoted 14 lines
sequencer.c +12 -0
+
+    /*
+     * Insert <commands> after every pick. Here, fixup/squash chains
+     * are considered part of the pick, so we insert the commands
 *after*
+     * those chains if there are any.
+     */
+    for (i = 0; i < todo_list->nr; i++) {
+        enum todo_command command = todo_list->items[i].command;
+        if (todo_list->items[i].arg)
+            todo_list->items[i].arg = todo_list->items[i].arg -
 old_buf + buf->buf;
+
+        if (command == TODO_PICK && !first) {
Quoted 9 lines

sequencer.c int sequencer_make_script(FILE *out, int argc, · line 4247

Quoted 2 lines
sequencer.c +2 -0
+            ALLOC_GROW(items, nr + 1, alloc);
+            memcpy(items + nr++, &base_item, sizeof(struct todo_item));

I think it would be clearer to say
    items[nr++] = base_item;
rather than using memcpy. This applies below and to some of the other patches as well. Also this needs to loop over all the base_items if the user gave --exec more than once on the command line.

I agree with you, it’s way more readable, IMO. But for some reason, I thought it was not possible to assign a struct to another in C.

Quoted 4 lines

Best Wishes

Phillip

Cheers,
Alban

#40 lore raw

Re: [PATCH v2 06/16] sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

Hi Alban

Quoted 63 lines · Alban Gruin

Le 30/10/2018 à 17:47, Phillip Wood a écrit :

Quoted 62 lines · Alban Gruin
Quoted 10 lines

This refactors sequencer_add_exec_commands() to work on a todo_list to avoid redundant reads and writes to the disk.

An obvious way to do this would be to insert the `exec' command between the other commands, and reparse it once this is done.  This is not what is done here.  Instead, the command is appended to the buffer once, and a new list of items is created.  Items from the old list are copied to it, and new `exec' items are appended when necessary.  This eliminates the need to reparse the todo list, but this also means its buffer cannot be directly written to the disk, hence todo_list_write_to_disk().

I'd reword this slightly, maybe

Instead of just inserting the `exec' command between the other commands, and re-parsing the buffer at the end the exec command is appended to the buffer once, and a new list of items is created.  Items from the old list are copied across and new `exec' items are appended when necessary. This eliminates the need to reparse the buffer, but this also means we have to use todo_list_write_to_disk() to write the file.

Quoted 3 lines

sequencer_add_exec_commands() still reads the todo list from the disk, as it is needed by rebase -p.  todo_list_add_exec_commands() works on a todo_list structure, and reparses it at the end.

I think the saying 'reparses' is confusing as that is what we're trying to avoid.

Quoted 13 lines

complete_action() still uses sequencer_add_exec_commands() for now. This will be changed in a future commit.

---
  sequencer.c | 69 +++++++++++++++++++++++++++++++++++++----------------

  1 file changed, 49 insertions(+), 20 deletions(-)
sequencer.c +0 -0
index e12860c047..12a3efeca8 100644

sequencer.c int sequencer_make_script(FILE *out, int argc, · line 4225

Quoted 10 lines
sequencer.c +6 -0
@@ -4216,6 +4216,50 @@ int sequencer_make_script(FILE *out, int argc,
 const char **argv,
       return 0;
   }
   +static void todo_list_add_exec_commands(struct todo_list *todo_list,
+                    const char *commands)
+{
+    struct strbuf *buf = &todo_list->buf;
+    const char *old_buf = buf->buf;
+    size_t commands_len = strlen(commands + strlen("exec ")) - 1;
+    int i, first = 1, nr = 0, alloc = 0;

Minor nit pick, I think it is clearer if first is initialized just before the loop as it is in the deleted code below.

Quoted 6 lines
sequencer.c +6 -0
+    struct todo_item *items = NULL,
+        base_item = {TODO_EXEC, NULL, 0, 0, commands_len, 0};
+
+    strbuf_addstr(buf, commands);
+    base_item.offset_in_buf = buf->len - commands_len - 1;
+    base_item.arg = buf->buf + base_item.offset_in_buf;
Quoted 6 lines
Quoted 4 lines

I think if the user gives --exec more than once on the command line then commands will contain more than one exec command so this needs to parse commands and create one todo_item for each command.

Ouch, you’re right. Thanks for the heads up.

I haven't looked how difficult it would be but it might be best to change the option parsing to pass an array of strings containing the exec commands rather than one long string so we can just loop over the array here.

Quoted 25 lines
Quoted 15 lines
Quoted 15 lines
sequencer.c +12 -0
+
+    /*
+     * Insert <commands> after every pick. Here, fixup/squash chains
+     * are considered part of the pick, so we insert the commands
 *after*
+     * those chains if there are any.
+     */
+    for (i = 0; i < todo_list->nr; i++) {
+        enum todo_command command = todo_list->items[i].command;
+        if (todo_list->items[i].arg)
+            todo_list->items[i].arg = todo_list->items[i].arg -
 old_buf + buf->buf;
+
+        if (command == TODO_PICK && !first) {
Quoted 9 lines

sequencer.c int sequencer_make_script(FILE *out, int argc, · line 4247

Quoted 2 lines
sequencer.c +2 -0
+            ALLOC_GROW(items, nr + 1, alloc);
+            memcpy(items + nr++, &base_item, sizeof(struct todo_item));

I think it would be clearer to say
    items[nr++] = base_item;
rather than using memcpy. This applies below and to some of the other patches as well. Also this needs to loop over all the base_items if the user gave --exec more than once on the command line.

Quoted 2 lines

I agree with you, it’s way more readable, IMO. But for some reason, I thought it was not possible to assign a struct to another in C.

In general one needs to be careful as it is only does a shallow copy but that is exactly what we want here. Having said that if we have an array of exec commands to insert then it might be worth sticking with memcpy() here so the whole array can be copied in one go.

Best Wishes

Phillip

Quoted 8 lines
Quoted 4 lines

Best Wishes

Phillip

Cheers,
Alban

#41 lore raw

Re: [PATCH v2 06/16] sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

Hi Phillip,

Le 02/11/2018 à 11:09, Phillip Wood a écrit :

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 6 lines

+    struct todo_item *items = NULL,
+        base_item = {TODO_EXEC, NULL, 0, 0, commands_len, 0}; +
+    strbuf_addstr(buf, commands);
+    base_item.offset_in_buf = buf->len - commands_len - 1; +    base_item.arg = buf->buf + base_item.offset_in_buf;

I think if the user gives --exec more than once on the command line then commands will contain more than one exec command so this needs to parse commands and create one todo_item for each command.

Quoted 7 lines
Quoted 1 line

Ouch, you’re right. Thanks for the heads up.

I haven't looked how difficult it would be but it might be best to change the option parsing to pass an array of strings containing the exec commands rather than one long string so we can just loop over the array here.

It would be the best way to do so. This string comes from git-rebase.sh (or builtin/rebase.c) -- they format it this way before invoking git-rebase--interactive. So either I modify both of them (for this, I would need to rebase my branch on master), or I can split this string in builtin/rebase--interactive.c. I prefer the first option, but maybe changing the base of this series will not please Junio.

Cheers,
Alban

#42 lore raw

Re: [PATCH v2 06/16] sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

Hi Alban

Quoted 23 lines · Alban Gruin

Hi Phillip,

Le 02/11/2018 à 11:09, Phillip Wood a écrit :

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 6 lines

+    struct todo_item *items = NULL,
+        base_item = {TODO_EXEC, NULL, 0, 0, commands_len, 0}; +
+    strbuf_addstr(buf, commands);
+    base_item.offset_in_buf = buf->len - commands_len - 1; +    base_item.arg = buf->buf + base_item.offset_in_buf;

I think if the user gives --exec more than once on the command line then commands will contain more than one exec command so this needs to parse commands and create one todo_item for each command.

Quoted 7 lines
Quoted 1 line

Ouch, you’re right. Thanks for the heads up.

I haven't looked how difficult it would be but it might be best to change the option parsing to pass an array of strings containing the exec commands rather than one long string so we can just loop over the array here.

Quoted 6 lines

It would be the best way to do so. This string comes from git-rebase.sh (or builtin/rebase.c) -- they format it this way before invoking git-rebase--interactive. So either I modify both of them (for this, I would need to rebase my branch on master), or I can split this string in builtin/rebase--interactive.c. I prefer the first option, but maybe changing the base of this series will not please Junio.

I think in the last 'what's cooking' email Junio said he was going to merge all the builtin/rebase.c stuff to master so there may not be a problem if you wait a couple of days.

Best Wishes

Phillip

Quoted 3 lines

Cheers,
Alban

#43 lore raw

PATCH v2 7/16 sequencer: refactor rearrange_squash() to work on a todo_list

In reply to Alban Gruin

This refactors rearrange_squash() to work on a todo_list to avoid redundant reads and writes. The function is renamed todo_list_rearrange_squash().

The old version created a new buffer, which was directly written to the disk. This new version creates a new item list by just memcpying items from the old item list, without creating a new buffer. This eliminates the need to reparse the todo list, but this also means its buffer cannot be directly written to the disk.

As rebase -p still need to check the todo list from the disk, a new function is introduced, rearrange_squash_in_todo_file().

complete_action() still uses rearrange_squash_in_todo_file() for now. This will be changed in a future commit.

---
 builtin/rebase--interactive.c |  2 +-
 sequencer.c                   | 87 +++++++++++++++++------------------
 sequencer.h                   |  2 +-
 3 files changed, 45 insertions(+), 46 deletions(-)
builtin/rebase--interactive.c +1 -1
index c1d87c0fe6..f827e53f05 100644
@@ -258,7 +258,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = check_todo_list_from_file();
 		break;
 	case REARRANGE_SQUASH:
-		ret = rearrange_squash();
+		ret = rearrange_squash_in_todo_file();
 		break;
 	case ADD_EXEC:
 		ret = sequencer_add_exec_commands(cmd);
sequencer.c +43 -44

152 lines not on this page · raw message

sequencer.h +1 -1
index e203930084..0c8074cab0 100644
@@ -139,7 +139,7 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, const char *cmd,
 		    unsigned autosquash);
-int rearrange_squash(void);
+int rearrange_squash_in_todo_file(void);
 
 extern const char sign_off_header[];
 
Signature · 1 line
2.19.1
#44 lore raw

PATCH v2 8/16 sequencer: make sequencer_make_script() write its script to a strbuf

In reply to Alban Gruin

This makes sequencer_make_script() write its script to a strbuf (ie. the buffer of a todo_list) instead of a FILE. This reduce the amount of read/write made by rebase interactive.

---
 builtin/rebase--interactive.c | 13 +++++++-----
 sequencer.c                   | 38 ++++++++++++++++-------------------
 sequencer.h                   |  2 +-
 3 files changed, 26 insertions(+), 27 deletions(-)
builtin/rebase--interactive.c +8 -5
index f827e53f05..eef1ff2e83 100644
@@ -71,7 +71,8 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	const char *head_hash = NULL;
 	char *revisions = NULL, *shortrevisions = NULL;
 	struct argv_array make_script_args = ARGV_ARRAY_INIT;
-	FILE *todo_list;
+	FILE *todo_list_file;
+	struct todo_list todo_list = TODO_LIST_INIT;
 
 	if (prepare_branch_to_be_rebased(opts, switch_to))
 		return -1;
@@ -93,8 +94,8 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	if (!upstream && squash_onto)
 		write_file(path_squash_onto(), "%s\n", squash_onto);
 
-	todo_list = fopen(rebase_path_todo(), "w");
-	if (!todo_list) {
+	todo_list_file = fopen(rebase_path_todo(), "w");
+	if (!todo_list_file) {
 		free(revisions);
 		free(shortrevisions);
 
@@ -105,10 +106,11 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	if (restrict_revision)
 		argv_array_push(&make_script_args, restrict_revision);
 
-	ret = sequencer_make_script(todo_list,
+	ret = sequencer_make_script(&todo_list.buf,
 				    make_script_args.argc, make_script_args.argv,
 				    flags);
-	fclose(todo_list);
+	fputs(todo_list.buf.buf, todo_list_file);
+	fclose(todo_list_file);
 
 	if (ret)
 		error(_("could not generate todo list"));
@@ -120,6 +122,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 
 	free(revisions);
 	free(shortrevisions);
+	todo_list_release(&todo_list);
 	argv_array_clear(&make_script_args);
 
 	return ret;
sequencer.c +17 -21
index 09e32f3e5a..94167588a2 100644
@@ -3927,7 +3927,7 @@ static const char *label_oid(struct object_id *oid, const char *label,
 }
 
 static int make_script_with_merges(struct pretty_print_context *pp,
-				   struct rev_info *revs, FILE *out,
+				   struct rev_info *revs, struct strbuf *out,
 				   unsigned flags)
 {
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
@@ -4076,7 +4076,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 	 * gathering commits not yet shown, reversing the list on the fly,
 	 * then outputting that list (labeling revisions as needed).
 	 */
-	fprintf(out, "%s onto\n", cmd_label);
+	strbuf_addf(out, "%s onto\n", cmd_label);
 	for (iter = tips; iter; iter = iter->next) {
 		struct commit_list *list = NULL, *iter2;
 
@@ -4086,9 +4086,9 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 		entry = oidmap_get(&state.commit2label, &commit->object.oid);
 
 		if (entry)
-			fprintf(out, "\n# Branch %s\n", entry->string);
+			strbuf_addf(out, "\n%c Branch %s\n", comment_line_char, entry->string);
 		else
-			fprintf(out, "\n");
+			strbuf_addch(out, '\n');
 
 		while (oidset_contains(&interesting, &commit->object.oid) &&
 		       !oidset_contains(&shown, &commit->object.oid)) {
@@ -4101,8 +4101,8 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 		}
 
 		if (!commit)
-			fprintf(out, "%s %s\n", cmd_reset,
-				rebase_cousins ? "onto" : "[new root]");
+			strbuf_addf(out, "%s %s\n", cmd_reset,
+				    rebase_cousins ? "onto" : "[new root]");
 		else {
 			const char *to = NULL;
 
@@ -4115,12 +4115,12 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 					       &state);
 
 			if (!to || !strcmp(to, "onto"))
-				fprintf(out, "%s onto\n", cmd_reset);
+				strbuf_addf(out, "%s onto\n", cmd_reset);
 			else {
 				strbuf_reset(&oneline);
 				pretty_print_commit(pp, commit, &oneline);
-				fprintf(out, "%s %s # %s\n",
-					cmd_reset, to, oneline.buf);
+				strbuf_addf(out, "%s %s # %s\n",
+					    cmd_reset, to, oneline.buf);
 			}
 		}
 
@@ -4129,11 +4129,11 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 			entry = oidmap_get(&commit2todo, oid);
 			/* only show if not already upstream */
 			if (entry)
-				fprintf(out, "%s\n", entry->string);
+				strbuf_addf(out, "%s\n", entry->string);
 			entry = oidmap_get(&state.commit2label, oid);
 			if (entry)
-				fprintf(out, "%s %s\n",
-					cmd_label, entry->string);
+				strbuf_addf(out, "%s %s\n",
+					    cmd_label, entry->string);
 			oidset_insert(&shown, oid);
 		}
 
@@ -4155,12 +4155,11 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 	return 0;
 }
 
-int sequencer_make_script(FILE *out, int argc, const char **argv,
+int sequencer_make_script(struct strbuf *out, int argc, const char **argv,
 			  unsigned flags)
 {
 	char *format = NULL;
 	struct pretty_print_context pp = {0};
-	struct strbuf buf = STRBUF_INIT;
 	struct rev_info revs;
 	struct commit *commit;
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
@@ -4203,16 +4202,13 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 
 		if (!is_empty && (commit->object.flags & PATCHSAME))
 			continue;
-		strbuf_reset(&buf);
 		if (!keep_empty && is_empty)
-			strbuf_addf(&buf, "%c ", comment_line_char);
-		strbuf_addf(&buf, "%s %s ", insn,
+			strbuf_addf(out, "%c ", comment_line_char);
+		strbuf_addf(out, "%s %s ", insn,
 			    oid_to_hex(&commit->object.oid));
-		pretty_print_commit(&pp, commit, &buf);
-		strbuf_addch(&buf, '\n');
-		fputs(buf.buf, out);
+		pretty_print_commit(&pp, commit, out);
+		strbuf_addch(out, '\n');
 	}
-	strbuf_release(&buf);
 	return 0;
 }
 
sequencer.h +1 -1
index 0c8074cab0..5af96723e6 100644
@@ -129,7 +129,7 @@ int sequencer_remove_state(struct replay_opts *opts);
  * commits should be rebased onto the new base, this flag needs to be passed.
  */
 #define TODO_LIST_REBASE_COUSINS (1U << 4)
-int sequencer_make_script(FILE *out, int argc, const char **argv,
+int sequencer_make_script(struct strbuf *out, int argc, const char **argv,
 			  unsigned flags);
 
 int sequencer_add_exec_commands(const char *command);
Signature · 1 line
2.19.1
#45 lore raw

PATCH v2 9/16 sequencer: change complete_action() to use the refactored functions

In reply to Alban Gruin

complete_action() used functions that read the todo-list file, made some changes to it, and wrote it back to the disk.

The previous commits were dedicated to separate the part that deals with the file from the actual logic of these functions. Now that this is done, we can call directly the "logic" functions to avoid useless file access.

The parsing of the list has to be done by the caller. If the buffer of the todo list provided by the caller is empty, a `noop' command is directly added to the todo list, without touching to the buffer.

---
 builtin/rebase--interactive.c | 16 ++-----
 sequencer.c                   | 82 ++++++++++++++++-------------------
 sequencer.h                   |  2 +-
 3 files changed, 42 insertions(+), 58 deletions(-)
builtin/rebase--interactive.c +4 -12
index eef1ff2e83..c07330bdff 100644
@@ -71,7 +71,6 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	const char *head_hash = NULL;
 	char *revisions = NULL, *shortrevisions = NULL;
 	struct argv_array make_script_args = ARGV_ARRAY_INIT;
-	FILE *todo_list_file;
 	struct todo_list todo_list = TODO_LIST_INIT;
 
 	if (prepare_branch_to_be_rebased(opts, switch_to))
@@ -94,14 +93,6 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	if (!upstream && squash_onto)
 		write_file(path_squash_onto(), "%s\n", squash_onto);
 
-	todo_list_file = fopen(rebase_path_todo(), "w");
-	if (!todo_list_file) {
-		free(revisions);
-		free(shortrevisions);
-
-		return error_errno(_("could not open %s"), rebase_path_todo());
-	}
-
 	argv_array_pushl(&make_script_args, "", revisions, NULL);
 	if (restrict_revision)
 		argv_array_push(&make_script_args, restrict_revision);
@@ -109,15 +100,16 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	ret = sequencer_make_script(&todo_list.buf,
 				    make_script_args.argc, make_script_args.argv,
 				    flags);
-	fputs(todo_list.buf.buf, todo_list_file);
-	fclose(todo_list_file);
 
 	if (ret)
 		error(_("could not generate todo list"));
 	else {
 		discard_cache();
+		if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list))
+			BUG("unusable todo list");
+
 		ret = complete_action(opts, flags, shortrevisions, onto_name, onto,
-				      head_hash, cmd, autosquash);
+				      head_hash, cmd, autosquash, &todo_list);
 	}
 
 	free(revisions);
sequencer.c +37 -45

130 lines not on this page · raw message

sequencer.h +1 -1
index 5af96723e6..4a051e5dc9 100644
@@ -138,7 +138,7 @@ int check_todo_list_from_file(void);
 int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, const char *cmd,
-		    unsigned autosquash);
+		    unsigned autosquash, struct todo_list *todo_list);
 int rearrange_squash_in_todo_file(void);
 
 extern const char sign_off_header[];
Signature · 1 line
2.19.1
#46 lore raw

PATCH v2 10/16 sequencer: refactor skip_unnecessary_picks() to work on a todo_list

In reply to Alban Gruin

This refactors skip_unnecessary_picks() to work on a todo_list. The file-handling logic is completely dropped here, as its only usage is made by complete_action().

Instead of truncating the todo list’s buffer, the items are moved to the beginning of the list, eliminating the need to reparse the list. This also means its buffer cannot be directly written to the disk.

rewrite_file() is then removed, as it is now unused.

---
 sequencer.c | 80 ++++++++++++-----------------------------------------
 1 file changed, 18 insertions(+), 62 deletions(-)
sequencer.c +18 -62
index 1f7579328b..7286498572 100644
@@ -4408,52 +4408,21 @@ int check_todo_list_from_file(void)
 	return res;
 }
 
-static int rewrite_file(const char *path, const char *buf, size_t len)
-{
-	int rc = 0;
-	int fd = open(path, O_WRONLY | O_TRUNC);
-	if (fd < 0)
-		return error_errno(_("could not open '%s' for writing"), path);
-	if (write_in_full(fd, buf, len) < 0)
-		rc = error_errno(_("could not write to '%s'"), path);
-	if (close(fd) && !rc)
-		rc = error_errno(_("could not close '%s'"), path);
-	return rc;
-}
-
 /* skip picking commits whose parents are unchanged */
-static int skip_unnecessary_picks(struct object_id *output_oid)
+static int skip_unnecessary_picks(struct todo_list *todo_list,
+				  struct object_id *output_oid)
 {
-	const char *todo_file = rebase_path_todo();
-	struct strbuf buf = STRBUF_INIT;
-	struct todo_list todo_list = TODO_LIST_INIT;
 	struct object_id *parent_oid;
-	int fd, i;
-
-	if (!read_oneliner(&buf, rebase_path_onto(), 0))
-		return error(_("could not read 'onto'"));
-	if (get_oid(buf.buf, output_oid)) {
-		strbuf_release(&buf);
-		return error(_("need a HEAD to fixup"));
-	}
-	strbuf_release(&buf);
-
-	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
-		return -1;
-	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {
-		todo_list_release(&todo_list);
-		return -1;
-	}
+	int i;
 
-	for (i = 0; i < todo_list.nr; i++) {
-		struct todo_item *item = todo_list.items + i;
+	for (i = 0; i < todo_list->nr; i++) {
+		struct todo_item *item = todo_list->items + i;
 
 		if (item->command >= TODO_NOOP)
 			continue;
 		if (item->command != TODO_PICK)
 			break;
 		if (parse_commit(item->commit)) {
-			todo_list_release(&todo_list);
 			return error(_("could not parse commit '%s'"),
 				oid_to_hex(&item->commit->object.oid));
 		}
@@ -4467,37 +4436,22 @@ static int skip_unnecessary_picks(struct object_id *output_oid)
 		oidcpy(output_oid, &item->commit->object.oid);
 	}
 	if (i > 0) {
-		int offset = get_item_line_offset(&todo_list, i);
 		const char *done_path = rebase_path_done();
 
-		fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
-		if (fd < 0) {
-			error_errno(_("could not open '%s' for writing"),
-				    done_path);
-			todo_list_release(&todo_list);
-			return -1;
-		}
-		if (write_in_full(fd, todo_list.buf.buf, offset) < 0) {
+		if (todo_list_write_to_file(todo_list, done_path, NULL, NULL, 0, 0, i, 0)) {
 			error_errno(_("could not write to '%s'"), done_path);
-			todo_list_release(&todo_list);
-			close(fd);
 			return -1;
 		}
-		close(fd);
 
-		if (rewrite_file(rebase_path_todo(), todo_list.buf.buf + offset,
-				 todo_list.buf.len - offset) < 0) {
-			todo_list_release(&todo_list);
-			return -1;
-		}
+		memmove(todo_list->items, todo_list->items + i,
+			sizeof(struct todo_item) * (todo_list->nr - i));
+		todo_list->nr -= i;
+		todo_list->current = 0;
 
-		todo_list.current = i;
-		if (is_fixup(peek_command(&todo_list, 0)))
-			record_in_rewritten(output_oid, peek_command(&todo_list, 0));
+		if (is_fixup(peek_command(todo_list, 0)))
+			record_in_rewritten(output_oid, peek_command(todo_list, 0));
 	}
 
-	todo_list_release(&todo_list);
-
 	return 0;
 }
 
@@ -4573,6 +4527,11 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 		return -1;
 	}
 
+	if (opts->allow_ff && skip_unnecessary_picks(&new_todo, &oid)) {
+		todo_list_release(&new_todo);
+		return error(_("could not skip unnecessary pick commands"));
+	}
+
 	if (todo_list_write_to_file(&new_todo, todo_file, NULL, NULL, 0, 0, -1,
 				    flags & ~(TODO_LIST_SHORTEN_IDS))) {
 		todo_list_release(&new_todo);
@@ -4581,12 +4540,9 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 
 	todo_list_release(&new_todo);
 
-	if (opts->allow_ff && skip_unnecessary_picks(&oid))
-		return error(_("could not skip unnecessary pick commands"));
-
 	if (checkout_onto(opts, onto_name, oid_to_hex(&oid), orig_head))
 		return -1;
-;
+
 	if (require_clean_work_tree("rebase", "", 1, 1))
 		return -1;
 
Signature · 1 line
2.19.1
#47 lore raw

PATCH v2 11/16 rebase-interactive: use todo_list_write_to_file() in edit_todo_list()

In reply to Alban Gruin

Just like complete_action(), edit_todo_list() used a function (transform_todo_file()) that read the todo list from the disk and wrote it back, resulting in useless disk accesses.

This changes edit_todo_list() to call directly todo_list_write_to_file() instead.

---
 rebase-interactive.c | 38 ++++++++++++++++----------------------
 1 file changed, 16 insertions(+), 22 deletions(-)
rebase-interactive.c +16 -22
index 9e81ecfe9f..b2be99a900 100644
@@ -78,39 +78,33 @@ void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 
 int edit_todo_list(unsigned flags)
 {
-	struct strbuf buf = STRBUF_INIT;
 	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res = 0;
 
-	if (strbuf_read_file(&buf, todo_file, 0) < 0)
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
 
-	strbuf_stripspace(&buf, 1);
-	if (write_message(buf.buf, buf.len, todo_file, 0)) {
-		strbuf_release(&buf);
+	strbuf_stripspace(&todo_list.buf, 1);
+	todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list);
+	if (todo_list_write_to_file(&todo_list, todo_file, NULL, NULL, 0, 1, -1,
+				    flags | TODO_LIST_SHORTEN_IDS)) {
+		todo_list_release(&todo_list);
 		return -1;
 	}
 
-	strbuf_release(&buf);
-
-	transform_todo_file(flags | TODO_LIST_SHORTEN_IDS);
-
-	if (strbuf_read_file(&buf, todo_file, 0) < 0)
-		return error_errno(_("could not read '%s'."), todo_file);
-
-	append_todo_help(1, 0, &buf);
-	if (write_message(buf.buf, buf.len, todo_file, 0)) {
-		strbuf_release(&buf);
+	strbuf_reset(&todo_list.buf);
+	if (launch_sequence_editor(todo_file, &todo_list.buf, NULL)) {
+		todo_list_release(&todo_list);
 		return -1;
 	}
 
-	strbuf_release(&buf);
+	if (!todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list))
+		res = todo_list_write_to_file(&todo_list, todo_file, NULL, NULL, 0, 0, -1,
+					      flags & ~(TODO_LIST_SHORTEN_IDS));
 
-	if (launch_sequence_editor(todo_file, NULL, NULL))
-		return -1;
-
-	transform_todo_file(flags & ~(TODO_LIST_SHORTEN_IDS));
-
-	return 0;
+	todo_list_release(&todo_list);
+	return res;
 }
 
 define_commit_slab(commit_seen, unsigned char);
Signature · 1 line
2.19.1
#48 lore raw

PATCH v2 12/16 rebase-interactive: append_todo_help() changes

In reply to Alban Gruin

This moves the writing of the comment "Rebase $shortrevisions onto $shortonto ($command_count commands)" from complete_action() to append_todo_help().

shortrevisions, shortonto, and command_count are passed as parameters to append_todo_help().

During the initial edit of the todo list, shortrevisions and shortonto are not NULL. Therefore, if shortrevisions or shortonto is NULL, then edit_todo would be true, otherwise it would be false. Thus, edit_todo is removed from the parameters of append_todo_help().

edit_todo_list() and complete_action() are modified to fit these changes.

---
 rebase-interactive.c | 12 +++++++++++-
 rebase-interactive.h |  3 ++-
 sequencer.c          | 16 ++++------------
 3 files changed, 17 insertions(+), 14 deletions(-)
rebase-interactive.c +11 -1
index b2be99a900..0643d54b1b 100644
@@ -28,7 +28,8 @@ static enum missing_commit_check_level get_missing_commit_check_level(void)
 	return MISSING_COMMIT_CHECK_IGNORE;
 }
 
-void append_todo_help(unsigned edit_todo, unsigned keep_empty,
+void append_todo_help(unsigned keep_empty, int command_count,
+		      const char *shortrevisions, const char *shortonto,
 		      struct strbuf *buf)
 {
 	const char *msg = _("\nCommands:\n"
@@ -47,6 +48,15 @@ void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 ".       specified). Use -c <commit> to reword the commit message.\n"
 "\n"
 "These lines can be re-ordered; they are executed from top to bottom.\n");
+	unsigned edit_todo = !(shortrevisions && shortonto);
+
+	if (!edit_todo) {
+		strbuf_addch(buf, '\n');
+		strbuf_commented_addf(buf, Q_("Rebase %s onto %s (%d command)",
+					      "Rebase %s onto %s (%d commands)",
+					      command_count),
+				      shortrevisions, shortonto, command_count);
+	}
 
 	strbuf_add_commented_lines(buf, msg, strlen(msg));
 
rebase-interactive.h +2 -1
index 6bc7bc315d..61858f3a02 100644
@@ -1,7 +1,8 @@
 #ifndef REBASE_INTERACTIVE_H
 #define REBASE_INTERACTIVE_H
 
-void append_todo_help(unsigned edit_todo, unsigned keep_empty,
+void append_todo_help(unsigned keep_empty, int command_count,
+		      const char *shortrevisions, const char *shortonto,
 		      struct strbuf *buf);
 int edit_todo_list(unsigned flags);
 int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
sequencer.c +4 -12
index 7286498572..4c664d6388 100644
@@ -4331,21 +4331,13 @@ int todo_list_write_to_file(struct todo_list *todo_list, const char *file,
 			    const char *shortrevisions, const char *shortonto,
 			    int command_count, int append_help, int num, unsigned flags)
 {
-	int edit_todo = !(shortrevisions && shortonto), res;
+	int res;
 	struct strbuf buf = STRBUF_INIT;
 
 	todo_list_to_strbuf(todo_list, &buf, num, flags);
-
-	if (append_help) {
-		if (!edit_todo) {
-			strbuf_addch(&buf, '\n');
-			strbuf_commented_addf(&buf, Q_("Rebase %s onto %s (%d command)",
-						       "Rebase %s onto %s (%d commands)",
-						       command_count),
-					      shortrevisions, shortonto, command_count);
-		}
-		append_todo_help(edit_todo, flags & TODO_LIST_KEEP_EMPTY, &buf);
-	}
+	if (append_help)
+		append_todo_help(flags & TODO_LIST_KEEP_EMPTY, command_count,
+				 shortrevisions, shortonto, &buf);
 
 	res = write_message(buf.buf, buf.len, file, 0);
 	strbuf_release(&buf);
Signature · 1 line
2.19.1
#49 lore raw

PATCH v2 13/16 rebase-interactive: rewrite edit_todo_list() to handle the initial edit

In reply to Alban Gruin

edit_todo_list() is changed to work on a todo_list, and to handle the initial edition of the todo list (ie. making a backup of the todo list).

It does not check for dropped commits yet, as todo_list_check() does not take the commits that have already been processed by the rebase (ie. the todo list is edited in the middle of a rebase session).

---
 builtin/rebase--interactive.c | 23 ++++++++++++++++-
 rebase-interactive.c          | 48 ++++++++++++++++++-----------------
 rebase-interactive.h          |  4 ++-
 sequencer.c                   |  3 +--
 sequencer.h                   |  1 +
 5 files changed, 52 insertions(+), 27 deletions(-)
builtin/rebase--interactive.c +22 -1
index c07330bdff..0f83422aa0 100644
@@ -13,6 +13,27 @@ static GIT_PATH_FUNC(path_state_dir, "rebase-merge/")
 static GIT_PATH_FUNC(path_squash_onto, "rebase-merge/squash-onto")
 static GIT_PATH_FUNC(path_interactive, "rebase-merge/interactive")
 
+static int edit_todo_file(unsigned flags)
+{
+	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT,
+		new_todo = TODO_LIST_INIT;
+
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
+		return error_errno(_("could not read '%s'."), todo_file);
+
+	strbuf_stripspace(&todo_list.buf, 1);
+	if (!edit_todo_list(&todo_list, &new_todo, flags, 0, NULL, NULL) &&
+	    todo_list_write_to_file(&new_todo, todo_file, NULL, NULL, 0, 0, -1,
+				    flags & ~(TODO_LIST_SHORTEN_IDS)) < 0)
+		return error_errno(_("could not write '%s'"), todo_file);
+
+	todo_list_release(&todo_list);
+	todo_list_release(&new_todo);
+
+	return 0;
+}
+
 static int get_revision_ranges(const char *upstream, const char *onto,
 			       const char **head_hash,
 			       char **revisions, char **shortrevisions)
@@ -234,7 +255,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		break;
 	}
 	case EDIT_TODO:
-		ret = edit_todo_list(flags);
+		ret = edit_todo_file(flags);
 		break;
 	case SHOW_CURRENT_PATCH: {
 		struct child_process cmd = CHILD_PROCESS_INIT;
rebase-interactive.c +25 -23

60 lines not on this page · raw message

rebase-interactive.h +3 -1
index 61858f3a02..83cde455e6 100644
@@ -4,7 +4,9 @@
 void append_todo_help(unsigned keep_empty, int command_count,
 		      const char *shortrevisions, const char *shortonto,
 		      struct strbuf *buf);
-int edit_todo_list(unsigned flags);
+int edit_todo_list(struct todo_list *todo_list, struct todo_list *new_todo,
+		   unsigned flags, int command_count,
+		   const char *shortrevisions, const char *shortonto);
 int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
 
 #endif
sequencer.c +1 -2
index 4c664d6388..a03505f582 100644
@@ -54,8 +54,7 @@ static GIT_PATH_FUNC(rebase_path, "rebase-merge")
  * file and written to the tail of 'done'.
  */
 GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
-static GIT_PATH_FUNC(rebase_path_todo_backup,
-		     "rebase-merge/git-rebase-todo.backup")
+GIT_PATH_FUNC(rebase_path_todo_backup, "rebase-merge/git-rebase-todo.backup")
 
 /*
  * The rebase command lines that have already been processed. A line
sequencer.h +1 -0
index 4a051e5dc9..6a1a23541f 100644
@@ -4,6 +4,7 @@
 const char *git_path_commit_editmsg(void);
 const char *git_path_seq_dir(void);
 const char *rebase_path_todo(void);
+const char *rebase_path_todo_backup(void);
 
 #define APPEND_SIGNOFF_DEDUP (1u << 0)
 
Signature · 1 line
2.19.1
#50 lore raw

PATCH v2 14/16 sequencer: use edit_todo_list() in complete_action()

In reply to Alban Gruin

This changes complete_action() to use edit_todo_list(), now that it can handle the initial edit of the todo list.

---
 sequencer.c | 22 +++++++---------------
 1 file changed, 7 insertions(+), 15 deletions(-)
sequencer.c +7 -15
index a03505f582..0763eeae25 100644
@@ -4457,7 +4457,7 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 	struct todo_list new_todo = TODO_LIST_INIT;
 	struct strbuf *buf = &todo_list->buf;
 	struct object_id oid;
-	int command_count;
+	int command_count, res;
 
 	get_oid(onto, &oid);
 	shortonto = find_unique_abbrev(&oid, DEFAULT_ABBREV);
@@ -4484,24 +4484,16 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 		return error(_("nothing to do"));
 	}
 
-	if (todo_list_write_to_file(todo_list, todo_file,
-				    shortrevisions, shortonto, command_count,
-				    1, -1, flags | TODO_LIST_SHORTEN_IDS))
-		return error_errno(_("could not write '%s'"), todo_file);
-
-	if (copy_file(rebase_path_todo_backup(), todo_file, 0666))
-		return error(_("could not copy '%s' to '%s'."), todo_file,
-			     rebase_path_todo_backup());
-
-	if (launch_sequence_editor(todo_file, &new_todo.buf, NULL)) {
+	res = edit_todo_list(todo_list, &new_todo, flags,
+			     command_count, shortrevisions, shortonto);
+	if (res == -1)
+		return -1;
+	else if (res == -2) {
 		apply_autostash(opts);
 		sequencer_remove_state(opts);
 
 		return -1;
-	}
-
-	strbuf_stripspace(&new_todo.buf, 1);
-	if (new_todo.buf.len == 0) {
+	} else if (res == -3) {
 		apply_autostash(opts);
 		sequencer_remove_state(opts);
 		todo_list_release(&new_todo);
Signature · 1 line
2.19.1
#51 lore raw

PATCH v2 15/16 sequencer: fix a call to error() in transform_todo_file()

In reply to Alban Gruin

This replaces a call to error() by a call to error_errno() after writing the content of the todo list to the disk in transform_todo_file().

---
No changes since v1, was commit 14/15.

 sequencer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
sequencer.c +1 -1
index 0763eeae25..3069d4c6a2 100644
@@ -4351,7 +4351,7 @@ int transform_todo_file(unsigned flags)
 	int res;
 
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
-		return error(_("could not read '%s'."), todo_file);
+		return error_errno(_("could not read '%s'."), todo_file);
 
 	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
 		todo_list_release(&todo_list);
Signature · 1 line
2.19.1
#52 lore raw

PATCH v2 16/16 rebase--interactive: move transform_todo_file() to rebase--interactive.c

In reply to Alban Gruin

As transform_todo_file() is only needed inside of rebase--interactive.c, it is moved there from sequencer.c.

---
 builtin/rebase--interactive.c | 20 ++++++++++++++++++++
 sequencer.c                   | 20 --------------------
 sequencer.h                   |  1 -
 3 files changed, 20 insertions(+), 21 deletions(-)
builtin/rebase--interactive.c +20 -0
index 0f83422aa0..28e6947ecb 100644
@@ -34,6 +34,26 @@ static int edit_todo_file(unsigned flags)
 	return 0;
 }
 
+static int transform_todo_file(unsigned flags)
+{
+	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res;
+
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
+		return error_errno(_("could not read '%s'."), todo_file);
+
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
+		todo_list_release(&todo_list);
+		return error(_("unusable todo list: '%s'"), todo_file);
+	}
+
+	res = todo_list_write_to_file(&todo_list, todo_file,
+				      NULL, NULL, 0, 0, -1, flags);
+	todo_list_release(&todo_list);
+	return res;
+}
+
 static int get_revision_ranges(const char *upstream, const char *onto,
 			       const char **head_hash,
 			       char **revisions, char **shortrevisions)
sequencer.c +0 -20
index 3069d4c6a2..a1b81bd544 100644
@@ -4344,26 +4344,6 @@ int todo_list_write_to_file(struct todo_list *todo_list, const char *file,
 	return res;
 }
 
-int transform_todo_file(unsigned flags)
-{
-	const char *todo_file = rebase_path_todo();
-	struct todo_list todo_list = TODO_LIST_INIT;
-	int res;
-
-	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
-		return error_errno(_("could not read '%s'."), todo_file);
-
-	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
-		todo_list_release(&todo_list);
-		return error(_("unusable todo list: '%s'"), todo_file);
-	}
-
-	res = todo_list_write_to_file(&todo_list, todo_file,
-				      NULL, NULL, 0, 0, -1, flags);
-	todo_list_release(&todo_list);
-	return res;
-}
-
 static const char edit_todo_list_advice[] =
 N_("You can fix this with 'git rebase --edit-todo' "
 "and then run 'git rebase --continue'.\n"
sequencer.h +0 -1
index 6a1a23541f..d35ca61d5d 100644
@@ -134,7 +134,6 @@ int sequencer_make_script(struct strbuf *out, int argc, const char **argv,
 			  unsigned flags);
 
 int sequencer_add_exec_commands(const char *command);
-int transform_todo_file(unsigned flags);
 int check_todo_list_from_file(void);
 int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
Signature · 1 line
2.19.1
#53 lore raw
In reply to Alban Gruin
Quoted 26 lines · Alban Gruin

At the center of the "interactive" part of the interactive rebase lies the todo list. When the user starts an interactive rebase, a todo list is generated, presented to the user (who then edits it using a text editor), read back, and then is checked and processed before the actual rebase takes place.

Some of this processing includes adding execs commands, reordering fixup! and squash! commits, and checking if no commits were accidentally dropped by the user.

Before I converted the interactive rebase in C, these functions were called by git-rebase--interactive.sh through git-rebase--helper. Since the only way to pass around a large amount of data between a shell script and a C program is to use a file (or any declination of a file), the functions that checked and processed the todo list were directly working on a file, the same file that the user edited.

During the conversion, I did not address this issue, which lead to a complete_action() that reads the todo list file, does some computation based on its content, and writes it back to the disk, several times in the same function.

As it is not an efficient way to handle a data structure, this patch series refactor the functions that processes the todo list to work on a todo_list structure instead of reading it from the disk.

Quoted 6 lines

Some commits consists in modifying edit_todo_list() (initially used by --edit-todo) to handle the initial edition of the todo list, to increase code sharing.

It is based onto ag/rebase-i-in-c (34b4731, "rebase -i: move rebase--helper modes to rebase--interactive").

As there are quite a lot of fixes to the sequencer machinery since that topic forked from the mainline. For example, [06/16] has unpleasant merge conflicts with 1ace63bc ("rebase --exec: make it work with --rebase-merges", 2018-08-09) that has been in master for the past couple of months. IOW, the tip of ag/rebase-i-in-c is a bit too old a base to work on by now.

I think I queued the previous round on the result of merging ag/rebase-i-in-c into master, i.e. 61dc7b24 ("Merge branch 'ag/rebase-i-in-c' into ag/sequencer-reduce-rewriting-todo", 2018-10-09). That may be a more reasonable place to start this update on.

#54 lore raw

Hi Junio,

Le 29/10/2018 à 04:05, Junio C Hamano a écrit :

Quoted 1 line

Alban Gruin <[email protected]> writes:

[…]

Quoted 10 lines
Quoted 2 lines

It is based onto ag/rebase-i-in-c (34b4731, "rebase -i: move rebase--helper modes to rebase--interactive").

As there are quite a lot of fixes to the sequencer machinery since that topic forked from the mainline. For example, [06/16] has unpleasant merge conflicts with 1ace63bc ("rebase --exec: make it work with --rebase-merges", 2018-08-09) that has been in master for the past couple of months. IOW, the tip of ag/rebase-i-in-c is a bit too old a base to work on by now.

Quoted 6 lines

I think I queued the previous round on the result of merging ag/rebase-i-in-c into master, i.e. 61dc7b24 ("Merge branch 'ag/rebase-i-in-c' into ag/sequencer-reduce-rewriting-todo", 2018-10-09). That may be a more reasonable place to start this update on.

Right.

My next iteration will be based on 61dc7b24, I just rebased my branch onto it.

#55 lore raw

v3 cover sequencer: refactor functions working on a todo_list

In reply to Alban Gruin

At the center of the "interactive" part of the interactive rebase lies the todo list. When the user starts an interactive rebase, a todo list is generated, presented to the user (who then edits it using a text editor), read back, and then is checked and processed before the actual rebase takes place.

Some of this processing includes adding execs commands, reordering fixup! and squash! commits, and checking if no commits were accidentally dropped by the user.

Before I converted the interactive rebase in C, these functions were called by git-rebase--interactive.sh through git-rebase--helper. Since the only way to pass around a large amount of data between a shell script and a C program is to use a file (or any declination of a file), the functions that checked and processed the todo list were directly working on a file, the same file that the user edited.

During the conversion, I did not address this issue, which lead to a complete_action() that reads the todo list file, does some computation based on its content, and writes it back to the disk, several times in the same function.

As it is not an efficient way to handle a data structure, this patch series refactor the functions that processes the todo list to work on a todo_list structure instead of reading it from the disk.

Some commits consists in modifying edit_todo_list() (initially used by --edit-todo) to handle the initial edition of the todo list, to increase code sharing.

This is based on 8858448bb4 ("Ninth batch for 2.20", 2018-11-06) to avoid conflicts with 'js/rebase-i-break'.

Changes since v2:

- Clarifying some commit messages

- Reducing the number of parameters in todo_list_write_to_file()

- sequencer_add_exec_commands() now requests a string list instead of a string.

- Replacing calls to memcpy() by shallow copies

- Applying array.cocci.patch

Alban Gruin (16):
sequencer: changes in parse_insn_buffer()
sequencer: make the todo_list structure public
sequencer: refactor transform_todos() to work on a todo_list
sequencer: introduce todo_list_write_to_file()
sequencer: refactor check_todo_list() to work on a todo_list
sequencer: refactor sequencer_add_exec_commands() to work on a
todo_list
sequencer: refactor rearrange_squash() to work on a todo_list
sequencer: make sequencer_make_script() write its script to a strbuf
sequencer: change complete_action() to use the refactored functions
sequencer: refactor skip_unnecessary_picks() to work on a todo_list
rebase-interactive: use todo_list_write_to_file() in edit_todo_list()
rebase-interactive: append_todo_help() changes
rebase-interactive: rewrite edit_todo_list() to handle the initial
edit
sequencer: use edit_todo_list() in complete_action()
sequencer: fix a call to error() in transform_todo_file()
rebase--interactive: move transform_todo_file() to
rebase--interactive.c

 builtin/rebase--interactive.c |  81 +++--
 rebase-interactive.c          | 142 ++++++--
 rebase-interactive.h          |   8 +-
 sequencer.c                   | 615 +++++++++++++---------------------
 sequencer.h                   |  75 ++++-
 5 files changed, 481 insertions(+), 440 deletions(-)

Range-diff against v2:

 1:  55fa1fff03 =  1:  9ae965b73e sequencer: changes in parse_insn_buffer()
 2:  192fb771ed <  -:  ---------- sequencer: make the todo_list structure public
 -:  ---------- >  2:  9c15cdede4 sequencer: make the todo_list structure public
 3:  e2f821ffbe =  3:  a5c01d5a95 sequencer: refactor transform_todos() to work on a todo_list
 4:  0001e8dbde <  -:  ---------- sequencer: introduce todo_list_write_to_file()
 -:  ---------- >  4:  f2781fe4c3 sequencer: introduce todo_list_write_to_file()
 5:  aa9554d81d =  5:  337e0dce57 sequencer: refactor check_todo_list() to work on a todo_list
 6:  8bd793caf5 <  -:  ---------- sequencer: refactor sequencer_add_exec_commands() to work on a todo_list
 7:  e984946cef <  -:  ---------- sequencer: refactor rearrange_squash() to work on a todo_list
 8:  cf05254acf <  -:  ---------- sequencer: make sequencer_make_script() write its script to a strbuf
 9:  ac79151792 <  -:  ---------- sequencer: change complete_action() to use the refactored functions
10:  e227e38b24 <  -:  ---------- sequencer: refactor skip_unnecessary_picks() to work on a todo_list
11:  5bfd9d3460 <  -:  ---------- rebase-interactive: use todo_list_write_to_file() in edit_todo_list()
12:  f4d7578a77 <  -:  ---------- rebase-interactive: append_todo_help() changes
13:  11c43f1dfe <  -:  ---------- rebase-interactive: rewrite edit_todo_list() to handle the initial edit
14:  e9a6396c26 <  -:  ---------- sequencer: use edit_todo_list() in complete_action()
 -:  ---------- >  6:  2c75eed153 sequencer: refactor sequencer_add_exec_commands() to work on a todo_list
 -:  ---------- >  7:  90e01bc713 sequencer: refactor rearrange_squash() to work on a todo_list
 -:  ---------- >  8:  850261fbf7 sequencer: make sequencer_make_script() write its script to a strbuf
 -:  ---------- >  9:  7bcfe98241 sequencer: change complete_action() to use the refactored functions
 -:  ---------- > 10:  50cdea7ef7 sequencer: refactor skip_unnecessary_picks() to work on a todo_list
 -:  ---------- > 11:  6d4fcf96e0 rebase-interactive: use todo_list_write_to_file() in edit_todo_list()
 -:  ---------- > 12:  3bea4eb9ba rebase-interactive: append_todo_help() changes
 -:  ---------- > 13:  bfe8f568ee rebase-interactive: rewrite edit_todo_list() to handle the initial edit
 -:  ---------- > 14:  0d6499b7c8 sequencer: use edit_todo_list() in complete_action()
15:  a36d094610 = 15:  21d3d1abd7 sequencer: fix a call to error() in transform_todo_file()
16:  344e8ceeed <  -:  ---------- rebase--interactive: move transform_todo_file() to rebase--interactive.c
 -:  ---------- > 16:  22fd27fa2f rebase--interactive: move transform_todo_file() to rebase--interactive.c
Signature · 1 line
2.19.1.872.ga867da739e
#56 lore raw

PATCH v3 1/16 sequencer: changes in parse_insn_buffer()

This clears the number of items of a todo_list before parsing it to allow to parse the same list multiple times without issues. As its items are not dynamically allocated, or don’t need to allocate memory, no additionnal memory management is required here.

Furthermore, if a line is invalid, the type of the corresponding command is set to a garbage value, and its argument is defined properly. This will allow to recreate the text of a todo list from its commands, even if one of them is incorrect.

---
No changes since v2.

 sequencer.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
sequencer.c +6 -1
index 9e1ab3a2a7..07cc91d8db 100644
@@ -2066,6 +2066,8 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 	char *p = buf, *next_p;
 	int i, res = 0, fixup_okay = file_exists(rebase_path_done());
 
+	todo_list->current = todo_list->nr = 0;
+
 	for (i = 1; *p; i++, p = next_p) {
 		char *eol = strchrnul(p, '\n');
 
@@ -2079,7 +2081,10 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		if (parse_insn_line(item, p, eol)) {
 			res = error(_("invalid line %d: %.*s"),
 				i, (int)(eol - p), p);
-			item->command = TODO_NOOP;
+			item->command = TODO_COMMENT + 1;
+			item->arg = p;
+			item->arg_len = (int)(eol - p);
+			item->commit = NULL;
 		}
 
 		if (fixup_okay)
Signature · 1 line
2.19.1.872.ga867da739e
#57 lore raw

PATCH v3 2/16 sequencer: make the todo_list structure public

In reply to Alban Gruin

This makes the structures todo_list and todo_item, and the functions todo_list_release() and parse_insn_buffer(), accessible outside of sequencer.c.

---
 sequencer.c | 67 +++++++++--------------------------------------------
 sequencer.h | 49 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 60 insertions(+), 56 deletions(-)
sequencer.c +11 -56
index 07cc91d8db..7adbeaa27d 100644
@@ -1443,32 +1443,6 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 		return 1;
 }
 
-/*
- * Note that ordering matters in this enum. Not only must it match the mapping
- * below, it is also divided into several sections that matter.  When adding
- * new commands, make sure you add it in the right section.
- */
-enum todo_command {
-	/* commands that handle commits */
-	TODO_PICK = 0,
-	TODO_REVERT,
-	TODO_EDIT,
-	TODO_REWORD,
-	TODO_FIXUP,
-	TODO_SQUASH,
-	/* commands that do something else than handling a single commit */
-	TODO_EXEC,
-	TODO_BREAK,
-	TODO_LABEL,
-	TODO_RESET,
-	TODO_MERGE,
-	/* commands that do nothing but are counted for reporting progress */
-	TODO_NOOP,
-	TODO_DROP,
-	/* comments (not counted for reporting progress) */
-	TODO_COMMENT
-};
-
 static struct {
 	char c;
 	const char *str;
@@ -1939,26 +1913,7 @@ enum todo_item_flags {
 	TODO_EDIT_MERGE_MSG = 1
 };
 
-struct todo_item {
-	enum todo_command command;
-	struct commit *commit;
-	unsigned int flags;
-	const char *arg;
-	int arg_len;
-	size_t offset_in_buf;
-};
-
-struct todo_list {
-	struct strbuf buf;
-	struct todo_item *items;
-	int nr, alloc, current;
-	int done_nr, total_nr;
-	struct stat_data stat;
-};
-
-#define TODO_LIST_INIT { STRBUF_INIT }
-
-static void todo_list_release(struct todo_list *todo_list)
+void todo_list_release(struct todo_list *todo_list)
 {
 	strbuf_release(&todo_list->buf);
 	FREE_AND_NULL(todo_list->items);
@@ -2060,7 +2015,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	return !item->commit;
 }
 
-static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
+int todo_list_parse_insn_buffer(char *buf, struct todo_list *todo_list)
 {
 	struct todo_item *item;
 	char *p = buf, *next_p;
@@ -2158,7 +2113,7 @@ static int read_populate_todo(struct todo_list *todo_list,
 		return error(_("could not stat '%s'"), todo_file);
 	fill_stat_data(&todo_list->stat, &st);
 
-	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
+	res = todo_list_parse_insn_buffer(todo_list->buf.buf, todo_list);
 	if (res) {
 		if (is_rebase_i(opts))
 			return error(_("please fix this using "
@@ -2189,7 +2144,7 @@ static int read_populate_todo(struct todo_list *todo_list,
 		FILE *f = fopen_or_warn(rebase_path_msgtotal(), "w");
 
 		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
-				!parse_insn_buffer(done.buf.buf, &done))
+				!todo_list_parse_insn_buffer(done.buf.buf, &done))
 			todo_list->done_nr = count_commands(&done);
 		else
 			todo_list->done_nr = 0;
@@ -4454,7 +4409,7 @@ int sequencer_add_exec_commands(const char *commands)
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4510,7 +4465,7 @@ int transform_todos(unsigned flags)
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4596,7 +4551,7 @@ int check_todo_list(void)
 		goto leave_check;
 	}
 	advise_to_edit_todo = res =
-		parse_insn_buffer(todo_list.buf.buf, &todo_list);
+		todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list);
 
 	if (res || check_level == MISSING_COMMIT_CHECK_IGNORE)
 		goto leave_check;
@@ -4615,7 +4570,7 @@ int check_todo_list(void)
 		goto leave_check;
 	}
 	strbuf_release(&todo_file);
-	res = !!parse_insn_buffer(todo_list.buf.buf, &todo_list);
+	res = !!todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list);
 
 	/* Find commits in git-rebase-todo.backup yet unseen */
 	for (i = todo_list.nr - 1; i >= 0; i--) {
@@ -4697,7 +4652,7 @@ static int skip_unnecessary_picks(struct object_id *output_oid)
 
 	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
 		return -1;
-	if (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {
 		todo_list_release(&todo_list);
 		return -1;
 	}
@@ -4785,7 +4740,7 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 	if (strbuf_read_file(buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(buf->buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(buf->buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4893,7 +4848,7 @@ int rearrange_squash(void)
 
 	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
 		return -1;
-	if (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {
 		todo_list_release(&todo_list);
 		return -1;
 	}
sequencer.h +49 -0

55 lines not on this page · raw message

Signature · 1 line
2.19.1.872.ga867da739e
#58 lore raw

PATCH v3 3/16 sequencer: refactor transform_todos() to work on a todo_list

In reply to Alban Gruin

This refactors transform_todos() to work on a todo_list. The function is renamed todo_list_transform().

As rebase -p still need to check the todo list from the disk, a new function is introduced, transform_todo_file(). It is still used by complete_action() and edit_todo_list() for now, but they will be replaced in a future commit.

todo_list_transform() is not a static function, because it will be used by edit_todo_list() from rebase-interactive.c in a future commit.

---
No changes since v2.

 builtin/rebase--interactive.c |  2 +-
 rebase-interactive.c          |  4 +--
 sequencer.c                   | 46 +++++++++++++++++++++++------------
 sequencer.h                   |  3 ++-
 4 files changed, 35 insertions(+), 20 deletions(-)
builtin/rebase--interactive.c +1 -1
index a2ab68ed06..abdf6126df 100644
@@ -252,7 +252,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 	}
 	case SHORTEN_OIDS:
 	case EXPAND_OIDS:
-		ret = transform_todos(flags);
+		ret = transform_todo_file(flags);
 		break;
 	case CHECK_TODO_LIST:
 		ret = check_todo_list();
rebase-interactive.c +2 -2
index 78f3263fc1..4cd487a450 100644
@@ -69,7 +69,7 @@ int edit_todo_list(unsigned flags)
 
 	strbuf_release(&buf);
 
-	transform_todos(flags | TODO_LIST_SHORTEN_IDS);
+	transform_todo_file(flags | TODO_LIST_SHORTEN_IDS);
 
 	if (strbuf_read_file(&buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
@@ -85,7 +85,7 @@ int edit_todo_list(unsigned flags)
 	if (launch_sequence_editor(todo_file, NULL, NULL))
 		return -1;
 
-	transform_todos(flags & ~(TODO_LIST_SHORTEN_IDS));
+	transform_todo_file(flags & ~(TODO_LIST_SHORTEN_IDS));
 
 	return 0;
 }
sequencer.c +30 -16

76 lines not on this page · raw message

sequencer.h +2 -1
index 84f813305f..708d042f0e 100644
@@ -119,6 +119,7 @@ struct todo_list {
 #define TODO_LIST_INIT { STRBUF_INIT }
 
 int todo_list_parse_insn_buffer(char *buf, struct todo_list *todo_list);
+void todo_list_transform(struct todo_list *todo_list, unsigned flags);
 void todo_list_release(struct todo_list *todo_list);
 
 /* Call this to setup defaults before parsing command line options */
@@ -142,7 +143,7 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
 int sequencer_add_exec_commands(const char *command);
-int transform_todos(unsigned flags);
+int transform_todo_file(unsigned flags);
 enum missing_commit_check_level get_missing_commit_check_level(void);
 int check_todo_list(void);
 int complete_action(struct replay_opts *opts, unsigned flags,
Signature · 1 line
2.19.1.872.ga867da739e
#59 lore raw

PATCH v3 4/16 sequencer: introduce todo_list_write_to_file()

In reply to Alban Gruin

This introduces a new function to recreate the text of a todo list from its commands and write it to a file. This will be useful as the next few commits will change the use of the buffer in struct todo_list so it will no longer be a mirror of the file on disk.

This functionality can already be found in todo_list_transform(), but it is specifically made to replace the buffer of a todo list, which is not the desired behaviour. Thus, the part of todo_list_transform() that actually creates the buffer is moved to a new function, todo_list_to_strbuf(). The rest is unused, and so is dropped.

todo_list_write_to_file() can also take care of appending the help text to the buffer before writing it to the disk, or to write only the first n items of the list. This feature will be used by skip_unnecessary_picks(), which has to write done commands in a file.

---
 sequencer.c | 60 ++++++++++++++++++++++++++++++++++++-----------------
 sequencer.h |  6 +++++-
 2 files changed, 46 insertions(+), 20 deletions(-)
sequencer.c +41 -19
index e1097660ed..19f24825d4 100644
@@ -4454,24 +4454,27 @@ int sequencer_add_exec_commands(const char *commands)
 	return i;
 }
 
-void todo_list_transform(struct todo_list *todo_list, unsigned flags)
+static void todo_list_to_strbuf(struct todo_list *todo_list, struct strbuf *buf,
+				int num, unsigned flags)
 {
-	struct strbuf buf = STRBUF_INIT;
 	struct todo_item *item;
-	int i;
+	int i, max = todo_list->nr;
 
-	for (item = todo_list->items, i = 0; i < todo_list->nr; i++, item++) {
+	if (num > 0 && num < max)
+		max = num;
+
+	for (item = todo_list->items, i = 0; i < max; i++, item++) {
 		/* if the item is not a command write it and continue */
 		if (item->command >= TODO_COMMENT) {
-			strbuf_addf(&buf, "%.*s\n", item->arg_len, item->arg);
+			strbuf_addf(buf, "%.*s\n", item->arg_len, item->arg);
 			continue;
 		}
 
 		/* add command to the buffer */
 		if (flags & TODO_LIST_ABBREVIATE_CMDS)
-			strbuf_addch(&buf, command_to_char(item->command));
+			strbuf_addch(buf, command_to_char(item->command));
 		else
-			strbuf_addstr(&buf, command_to_string(item->command));
+			strbuf_addstr(buf, command_to_string(item->command));
 
 		/* add commit id */
 		if (item->commit) {
@@ -4481,27 +4484,47 @@ void todo_list_transform(struct todo_list *todo_list, unsigned flags)
 
 			if (item->command == TODO_MERGE) {
 				if (item->flags & TODO_EDIT_MERGE_MSG)
-					strbuf_addstr(&buf, " -c");
+					strbuf_addstr(buf, " -c");
 				else
-					strbuf_addstr(&buf, " -C");
+					strbuf_addstr(buf, " -C");
 			}
 
-			strbuf_addf(&buf, " %s", oid);
+			strbuf_addf(buf, " %s", oid);
 		}
 
 		/* add all the rest */
 		if (!item->arg_len)
-			strbuf_addch(&buf, '\n');
+			strbuf_addch(buf, '\n');
 		else
-			strbuf_addf(&buf, " %.*s\n", item->arg_len, item->arg);
+			strbuf_addf(buf, " %.*s\n", item->arg_len, item->arg);
 	}
+}
 
-	strbuf_reset(&todo_list->buf);
-	strbuf_add(&todo_list->buf, buf.buf, buf.len);
+int todo_list_write_to_file(struct todo_list *todo_list, const char *file,
+			    const char *shortrevisions, const char *shortonto,
+			    int num, unsigned flags)
+{
+	int edit_todo = !(shortrevisions && shortonto), res;
+	struct strbuf buf = STRBUF_INIT;
+
+	todo_list_to_strbuf(todo_list, &buf, num, flags);
+
+	if (flags & TODO_LIST_APPEND_TODO_HELP) {
+		int command_count = count_commands(todo_list);
+		if (!edit_todo) {
+			strbuf_addch(&buf, '\n');
+			strbuf_commented_addf(&buf, Q_("Rebase %s onto %s (%d command)",
+						       "Rebase %s onto %s (%d commands)",
+						       command_count),
+					      shortrevisions, shortonto, command_count);
+		}
+		append_todo_help(edit_todo, flags & TODO_LIST_KEEP_EMPTY, &buf);
+	}
+
+	res = write_message(buf.buf, buf.len, file, 0);
 	strbuf_release(&buf);
 
-	if (todo_list_parse_insn_buffer(todo_list->buf.buf, todo_list))
-		BUG("unusable todo list");
+	return res;
 }
 
 int transform_todo_file(unsigned flags)
@@ -4518,9 +4541,8 @@ int transform_todo_file(unsigned flags)
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
 
-	todo_list_transform(&todo_list, flags);
-
-	res = write_message(todo_list.buf.buf, todo_list.buf.len, todo_file, 0);
+	res = todo_list_write_to_file(&todo_list, todo_file,
+				      NULL, NULL, -1, flags);
 	todo_list_release(&todo_list);
 	return res;
 }
sequencer.h +5 -1
index 708d042f0e..e69357c95b 100644
@@ -119,7 +119,9 @@ struct todo_list {
 #define TODO_LIST_INIT { STRBUF_INIT }
 
 int todo_list_parse_insn_buffer(char *buf, struct todo_list *todo_list);
-void todo_list_transform(struct todo_list *todo_list, unsigned flags);
+int todo_list_write_to_file(struct todo_list *todo_list, const char *file,
+			    const char *shortrevisions, const char *shortonto,
+			    int num, unsigned flags);
 void todo_list_release(struct todo_list *todo_list);
 
 /* Call this to setup defaults before parsing command line options */
@@ -139,6 +141,8 @@ int sequencer_remove_state(struct replay_opts *opts);
  * commits should be rebased onto the new base, this flag needs to be passed.
  */
 #define TODO_LIST_REBASE_COUSINS (1U << 4)
+#define TODO_LIST_APPEND_TODO_HELP (1U << 5)
+
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
Signature · 1 line
2.19.1.872.ga867da739e
#60 lore raw

PATCH v3 5/16 sequencer: refactor check_todo_list() to work on a todo_list

In reply to Alban Gruin

This refactors check_todo_list() to work on a todo_list to avoid redundant reads and writes to the disk. The function is renamed todo_list_check(). The parsing of the two todo lists is left to the caller.

As rebase -p still need to check the todo list from the disk, a new function is introduced, check_todo_list_from_file(). It reads the file from the disk, parses it, pass the todo_list to todo_list_check(), and writes it back to the disk.

As get_missing_commit_check_level() and the enum missing_commit_check_level are no longer needed inside of sequencer.c, they are moved to rebase-interactive.c, and made static again.

---
No changes since v2.

 builtin/rebase--interactive.c |   2 +-
 rebase-interactive.c          |  90 ++++++++++++++++++++++++-
 rebase-interactive.h          |   1 +
 sequencer.c                   | 120 +++++++---------------------------
 sequencer.h                   |   9 +--
 5 files changed, 115 insertions(+), 107 deletions(-)
builtin/rebase--interactive.c +1 -1
index abdf6126df..c1d87c0fe6 100644
@@ -255,7 +255,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = transform_todo_file(flags);
 		break;
 	case CHECK_TODO_LIST:
-		ret = check_todo_list();
+		ret = check_todo_list_from_file();
 		break;
 	case REARRANGE_SQUASH:
 		ret = rearrange_squash();
rebase-interactive.c +89 -1

100 lines not on this page · raw message

rebase-interactive.h +1 -0
index 971da03776..6bc7bc315d 100644
@@ -4,5 +4,6 @@
 void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 		      struct strbuf *buf);
 int edit_todo_list(unsigned flags);
+int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
 
 #endif
sequencer.c +23 -97

141 lines not on this page · raw message

sequencer.h +1 -8
index e69357c95b..c78f00aadd 100644
@@ -63,12 +63,6 @@ struct replay_opts {
 };
 #define REPLAY_OPTS_INIT { .action = -1, .current_fixups = STRBUF_INIT }
 
-enum missing_commit_check_level {
-	MISSING_COMMIT_CHECK_IGNORE = 0,
-	MISSING_COMMIT_CHECK_WARN,
-	MISSING_COMMIT_CHECK_ERROR
-};
-
 int write_message(const void *buf, size_t len, const char *filename,
 		  int append_eol);
 
@@ -148,8 +142,7 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 
 int sequencer_add_exec_commands(const char *command);
 int transform_todo_file(unsigned flags);
-enum missing_commit_check_level get_missing_commit_check_level(void);
-int check_todo_list(void);
+int check_todo_list_from_file(void);
 int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, const char *cmd,
Signature · 1 line
2.19.1.872.ga867da739e
#61 lore raw

PATCH v3 6/16 sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

In reply to Alban Gruin

This refactors sequencer_add_exec_commands() to work on a todo_list to avoid redundant reads and writes to the disk.

Instead of just inserting the `exec' command between the other commands, and re-parsing the buffer at the end the exec command is appended to the buffer once, and a new list of items is created. Items from the old list are copied across and new `exec' items are appended when necessary. This eliminates the need to reparse the buffer, but this also means we have to use todo_list_write_to_disk() to write the file().

todo_list_add_exec_commands() and sequencer_add_exec_commands() are modified to take a string list instead of a string -- one item for each command. This makes it easier to insert a new command to the todo list for each command to execute.

sequencer_add_exec_commands() still reads the todo list from the disk, as it is needed by rebase -p.

complete_action() still uses sequencer_add_exec_commands() for now. This will be changed in a future commit.

---
 builtin/rebase--interactive.c |  15 +++--
 sequencer.c                   | 111 +++++++++++++++++++++-------------
 sequencer.h                   |   4 +-
 3 files changed, 83 insertions(+), 47 deletions(-)
builtin/rebase--interactive.c +11 -4
index c1d87c0fe6..1fb5a43c0d 100644
@@ -65,7 +65,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 				 const char *onto, const char *onto_name,
 				 const char *squash_onto, const char *head_name,
 				 const char *restrict_revision, char *raw_strategies,
-				 const char *cmd, unsigned autosquash)
+				 struct string_list *commands, unsigned autosquash)
 {
 	int ret;
 	const char *head_hash = NULL;
@@ -115,7 +115,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	else {
 		discard_cache();
 		ret = complete_action(opts, flags, shortrevisions, onto_name, onto,
-				      head_hash, cmd, autosquash);
+				      head_hash, commands, autosquash);
 	}
 
 	free(revisions);
@@ -138,6 +138,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 	const char *onto = NULL, *onto_name = NULL, *restrict_revision = NULL,
 		*squash_onto = NULL, *upstream = NULL, *head_name = NULL,
 		*switch_to = NULL, *cmd = NULL;
+	struct string_list commands = STRING_LIST_INIT_DUP;
 	char *raw_strategies = NULL;
 	enum {
 		NONE = 0, CONTINUE, SKIP, EDIT_TODO, SHOW_CURRENT_PATCH,
@@ -220,6 +221,12 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		warning(_("--[no-]rebase-cousins has no effect without "
 			  "--rebase-merges"));
 
+	if (cmd && *cmd) {
+		string_list_split(&commands, cmd, '\n', -1);
+		if (strlen(commands.items[commands.nr - 1].string) == 0)
+			--commands.nr;
+	}
+
 	switch (command) {
 	case NONE:
 		if (!onto && !upstream)
@@ -227,7 +234,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 
 		ret = do_interactive_rebase(&opts, flags, switch_to, upstream, onto,
 					    onto_name, squash_onto, head_name, restrict_revision,
-					    raw_strategies, cmd, autosquash);
+					    raw_strategies, &commands, autosquash);
 		break;
 	case SKIP: {
 		struct string_list merge_rr = STRING_LIST_INIT_DUP;
@@ -261,7 +268,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = rearrange_squash();
 		break;
 	case ADD_EXEC:
-		ret = sequencer_add_exec_commands(cmd);
+		ret = sequencer_add_exec_commands(&commands);
 		break;
 	default:
 		BUG("invalid command '%d'", command);
sequencer.c +70 -41

147 lines not on this page · raw message

sequencer.h +2 -2
index c78f00aadd..d7b52044bd 100644
@@ -140,12 +140,12 @@ int sequencer_remove_state(struct replay_opts *opts);
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
-int sequencer_add_exec_commands(const char *command);
+int sequencer_add_exec_commands(struct string_list *commands);
 int transform_todo_file(unsigned flags);
 int check_todo_list_from_file(void);
 int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
-		    const char *onto, const char *orig_head, const char *cmd,
+		    const char *onto, const char *orig_head, struct string_list *commands,
 		    unsigned autosquash);
 int rearrange_squash(void);
 
Signature · 1 line
2.19.1.872.ga867da739e
#62 lore raw

Re: [PATCH v3 06/16] sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

Hi Alban

Sorry it has taken me a while to look at the latest iteration. I like the changes to pass a list of strings for the exec commands. I've only had a chance to take a quick look, but I've got a couple of comments below

Quoted 112 lines · Alban Gruin

This refactors sequencer_add_exec_commands() to work on a todo_list to avoid redundant reads and writes to the disk.

Instead of just inserting the `exec' command between the other commands, and re-parsing the buffer at the end the exec command is appended to the buffer once, and a new list of items is created. Items from the old list are copied across and new `exec' items are appended when necessary. This eliminates the need to reparse the buffer, but this also means we have to use todo_list_write_to_disk() to write the file().

todo_list_add_exec_commands() and sequencer_add_exec_commands() are modified to take a string list instead of a string -- one item for each command. This makes it easier to insert a new command to the todo list for each command to execute.

sequencer_add_exec_commands() still reads the todo list from the disk, as it is needed by rebase -p.

complete_action() still uses sequencer_add_exec_commands() for now. This will be changed in a future commit.

---
 builtin/rebase--interactive.c |  15 +++--
 sequencer.c                   | 111 +++++++++++++++++++++-------------
 sequencer.h                   |   4 +-
 3 files changed, 83 insertions(+), 47 deletions(-)
builtin/rebase--interactive.c +11 -4
index c1d87c0fe6..1fb5a43c0d 100644
@@ -65,7 +65,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 				 const char *onto, const char *onto_name,
 				 const char *squash_onto, const char *head_name,
 				 const char *restrict_revision, char *raw_strategies,
-				 const char *cmd, unsigned autosquash)
+				 struct string_list *commands, unsigned autosquash)
 {
 	int ret;
 	const char *head_hash = NULL;
@@ -115,7 +115,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	else {
 		discard_cache();
 		ret = complete_action(opts, flags, shortrevisions, onto_name, onto,
-				      head_hash, cmd, autosquash);
+				      head_hash, commands, autosquash);
 	}
 
 	free(revisions);
@@ -138,6 +138,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 	const char *onto = NULL, *onto_name = NULL, *restrict_revision = NULL,
 		*squash_onto = NULL, *upstream = NULL, *head_name = NULL,
 		*switch_to = NULL, *cmd = NULL;
+	struct string_list commands = STRING_LIST_INIT_DUP;
 	char *raw_strategies = NULL;
 	enum {
 		NONE = 0, CONTINUE, SKIP, EDIT_TODO, SHOW_CURRENT_PATCH,
@@ -220,6 +221,12 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		warning(_("--[no-]rebase-cousins has no effect without "
 			  "--rebase-merges"));
 
+	if (cmd && *cmd) {
+		string_list_split(&commands, cmd, '\n', -1);
+		if (strlen(commands.items[commands.nr - 1].string) == 0)
+			--commands.nr;
+	}
+
 	switch (command) {
 	case NONE:
 		if (!onto && !upstream)
@@ -227,7 +234,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 
 		ret = do_interactive_rebase(&opts, flags, switch_to, upstream, onto,
 					    onto_name, squash_onto, head_name, restrict_revision,
-					    raw_strategies, cmd, autosquash);
+					    raw_strategies, &commands, autosquash);
 		break;
 	case SKIP: {
 		struct string_list merge_rr = STRING_LIST_INIT_DUP;
@@ -261,7 +268,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = rearrange_squash();
 		break;
 	case ADD_EXEC:
-		ret = sequencer_add_exec_commands(cmd);
+		ret = sequencer_add_exec_commands(&commands);
 		break;
 	default:
 		BUG("invalid command '%d'", command);
sequencer.c +4 -10
index 900899ef20..11692d0b98 100644
@@ -4394,24 +4394,29 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	return 0;
 }
 
-/*
- * Add commands after pick and (series of) squash/fixup commands
- * in the todo list.
- */
-int sequencer_add_exec_commands(const char *commands)
+static void todo_list_add_exec_commands(struct todo_list *todo_list,
+					struct string_list *commands)
 {
-	const char *todo_file = rebase_path_todo();
-	struct todo_list todo_list = TODO_LIST_INIT;
-	struct strbuf *buf = &todo_list.buf;
-	size_t offset = 0, commands_len = strlen(commands);
-	int i, insert;
+	struct strbuf *buf = &todo_list->buf;
+	const char *old_buf = buf->buf;

sequencer.c int sequencer_make_script(FILE *out, int argc, const char **argv, · line 4409

Quoted 10 lines
sequencer.c +7 -2
+	size_t base_length = buf->len;
+	int i, insert, nr = 0, alloc = 0;
+	struct todo_item *items = NULL, *base_items = NULL;
 
-	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
-		return error(_("could not read '%s'."), todo_file);
+	for (i = 0; i < commands->nr; ++i) {
+		strbuf_addstr(buf, commands->items[i].string);
+		strbuf_addch(buf, '\n');
+	}

This could cause buf->buf to be reallocated in which case all the todo_list_item.arg pointers will be invalidated. I think you want to covert all those pointers to offsets stored in a temporary array and then recreate the correct pointers from the offsets after appending to buf.

sequencer.c int sequencer_make_script(FILE *out, int argc, const char **argv, · line 4416

Quoted 9 lines
sequencer.c +6 -3
-	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
-		todo_list_release(&todo_list);
-		return error(_("unusable todo list: '%s'"), todo_file);
+	base_items = xcalloc(commands->nr, sizeof(struct todo_item));
+	for (i = 0; i < commands->nr; ++i) {
+		size_t command_len = strlen(commands->items[i].string) - strlen("exec ");
+
+		base_items[i].command = TODO_EXEC;
+		base_items[i].offset_in_buf = base_length + strlen("exec ");

Currently parse_insn_buffer() sets offset_in_buf to the beginning of the
line, it does not skip the command name I think it would be good to keep this consistent with all the other commands.

sequencer.c int sequencer_make_script(FILE *out, int argc, const char **argv, · line 4419

Quoted 3 lines
sequencer.c +3 -0
+		base_items[i].arg = buf->buf + base_items[i].offset_in_buf;
+		base_items[i].arg_len = command_len;
+		base_length = base_items[i].offset_in_buf + base_items[i].arg_len + 1;

base_length is actually on offset into the buffer, it took me a while to figure out that the calculation is correct. It might be simpler to do

base_offset += strlen(commands->items[i].string) + 1;

Quoted 16 lines
sequencer.c +0 -9
 	}
 
 	/*
@@ -4420,38 +4425,62 @@ int sequencer_add_exec_commands(const char *commands)
 	 * those chains if there are any.
 	 */
 	insert = -1;
-	for (i = 0; i < todo_list.nr; i++) {
-		enum todo_command command = todo_list.items[i].command;
-
-		if (insert >= 0) {
-			/* skip fixup/squash chains */
-			if (command == TODO_COMMENT)
-				continue;
-			else if (is_fixup(command)) {
-				insert = i + 1;

sequencer.c int sequencer_add_exec_commands(const char *commands) · line 4431

Quoted 10 lines
sequencer.c +4 -6
-				continue;
-			}
-			strbuf_insert(buf,
-				      todo_list.items[insert].offset_in_buf +
-				      offset, commands, commands_len);
-			offset += commands_len;
+	for (i = 0; i < todo_list->nr; i++) {
+		enum todo_command command = todo_list->items[i].command;
+		if (todo_list->items[i].arg)
+			todo_list->items[i].arg = todo_list->items[i].arg - old_buf + buf->buf;

I think that is undefined behavior the old value of a pointer passed realloc() is undefined so cannot be used for pointer arithmetic. Using a temporary array of offsets as suggested above would avoid this.

Best Wishes

Phillip

Quoted 103 lines
sequencer.c +46 -11
+
+		if (insert >= 0 && command != TODO_COMMENT && !is_fixup(command)) {
+			ALLOC_GROW(items, nr + commands->nr, alloc);
+			COPY_ARRAY(items + nr, base_items, commands->nr);
+			nr += commands->nr;
 			insert = -1;
 		}
 
-		if (command == TODO_PICK || command == TODO_MERGE)
+		ALLOC_GROW(items, nr + 1, alloc);
+		items[nr++] = todo_list->items[i];
+
+		if (command == TODO_PICK || command == TODO_MERGE || is_fixup(command))
 			insert = i + 1;
 	}
 
 	/* insert or append final <commands> */
-	if (insert >= 0 && insert < todo_list.nr)
-		strbuf_insert(buf, todo_list.items[insert].offset_in_buf +
-			      offset, commands, commands_len);
-	else if (insert >= 0 || !offset)
-		strbuf_add(buf, commands, commands_len);
+	if (insert >= 0 || nr == todo_list->nr) {
+		ALLOC_GROW(items, nr + commands->nr, alloc);
+		COPY_ARRAY(items + nr, base_items, commands->nr);
+		nr += commands->nr;
+	}
+
+	free(base_items);
+	FREE_AND_NULL(todo_list->items);
+	todo_list->items = items;
+	todo_list->nr = nr;
+	todo_list->alloc = alloc;
+}
+
+/*
+ * Add commands after pick and (series of) squash/fixup commands
+ * in the todo list.
+ */
+int sequencer_add_exec_commands(struct string_list *commands)
+{
+	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res;
 
-	i = write_message(buf->buf, buf->len, todo_file, 0);
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
+		return error(_("could not read '%s'."), todo_file);
+
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
+		todo_list_release(&todo_list);
+		return error(_("unusable todo list: '%s'"), todo_file);
+	}
+
+	todo_list_add_exec_commands(&todo_list, commands);
+	res = todo_list_write_to_file(&todo_list, todo_file, NULL, NULL, -1, 0);
 	todo_list_release(&todo_list);
-	return i;
+
+	return res;
 }
 
 static void todo_list_to_strbuf(struct todo_list *todo_list, struct strbuf *buf,
@@ -4677,7 +4706,7 @@ static int skip_unnecessary_picks(struct object_id *output_oid)
 
 int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
-		    const char *onto, const char *orig_head, const char *cmd,
+		    const char *onto, const char *orig_head, struct string_list *commands,
 		    unsigned autosquash)
 {
 	const char *shortonto, *todo_file = rebase_path_todo();
@@ -4696,8 +4725,8 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 	if (autosquash && rearrange_squash())
 		return -1;
 
-	if (cmd && *cmd)
-		sequencer_add_exec_commands(cmd);
+	if (commands->nr)
+		sequencer_add_exec_commands(commands);
 
 	if (strbuf_read_file(buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
sequencer.h +2 -2
index c78f00aadd..d7b52044bd 100644
@@ -140,12 +140,12 @@ int sequencer_remove_state(struct replay_opts *opts);
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
-int sequencer_add_exec_commands(const char *command);
+int sequencer_add_exec_commands(struct string_list *commands);
 int transform_todo_file(unsigned flags);
 int check_todo_list_from_file(void);
 int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
-		    const char *onto, const char *orig_head, const char *cmd,
+		    const char *onto, const char *orig_head, struct string_list *commands,
 		    unsigned autosquash);
 int rearrange_squash(void);
 
#63 lore raw

Re: [PATCH v3 06/16] sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

Hi,

Quoted 28 lines · Phillip Wood
Quoted 28 lines
sequencer.c +7 -10
index 900899ef20..11692d0b98 100644
@@ -4394,24 +4394,29 @@ int sequencer_make_script(FILE *out, int argc, const
 char **argv,
 	return 0;
 }
 
-/*
- * Add commands after pick and (series of) squash/fixup commands
- * in the todo list.
- */
-int sequencer_add_exec_commands(const char *commands)
+static void todo_list_add_exec_commands(struct todo_list *todo_list,
+					struct string_list *commands)
 {
-	const char *todo_file = rebase_path_todo();
-	struct todo_list todo_list = TODO_LIST_INIT;
-	struct strbuf *buf = &todo_list.buf;
-	size_t offset = 0, commands_len = strlen(commands);
-	int i, insert;
+	struct strbuf *buf = &todo_list->buf;
+	const char *old_buf = buf->buf;
+	size_t base_length = buf->len;
+	int i, insert, nr = 0, alloc = 0;
+	struct todo_item *items = NULL, *base_items = NULL;
 
Quoted 9 lines

sequencer.c int sequencer_make_script(FILE *out, int argc, const char **argv, · line 4409

Quoted 6 lines
sequencer.c +4 -2
-	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
-		return error(_("could not read '%s'."), todo_file);
+	for (i = 0; i < commands->nr; ++i) {
+		strbuf_addstr(buf, commands->items[i].string);
+		strbuf_addch(buf, '\n');
+	}

This could cause buf->buf to be reallocated in which case all the todo_list_item.arg pointers will be invalidated.

I guess it is a good time for me to admit that the arg idea was not my best. Maybe it would be good to convert that from a pointer to an offset, e.g. size_t arg_offset_in_buf;? Or maybe we should just drop the _in_buf suffix and clarify in a comment next to the declaration of the fields that they are offsets in the strbuf?

Ciao,
Dscho

#64 lore raw

Re: [PATCH v3 06/16] sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

Quoted 42 lines · Johannes Schindelin

Hi,

Quoted 28 lines · Phillip Wood
Quoted 28 lines
sequencer.c +7 -10
index 900899ef20..11692d0b98 100644
@@ -4394,24 +4394,29 @@ int sequencer_make_script(FILE *out, int argc, const
 char **argv,
  	return 0;
  }
 
-/*
- * Add commands after pick and (series of) squash/fixup commands
- * in the todo list.
- */
-int sequencer_add_exec_commands(const char *commands)
+static void todo_list_add_exec_commands(struct todo_list *todo_list,
+					struct string_list *commands)
 {
-	const char *todo_file = rebase_path_todo();
-	struct todo_list todo_list = TODO_LIST_INIT;
-	struct strbuf *buf = &todo_list.buf;
-	size_t offset = 0, commands_len = strlen(commands);
-	int i, insert;
+	struct strbuf *buf = &todo_list->buf;
+	const char *old_buf = buf->buf;
+	size_t base_length = buf->len;
+	int i, insert, nr = 0, alloc = 0;
+	struct todo_item *items = NULL, *base_items = NULL;
 
Quoted 9 lines

sequencer.c int sequencer_make_script(FILE *out, int argc, const char **argv, · line 4409

Quoted 6 lines
sequencer.c +4 -2
-	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
-		return error(_("could not read '%s'."), todo_file);
+	for (i = 0; i < commands->nr; ++i) {
+		strbuf_addstr(buf, commands->items[i].string);
+		strbuf_addch(buf, '\n');
+	}

This could cause buf->buf to be reallocated in which case all the todo_list_item.arg pointers will be invalidated.

Quoted 5 lines

I guess it is a good time for me to admit that the arg idea was not my best. Maybe it would be good to convert that from a pointer to an offset, e.g. size_t arg_offset_in_buf;? Or maybe we should just drop the _in_buf suffix and clarify in a comment next to the declaration of the fields that they are offsets in the strbuf?

I think that sounds sensible (though I haven't looked at how much work it would be), having a comment and calling it arg_offset would be my preference.

Best Wishes

Phillip

Quoted 4 lines

Ciao,
Dscho

#65 lore raw

Re: [PATCH v3 06/16] sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

Hi Phillip and Johannes,

many thanks for your suggestions and feedback, I really appreciate it.

Le 10/12/2018 à 15:33, Phillip Wood a écrit :

Quoted 49 lines · Johannes Schindelin
Quoted 42 lines

Hi,

Quoted 28 lines · Phillip Wood
Quoted 28 lines
sequencer.c +0 -0
index 900899ef20..11692d0b98 100644
@@ -4394,24 +4394,29 @@ int sequencer_make_script(FILE *out, int

argc, const
char **argv,
      return 0;
  }
  -/*
- * Add commands after pick and (series of) squash/fixup commands
- * in the todo list.
- */

sequencer.c +2 -1
-int sequencer_add_exec_commands(const char *commands)
+static void todo_list_add_exec_commands(struct todo_list *todo_list,
+                    struct string_list *commands)

{
-    const char *todo_file = rebase_path_todo(); -    struct todo_list todo_list = TODO_LIST_INIT; -    struct strbuf *buf = &todo_list.buf;
-    size_t offset = 0, commands_len = strlen(commands); -    int i, insert;
+    struct strbuf *buf = &todo_list->buf;
+    const char *old_buf = buf->buf;
+    size_t base_length = buf->len;
+    int i, insert, nr = 0, alloc = 0;
+    struct todo_item *items = NULL, *base_items = NULL;

Quoted 9 lines
Quoted 6 lines

-    if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0) -        return error(_("could not read '%s'."), todo_file); +    for (i = 0; i < commands->nr; ++i) {
+        strbuf_addstr(buf, commands->items[i].string); +        strbuf_addch(buf, '\n');
+    }

This could cause buf->buf to be reallocated in which case all the todo_list_item.arg pointers will be invalidated.

Quoted 5 lines

I guess it is a good time for me to admit that the arg idea was not my best. Maybe it would be good to convert that from a pointer to an offset, e.g. size_t arg_offset_in_buf;? Or maybe we should just drop the _in_buf suffix and clarify in a comment next to the declaration of the fields that they are offsets in the strbuf?

Quoted 4 lines

I think that sounds sensible (though I haven't looked at how much work it would be), having a comment and calling it arg_offset would be my preference.

It’s not a lot of work, actually. Most usages of 'arg' are concentrated in two functions (parse_insn_line() and pick_commits()). Some of the subsequent patches of this series also use 'arg', and adapting them is trivial.

In the end, most of the work went into todo_list_add_exec_commands(), and the result is pretty clean.

Cheers,
Alban

#66 lore raw

PATCH v3 7/16 sequencer: refactor rearrange_squash() to work on a todo_list

In reply to Alban Gruin

This refactors rearrange_squash() to work on a todo_list to avoid redundant reads and writes. The function is renamed todo_list_rearrange_squash().

The old version created a new buffer, which was directly written to the disk. This new version creates a new item list by just memcpying items from the old item list, without creating a new buffer. This eliminates the need to reparse the todo list, but this also means its buffer cannot be directly written to the disk.

As rebase -p still need to check the todo list from the disk, a new function is introduced, rearrange_squash_in_todo_file().

complete_action() still uses rearrange_squash_in_todo_file() for now. This will be changed in a future commit.

---
 builtin/rebase--interactive.c |  2 +-
 sequencer.c                   | 87 +++++++++++++++++------------------
 sequencer.h                   |  2 +-
 3 files changed, 45 insertions(+), 46 deletions(-)
builtin/rebase--interactive.c +1 -1
index 1fb5a43c0d..a6d83a684e 100644
@@ -265,7 +265,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = check_todo_list_from_file();
 		break;
 	case REARRANGE_SQUASH:
-		ret = rearrange_squash();
+		ret = rearrange_squash_in_todo_file();
 		break;
 	case ADD_EXEC:
 		ret = sequencer_add_exec_commands(&commands);
sequencer.c +43 -44

152 lines not on this page · raw message

sequencer.h +1 -1
index d7b52044bd..1cd9d7f1e2 100644
@@ -147,7 +147,7 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, struct string_list *commands,
 		    unsigned autosquash);
-int rearrange_squash(void);
+int rearrange_squash_in_todo_file(void);
 
 extern const char sign_off_header[];
 
Signature · 1 line
2.19.1.872.ga867da739e
#67 lore raw

PATCH v3 8/16 sequencer: make sequencer_make_script() write its script to a strbuf

In reply to Alban Gruin

This makes sequencer_make_script() write its script to a strbuf (ie. the buffer of a todo_list) instead of a FILE. This reduce the amount of read/write made by rebase interactive.

---
 builtin/rebase--interactive.c | 13 +++++++-----
 sequencer.c                   | 38 ++++++++++++++++-------------------
 sequencer.h                   |  2 +-
 3 files changed, 26 insertions(+), 27 deletions(-)
builtin/rebase--interactive.c +8 -5
index a6d83a684e..c740a7dd5d 100644
@@ -71,7 +71,8 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	const char *head_hash = NULL;
 	char *revisions = NULL, *shortrevisions = NULL;
 	struct argv_array make_script_args = ARGV_ARRAY_INIT;
-	FILE *todo_list;
+	FILE *todo_list_file;
+	struct todo_list todo_list = TODO_LIST_INIT;
 
 	if (prepare_branch_to_be_rebased(opts, switch_to))
 		return -1;
@@ -93,8 +94,8 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	if (!upstream && squash_onto)
 		write_file(path_squash_onto(), "%s\n", squash_onto);
 
-	todo_list = fopen(rebase_path_todo(), "w");
-	if (!todo_list) {
+	todo_list_file = fopen(rebase_path_todo(), "w");
+	if (!todo_list_file) {
 		free(revisions);
 		free(shortrevisions);
 
@@ -105,10 +106,11 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	if (restrict_revision)
 		argv_array_push(&make_script_args, restrict_revision);
 
-	ret = sequencer_make_script(todo_list,
+	ret = sequencer_make_script(&todo_list.buf,
 				    make_script_args.argc, make_script_args.argv,
 				    flags);
-	fclose(todo_list);
+	fputs(todo_list.buf.buf, todo_list_file);
+	fclose(todo_list_file);
 
 	if (ret)
 		error(_("could not generate todo list"));
@@ -120,6 +122,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 
 	free(revisions);
 	free(shortrevisions);
+	todo_list_release(&todo_list);
 	argv_array_clear(&make_script_args);
 
 	return ret;
sequencer.c +17 -21
index fce97e5f11..3389a753b6 100644
@@ -4109,7 +4109,7 @@ static const char *label_oid(struct object_id *oid, const char *label,
 }
 
 static int make_script_with_merges(struct pretty_print_context *pp,
-				   struct rev_info *revs, FILE *out,
+				   struct rev_info *revs, struct strbuf *out,
 				   unsigned flags)
 {
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
@@ -4254,7 +4254,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 	 * gathering commits not yet shown, reversing the list on the fly,
 	 * then outputting that list (labeling revisions as needed).
 	 */
-	fprintf(out, "%s onto\n", cmd_label);
+	strbuf_addf(out, "%s onto\n", cmd_label);
 	for (iter = tips; iter; iter = iter->next) {
 		struct commit_list *list = NULL, *iter2;
 
@@ -4264,9 +4264,9 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 		entry = oidmap_get(&state.commit2label, &commit->object.oid);
 
 		if (entry)
-			fprintf(out, "\n%c Branch %s\n", comment_line_char, entry->string);
+			strbuf_addf(out, "\n%c Branch %s\n", comment_line_char, entry->string);
 		else
-			fprintf(out, "\n");
+			strbuf_addch(out, '\n');
 
 		while (oidset_contains(&interesting, &commit->object.oid) &&
 		       !oidset_contains(&shown, &commit->object.oid)) {
@@ -4279,8 +4279,8 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 		}
 
 		if (!commit)
-			fprintf(out, "%s %s\n", cmd_reset,
-				rebase_cousins ? "onto" : "[new root]");
+			strbuf_addf(out, "%s %s\n", cmd_reset,
+				    rebase_cousins ? "onto" : "[new root]");
 		else {
 			const char *to = NULL;
 
@@ -4293,12 +4293,12 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 					       &state);
 
 			if (!to || !strcmp(to, "onto"))
-				fprintf(out, "%s onto\n", cmd_reset);
+				strbuf_addf(out, "%s onto\n", cmd_reset);
 			else {
 				strbuf_reset(&oneline);
 				pretty_print_commit(pp, commit, &oneline);
-				fprintf(out, "%s %s # %s\n",
-					cmd_reset, to, oneline.buf);
+				strbuf_addf(out, "%s %s # %s\n",
+					    cmd_reset, to, oneline.buf);
 			}
 		}
 
@@ -4307,11 +4307,11 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 			entry = oidmap_get(&commit2todo, oid);
 			/* only show if not already upstream */
 			if (entry)
-				fprintf(out, "%s\n", entry->string);
+				strbuf_addf(out, "%s\n", entry->string);
 			entry = oidmap_get(&state.commit2label, oid);
 			if (entry)
-				fprintf(out, "%s %s\n",
-					cmd_label, entry->string);
+				strbuf_addf(out, "%s %s\n",
+					    cmd_label, entry->string);
 			oidset_insert(&shown, oid);
 		}
 
@@ -4333,12 +4333,11 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 	return 0;
 }
 
-int sequencer_make_script(FILE *out, int argc, const char **argv,
+int sequencer_make_script(struct strbuf *out, int argc, const char **argv,
 			  unsigned flags)
 {
 	char *format = NULL;
 	struct pretty_print_context pp = {0};
-	struct strbuf buf = STRBUF_INIT;
 	struct rev_info revs;
 	struct commit *commit;
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
@@ -4381,16 +4380,13 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 
 		if (!is_empty && (commit->object.flags & PATCHSAME))
 			continue;
-		strbuf_reset(&buf);
 		if (!keep_empty && is_empty)
-			strbuf_addf(&buf, "%c ", comment_line_char);
-		strbuf_addf(&buf, "%s %s ", insn,
+			strbuf_addf(out, "%c ", comment_line_char);
+		strbuf_addf(out, "%s %s ", insn,
 			    oid_to_hex(&commit->object.oid));
-		pretty_print_commit(&pp, commit, &buf);
-		strbuf_addch(&buf, '\n');
-		fputs(buf.buf, out);
+		pretty_print_commit(&pp, commit, out);
+		strbuf_addch(out, '\n');
 	}
-	strbuf_release(&buf);
 	return 0;
 }
 
sequencer.h +1 -1
index 1cd9d7f1e2..fcde6e739d 100644
@@ -137,7 +137,7 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_REBASE_COUSINS (1U << 4)
 #define TODO_LIST_APPEND_TODO_HELP (1U << 5)
 
-int sequencer_make_script(FILE *out, int argc, const char **argv,
+int sequencer_make_script(struct strbuf *out, int argc, const char **argv,
 			  unsigned flags);
 
 int sequencer_add_exec_commands(struct string_list *commands);
Signature · 1 line
2.19.1.872.ga867da739e
#68 lore raw

PATCH v3 9/16 sequencer: change complete_action() to use the refactored functions

In reply to Alban Gruin

complete_action() used functions that read the todo-list file, made some changes to it, and wrote it back to the disk.

The previous commits were dedicated to separate the part that deals with the file from the actual logic of these functions. Now that this is done, we can call directly the "logic" functions to avoid useless file access.

The parsing of the list has to be done by the caller. If the buffer of the todo list provided by the caller is empty, a `noop' command is directly added to the todo list, without touching to the buffer.

---
 builtin/rebase--interactive.c | 16 ++-----
 sequencer.c                   | 80 +++++++++++++++--------------------
 sequencer.h                   |  2 +-
 3 files changed, 40 insertions(+), 58 deletions(-)
builtin/rebase--interactive.c +4 -12
index c740a7dd5d..99cbd1e8e3 100644
@@ -71,7 +71,6 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	const char *head_hash = NULL;
 	char *revisions = NULL, *shortrevisions = NULL;
 	struct argv_array make_script_args = ARGV_ARRAY_INIT;
-	FILE *todo_list_file;
 	struct todo_list todo_list = TODO_LIST_INIT;
 
 	if (prepare_branch_to_be_rebased(opts, switch_to))
@@ -94,14 +93,6 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	if (!upstream && squash_onto)
 		write_file(path_squash_onto(), "%s\n", squash_onto);
 
-	todo_list_file = fopen(rebase_path_todo(), "w");
-	if (!todo_list_file) {
-		free(revisions);
-		free(shortrevisions);
-
-		return error_errno(_("could not open %s"), rebase_path_todo());
-	}
-
 	argv_array_pushl(&make_script_args, "", revisions, NULL);
 	if (restrict_revision)
 		argv_array_push(&make_script_args, restrict_revision);
@@ -109,15 +100,16 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	ret = sequencer_make_script(&todo_list.buf,
 				    make_script_args.argc, make_script_args.argv,
 				    flags);
-	fputs(todo_list.buf.buf, todo_list_file);
-	fclose(todo_list_file);
 
 	if (ret)
 		error(_("could not generate todo list"));
 	else {
 		discard_cache();
+		if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list))
+			BUG("unusable todo list");
+
 		ret = complete_action(opts, flags, shortrevisions, onto_name, onto,
-				      head_hash, commands, autosquash);
+				      head_hash, commands, autosquash, &todo_list);
 	}
 
 	free(revisions);
sequencer.c +35 -45

128 lines not on this page · raw message

sequencer.h +1 -1
index fcde6e739d..996e8ea2a6 100644
@@ -146,7 +146,7 @@ int check_todo_list_from_file(void);
 int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, struct string_list *commands,
-		    unsigned autosquash);
+		    unsigned autosquash, struct todo_list *todo_list);
 int rearrange_squash_in_todo_file(void);
 
 extern const char sign_off_header[];
Signature · 1 line
2.19.1.872.ga867da739e
#69 lore raw

PATCH v3 10/16 sequencer: refactor skip_unnecessary_picks() to work on a todo_list

In reply to Alban Gruin

This refactors skip_unnecessary_picks() to work on a todo_list. The file-handling logic is completely dropped here, as its only usage is made by complete_action().

Instead of truncating the todo list’s buffer, the items are moved to the beginning of the list, eliminating the need to reparse the list. This also means its buffer cannot be directly written to the disk.

rewrite_file() is then removed, as it is now unused.

---
 sequencer.c | 79 ++++++++++++-----------------------------------------
 1 file changed, 17 insertions(+), 62 deletions(-)
sequencer.c +17 -62
index 64a99ab84f..1c405763c3 100644
@@ -4607,52 +4607,21 @@ int check_todo_list_from_file(void)
 	return res;
 }
 
-static int rewrite_file(const char *path, const char *buf, size_t len)
-{
-	int rc = 0;
-	int fd = open(path, O_WRONLY | O_TRUNC);
-	if (fd < 0)
-		return error_errno(_("could not open '%s' for writing"), path);
-	if (write_in_full(fd, buf, len) < 0)
-		rc = error_errno(_("could not write to '%s'"), path);
-	if (close(fd) && !rc)
-		rc = error_errno(_("could not close '%s'"), path);
-	return rc;
-}
-
 /* skip picking commits whose parents are unchanged */
-static int skip_unnecessary_picks(struct object_id *output_oid)
+static int skip_unnecessary_picks(struct todo_list *todo_list,
+				  struct object_id *output_oid)
 {
-	const char *todo_file = rebase_path_todo();
-	struct strbuf buf = STRBUF_INIT;
-	struct todo_list todo_list = TODO_LIST_INIT;
 	struct object_id *parent_oid;
-	int fd, i;
-
-	if (!read_oneliner(&buf, rebase_path_onto(), 0))
-		return error(_("could not read 'onto'"));
-	if (get_oid(buf.buf, output_oid)) {
-		strbuf_release(&buf);
-		return error(_("need a HEAD to fixup"));
-	}
-	strbuf_release(&buf);
-
-	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
-		return -1;
-	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {
-		todo_list_release(&todo_list);
-		return -1;
-	}
+	int i;
 
-	for (i = 0; i < todo_list.nr; i++) {
-		struct todo_item *item = todo_list.items + i;
+	for (i = 0; i < todo_list->nr; i++) {
+		struct todo_item *item = todo_list->items + i;
 
 		if (item->command >= TODO_NOOP)
 			continue;
 		if (item->command != TODO_PICK)
 			break;
 		if (parse_commit(item->commit)) {
-			todo_list_release(&todo_list);
 			return error(_("could not parse commit '%s'"),
 				oid_to_hex(&item->commit->object.oid));
 		}
@@ -4666,37 +4635,21 @@ static int skip_unnecessary_picks(struct object_id *output_oid)
 		oidcpy(output_oid, &item->commit->object.oid);
 	}
 	if (i > 0) {
-		int offset = get_item_line_offset(&todo_list, i);
 		const char *done_path = rebase_path_done();
 
-		fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
-		if (fd < 0) {
-			error_errno(_("could not open '%s' for writing"),
-				    done_path);
-			todo_list_release(&todo_list);
-			return -1;
-		}
-		if (write_in_full(fd, todo_list.buf.buf, offset) < 0) {
+		if (todo_list_write_to_file(todo_list, done_path, NULL, NULL, i, 0)) {
 			error_errno(_("could not write to '%s'"), done_path);
-			todo_list_release(&todo_list);
-			close(fd);
 			return -1;
 		}
-		close(fd);
 
-		if (rewrite_file(rebase_path_todo(), todo_list.buf.buf + offset,
-				 todo_list.buf.len - offset) < 0) {
-			todo_list_release(&todo_list);
-			return -1;
-		}
+		MOVE_ARRAY(todo_list->items, todo_list->items + i, todo_list->nr - i);
+		todo_list->nr -= i;
+		todo_list->current = 0;
 
-		todo_list.current = i;
-		if (is_fixup(peek_command(&todo_list, 0)))
-			record_in_rewritten(output_oid, peek_command(&todo_list, 0));
+		if (is_fixup(peek_command(todo_list, 0)))
+			record_in_rewritten(output_oid, peek_command(todo_list, 0));
 	}
 
-	todo_list_release(&todo_list);
-
 	return 0;
 }
 
@@ -4770,6 +4723,11 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 		return -1;
 	}
 
+	if (opts->allow_ff && skip_unnecessary_picks(&new_todo, &oid)) {
+		todo_list_release(&new_todo);
+		return error(_("could not skip unnecessary pick commands"));
+	}
+
 	if (todo_list_write_to_file(&new_todo, todo_file, NULL, NULL, -1,
 				    flags & ~(TODO_LIST_SHORTEN_IDS))) {
 		todo_list_release(&new_todo);
@@ -4778,12 +4736,9 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 
 	todo_list_release(&new_todo);
 
-	if (opts->allow_ff && skip_unnecessary_picks(&oid))
-		return error(_("could not skip unnecessary pick commands"));
-
 	if (checkout_onto(opts, onto_name, oid_to_hex(&oid), orig_head))
 		return -1;
-;
+
 	if (require_clean_work_tree("rebase", "", 1, 1))
 		return -1;
 
Signature · 1 line
2.19.1.872.ga867da739e
#70 lore raw

PATCH v3 11/16 rebase-interactive: use todo_list_write_to_file() in edit_todo_list()

In reply to Alban Gruin

Just like complete_action(), edit_todo_list() used a function (transform_todo_file()) that read the todo list from the disk and wrote it back, resulting in useless disk accesses.

This changes edit_todo_list() to call directly todo_list_write_to_file() instead.

---
 rebase-interactive.c | 38 ++++++++++++++++----------------------
 1 file changed, 16 insertions(+), 22 deletions(-)
rebase-interactive.c +16 -22
index 3adcf39e07..23569cfa3c 100644
@@ -79,39 +79,33 @@ void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 
 int edit_todo_list(unsigned flags)
 {
-	struct strbuf buf = STRBUF_INIT;
 	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res = 0;
 
-	if (strbuf_read_file(&buf, todo_file, 0) < 0)
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
 
-	strbuf_stripspace(&buf, 1);
-	if (write_message(buf.buf, buf.len, todo_file, 0)) {
-		strbuf_release(&buf);
+	strbuf_stripspace(&todo_list.buf, 1);
+	todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list);
+	if (todo_list_write_to_file(&todo_list, todo_file, NULL, NULL, -1,
+				    flags | TODO_LIST_SHORTEN_IDS | TODO_LIST_APPEND_TODO_HELP)) {
+		todo_list_release(&todo_list);
 		return -1;
 	}
 
-	strbuf_release(&buf);
-
-	transform_todo_file(flags | TODO_LIST_SHORTEN_IDS);
-
-	if (strbuf_read_file(&buf, todo_file, 0) < 0)
-		return error_errno(_("could not read '%s'."), todo_file);
-
-	append_todo_help(1, 0, &buf);
-	if (write_message(buf.buf, buf.len, todo_file, 0)) {
-		strbuf_release(&buf);
+	strbuf_reset(&todo_list.buf);
+	if (launch_sequence_editor(todo_file, &todo_list.buf, NULL)) {
+		todo_list_release(&todo_list);
 		return -1;
 	}
 
-	strbuf_release(&buf);
+	if (!todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list))
+		res = todo_list_write_to_file(&todo_list, todo_file, NULL, NULL, -1,
+					      flags & ~(TODO_LIST_SHORTEN_IDS));
 
-	if (launch_sequence_editor(todo_file, NULL, NULL))
-		return -1;
-
-	transform_todo_file(flags & ~(TODO_LIST_SHORTEN_IDS));
-
-	return 0;
+	todo_list_release(&todo_list);
+	return res;
 }
 
 define_commit_slab(commit_seen, unsigned char);
Signature · 1 line
2.19.1.872.ga867da739e
#71 lore raw

PATCH v3 12/16 rebase-interactive: append_todo_help() changes

In reply to Alban Gruin

This moves the writing of the comment "Rebase $shortrevisions onto $shortonto ($command_count commands)" from complete_action() to append_todo_help().

shortrevisions, shortonto, and command_count are passed as parameters to append_todo_help().

During the initial edit of the todo list, shortrevisions and shortonto are not NULL. Therefore, if shortrevisions or shortonto is NULL, then edit_todo would be true, otherwise it would be false. Thus, edit_todo is removed from the parameters of append_todo_help().

edit_todo_list() and complete_action() are modified to fit these changes.

---
 rebase-interactive.c | 12 +++++++++++-
 rebase-interactive.h |  3 ++-
 sequencer.c          | 17 ++++-------------
 3 files changed, 17 insertions(+), 15 deletions(-)
rebase-interactive.c +11 -1
index 23569cfa3c..3b7b5e3382 100644
@@ -28,7 +28,8 @@ static enum missing_commit_check_level get_missing_commit_check_level(void)
 	return MISSING_COMMIT_CHECK_IGNORE;
 }
 
-void append_todo_help(unsigned edit_todo, unsigned keep_empty,
+void append_todo_help(unsigned keep_empty, int command_count,
+		      const char *shortrevisions, const char *shortonto,
 		      struct strbuf *buf)
 {
 	const char *msg = _("\nCommands:\n"
@@ -48,6 +49,15 @@ void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 ".       specified). Use -c <commit> to reword the commit message.\n"
 "\n"
 "These lines can be re-ordered; they are executed from top to bottom.\n");
+	unsigned edit_todo = !(shortrevisions && shortonto);
+
+	if (!edit_todo) {
+		strbuf_addch(buf, '\n');
+		strbuf_commented_addf(buf, Q_("Rebase %s onto %s (%d command)",
+					      "Rebase %s onto %s (%d commands)",
+					      command_count),
+				      shortrevisions, shortonto, command_count);
+	}
 
 	strbuf_add_commented_lines(buf, msg, strlen(msg));
 
rebase-interactive.h +2 -1
index 6bc7bc315d..61858f3a02 100644
@@ -1,7 +1,8 @@
 #ifndef REBASE_INTERACTIVE_H
 #define REBASE_INTERACTIVE_H
 
-void append_todo_help(unsigned edit_todo, unsigned keep_empty,
+void append_todo_help(unsigned keep_empty, int command_count,
+		      const char *shortrevisions, const char *shortonto,
 		      struct strbuf *buf);
 int edit_todo_list(unsigned flags);
 int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
sequencer.c +4 -13
index 1c405763c3..cf6f69c93e 100644
@@ -4529,22 +4529,13 @@ int todo_list_write_to_file(struct todo_list *todo_list, const char *file,
 			    const char *shortrevisions, const char *shortonto,
 			    int num, unsigned flags)
 {
-	int edit_todo = !(shortrevisions && shortonto), res;
+	int res;
 	struct strbuf buf = STRBUF_INIT;
 
 	todo_list_to_strbuf(todo_list, &buf, num, flags);
-
-	if (flags & TODO_LIST_APPEND_TODO_HELP) {
-		int command_count = count_commands(todo_list);
-		if (!edit_todo) {
-			strbuf_addch(&buf, '\n');
-			strbuf_commented_addf(&buf, Q_("Rebase %s onto %s (%d command)",
-						       "Rebase %s onto %s (%d commands)",
-						       command_count),
-					      shortrevisions, shortonto, command_count);
-		}
-		append_todo_help(edit_todo, flags & TODO_LIST_KEEP_EMPTY, &buf);
-	}
+	if (flags & TODO_LIST_APPEND_TODO_HELP)
+		append_todo_help(flags & TODO_LIST_KEEP_EMPTY, count_commands(todo_list),
+				 shortrevisions, shortonto, &buf);
 
 	res = write_message(buf.buf, buf.len, file, 0);
 	strbuf_release(&buf);
Signature · 1 line
2.19.1.872.ga867da739e
#72 lore raw

PATCH v3 13/16 rebase-interactive: rewrite edit_todo_list() to handle the initial edit

In reply to Alban Gruin

edit_todo_list() is changed to work on a todo_list, and to handle the initial edition of the todo list (ie. making a backup of the todo list).

It does not check for dropped commits yet, as todo_list_check() does not take the commits that have already been processed by the rebase (ie. the todo list is edited in the middle of a rebase session).

---
 builtin/rebase--interactive.c | 23 ++++++++++++++++-
 rebase-interactive.c          | 48 ++++++++++++++++++-----------------
 rebase-interactive.h          |  4 ++-
 sequencer.c                   |  3 +--
 sequencer.h                   |  1 +
 5 files changed, 52 insertions(+), 27 deletions(-)
builtin/rebase--interactive.c +22 -1
index 99cbd1e8e3..6871990544 100644
@@ -13,6 +13,27 @@ static GIT_PATH_FUNC(path_state_dir, "rebase-merge/")
 static GIT_PATH_FUNC(path_squash_onto, "rebase-merge/squash-onto")
 static GIT_PATH_FUNC(path_interactive, "rebase-merge/interactive")
 
+static int edit_todo_file(unsigned flags)
+{
+	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT,
+		new_todo = TODO_LIST_INIT;
+
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
+		return error_errno(_("could not read '%s'."), todo_file);
+
+	strbuf_stripspace(&todo_list.buf, 1);
+	if (!edit_todo_list(&todo_list, &new_todo, NULL, NULL, flags) &&
+	    todo_list_write_to_file(&new_todo, todo_file, NULL, NULL, -1,
+				    flags & ~(TODO_LIST_SHORTEN_IDS)) < 0)
+		return error_errno(_("could not write '%s'"), todo_file);
+
+	todo_list_release(&todo_list);
+	todo_list_release(&new_todo);
+
+	return 0;
+}
+
 static int get_revision_ranges(const char *upstream, const char *onto,
 			       const char **head_hash,
 			       char **revisions, char **shortrevisions)
@@ -241,7 +262,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		break;
 	}
 	case EDIT_TODO:
-		ret = edit_todo_list(flags);
+		ret = edit_todo_file(flags);
 		break;
 	case SHOW_CURRENT_PATCH: {
 		struct child_process cmd = CHILD_PROCESS_INIT;
rebase-interactive.c +25 -23

60 lines not on this page · raw message

rebase-interactive.h +3 -1
index 61858f3a02..8d5c04521e 100644
@@ -4,7 +4,9 @@
 void append_todo_help(unsigned keep_empty, int command_count,
 		      const char *shortrevisions, const char *shortonto,
 		      struct strbuf *buf);
-int edit_todo_list(unsigned flags);
+int edit_todo_list(struct todo_list *todo_list, struct todo_list *new_todo,
+		   const char *shortrevisions, const char *shortonto,
+		   unsigned flags);
 int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
 
 #endif
sequencer.c +1 -2
index cf6f69c93e..f04b002e37 100644
@@ -55,8 +55,7 @@ static GIT_PATH_FUNC(rebase_path, "rebase-merge")
  * file and written to the tail of 'done'.
  */
 GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
-static GIT_PATH_FUNC(rebase_path_todo_backup,
-		     "rebase-merge/git-rebase-todo.backup")
+GIT_PATH_FUNC(rebase_path_todo_backup, "rebase-merge/git-rebase-todo.backup")
 
 /*
  * The rebase command lines that have already been processed. A line
sequencer.h +1 -0
index 996e8ea2a6..f6751d53b9 100644
@@ -9,6 +9,7 @@ struct commit;
 const char *git_path_commit_editmsg(void);
 const char *git_path_seq_dir(void);
 const char *rebase_path_todo(void);
+const char *rebase_path_todo_backup(void);
 
 #define APPEND_SIGNOFF_DEDUP (1u << 0)
 
Signature · 1 line
2.19.1.872.ga867da739e
#73 lore raw

PATCH v3 14/16 sequencer: use edit_todo_list() in complete_action()

In reply to Alban Gruin

This changes complete_action() to use edit_todo_list(), now that it can handle the initial edit of the todo list.

---
 sequencer.c | 20 ++++++--------------
 1 file changed, 6 insertions(+), 14 deletions(-)
sequencer.c +6 -14
index f04b002e37..02afd2f5cd 100644
@@ -4654,6 +4654,7 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 	struct todo_list new_todo = TODO_LIST_INIT;
 	struct strbuf *buf = &todo_list->buf;
 	struct object_id oid;
+	int res;
 
 	get_oid(onto, &oid);
 	shortonto = find_unique_abbrev(&oid, DEFAULT_ABBREV);
@@ -4679,24 +4680,15 @@ int complete_action(struct replay_opts *opts, unsigned flags,
 		return error(_("nothing to do"));
 	}
 
-	if (todo_list_write_to_file(todo_list, todo_file,
-				    shortrevisions, shortonto, -1,
-				    flags | TODO_LIST_SHORTEN_IDS | TODO_LIST_APPEND_TODO_HELP))
-		return error_errno(_("could not write '%s'"), todo_file);
-
-	if (copy_file(rebase_path_todo_backup(), todo_file, 0666))
-		return error(_("could not copy '%s' to '%s'."), todo_file,
-			     rebase_path_todo_backup());
-
-	if (launch_sequence_editor(todo_file, &new_todo.buf, NULL)) {
+	res = edit_todo_list(todo_list, &new_todo, shortrevisions, shortonto, flags);
+	if (res == -1)
+		return -1;
+	else if (res == -2) {
 		apply_autostash(opts);
 		sequencer_remove_state(opts);
 
 		return -1;
-	}
-
-	strbuf_stripspace(&new_todo.buf, 1);
-	if (new_todo.buf.len == 0) {
+	} else if (res == -3) {
 		apply_autostash(opts);
 		sequencer_remove_state(opts);
 		todo_list_release(&new_todo);
Signature · 1 line
2.19.1.872.ga867da739e
#74 lore raw

PATCH v3 15/16 sequencer: fix a call to error() in transform_todo_file()

In reply to Alban Gruin

This replaces a call to error() by a call to error_errno() after writing the content of the todo list to the disk in transform_todo_file().

---
No changes since v2.

 sequencer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
sequencer.c +1 -1
index 02afd2f5cd..a55df3526f 100644
@@ -4464,7 +4464,7 @@ int sequencer_add_exec_commands(struct string_list *commands)
 	int res;
 
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
-		return error(_("could not read '%s'."), todo_file);
+		return error_errno(_("could not read '%s'."), todo_file);
 
 	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
 		todo_list_release(&todo_list);
Signature · 1 line
2.19.1.872.ga867da739e
#75 lore raw

PATCH v3 16/16 rebase--interactive: move transform_todo_file() to rebase--interactive.c

In reply to Alban Gruin

As transform_todo_file() is only needed inside of rebase--interactive.c, it is moved there from sequencer.c.

---
 builtin/rebase--interactive.c | 20 ++++++++++++++++++++
 sequencer.c                   | 20 --------------------
 sequencer.h                   |  1 -
 3 files changed, 20 insertions(+), 21 deletions(-)
builtin/rebase--interactive.c +20 -0
index 6871990544..580c6a3822 100644
@@ -34,6 +34,26 @@ static int edit_todo_file(unsigned flags)
 	return 0;
 }
 
+static int transform_todo_file(unsigned flags)
+{
+	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res;
+
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
+		return error_errno(_("could not read '%s'."), todo_file);
+
+	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
+		todo_list_release(&todo_list);
+		return error(_("unusable todo list: '%s'"), todo_file);
+	}
+
+	res = todo_list_write_to_file(&todo_list, todo_file,
+				      NULL, NULL, -1, flags);
+	todo_list_release(&todo_list);
+	return res;
+}
+
 static int get_revision_ranges(const char *upstream, const char *onto,
 			       const char **head_hash,
 			       char **revisions, char **shortrevisions)
sequencer.c +0 -20
index a55df3526f..896dd04150 100644
@@ -4542,26 +4542,6 @@ int todo_list_write_to_file(struct todo_list *todo_list, const char *file,
 	return res;
 }
 
-int transform_todo_file(unsigned flags)
-{
-	const char *todo_file = rebase_path_todo();
-	struct todo_list todo_list = TODO_LIST_INIT;
-	int res;
-
-	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
-		return error(_("could not read '%s'."), todo_file);
-
-	if (todo_list_parse_insn_buffer(todo_list.buf.buf, &todo_list)) {
-		todo_list_release(&todo_list);
-		return error(_("unusable todo list: '%s'"), todo_file);
-	}
-
-	res = todo_list_write_to_file(&todo_list, todo_file,
-				      NULL, NULL, -1, flags);
-	todo_list_release(&todo_list);
-	return res;
-}
-
 static const char edit_todo_list_advice[] =
 N_("You can fix this with 'git rebase --edit-todo' "
 "and then run 'git rebase --continue'.\n"
sequencer.h +0 -1
index f6751d53b9..ee59233344 100644
@@ -142,7 +142,6 @@ int sequencer_make_script(struct strbuf *out, int argc, const char **argv,
 			  unsigned flags);
 
 int sequencer_add_exec_commands(struct string_list *commands);
-int transform_todo_file(unsigned flags);
 int check_todo_list_from_file(void);
 int complete_action(struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
Signature · 1 line
2.19.1.872.ga867da739e
#76 lore raw

v4 cover sequencer: refactor functions working on a todo_list

In reply to Alban Gruin

At the center of the "interactive" part of the interactive rebase lies the todo list. When the user starts an interactive rebase, a todo list is generated, presented to the user (who then edits it using a text editor), read back, and then is checked and processed before the actual rebase takes place.

Some of this processing includes adding execs commands, reordering fixup! and squash! commits, and checking if no commits were accidentally dropped by the user.

Before I converted the interactive rebase in C, these functions were called by git-rebase--interactive.sh through git-rebase--helper. Since the only way to pass around a large amount of data between a shell script and a C program is to use a file (or any declination of a file), the functions that checked and processed the todo list were directly working on a file, the same file that the user edited.

During the conversion, I did not address this issue, which lead to a complete_action() that reads the todo list file, does some computation based on its content, and writes it back to the disk, several times in the same function.

As it is not an efficient way to handle a data structure, this patch series refactor the functions that processes the todo list to work on a todo_list structure instead of reading it from the disk.

Some commits consists in modifying edit_todo_list() (initially used by --edit-todo) to handle the initial edition of the todo list, to increase code sharing.

This is based on nd/the-index (36e7ed69de, "rebase-interactive.c: remove the_repository references"), as it introduced a lot of conflicts. The result does not conflict with pu (e31bc98f4b, "Merge branch 'md/list-objects-filter-by-depth' into pu").

Changes since v3:

- Replacing the 'arg' field from todo_item by 'arg_offset' to avoid dealing with pointers on the todo list buffer in todo_list_add_exec_commands(). This has led to some additionnal changes.

- Rewording some commits.

- Dropping the commit "sequencer: fix a call to error() in transform_todo_file()". The call to error() after reading the todo file is replaced by error_errno() in "sequencer: refactor transform_todos() to work on a todo_list". The same change has been applied to sequencer_add_exec_commands() in "sequencer: refactor sequencer_add_exec_commands() to work on a todo_list".

- transform_todo_file(), sequencer_add_exec_commands() and rearrange_squash_in_todo_file() now print an error if they fail to write to the todo file.

- A lot of changes were introduced by the conflict resolution with nd/the-index.

Alban Gruin (16):
sequencer: changes in parse_insn_buffer()
sequencer: make the todo_list structure public
sequencer: remove the 'arg' field from todo_item
sequencer: refactor transform_todos() to work on a todo_list
sequencer: introduce todo_list_write_to_file()
sequencer: refactor check_todo_list() to work on a todo_list
sequencer: refactor sequencer_add_exec_commands() to work on a
todo_list
sequencer: refactor rearrange_squash() to work on a todo_list
sequencer: make sequencer_make_script() write its script to a strbuf
sequencer: change complete_action() to use the refactored functions
sequencer: refactor skip_unnecessary_picks() to work on a todo_list
rebase-interactive: use todo_list_write_to_file() in edit_todo_list()
rebase-interactive: append_todo_help() changes
rebase-interactive: rewrite edit_todo_list() to handle the initial
edit
sequencer: use edit_todo_list() in complete_action()
rebase--interactive: move transform_todo_file() to
rebase--interactive.c

 builtin/rebase--interactive.c |  90 +++--
 rebase-interactive.c          | 143 +++++--
 rebase-interactive.h          |   8 +-
 sequencer.c                   | 677 +++++++++++++---------------------
 sequencer.h                   |  82 +++-
 5 files changed, 525 insertions(+), 475 deletions(-)
Signature · 1 line
2.20.1
#77 lore raw

PATCH v4 1/16 sequencer: changes in parse_insn_buffer()

This clears the number of items of a todo_list before parsing it to allow to parse the same list multiple times without issues. As its items are not dynamically allocated, or don’t need to allocate memory, no additionnal memory management is required here.

Furthermore, if a line is invalid, the type of the corresponding command is set to a garbage value, and its argument is defined properly. This will allow to recreate the text of a todo list from its commands, even if one of them is incorrect.

---
The full diff changed due to the conflict resolution with nd/the-index, but the "core change" is the same as in the v3.

 sequencer.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
sequencer.c +6 -1
index d726f77e11..a7afaf6882 100644
@@ -2091,6 +2091,8 @@ static int parse_insn_buffer(struct repository *r, char *buf,
 	char *p = buf, *next_p;
 	int i, res = 0, fixup_okay = file_exists(rebase_path_done());
 
+	todo_list->current = todo_list->nr = 0;
+
 	for (i = 1; *p; i++, p = next_p) {
 		char *eol = strchrnul(p, '\n');
 
@@ -2104,7 +2106,10 @@ static int parse_insn_buffer(struct repository *r, char *buf,
 		if (parse_insn_line(r, item, p, eol)) {
 			res = error(_("invalid line %d: %.*s"),
 				i, (int)(eol - p), p);
-			item->command = TODO_NOOP;
+			item->command = TODO_COMMENT + 1;
+			item->arg = p;
+			item->arg_len = (int)(eol - p);
+			item->commit = NULL;
 		}
 
 		if (fixup_okay)
Signature · 1 line
2.20.1
#78 lore raw

PATCH v4 2/16 sequencer: make the todo_list structure public

In reply to Alban Gruin

This makes the structures todo_list and todo_item, and the functions todo_list_release() and parse_insn_buffer(), accessible outside of sequencer.c.

---
Some changes were introduced because of the conflict resolution with nd/the-index. They mostly consist of adding a parameter ("r") when calling todo_list_parse_insn_buffer.

 sequencer.c | 69 ++++++++++-------------------------------------------
 sequencer.h | 50 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 62 insertions(+), 57 deletions(-)
sequencer.c +12 -57
index a7afaf6882..5b84a20532 100644
@@ -1460,32 +1460,6 @@ static int allow_empty(struct repository *r,
 		return 1;
 }
 
-/*
- * Note that ordering matters in this enum. Not only must it match the mapping
- * below, it is also divided into several sections that matter.  When adding
- * new commands, make sure you add it in the right section.
- */
-enum todo_command {
-	/* commands that handle commits */
-	TODO_PICK = 0,
-	TODO_REVERT,
-	TODO_EDIT,
-	TODO_REWORD,
-	TODO_FIXUP,
-	TODO_SQUASH,
-	/* commands that do something else than handling a single commit */
-	TODO_EXEC,
-	TODO_BREAK,
-	TODO_LABEL,
-	TODO_RESET,
-	TODO_MERGE,
-	/* commands that do nothing but are counted for reporting progress */
-	TODO_NOOP,
-	TODO_DROP,
-	/* comments (not counted for reporting progress) */
-	TODO_COMMENT
-};
-
 static struct {
 	char c;
 	const char *str;
@@ -1962,26 +1936,7 @@ enum todo_item_flags {
 	TODO_EDIT_MERGE_MSG = 1
 };
 
-struct todo_item {
-	enum todo_command command;
-	struct commit *commit;
-	unsigned int flags;
-	const char *arg;
-	int arg_len;
-	size_t offset_in_buf;
-};
-
-struct todo_list {
-	struct strbuf buf;
-	struct todo_item *items;
-	int nr, alloc, current;
-	int done_nr, total_nr;
-	struct stat_data stat;
-};
-
-#define TODO_LIST_INIT { STRBUF_INIT }
-
-static void todo_list_release(struct todo_list *todo_list)
+void todo_list_release(struct todo_list *todo_list)
 {
 	strbuf_release(&todo_list->buf);
 	FREE_AND_NULL(todo_list->items);
@@ -2084,8 +2039,8 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 	return !item->commit;
 }
 
-static int parse_insn_buffer(struct repository *r, char *buf,
-			     struct todo_list *todo_list)
+int todo_list_parse_insn_buffer(struct repository *r, char *buf,
+				struct todo_list *todo_list)
 {
 	struct todo_item *item;
 	char *p = buf, *next_p;
@@ -2184,7 +2139,7 @@ static int read_populate_todo(struct repository *r,
 		return error(_("could not stat '%s'"), todo_file);
 	fill_stat_data(&todo_list->stat, &st);
 
-	res = parse_insn_buffer(r, todo_list->buf.buf, todo_list);
+	res = todo_list_parse_insn_buffer(r, todo_list->buf.buf, todo_list);
 	if (res) {
 		if (is_rebase_i(opts))
 			return error(_("please fix this using "
@@ -2215,7 +2170,7 @@ static int read_populate_todo(struct repository *r,
 		FILE *f = fopen_or_warn(rebase_path_msgtotal(), "w");
 
 		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
-		    !parse_insn_buffer(r, done.buf.buf, &done))
+		    !todo_list_parse_insn_buffer(r, done.buf.buf, &done))
 			todo_list->done_nr = count_commands(&done);
 		else
 			todo_list->done_nr = 0;
@@ -4501,7 +4456,7 @@ int sequencer_add_exec_commands(struct repository *r,
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4557,7 +4512,7 @@ int transform_todos(struct repository *r, unsigned flags)
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4643,7 +4598,7 @@ int check_todo_list(struct repository *r)
 		goto leave_check;
 	}
 	advise_to_edit_todo = res =
-		parse_insn_buffer(r, todo_list.buf.buf, &todo_list);
+		todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list);
 
 	if (res || check_level == MISSING_COMMIT_CHECK_IGNORE)
 		goto leave_check;
@@ -4662,7 +4617,7 @@ int check_todo_list(struct repository *r)
 		goto leave_check;
 	}
 	strbuf_release(&todo_file);
-	res = !!parse_insn_buffer(r, todo_list.buf.buf, &todo_list);
+	res = !!todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list);
 
 	/* Find commits in git-rebase-todo.backup yet unseen */
 	for (i = todo_list.nr - 1; i >= 0; i--) {
@@ -4744,7 +4699,7 @@ static int skip_unnecessary_picks(struct repository *r, struct object_id *output
 
 	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
 		return -1;
-	if (parse_insn_buffer(r, todo_list.buf.buf, &todo_list) < 0) {
+	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list) < 0) {
 		todo_list_release(&todo_list);
 		return -1;
 	}
@@ -4832,7 +4787,7 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla
 	if (strbuf_read_file(buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(r, buf->buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(r, buf->buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4940,7 +4895,7 @@ int rearrange_squash(struct repository *r)
 
 	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
 		return -1;
-	if (parse_insn_buffer(r, todo_list.buf.buf, &todo_list) < 0) {
+	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list) < 0) {
 		todo_list_release(&todo_list);
 		return -1;
 	}
sequencer.h +50 -0

56 lines not on this page · raw message

Signature · 1 line
2.20.1
#79 lore raw

PATCH v4 3/16 sequencer: remove the 'arg' field from todo_item

In reply to Alban Gruin

The 'arg' field of todo_item used to store the address of the first byte of the parameter of a command in a todo list. It was associated with the length of the parameter (the 'arg_len' field).

This replaces the 'arg' field by 'arg_offset'. This new field does not store the address of the parameter, but the position of the first character of the parameter in the buffer.

This will prevent todo_list_add_exec_commands() from having to do awful pointer arithmetics when growing the todo list buffer.

---
This is a new commit.

 sequencer.c | 61 ++++++++++++++++++++++++++++-------------------------
 sequencer.h |  4 ++--
 2 files changed, 34 insertions(+), 31 deletions(-)
sequencer.c +32 -29
index 5b84a20532..61be04ccc5 100644
@@ -1950,7 +1950,7 @@ static struct todo_item *append_new_todo(struct todo_list *todo_list)
 }
 
 static int parse_insn_line(struct repository *r, struct todo_item *item,
-			   const char *bol, char *eol)
+			   const char *buf, const char *bol, char *eol)
 {
 	struct object_id commit_oid;
 	char *end_of_object_name;
@@ -1964,7 +1964,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
 		item->command = TODO_COMMENT;
 		item->commit = NULL;
-		item->arg = bol;
+		item->arg_offset = bol - buf;
 		item->arg_len = eol - bol;
 		return 0;
 	}
@@ -1991,7 +1991,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 			return error(_("%s does not accept arguments: '%s'"),
 				     command_to_string(item->command), bol);
 		item->commit = NULL;
-		item->arg = bol;
+		item->arg_offset = bol - buf;
 		item->arg_len = eol - bol;
 		return 0;
 	}
@@ -2003,7 +2003,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
 	    item->command == TODO_RESET) {
 		item->commit = NULL;
-		item->arg = bol;
+		item->arg_offset = bol - buf;
 		item->arg_len = (int)(eol - bol);
 		return 0;
 	}
@@ -2017,7 +2017,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 		} else {
 			item->flags |= TODO_EDIT_MERGE_MSG;
 			item->commit = NULL;
-			item->arg = bol;
+			item->arg_offset = bol - buf;
 			item->arg_len = (int)(eol - bol);
 			return 0;
 		}
@@ -2029,8 +2029,9 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 	status = get_oid(bol, &commit_oid);
 	*end_of_object_name = saved;
 
-	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
-	item->arg_len = (int)(eol - item->arg);
+	bol = end_of_object_name + strspn(end_of_object_name, " \t");
+	item->arg_offset = bol - buf;
+	item->arg_len = (int)(eol - bol);
 
 	if (status < 0)
 		return -1;
@@ -2058,11 +2059,11 @@ int todo_list_parse_insn_buffer(struct repository *r, char *buf,
 
 		item = append_new_todo(todo_list);
 		item->offset_in_buf = p - todo_list->buf.buf;
-		if (parse_insn_line(r, item, p, eol)) {
+		if (parse_insn_line(r, item, buf, p, eol)) {
 			res = error(_("invalid line %d: %.*s"),
 				i, (int)(eol - p), p);
 			item->command = TODO_COMMENT + 1;
-			item->arg = p;
+			item->arg_offset = p - buf;
 			item->arg_len = (int)(eol - p);
 			item->commit = NULL;
 		}
@@ -2402,7 +2403,7 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 
 		item->command = command;
 		item->commit = commit;
-		item->arg = NULL;
+		item->arg_offset = 0;
 		item->arg_len = 0;
 		item->offset_in_buf = todo_list->buf.len;
 		subject_len = find_commit_subject(commit_buffer, &subject);
@@ -3438,6 +3439,8 @@ static int pick_commits(struct repository *r,
 
 	while (todo_list->current < todo_list->nr) {
 		struct todo_item *item = todo_list->items + todo_list->current;
+		const char *arg = todo_list->buf.buf + item->arg_offset;
+
 		if (save_todo(todo_list, opts))
 			return -1;
 		if (is_rebase_i(opts)) {
@@ -3488,10 +3491,9 @@ static int pick_commits(struct repository *r,
 					fprintf(stderr,
 						_("Stopped at %s...  %.*s\n"),
 						short_commit_name(commit),
-						item->arg_len, item->arg);
+						item->arg_len, arg);
 				return error_with_patch(r, commit,
-					item->arg, item->arg_len, opts, res,
-					!res);
+					arg, item->arg_len, opts, res, !res);
 			}
 			if (is_rebase_i(opts) && !res)
 				record_in_rewritten(&item->commit->object.oid,
@@ -3500,7 +3502,7 @@ static int pick_commits(struct repository *r,
 				if (res == 1)
 					intend_to_amend();
 				return error_failed_squash(r, item->commit, opts,
-					item->arg_len, item->arg);
+					item->arg_len, arg);
 			} else if (res && is_rebase_i(opts) && item->commit) {
 				int to_amend = 0;
 				struct object_id oid;
@@ -3519,16 +3521,16 @@ static int pick_commits(struct repository *r,
 					to_amend = 1;
 
 				return res | error_with_patch(r, item->commit,
-						item->arg, item->arg_len, opts,
+						arg, item->arg_len, opts,
 						res, to_amend);
 			}
 		} else if (item->command == TODO_EXEC) {
-			char *end_of_arg = (char *)(item->arg + item->arg_len);
+			char *end_of_arg = (char *)(arg + item->arg_len);
 			int saved = *end_of_arg;
 			struct stat st;
 
 			*end_of_arg = '\0';
-			res = do_exec(r, item->arg);
+			res = do_exec(r, arg);
 			*end_of_arg = saved;
 
 			/* Reread the todo file if it has changed. */
@@ -3545,14 +3547,14 @@ static int pick_commits(struct repository *r,
 				todo_list->current = -1;
 			}
 		} else if (item->command == TODO_LABEL) {
-			if ((res = do_label(r, item->arg, item->arg_len)))
+			if ((res = do_label(r, arg, item->arg_len)))
 				reschedule = 1;
 		} else if (item->command == TODO_RESET) {
-			if ((res = do_reset(r, item->arg, item->arg_len, opts)))
+			if ((res = do_reset(r, arg, item->arg_len, opts)))
 				reschedule = 1;
 		} else if (item->command == TODO_MERGE) {
 			if ((res = do_merge(r, item->commit,
-					    item->arg, item->arg_len,
+					    arg, item->arg_len,
 					    item->flags, opts)) < 0)
 				reschedule = 1;
 			else if (item->commit)
@@ -3561,9 +3563,8 @@ static int pick_commits(struct repository *r,
 			if (res > 0)
 				/* failed with merge conflicts */
 				return error_with_patch(r, item->commit,
-							item->arg,
-							item->arg_len, opts,
-							res, 0);
+							arg, item->arg_len,
+							opts, res, 0);
 		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
@@ -3578,9 +3579,8 @@ static int pick_commits(struct repository *r,
 			if (item->commit)
 				return error_with_patch(r,
 							item->commit,
-							item->arg,
-							item->arg_len, opts,
-							res, 0);
+							arg, item->arg_len,
+							opts, res, 0);
 		}
 
 		todo_list->current++;
@@ -4520,7 +4520,8 @@ int transform_todos(struct repository *r, unsigned flags)
 	for (item = todo_list.items, i = 0; i < todo_list.nr; i++, item++) {
 		/* if the item is not a command write it and continue */
 		if (item->command >= TODO_COMMENT) {
-			strbuf_addf(&buf, "%.*s\n", item->arg_len, item->arg);
+			strbuf_addf(&buf, "%.*s\n", item->arg_len,
+				    todo_list.buf.buf + item->arg_offset);
 			continue;
 		}
 
@@ -4550,7 +4551,8 @@ int transform_todos(struct repository *r, unsigned flags)
 		if (!item->arg_len)
 			strbuf_addch(&buf, '\n');
 		else
-			strbuf_addf(&buf, " %.*s\n", item->arg_len, item->arg);
+			strbuf_addf(&buf, " %.*s\n", item->arg_len,
+				    todo_list.buf.buf + item->arg_offset);
 	}
 
 	i = write_message(buf.buf, buf.len, todo_file, 0);
@@ -4626,7 +4628,8 @@ int check_todo_list(struct repository *r)
 		if (commit && !*commit_seen_at(&commit_seen, commit)) {
 			strbuf_addf(&missing, " - %s %.*s\n",
 				    short_commit_name(commit),
-				    item->arg_len, item->arg);
+				    item->arg_len,
+				    todo_list.buf.buf + item->arg_offset);
 			*commit_seen_at(&commit_seen, commit) = 1;
 		}
 	}
sequencer.h +2 -2
index 7dc4d8946b..70a896e756 100644
@@ -104,9 +104,9 @@ struct todo_item {
 	enum todo_command command;
 	struct commit *commit;
 	unsigned int flags;
-	const char *arg;
 	int arg_len;
-	size_t offset_in_buf;
+	/* The offset of the command and its argument in the strbuf */
+	size_t offset_in_buf, arg_offset;
 };
 
 struct todo_list {
Signature · 1 line
2.20.1
#80 lore raw

Re: [PATCH v4 03/16] sequencer: remove the 'arg' field from todo_item

Hi Alban

I think this is a good change, I just wonder if it would be better to have an accessor function to get the arg pointer from a todo list item see below

Quoted 197 lines · Alban Gruin

The 'arg' field of todo_item used to store the address of the first byte of the parameter of a command in a todo list. It was associated with the length of the parameter (the 'arg_len' field).

This replaces the 'arg' field by 'arg_offset'. This new field does not store the address of the parameter, but the position of the first character of the parameter in the buffer.

This will prevent todo_list_add_exec_commands() from having to do awful pointer arithmetics when growing the todo list buffer.

---
This is a new commit.

  sequencer.c | 61 ++++++++++++++++++++++++++++-------------------------
  sequencer.h |  4 ++--
  2 files changed, 34 insertions(+), 31 deletions(-)
sequencer.c +26 -26
index 5b84a20532..61be04ccc5 100644
@@ -1950,7 +1950,7 @@ static struct todo_item *append_new_todo(struct todo_list *todo_list)
  }
 
  static int parse_insn_line(struct repository *r, struct todo_item *item,
-			   const char *bol, char *eol)
+			   const char *buf, const char *bol, char *eol)
  {
  	struct object_id commit_oid;
  	char *end_of_object_name;
@@ -1964,7 +1964,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
  	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
  		item->command = TODO_COMMENT;
  		item->commit = NULL;
-		item->arg = bol;
+		item->arg_offset = bol - buf;
  		item->arg_len = eol - bol;
  		return 0;
  	}
@@ -1991,7 +1991,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
  			return error(_("%s does not accept arguments: '%s'"),
  				     command_to_string(item->command), bol);
  		item->commit = NULL;
-		item->arg = bol;
+		item->arg_offset = bol - buf;
  		item->arg_len = eol - bol;
  		return 0;
  	}
@@ -2003,7 +2003,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
  	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
  	    item->command == TODO_RESET) {
  		item->commit = NULL;
-		item->arg = bol;
+		item->arg_offset = bol - buf;
  		item->arg_len = (int)(eol - bol);
  		return 0;
  	}
@@ -2017,7 +2017,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
  		} else {
  			item->flags |= TODO_EDIT_MERGE_MSG;
  			item->commit = NULL;
-			item->arg = bol;
+			item->arg_offset = bol - buf;
  			item->arg_len = (int)(eol - bol);
  			return 0;
  		}
@@ -2029,8 +2029,9 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
  	status = get_oid(bol, &commit_oid);
  	*end_of_object_name = saved;
 
-	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
-	item->arg_len = (int)(eol - item->arg);
+	bol = end_of_object_name + strspn(end_of_object_name, " \t");
+	item->arg_offset = bol - buf;
+	item->arg_len = (int)(eol - bol);
 
  	if (status < 0)
  		return -1;
@@ -2058,11 +2059,11 @@ int todo_list_parse_insn_buffer(struct repository *r, char *buf,
 
  		item = append_new_todo(todo_list);
  		item->offset_in_buf = p - todo_list->buf.buf;
-		if (parse_insn_line(r, item, p, eol)) {
+		if (parse_insn_line(r, item, buf, p, eol)) {
  			res = error(_("invalid line %d: %.*s"),
  				i, (int)(eol - p), p);
  			item->command = TODO_COMMENT + 1;
-			item->arg = p;
+			item->arg_offset = p - buf;
  			item->arg_len = (int)(eol - p);
  			item->commit = NULL;
  		}
@@ -2402,7 +2403,7 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 
  		item->command = command;
  		item->commit = commit;
-		item->arg = NULL;
+		item->arg_offset = 0;
  		item->arg_len = 0;
  		item->offset_in_buf = todo_list->buf.len;
  		subject_len = find_commit_subject(commit_buffer, &subject);
@@ -3438,6 +3439,8 @@ static int pick_commits(struct repository *r,
 
  	while (todo_list->current < todo_list->nr) {
  		struct todo_item *item = todo_list->items + todo_list->current;
+		const char *arg = todo_list->buf.buf + item->arg_offset;
+
  		if (save_todo(todo_list, opts))
  			return -1;
  		if (is_rebase_i(opts)) {
@@ -3488,10 +3491,9 @@ static int pick_commits(struct repository *r,
  					fprintf(stderr,
  						_("Stopped at %s...  %.*s\n"),
  						short_commit_name(commit),
-						item->arg_len, item->arg);
+						item->arg_len, arg);
  				return error_with_patch(r, commit,
-					item->arg, item->arg_len, opts, res,
-					!res);
+					arg, item->arg_len, opts, res, !res);
  			}
  			if (is_rebase_i(opts) && !res)
  				record_in_rewritten(&item->commit->object.oid,
@@ -3500,7 +3502,7 @@ static int pick_commits(struct repository *r,
  				if (res == 1)
  					intend_to_amend();
  				return error_failed_squash(r, item->commit, opts,
-					item->arg_len, item->arg);
+					item->arg_len, arg);
  			} else if (res && is_rebase_i(opts) && item->commit) {
  				int to_amend = 0;
  				struct object_id oid;
@@ -3519,16 +3521,16 @@ static int pick_commits(struct repository *r,
  					to_amend = 1;
 
  				return res | error_with_patch(r, item->commit,
-						item->arg, item->arg_len, opts,
+						arg, item->arg_len, opts,
  						res, to_amend);
  			}
  		} else if (item->command == TODO_EXEC) {
-			char *end_of_arg = (char *)(item->arg + item->arg_len);
+			char *end_of_arg = (char *)(arg + item->arg_len);
  			int saved = *end_of_arg;
  			struct stat st;
 
  			*end_of_arg = '\0';
-			res = do_exec(r, item->arg);
+			res = do_exec(r, arg);
  			*end_of_arg = saved;
 
  			/* Reread the todo file if it has changed. */
@@ -3545,14 +3547,14 @@ static int pick_commits(struct repository *r,
  				todo_list->current = -1;
  			}
  		} else if (item->command == TODO_LABEL) {
-			if ((res = do_label(r, item->arg, item->arg_len)))
+			if ((res = do_label(r, arg, item->arg_len)))
  				reschedule = 1;
  		} else if (item->command == TODO_RESET) {
-			if ((res = do_reset(r, item->arg, item->arg_len, opts)))
+			if ((res = do_reset(r, arg, item->arg_len, opts)))
  				reschedule = 1;
  		} else if (item->command == TODO_MERGE) {
  			if ((res = do_merge(r, item->commit,
-					    item->arg, item->arg_len,
+					    arg, item->arg_len,
  					    item->flags, opts)) < 0)
  				reschedule = 1;
  			else if (item->commit)
@@ -3561,9 +3563,8 @@ static int pick_commits(struct repository *r,
  			if (res > 0)
  				/* failed with merge conflicts */
  				return error_with_patch(r, item->commit,
-							item->arg,
-							item->arg_len, opts,
-							res, 0);
+							arg, item->arg_len,
+							opts, res, 0);
  		} else if (!is_noop(item->command))
  			return error(_("unknown command %d"), item->command);
 
@@ -3578,9 +3579,8 @@ static int pick_commits(struct repository *r,
  			if (item->commit)
  				return error_with_patch(r,
  							item->commit,
-							item->arg,
-							item->arg_len, opts,
-							res, 0);
+							arg, item->arg_len,
+							opts, res, 0);
  		}
 
  		todo_list->current++;

sequencer.c int transform_todos(struct repository *r, unsigned flags) · line 4520

Quoted 1 line
sequencer.c +0 -0
@@ -4520,7 +4520,8 @@ int transform_todos(struct repository *r, unsigned flags)
  	for (item = todo_list.items, i = 0; i < todo_list.nr; i++, item++) {

I think the changes to this function would be clearer with
+ const char *arg = todo_list.buf.buf + item->arg_offset
here and then used arg instead of item->arg rather than calculating arg twice. Given that we are going to need to calculate arg again in check_todo_list() maybe it would be better to have a function

const char *get_item_arg(struct todo_list *todo, struct todo_list_item *item) {
	return todo->buf.buf + item->arg_offset;
}

and do
+ const char *arg = get_item_arg(&todo_list, item);
to avoid any accidents when calculating arg in future.

Best Wishes

Phillip

Quoted 45 lines
sequencer.c +6 -3
  		/* if the item is not a command write it and continue */
  		if (item->command >= TODO_COMMENT) {
-			strbuf_addf(&buf, "%.*s\n", item->arg_len, item->arg);
+			strbuf_addf(&buf, "%.*s\n", item->arg_len,
+				    todo_list.buf.buf + item->arg_offset);
  			continue;
  		}
 
@@ -4550,7 +4551,8 @@ int transform_todos(struct repository *r, unsigned flags)
  		if (!item->arg_len)
  			strbuf_addch(&buf, '\n');
  		else
-			strbuf_addf(&buf, " %.*s\n", item->arg_len, item->arg);
+			strbuf_addf(&buf, " %.*s\n", item->arg_len,
+				    todo_list.buf.buf + item->arg_offset);
  	}
 
  	i = write_message(buf.buf, buf.len, todo_file, 0);
@@ -4626,7 +4628,8 @@ int check_todo_list(struct repository *r)
  		if (commit && !*commit_seen_at(&commit_seen, commit)) {
  			strbuf_addf(&missing, " - %s %.*s\n",
  				    short_commit_name(commit),
-				    item->arg_len, item->arg);
+				    item->arg_len,
+				    todo_list.buf.buf + item->arg_offset);
  			*commit_seen_at(&commit_seen, commit) = 1;
  		}
  	}
sequencer.h +2 -2
index 7dc4d8946b..70a896e756 100644
@@ -104,9 +104,9 @@ struct todo_item {
  	enum todo_command command;
  	struct commit *commit;
  	unsigned int flags;
-	const char *arg;
  	int arg_len;
-	size_t offset_in_buf;
+	/* The offset of the command and its argument in the strbuf */
+	size_t offset_in_buf, arg_offset;
  };
 
  struct todo_list {
#81 lore raw

Re: [PATCH v4 03/16] sequencer: remove the 'arg' field from todo_item

Hi Phillip,

Quoted 4 lines · Phillip Wood

const char *get_item_arg(struct todo_list *todo, struct todo_list_item *item)

{
	return todo->buf.buf + item->arg_offset;
}

I like that a lot.

Thanks,
Dscho

#82 lore raw

PATCH v4 4/16 sequencer: refactor transform_todos() to work on a todo_list

In reply to Alban Gruin

This refactors transform_todos() to work on a todo_list. The function is renamed todo_list_transform().

As rebase -p still need to check the todo list from the disk, a new function is introduced, transform_todo_file(). It is still used by complete_action() and edit_todo_list() for now, but they will be replaced in a future commit.

todo_list_transform() is not a static function, because it will be used by edit_todo_list() from rebase-interactive.c in a future commit.

---
 builtin/rebase--interactive.c |  2 +-
 rebase-interactive.c          |  4 +--
 sequencer.c                   | 54 +++++++++++++++++++++++------------
 sequencer.h                   |  4 ++-
 4 files changed, 42 insertions(+), 22 deletions(-)
builtin/rebase--interactive.c +1 -1
index dd2a55ab1d..0898eb4c59 100644
@@ -253,7 +253,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 	}
 	case SHORTEN_OIDS:
 	case EXPAND_OIDS:
-		ret = transform_todos(the_repository, flags);
+		ret = transform_todo_file(the_repository, flags);
 		break;
 	case CHECK_TODO_LIST:
 		ret = check_todo_list(the_repository);
rebase-interactive.c +2 -2
index 68aff1dac2..842fa07e7e 100644
@@ -69,7 +69,7 @@ int edit_todo_list(struct repository *r, unsigned flags)
 
 	strbuf_release(&buf);
 
-	transform_todos(r, flags | TODO_LIST_SHORTEN_IDS);
+	transform_todo_file(r, flags | TODO_LIST_SHORTEN_IDS);
 
 	if (strbuf_read_file(&buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
@@ -85,7 +85,7 @@ int edit_todo_list(struct repository *r, unsigned flags)
 	if (launch_sequence_editor(todo_file, NULL, NULL))
 		return -1;
 
-	transform_todos(r, flags & ~(TODO_LIST_SHORTEN_IDS));
+	transform_todo_file(r, flags & ~(TODO_LIST_SHORTEN_IDS));
 
 	return 0;
 }
sequencer.c +36 -18

89 lines not on this page · raw message

sequencer.h +3 -1
index 70a896e756..53ffe7371c 100644
@@ -121,6 +121,8 @@ struct todo_list {
 
 int todo_list_parse_insn_buffer(struct repository *r, char *buf,
 				struct todo_list *todo_list);
+void todo_list_transform(struct repository *r, struct todo_list *todo_list,
+			 unsigned flags);
 void todo_list_release(struct todo_list *todo_list);
 
 /* Call this to setup defaults before parsing command line options */
@@ -146,7 +148,7 @@ int sequencer_make_script(struct repository *repo, FILE *out,
 			  unsigned flags);
 
 int sequencer_add_exec_commands(struct repository *r, const char *command);
-int transform_todos(struct repository *r, unsigned flags);
+int transform_todo_file(struct repository *r, unsigned flags);
 enum missing_commit_check_level get_missing_commit_check_level(void);
 int check_todo_list(struct repository *r);
 int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
Signature · 1 line
2.20.1
#83 lore raw

PATCH v4 5/16 sequencer: introduce todo_list_write_to_file()

In reply to Alban Gruin

This introduces a new function to recreate the text of a todo list from its commands and write it to a file. This will be useful as the next few commits will change the use of the buffer in struct todo_list so it will no longer be a mirror of the file on disk.

This functionality already exists in todo_list_transform(), but this function was made to replace the buffer of a todo list, which is not what we want here. Thus, the part of todo_list_transform() that replaces the buffer is dropped, and the function is renamed todo_list_to_strbuf(). It is called by todo_list_write_to_file() to fill the buffer to write to the disk.

todo_list_write_to_file() can also take care of appending the help text to the buffer before writing it to the disk, or to write only the first n items of the list. This feature will be used by skip_unnecessary_picks(), which has to write done commands in a file.

---
 sequencer.c | 61 +++++++++++++++++++++++++++++++++++------------------
 sequencer.h | 11 ++++++----
 2 files changed, 48 insertions(+), 24 deletions(-)
sequencer.c +41 -20
index 6b4eb7a9ba..b932ca34f2 100644
@@ -4501,26 +4501,28 @@ int sequencer_add_exec_commands(struct repository *r,
 	return i;
 }
 
-void todo_list_transform(struct repository *r, struct todo_list *todo_list,
-			 unsigned flags)
+static void todo_list_to_strbuf(struct repository *r, struct todo_list *todo_list,
+				struct strbuf *buf, int num, unsigned flags)
 {
-	struct strbuf buf = STRBUF_INIT;
 	struct todo_item *item;
-	int i;
+	int i, max = todo_list->nr;
 
-	for (item = todo_list->items, i = 0; i < todo_list->nr; i++, item++) {
+	if (num > 0 && num < max)
+		max = num;
+
+	for (item = todo_list->items, i = 0; i < max; i++, item++) {
 		/* if the item is not a command write it and continue */
 		if (item->command >= TODO_COMMENT) {
-			strbuf_addf(&buf, "%.*s\n", item->arg_len,
+			strbuf_addf(buf, "%.*s\n", item->arg_len,
 				    todo_list->buf.buf + item->arg_offset);
 			continue;
 		}
 
 		/* add command to the buffer */
 		if (flags & TODO_LIST_ABBREVIATE_CMDS)
-			strbuf_addch(&buf, command_to_char(item->command));
+			strbuf_addch(buf, command_to_char(item->command));
 		else
-			strbuf_addstr(&buf, command_to_string(item->command));
+			strbuf_addstr(buf, command_to_string(item->command));
 
 		/* add commit id */
 		if (item->commit) {
@@ -4530,28 +4532,48 @@ void todo_list_transform(struct repository *r, struct todo_list *todo_list,
 
 			if (item->command == TODO_MERGE) {
 				if (item->flags & TODO_EDIT_MERGE_MSG)
-					strbuf_addstr(&buf, " -c");
+					strbuf_addstr(buf, " -c");
 				else
-					strbuf_addstr(&buf, " -C");
+					strbuf_addstr(buf, " -C");
 			}
 
-			strbuf_addf(&buf, " %s", oid);
+			strbuf_addf(buf, " %s", oid);
 		}
 
 		/* add all the rest */
 		if (!item->arg_len)
-			strbuf_addch(&buf, '\n');
+			strbuf_addch(buf, '\n');
 		else
-			strbuf_addf(&buf, " %.*s\n", item->arg_len,
+			strbuf_addf(buf, " %.*s\n", item->arg_len,
 				    todo_list->buf.buf + item->arg_offset);
 	}
+}
 
-	strbuf_reset(&todo_list->buf);
-	strbuf_add(&todo_list->buf, buf.buf, buf.len);
+int todo_list_write_to_file(struct repository *r, struct todo_list *todo_list,
+			    const char *file, const char *shortrevisions,
+			    const char *shortonto, int num, unsigned flags)
+{
+	int edit_todo = !(shortrevisions && shortonto), res;
+	struct strbuf buf = STRBUF_INIT;
+
+	todo_list_to_strbuf(r, todo_list, &buf, num, flags);
+
+	if (flags & TODO_LIST_APPEND_TODO_HELP) {
+		int command_count = count_commands(todo_list);
+		if (!edit_todo) {
+			strbuf_addch(&buf, '\n');
+			strbuf_commented_addf(&buf, Q_("Rebase %s onto %s (%d command)",
+						       "Rebase %s onto %s (%d commands)",
+						       command_count),
+					      shortrevisions, shortonto, command_count);
+		}
+		append_todo_help(edit_todo, flags & TODO_LIST_KEEP_EMPTY, &buf);
+	}
+
+	res = write_message(buf.buf, buf.len, file, 0);
 	strbuf_release(&buf);
 
-	if (todo_list_parse_insn_buffer(r, todo_list->buf.buf, todo_list))
-		BUG("unusable todo list");
+	return res;
 }
 
 int transform_todo_file(struct repository *r, unsigned flags)
@@ -4568,9 +4590,8 @@ int transform_todo_file(struct repository *r, unsigned flags)
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
 
-	todo_list_transform(r, &todo_list, flags);
-
-	res = write_message(todo_list.buf.buf, todo_list.buf.len, todo_file, 0);
+	res = todo_list_write_to_file(r, &todo_list, todo_file,
+				      NULL, NULL, -1, flags);
 	todo_list_release(&todo_list);
 
 	if (res)
sequencer.h +7 -4
index 53ffe7371c..4643fb3b69 100644
@@ -121,8 +121,9 @@ struct todo_list {
 
 int todo_list_parse_insn_buffer(struct repository *r, char *buf,
 				struct todo_list *todo_list);
-void todo_list_transform(struct repository *r, struct todo_list *todo_list,
-			 unsigned flags);
+int todo_list_write_to_file(struct repository *r, struct todo_list *todo_list,
+			    const char *file, const char *shortrevisions,
+			    const char *shortonto, int num, unsigned flags);
 void todo_list_release(struct todo_list *todo_list);
 
 /* Call this to setup defaults before parsing command line options */
@@ -143,8 +144,10 @@ int sequencer_remove_state(struct replay_opts *opts);
  * commits should be rebased onto the new base, this flag needs to be passed.
  */
 #define TODO_LIST_REBASE_COUSINS (1U << 4)
-int sequencer_make_script(struct repository *repo, FILE *out,
-			  int argc, const char **argv,
+#define TODO_LIST_APPEND_TODO_HELP (1U << 5)
+
+int sequencer_make_script(struct repository *r, FILE *out, int argc,
+			  const char **argv,
 			  unsigned flags);
 
 int sequencer_add_exec_commands(struct repository *r, const char *command);
Signature · 1 line
2.20.1
#84 lore raw

PATCH v4 6/16 sequencer: refactor check_todo_list() to work on a todo_list

In reply to Alban Gruin

This refactors check_todo_list() to work on a todo_list to avoid redundant reads and writes to the disk. The function is renamed todo_list_check(). The parsing of the two todo lists is left to the caller.

As rebase -p still need to check the todo list from the disk, a new function is introduced, check_todo_list_from_file(). It reads the file from the disk, parses it, pass the todo_list to todo_list_check(), and writes it back to the disk.

As get_missing_commit_check_level() and the enum missing_commit_check_level are no longer needed inside of sequencer.c, they are moved to rebase-interactive.c, and made static again.

---
 builtin/rebase--interactive.c |   2 +-
 rebase-interactive.c          |  91 ++++++++++++++++++++++++-
 rebase-interactive.h          |   1 +
 sequencer.c                   | 121 +++++++---------------------------
 sequencer.h                   |   9 +--
 5 files changed, 116 insertions(+), 108 deletions(-)
builtin/rebase--interactive.c +1 -1
index 0898eb4c59..df19ccaeb9 100644
@@ -256,7 +256,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = transform_todo_file(the_repository, flags);
 		break;
 	case CHECK_TODO_LIST:
-		ret = check_todo_list(the_repository);
+		ret = check_todo_list_from_file(the_repository);
 		break;
 	case REARRANGE_SQUASH:
 		ret = rearrange_squash(the_repository);
rebase-interactive.c +90 -1

101 lines not on this page · raw message

rebase-interactive.h +1 -0
index 17b6c9f6d0..dd0d717bc1 100644
@@ -7,5 +7,6 @@ struct repository;
 void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 		      struct strbuf *buf);
 int edit_todo_list(struct repository *r, unsigned flags);
+int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
 
 #endif
sequencer.c +23 -98

142 lines not on this page · raw message

sequencer.h +1 -8
index 4643fb3b69..07a8418b0d 100644
@@ -64,12 +64,6 @@ struct replay_opts {
 };
 #define REPLAY_OPTS_INIT { .action = -1, .current_fixups = STRBUF_INIT }
 
-enum missing_commit_check_level {
-	MISSING_COMMIT_CHECK_IGNORE = 0,
-	MISSING_COMMIT_CHECK_WARN,
-	MISSING_COMMIT_CHECK_ERROR
-};
-
 int write_message(const void *buf, size_t len, const char *filename,
 		  int append_eol);
 
@@ -152,8 +146,7 @@ int sequencer_make_script(struct repository *r, FILE *out, int argc,
 
 int sequencer_add_exec_commands(struct repository *r, const char *command);
 int transform_todo_file(struct repository *r, unsigned flags);
-enum missing_commit_check_level get_missing_commit_check_level(void);
-int check_todo_list(struct repository *r);
+int check_todo_list_from_file(struct repository *r);
 int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, const char *cmd,
Signature · 1 line
2.20.1
#85 lore raw

PATCH v4 7/16 sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

In reply to Alban Gruin

This refactors sequencer_add_exec_commands() to work on a todo_list to avoid redundant reads and writes to the disk.

Instead of inserting the `exec' commands between the other commands and re-parsing the buffer at the end, they are appended to the buffer once, and a new list of items is created. Items from the old list are copied across and new `exec' items are appended when necessary. This eliminates the need to reparse the buffer, but this also means we have to use todo_list_write_to_disk() to write the file.

todo_list_add_exec_commands() and sequencer_add_exec_commands() are modified to take a string list instead of a string -- one item for each command. This makes it easier to insert a new command to the todo list for each command to execute.

sequencer_add_exec_commands() still reads the todo list from the disk, as it is needed by rebase -p.

complete_action() still uses sequencer_add_exec_commands() for now. This will be changed in a future commit.

---
 builtin/rebase--interactive.c |  15 +++--
 sequencer.c                   | 110 +++++++++++++++++++++-------------
 sequencer.h                   |   5 +-
 3 files changed, 82 insertions(+), 48 deletions(-)
builtin/rebase--interactive.c +11 -4
index df19ccaeb9..53056ee713 100644
@@ -65,7 +65,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 				 const char *onto, const char *onto_name,
 				 const char *squash_onto, const char *head_name,
 				 const char *restrict_revision, char *raw_strategies,
-				 const char *cmd, unsigned autosquash)
+				 struct string_list *commands, unsigned autosquash)
 {
 	int ret;
 	const char *head_hash = NULL;
@@ -116,7 +116,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 		discard_cache();
 		ret = complete_action(the_repository, opts, flags,
 				      shortrevisions, onto_name, onto,
-				      head_hash, cmd, autosquash);
+				      head_hash, commands, autosquash);
 	}
 
 	free(revisions);
@@ -139,6 +139,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 	const char *onto = NULL, *onto_name = NULL, *restrict_revision = NULL,
 		*squash_onto = NULL, *upstream = NULL, *head_name = NULL,
 		*switch_to = NULL, *cmd = NULL;
+	struct string_list commands = STRING_LIST_INIT_DUP;
 	char *raw_strategies = NULL;
 	enum {
 		NONE = 0, CONTINUE, SKIP, EDIT_TODO, SHOW_CURRENT_PATCH,
@@ -221,6 +222,12 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		warning(_("--[no-]rebase-cousins has no effect without "
 			  "--rebase-merges"));
 
+	if (cmd && *cmd) {
+		string_list_split(&commands, cmd, '\n', -1);
+		if (strlen(commands.items[commands.nr - 1].string) == 0)
+			--commands.nr;
+	}
+
 	switch (command) {
 	case NONE:
 		if (!onto && !upstream)
@@ -228,7 +235,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 
 		ret = do_interactive_rebase(&opts, flags, switch_to, upstream, onto,
 					    onto_name, squash_onto, head_name, restrict_revision,
-					    raw_strategies, cmd, autosquash);
+					    raw_strategies, &commands, autosquash);
 		break;
 	case SKIP: {
 		struct string_list merge_rr = STRING_LIST_INIT_DUP;
@@ -262,7 +269,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = rearrange_squash(the_repository);
 		break;
 	case ADD_EXEC:
-		ret = sequencer_add_exec_commands(the_repository, cmd);
+		ret = sequencer_add_exec_commands(the_repository, &commands);
 		break;
 	default:
 		BUG("invalid command '%d'", command);
sequencer.c +68 -42

146 lines not on this page · raw message

sequencer.h +3 -2
index 07a8418b0d..1d9d66b0cd 100644
@@ -144,12 +144,13 @@ int sequencer_make_script(struct repository *r, FILE *out, int argc,
 			  const char **argv,
 			  unsigned flags);
 
-int sequencer_add_exec_commands(struct repository *r, const char *command);
+int sequencer_add_exec_commands(struct repository *r,
+				struct string_list *commands);
 int transform_todo_file(struct repository *r, unsigned flags);
 int check_todo_list_from_file(struct repository *r);
 int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
-		    const char *onto, const char *orig_head, const char *cmd,
+		    const char *onto, const char *orig_head, struct string_list *commands,
 		    unsigned autosquash);
 int rearrange_squash(struct repository *r);
 
Signature · 1 line
2.20.1
#86 lore raw

PATCH v4 8/16 sequencer: refactor rearrange_squash() to work on a todo_list

In reply to Alban Gruin

This refactors rearrange_squash() to work on a todo_list to avoid redundant reads and writes. The function is renamed todo_list_rearrange_squash().

The old version created a new buffer, which was directly written to the disk. This new version creates a new item list by just copying items from the old item list, without creating a new buffer. This eliminates the need to reparse the todo list, but this also means its buffer cannot be directly written to the disk.

As rebase -p still need to check the todo list from the disk, a new function is introduced, rearrange_squash_in_todo_file().

complete_action() still uses rearrange_squash_in_todo_file() for now. This will be changed in a future commit.

---
Slight rewording of the commit message and changes due to conflicts with nd/the-index.

 builtin/rebase--interactive.c |  2 +-
 sequencer.c                   | 92 ++++++++++++++++++-----------------
 sequencer.h                   |  2 +-
 3 files changed, 49 insertions(+), 47 deletions(-)
builtin/rebase--interactive.c +1 -1
index 53056ee713..4f0eae9239 100644
@@ -266,7 +266,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = check_todo_list_from_file(the_repository);
 		break;
 	case REARRANGE_SQUASH:
-		ret = rearrange_squash(the_repository);
+		ret = rearrange_squash_in_todo_file(the_repository);
 		break;
 	case ADD_EXEC:
 		ret = sequencer_add_exec_commands(the_repository, &commands);
sequencer.c +47 -45

156 lines not on this page · raw message

sequencer.h +1 -1
index 1d9d66b0cd..e680fe2daa 100644
@@ -152,7 +152,7 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, struct string_list *commands,
 		    unsigned autosquash);
-int rearrange_squash(struct repository *r);
+int rearrange_squash_in_todo_file(struct repository *r);
 
 extern const char sign_off_header[];
 
Signature · 1 line
2.20.1
#87 lore raw

PATCH v4 9/16 sequencer: make sequencer_make_script() write its script to a strbuf

In reply to Alban Gruin

This makes sequencer_make_script() write its script to a strbuf (ie. the buffer of a todo_list) instead of a FILE. This reduce the amount of read/write made by rebase interactive.

---
Changes due to conflicts with nd/the-index.

 builtin/rebase--interactive.c | 13 ++++++-----
 sequencer.c                   | 41 +++++++++++++++--------------------
 sequencer.h                   |  5 ++---
 3 files changed, 28 insertions(+), 31 deletions(-)
builtin/rebase--interactive.c +8 -5
index 4f0eae9239..92026739c9 100644
@@ -71,7 +71,8 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	const char *head_hash = NULL;
 	char *revisions = NULL, *shortrevisions = NULL;
 	struct argv_array make_script_args = ARGV_ARRAY_INIT;
-	FILE *todo_list;
+	FILE *todo_list_file;
+	struct todo_list todo_list = TODO_LIST_INIT;
 
 	if (prepare_branch_to_be_rebased(opts, switch_to))
 		return -1;
@@ -93,8 +94,8 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	if (!upstream && squash_onto)
 		write_file(path_squash_onto(), "%s\n", squash_onto);
 
-	todo_list = fopen(rebase_path_todo(), "w");
-	if (!todo_list) {
+	todo_list_file = fopen(rebase_path_todo(), "w");
+	if (!todo_list_file) {
 		free(revisions);
 		free(shortrevisions);
 
@@ -105,10 +106,11 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	if (restrict_revision)
 		argv_array_push(&make_script_args, restrict_revision);
 
-	ret = sequencer_make_script(the_repository, todo_list,
+	ret = sequencer_make_script(the_repository, &todo_list.buf,
 				    make_script_args.argc, make_script_args.argv,
 				    flags);
-	fclose(todo_list);
+	fputs(todo_list.buf.buf, todo_list_file);
+	fclose(todo_list_file);
 
 	if (ret)
 		error(_("could not generate todo list"));
@@ -121,6 +123,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 
 	free(revisions);
 	free(shortrevisions);
+	todo_list_release(&todo_list);
 	argv_array_clear(&make_script_args);
 
 	return ret;
sequencer.c +18 -23

101 lines not on this page · raw message

sequencer.h +2 -3
index e680fe2daa..d75049c15f 100644
@@ -140,9 +140,8 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_REBASE_COUSINS (1U << 4)
 #define TODO_LIST_APPEND_TODO_HELP (1U << 5)
 
-int sequencer_make_script(struct repository *r, FILE *out, int argc,
-			  const char **argv,
-			  unsigned flags);
+int sequencer_make_script(struct repository *r, struct strbuf *out, int argc,
+			  const char **argv, unsigned flags);
 
 int sequencer_add_exec_commands(struct repository *r,
 				struct string_list *commands);
Signature · 1 line
2.20.1
#88 lore raw

PATCH v4 10/16 sequencer: change complete_action() to use the refactored functions

In reply to Alban Gruin

complete_action() used functions that read the todo-list file, made some changes to it, and wrote it back to the disk.

The previous commits were dedicated to separate the part that deals with the file from the actual logic of these functions. Now that this is done, we can call directly the "logic" functions to avoid useless file access.

The parsing of the list has to be done by the caller. If the buffer of the todo list provided by the caller is empty, a `noop' command is directly added to the todo list, without touching the buffer.

---
 builtin/rebase--interactive.c | 20 +++------
 sequencer.c                   | 81 +++++++++++++++--------------------
 sequencer.h                   |  2 +-
 3 files changed, 42 insertions(+), 61 deletions(-)
builtin/rebase--interactive.c +6 -14
index 92026739c9..2dbf8fc08b 100644
@@ -71,7 +71,6 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	const char *head_hash = NULL;
 	char *revisions = NULL, *shortrevisions = NULL;
 	struct argv_array make_script_args = ARGV_ARRAY_INIT;
-	FILE *todo_list_file;
 	struct todo_list todo_list = TODO_LIST_INIT;
 
 	if (prepare_branch_to_be_rebased(opts, switch_to))
@@ -94,14 +93,6 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	if (!upstream && squash_onto)
 		write_file(path_squash_onto(), "%s\n", squash_onto);
 
-	todo_list_file = fopen(rebase_path_todo(), "w");
-	if (!todo_list_file) {
-		free(revisions);
-		free(shortrevisions);
-
-		return error_errno(_("could not open %s"), rebase_path_todo());
-	}
-
 	argv_array_pushl(&make_script_args, "", revisions, NULL);
 	if (restrict_revision)
 		argv_array_push(&make_script_args, restrict_revision);
@@ -109,16 +100,17 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	ret = sequencer_make_script(the_repository, &todo_list.buf,
 				    make_script_args.argc, make_script_args.argv,
 				    flags);
-	fputs(todo_list.buf.buf, todo_list_file);
-	fclose(todo_list_file);
 
 	if (ret)
 		error(_("could not generate todo list"));
 	else {
 		discard_cache();
-		ret = complete_action(the_repository, opts, flags,
-				      shortrevisions, onto_name, onto,
-				      head_hash, commands, autosquash);
+		if (todo_list_parse_insn_buffer(the_repository, todo_list.buf.buf,
+						&todo_list))
+			BUG("unusable todo list");
+
+		ret = complete_action(the_repository, opts, flags, shortrevisions, onto_name,
+				      onto, head_hash, commands, autosquash, &todo_list);
 	}
 
 	free(revisions);
sequencer.c +35 -46

135 lines not on this page · raw message

sequencer.h +1 -1
index d75049c15f..dcbd682e63 100644
@@ -150,7 +150,7 @@ int check_todo_list_from_file(struct repository *r);
 int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, struct string_list *commands,
-		    unsigned autosquash);
+		    unsigned autosquash, struct todo_list *todo_list);
 int rearrange_squash_in_todo_file(struct repository *r);
 
 extern const char sign_off_header[];
Signature · 1 line
2.20.1
#89 lore raw

PATCH v4 11/16 sequencer: refactor skip_unnecessary_picks() to work on a todo_list

In reply to Alban Gruin

This refactors skip_unnecessary_picks() to work on a todo_list. As this function is only called by complete_action() (and thus is not used by rebase -p), the file-handling logic is completely dropped here.

Instead of truncating the todo list’s buffer, the items are moved to the beginning of the list, eliminating the need to reparse the list. This also means its buffer cannot be directly written to the disk.

rewrite_file() is then removed, as it is now unused.

---
 sequencer.c | 78 ++++++++++++-----------------------------------------
 1 file changed, 17 insertions(+), 61 deletions(-)
sequencer.c +17 -61
index ea5cea81a8..dfdba5cec0 100644
@@ -4655,52 +4655,22 @@ int check_todo_list_from_file(struct repository *r)
 	return res;
 }
 
-static int rewrite_file(const char *path, const char *buf, size_t len)
-{
-	int rc = 0;
-	int fd = open(path, O_WRONLY | O_TRUNC);
-	if (fd < 0)
-		return error_errno(_("could not open '%s' for writing"), path);
-	if (write_in_full(fd, buf, len) < 0)
-		rc = error_errno(_("could not write to '%s'"), path);
-	if (close(fd) && !rc)
-		rc = error_errno(_("could not close '%s'"), path);
-	return rc;
-}
-
 /* skip picking commits whose parents are unchanged */
-static int skip_unnecessary_picks(struct repository *r, struct object_id *output_oid)
+static int skip_unnecessary_picks(struct repository *r,
+				  struct todo_list *todo_list,
+				  struct object_id *output_oid)
 {
-	const char *todo_file = rebase_path_todo();
-	struct strbuf buf = STRBUF_INIT;
-	struct todo_list todo_list = TODO_LIST_INIT;
 	struct object_id *parent_oid;
-	int fd, i;
-
-	if (!read_oneliner(&buf, rebase_path_onto(), 0))
-		return error(_("could not read 'onto'"));
-	if (get_oid(buf.buf, output_oid)) {
-		strbuf_release(&buf);
-		return error(_("need a HEAD to fixup"));
-	}
-	strbuf_release(&buf);
-
-	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
-		return -1;
-	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list) < 0) {
-		todo_list_release(&todo_list);
-		return -1;
-	}
+	int i;
 
-	for (i = 0; i < todo_list.nr; i++) {
-		struct todo_item *item = todo_list.items + i;
+	for (i = 0; i < todo_list->nr; i++) {
+		struct todo_item *item = todo_list->items + i;
 
 		if (item->command >= TODO_NOOP)
 			continue;
 		if (item->command != TODO_PICK)
 			break;
 		if (parse_commit(item->commit)) {
-			todo_list_release(&todo_list);
 			return error(_("could not parse commit '%s'"),
 				oid_to_hex(&item->commit->object.oid));
 		}
@@ -4714,37 +4684,21 @@ static int skip_unnecessary_picks(struct repository *r, struct object_id *output
 		oidcpy(output_oid, &item->commit->object.oid);
 	}
 	if (i > 0) {
-		int offset = get_item_line_offset(&todo_list, i);
 		const char *done_path = rebase_path_done();
 
-		fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
-		if (fd < 0) {
-			error_errno(_("could not open '%s' for writing"),
-				    done_path);
-			todo_list_release(&todo_list);
-			return -1;
-		}
-		if (write_in_full(fd, todo_list.buf.buf, offset) < 0) {
+		if (todo_list_write_to_file(r, todo_list, done_path, NULL, NULL, i, 0)) {
 			error_errno(_("could not write to '%s'"), done_path);
-			todo_list_release(&todo_list);
-			close(fd);
 			return -1;
 		}
-		close(fd);
 
-		if (rewrite_file(rebase_path_todo(), todo_list.buf.buf + offset,
-				 todo_list.buf.len - offset) < 0) {
-			todo_list_release(&todo_list);
-			return -1;
-		}
+		MOVE_ARRAY(todo_list->items, todo_list->items + i, todo_list->nr - i);
+		todo_list->nr -= i;
+		todo_list->current = 0;
 
-		todo_list.current = i;
-		if (is_fixup(peek_command(&todo_list, 0)))
-			record_in_rewritten(output_oid, peek_command(&todo_list, 0));
+		if (is_fixup(peek_command(todo_list, 0)))
+			record_in_rewritten(output_oid, peek_command(todo_list, 0));
 	}
 
-	todo_list_release(&todo_list);
-
 	return 0;
 }
 
@@ -4817,6 +4771,11 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla
 		return -1;
 	}
 
+	if (opts->allow_ff && skip_unnecessary_picks(r, &new_todo, &oid)) {
+		todo_list_release(&new_todo);
+		return error(_("could not skip unnecessary pick commands"));
+	}
+
 	if (todo_list_write_to_file(r, &new_todo, todo_file, NULL, NULL, -1,
 				    flags & ~(TODO_LIST_SHORTEN_IDS))) {
 		todo_list_release(&new_todo);
@@ -4825,9 +4784,6 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla
 
 	todo_list_release(&new_todo);
 
-	if (opts->allow_ff && skip_unnecessary_picks(r, &oid))
-		return error(_("could not skip unnecessary pick commands"));
-
 	if (checkout_onto(opts, onto_name, oid_to_hex(&oid), orig_head))
 		return -1;
 
Signature · 1 line
2.20.1
#90 lore raw

PATCH v4 12/16 rebase-interactive: use todo_list_write_to_file() in edit_todo_list()

In reply to Alban Gruin

Just like complete_action(), edit_todo_list() used a function (transform_todo_file()) that read the todo list from the disk and wrote it back, resulting in useless disk accesses.

This changes edit_todo_list() to call directly todo_list_write_to_file() instead.

---
Changes due to conflicts with nd/the-index.

 rebase-interactive.c | 38 ++++++++++++++++----------------------
 1 file changed, 16 insertions(+), 22 deletions(-)
rebase-interactive.c +16 -22
index 6157247e1f..994f0f9753 100644
@@ -79,39 +79,33 @@ void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 
 int edit_todo_list(struct repository *r, unsigned flags)
 {
-	struct strbuf buf = STRBUF_INIT;
 	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res = 0;
 
-	if (strbuf_read_file(&buf, todo_file, 0) < 0)
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
 
-	strbuf_stripspace(&buf, 1);
-	if (write_message(buf.buf, buf.len, todo_file, 0)) {
-		strbuf_release(&buf);
+	strbuf_stripspace(&todo_list.buf, 1);
+	todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list);
+	if (todo_list_write_to_file(r, &todo_list, todo_file, NULL, NULL, -1,
+				    flags | TODO_LIST_SHORTEN_IDS | TODO_LIST_APPEND_TODO_HELP)) {
+		todo_list_release(&todo_list);
 		return -1;
 	}
 
-	strbuf_release(&buf);
-
-	transform_todo_file(r, flags | TODO_LIST_SHORTEN_IDS);
-
-	if (strbuf_read_file(&buf, todo_file, 0) < 0)
-		return error_errno(_("could not read '%s'."), todo_file);
-
-	append_todo_help(1, 0, &buf);
-	if (write_message(buf.buf, buf.len, todo_file, 0)) {
-		strbuf_release(&buf);
+	strbuf_reset(&todo_list.buf);
+	if (launch_sequence_editor(todo_file, &todo_list.buf, NULL)) {
+		todo_list_release(&todo_list);
 		return -1;
 	}
 
-	strbuf_release(&buf);
+	if (!todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list))
+		res = todo_list_write_to_file(r, &todo_list, todo_file, NULL, NULL, -1,
+					      flags & ~(TODO_LIST_SHORTEN_IDS));
 
-	if (launch_sequence_editor(todo_file, NULL, NULL))
-		return -1;
-
-	transform_todo_file(r, flags & ~(TODO_LIST_SHORTEN_IDS));
-
-	return 0;
+	todo_list_release(&todo_list);
+	return res;
 }
 
 define_commit_slab(commit_seen, unsigned char);
Signature · 1 line
2.20.1
#91 lore raw

PATCH v4 13/16 rebase-interactive: append_todo_help() changes

In reply to Alban Gruin

This moves the writing of the comment "Rebase $shortrevisions onto $shortonto ($command_count commands)" from todo_list_write_to_file() to append_todo_help().

shortrevisions, shortonto, and command_count are passed as parameters to append_todo_help().

During the initial edit of the todo list, shortrevisions and shortonto are not NULL. Therefore, if shortrevisions or shortonto is NULL, then edit_todo would be true, otherwise it would be false. Thus, edit_todo is removed from the parameters of append_todo_help().

---
Slight rewording of the message and changes due to conflicts with nd/the-index.

 rebase-interactive.c | 12 +++++++++++-
 rebase-interactive.h |  3 ++-
 sequencer.c          | 17 ++++-------------
 3 files changed, 17 insertions(+), 15 deletions(-)
rebase-interactive.c +11 -1
index 994f0f9753..32f95002df 100644
@@ -28,7 +28,8 @@ static enum missing_commit_check_level get_missing_commit_check_level(void)
 	return MISSING_COMMIT_CHECK_IGNORE;
 }
 
-void append_todo_help(unsigned edit_todo, unsigned keep_empty,
+void append_todo_help(unsigned keep_empty, int command_count,
+		      const char *shortrevisions, const char *shortonto,
 		      struct strbuf *buf)
 {
 	const char *msg = _("\nCommands:\n"
@@ -48,6 +49,15 @@ void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 ".       specified). Use -c <commit> to reword the commit message.\n"
 "\n"
 "These lines can be re-ordered; they are executed from top to bottom.\n");
+	unsigned edit_todo = !(shortrevisions && shortonto);
+
+	if (!edit_todo) {
+		strbuf_addch(buf, '\n');
+		strbuf_commented_addf(buf, Q_("Rebase %s onto %s (%d command)",
+					      "Rebase %s onto %s (%d commands)",
+					      command_count),
+				      shortrevisions, shortonto, command_count);
+	}
 
 	strbuf_add_commented_lines(buf, msg, strlen(msg));
 
rebase-interactive.h +2 -1
index dd0d717bc1..2f6675eabd 100644
@@ -4,7 +4,8 @@
 struct strbuf;
 struct repository;
 
-void append_todo_help(unsigned edit_todo, unsigned keep_empty,
+void append_todo_help(unsigned keep_empty, int command_count,
+		      const char *shortrevisions, const char *shortonto,
 		      struct strbuf *buf);
 int edit_todo_list(struct repository *r, unsigned flags);
 int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
sequencer.c +4 -13
index dfdba5cec0..dfc705291d 100644
@@ -4574,22 +4574,13 @@ int todo_list_write_to_file(struct repository *r, struct todo_list *todo_list,
 			    const char *file, const char *shortrevisions,
 			    const char *shortonto, int num, unsigned flags)
 {
-	int edit_todo = !(shortrevisions && shortonto), res;
+	int res;
 	struct strbuf buf = STRBUF_INIT;
 
 	todo_list_to_strbuf(r, todo_list, &buf, num, flags);
-
-	if (flags & TODO_LIST_APPEND_TODO_HELP) {
-		int command_count = count_commands(todo_list);
-		if (!edit_todo) {
-			strbuf_addch(&buf, '\n');
-			strbuf_commented_addf(&buf, Q_("Rebase %s onto %s (%d command)",
-						       "Rebase %s onto %s (%d commands)",
-						       command_count),
-					      shortrevisions, shortonto, command_count);
-		}
-		append_todo_help(edit_todo, flags & TODO_LIST_KEEP_EMPTY, &buf);
-	}
+	if (flags & TODO_LIST_APPEND_TODO_HELP)
+		append_todo_help(flags & TODO_LIST_KEEP_EMPTY, count_commands(todo_list),
+				 shortrevisions, shortonto, &buf);
 
 	res = write_message(buf.buf, buf.len, file, 0);
 	strbuf_release(&buf);
Signature · 1 line
2.20.1
#92 lore raw

PATCH v4 14/16 rebase-interactive: rewrite edit_todo_list() to handle the initial edit

In reply to Alban Gruin

edit_todo_list() is changed to work on a todo_list, and to handle the initial edition of the todo list (ie. making a backup of the todo list).

It does not check for dropped commits yet, as todo_list_check() does not take the commits that have already been processed by the rebase (ie. the todo list is edited in the middle of a rebase session).

---
Changes due to conflicts with nd/the-index.

 builtin/rebase--interactive.c | 24 +++++++++++++++++-
 rebase-interactive.c          | 48 ++++++++++++++++++-----------------
 rebase-interactive.h          |  4 ++-
 sequencer.c                   |  3 +--
 sequencer.h                   |  1 +
 5 files changed, 53 insertions(+), 27 deletions(-)
builtin/rebase--interactive.c +23 -1
index 2dbf8fc08b..645ac587f7 100644
@@ -13,6 +13,28 @@ static GIT_PATH_FUNC(path_state_dir, "rebase-merge/")
 static GIT_PATH_FUNC(path_squash_onto, "rebase-merge/squash-onto")
 static GIT_PATH_FUNC(path_interactive, "rebase-merge/interactive")
 
+static int edit_todo_file(unsigned flags)
+{
+	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT,
+		new_todo = TODO_LIST_INIT;
+
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
+		return error_errno(_("could not read '%s'."), todo_file);
+
+	strbuf_stripspace(&todo_list.buf, 1);
+	if (!edit_todo_list(the_repository, &todo_list,
+			    &new_todo, NULL, NULL, flags) &&
+	    todo_list_write_to_file(the_repository, &new_todo, todo_file, NULL, NULL,
+				    -1, flags & ~(TODO_LIST_SHORTEN_IDS)) < 0)
+		return error_errno(_("could not write '%s'"), todo_file);
+
+	todo_list_release(&todo_list);
+	todo_list_release(&new_todo);
+
+	return 0;
+}
+
 static int get_revision_ranges(const char *upstream, const char *onto,
 			       const char **head_hash,
 			       char **revisions, char **shortrevisions)
@@ -242,7 +264,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		break;
 	}
 	case EDIT_TODO:
-		ret = edit_todo_list(the_repository, flags);
+		ret = edit_todo_file(flags);
 		break;
 	case SHOW_CURRENT_PATCH: {
 		struct child_process cmd = CHILD_PROCESS_INIT;
rebase-interactive.c +25 -23

60 lines not on this page · raw message

rebase-interactive.h +3 -1
index 2f6675eabd..42cc3f865d 100644
@@ -7,7 +7,9 @@ struct repository;
 void append_todo_help(unsigned keep_empty, int command_count,
 		      const char *shortrevisions, const char *shortonto,
 		      struct strbuf *buf);
-int edit_todo_list(struct repository *r, unsigned flags);
+int edit_todo_list(struct repository *r, struct todo_list *todo_list,
+		   struct todo_list *new_todo, const char *shortrevisions,
+		   const char *shortonto, unsigned flags);
 int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
 
 #endif
sequencer.c +1 -2
index dfc705291d..0500a32e80 100644
@@ -55,8 +55,7 @@ static GIT_PATH_FUNC(rebase_path, "rebase-merge")
  * file and written to the tail of 'done'.
  */
 GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
-static GIT_PATH_FUNC(rebase_path_todo_backup,
-		     "rebase-merge/git-rebase-todo.backup")
+GIT_PATH_FUNC(rebase_path_todo_backup, "rebase-merge/git-rebase-todo.backup")
 
 /*
  * The rebase command lines that have already been processed. A line
sequencer.h +1 -0
index dcbd682e63..7dd85bb399 100644
@@ -10,6 +10,7 @@ struct repository;
 const char *git_path_commit_editmsg(void);
 const char *git_path_seq_dir(void);
 const char *rebase_path_todo(void);
+const char *rebase_path_todo_backup(void);
 
 #define APPEND_SIGNOFF_DEDUP (1u << 0)
 
Signature · 1 line
2.20.1
#93 lore raw

PATCH v4 15/16 sequencer: use edit_todo_list() in complete_action()

In reply to Alban Gruin

This changes complete_action() to use edit_todo_list(), now that it can handle the initial edit of the todo list.

---
Changes due to conflicts with nd/the-index.

 sequencer.c | 21 +++++++--------------
 1 file changed, 7 insertions(+), 14 deletions(-)
sequencer.c +7 -14
index 0500a32e80..127bb0b68e 100644
@@ -4703,6 +4703,7 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla
 	struct todo_list new_todo = TODO_LIST_INIT;
 	struct strbuf *buf = &todo_list->buf;
 	struct object_id oid;
+	int res;
 
 	get_oid(onto, &oid);
 	shortonto = find_unique_abbrev(&oid, DEFAULT_ABBREV);
@@ -4727,24 +4728,16 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla
 		return error(_("nothing to do"));
 	}
 
-	if (todo_list_write_to_file(r, todo_list, todo_file,
-				    shortrevisions, shortonto, -1,
-				    flags | TODO_LIST_SHORTEN_IDS | TODO_LIST_APPEND_TODO_HELP))
-		return error_errno(_("could not write '%s'"), todo_file);
-
-	if (copy_file(rebase_path_todo_backup(), todo_file, 0666))
-		return error(_("could not copy '%s' to '%s'."), todo_file,
-			     rebase_path_todo_backup());
-
-	if (launch_sequence_editor(todo_file, &new_todo.buf, NULL)) {
+	res = edit_todo_list(r, todo_list, &new_todo, shortrevisions,
+			     shortonto, flags);
+	if (res == -1)
+		return -1;
+	else if (res == -2) {
 		apply_autostash(opts);
 		sequencer_remove_state(opts);
 
 		return -1;
-	}
-
-	strbuf_stripspace(&new_todo.buf, 1);
-	if (new_todo.buf.len == 0) {
+	} else if (res == -3) {
 		apply_autostash(opts);
 		sequencer_remove_state(opts);
 		todo_list_release(&new_todo);
Signature · 1 line
2.20.1
#94 lore raw

PATCH v4 16/16 rebase--interactive: move transform_todo_file() to rebase--interactive.c

In reply to Alban Gruin

As transform_todo_file() is only needed inside of rebase--interactive.c, it is moved there from sequencer.c.

---
 builtin/rebase--interactive.c | 26 +++++++++++++++++++++++++-
 sequencer.c                   | 23 -----------------------
 sequencer.h                   |  1 -
 3 files changed, 25 insertions(+), 25 deletions(-)
builtin/rebase--interactive.c +25 -1
index 645ac587f7..7f1e88a087 100644
@@ -35,6 +35,30 @@ static int edit_todo_file(unsigned flags)
 	return 0;
 }
 
+static int transform_todo_file(unsigned flags)
+{
+	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res;
+
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
+		return error_errno(_("could not read '%s'."), todo_file);
+
+	if (todo_list_parse_insn_buffer(the_repository, todo_list.buf.buf,
+					&todo_list)) {
+		todo_list_release(&todo_list);
+		return error(_("unusable todo list: '%s'"), todo_file);
+	}
+
+	res = todo_list_write_to_file(the_repository, &todo_list, todo_file,
+				      NULL, NULL, -1, flags);
+	todo_list_release(&todo_list);
+
+	if (res)
+		return error_errno(_("could not write '%s'."), todo_file);
+	return 0;
+}
+
 static int get_revision_ranges(const char *upstream, const char *onto,
 			       const char **head_hash,
 			       char **revisions, char **shortrevisions)
@@ -277,7 +301,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 	}
 	case SHORTEN_OIDS:
 	case EXPAND_OIDS:
-		ret = transform_todo_file(the_repository, flags);
+		ret = transform_todo_file(flags);
 		break;
 	case CHECK_TODO_LIST:
 		ret = check_todo_list_from_file(the_repository);
sequencer.c +0 -23
index 127bb0b68e..6566247a7c 100644
@@ -4587,29 +4587,6 @@ int todo_list_write_to_file(struct repository *r, struct todo_list *todo_list,
 	return res;
 }
 
-int transform_todo_file(struct repository *r, unsigned flags)
-{
-	const char *todo_file = rebase_path_todo();
-	struct todo_list todo_list = TODO_LIST_INIT;
-	int res;
-
-	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
-		return error_errno(_("could not read '%s'."), todo_file);
-
-	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
-		todo_list_release(&todo_list);
-		return error(_("unusable todo list: '%s'"), todo_file);
-	}
-
-	res = todo_list_write_to_file(r, &todo_list, todo_file,
-				      NULL, NULL, -1, flags);
-	todo_list_release(&todo_list);
-
-	if (res)
-		return error_errno(_("could not write '%s'."), todo_file);
-	return 0;
-}
-
 static const char edit_todo_list_advice[] =
 N_("You can fix this with 'git rebase --edit-todo' "
 "and then run 'git rebase --continue'.\n"
sequencer.h +0 -1
index 7dd85bb399..bb3ca5783b 100644
@@ -146,7 +146,6 @@ int sequencer_make_script(struct repository *r, struct strbuf *out, int argc,
 
 int sequencer_add_exec_commands(struct repository *r,
 				struct string_list *commands);
-int transform_todo_file(struct repository *r, unsigned flags);
 int check_todo_list_from_file(struct repository *r);
 int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
Signature · 1 line
2.20.1
#95 lore raw

v5 cover sequencer: refactor functions working on a todo_list

In reply to Alban Gruin

At the center of the "interactive" part of the interactive rebase lies the todo list. When the user starts an interactive rebase, a todo list is generated, presented to the user (who then edits it using a text editor), read back, and then is checked and processed before the actual rebase takes place.

Some of this processing includes adding execs commands, reordering fixup! and squash! commits, and checking if no commits were accidentally dropped by the user.

Before I converted the interactive rebase in C, these functions were called by git-rebase--interactive.sh through git-rebase--helper. Since the only way to pass around a large amount of data between a shell script and a C program is to use a file (or any declination of a file), the functions that checked and processed the todo list were directly working on a file, the same file that the user edited.

During the conversion, I did not address this issue, which lead to a complete_action() that reads the todo list file, does some computation based on its content, and writes it back to the disk, several times in the same function.

As it is not an efficient way to handle a data structure, this patch series refactor the functions that processes the todo list to work on a todo_list structure instead of reading it from the disk.

Some commits consists in modifying edit_todo_list() (initially used by --edit-todo) to handle the initial edition of the todo list, to increase code sharing.

This is based on nd/the-index (36e7ed69de, "rebase-interactive.c: remove the_repository references").

Changes since v4:

- Adding todo_item_get_arg() to return the address of the parameter of an item to avoid confusion.

- Squashed <[email protected]> and <[email protected]> from Ramsay Jones.

Alban Gruin (16):
sequencer: changes in parse_insn_buffer()
sequencer: make the todo_list structure public
sequencer: remove the 'arg' field from todo_item
sequencer: refactor transform_todos() to work on a todo_list
sequencer: introduce todo_list_write_to_file()
sequencer: refactor check_todo_list() to work on a todo_list
sequencer: refactor sequencer_add_exec_commands() to work on a
todo_list
sequencer: refactor rearrange_squash() to work on a todo_list
sequencer: make sequencer_make_script() write its script to a strbuf
sequencer: change complete_action() to use the refactored functions
sequencer: refactor skip_unnecessary_picks() to work on a todo_list
rebase-interactive: use todo_list_write_to_file() in edit_todo_list()
rebase-interactive: append_todo_help() changes
rebase-interactive: rewrite edit_todo_list() to handle the initial
edit
sequencer: use edit_todo_list() in complete_action()
rebase--interactive: move transform_todo_file() to
rebase--interactive.c

 builtin/rebase--interactive.c |  90 +++--
 rebase-interactive.c          | 143 +++++--
 rebase-interactive.h          |   9 +-
 sequencer.c                   | 687 ++++++++++++++--------------------
 sequencer.h                   |  81 +++-
 5 files changed, 533 insertions(+), 477 deletions(-)
Signature · 1 line
2.20.1
#96 lore raw

PATCH v5 1/16 sequencer: changes in parse_insn_buffer()

This clears the number of items of a todo_list before parsing it to allow to parse the same list multiple times without issues. As its items are not dynamically allocated, or don’t need to allocate memory, no additionnal memory management is required here.

Furthermore, if a line is invalid, the type of the corresponding command is set to a garbage value, and its argument is defined properly. This will allow to recreate the text of a todo list from its commands, even if one of them is incorrect.

---
Unchanged since v4.

 sequencer.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
sequencer.c +6 -1
index d726f77e11..a7afaf6882 100644
@@ -2091,6 +2091,8 @@ static int parse_insn_buffer(struct repository *r, char *buf,
 	char *p = buf, *next_p;
 	int i, res = 0, fixup_okay = file_exists(rebase_path_done());
 
+	todo_list->current = todo_list->nr = 0;
+
 	for (i = 1; *p; i++, p = next_p) {
 		char *eol = strchrnul(p, '\n');
 
@@ -2104,7 +2106,10 @@ static int parse_insn_buffer(struct repository *r, char *buf,
 		if (parse_insn_line(r, item, p, eol)) {
 			res = error(_("invalid line %d: %.*s"),
 				i, (int)(eol - p), p);
-			item->command = TODO_NOOP;
+			item->command = TODO_COMMENT + 1;
+			item->arg = p;
+			item->arg_len = (int)(eol - p);
+			item->commit = NULL;
 		}
 
 		if (fixup_okay)
Signature · 1 line
2.20.1
#97 lore raw

PATCH v5 2/16 sequencer: make the todo_list structure public

In reply to Alban Gruin

This makes the structures todo_list and todo_item, and the functions todo_list_release() and parse_insn_buffer(), accessible outside of sequencer.c.

---
Unchanged since v4.

 sequencer.c | 69 ++++++++++-------------------------------------------
 sequencer.h | 50 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 62 insertions(+), 57 deletions(-)
sequencer.c +12 -57
index a7afaf6882..5b84a20532 100644
@@ -1460,32 +1460,6 @@ static int allow_empty(struct repository *r,
 		return 1;
 }
 
-/*
- * Note that ordering matters in this enum. Not only must it match the mapping
- * below, it is also divided into several sections that matter.  When adding
- * new commands, make sure you add it in the right section.
- */
-enum todo_command {
-	/* commands that handle commits */
-	TODO_PICK = 0,
-	TODO_REVERT,
-	TODO_EDIT,
-	TODO_REWORD,
-	TODO_FIXUP,
-	TODO_SQUASH,
-	/* commands that do something else than handling a single commit */
-	TODO_EXEC,
-	TODO_BREAK,
-	TODO_LABEL,
-	TODO_RESET,
-	TODO_MERGE,
-	/* commands that do nothing but are counted for reporting progress */
-	TODO_NOOP,
-	TODO_DROP,
-	/* comments (not counted for reporting progress) */
-	TODO_COMMENT
-};
-
 static struct {
 	char c;
 	const char *str;
@@ -1962,26 +1936,7 @@ enum todo_item_flags {
 	TODO_EDIT_MERGE_MSG = 1
 };
 
-struct todo_item {
-	enum todo_command command;
-	struct commit *commit;
-	unsigned int flags;
-	const char *arg;
-	int arg_len;
-	size_t offset_in_buf;
-};
-
-struct todo_list {
-	struct strbuf buf;
-	struct todo_item *items;
-	int nr, alloc, current;
-	int done_nr, total_nr;
-	struct stat_data stat;
-};
-
-#define TODO_LIST_INIT { STRBUF_INIT }
-
-static void todo_list_release(struct todo_list *todo_list)
+void todo_list_release(struct todo_list *todo_list)
 {
 	strbuf_release(&todo_list->buf);
 	FREE_AND_NULL(todo_list->items);
@@ -2084,8 +2039,8 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 	return !item->commit;
 }
 
-static int parse_insn_buffer(struct repository *r, char *buf,
-			     struct todo_list *todo_list)
+int todo_list_parse_insn_buffer(struct repository *r, char *buf,
+				struct todo_list *todo_list)
 {
 	struct todo_item *item;
 	char *p = buf, *next_p;
@@ -2184,7 +2139,7 @@ static int read_populate_todo(struct repository *r,
 		return error(_("could not stat '%s'"), todo_file);
 	fill_stat_data(&todo_list->stat, &st);
 
-	res = parse_insn_buffer(r, todo_list->buf.buf, todo_list);
+	res = todo_list_parse_insn_buffer(r, todo_list->buf.buf, todo_list);
 	if (res) {
 		if (is_rebase_i(opts))
 			return error(_("please fix this using "
@@ -2215,7 +2170,7 @@ static int read_populate_todo(struct repository *r,
 		FILE *f = fopen_or_warn(rebase_path_msgtotal(), "w");
 
 		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
-		    !parse_insn_buffer(r, done.buf.buf, &done))
+		    !todo_list_parse_insn_buffer(r, done.buf.buf, &done))
 			todo_list->done_nr = count_commands(&done);
 		else
 			todo_list->done_nr = 0;
@@ -4501,7 +4456,7 @@ int sequencer_add_exec_commands(struct repository *r,
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4557,7 +4512,7 @@ int transform_todos(struct repository *r, unsigned flags)
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4643,7 +4598,7 @@ int check_todo_list(struct repository *r)
 		goto leave_check;
 	}
 	advise_to_edit_todo = res =
-		parse_insn_buffer(r, todo_list.buf.buf, &todo_list);
+		todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list);
 
 	if (res || check_level == MISSING_COMMIT_CHECK_IGNORE)
 		goto leave_check;
@@ -4662,7 +4617,7 @@ int check_todo_list(struct repository *r)
 		goto leave_check;
 	}
 	strbuf_release(&todo_file);
-	res = !!parse_insn_buffer(r, todo_list.buf.buf, &todo_list);
+	res = !!todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list);
 
 	/* Find commits in git-rebase-todo.backup yet unseen */
 	for (i = todo_list.nr - 1; i >= 0; i--) {
@@ -4744,7 +4699,7 @@ static int skip_unnecessary_picks(struct repository *r, struct object_id *output
 
 	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
 		return -1;
-	if (parse_insn_buffer(r, todo_list.buf.buf, &todo_list) < 0) {
+	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list) < 0) {
 		todo_list_release(&todo_list);
 		return -1;
 	}
@@ -4832,7 +4787,7 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla
 	if (strbuf_read_file(buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(r, buf->buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(r, buf->buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4940,7 +4895,7 @@ int rearrange_squash(struct repository *r)
 
 	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
 		return -1;
-	if (parse_insn_buffer(r, todo_list.buf.buf, &todo_list) < 0) {
+	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list) < 0) {
 		todo_list_release(&todo_list);
 		return -1;
 	}
sequencer.h +50 -0

56 lines not on this page · raw message

Signature · 1 line
2.20.1
#98 lore raw

PATCH v5 3/16 sequencer: remove the 'arg' field from todo_item

In reply to Alban Gruin

The 'arg' field of todo_item used to store the address of the first byte of the parameter of a command in a todo list. It was associated with the length of the parameter (the 'arg_len' field).

This replaces the 'arg' field by 'arg_offset'. This new field does not store the address of the parameter, but the position of the first character of the parameter in the buffer. todo_item_get_arg() is added to return the address of the parameter of an item.

This will prevent todo_list_add_exec_commands() from having to do awful pointer arithmetics when growing the todo list buffer.

---
Introduction and use of todo_item_get_arg().

 sequencer.c | 67 ++++++++++++++++++++++++++++++-----------------------
 sequencer.h |  6 +++--
 2 files changed, 42 insertions(+), 31 deletions(-)
sequencer.c +38 -29
index 5b84a20532..0e7ab16a05 100644
@@ -1949,8 +1949,14 @@ static struct todo_item *append_new_todo(struct todo_list *todo_list)
 	return todo_list->items + todo_list->nr++;
 }
 
+const char *todo_item_get_arg(struct todo_list *todo_list,
+			      struct todo_item *item)
+{
+	return todo_list->buf.buf + item->arg_offset;
+}
+
 static int parse_insn_line(struct repository *r, struct todo_item *item,
-			   const char *bol, char *eol)
+			   const char *buf, const char *bol, char *eol)
 {
 	struct object_id commit_oid;
 	char *end_of_object_name;
@@ -1964,7 +1970,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
 		item->command = TODO_COMMENT;
 		item->commit = NULL;
-		item->arg = bol;
+		item->arg_offset = bol - buf;
 		item->arg_len = eol - bol;
 		return 0;
 	}
@@ -1991,7 +1997,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 			return error(_("%s does not accept arguments: '%s'"),
 				     command_to_string(item->command), bol);
 		item->commit = NULL;
-		item->arg = bol;
+		item->arg_offset = bol - buf;
 		item->arg_len = eol - bol;
 		return 0;
 	}
@@ -2003,7 +2009,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
 	    item->command == TODO_RESET) {
 		item->commit = NULL;
-		item->arg = bol;
+		item->arg_offset = bol - buf;
 		item->arg_len = (int)(eol - bol);
 		return 0;
 	}
@@ -2017,7 +2023,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 		} else {
 			item->flags |= TODO_EDIT_MERGE_MSG;
 			item->commit = NULL;
-			item->arg = bol;
+			item->arg_offset = bol - buf;
 			item->arg_len = (int)(eol - bol);
 			return 0;
 		}
@@ -2029,8 +2035,9 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 	status = get_oid(bol, &commit_oid);
 	*end_of_object_name = saved;
 
-	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
-	item->arg_len = (int)(eol - item->arg);
+	bol = end_of_object_name + strspn(end_of_object_name, " \t");
+	item->arg_offset = bol - buf;
+	item->arg_len = (int)(eol - bol);
 
 	if (status < 0)
 		return -1;
@@ -2058,11 +2065,11 @@ int todo_list_parse_insn_buffer(struct repository *r, char *buf,
 
 		item = append_new_todo(todo_list);
 		item->offset_in_buf = p - todo_list->buf.buf;
-		if (parse_insn_line(r, item, p, eol)) {
+		if (parse_insn_line(r, item, buf, p, eol)) {
 			res = error(_("invalid line %d: %.*s"),
 				i, (int)(eol - p), p);
 			item->command = TODO_COMMENT + 1;
-			item->arg = p;
+			item->arg_offset = p - buf;
 			item->arg_len = (int)(eol - p);
 			item->commit = NULL;
 		}
@@ -2402,7 +2409,7 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 
 		item->command = command;
 		item->commit = commit;
-		item->arg = NULL;
+		item->arg_offset = 0;
 		item->arg_len = 0;
 		item->offset_in_buf = todo_list->buf.len;
 		subject_len = find_commit_subject(commit_buffer, &subject);
@@ -3438,6 +3445,8 @@ static int pick_commits(struct repository *r,
 
 	while (todo_list->current < todo_list->nr) {
 		struct todo_item *item = todo_list->items + todo_list->current;
+		const char *arg = todo_item_get_arg(todo_list, item);
+
 		if (save_todo(todo_list, opts))
 			return -1;
 		if (is_rebase_i(opts)) {
@@ -3488,10 +3497,9 @@ static int pick_commits(struct repository *r,
 					fprintf(stderr,
 						_("Stopped at %s...  %.*s\n"),
 						short_commit_name(commit),
-						item->arg_len, item->arg);
+						item->arg_len, arg);
 				return error_with_patch(r, commit,
-					item->arg, item->arg_len, opts, res,
-					!res);
+					arg, item->arg_len, opts, res, !res);
 			}
 			if (is_rebase_i(opts) && !res)
 				record_in_rewritten(&item->commit->object.oid,
@@ -3500,7 +3508,7 @@ static int pick_commits(struct repository *r,
 				if (res == 1)
 					intend_to_amend();
 				return error_failed_squash(r, item->commit, opts,
-					item->arg_len, item->arg);
+					item->arg_len, arg);
 			} else if (res && is_rebase_i(opts) && item->commit) {
 				int to_amend = 0;
 				struct object_id oid;
@@ -3519,16 +3527,16 @@ static int pick_commits(struct repository *r,
 					to_amend = 1;
 
 				return res | error_with_patch(r, item->commit,
-						item->arg, item->arg_len, opts,
+						arg, item->arg_len, opts,
 						res, to_amend);
 			}
 		} else if (item->command == TODO_EXEC) {
-			char *end_of_arg = (char *)(item->arg + item->arg_len);
+			char *end_of_arg = (char *)(arg + item->arg_len);
 			int saved = *end_of_arg;
 			struct stat st;
 
 			*end_of_arg = '\0';
-			res = do_exec(r, item->arg);
+			res = do_exec(r, arg);
 			*end_of_arg = saved;
 
 			/* Reread the todo file if it has changed. */
@@ -3545,14 +3553,14 @@ static int pick_commits(struct repository *r,
 				todo_list->current = -1;
 			}
 		} else if (item->command == TODO_LABEL) {
-			if ((res = do_label(r, item->arg, item->arg_len)))
+			if ((res = do_label(r, arg, item->arg_len)))
 				reschedule = 1;
 		} else if (item->command == TODO_RESET) {
-			if ((res = do_reset(r, item->arg, item->arg_len, opts)))
+			if ((res = do_reset(r, arg, item->arg_len, opts)))
 				reschedule = 1;
 		} else if (item->command == TODO_MERGE) {
 			if ((res = do_merge(r, item->commit,
-					    item->arg, item->arg_len,
+					    arg, item->arg_len,
 					    item->flags, opts)) < 0)
 				reschedule = 1;
 			else if (item->commit)
@@ -3561,9 +3569,8 @@ static int pick_commits(struct repository *r,
 			if (res > 0)
 				/* failed with merge conflicts */
 				return error_with_patch(r, item->commit,
-							item->arg,
-							item->arg_len, opts,
-							res, 0);
+							arg, item->arg_len,
+							opts, res, 0);
 		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
@@ -3578,9 +3585,8 @@ static int pick_commits(struct repository *r,
 			if (item->commit)
 				return error_with_patch(r,
 							item->commit,
-							item->arg,
-							item->arg_len, opts,
-							res, 0);
+							arg, item->arg_len,
+							opts, res, 0);
 		}
 
 		todo_list->current++;
@@ -4520,7 +4526,8 @@ int transform_todos(struct repository *r, unsigned flags)
 	for (item = todo_list.items, i = 0; i < todo_list.nr; i++, item++) {
 		/* if the item is not a command write it and continue */
 		if (item->command >= TODO_COMMENT) {
-			strbuf_addf(&buf, "%.*s\n", item->arg_len, item->arg);
+			strbuf_addf(&buf, "%.*s\n", item->arg_len,
+				    todo_item_get_arg(&todo_list, item));
 			continue;
 		}
 
@@ -4550,7 +4557,8 @@ int transform_todos(struct repository *r, unsigned flags)
 		if (!item->arg_len)
 			strbuf_addch(&buf, '\n');
 		else
-			strbuf_addf(&buf, " %.*s\n", item->arg_len, item->arg);
+			strbuf_addf(&buf, " %.*s\n", item->arg_len,
+				    todo_item_get_arg(&todo_list, item));
 	}
 
 	i = write_message(buf.buf, buf.len, todo_file, 0);
@@ -4626,7 +4634,8 @@ int check_todo_list(struct repository *r)
 		if (commit && !*commit_seen_at(&commit_seen, commit)) {
 			strbuf_addf(&missing, " - %s %.*s\n",
 				    short_commit_name(commit),
-				    item->arg_len, item->arg);
+				    item->arg_len,
+				    todo_item_get_arg(&todo_list, item));
 			*commit_seen_at(&commit_seen, commit) = 1;
 		}
 	}
sequencer.h +4 -2
index 7dc4d8946b..cdb9ed7a2d 100644
@@ -104,9 +104,9 @@ struct todo_item {
 	enum todo_command command;
 	struct commit *commit;
 	unsigned int flags;
-	const char *arg;
 	int arg_len;
-	size_t offset_in_buf;
+	/* The offset of the command and its argument in the strbuf */
+	size_t offset_in_buf, arg_offset;
 };
 
 struct todo_list {
@@ -122,6 +122,8 @@ struct todo_list {
 int todo_list_parse_insn_buffer(struct repository *r, char *buf,
 				struct todo_list *todo_list);
 void todo_list_release(struct todo_list *todo_list);
+const char *todo_item_get_arg(struct todo_list *todo_list,
+			      struct todo_item *item);
 
 /* Call this to setup defaults before parsing command line options */
 void sequencer_init_config(struct replay_opts *opts);
Signature · 1 line
2.20.1
#99 lore raw

PATCH v5 4/16 sequencer: refactor transform_todos() to work on a todo_list

In reply to Alban Gruin

This refactors transform_todos() to work on a todo_list. The function is renamed todo_list_transform().

As rebase -p still need to check the todo list from the disk, a new function is introduced, transform_todo_file(). It is still used by complete_action() and edit_todo_list() for now, but they will be replaced in a future commit.

todo_list_transform() is not a static function, because it will be used by edit_todo_list() from rebase-interactive.c in a future commit.

---
 builtin/rebase--interactive.c |  2 +-
 rebase-interactive.c          |  4 +--
 sequencer.c                   | 54 +++++++++++++++++++++++------------
 sequencer.h                   |  4 ++-
 4 files changed, 42 insertions(+), 22 deletions(-)
builtin/rebase--interactive.c +1 -1
index dd2a55ab1d..0898eb4c59 100644
@@ -253,7 +253,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 	}
 	case SHORTEN_OIDS:
 	case EXPAND_OIDS:
-		ret = transform_todos(the_repository, flags);
+		ret = transform_todo_file(the_repository, flags);
 		break;
 	case CHECK_TODO_LIST:
 		ret = check_todo_list(the_repository);
rebase-interactive.c +2 -2
index 68aff1dac2..842fa07e7e 100644
@@ -69,7 +69,7 @@ int edit_todo_list(struct repository *r, unsigned flags)
 
 	strbuf_release(&buf);
 
-	transform_todos(r, flags | TODO_LIST_SHORTEN_IDS);
+	transform_todo_file(r, flags | TODO_LIST_SHORTEN_IDS);
 
 	if (strbuf_read_file(&buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
@@ -85,7 +85,7 @@ int edit_todo_list(struct repository *r, unsigned flags)
 	if (launch_sequence_editor(todo_file, NULL, NULL))
 		return -1;
 
-	transform_todos(r, flags & ~(TODO_LIST_SHORTEN_IDS));
+	transform_todo_file(r, flags & ~(TODO_LIST_SHORTEN_IDS));
 
 	return 0;
 }
sequencer.c +36 -18

89 lines not on this page · raw message

sequencer.h +3 -1
index cdb9ed7a2d..35472c137b 100644
@@ -121,6 +121,8 @@ struct todo_list {
 
 int todo_list_parse_insn_buffer(struct repository *r, char *buf,
 				struct todo_list *todo_list);
+void todo_list_transform(struct repository *r, struct todo_list *todo_list,
+			 unsigned flags);
 void todo_list_release(struct todo_list *todo_list);
 const char *todo_item_get_arg(struct todo_list *todo_list,
 			      struct todo_item *item);
@@ -148,7 +150,7 @@ int sequencer_make_script(struct repository *repo, FILE *out,
 			  unsigned flags);
 
 int sequencer_add_exec_commands(struct repository *r, const char *command);
-int transform_todos(struct repository *r, unsigned flags);
+int transform_todo_file(struct repository *r, unsigned flags);
 enum missing_commit_check_level get_missing_commit_check_level(void);
 int check_todo_list(struct repository *r);
 int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
Signature · 1 line
2.20.1
#100 lore raw

PATCH v5 5/16 sequencer: introduce todo_list_write_to_file()

In reply to Alban Gruin

This introduces a new function to recreate the text of a todo list from its commands and write it to a file. This will be useful as the next few commits will change the use of the buffer in struct todo_list so it will no longer be a mirror of the file on disk.

This functionality already exists in todo_list_transform(), but this function was made to replace the buffer of a todo list, which is not what we want here. Thus, the part of todo_list_transform() that replaces the buffer is dropped, and the function is renamed todo_list_to_strbuf(). It is called by todo_list_write_to_file() to fill the buffer to write to the disk.

todo_list_write_to_file() can also take care of appending the help text to the buffer before writing it to the disk, or to write only the first n items of the list. This feature will be used by skip_unnecessary_picks(), which has to write done commands in a file.

---
 sequencer.c | 61 +++++++++++++++++++++++++++++++++++------------------
 sequencer.h | 11 ++++++----
 2 files changed, 48 insertions(+), 24 deletions(-)
sequencer.c +0 -0
index 7a295cbd3f..87b43994ff 100644

--- a/sequencer.c

+++ b/sequencer.c
@@ -4507,26 +4507,28 @@ int sequencer_add_exec_commands(struct repository *r,
return i;
}

-void todo_list_transform(struct repository *r, struct todo_list *todo_list,
- unsigned flags)
+static void todo_list_to_strbuf(struct repository *r, struct todo_list *todo_list,
+ struct strbuf *buf, int num, unsigned flags)
{
- struct strbuf buf = STRBUF_INIT;
struct todo_item *item;
- int i;
+ int i, max = todo_list->nr;

- for (item = todo_list->items, i = 0; i < todo_list->nr; i++, item++) {
+ if (num > 0 && num < max)
+ max = num;
+
+ for (item = todo_list->items, i = 0; i < max; i++, item++) {
/* if the item is not a command write it and continue */ if (item->command >= TODO_COMMENT) {
- strbuf_addf(&buf, "%.*s\n", item->arg_len,
+ strbuf_addf(buf, "%.*s\n", item->arg_len,
todo_item_get_arg(todo_list, item));
continue;
}

/* add command to the buffer */ if (flags & TODO_LIST_ABBREVIATE_CMDS)
- strbuf_addch(&buf, command_to_char(item->command));
+ strbuf_addch(buf, command_to_char(item->command));
else
- strbuf_addstr(&buf, command_to_string(item->command));
+ strbuf_addstr(buf, command_to_string(item->command));

/* add commit id */
if (item->commit) {
@@ -4536,28 +4538,48 @@ void todo_list_transform(struct repository *r, struct todo_list *todo_list,

if (item->command == TODO_MERGE) {
if (item->flags & TODO_EDIT_MERGE_MSG)
- strbuf_addstr(&buf, " -c");
+ strbuf_addstr(buf, " -c");
else
- strbuf_addstr(&buf, " -C");
+ strbuf_addstr(buf, " -C");
}

- strbuf_addf(&buf, " %s", oid);
+ strbuf_addf(buf, " %s", oid);
}

/* add all the rest */
if (!item->arg_len)
- strbuf_addch(&buf, '\n');
+ strbuf_addch(buf, '\n');
else
- strbuf_addf(&buf, " %.*s\n", item->arg_len,
+ strbuf_addf(buf, " %.*s\n", item->arg_len,
todo_item_get_arg(todo_list, item));
}
+}

- strbuf_reset(&todo_list->buf);
- strbuf_add(&todo_list->buf, buf.buf, buf.len);
+int todo_list_write_to_file(struct repository *r, struct todo_list *todo_list,
+ const char *file, const char *shortrevisions,
+ const char *shortonto, int num, unsigned flags)
+{
+ int edit_todo = !(shortrevisions && shortonto), res;
+ struct strbuf buf = STRBUF_INIT;
+
+ todo_list_to_strbuf(r, todo_list, &buf, num, flags);
+
+ if (flags & TODO_LIST_APPEND_TODO_HELP) {
+ int command_count = count_commands(todo_list);
+ if (!edit_todo) {
+ strbuf_addch(&buf, '\n');
+ strbuf_commented_addf(&buf, Q_("Rebase %s onto %s (%d command)",
+ "Rebase %s onto %s (%d commands)",
+ command_count),
+ shortrevisions, shortonto, command_count);
+ }
+ append_todo_help(edit_todo, flags & TODO_LIST_KEEP_EMPTY, &buf);
+ }
+
+ res = write_message(buf.buf, buf.len, file, 0);
strbuf_release(&buf);

- if (todo_list_parse_insn_buffer(r, todo_list->buf.buf, todo_list))
- BUG("unusable todo list");
+ return res;
}

int transform_todo_file(struct repository *r, unsigned flags)
@@ -4574,9 +4596,8 @@ int transform_todo_file(struct repository *r, unsigned flags)
return error(_("unusable todo list: '%s'"), todo_file);
}

- todo_list_transform(r, &todo_list, flags);
-
- res = write_message(todo_list.buf.buf, todo_list.buf.len, todo_file, 0);
+ res = todo_list_write_to_file(r, &todo_list, todo_file,
+ NULL, NULL, -1, flags);
todo_list_release(&todo_list);

if (res)

sequencer.h +7 -4
index 35472c137b..dc3d8f76f7 100644
@@ -121,8 +121,9 @@ struct todo_list {
 
 int todo_list_parse_insn_buffer(struct repository *r, char *buf,
 				struct todo_list *todo_list);
-void todo_list_transform(struct repository *r, struct todo_list *todo_list,
-			 unsigned flags);
+int todo_list_write_to_file(struct repository *r, struct todo_list *todo_list,
+			    const char *file, const char *shortrevisions,
+			    const char *shortonto, int num, unsigned flags);
 void todo_list_release(struct todo_list *todo_list);
 const char *todo_item_get_arg(struct todo_list *todo_list,
 			      struct todo_item *item);
@@ -145,8 +146,10 @@ int sequencer_remove_state(struct replay_opts *opts);
  * commits should be rebased onto the new base, this flag needs to be passed.
  */
 #define TODO_LIST_REBASE_COUSINS (1U << 4)
-int sequencer_make_script(struct repository *repo, FILE *out,
-			  int argc, const char **argv,
+#define TODO_LIST_APPEND_TODO_HELP (1U << 5)
+
+int sequencer_make_script(struct repository *r, FILE *out, int argc,
+			  const char **argv,
 			  unsigned flags);
 
 int sequencer_add_exec_commands(struct repository *r, const char *command);
Signature · 1 line
2.20.1
#101 lore raw

PATCH v5 6/16 sequencer: refactor check_todo_list() to work on a todo_list

In reply to Alban Gruin

This refactors check_todo_list() to work on a todo_list to avoid redundant reads and writes to the disk. The function is renamed todo_list_check(). The parsing of the two todo lists is left to the caller.

As rebase -p still need to check the todo list from the disk, a new function is introduced, check_todo_list_from_file(). It reads the file from the disk, parses it, pass the todo_list to todo_list_check(), and writes it back to the disk.

As get_missing_commit_check_level() and the enum missing_commit_check_level are no longer needed inside of sequencer.c, they are moved to rebase-interactive.c, and made static again.

---
Squashed <[email protected]> from Ramsay Jones.

 builtin/rebase--interactive.c |   2 +-
 rebase-interactive.c          |  91 ++++++++++++++++++++++++-
 rebase-interactive.h          |   2 +
 sequencer.c                   | 121 +++++++---------------------------
 sequencer.h                   |   9 +--
 5 files changed, 117 insertions(+), 108 deletions(-)
builtin/rebase--interactive.c +1 -1
index 0898eb4c59..df19ccaeb9 100644
@@ -256,7 +256,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = transform_todo_file(the_repository, flags);
 		break;
 	case CHECK_TODO_LIST:
-		ret = check_todo_list(the_repository);
+		ret = check_todo_list_from_file(the_repository);
 		break;
 	case REARRANGE_SQUASH:
 		ret = rearrange_squash(the_repository);
rebase-interactive.c +90 -1

101 lines not on this page · raw message

rebase-interactive.h +2 -0
index 17b6c9f6d0..187b5032d6 100644
@@ -3,9 +3,11 @@
 
 struct strbuf;
 struct repository;
+struct todo_list;
 
 void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 		      struct strbuf *buf);
 int edit_todo_list(struct repository *r, unsigned flags);
+int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
 
 #endif
sequencer.c +23 -98

142 lines not on this page · raw message

sequencer.h +1 -8
index dc3d8f76f7..1de97f188d 100644
@@ -64,12 +64,6 @@ struct replay_opts {
 };
 #define REPLAY_OPTS_INIT { .action = -1, .current_fixups = STRBUF_INIT }
 
-enum missing_commit_check_level {
-	MISSING_COMMIT_CHECK_IGNORE = 0,
-	MISSING_COMMIT_CHECK_WARN,
-	MISSING_COMMIT_CHECK_ERROR
-};
-
 int write_message(const void *buf, size_t len, const char *filename,
 		  int append_eol);
 
@@ -154,8 +148,7 @@ int sequencer_make_script(struct repository *r, FILE *out, int argc,
 
 int sequencer_add_exec_commands(struct repository *r, const char *command);
 int transform_todo_file(struct repository *r, unsigned flags);
-enum missing_commit_check_level get_missing_commit_check_level(void);
-int check_todo_list(struct repository *r);
+int check_todo_list_from_file(struct repository *r);
 int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, const char *cmd,
Signature · 1 line
2.20.1
#102 lore raw

PATCH v5 7/16 sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

In reply to Alban Gruin

This refactors sequencer_add_exec_commands() to work on a todo_list to avoid redundant reads and writes to the disk.

Instead of inserting the `exec' commands between the other commands and re-parsing the buffer at the end, they are appended to the buffer once, and a new list of items is created. Items from the old list are copied across and new `exec' items are appended when necessary. This eliminates the need to reparse the buffer, but this also means we have to use todo_list_write_to_disk() to write the file.

todo_list_add_exec_commands() and sequencer_add_exec_commands() are modified to take a string list instead of a string -- one item for each command. This makes it easier to insert a new command to the todo list for each command to execute.

sequencer_add_exec_commands() still reads the todo list from the disk, as it is needed by rebase -p.

complete_action() still uses sequencer_add_exec_commands() for now. This will be changed in a future commit.

---
Unchanged since v4.

 builtin/rebase--interactive.c |  15 +++--
 sequencer.c                   | 110 +++++++++++++++++++++-------------
 sequencer.h                   |   5 +-
 3 files changed, 82 insertions(+), 48 deletions(-)
builtin/rebase--interactive.c +11 -4
index df19ccaeb9..53056ee713 100644
@@ -65,7 +65,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 				 const char *onto, const char *onto_name,
 				 const char *squash_onto, const char *head_name,
 				 const char *restrict_revision, char *raw_strategies,
-				 const char *cmd, unsigned autosquash)
+				 struct string_list *commands, unsigned autosquash)
 {
 	int ret;
 	const char *head_hash = NULL;
@@ -116,7 +116,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 		discard_cache();
 		ret = complete_action(the_repository, opts, flags,
 				      shortrevisions, onto_name, onto,
-				      head_hash, cmd, autosquash);
+				      head_hash, commands, autosquash);
 	}
 
 	free(revisions);
@@ -139,6 +139,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 	const char *onto = NULL, *onto_name = NULL, *restrict_revision = NULL,
 		*squash_onto = NULL, *upstream = NULL, *head_name = NULL,
 		*switch_to = NULL, *cmd = NULL;
+	struct string_list commands = STRING_LIST_INIT_DUP;
 	char *raw_strategies = NULL;
 	enum {
 		NONE = 0, CONTINUE, SKIP, EDIT_TODO, SHOW_CURRENT_PATCH,
@@ -221,6 +222,12 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		warning(_("--[no-]rebase-cousins has no effect without "
 			  "--rebase-merges"));
 
+	if (cmd && *cmd) {
+		string_list_split(&commands, cmd, '\n', -1);
+		if (strlen(commands.items[commands.nr - 1].string) == 0)
+			--commands.nr;
+	}
+
 	switch (command) {
 	case NONE:
 		if (!onto && !upstream)
@@ -228,7 +235,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 
 		ret = do_interactive_rebase(&opts, flags, switch_to, upstream, onto,
 					    onto_name, squash_onto, head_name, restrict_revision,
-					    raw_strategies, cmd, autosquash);
+					    raw_strategies, &commands, autosquash);
 		break;
 	case SKIP: {
 		struct string_list merge_rr = STRING_LIST_INIT_DUP;
@@ -262,7 +269,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = rearrange_squash(the_repository);
 		break;
 	case ADD_EXEC:
-		ret = sequencer_add_exec_commands(the_repository, cmd);
+		ret = sequencer_add_exec_commands(the_repository, &commands);
 		break;
 	default:
 		BUG("invalid command '%d'", command);
sequencer.c +68 -42

146 lines not on this page · raw message

sequencer.h +3 -2
index 1de97f188d..e79f03e213 100644
@@ -146,12 +146,13 @@ int sequencer_make_script(struct repository *r, FILE *out, int argc,
 			  const char **argv,
 			  unsigned flags);
 
-int sequencer_add_exec_commands(struct repository *r, const char *command);
+int sequencer_add_exec_commands(struct repository *r,
+				struct string_list *commands);
 int transform_todo_file(struct repository *r, unsigned flags);
 int check_todo_list_from_file(struct repository *r);
 int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
-		    const char *onto, const char *orig_head, const char *cmd,
+		    const char *onto, const char *orig_head, struct string_list *commands,
 		    unsigned autosquash);
 int rearrange_squash(struct repository *r);
 
Signature · 1 line
2.20.1
#103 lore raw

PATCH v5 8/16 sequencer: refactor rearrange_squash() to work on a todo_list

In reply to Alban Gruin

This refactors rearrange_squash() to work on a todo_list to avoid redundant reads and writes. The function is renamed todo_list_rearrange_squash().

The old version created a new buffer, which was directly written to the disk. This new version creates a new item list by just copying items from the old item list, without creating a new buffer. This eliminates the need to reparse the todo list, but this also means its buffer cannot be directly written to the disk.

As rebase -p still need to check the todo list from the disk, a new function is introduced, rearrange_squash_in_todo_file().

complete_action() still uses rearrange_squash_in_todo_file() for now. This will be changed in a future commit.

---
Unchanged since v4.

 builtin/rebase--interactive.c |  2 +-
 sequencer.c                   | 92 ++++++++++++++++++-----------------
 sequencer.h                   |  2 +-
 3 files changed, 49 insertions(+), 47 deletions(-)
builtin/rebase--interactive.c +1 -1
index 53056ee713..4f0eae9239 100644
@@ -266,7 +266,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = check_todo_list_from_file(the_repository);
 		break;
 	case REARRANGE_SQUASH:
-		ret = rearrange_squash(the_repository);
+		ret = rearrange_squash_in_todo_file(the_repository);
 		break;
 	case ADD_EXEC:
 		ret = sequencer_add_exec_commands(the_repository, &commands);
sequencer.c +47 -45

156 lines not on this page · raw message

sequencer.h +1 -1
index e79f03e213..c7bb38d6df 100644
@@ -154,7 +154,7 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, struct string_list *commands,
 		    unsigned autosquash);
-int rearrange_squash(struct repository *r);
+int rearrange_squash_in_todo_file(struct repository *r);
 
 extern const char sign_off_header[];
 
Signature · 1 line
2.20.1
#104 lore raw

PATCH v5 9/16 sequencer: make sequencer_make_script() write its script to a strbuf

In reply to Alban Gruin

This makes sequencer_make_script() write its script to a strbuf (ie. the buffer of a todo_list) instead of a FILE. This reduce the amount of read/write made by rebase interactive.

---
Unchanged since v4.

 builtin/rebase--interactive.c | 13 ++++++-----
 sequencer.c                   | 41 +++++++++++++++--------------------
 sequencer.h                   |  5 ++---
 3 files changed, 28 insertions(+), 31 deletions(-)
builtin/rebase--interactive.c +8 -5
index 4f0eae9239..92026739c9 100644
@@ -71,7 +71,8 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	const char *head_hash = NULL;
 	char *revisions = NULL, *shortrevisions = NULL;
 	struct argv_array make_script_args = ARGV_ARRAY_INIT;
-	FILE *todo_list;
+	FILE *todo_list_file;
+	struct todo_list todo_list = TODO_LIST_INIT;
 
 	if (prepare_branch_to_be_rebased(opts, switch_to))
 		return -1;
@@ -93,8 +94,8 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	if (!upstream && squash_onto)
 		write_file(path_squash_onto(), "%s\n", squash_onto);
 
-	todo_list = fopen(rebase_path_todo(), "w");
-	if (!todo_list) {
+	todo_list_file = fopen(rebase_path_todo(), "w");
+	if (!todo_list_file) {
 		free(revisions);
 		free(shortrevisions);
 
@@ -105,10 +106,11 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	if (restrict_revision)
 		argv_array_push(&make_script_args, restrict_revision);
 
-	ret = sequencer_make_script(the_repository, todo_list,
+	ret = sequencer_make_script(the_repository, &todo_list.buf,
 				    make_script_args.argc, make_script_args.argv,
 				    flags);
-	fclose(todo_list);
+	fputs(todo_list.buf.buf, todo_list_file);
+	fclose(todo_list_file);
 
 	if (ret)
 		error(_("could not generate todo list"));
@@ -121,6 +123,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 
 	free(revisions);
 	free(shortrevisions);
+	todo_list_release(&todo_list);
 	argv_array_clear(&make_script_args);
 
 	return ret;
sequencer.c +18 -23

101 lines not on this page · raw message

sequencer.h +2 -3
index c7bb38d6df..9b8edd7df6 100644
@@ -142,9 +142,8 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_REBASE_COUSINS (1U << 4)
 #define TODO_LIST_APPEND_TODO_HELP (1U << 5)
 
-int sequencer_make_script(struct repository *r, FILE *out, int argc,
-			  const char **argv,
-			  unsigned flags);
+int sequencer_make_script(struct repository *r, struct strbuf *out, int argc,
+			  const char **argv, unsigned flags);
 
 int sequencer_add_exec_commands(struct repository *r,
 				struct string_list *commands);
Signature · 1 line
2.20.1
#105 lore raw

PATCH v5 10/16 sequencer: change complete_action() to use the refactored functions

In reply to Alban Gruin

complete_action() used functions that read the todo-list file, made some changes to it, and wrote it back to the disk.

The previous commits were dedicated to separate the part that deals with the file from the actual logic of these functions. Now that this is done, we can call directly the "logic" functions to avoid useless file access.

The parsing of the list has to be done by the caller. If the buffer of the todo list provided by the caller is empty, a `noop' command is directly added to the todo list, without touching the buffer.

---
Unchanged since v4.

 builtin/rebase--interactive.c | 20 +++------
 sequencer.c                   | 81 +++++++++++++++--------------------
 sequencer.h                   |  2 +-
 3 files changed, 42 insertions(+), 61 deletions(-)
builtin/rebase--interactive.c +6 -14
index 92026739c9..2dbf8fc08b 100644
@@ -71,7 +71,6 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	const char *head_hash = NULL;
 	char *revisions = NULL, *shortrevisions = NULL;
 	struct argv_array make_script_args = ARGV_ARRAY_INIT;
-	FILE *todo_list_file;
 	struct todo_list todo_list = TODO_LIST_INIT;
 
 	if (prepare_branch_to_be_rebased(opts, switch_to))
@@ -94,14 +93,6 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	if (!upstream && squash_onto)
 		write_file(path_squash_onto(), "%s\n", squash_onto);
 
-	todo_list_file = fopen(rebase_path_todo(), "w");
-	if (!todo_list_file) {
-		free(revisions);
-		free(shortrevisions);
-
-		return error_errno(_("could not open %s"), rebase_path_todo());
-	}
-
 	argv_array_pushl(&make_script_args, "", revisions, NULL);
 	if (restrict_revision)
 		argv_array_push(&make_script_args, restrict_revision);
@@ -109,16 +100,17 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 	ret = sequencer_make_script(the_repository, &todo_list.buf,
 				    make_script_args.argc, make_script_args.argv,
 				    flags);
-	fputs(todo_list.buf.buf, todo_list_file);
-	fclose(todo_list_file);
 
 	if (ret)
 		error(_("could not generate todo list"));
 	else {
 		discard_cache();
-		ret = complete_action(the_repository, opts, flags,
-				      shortrevisions, onto_name, onto,
-				      head_hash, commands, autosquash);
+		if (todo_list_parse_insn_buffer(the_repository, todo_list.buf.buf,
+						&todo_list))
+			BUG("unusable todo list");
+
+		ret = complete_action(the_repository, opts, flags, shortrevisions, onto_name,
+				      onto, head_hash, commands, autosquash, &todo_list);
 	}
 
 	free(revisions);
sequencer.c +35 -46

135 lines not on this page · raw message

sequencer.h +1 -1
index 9b8edd7df6..0c30e43f0a 100644
@@ -152,7 +152,7 @@ int check_todo_list_from_file(struct repository *r);
 int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, struct string_list *commands,
-		    unsigned autosquash);
+		    unsigned autosquash, struct todo_list *todo_list);
 int rearrange_squash_in_todo_file(struct repository *r);
 
 extern const char sign_off_header[];
Signature · 1 line
2.20.1
#106 lore raw

PATCH v5 11/16 sequencer: refactor skip_unnecessary_picks() to work on a todo_list

In reply to Alban Gruin

This refactors skip_unnecessary_picks() to work on a todo_list. As this function is only called by complete_action() (and thus is not used by rebase -p), the file-handling logic is completely dropped here.

Instead of truncating the todo list’s buffer, the items are moved to the beginning of the list, eliminating the need to reparse the list. This also means its buffer cannot be directly written to the disk.

rewrite_file() is then removed, as it is now unused.

---
Unchanged since v4.

 sequencer.c | 78 ++++++++++++-----------------------------------------
 1 file changed, 17 insertions(+), 61 deletions(-)
sequencer.c +17 -61
index 2a43ca685b..a817afffa9 100644
@@ -4661,52 +4661,22 @@ int check_todo_list_from_file(struct repository *r)
 	return res;
 }
 
-static int rewrite_file(const char *path, const char *buf, size_t len)
-{
-	int rc = 0;
-	int fd = open(path, O_WRONLY | O_TRUNC);
-	if (fd < 0)
-		return error_errno(_("could not open '%s' for writing"), path);
-	if (write_in_full(fd, buf, len) < 0)
-		rc = error_errno(_("could not write to '%s'"), path);
-	if (close(fd) && !rc)
-		rc = error_errno(_("could not close '%s'"), path);
-	return rc;
-}
-
 /* skip picking commits whose parents are unchanged */
-static int skip_unnecessary_picks(struct repository *r, struct object_id *output_oid)
+static int skip_unnecessary_picks(struct repository *r,
+				  struct todo_list *todo_list,
+				  struct object_id *output_oid)
 {
-	const char *todo_file = rebase_path_todo();
-	struct strbuf buf = STRBUF_INIT;
-	struct todo_list todo_list = TODO_LIST_INIT;
 	struct object_id *parent_oid;
-	int fd, i;
-
-	if (!read_oneliner(&buf, rebase_path_onto(), 0))
-		return error(_("could not read 'onto'"));
-	if (get_oid(buf.buf, output_oid)) {
-		strbuf_release(&buf);
-		return error(_("need a HEAD to fixup"));
-	}
-	strbuf_release(&buf);
-
-	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
-		return -1;
-	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list) < 0) {
-		todo_list_release(&todo_list);
-		return -1;
-	}
+	int i;
 
-	for (i = 0; i < todo_list.nr; i++) {
-		struct todo_item *item = todo_list.items + i;
+	for (i = 0; i < todo_list->nr; i++) {
+		struct todo_item *item = todo_list->items + i;
 
 		if (item->command >= TODO_NOOP)
 			continue;
 		if (item->command != TODO_PICK)
 			break;
 		if (parse_commit(item->commit)) {
-			todo_list_release(&todo_list);
 			return error(_("could not parse commit '%s'"),
 				oid_to_hex(&item->commit->object.oid));
 		}
@@ -4720,37 +4690,21 @@ static int skip_unnecessary_picks(struct repository *r, struct object_id *output
 		oidcpy(output_oid, &item->commit->object.oid);
 	}
 	if (i > 0) {
-		int offset = get_item_line_offset(&todo_list, i);
 		const char *done_path = rebase_path_done();
 
-		fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
-		if (fd < 0) {
-			error_errno(_("could not open '%s' for writing"),
-				    done_path);
-			todo_list_release(&todo_list);
-			return -1;
-		}
-		if (write_in_full(fd, todo_list.buf.buf, offset) < 0) {
+		if (todo_list_write_to_file(r, todo_list, done_path, NULL, NULL, i, 0)) {
 			error_errno(_("could not write to '%s'"), done_path);
-			todo_list_release(&todo_list);
-			close(fd);
 			return -1;
 		}
-		close(fd);
 
-		if (rewrite_file(rebase_path_todo(), todo_list.buf.buf + offset,
-				 todo_list.buf.len - offset) < 0) {
-			todo_list_release(&todo_list);
-			return -1;
-		}
+		MOVE_ARRAY(todo_list->items, todo_list->items + i, todo_list->nr - i);
+		todo_list->nr -= i;
+		todo_list->current = 0;
 
-		todo_list.current = i;
-		if (is_fixup(peek_command(&todo_list, 0)))
-			record_in_rewritten(output_oid, peek_command(&todo_list, 0));
+		if (is_fixup(peek_command(todo_list, 0)))
+			record_in_rewritten(output_oid, peek_command(todo_list, 0));
 	}
 
-	todo_list_release(&todo_list);
-
 	return 0;
 }
 
@@ -4823,6 +4777,11 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla
 		return -1;
 	}
 
+	if (opts->allow_ff && skip_unnecessary_picks(r, &new_todo, &oid)) {
+		todo_list_release(&new_todo);
+		return error(_("could not skip unnecessary pick commands"));
+	}
+
 	if (todo_list_write_to_file(r, &new_todo, todo_file, NULL, NULL, -1,
 				    flags & ~(TODO_LIST_SHORTEN_IDS))) {
 		todo_list_release(&new_todo);
@@ -4831,9 +4790,6 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla
 
 	todo_list_release(&new_todo);
 
-	if (opts->allow_ff && skip_unnecessary_picks(r, &oid))
-		return error(_("could not skip unnecessary pick commands"));
-
 	if (checkout_onto(opts, onto_name, oid_to_hex(&oid), orig_head))
 		return -1;
 
Signature · 1 line
2.20.1
#107 lore raw

PATCH v5 12/16 rebase-interactive: use todo_list_write_to_file() in edit_todo_list()

In reply to Alban Gruin

Just like complete_action(), edit_todo_list() used a function (transform_todo_file()) that read the todo list from the disk and wrote it back, resulting in useless disk accesses.

This changes edit_todo_list() to call directly todo_list_write_to_file() instead.

---
Squashed <[email protected]> from Ramsay Jones.

 rebase-interactive.c | 38 ++++++++++++++++----------------------
 sequencer.c          |  4 ++--
 sequencer.h          |  3 ---
 3 files changed, 18 insertions(+), 27 deletions(-)
rebase-interactive.c +16 -22
index dfa6dd530f..d396ecc599 100644
@@ -79,39 +79,33 @@ void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 
 int edit_todo_list(struct repository *r, unsigned flags)
 {
-	struct strbuf buf = STRBUF_INIT;
 	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res = 0;
 
-	if (strbuf_read_file(&buf, todo_file, 0) < 0)
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
 
-	strbuf_stripspace(&buf, 1);
-	if (write_message(buf.buf, buf.len, todo_file, 0)) {
-		strbuf_release(&buf);
+	strbuf_stripspace(&todo_list.buf, 1);
+	todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list);
+	if (todo_list_write_to_file(r, &todo_list, todo_file, NULL, NULL, -1,
+				    flags | TODO_LIST_SHORTEN_IDS | TODO_LIST_APPEND_TODO_HELP)) {
+		todo_list_release(&todo_list);
 		return -1;
 	}
 
-	strbuf_release(&buf);
-
-	transform_todo_file(r, flags | TODO_LIST_SHORTEN_IDS);
-
-	if (strbuf_read_file(&buf, todo_file, 0) < 0)
-		return error_errno(_("could not read '%s'."), todo_file);
-
-	append_todo_help(1, 0, &buf);
-	if (write_message(buf.buf, buf.len, todo_file, 0)) {
-		strbuf_release(&buf);
+	strbuf_reset(&todo_list.buf);
+	if (launch_sequence_editor(todo_file, &todo_list.buf, NULL)) {
+		todo_list_release(&todo_list);
 		return -1;
 	}
 
-	strbuf_release(&buf);
+	if (!todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list))
+		res = todo_list_write_to_file(r, &todo_list, todo_file, NULL, NULL, -1,
+					      flags & ~(TODO_LIST_SHORTEN_IDS));
 
-	if (launch_sequence_editor(todo_file, NULL, NULL))
-		return -1;
-
-	transform_todo_file(r, flags & ~(TODO_LIST_SHORTEN_IDS));
-
-	return 0;
+	todo_list_release(&todo_list);
+	return res;
 }
 
 define_commit_slab(commit_seen, unsigned char);
sequencer.c +2 -2
index a817afffa9..d8d045067c 100644
@@ -383,8 +383,8 @@ static void print_advice(struct repository *r, int show_hint,
 	}
 }
 
-int write_message(const void *buf, size_t len, const char *filename,
-		  int append_eol)
+static int write_message(const void *buf, size_t len, const char *filename,
+			 int append_eol)
 {
 	struct lock_file msg_file = LOCK_INIT;
 
sequencer.h +0 -3
index 0c30e43f0a..c5bee8124c 100644
@@ -64,9 +64,6 @@ struct replay_opts {
 };
 #define REPLAY_OPTS_INIT { .action = -1, .current_fixups = STRBUF_INIT }
 
-int write_message(const void *buf, size_t len, const char *filename,
-		  int append_eol);
-
 /*
  * Note that ordering matters in this enum. Not only must it match the mapping
  * of todo_command_info (in sequencer.c), it is also divided into several
Signature · 1 line
2.20.1
#108 lore raw

PATCH v5 13/16 rebase-interactive: append_todo_help() changes

In reply to Alban Gruin

This moves the writing of the comment "Rebase $shortrevisions onto $shortonto ($command_count commands)" from todo_list_write_to_file() to append_todo_help().

shortrevisions, shortonto, and command_count are passed as parameters to append_todo_help().

During the initial edit of the todo list, shortrevisions and shortonto are not NULL. Therefore, if shortrevisions or shortonto is NULL, then edit_todo would be true, otherwise it would be false. Thus, edit_todo is removed from the parameters of append_todo_help().

---
 rebase-interactive.c | 12 +++++++++++-
 rebase-interactive.h |  3 ++-
 sequencer.c          | 17 ++++-------------
 3 files changed, 17 insertions(+), 15 deletions(-)
rebase-interactive.c +11 -1
index d396ecc599..807f8370db 100644
@@ -28,7 +28,8 @@ static enum missing_commit_check_level get_missing_commit_check_level(void)
 	return MISSING_COMMIT_CHECK_IGNORE;
 }
 
-void append_todo_help(unsigned edit_todo, unsigned keep_empty,
+void append_todo_help(unsigned keep_empty, int command_count,
+		      const char *shortrevisions, const char *shortonto,
 		      struct strbuf *buf)
 {
 	const char *msg = _("\nCommands:\n"
@@ -48,6 +49,15 @@ void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 ".       specified). Use -c <commit> to reword the commit message.\n"
 "\n"
 "These lines can be re-ordered; they are executed from top to bottom.\n");
+	unsigned edit_todo = !(shortrevisions && shortonto);
+
+	if (!edit_todo) {
+		strbuf_addch(buf, '\n');
+		strbuf_commented_addf(buf, Q_("Rebase %s onto %s (%d command)",
+					      "Rebase %s onto %s (%d commands)",
+					      command_count),
+				      shortrevisions, shortonto, command_count);
+	}
 
 	strbuf_add_commented_lines(buf, msg, strlen(msg));
 
rebase-interactive.h +2 -1
index 187b5032d6..0e5925e3aa 100644
@@ -5,7 +5,8 @@ struct strbuf;
 struct repository;
 struct todo_list;
 
-void append_todo_help(unsigned edit_todo, unsigned keep_empty,
+void append_todo_help(unsigned keep_empty, int command_count,
+		      const char *shortrevisions, const char *shortonto,
 		      struct strbuf *buf);
 int edit_todo_list(struct repository *r, unsigned flags);
 int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
sequencer.c +4 -13
index d8d045067c..92de982bc4 100644
@@ -4580,22 +4580,13 @@ int todo_list_write_to_file(struct repository *r, struct todo_list *todo_list,
 			    const char *file, const char *shortrevisions,
 			    const char *shortonto, int num, unsigned flags)
 {
-	int edit_todo = !(shortrevisions && shortonto), res;
+	int res;
 	struct strbuf buf = STRBUF_INIT;
 
 	todo_list_to_strbuf(r, todo_list, &buf, num, flags);
-
-	if (flags & TODO_LIST_APPEND_TODO_HELP) {
-		int command_count = count_commands(todo_list);
-		if (!edit_todo) {
-			strbuf_addch(&buf, '\n');
-			strbuf_commented_addf(&buf, Q_("Rebase %s onto %s (%d command)",
-						       "Rebase %s onto %s (%d commands)",
-						       command_count),
-					      shortrevisions, shortonto, command_count);
-		}
-		append_todo_help(edit_todo, flags & TODO_LIST_KEEP_EMPTY, &buf);
-	}
+	if (flags & TODO_LIST_APPEND_TODO_HELP)
+		append_todo_help(flags & TODO_LIST_KEEP_EMPTY, count_commands(todo_list),
+				 shortrevisions, shortonto, &buf);
 
 	res = write_message(buf.buf, buf.len, file, 0);
 	strbuf_release(&buf);
Signature · 1 line
2.20.1
#109 lore raw

PATCH v5 14/16 rebase-interactive: rewrite edit_todo_list() to handle the initial edit

In reply to Alban Gruin

edit_todo_list() is changed to work on a todo_list, and to handle the initial edition of the todo list (ie. making a backup of the todo list).

It does not check for dropped commits yet, as todo_list_check() does not take the commits that have already been processed by the rebase (ie. the todo list is edited in the middle of a rebase session).

---
Unchanged since v4.

 builtin/rebase--interactive.c | 24 +++++++++++++++++-
 rebase-interactive.c          | 48 ++++++++++++++++++-----------------
 rebase-interactive.h          |  4 ++-
 sequencer.c                   |  3 +--
 sequencer.h                   |  1 +
 5 files changed, 53 insertions(+), 27 deletions(-)
builtin/rebase--interactive.c +23 -1
index 2dbf8fc08b..645ac587f7 100644
@@ -13,6 +13,28 @@ static GIT_PATH_FUNC(path_state_dir, "rebase-merge/")
 static GIT_PATH_FUNC(path_squash_onto, "rebase-merge/squash-onto")
 static GIT_PATH_FUNC(path_interactive, "rebase-merge/interactive")
 
+static int edit_todo_file(unsigned flags)
+{
+	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT,
+		new_todo = TODO_LIST_INIT;
+
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
+		return error_errno(_("could not read '%s'."), todo_file);
+
+	strbuf_stripspace(&todo_list.buf, 1);
+	if (!edit_todo_list(the_repository, &todo_list,
+			    &new_todo, NULL, NULL, flags) &&
+	    todo_list_write_to_file(the_repository, &new_todo, todo_file, NULL, NULL,
+				    -1, flags & ~(TODO_LIST_SHORTEN_IDS)) < 0)
+		return error_errno(_("could not write '%s'"), todo_file);
+
+	todo_list_release(&todo_list);
+	todo_list_release(&new_todo);
+
+	return 0;
+}
+
 static int get_revision_ranges(const char *upstream, const char *onto,
 			       const char **head_hash,
 			       char **revisions, char **shortrevisions)
@@ -242,7 +264,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		break;
 	}
 	case EDIT_TODO:
-		ret = edit_todo_list(the_repository, flags);
+		ret = edit_todo_file(flags);
 		break;
 	case SHOW_CURRENT_PATCH: {
 		struct child_process cmd = CHILD_PROCESS_INIT;
rebase-interactive.c +25 -23

60 lines not on this page · raw message

rebase-interactive.h +3 -1
index 0e5925e3aa..44dbb06311 100644
@@ -8,7 +8,9 @@ struct todo_list;
 void append_todo_help(unsigned keep_empty, int command_count,
 		      const char *shortrevisions, const char *shortonto,
 		      struct strbuf *buf);
-int edit_todo_list(struct repository *r, unsigned flags);
+int edit_todo_list(struct repository *r, struct todo_list *todo_list,
+		   struct todo_list *new_todo, const char *shortrevisions,
+		   const char *shortonto, unsigned flags);
 int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
 
 #endif
sequencer.c +1 -2
index 92de982bc4..8f47f0cf39 100644
@@ -55,8 +55,7 @@ static GIT_PATH_FUNC(rebase_path, "rebase-merge")
  * file and written to the tail of 'done'.
  */
 GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
-static GIT_PATH_FUNC(rebase_path_todo_backup,
-		     "rebase-merge/git-rebase-todo.backup")
+GIT_PATH_FUNC(rebase_path_todo_backup, "rebase-merge/git-rebase-todo.backup")
 
 /*
  * The rebase command lines that have already been processed. A line
sequencer.h +1 -0
index c5bee8124c..68acab980b 100644
@@ -10,6 +10,7 @@ struct repository;
 const char *git_path_commit_editmsg(void);
 const char *git_path_seq_dir(void);
 const char *rebase_path_todo(void);
+const char *rebase_path_todo_backup(void);
 
 #define APPEND_SIGNOFF_DEDUP (1u << 0)
 
Signature · 1 line
2.20.1
#110 lore raw

PATCH v5 15/16 sequencer: use edit_todo_list() in complete_action()

In reply to Alban Gruin

This changes complete_action() to use edit_todo_list(), now that it can handle the initial edit of the todo list.

---
Unchanged since v4.

 sequencer.c | 21 +++++++--------------
 1 file changed, 7 insertions(+), 14 deletions(-)
sequencer.c +7 -14
index 8f47f0cf39..21b04e0642 100644
@@ -4709,6 +4709,7 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla
 	struct todo_list new_todo = TODO_LIST_INIT;
 	struct strbuf *buf = &todo_list->buf;
 	struct object_id oid;
+	int res;
 
 	get_oid(onto, &oid);
 	shortonto = find_unique_abbrev(&oid, DEFAULT_ABBREV);
@@ -4733,24 +4734,16 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla
 		return error(_("nothing to do"));
 	}
 
-	if (todo_list_write_to_file(r, todo_list, todo_file,
-				    shortrevisions, shortonto, -1,
-				    flags | TODO_LIST_SHORTEN_IDS | TODO_LIST_APPEND_TODO_HELP))
-		return error_errno(_("could not write '%s'"), todo_file);
-
-	if (copy_file(rebase_path_todo_backup(), todo_file, 0666))
-		return error(_("could not copy '%s' to '%s'."), todo_file,
-			     rebase_path_todo_backup());
-
-	if (launch_sequence_editor(todo_file, &new_todo.buf, NULL)) {
+	res = edit_todo_list(r, todo_list, &new_todo, shortrevisions,
+			     shortonto, flags);
+	if (res == -1)
+		return -1;
+	else if (res == -2) {
 		apply_autostash(opts);
 		sequencer_remove_state(opts);
 
 		return -1;
-	}
-
-	strbuf_stripspace(&new_todo.buf, 1);
-	if (new_todo.buf.len == 0) {
+	} else if (res == -3) {
 		apply_autostash(opts);
 		sequencer_remove_state(opts);
 		todo_list_release(&new_todo);
Signature · 1 line
2.20.1
#111 lore raw

PATCH v5 16/16 rebase--interactive: move transform_todo_file() to rebase--interactive.c

In reply to Alban Gruin

As transform_todo_file() is only needed inside of rebase--interactive.c, it is moved there from sequencer.c.

---
Unchanged since v4.

 builtin/rebase--interactive.c | 26 +++++++++++++++++++++++++-
 sequencer.c                   | 23 -----------------------
 sequencer.h                   |  1 -
 3 files changed, 25 insertions(+), 25 deletions(-)
builtin/rebase--interactive.c +25 -1
index 645ac587f7..7f1e88a087 100644
@@ -35,6 +35,30 @@ static int edit_todo_file(unsigned flags)
 	return 0;
 }
 
+static int transform_todo_file(unsigned flags)
+{
+	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res;
+
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
+		return error_errno(_("could not read '%s'."), todo_file);
+
+	if (todo_list_parse_insn_buffer(the_repository, todo_list.buf.buf,
+					&todo_list)) {
+		todo_list_release(&todo_list);
+		return error(_("unusable todo list: '%s'"), todo_file);
+	}
+
+	res = todo_list_write_to_file(the_repository, &todo_list, todo_file,
+				      NULL, NULL, -1, flags);
+	todo_list_release(&todo_list);
+
+	if (res)
+		return error_errno(_("could not write '%s'."), todo_file);
+	return 0;
+}
+
 static int get_revision_ranges(const char *upstream, const char *onto,
 			       const char **head_hash,
 			       char **revisions, char **shortrevisions)
@@ -277,7 +301,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 	}
 	case SHORTEN_OIDS:
 	case EXPAND_OIDS:
-		ret = transform_todo_file(the_repository, flags);
+		ret = transform_todo_file(flags);
 		break;
 	case CHECK_TODO_LIST:
 		ret = check_todo_list_from_file(the_repository);
sequencer.c +0 -23
index 21b04e0642..5239700efc 100644
@@ -4593,29 +4593,6 @@ int todo_list_write_to_file(struct repository *r, struct todo_list *todo_list,
 	return res;
 }
 
-int transform_todo_file(struct repository *r, unsigned flags)
-{
-	const char *todo_file = rebase_path_todo();
-	struct todo_list todo_list = TODO_LIST_INIT;
-	int res;
-
-	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
-		return error_errno(_("could not read '%s'."), todo_file);
-
-	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
-		todo_list_release(&todo_list);
-		return error(_("unusable todo list: '%s'"), todo_file);
-	}
-
-	res = todo_list_write_to_file(r, &todo_list, todo_file,
-				      NULL, NULL, -1, flags);
-	todo_list_release(&todo_list);
-
-	if (res)
-		return error_errno(_("could not write '%s'."), todo_file);
-	return 0;
-}
-
 static const char edit_todo_list_advice[] =
 N_("You can fix this with 'git rebase --edit-todo' "
 "and then run 'git rebase --continue'.\n"
sequencer.h +0 -1
index 68acab980b..11afd47aa9 100644
@@ -145,7 +145,6 @@ int sequencer_make_script(struct repository *r, struct strbuf *out, int argc,
 
 int sequencer_add_exec_commands(struct repository *r,
 				struct string_list *commands);
-int transform_todo_file(struct repository *r, unsigned flags);
 int check_todo_list_from_file(struct repository *r);
 int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
Signature · 1 line
2.20.1
#112 lore raw
In reply to Alban Gruin

Alban Gruin <[email protected]> writes:

Before I comment on anything else.

Quoted 2 lines

This is based on nd/the-index (36e7ed69de, "rebase-interactive.c: remove the_repository references").

My attempt to apply these in order on top of that commit seems to stop at step 5/16. Are you sure you based them on it?

Thanks.

#113 lore raw

[I’m resending this as I clicked on the wrong button…]

Hi,

Quoted 11 lines · Junio C Hamano

Alban Gruin <[email protected]> writes:

Before I comment on anything else.

Quoted 2 lines

This is based on nd/the-index (36e7ed69de, "rebase-interactive.c: remove the_repository references").

My attempt to apply these in order on top of that commit seems to stop at step 5/16. Are you sure you based them on it?

Thanks.

It is based on that commit, but I mistakenly added a newline between --- a/ sequencer.c and +++ b/sequencer.c before sending this series. Sorry about this.

I just reapplied all the patches I sent, and there is no other problem. Should I send this back?

-- Alban

#114 lore raw

v6 cover sequencer: refactor functions working on a todo_list

In reply to Alban Gruin

At the center of the "interactive" part of the interactive rebase lies the todo list. When the user starts an interactive rebase, a todo list is generated, presented to the user (who then edits it using a text editor), read back, and then is checked and processed before the actual rebase takes place.

Some of this processing includes adding execs commands, reordering fixup! and squash! commits, and checking if no commits were accidentally dropped by the user.

Before I converted the interactive rebase in C, these functions were called by git-rebase--interactive.sh through git-rebase--helper. Since the only way to pass around a large amount of data between a shell script and a C program is to use a file (or any declination of a file), the functions that checked and processed the todo list were directly working on a file, the same file that the user edited.

During the conversion, I did not address this issue, which lead to a complete_action() that reads the todo list file, does some computation based on its content, and writes it back to the disk, several times in the same function.

As it is not an efficient way to handle a data structure, this patch series refactor the functions that processes the todo list to work on a todo_list structure instead of reading it from the disk.

Some commits consists in modifying edit_todo_list() (initially used by --edit-todo) to handle the initial edition of the todo list, to increase code sharing.

This is based on nd/the-index (36e7ed69de, "rebase-interactive.c: remove the_repository references").

I am rerolling this because I accidentally added a newline to a place I shouldn’t in the patch 05/16 of the v5. There is no change in this version.

Alban Gruin (16):
sequencer: changes in parse_insn_buffer()
sequencer: make the todo_list structure public
sequencer: remove the 'arg' field from todo_item
sequencer: refactor transform_todos() to work on a todo_list
sequencer: introduce todo_list_write_to_file()
sequencer: refactor check_todo_list() to work on a todo_list
sequencer: refactor sequencer_add_exec_commands() to work on a
todo_list
sequencer: refactor rearrange_squash() to work on a todo_list
sequencer: make sequencer_make_script() write its script to a strbuf
sequencer: change complete_action() to use the refactored functions
sequencer: refactor skip_unnecessary_picks() to work on a todo_list
rebase-interactive: use todo_list_write_to_file() in edit_todo_list()
rebase-interactive: append_todo_help() changes
rebase-interactive: rewrite edit_todo_list() to handle the initial
edit
sequencer: use edit_todo_list() in complete_action()
rebase--interactive: move transform_todo_file() to
rebase--interactive.c

 builtin/rebase--interactive.c |  90 +++--
 rebase-interactive.c          | 143 +++++--
 rebase-interactive.h          |   9 +-
 sequencer.c                   | 687 ++++++++++++++--------------------
 sequencer.h                   |  81 +++-
 5 files changed, 533 insertions(+), 477 deletions(-)
Signature · 1 line
2.20.1
#115 lore raw

PATCH v6 1/16 sequencer: changes in parse_insn_buffer()

This clears the number of items of a todo_list before parsing it to allow to parse the same list multiple times without issues. As its items are not dynamically allocated, or don’t need to allocate memory, no additionnal memory management is required here.

Furthermore, if a line is invalid, the type of the corresponding command is set to a garbage value, and its argument is defined properly. This will allow to recreate the text of a todo list from its commands, even if one of them is incorrect.

---
 sequencer.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
sequencer.c +6 -1
index d726f77e11..a7afaf6882 100644
@@ -2091,6 +2091,8 @@ static int parse_insn_buffer(struct repository *r, char *buf,
 	char *p = buf, *next_p;
 	int i, res = 0, fixup_okay = file_exists(rebase_path_done());
 
+	todo_list->current = todo_list->nr = 0;
+
 	for (i = 1; *p; i++, p = next_p) {
 		char *eol = strchrnul(p, '\n');
 
@@ -2104,7 +2106,10 @@ static int parse_insn_buffer(struct repository *r, char *buf,
 		if (parse_insn_line(r, item, p, eol)) {
 			res = error(_("invalid line %d: %.*s"),
 				i, (int)(eol - p), p);
-			item->command = TODO_NOOP;
+			item->command = TODO_COMMENT + 1;
+			item->arg = p;
+			item->arg_len = (int)(eol - p);
+			item->commit = NULL;
 		}
 
 		if (fixup_okay)
Signature · 1 line
2.20.1
#116 lore raw

PATCH v6 2/16 sequencer: make the todo_list structure public

In reply to Alban Gruin

This makes the structures todo_list and todo_item, and the functions todo_list_release() and parse_insn_buffer(), accessible outside of sequencer.c.

---
 sequencer.c | 69 ++++++++++-------------------------------------------
 sequencer.h | 50 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 62 insertions(+), 57 deletions(-)
sequencer.c +12 -57
index a7afaf6882..5b84a20532 100644
@@ -1460,32 +1460,6 @@ static int allow_empty(struct repository *r,
 		return 1;
 }
 
-/*
- * Note that ordering matters in this enum. Not only must it match the mapping
- * below, it is also divided into several sections that matter.  When adding
- * new commands, make sure you add it in the right section.
- */
-enum todo_command {
-	/* commands that handle commits */
-	TODO_PICK = 0,
-	TODO_REVERT,
-	TODO_EDIT,
-	TODO_REWORD,
-	TODO_FIXUP,
-	TODO_SQUASH,
-	/* commands that do something else than handling a single commit */
-	TODO_EXEC,
-	TODO_BREAK,
-	TODO_LABEL,
-	TODO_RESET,
-	TODO_MERGE,
-	/* commands that do nothing but are counted for reporting progress */
-	TODO_NOOP,
-	TODO_DROP,
-	/* comments (not counted for reporting progress) */
-	TODO_COMMENT
-};
-
 static struct {
 	char c;
 	const char *str;
@@ -1962,26 +1936,7 @@ enum todo_item_flags {
 	TODO_EDIT_MERGE_MSG = 1
 };
 
-struct todo_item {
-	enum todo_command command;
-	struct commit *commit;
-	unsigned int flags;
-	const char *arg;
-	int arg_len;
-	size_t offset_in_buf;
-};
-
-struct todo_list {
-	struct strbuf buf;
-	struct todo_item *items;
-	int nr, alloc, current;
-	int done_nr, total_nr;
-	struct stat_data stat;
-};
-
-#define TODO_LIST_INIT { STRBUF_INIT }
-
-static void todo_list_release(struct todo_list *todo_list)
+void todo_list_release(struct todo_list *todo_list)
 {
 	strbuf_release(&todo_list->buf);
 	FREE_AND_NULL(todo_list->items);
@@ -2084,8 +2039,8 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 	return !item->commit;
 }
 
-static int parse_insn_buffer(struct repository *r, char *buf,
-			     struct todo_list *todo_list)
+int todo_list_parse_insn_buffer(struct repository *r, char *buf,
+				struct todo_list *todo_list)
 {
 	struct todo_item *item;
 	char *p = buf, *next_p;
@@ -2184,7 +2139,7 @@ static int read_populate_todo(struct repository *r,
 		return error(_("could not stat '%s'"), todo_file);
 	fill_stat_data(&todo_list->stat, &st);
 
-	res = parse_insn_buffer(r, todo_list->buf.buf, todo_list);
+	res = todo_list_parse_insn_buffer(r, todo_list->buf.buf, todo_list);
 	if (res) {
 		if (is_rebase_i(opts))
 			return error(_("please fix this using "
@@ -2215,7 +2170,7 @@ static int read_populate_todo(struct repository *r,
 		FILE *f = fopen_or_warn(rebase_path_msgtotal(), "w");
 
 		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
-		    !parse_insn_buffer(r, done.buf.buf, &done))
+		    !todo_list_parse_insn_buffer(r, done.buf.buf, &done))
 			todo_list->done_nr = count_commands(&done);
 		else
 			todo_list->done_nr = 0;
@@ -4501,7 +4456,7 @@ int sequencer_add_exec_commands(struct repository *r,
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4557,7 +4512,7 @@ int transform_todos(struct repository *r, unsigned flags)
 	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
 		return error(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4643,7 +4598,7 @@ int check_todo_list(struct repository *r)
 		goto leave_check;
 	}
 	advise_to_edit_todo = res =
-		parse_insn_buffer(r, todo_list.buf.buf, &todo_list);
+		todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list);
 
 	if (res || check_level == MISSING_COMMIT_CHECK_IGNORE)
 		goto leave_check;
@@ -4662,7 +4617,7 @@ int check_todo_list(struct repository *r)
 		goto leave_check;
 	}
 	strbuf_release(&todo_file);
-	res = !!parse_insn_buffer(r, todo_list.buf.buf, &todo_list);
+	res = !!todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list);
 
 	/* Find commits in git-rebase-todo.backup yet unseen */
 	for (i = todo_list.nr - 1; i >= 0; i--) {
@@ -4744,7 +4699,7 @@ static int skip_unnecessary_picks(struct repository *r, struct object_id *output
 
 	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
 		return -1;
-	if (parse_insn_buffer(r, todo_list.buf.buf, &todo_list) < 0) {
+	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list) < 0) {
 		todo_list_release(&todo_list);
 		return -1;
 	}
@@ -4832,7 +4787,7 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla
 	if (strbuf_read_file(buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
 
-	if (parse_insn_buffer(r, buf->buf, &todo_list)) {
+	if (todo_list_parse_insn_buffer(r, buf->buf, &todo_list)) {
 		todo_list_release(&todo_list);
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
@@ -4940,7 +4895,7 @@ int rearrange_squash(struct repository *r)
 
 	if (strbuf_read_file_or_whine(&todo_list.buf, todo_file) < 0)
 		return -1;
-	if (parse_insn_buffer(r, todo_list.buf.buf, &todo_list) < 0) {
+	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list) < 0) {
 		todo_list_release(&todo_list);
 		return -1;
 	}
sequencer.h +50 -0

56 lines not on this page · raw message

Signature · 1 line
2.20.1
#117 lore raw

PATCH v6 3/16 sequencer: remove the 'arg' field from todo_item

In reply to Alban Gruin

The 'arg' field of todo_item used to store the address of the first byte of the parameter of a command in a todo list. It was associated with the length of the parameter (the 'arg_len' field).

This replaces the 'arg' field by 'arg_offset'. This new field does not store the address of the parameter, but the position of the first character of the parameter in the buffer. todo_item_get_arg() is added to return the address of the parameter of an item.

This will prevent todo_list_add_exec_commands() from having to do awful pointer arithmetics when growing the todo list buffer.

---
 sequencer.c | 67 ++++++++++++++++++++++++++++++-----------------------
 sequencer.h |  6 +++--
 2 files changed, 42 insertions(+), 31 deletions(-)
sequencer.c +38 -29
index 5b84a20532..0e7ab16a05 100644
@@ -1949,8 +1949,14 @@ static struct todo_item *append_new_todo(struct todo_list *todo_list)
 	return todo_list->items + todo_list->nr++;
 }
 
+const char *todo_item_get_arg(struct todo_list *todo_list,
+			      struct todo_item *item)
+{
+	return todo_list->buf.buf + item->arg_offset;
+}
+
 static int parse_insn_line(struct repository *r, struct todo_item *item,
-			   const char *bol, char *eol)
+			   const char *buf, const char *bol, char *eol)
 {
 	struct object_id commit_oid;
 	char *end_of_object_name;
@@ -1964,7 +1970,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
 		item->command = TODO_COMMENT;
 		item->commit = NULL;
-		item->arg = bol;
+		item->arg_offset = bol - buf;
 		item->arg_len = eol - bol;
 		return 0;
 	}
@@ -1991,7 +1997,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 			return error(_("%s does not accept arguments: '%s'"),
 				     command_to_string(item->command), bol);
 		item->commit = NULL;
-		item->arg = bol;
+		item->arg_offset = bol - buf;
 		item->arg_len = eol - bol;
 		return 0;
 	}
@@ -2003,7 +2009,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
 	    item->command == TODO_RESET) {
 		item->commit = NULL;
-		item->arg = bol;
+		item->arg_offset = bol - buf;
 		item->arg_len = (int)(eol - bol);
 		return 0;
 	}
@@ -2017,7 +2023,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 		} else {
 			item->flags |= TODO_EDIT_MERGE_MSG;
 			item->commit = NULL;
-			item->arg = bol;
+			item->arg_offset = bol - buf;
 			item->arg_len = (int)(eol - bol);
 			return 0;
 		}
@@ -2029,8 +2035,9 @@ static int parse_insn_line(struct repository *r, struct todo_item *item,
 	status = get_oid(bol, &commit_oid);
 	*end_of_object_name = saved;
 
-	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
-	item->arg_len = (int)(eol - item->arg);
+	bol = end_of_object_name + strspn(end_of_object_name, " \t");
+	item->arg_offset = bol - buf;
+	item->arg_len = (int)(eol - bol);
 
 	if (status < 0)
 		return -1;
@@ -2058,11 +2065,11 @@ int todo_list_parse_insn_buffer(struct repository *r, char *buf,
 
 		item = append_new_todo(todo_list);
 		item->offset_in_buf = p - todo_list->buf.buf;
-		if (parse_insn_line(r, item, p, eol)) {
+		if (parse_insn_line(r, item, buf, p, eol)) {
 			res = error(_("invalid line %d: %.*s"),
 				i, (int)(eol - p), p);
 			item->command = TODO_COMMENT + 1;
-			item->arg = p;
+			item->arg_offset = p - buf;
 			item->arg_len = (int)(eol - p);
 			item->commit = NULL;
 		}
@@ -2402,7 +2409,7 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 
 		item->command = command;
 		item->commit = commit;
-		item->arg = NULL;
+		item->arg_offset = 0;
 		item->arg_len = 0;
 		item->offset_in_buf = todo_list->buf.len;
 		subject_len = find_commit_subject(commit_buffer, &subject);
@@ -3438,6 +3445,8 @@ static int pick_commits(struct repository *r,
 
 	while (todo_list->current < todo_list->nr) {
 		struct todo_item *item = todo_list->items + todo_list->current;
+		const char *arg = todo_item_get_arg(todo_list, item);
+
 		if (save_todo(todo_list, opts))
 			return -1;
 		if (is_rebase_i(opts)) {
@@ -3488,10 +3497,9 @@ static int pick_commits(struct repository *r,
 					fprintf(stderr,
 						_("Stopped at %s...  %.*s\n"),
 						short_commit_name(commit),
-						item->arg_len, item->arg);
+						item->arg_len, arg);
 				return error_with_patch(r, commit,
-					item->arg, item->arg_len, opts, res,
-					!res);
+					arg, item->arg_len, opts, res, !res);
 			}
 			if (is_rebase_i(opts) && !res)
 				record_in_rewritten(&item->commit->object.oid,
@@ -3500,7 +3508,7 @@ static int pick_commits(struct repository *r,
 				if (res == 1)
 					intend_to_amend();
 				return error_failed_squash(r, item->commit, opts,
-					item->arg_len, item->arg);
+					item->arg_len, arg);
 			} else if (res && is_rebase_i(opts) && item->commit) {
 				int to_amend = 0;
 				struct object_id oid;
@@ -3519,16 +3527,16 @@ static int pick_commits(struct repository *r,
 					to_amend = 1;
 
 				return res | error_with_patch(r, item->commit,
-						item->arg, item->arg_len, opts,
+						arg, item->arg_len, opts,
 						res, to_amend);
 			}
 		} else if (item->command == TODO_EXEC) {
-			char *end_of_arg = (char *)(item->arg + item->arg_len);
+			char *end_of_arg = (char *)(arg + item->arg_len);
 			int saved = *end_of_arg;
 			struct stat st;
 
 			*end_of_arg = '\0';
-			res = do_exec(r, item->arg);
+			res = do_exec(r, arg);
 			*end_of_arg = saved;
 
 			/* Reread the todo file if it has changed. */
@@ -3545,14 +3553,14 @@ static int pick_commits(struct repository *r,
 				todo_list->current = -1;
 			}
 		} else if (item->command == TODO_LABEL) {
-			if ((res = do_label(r, item->arg, item->arg_len)))
+			if ((res = do_label(r, arg, item->arg_len)))
 				reschedule = 1;
 		} else if (item->command == TODO_RESET) {
-			if ((res = do_reset(r, item->arg, item->arg_len, opts)))
+			if ((res = do_reset(r, arg, item->arg_len, opts)))
 				reschedule = 1;
 		} else if (item->command == TODO_MERGE) {
 			if ((res = do_merge(r, item->commit,
-					    item->arg, item->arg_len,
+					    arg, item->arg_len,
 					    item->flags, opts)) < 0)
 				reschedule = 1;
 			else if (item->commit)
@@ -3561,9 +3569,8 @@ static int pick_commits(struct repository *r,
 			if (res > 0)
 				/* failed with merge conflicts */
 				return error_with_patch(r, item->commit,
-							item->arg,
-							item->arg_len, opts,
-							res, 0);
+							arg, item->arg_len,
+							opts, res, 0);
 		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
@@ -3578,9 +3585,8 @@ static int pick_commits(struct repository *r,
 			if (item->commit)
 				return error_with_patch(r,
 							item->commit,
-							item->arg,
-							item->arg_len, opts,
-							res, 0);
+							arg, item->arg_len,
+							opts, res, 0);
 		}
 
 		todo_list->current++;
@@ -4520,7 +4526,8 @@ int transform_todos(struct repository *r, unsigned flags)
 	for (item = todo_list.items, i = 0; i < todo_list.nr; i++, item++) {
 		/* if the item is not a command write it and continue */
 		if (item->command >= TODO_COMMENT) {
-			strbuf_addf(&buf, "%.*s\n", item->arg_len, item->arg);
+			strbuf_addf(&buf, "%.*s\n", item->arg_len,
+				    todo_item_get_arg(&todo_list, item));
 			continue;
 		}
 
@@ -4550,7 +4557,8 @@ int transform_todos(struct repository *r, unsigned flags)
 		if (!item->arg_len)
 			strbuf_addch(&buf, '\n');
 		else
-			strbuf_addf(&buf, " %.*s\n", item->arg_len, item->arg);
+			strbuf_addf(&buf, " %.*s\n", item->arg_len,
+				    todo_item_get_arg(&todo_list, item));
 	}
 
 	i = write_message(buf.buf, buf.len, todo_file, 0);
@@ -4626,7 +4634,8 @@ int check_todo_list(struct repository *r)
 		if (commit && !*commit_seen_at(&commit_seen, commit)) {
 			strbuf_addf(&missing, " - %s %.*s\n",
 				    short_commit_name(commit),
-				    item->arg_len, item->arg);
+				    item->arg_len,
+				    todo_item_get_arg(&todo_list, item));
 			*commit_seen_at(&commit_seen, commit) = 1;
 		}
 	}
sequencer.h +4 -2
index 7dc4d8946b..cdb9ed7a2d 100644
@@ -104,9 +104,9 @@ struct todo_item {
 	enum todo_command command;
 	struct commit *commit;
 	unsigned int flags;
-	const char *arg;
 	int arg_len;
-	size_t offset_in_buf;
+	/* The offset of the command and its argument in the strbuf */
+	size_t offset_in_buf, arg_offset;
 };
 
 struct todo_list {
@@ -122,6 +122,8 @@ struct todo_list {
 int todo_list_parse_insn_buffer(struct repository *r, char *buf,
 				struct todo_list *todo_list);
 void todo_list_release(struct todo_list *todo_list);
+const char *todo_item_get_arg(struct todo_list *todo_list,
+			      struct todo_item *item);
 
 /* Call this to setup defaults before parsing command line options */
 void sequencer_init_config(struct replay_opts *opts);
Signature · 1 line
2.20.1
#118 lore raw

PATCH v6 4/16 sequencer: refactor transform_todos() to work on a todo_list

In reply to Alban Gruin

This refactors transform_todos() to work on a todo_list. The function is renamed todo_list_transform().

As rebase -p still need to check the todo list from the disk, a new function is introduced, transform_todo_file(). It is still used by complete_action() and edit_todo_list() for now, but they will be replaced in a future commit.

todo_list_transform() is not a static function, because it will be used by edit_todo_list() from rebase-interactive.c in a future commit.

---
 builtin/rebase--interactive.c |  2 +-
 rebase-interactive.c          |  4 +--
 sequencer.c                   | 54 +++++++++++++++++++++++------------
 sequencer.h                   |  4 ++-
 4 files changed, 42 insertions(+), 22 deletions(-)
builtin/rebase--interactive.c +1 -1
index dd2a55ab1d..0898eb4c59 100644
@@ -253,7 +253,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 	}
 	case SHORTEN_OIDS:
 	case EXPAND_OIDS:
-		ret = transform_todos(the_repository, flags);
+		ret = transform_todo_file(the_repository, flags);
 		break;
 	case CHECK_TODO_LIST:
 		ret = check_todo_list(the_repository);
rebase-interactive.c +2 -2
index 68aff1dac2..842fa07e7e 100644
@@ -69,7 +69,7 @@ int edit_todo_list(struct repository *r, unsigned flags)
 
 	strbuf_release(&buf);
 
-	transform_todos(r, flags | TODO_LIST_SHORTEN_IDS);
+	transform_todo_file(r, flags | TODO_LIST_SHORTEN_IDS);
 
 	if (strbuf_read_file(&buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
@@ -85,7 +85,7 @@ int edit_todo_list(struct repository *r, unsigned flags)
 	if (launch_sequence_editor(todo_file, NULL, NULL))
 		return -1;
 
-	transform_todos(r, flags & ~(TODO_LIST_SHORTEN_IDS));
+	transform_todo_file(r, flags & ~(TODO_LIST_SHORTEN_IDS));
 
 	return 0;
 }
sequencer.c +36 -18

89 lines not on this page · raw message

sequencer.h +3 -1
index cdb9ed7a2d..35472c137b 100644
@@ -121,6 +121,8 @@ struct todo_list {
 
 int todo_list_parse_insn_buffer(struct repository *r, char *buf,
 				struct todo_list *todo_list);
+void todo_list_transform(struct repository *r, struct todo_list *todo_list,
+			 unsigned flags);
 void todo_list_release(struct todo_list *todo_list);
 const char *todo_item_get_arg(struct todo_list *todo_list,
 			      struct todo_item *item);
@@ -148,7 +150,7 @@ int sequencer_make_script(struct repository *repo, FILE *out,
 			  unsigned flags);
 
 int sequencer_add_exec_commands(struct repository *r, const char *command);
-int transform_todos(struct repository *r, unsigned flags);
+int transform_todo_file(struct repository *r, unsigned flags);
 enum missing_commit_check_level get_missing_commit_check_level(void);
 int check_todo_list(struct repository *r);
 int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
Signature · 1 line
2.20.1
#119 lore raw

PATCH v6 5/16 sequencer: introduce todo_list_write_to_file()

In reply to Alban Gruin

This introduces a new function to recreate the text of a todo list from its commands and write it to a file. This will be useful as the next few commits will change the use of the buffer in struct todo_list so it will no longer be a mirror of the file on disk.

This functionality already exists in todo_list_transform(), but this function was made to replace the buffer of a todo list, which is not what we want here. Thus, the part of todo_list_transform() that replaces the buffer is dropped, and the function is renamed todo_list_to_strbuf(). It is called by todo_list_write_to_file() to fill the buffer to write to the disk.

todo_list_write_to_file() can also take care of appending the help text to the buffer before writing it to the disk, or to write only the first n items of the list. This feature will be used by skip_unnecessary_picks(), which has to write done commands in a file.

---
 sequencer.c | 61 +++++++++++++++++++++++++++++++++++------------------
 sequencer.h | 11 ++++++----
 2 files changed, 48 insertions(+), 24 deletions(-)
sequencer.c +41 -20
index 7a295cbd3f..87b43994ff 100644
@@ -4507,26 +4507,28 @@ int sequencer_add_exec_commands(struct repository *r,
 	return i;
 }
 
-void todo_list_transform(struct repository *r, struct todo_list *todo_list,
-			 unsigned flags)
+static void todo_list_to_strbuf(struct repository *r, struct todo_list *todo_list,
+				struct strbuf *buf, int num, unsigned flags)
 {
-	struct strbuf buf = STRBUF_INIT;
 	struct todo_item *item;
-	int i;
+	int i, max = todo_list->nr;
 
-	for (item = todo_list->items, i = 0; i < todo_list->nr; i++, item++) {
+	if (num > 0 && num < max)
+		max = num;
+
+	for (item = todo_list->items, i = 0; i < max; i++, item++) {
 		/* if the item is not a command write it and continue */
 		if (item->command >= TODO_COMMENT) {
-			strbuf_addf(&buf, "%.*s\n", item->arg_len,
+			strbuf_addf(buf, "%.*s\n", item->arg_len,
 				    todo_item_get_arg(todo_list, item));
 			continue;
 		}
 
 		/* add command to the buffer */
 		if (flags & TODO_LIST_ABBREVIATE_CMDS)
-			strbuf_addch(&buf, command_to_char(item->command));
+			strbuf_addch(buf, command_to_char(item->command));
 		else
-			strbuf_addstr(&buf, command_to_string(item->command));
+			strbuf_addstr(buf, command_to_string(item->command));
 
 		/* add commit id */
 		if (item->commit) {
@@ -4536,28 +4538,48 @@ void todo_list_transform(struct repository *r, struct todo_list *todo_list,
 
 			if (item->command == TODO_MERGE) {
 				if (item->flags & TODO_EDIT_MERGE_MSG)
-					strbuf_addstr(&buf, " -c");
+					strbuf_addstr(buf, " -c");
 				else
-					strbuf_addstr(&buf, " -C");
+					strbuf_addstr(buf, " -C");
 			}
 
-			strbuf_addf(&buf, " %s", oid);
+			strbuf_addf(buf, " %s", oid);
 		}
 
 		/* add all the rest */
 		if (!item->arg_len)
-			strbuf_addch(&buf, '\n');
+			strbuf_addch(buf, '\n');
 		else
-			strbuf_addf(&buf, " %.*s\n", item->arg_len,
+			strbuf_addf(buf, " %.*s\n", item->arg_len,
 				    todo_item_get_arg(todo_list, item));
 	}
+}
 
-	strbuf_reset(&todo_list->buf);
-	strbuf_add(&todo_list->buf, buf.buf, buf.len);
+int todo_list_write_to_file(struct repository *r, struct todo_list *todo_list,
+			    const char *file, const char *shortrevisions,
+			    const char *shortonto, int num, unsigned flags)
+{
+	int edit_todo = !(shortrevisions && shortonto), res;
+	struct strbuf buf = STRBUF_INIT;
+
+	todo_list_to_strbuf(r, todo_list, &buf, num, flags);
+
+	if (flags & TODO_LIST_APPEND_TODO_HELP) {
+		int command_count = count_commands(todo_list);
+		if (!edit_todo) {
+			strbuf_addch(&buf, '\n');
+			strbuf_commented_addf(&buf, Q_("Rebase %s onto %s (%d command)",
+						       "Rebase %s onto %s (%d commands)",
+						       command_count),
+					      shortrevisions, shortonto, command_count);
+		}
+		append_todo_help(edit_todo, flags & TODO_LIST_KEEP_EMPTY, &buf);
+	}
+
+	res = write_message(buf.buf, buf.len, file, 0);
 	strbuf_release(&buf);
 
-	if (todo_list_parse_insn_buffer(r, todo_list->buf.buf, todo_list))
-		BUG("unusable todo list");
+	return res;
 }
 
 int transform_todo_file(struct repository *r, unsigned flags)
@@ -4574,9 +4596,8 @@ int transform_todo_file(struct repository *r, unsigned flags)
 		return error(_("unusable todo list: '%s'"), todo_file);
 	}
 
-	todo_list_transform(r, &todo_list, flags);
-
-	res = write_message(todo_list.buf.buf, todo_list.buf.len, todo_file, 0);
+	res = todo_list_write_to_file(r, &todo_list, todo_file,
+				      NULL, NULL, -1, flags);
 	todo_list_release(&todo_list);
 
 	if (res)
sequencer.h +7 -4
index 35472c137b..dc3d8f76f7 100644
@@ -121,8 +121,9 @@ struct todo_list {
 
 int todo_list_parse_insn_buffer(struct repository *r, char *buf,
 				struct todo_list *todo_list);
-void todo_list_transform(struct repository *r, struct todo_list *todo_list,
-			 unsigned flags);
+int todo_list_write_to_file(struct repository *r, struct todo_list *todo_list,
+			    const char *file, const char *shortrevisions,
+			    const char *shortonto, int num, unsigned flags);
 void todo_list_release(struct todo_list *todo_list);
 const char *todo_item_get_arg(struct todo_list *todo_list,
 			      struct todo_item *item);
@@ -145,8 +146,10 @@ int sequencer_remove_state(struct replay_opts *opts);
  * commits should be rebased onto the new base, this flag needs to be passed.
  */
 #define TODO_LIST_REBASE_COUSINS (1U << 4)
-int sequencer_make_script(struct repository *repo, FILE *out,
-			  int argc, const char **argv,
+#define TODO_LIST_APPEND_TODO_HELP (1U << 5)
+
+int sequencer_make_script(struct repository *r, FILE *out, int argc,
+			  const char **argv,
 			  unsigned flags);
 
 int sequencer_add_exec_commands(struct repository *r, const char *command);
Signature · 1 line
2.20.1
#120 lore raw

PATCH v6 6/16 sequencer: refactor check_todo_list() to work on a todo_list

In reply to Alban Gruin

This refactors check_todo_list() to work on a todo_list to avoid redundant reads and writes to the disk. The function is renamed todo_list_check(). The parsing of the two todo lists is left to the caller.

As rebase -p still need to check the todo list from the disk, a new function is introduced, check_todo_list_from_file(). It reads the file from the disk, parses it, pass the todo_list to todo_list_check(), and writes it back to the disk.

As get_missing_commit_check_level() and the enum missing_commit_check_level are no longer needed inside of sequencer.c, they are moved to rebase-interactive.c, and made static again.

---
 builtin/rebase--interactive.c |   2 +-
 rebase-interactive.c          |  91 ++++++++++++++++++++++++-
 rebase-interactive.h          |   2 +
 sequencer.c                   | 121 +++++++---------------------------
 sequencer.h                   |   9 +--
 5 files changed, 117 insertions(+), 108 deletions(-)
builtin/rebase--interactive.c +1 -1
index 0898eb4c59..df19ccaeb9 100644
@@ -256,7 +256,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = transform_todo_file(the_repository, flags);
 		break;
 	case CHECK_TODO_LIST:
-		ret = check_todo_list(the_repository);
+		ret = check_todo_list_from_file(the_repository);
 		break;
 	case REARRANGE_SQUASH:
 		ret = rearrange_squash(the_repository);
rebase-interactive.c +90 -1

101 lines not on this page · raw message

rebase-interactive.h +2 -0
index 17b6c9f6d0..187b5032d6 100644
@@ -3,9 +3,11 @@
 
 struct strbuf;
 struct repository;
+struct todo_list;
 
 void append_todo_help(unsigned edit_todo, unsigned keep_empty,
 		      struct strbuf *buf);
 int edit_todo_list(struct repository *r, unsigned flags);
+int todo_list_check(struct todo_list *old_todo, struct todo_list *new_todo);
 
 #endif
sequencer.c +23 -98

142 lines not on this page · raw message

sequencer.h +1 -8
index dc3d8f76f7..1de97f188d 100644
@@ -64,12 +64,6 @@ struct replay_opts {
 };
 #define REPLAY_OPTS_INIT { .action = -1, .current_fixups = STRBUF_INIT }
 
-enum missing_commit_check_level {
-	MISSING_COMMIT_CHECK_IGNORE = 0,
-	MISSING_COMMIT_CHECK_WARN,
-	MISSING_COMMIT_CHECK_ERROR
-};
-
 int write_message(const void *buf, size_t len, const char *filename,
 		  int append_eol);
 
@@ -154,8 +148,7 @@ int sequencer_make_script(struct repository *r, FILE *out, int argc,
 
 int sequencer_add_exec_commands(struct repository *r, const char *command);
 int transform_todo_file(struct repository *r, unsigned flags);
-enum missing_commit_check_level get_missing_commit_check_level(void);
-int check_todo_list(struct repository *r);
+int check_todo_list_from_file(struct repository *r);
 int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
 		    const char *onto, const char *orig_head, const char *cmd,
Signature · 1 line
2.20.1
#121 lore raw

PATCH v6 7/16 sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

In reply to Alban Gruin

This refactors sequencer_add_exec_commands() to work on a todo_list to avoid redundant reads and writes to the disk.

Instead of inserting the `exec' commands between the other commands and re-parsing the buffer at the end, they are appended to the buffer once, and a new list of items is created. Items from the old list are copied across and new `exec' items are appended when necessary. This eliminates the need to reparse the buffer, but this also means we have to use todo_list_write_to_disk() to write the file.

todo_list_add_exec_commands() and sequencer_add_exec_commands() are modified to take a string list instead of a string -- one item for each command. This makes it easier to insert a new command to the todo list for each command to execute.

sequencer_add_exec_commands() still reads the todo list from the disk, as it is needed by rebase -p.

complete_action() still uses sequencer_add_exec_commands() for now. This will be changed in a future commit.

---
 builtin/rebase--interactive.c |  15 +++--
 sequencer.c                   | 110 +++++++++++++++++++++-------------
 sequencer.h                   |   5 +-
 3 files changed, 82 insertions(+), 48 deletions(-)
builtin/rebase--interactive.c +11 -4
index df19ccaeb9..53056ee713 100644
@@ -65,7 +65,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 				 const char *onto, const char *onto_name,
 				 const char *squash_onto, const char *head_name,
 				 const char *restrict_revision, char *raw_strategies,
-				 const char *cmd, unsigned autosquash)
+				 struct string_list *commands, unsigned autosquash)
 {
 	int ret;
 	const char *head_hash = NULL;
@@ -116,7 +116,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 		discard_cache();
 		ret = complete_action(the_repository, opts, flags,
 				      shortrevisions, onto_name, onto,
-				      head_hash, cmd, autosquash);
+				      head_hash, commands, autosquash);
 	}
 
 	free(revisions);
@@ -139,6 +139,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 	const char *onto = NULL, *onto_name = NULL, *restrict_revision = NULL,
 		*squash_onto = NULL, *upstream = NULL, *head_name = NULL,
 		*switch_to = NULL, *cmd = NULL;
+	struct string_list commands = STRING_LIST_INIT_DUP;
 	char *raw_strategies = NULL;
 	enum {
 		NONE = 0, CONTINUE, SKIP, EDIT_TODO, SHOW_CURRENT_PATCH,
@@ -221,6 +222,12 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		warning(_("--[no-]rebase-cousins has no effect without "
 			  "--rebase-merges"));
 
+	if (cmd && *cmd) {
+		string_list_split(&commands, cmd, '\n', -1);
+		if (strlen(commands.items[commands.nr - 1].string) == 0)
+			--commands.nr;
+	}
+
 	switch (command) {
 	case NONE:
 		if (!onto && !upstream)
@@ -228,7 +235,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 
 		ret = do_interactive_rebase(&opts, flags, switch_to, upstream, onto,
 					    onto_name, squash_onto, head_name, restrict_revision,
-					    raw_strategies, cmd, autosquash);
+					    raw_strategies, &commands, autosquash);
 		break;
 	case SKIP: {
 		struct string_list merge_rr = STRING_LIST_INIT_DUP;
@@ -262,7 +269,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = rearrange_squash(the_repository);
 		break;
 	case ADD_EXEC:
-		ret = sequencer_add_exec_commands(the_repository, cmd);
+		ret = sequencer_add_exec_commands(the_repository, &commands);
 		break;
 	default:
 		BUG("invalid command '%d'", command);
sequencer.c +68 -42

146 lines not on this page · raw message

sequencer.h +3 -2
index 1de97f188d..e79f03e213 100644
@@ -146,12 +146,13 @@ int sequencer_make_script(struct repository *r, FILE *out, int argc,
 			  const char **argv,
 			  unsigned flags);
 
-int sequencer_add_exec_commands(struct repository *r, const char *command);
+int sequencer_add_exec_commands(struct repository *r,
+				struct string_list *commands);
 int transform_todo_file(struct repository *r, unsigned flags);
 int check_todo_list_from_file(struct repository *r);
 int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
-		    const char *onto, const char *orig_head, const char *cmd,
+		    const char *onto, const char *orig_head, struct string_list *commands,
 		    unsigned autosquash);
 int rearrange_squash(struct repository *r);
 
Signature · 1 line
2.20.1
#122 lore raw

Re: [PATCH v6 07/16] sequencer: refactor sequencer_add_exec_commands() to work on a todo_list

Hi Alban

Quoted 59 lines · Alban Gruin

This refactors sequencer_add_exec_commands() to work on a todo_list to avoid redundant reads and writes to the disk.

Instead of inserting the `exec' commands between the other commands and re-parsing the buffer at the end, they are appended to the buffer once, and a new list of items is created. Items from the old list are copied across and new `exec' items are appended when necessary. This eliminates the need to reparse the buffer, but this also means we have to use todo_list_write_to_disk() to write the file.

todo_list_add_exec_commands() and sequencer_add_exec_commands() are modified to take a string list instead of a string -- one item for each command. This makes it easier to insert a new command to the todo list for each command to execute.

sequencer_add_exec_commands() still reads the todo list from the disk, as it is needed by rebase -p.

complete_action() still uses sequencer_add_exec_commands() for now. This will be changed in a future commit.

---
 builtin/rebase--interactive.c |  15 +++--
 sequencer.c                   | 110 +++++++++++++++++++++-------------
 sequencer.h                   |   5 +-
 3 files changed, 82 insertions(+), 48 deletions(-)
builtin/rebase--interactive.c +3 -2
index df19ccaeb9..53056ee713 100644
@@ -65,7 +65,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 				 const char *onto, const char *onto_name,
 				 const char *squash_onto, const char *head_name,
 				 const char *restrict_revision, char *raw_strategies,
-				 const char *cmd, unsigned autosquash)
+				 struct string_list *commands, unsigned autosquash)
 {
 	int ret;
 	const char *head_hash = NULL;
@@ -116,7 +116,7 @@ static int do_interactive_rebase(struct replay_opts *opts, unsigned flags,
 		discard_cache();
 		ret = complete_action(the_repository, opts, flags,
 				      shortrevisions, onto_name, onto,
-				      head_hash, cmd, autosquash);
+				      head_hash, commands, autosquash);
 	}
 
 	free(revisions);
@@ -139,6 +139,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 	const char *onto = NULL, *onto_name = NULL, *restrict_revision = NULL,
 		*squash_onto = NULL, *upstream = NULL, *head_name = NULL,
 		*switch_to = NULL, *cmd = NULL;
+	struct string_list commands = STRING_LIST_INIT_DUP;
 	char *raw_strategies = NULL;
 	enum {
 		NONE = 0, CONTINUE, SKIP, EDIT_TODO, SHOW_CURRENT_PATCH,

builtin/rebase--interactive.c int cmd_rebase__interactive(int argc, const char **argv, const char *prefix) · line 226

Quoted 5 lines
builtin/rebase--interactive.c +2 -0
@@ -221,6 +222,12 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		warning(_("--[no-]rebase-cousins has no effect without "
 			  "--rebase-merges"));
 
+	if (cmd && *cmd) {
+		string_list_split(&commands, cmd, '\n', -1);

This whole splitting and later skipping 'exec ' is a bit of a shame - it would be much nicer if we could just have one exec command per -x option but I think that is outside the scope of this series (If I have time I'd like to look at calling do_interactive_rebase() directly from builtin/rebase.c without forking rebase--interactive).

builtin/rebase--interactive.c int cmd_rebase__interactive(int argc, const char **argv, const char *prefix) · line 227

Quoted 1 line
builtin/rebase--interactive.c +1 -0
+		if (strlen(commands.items[commands.nr - 1].string) == 0)

I'd be tempted just to test the string using !* rather than calling strlen. Also is there ever a case where the last string isn't empty?

Quoted 80 lines
builtin/rebase--interactive.c +5 -2
+			--commands.nr;
+	}
+
 	switch (command) {
 	case NONE:
 		if (!onto && !upstream)
@@ -228,7 +235,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 
 		ret = do_interactive_rebase(&opts, flags, switch_to, upstream, onto,
 					    onto_name, squash_onto, head_name, restrict_revision,
-					    raw_strategies, cmd, autosquash);
+					    raw_strategies, &commands, autosquash);
 		break;
 	case SKIP: {
 		struct string_list merge_rr = STRING_LIST_INIT_DUP;
@@ -262,7 +269,7 @@ int cmd_rebase__interactive(int argc, const char **argv, const char *prefix)
 		ret = rearrange_squash(the_repository);
 		break;
 	case ADD_EXEC:
-		ret = sequencer_add_exec_commands(the_repository, cmd);
+		ret = sequencer_add_exec_commands(the_repository, &commands);
 		break;
 	default:
 		BUG("invalid command '%d'", command);
sequencer.c +18 -20
index 266f80d704..3a90b419d7 100644
@@ -4446,25 +4446,27 @@ int sequencer_make_script(struct repository *r, FILE *out,
 	return 0;
 }
 
-/*
- * Add commands after pick and (series of) squash/fixup commands
- * in the todo list.
- */
-int sequencer_add_exec_commands(struct repository *r,
-				const char *commands)
+static void todo_list_add_exec_commands(struct todo_list *todo_list,
+					struct string_list *commands)
 {
-	const char *todo_file = rebase_path_todo();
-	struct todo_list todo_list = TODO_LIST_INIT;
-	struct strbuf *buf = &todo_list.buf;
-	size_t offset = 0, commands_len = strlen(commands);
-	int i, insert;
+	struct strbuf *buf = &todo_list->buf;
+	size_t base_offset = buf->len;
+	int i, insert, nr = 0, alloc = 0;
+	struct todo_item *items = NULL, *base_items = NULL;
 
-	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
-		return error(_("could not read '%s'."), todo_file);
+	base_items = xcalloc(commands->nr, sizeof(struct todo_item));
+	for (i = 0; i < commands->nr; ++i) {
+		size_t command_len = strlen(commands->items[i].string);
 
-	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
-		todo_list_release(&todo_list);
-		return error(_("unusable todo list: '%s'"), todo_file);
+		strbuf_addstr(buf, commands->items[i].string);
+		strbuf_addch(buf, '\n');
+
+		base_items[i].command = TODO_EXEC;
+		base_items[i].offset_in_buf = base_offset;
+		base_items[i].arg_offset = base_offset + strlen("exec ");
+		base_items[i].arg_len = command_len - strlen("exec ");
+
+		base_offset += command_len + 1;
 	}
 
 	/*
@@ -4473,38 +4475,62 @@ int sequencer_add_exec_commands(struct repository *r,
 	 * those chains if there are any.
 	 */
 	insert = -1;
-	for (i = 0; i < todo_list.nr; i++) {
-		enum todo_command command = todo_list.items[i].command;
-
-		if (insert >= 0) {

sequencer.c int sequencer_add_exec_commands(struct repository *r, · line 4489

Quoted 10 lines
sequencer.c +0 -10
-			/* skip fixup/squash chains */
-			if (command == TODO_COMMENT)
-				continue;
-			else if (is_fixup(command)) {
-				insert = i + 1;
-				continue;
-			}
-			strbuf_insert(buf,
-				      todo_list.items[insert].offset_in_buf +
-				      offset, commands, commands_len);

In a todo list that looks like
pick abc message
#pick cde empty commit
This inserts the exec command for the first pick above the commented out pick. I think your translation puts it below the commented out pick as it ignores the value of insert. I think it's probably easiest to add an INSERT_ARRAY macro to insert it in the right place. An alternative might be to track the last insert position and only copy commands across when there is another exec to insert but that might get complicated in cases such as

pick abc message
#squash cde squash! message //empty commit for rewording fixup 123 fixup! message
#pick 456 empty commit

Best Wishes

Phillip

Quoted 108 lines
sequencer.c +49 -12
-			offset += commands_len;
+	for (i = 0; i < todo_list->nr; i++) {
+		enum todo_command command = todo_list->items[i].command;
+		if (insert >= 0 && command != TODO_COMMENT && !is_fixup(command)) {
+			ALLOC_GROW(items, nr + commands->nr, alloc);
+			COPY_ARRAY(items + nr, base_items, commands->nr);
+			nr += commands->nr;
 			insert = -1;
 		}
 
-		if (command == TODO_PICK || command == TODO_MERGE)
+		ALLOC_GROW(items, nr + 1, alloc);
+		items[nr++] = todo_list->items[i];
+
+		if (command == TODO_PICK || command == TODO_MERGE || is_fixup(command))
 			insert = i + 1;
 	}
 
 	/* insert or append final <commands> */
-	if (insert >= 0 && insert < todo_list.nr)
-		strbuf_insert(buf, todo_list.items[insert].offset_in_buf +
-			      offset, commands, commands_len);
-	else if (insert >= 0 || !offset)
-		strbuf_add(buf, commands, commands_len);
+	if (insert >= 0 || nr == todo_list->nr) {
+		ALLOC_GROW(items, nr + commands->nr, alloc);
+		COPY_ARRAY(items + nr, base_items, commands->nr);
+		nr += commands->nr;
+	}
+
+	free(base_items);
+	FREE_AND_NULL(todo_list->items);
+	todo_list->items = items;
+	todo_list->nr = nr;
+	todo_list->alloc = alloc;> +}
 
-	i = write_message(buf->buf, buf->len, todo_file, 0);
+/*
+ * Add commands after pick and (series of) squash/fixup commands
+ * in the todo list.
+ */
+int sequencer_add_exec_commands(struct repository *r,
+				struct string_list *commands)
+{
+	const char *todo_file = rebase_path_todo();
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res;
+
+	if (strbuf_read_file(&todo_list.buf, todo_file, 0) < 0)
+		return error_errno(_("could not read '%s'."), todo_file);
+
+	if (todo_list_parse_insn_buffer(r, todo_list.buf.buf, &todo_list)) {
+		todo_list_release(&todo_list);
+		return error(_("unusable todo list: '%s'"), todo_file);
+	}
+
+	todo_list_add_exec_commands(&todo_list, commands);
+	res = todo_list_write_to_file(r, &todo_list, todo_file, NULL, NULL, -1, 0);
 	todo_list_release(&todo_list);
-	return i;
+
+	if (res)
+		return error_errno(_("could not write '%s'."), todo_file);
+	return 0;
 }
 
 static void todo_list_to_strbuf(struct repository *r, struct todo_list *todo_list,
@@ -4735,7 +4761,7 @@ static int skip_unnecessary_picks(struct repository *r, struct object_id *output
 
 int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
-		    const char *onto, const char *orig_head, const char *cmd,
+		    const char *onto, const char *orig_head, struct string_list *commands,
 		    unsigned autosquash)
 {
 	const char *shortonto, *todo_file = rebase_path_todo();
@@ -4754,8 +4780,8 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla
 	if (autosquash && rearrange_squash(r))
 		return -1;
 
-	if (cmd && *cmd)
-		sequencer_add_exec_commands(r, cmd);
+	if (commands->nr)
+		sequencer_add_exec_commands(r, commands);
 
 	if (strbuf_read_file(buf, todo_file, 0) < 0)
 		return error_errno(_("could not read '%s'."), todo_file);
sequencer.h +3 -2
index 1de97f188d..e79f03e213 100644
@@ -146,12 +146,13 @@ int sequencer_make_script(struct repository *r, FILE *out, int argc,
 			  const char **argv,
 			  unsigned flags);
 
-int sequencer_add_exec_commands(struct repository *r, const char *command);
+int sequencer_add_exec_commands(struct repository *r,
+				struct string_list *commands);
 int transform_todo_file(struct repository *r, unsigned flags);
 int check_todo_list_from_file(struct repository *r);
 int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
 		    const char *shortrevisions, const char *onto_name,
-		    const char *onto, const char *orig_head, const char *cmd,
+		    const char *onto, const char *orig_head, struct string_list *commands,
 		    unsigned autosquash);
 int rearrange_squash(struct repository *r);