git config cache & special querying api utilizing the cache
Subsystems (2)
v9 cover git config cache & special querying api utilizing the cache
Hi,
[PATCH v9]: Applied most of the review comments mentioned in [11]. Mostly asthetic changes.
test-config now clears the config_set before exiting. Most of the tests now use the check-config function. check_config_init() now handles return values correctly. Diff between v8 and v9 is at the bottom. Thanks to Junio and Matthieu for the review.
[PATCH V8]: Moved the contents of config-set.c to config.c for future convenience. Reverted
test 'find value with misspelled key' to the one in v5. See [10] for the discussion.
[PATCH V7]: Style nits and a broken && chain corrected in t/t1308-config-set.sh. See
[9] for the nits.
[PATCH V6]: Style nits and mistakes corrected. Diff between v6 and v5[8] is at the bottom.
Thanks to Matthieu, Ramsay and Ram for their suggestions.
[PATCH V5]: config_set now uses a single hashmap. Corrected style nits raised in
the thread[7]. Thanks to Junio and Matthieu for their suggestions.
[PATCH v4]: Introduced config_set construct which points to a ordered set of
config-files cached as hashmaps. Added relevant API functions. For more details see the documentation. Rewrote the git_config_get* family to use config_set internally. Added tests for both config_set API and git_config_get family. Added type specific API functions which parses the found value and converts it into a specific type.
Most of the changes implemented are the result of discussion in [6]. Thanks to Eric, Ramsay, Junio, Matthieu & Karsten for their suggestions and review.
[PATCH v3]: Added flag for NULL values that were causing segfaults in some cases.
Added test-config for usage examples.
Minor changes and corrections. Refer to discussion thread[5] for more details. Thanks to Matthieu, Jeff and Junio for their valuable suggestions.
[PATCH v2]:Changed the string_list to a struct instead of pointer to a struct.
Added string-list initilization functions.
Minor mistakes corrected acoording to review comments[4]. Thanks to Eric and Matthieu for their review.
[PATCH V1]:Most of the invaluable suggestions by Eric Sunshine, Torsten Bogershausen and
Jeff King has been implemented[1]. Complete rewrite of config_cache*() family using git_config() as hook as suggested by Jeff. Thanks for the review.
[RFC V2]: Improved according to the suggestions by Eric Sunshine and Torsten Bogershausen.
Added cache invalidation when config file is changed.[2]
I am using git_config_set_multivar_in_file() as an update hook.
This is patch is for this year's GSoC. My project is
"Git Config API improvements". The link of my proposal is appended below [3].
The aim of this patch series is to generate a cache for querying values from the config files in a non-callback manner as the current method reads and parses the config files every time a value is queried for.
The cache is generated from hooking the update_cache function to the current parsing and callback mechanism in config.c. It is implemented as an hashmap using the hashmap-api with variables and its corresponding values list as its members. The values in the list are sorted in order of increasing priority. The cache is initialised the first time when any of the new query functions is called. It is invalidated by using git_config_set_multivar_in_file() as an update hook.
We add two new functions to the config-api, git_config_get_string() and git_config_get_string_multi() for querying in a non callback manner from the cache.
[1] http://marc.info/?t=140172066200006&r=1&w=2
[2] http://git.661346.n2.nabble.com/RFC-PATCH-0-2-Git-config-cache-amp-special-querying-api-utilizing-the-cache-td7611691.html
[3] https://drive.google.com/file/d/0B4suZ-aHqDcnSUZJRXVTTnZUN1E/edit?usp=sharing
[4] http://thread.gmane.org/gmane.comp.version-control.git/251073/focus=251369
[5] http://thread.gmane.org/gmane.comp.version-control.git/251704/
[6] http://thread.gmane.org/gmane.comp.version-control.git/252329/
[7] http://marc.info/?t=140428115200001&r=1&w=2
[8] http://article.gmane.org/gmane.comp.version-control.git/252942/
[9] http://thread.gmane.org/gmane.comp.version-control.git/252959/
[10] http://article.gmane.org/gmane.comp.version-control.git/253113
[11] http://thread.gmane.org/gmane.comp.version-control.git/253248
Tanay Abhra (2):
config set
test-config
.gitignore | 1 + Documentation/technical/api-config.txt | 135 ++++++++++++++++ Makefile | 1 + cache.h | 31 ++++ config.c | 276 +++++++++++++++++++++++++++++++++ t/t1308-config-set.sh | 212 +++++++++++++++++++++++++ test-config.c | 140 +++++++++++++++++ 7 files changed, 796 insertions(+) create mode 100755 t/t1308-config-set.sh create mode 100644 test-config.c
Signature · 3 lines
1.9.0.GIT -- 8< --
Documentation/technical/api-config.txt +2 -1
@@ -241,7 +241,8 @@ Configset API provides functions for the above mentioned work flow, including: `int git_configset_add_file(struct config_set *cs, const char *filename)`:: Parses the file and adds the variable-value pairs to the `config_set`, - dies if there is an error in parsing the file. + dies if there is an error in parsing the file. Returns 0 on success, or + -1 if the file doesnot exist or cannot be read. `int git_configset_get_value(struct config_set *cs, const char *key, const char **value)`::
config.c +68 -88
227 lines not on this page · raw message
t/t1308-config-set.sh +92 -50
225 lines not on this page · raw message
test-config.c +33 -18
109 lines not on this page · raw message
-- 8< --
PATCH v9 1/2 add `config_set` API for caching config-like files
Currently git_config() uses a callback mechanism and file rereads for config values. Due to this approach, it is not uncommon for the config files to be parsed several times during the run of a git program, with different callbacks picking out different variables useful to themselves.
Add a config_set, that can be used to construct an in-memory cache for config-like files that the caller specifies (i.e., files like .gitmodules, ~/.gitconfig etc.). Add two external functions git_configset_get_value and git_configset_get_value_multi for querying from the config sets. git_configset_get_value follows last one wins semantic (i.e. if there are multiple matches for the queried key in the files of the configset the value returned will be the last entry in value_list). git_configset_get_value_multi returns a list of values sorted in order of increasing priority (i.e. last match will be at the end of the list). Add type specific query functions like git_configset_get_bool and similar.
Add a default config_set, the_config_set to cache all key-value pairs read from usual config files (repo specific .git/config, user wide ~/.gitconfig, XDG config and the global /etc/gitconfig). the_config_set is populated using git_config().
Add two external functions git_config_get_value and git_config_get_value_multi for querying in a non-callback manner from the_config_set. Also, add type specific query functions that are implemented as a thin wrapper around the config_set API.
--- Documentation/technical/api-config.txt | 135 ++++++++++++++++ cache.h | 31 ++++ config.c | 276 +++++++++++++++++++++++++++++++++ 3 files changed, 442 insertions(+)
Documentation/technical/api-config.txt +135 -0
@@ -77,6 +77,81 @@ To read a specific file in git-config format, use `git_config_from_file`. This takes the same callback and data parameters as `git_config`. +Querying For Specific Variables +------------------------------- + +For programs wanting to query for specific variables in a non-callback +manner, the config API provides two functions `git_config_get_value` +and `git_config_get_value_multi`. They both read values from an internal +cache generated previously from reading the config files. + +`int git_config_get_value(const char *key, const char **value)`:: + + Finds the highest-priority value for the configuration variable `key`, + stores the pointer to it in `value` and returns 0. When the + configuration variable `key` is not found, returns 1 without touching + `value`. The caller should not free or modify `value`, as it is owned + by the cache. + +`const struct string_list *git_config_get_value_multi(const char *key)`:: + + Finds and returns the value list, sorted in order of increasing priority + for the configuration variable `key`. When the configuration variable + `key` is not found, returns NULL. The caller should not free or modify + the returned pointer, as it is owned by the cache. + +`void git_config_clear(void)`:: + + Resets and invalidates the config cache. + +The config API also provides type specific API functions which do conversion +as well as retrieval for the queried variable, including: + +`int git_config_get_int(const char *key, int *dest)`:: + + Finds and parses the value to an integer for the configuration variable + `key`. Dies on error; otherwise, stores the value of the parsed integer in + `dest` and returns 0. When the configuration variable `key` is not found, + returns 1 without touching `dest`. + +`int git_config_get_ulong(const char *key, unsigned long *dest)`:: + + Similar to `git_config_get_int` but for unsigned longs. + +`int git_config_get_bool(const char *key, int *dest)`:: + + Finds and parses the value into a boolean value, for the configuration + variable `key` respecting keywords like "true" and "false". Integer + values are converted into true/false values (when they are non-zero or + zero, respectively). Other values cause a die(). If parsing is successful, + stores the value of the parsed result in `dest` and returns 0. When the + configuration variable `key` is not found, returns 1 without touching + `dest`. + +`int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest)`:: + + Similar to `git_config_get_bool`, except that integers are copied as-is, + and `is_bool` flag is unset. + +`int git_config_get_maybe_bool(const char *key, int *dest)`:: + + Similar to `git_config_get_bool`, except that it returns -1 on error + rather than dying. + +`int git_config_get_string(const char *key, const char **dest)`:: + + Allocates and copies the retrieved string into the `dest` parameter for + the configuration variable `key`; if NULL string is given, prints an + error message and returns -1. When the configuration variable `key` is + not found, returns 1 without touching `dest`. + +`int git_config_get_pathname(const char *key, const char **dest)`:: + + Similar to `git_config_get_string`, but expands `~` or `~user` into + the user's home directory when found at the beginning of the path. + +See test-config.c for usage examples. + Value Parsing Helpers ---------------------
@@ -134,6 +209,66 @@ int read_file_with_include(const char *file, config_fn_t fn, void *data) `git_config` respects includes automatically. The lower-level `git_config_from_file` does not. +Custom Configsets +----------------- + +A `config_set` can be used to construct an in-memory cache for +config-like files that the caller specifies (i.e., files like `.gitmodules`, +`~/.gitconfig` etc.). For example, + +--------------------------------------- +struct config_set gm_config; +git_configset_init(&gm_config); +int b; +/* we add config files to the config_set */ +git_configset_add_file(&gm_config, ".gitmodules"); +git_configset_add_file(&gm_config, ".gitmodules_alt"); + +if (!git_configset_get_bool(gm_config, "submodule.frotz.ignore", &b)) { + /* hack hack hack */ +} + +/* when we are done with the configset */ +git_configset_clear(&gm_config); +---------------------------------------- + +Configset API provides functions for the above mentioned work flow, including: + +`void git_configset_init(struct config_set *cs)`:: + + Initializes the config_set `cs`. + +`int git_configset_add_file(struct config_set *cs, const char *filename)`:: + + Parses the file and adds the variable-value pairs to the `config_set`, + dies if there is an error in parsing the file. Returns 0 on success, or + -1 if the file doesnot exist or cannot be read. + +`int git_configset_get_value(struct config_set *cs, const char *key, const char **value)`:: + + Finds the highest-priority value for the configuration variable `key` + and config set `cs`, stores the pointer to it in `value` and returns 0. + When the configuration variable `key` is not found, returns 1 without + touching `value`. The caller should not free or modify `value`, as it + is owned by the cache. + +`const struct string_list *git_configset_get_value_multi(struct config_set *cs, const char *key)`:: + + Finds and returns the value list, sorted in order of increasing priority + for the configuration variable `key` and config set `cs`. When the + configuration variable `key` is not found, returns NULL. The caller + should not free or modify the returned pointer, as it is owned by the cache. + +`void git_configset_clear(struct config_set *cs)`:: + + Clears `config_set` structure, removes all saved variable-value pairs. + +In addition to above functions, the `config_set` API provides type specific +functions in the vein of `git_config_get_int` and family but with an extra +parameter, pointer to struct `config_set`. +They all behave similarly to the `git_config_get*()` family described in +"Querying For Specific Variables" above. + Writing Config Files --------------------
cache.h +31 -0
@@ -1329,6 +1329,37 @@ extern int parse_config_key(const char *var, const char **subsection, int *subsection_len, const char **key); +struct config_set { +struct hashmap config_hash; +int hash_initialized; +}; + +extern void git_configset_init (struct config_set *cs); +extern int git_configset_add_file (struct config_set *cs,const char *filename); +extern int git_configset_get_value (struct config_set *cs,const char *key,const char **value); +extern const struct string_list *git_configset_get_value_multi (struct config_set *cs,const char *key); +extern void git_configset_clear (struct config_set *cs); +extern void git_configset_iter (struct config_set *cs,config_fn_t fn,void *data); +extern int git_configset_get_string (struct config_set *cs,const char *key,const char **dest); +extern int git_configset_get_int (struct config_set *cs,const char *key,int *dest); +extern int git_configset_get_ulong (struct config_set *cs,const char *key,unsigned long *dest); +extern int git_configset_get_bool (struct config_set *cs,const char *key,int *dest); +extern int git_configset_get_bool_or_int (struct config_set *cs,const char *key,int *is_bool,int *dest); +extern int git_configset_get_maybe_bool (struct config_set *cs,const char *key,int *dest); +extern int git_configset_get_pathname (struct config_set *cs,const char *key,const char **dest); + +extern int git_config_get_value (const char *key,const char **value); +extern const struct string_list *git_config_get_value_multi (const char *key); +extern void git_config_clear (void ); +extern void git_config_iter (config_fn_t fn,void *data); +extern int git_config_get_string (const char *key,const char **dest); +extern int git_config_get_int (const char *key,int *dest); +extern int git_config_get_ulong (const char *key,unsigned long *dest); +extern int git_config_get_bool (const char *key,int *dest); +extern int git_config_get_bool_or_int (const char *key,int *is_bool,int *dest); +extern int git_config_get_maybe_bool (const char *key,int *dest); +extern int git_config_get_pathname (const char *key,const char **dest); +extern int committer_ident_sufficiently_given (void );extern int author_ident_sufficiently_given (void );
config.c +276 -0
304 lines not on this page · raw message
Signature · 1 line
1.9.0.GIT
Re: [PATCH v9 1/2] add `config_set` API for caching config-like files
config.c int git_config(config_fn_t fn, void *data) · line 1256
Quoted 10 lines · Tanay Abhra
config.c +5
-0+const struct string_list *git_configset_get_value_multi (struct config_set *cs,const char *key) +{ +struct config_set_element *e =configset_find_element (cs, key); +return e ? &e->value_list :NULL ; +}
Somehow I find the placement of this function out of place. Didn't you get the same impression while proofreading the entire file after you are done writing this patch?
The right place for it would probably be immediately before git_configset_get_value(), I would think.
config.c int git_config(config_fn_t fn, void *data) · line 1324
Quoted 11 lines
config.c +11
-0+int git_configset_add_file (struct config_set *cs,const char *filename) +{ +int ret =0 ; + ret =git_config_from_file (config_hash_callback, filename, cs); +if (!ret) +return 0 ; +else { +git_configset_clear (cs); +return -1 ; + } +}
If I add contents from file "A" successfully and then attempt to further add contents from file "B" which fails, I would lose contents I read from "A"?
It would not be worth trying to make it transactional (i.e. making sure cs contains what was there before the config-from-file was called if it failed), so in such a case we may end up seeing a mixture of full contents from A and partial contents from B, which is just as useless as a cleared configset, so it is not wrong to clear it per-se. An alternative might be to return without clearing, as we are leaving the configset in a useless state either way and give the caller a choice between clearing it and continue, and dying without even spending unnecessary cycles to clear. That might be more consistent with what git_config_check_init() does, where you die without bothering to clear what was half-read into the configset.
Whatever behaviour is chosen in the error codepath, it needs to be documented.
Thanks.
Re: [PATCH v9 1/2] add `config_set` API for caching config-like files
Quoted 17 lines · Junio C Hamano
config.c int git_config(config_fn_t fn, void *data) · line 1256
Quoted 10 lines · Tanay Abhra
config.c +5
-0+const struct string_list *git_configset_get_value_multi (struct config_set *cs,const char *key) +{ +struct config_set_element *e =configset_find_element (cs, key); +return e ? &e->value_list :NULL ; +}Somehow I find the placement of this function out of place. Didn't you get the same impression while proofreading the entire file after you are done writing this patch?
Quoted 3 lines
The right place for it would probably be immediately before
git_configset_get_value(), I would think.
Noted. My fault, will correct in the reroll.
Quoted 29 lines
config.c int git_config(config_fn_t fn, void *data) · line 1324
Quoted 11 lines
config.c +11
-0+int git_configset_add_file (struct config_set *cs,const char *filename) +{ +int ret =0 ; + ret =git_config_from_file (config_hash_callback, filename, cs); +if (!ret) +return 0 ; +else { +git_configset_clear (cs); +return -1 ; + } +}If I add contents from file "A" successfully and then attempt to further add contents from file "B" which fails, I would lose contents I read from "A"?
It would not be worth trying to make it transactional (i.e. making sure cs contains what was there before the config-from-file was called if it failed), so in such a case we may end up seeing a mixture of full contents from A and partial contents from B, which is just as useless as a cleared configset, so it is not wrong to clear it per-se. An alternative might be to return without clearing, as we are leaving the configset in a useless state either way and give the caller a choice between clearing it and continue, and dying without even spending unnecessary cycles to clear. That might be more consistent with what
git_config_check_init()does, where you die without bothering to clear what was half-read into the configset.
Quoted 3 lines
Whatever behaviour is chosen in the error codepath, it needs to be documented.
Since syntactical errors in reading the file will cause a die when we use either git_config_from_file or git_config, I don't think incomplete parsing would trigger the error path, as it will die before reaching there.
So, the best way would be just to return without clearing and let the user take the decision if he wants to go on with the incomplete config_set or clear it when he sees that configset_add_file() failed.
Die in git_config_check_init() is never triggered expect in rare race conditions like between access_or_die() and git_config_from_file() in
if (git_config_system() && !access_or_die(git_etc_gitconfig(), R_OK, 0)) {
ret += git_config_from_file(fn, git_etc_gitconfig(),
data);
found += 1;
}which I think would never happen in reality, dunno. I think I will take out the die() in git_config_check_init(). Thus, the behavior of both functions git_config_check_init() & git_configset_add_file will be consistent.
Thanks.
PATCH 1/2 add `config_set` API for caching config-like files
In reply to Junio C HamanoCurrently git_config() uses a callback mechanism and file rereads for config values. Due to this approach, it is not uncommon for the config files to be parsed several times during the run of a git program, with different callbacks picking out different variables useful to themselves.
Add a config_set, that can be used to construct an in-memory cache for config-like files that the caller specifies (i.e., files like .gitmodules, ~/.gitconfig etc.). Add two external functions git_configset_get_value and git_configset_get_value_multi for querying from the config sets. git_configset_get_value follows last one wins semantic (i.e. if there are multiple matches for the queried key in the files of the configset the value returned will be the last entry in value_list). git_configset_get_value_multi returns a list of values sorted in order of increasing priority (i.e. last match will be at the end of the list). Add type specific query functions like git_configset_get_bool and similar.
Add a default config_set, the_config_set to cache all key-value pairs read from usual config files (repo specific .git/config, user wide ~/.gitconfig, XDG config and the global /etc/gitconfig). the_config_set is populated using git_config().
Add two external functions git_config_get_value and git_config_get_value_multi for querying in a non-callback manner from the_config_set. Also, add type specific query functions that are implemented as a thin wrapper around the config_set API.
--- Documentation/technical/api-config.txt | 137 +++++++++++++++++ cache.h | 30 ++++ config.c | 264 +++++++++++++++++++++++++++++++++ 3 files changed, 431 insertions(+)
Documentation/technical/api-config.txt +137 -0
@@ -77,6 +77,81 @@ To read a specific file in git-config format, use `git_config_from_file`. This takes the same callback and data parameters as `git_config`. +Querying For Specific Variables +------------------------------- + +For programs wanting to query for specific variables in a non-callback +manner, the config API provides two functions `git_config_get_value` +and `git_config_get_value_multi`. They both read values from an internal +cache generated previously from reading the config files. + +`int git_config_get_value(const char *key, const char **value)`:: + + Finds the highest-priority value for the configuration variable `key`, + stores the pointer to it in `value` and returns 0. When the + configuration variable `key` is not found, returns 1 without touching + `value`. The caller should not free or modify `value`, as it is owned + by the cache. + +`const struct string_list *git_config_get_value_multi(const char *key)`:: + + Finds and returns the value list, sorted in order of increasing priority + for the configuration variable `key`. When the configuration variable + `key` is not found, returns NULL. The caller should not free or modify + the returned pointer, as it is owned by the cache. + +`void git_config_clear(void)`:: + + Resets and invalidates the config cache. + +The config API also provides type specific API functions which do conversion +as well as retrieval for the queried variable, including: + +`int git_config_get_int(const char *key, int *dest)`:: + + Finds and parses the value to an integer for the configuration variable + `key`. Dies on error; otherwise, stores the value of the parsed integer in + `dest` and returns 0. When the configuration variable `key` is not found, + returns 1 without touching `dest`. + +`int git_config_get_ulong(const char *key, unsigned long *dest)`:: + + Similar to `git_config_get_int` but for unsigned longs. + +`int git_config_get_bool(const char *key, int *dest)`:: + + Finds and parses the value into a boolean value, for the configuration + variable `key` respecting keywords like "true" and "false". Integer + values are converted into true/false values (when they are non-zero or + zero, respectively). Other values cause a die(). If parsing is successful, + stores the value of the parsed result in `dest` and returns 0. When the + configuration variable `key` is not found, returns 1 without touching + `dest`. + +`int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest)`:: + + Similar to `git_config_get_bool`, except that integers are copied as-is, + and `is_bool` flag is unset. + +`int git_config_get_maybe_bool(const char *key, int *dest)`:: + + Similar to `git_config_get_bool`, except that it returns -1 on error + rather than dying. + +`int git_config_get_string(const char *key, const char **dest)`:: + + Allocates and copies the retrieved string into the `dest` parameter for + the configuration variable `key`; if NULL string is given, prints an + error message and returns -1. When the configuration variable `key` is + not found, returns 1 without touching `dest`. + +`int git_config_get_pathname(const char *key, const char **dest)`:: + + Similar to `git_config_get_string`, but expands `~` or `~user` into + the user's home directory when found at the beginning of the path. + +See test-config.c for usage examples. + Value Parsing Helpers ---------------------
@@ -134,6 +209,68 @@ int read_file_with_include(const char *file, config_fn_t fn, void *data) `git_config` respects includes automatically. The lower-level `git_config_from_file` does not. +Custom Configsets +----------------- + +A `config_set` can be used to construct an in-memory cache for +config-like files that the caller specifies (i.e., files like `.gitmodules`, +`~/.gitconfig` etc.). For example, + +--------------------------------------- +struct config_set gm_config; +git_configset_init(&gm_config); +int b; +/* we add config files to the config_set */ +git_configset_add_file(&gm_config, ".gitmodules"); +git_configset_add_file(&gm_config, ".gitmodules_alt"); + +if (!git_configset_get_bool(gm_config, "submodule.frotz.ignore", &b)) { + /* hack hack hack */ +} + +/* when we are done with the configset */ +git_configset_clear(&gm_config); +---------------------------------------- + +Configset API provides functions for the above mentioned work flow, including: + +`void git_configset_init(struct config_set *cs)`:: + + Initializes the config_set `cs`. + +`int git_configset_add_file(struct config_set *cs, const char *filename)`:: + + Parses the file and adds the variable-value pairs to the `config_set`, + dies if there is an error in parsing the file. Returns 0 on success, or + -1 if the file doesnot exist or is inaccessible. The user has to decide + if he wants to free the incomplete configset or continue using it when + the function returns -1. + +`int git_configset_get_value(struct config_set *cs, const char *key, const char **value)`:: + + Finds the highest-priority value for the configuration variable `key` + and config set `cs`, stores the pointer to it in `value` and returns 0. + When the configuration variable `key` is not found, returns 1 without + touching `value`. The caller should not free or modify `value`, as it + is owned by the cache. + +`const struct string_list *git_configset_get_value_multi(struct config_set *cs, const char *key)`:: + + Finds and returns the value list, sorted in order of increasing priority + for the configuration variable `key` and config set `cs`. When the + configuration variable `key` is not found, returns NULL. The caller + should not free or modify the returned pointer, as it is owned by the cache. + +`void git_configset_clear(struct config_set *cs)`:: + + Clears `config_set` structure, removes all saved variable-value pairs. + +In addition to above functions, the `config_set` API provides type specific +functions in the vein of `git_config_get_int` and family but with an extra +parameter, pointer to struct `config_set`. +They all behave similarly to the `git_config_get*()` family described in +"Querying For Specific Variables" above. + Writing Config Files --------------------
cache.h +30 -0
@@ -1329,6 +1329,36 @@ extern int parse_config_key(const char *var, const char **subsection, int *subsection_len, const char **key); +struct config_set { +struct hashmap config_hash; +int hash_initialized; +}; + +extern void git_configset_init (struct config_set *cs); +extern int git_configset_add_file (struct config_set *cs,const char *filename); +extern int git_configset_get_value (struct config_set *cs,const char *key,const char **value); +extern const struct string_list *git_configset_get_value_multi (struct config_set *cs,const char *key); +extern void git_configset_clear (struct config_set *cs); +extern int git_configset_get_string (struct config_set *cs,const char *key,const char **dest); +extern int git_configset_get_int (struct config_set *cs,const char *key,int *dest); +extern int git_configset_get_ulong (struct config_set *cs,const char *key,unsigned long *dest); +extern int git_configset_get_bool (struct config_set *cs,const char *key,int *dest); +extern int git_configset_get_bool_or_int (struct config_set *cs,const char *key,int *is_bool,int *dest); +extern int git_configset_get_maybe_bool (struct config_set *cs,const char *key,int *dest); +extern int git_configset_get_pathname (struct config_set *cs,const char *key,const char **dest); + +extern int git_config_get_value (const char *key,const char **value); +extern const struct string_list *git_config_get_value_multi (const char *key); +extern void git_config_clear (void ); +extern void git_config_iter (config_fn_t fn,void *data); +extern int git_config_get_string (const char *key,const char **dest); +extern int git_config_get_int (const char *key,int *dest); +extern int git_config_get_ulong (const char *key,unsigned long *dest); +extern int git_config_get_bool (const char *key,int *dest); +extern int git_config_get_bool_or_int (const char *key,int *is_bool,int *dest); +extern int git_config_get_maybe_bool (const char *key,int *dest); +extern int git_config_get_pathname (const char *key,const char **dest); +extern int committer_ident_sufficiently_given (void );extern int author_ident_sufficiently_given (void );
config.c +264 -0
292 lines not on this page · raw message
Signature · 1 line
1.9.0.GIT
PATCH v9 2/2 test-config: add tests for the config_set API
In reply to Tanay AbhraExpose the config_set C API as a set of simple commands in order to facilitate testing. Add tests for the config_set API as well as for git_config_get_*() family for the usual config files.
--- .gitignore | 1 + Makefile | 1 + t/t1308-config-set.sh | 212 ++++++++++++++++++++++++++++++++++++++++++++++++++ test-config.c | 140 +++++++++++++++++++++++++++++++++ 4 files changed, 354 insertions(+) create mode 100755 t/t1308-config-set.sh create mode 100644 test-config.c
.gitignore +1 -0
@@ -177,6 +177,7 @@ /gitweb/static/gitweb.min.* /test-chmtime /test-ctype +/test-config /test-date /test-delta /test-dump-cache-tree
Makefile +1 -0
@@ -549,6 +549,7 @@ PROGRAMS += $(patsubst %.o,git-%$X,$(PROGRAM_OBJS)) TEST_PROGRAMS_NEED_X += test-chmtime TEST_PROGRAMS_NEED_X += test-ctype +TEST_PROGRAMS_NEED_X += test-config TEST_PROGRAMS_NEED_X += test-date TEST_PROGRAMS_NEED_X += test-delta TEST_PROGRAMS_NEED_X += test-dump-cache-tree
t/t1308-config-set.sh new file +212 -0
212 lines not on this page · raw message
test-config.c new file +140 -0
140 lines not on this page · raw message
Signature · 1 line
1.9.0.GIT
Re: [PATCH v9 2/2] test-config: add tests for the config_set API
Quoted 48 lines · Tanay Abhra
Expose the
config_setC API as a set of simple commands in order to facilitate testing. Add tests for theconfig_setAPI as well as forgit_config_get_*()family for the usual config files.Signed-off-by Matthieu Moy <[email protected]>Signed-off-by Tanay Abhra <[email protected]>--- .gitignore | 1 + Makefile | 1 + t/t1308-config-set.sh | 212 ++++++++++++++++++++++++++++++++++++++++++++++++++ test-config.c | 140 +++++++++++++++++++++++++++++++++ 4 files changed, 354 insertions(+) create mode 100755 t/t1308-config-set.sh create mode 100644 test-config.c.gitignore +1
-0@@ -177,6 +177,7 @@ /gitweb/static/gitweb.min.* /test-chmtime /test-ctype +/test-config /test-date /test-delta /test-dump-cache-treeMakefile +1
-0@@ -549,6 +549,7 @@ PROGRAMS += $(patsubst %.o,git-%$X,$(PROGRAM_OBJS)) TEST_PROGRAMS_NEED_X += test-chmtime TEST_PROGRAMS_NEED_X += test-ctype +TEST_PROGRAMS_NEED_X += test-config TEST_PROGRAMS_NEED_X += test-date TEST_PROGRAMS_NEED_X += test-delta TEST_PROGRAMS_NEED_X += test-dump-cache-treet/t1308-config-set.sh new file +3
-0@@ -0,0 +1,212 @@ +#!/bin/sh + +test_description='Test git config-set API in different settings'
t/t1308-config-set.sh line 13
Quoted 10 lines
t/t1308-config-set.sh new file +10
-0+ +. ./test-lib.sh + +# 'check_config get_* section.key value' verifies that the entry for +# section.key is 'value' +check_config () { + case "$1" in + expect_code) + if [ "$#" -lt 5 ]; + then
Spell out "test" and drop unnecessary semicolon, i.e.
if test $# -lt 5
then
t/t1308-config-set.sh line 16
Quoted 3 lines
t/t1308-config-set.sh new file +3
-0+ >expect + else + printf "%s\n" "$5" >expect
The other "expecting success" side of this function allows to expect more than one line of output, but this one only allows you to expect at most one line? Why?
t/t1308-config-set.sh line 28
Quoted 12 lines
t/t1308-config-set.sh new file +12
-0+ fi && + test_expect_code "$2" test-config "$3" "$4" >actual + ;; + *) + op=$1 key=$2 && + shift && + shift && + printf "%s\n" "$@" >expect && + test-config "$op" "$key" >actual + ;; + esac && + test_cmp expect actual
Perhaps you meant to say something like this instead?
if test "$1" = expect_code
then
expect_code="$2" && shift && shift
else
expect_code=0
fi &&
op=$1 key=$2 && shift && shift
if test $# != 0
then
printf "%s\n" "$@"
fi >expect &&
test_expect_code $expet_code test-config "$op" "$key" >actual &&
test_cmp expect actualI dunno.
t/t1308-config-set.sh line 32
Quoted 2 lines
t/t1308-config-set.sh new file +2
-0+test_expect_success 'setup default config' ' + cat >.git/config <<\EOF
So the default .git/config that was prepared by "git init" is discarded and replaced with this? Shouldn't it be
cat >>.git/config <<\EOF
instead?
t/t1308-config-set.sh line 148
Quoted 9 lines
t/t1308-config-set.sh new file +9
-0+test_expect_success 'find multiple values' ' + cat >expect <<-\EOF && + sam + bat + hask + EOF + test-config get_value_multi "case.baz">actual && + test_cmp expect actual +'
Hmmm, wasn't the whole point of the helper to allow us to make things like the above into a one-liner, perhaps like this?
check_config get_value_multi case.baz sam bat hask
I suspect the same applies to most if not all uses of test-config in the remainder of this patch.
Re: [PATCH v9 2/2] test-config: add tests for the config_set API
Quoted 10 lines · Junio C Hamano
t/t1308-config-set.sh line 32
Quoted 2 lines
t/t1308-config-set.sh new file +2
-0+test_expect_success 'setup default config' ' + cat >.git/config <<\EOFSo the default .git/config that was prepared by "git init" is discarded and replaced with this? Shouldn't it be
cat >>.git/config <<\EOFinstead?
Most of tests like t1300-repo-config.sh or t1303-wacky-config.sh clears the default config, will it be okay to clear it in this test series also? I need it fortest_expect_success 'proper error on error in default config files' ' which requires me to compare the line no at which the error was found.
Quoted 13 lines
t/t1308-config-set.sh line 148
Quoted 9 lines
t/t1308-config-set.sh new file +9
-0+test_expect_success 'find multiple values' ' + cat >expect <<-\EOF && + sam + bat + hask + EOF + test-config get_value_multi "case.baz">actual && + test_cmp expect actual +'Hmmm, wasn't the whole point of the helper to allow us to make things like the above into a one-liner, perhaps like this?
Quoted 1 line
check_configget_value_multicase.baz sam bat hask
Noted and corrected.
Quoted 3 lines
I suspect the same applies to most if not all uses of test-config in the remainder of this patch.
I cant use it for configset_get_value_* as it may have variable number of files as arguments. :)
Thanks,
Tanay Abhra.
Re: [PATCH v9 2/2] test-config: add tests for the config_set API
Quoted 3 lines · Tanay Abhra
... I need it for
test_expect_success'proper error on error in default config files' ' which requires me to compare the line no at which the error was found.
I see. We may want to later rethink the way you validate the line numbers, but in the meantime I think the version posted should suffice, if you cannot do better than that.
Thanks.
PATCH 2/2 test-config: add tests for the config_set API
In reply to Junio C HamanoExpose the config_set C API as a set of simple commands in order to facilitate testing. Add tests for the config_set API as well as for git_config_get_*() family for the usual config files.
--- .gitignore | 1 + Makefile | 1 + t/t1308-config-set.sh | 200 ++++++++++++++++++++++++++++++++++++++++++++++++++ test-config.c | 140 +++++++++++++++++++++++++++++++++++ 4 files changed, 342 insertions(+) create mode 100755 t/t1308-config-set.sh create mode 100644 test-config.c
.gitignore +1 -0
@@ -177,6 +177,7 @@ /gitweb/static/gitweb.min.* /test-chmtime /test-ctype +/test-config /test-date /test-delta /test-dump-cache-tree
Makefile +1 -0
@@ -549,6 +549,7 @@ PROGRAMS += $(patsubst %.o,git-%$X,$(PROGRAM_OBJS)) TEST_PROGRAMS_NEED_X += test-chmtime TEST_PROGRAMS_NEED_X += test-ctype +TEST_PROGRAMS_NEED_X += test-config TEST_PROGRAMS_NEED_X += test-date TEST_PROGRAMS_NEED_X += test-delta TEST_PROGRAMS_NEED_X += test-dump-cache-tree
t/t1308-config-set.sh new file +200 -0
200 lines not on this page · raw message
test-config.c new file +140 -0
140 lines not on this page · raw message
Signature · 1 line
1.9.0.GIT
Re: [PATCH v9r1 2/2] test-config: add tests for the config_set API
t/t1308-config-set.sh line 17
Quoted 11 lines · Tanay Abhra
t/t1308-config-set.sh new file +11
-0+# 'check_config get_* section.key value' verifies that the entry for +# section.key is 'value' +check_config () { + if test "$1" = expect_code + then + expect_code="$2" && shift && shift + else + expect_code=0 + fi && + op=$1 key=$2 && shift && shift + if test $# != 0
Broken &&-chain after shift.
(No time for more review right now, but I'll have time in ~3h from now)
Signature · 2 lines
Matthieu Moy http://www-verimag.imag.fr/~moy/
PATCH 1/2 add `config_set` API for caching config-like files
Currently git_config() uses a callback mechanism and file rereads for config values. Due to this approach, it is not uncommon for the config files to be parsed several times during the run of a git program, with different callbacks picking out different variables useful to themselves.
Add a config_set, that can be used to construct an in-memory cache for config-like files that the caller specifies (i.e., files like .gitmodules, ~/.gitconfig etc.). Add two external functions git_configset_get_value and git_configset_get_value_multi for querying from the config sets. git_configset_get_value follows last one wins semantic (i.e. if there are multiple matches for the queried key in the files of the configset the value returned will be the last entry in value_list). git_configset_get_value_multi returns a list of values sorted in order of increasing priority (i.e. last match will be at the end of the list). Add type specific query functions like git_configset_get_bool and similar.
Add a default config_set, the_config_set to cache all key-value pairs read from usual config files (repo specific .git/config, user wide ~/.gitconfig, XDG config and the global /etc/gitconfig). the_config_set is populated using git_config().
Add two external functions git_config_get_value and git_config_get_value_multi for querying in a non-callback manner from the_config_set. Also, add type specific query functions that are implemented as a thin wrapper around the config_set API.
Documentation/technical/api-config.txt | 137 +++++++++++++++++ cache.h | 30 ++++ config.c | 264 +++++++++++++++++++++++++++++++++ 3 files changed, 431 insertions(+)
Documentation/technical/api-config.txt +137 -0
@@ -77,6 +77,81 @@ To read a specific file in git-config format, use `git_config_from_file`. This takes the same callback and data parameters as `git_config`. +Querying For Specific Variables +------------------------------- + +For programs wanting to query for specific variables in a non-callback +manner, the config API provides two functions `git_config_get_value` +and `git_config_get_value_multi`. They both read values from an internal +cache generated previously from reading the config files. + +`int git_config_get_value(const char *key, const char **value)`:: + + Finds the highest-priority value for the configuration variable `key`, + stores the pointer to it in `value` and returns 0. When the + configuration variable `key` is not found, returns 1 without touching + `value`. The caller should not free or modify `value`, as it is owned + by the cache. + +`const struct string_list *git_config_get_value_multi(const char *key)`:: + + Finds and returns the value list, sorted in order of increasing priority + for the configuration variable `key`. When the configuration variable + `key` is not found, returns NULL. The caller should not free or modify + the returned pointer, as it is owned by the cache. + +`void git_config_clear(void)`:: + + Resets and invalidates the config cache. + +The config API also provides type specific API functions which do conversion +as well as retrieval for the queried variable, including: + +`int git_config_get_int(const char *key, int *dest)`:: + + Finds and parses the value to an integer for the configuration variable + `key`. Dies on error; otherwise, stores the value of the parsed integer in + `dest` and returns 0. When the configuration variable `key` is not found, + returns 1 without touching `dest`. + +`int git_config_get_ulong(const char *key, unsigned long *dest)`:: + + Similar to `git_config_get_int` but for unsigned longs. + +`int git_config_get_bool(const char *key, int *dest)`:: + + Finds and parses the value into a boolean value, for the configuration + variable `key` respecting keywords like "true" and "false". Integer + values are converted into true/false values (when they are non-zero or + zero, respectively). Other values cause a die(). If parsing is successful, + stores the value of the parsed result in `dest` and returns 0. When the + configuration variable `key` is not found, returns 1 without touching + `dest`. + +`int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest)`:: + + Similar to `git_config_get_bool`, except that integers are copied as-is, + and `is_bool` flag is unset. + +`int git_config_get_maybe_bool(const char *key, int *dest)`:: + + Similar to `git_config_get_bool`, except that it returns -1 on error + rather than dying. + +`int git_config_get_string(const char *key, const char **dest)`:: + + Allocates and copies the retrieved string into the `dest` parameter for + the configuration variable `key`; if NULL string is given, prints an + error message and returns -1. When the configuration variable `key` is + not found, returns 1 without touching `dest`. + +`int git_config_get_pathname(const char *key, const char **dest)`:: + + Similar to `git_config_get_string`, but expands `~` or `~user` into + the user's home directory when found at the beginning of the path. + +See test-config.c for usage examples. + Value Parsing Helpers ---------------------
@@ -134,6 +209,68 @@ int read_file_with_include(const char *file, config_fn_t fn, void *data) `git_config` respects includes automatically. The lower-level `git_config_from_file` does not. +Custom Configsets +----------------- + +A `config_set` can be used to construct an in-memory cache for +config-like files that the caller specifies (i.e., files like `.gitmodules`, +`~/.gitconfig` etc.). For example, + +--------------------------------------- +struct config_set gm_config; +git_configset_init(&gm_config); +int b; +/* we add config files to the config_set */ +git_configset_add_file(&gm_config, ".gitmodules"); +git_configset_add_file(&gm_config, ".gitmodules_alt"); + +if (!git_configset_get_bool(gm_config, "submodule.frotz.ignore", &b)) { + /* hack hack hack */ +} + +/* when we are done with the configset */ +git_configset_clear(&gm_config); +---------------------------------------- + +Configset API provides functions for the above mentioned work flow, including: + +`void git_configset_init(struct config_set *cs)`:: + + Initializes the config_set `cs`. + +`int git_configset_add_file(struct config_set *cs, const char *filename)`:: + + Parses the file and adds the variable-value pairs to the `config_set`, + dies if there is an error in parsing the file. Returns 0 on success, or + -1 if the file doesnot exist or is inaccessible. The user has to decide + if he wants to free the incomplete configset or continue using it when + the function returns -1. + +`int git_configset_get_value(struct config_set *cs, const char *key, const char **value)`:: + + Finds the highest-priority value for the configuration variable `key` + and config set `cs`, stores the pointer to it in `value` and returns 0. + When the configuration variable `key` is not found, returns 1 without + touching `value`. The caller should not free or modify `value`, as it + is owned by the cache. + +`const struct string_list *git_configset_get_value_multi(struct config_set *cs, const char *key)`:: + + Finds and returns the value list, sorted in order of increasing priority + for the configuration variable `key` and config set `cs`. When the + configuration variable `key` is not found, returns NULL. The caller + should not free or modify the returned pointer, as it is owned by the cache. + +`void git_configset_clear(struct config_set *cs)`:: + + Clears `config_set` structure, removes all saved variable-value pairs. + +In addition to above functions, the `config_set` API provides type specific +functions in the vein of `git_config_get_int` and family but with an extra +parameter, pointer to struct `config_set`. +They all behave similarly to the `git_config_get*()` family described in +"Querying For Specific Variables" above. + Writing Config Files --------------------
cache.h +30 -0
@@ -1329,6 +1329,36 @@ extern int parse_config_key(const char *var, const char **subsection, int *subsection_len, const char **key); +struct config_set { +struct hashmap config_hash; +int hash_initialized; +}; + +extern void git_configset_init (struct config_set *cs); +extern int git_configset_add_file (struct config_set *cs,const char *filename); +extern int git_configset_get_value (struct config_set *cs,const char *key,const char **value); +extern const struct string_list *git_configset_get_value_multi (struct config_set *cs,const char *key); +extern void git_configset_clear (struct config_set *cs); +extern int git_configset_get_string (struct config_set *cs,const char *key,const char **dest); +extern int git_configset_get_int (struct config_set *cs,const char *key,int *dest); +extern int git_configset_get_ulong (struct config_set *cs,const char *key,unsigned long *dest); +extern int git_configset_get_bool (struct config_set *cs,const char *key,int *dest); +extern int git_configset_get_bool_or_int (struct config_set *cs,const char *key,int *is_bool,int *dest); +extern int git_configset_get_maybe_bool (struct config_set *cs,const char *key,int *dest); +extern int git_configset_get_pathname (struct config_set *cs,const char *key,const char **dest); + +extern int git_config_get_value (const char *key,const char **value); +extern const struct string_list *git_config_get_value_multi (const char *key); +extern void git_config_clear (void ); +extern void git_config_iter (config_fn_t fn,void *data); +extern int git_config_get_string (const char *key,const char **dest); +extern int git_config_get_int (const char *key,int *dest); +extern int git_config_get_ulong (const char *key,unsigned long *dest); +extern int git_config_get_bool (const char *key,int *dest); +extern int git_config_get_bool_or_int (const char *key,int *is_bool,int *dest); +extern int git_config_get_maybe_bool (const char *key,int *dest); +extern int git_config_get_pathname (const char *key,const char **dest); +extern int committer_ident_sufficiently_given (void );extern int author_ident_sufficiently_given (void );
config.c +264 -0
292 lines not on this page · raw message
Signature · 1 line
1.9.0.GIT
Re: [PATCH v9r2 1/2] add `config_set` API for caching config-like files
Quoted 8 lines · Tanay Abhra
implemented as a thin wrapper around the
config_setAPI.Signed-off-by Matthieu Moy <[email protected]>Signed-off-by Tanay Abhra <[email protected]>Documentation/technical/api-config.txt | 137 +++++++++++++++++ cache.h | 30 ++++ config.c | 264 +++++++++++++++++++++++++++++++++ 3 files changed, 431 insertions(+)
(you broke the patch by removing the ---)
config.c int git_config(config_fn_t fn, void *data) · line 1413
Quoted 7 lines
config.c +7
-0+static void git_config_check_init (void ) +{ +if (the_config_set.hash_initialized) +return ; +git_configset_init (&the_config_set); +git_config (config_set_callback, &the_config_set); +}
So, you're now ignoring the return value of git_config. What is the rationale for this? In particular, why did you reject the "die" possibility (I understood that you were inclined to take this option, so I'm curious why you changed your mind).
OTOH, you're transmitting the return value without dying here:
+int git_configset_add_file(struct config_set *cs, const char *filename) +{
+ return git_config_from_file(config_set_callback, filename, cs);
+}
and I think this one is correct, as we cannot tell in advance how serious an error would be for any callers. And we do test this (I think we can improve a bit, I'll send a fixup patch).
Signature · 2 lines
Matthieu Moy http://www-verimag.imag.fr/~moy/
PATCH 1/3 fixup for patch 2: minor style fix
--- t/t1308-config-set.sh | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
t/t1308-config-set.sh +6 -6
@@ -150,8 +150,8 @@ test_expect_success 'find value from a configset' ' ' test_expect_success 'find value with highest priority from a configset' ' - echo hask > expect && - test-config configset_get_value case.baz config2 .git/config >actual && + echo hask >expect && + test-config configset_get_value case.baz config2 .git/config >actual && test_cmp expect actual '
@@ -163,7 +163,7 @@ test_expect_success 'find value_list for a key from a configset' ' lama ball EOF - test-config configset_get_value case.baz config2 .git/config >actual && + test-config configset_get_value case.baz config2 .git/config >actual && test_cmp expect actual '
@@ -173,7 +173,7 @@ test_expect_success 'proper error on non-existant files' ' test_cmp expect actual ' -test_expect_success 'proper error on non-accessible files' ' +test_expect_success 'proper error on non-accessible files' ' chmod -r .git/config && test_when_finished "chmod +r .git/config" && echo "Error reading configuration file .git/config." >expect &&
@@ -184,14 +184,14 @@ test_expect_success 'proper error on non-accessible files' ' test_expect_success 'proper error on error in default config files' ' cp .git/config .git/config.old && test_when_finished "mv .git/config.old .git/config" && - echo "[" >> .git/config && + echo "[" >>.git/config && echo "fatal: bad config file line 35 in .git/config" >expect && test_expect_code 128 test-config get_value foo.bar 2>actual && test_cmp expect actual ' test_expect_success 'proper error on error in custom config files' ' - echo "[" >> syntax-error && + echo "[" >>syntax-error && echo "fatal: bad config file line 1 in syntax-error" >expect && test_expect_code 128 test-config configset_get_value foo.bar syntax-error 2>actual && test_cmp expect actual
Signature · 1 line
2.0.0.262.gdafc651
PATCH 2/3 fixup for patch 2: actually check the return value
---
I won't fight for this, but I think it makes sense.
t/t1308-config-set.sh | 4 ++-- test-config.c | 10 ++++++---- 2 files changed, 8 insertions(+), 6 deletions(-)
t/t1308-config-set.sh +2 -2
@@ -168,7 +168,7 @@ test_expect_success 'find value_list for a key from a configset' ' ' test_expect_success 'proper error on non-existant files' ' - echo "Error reading configuration file non-existant-file." >expect && + echo "Error (-1) reading configuration file non-existant-file." >expect && test_expect_code 2 test-config configset_get_value foo.bar non-existant-file 2>actual && test_cmp expect actual '
@@ -176,7 +176,7 @@ test_expect_success 'proper error on non-existant files' ' test_expect_success 'proper error on non-accessible files' ' chmod -r .git/config && test_when_finished "chmod +r .git/config" && - echo "Error reading configuration file .git/config." >expect && + echo "Error (-1) reading configuration file .git/config." >expect && test_expect_code 2 test-config configset_get_value foo.bar .git/config 2>actual && test_cmp expect actual '
test-config.c +6 -4
@@ -86,8 +86,9 @@ int main(int argc, char **argv) } }else if (!strcmp (argv[1 ],"configset_get_value" )) {for (i =3 ; i < argc; i++) { -if (git_configset_add_file (&cs, argv[i])) { -fprintf (stderr,"Error reading configuration file %s.\n" , argv[i]); +int err; +if ((err =git_configset_add_file (&cs, argv[i]))) { +fprintf (stderr,"Error (%d) reading configuration file %s.\n" , err, argv[i]); goto exit2; } }
@@ -103,8 +104,9 @@ int main(int argc, char **argv) } }else if (!strcmp (argv[1 ],"configset_get_value_multi" )) {for (i =3 ; i < argc; i++) { -if (git_configset_add_file (&cs, argv[i])) { -fprintf (stderr,"Error reading configuration file %s.\n" , argv[i]); +int err; +if ((err =git_configset_add_file (&cs, argv[i]))) { +fprintf (stderr,"Error (%d) reading configuration file %s.\n" , err, argv[i]); goto exit2; } }
Signature · 1 line
2.0.0.262.gdafc651
Re: [PATCH 2/3] fixup for patch 2: actually check the return value
I think it would be unnecessary for the current iteration. Currently git_configset_add_file has only two possible return values -1 or 0. I could add specialized error values for ENOENT or ENOTDIR or EACCES, but the logs show that we silently ignore the first two. I can add an access warn for the third. What do you think?
Thanks,
Tanay.
Quoted 59 lines · Matthieu Moy
Signed-off-by Matthieu Moy <[email protected]>---
I won't fight for this, but I think it makes sense.t/t1308-config-set.sh | 4 ++-- test-config.c | 10 ++++++---- 2 files changed, 8 insertions(+), 6 deletions(-)t/t1308-config-set.sh +2
-2@@ -168,7 +168,7 @@ test_expect_success 'find value_list for a key from a configset' ' ' test_expect_success 'proper error on non-existant files' ' - echo "Error reading configuration file non-existant-file." >expect && + echo "Error (-1) reading configuration file non-existant-file." >expect && test_expect_code 2 test-config configset_get_value foo.bar non-existant-file 2>actual && test_cmp expect actual '@@ -176,7 +176,7 @@ test_expect_success 'proper error on non-existant files' ' test_expect_success 'proper error on non-accessible files' ' chmod -r .git/config && test_when_finished "chmod +r .git/config" && - echo "Error reading configuration file .git/config." >expect && + echo "Error (-1) reading configuration file .git/config." >expect && test_expect_code 2 test-config configset_get_value foo.bar .git/config 2>actual && test_cmp expect actual 'test-config.c +6
-4@@ -86,8 +86,9 @@ int main(int argc, char **argv) } }else if (!strcmp (argv[1 ],"configset_get_value" )) {for (i =3 ; i < argc; i++) { -if (git_configset_add_file (&cs, argv[i])) { -fprintf (stderr,"Error reading configuration file %s.\n" , argv[i]); +int err; +if ((err =git_configset_add_file (&cs, argv[i]))) { +fprintf (stderr,"Error (%d) reading configuration file %s.\n" , err, argv[i]); goto exit2; } }@@ -103,8 +104,9 @@ int main(int argc, char **argv) } }else if (!strcmp (argv[1 ],"configset_get_value_multi" )) {for (i =3 ; i < argc; i++) { -if (git_configset_add_file (&cs, argv[i])) { -fprintf (stderr,"Error reading configuration file %s.\n" , argv[i]); +int err; +if ((err =git_configset_add_file (&cs, argv[i]))) { +fprintf (stderr,"Error (%d) reading configuration file %s.\n" , err, argv[i]); goto exit2; } }
Re: [PATCH 2/3] fixup for patch 2: actually check the return value
Quoted 3 lines · Tanay Abhra
I think it would be unnecessary for the current iteration. Currently
git_configset_add_filehas only two possible return values -1 or 0.
Yes. My point is just to check that statement in tests. (I'm usually wary of statements like "is obviously true so I don't need to test it" ;-) )
Quoted 3 lines
I could add specialized error values for ENOENT or ENOTDIR or EACCES, but the logs show that we silently ignore the first two. I can add an access warn for the third. What do you think?
I think the code is fine as it is.
But anyway, it's not terribly important.
Signature · 2 lines
Matthieu Moy http://www-verimag.imag.fr/~moy/
PATCH 3/3 fixup for patch 1: typo
In reply to Matthieu Moy--- Documentation/technical/api-config.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/technical/api-config.txt +1 -1
@@ -242,7 +242,7 @@ Configset API provides functions for the above mentioned work flow, including: Parses the file and adds the variable-value pairs to the `config_set`, dies if there is an error in parsing the file. Returns 0 on success, or - -1 if the file doesnot exist or is inaccessible. The user has to decide + -1 if the file does not exist or is inaccessible. The user has to decide if he wants to free the incomplete configset or continue using it when the function returns -1.
Signature · 1 line
2.0.0.262.gdafc651
Re: [PATCH v9r2 1/2] add `config_set` API for caching config-like files
In reply to Matthieu MoyQuoted 4 lines · Matthieu Moy
Quoted 2 lines · Tanay Abhra
implemented as a thin wrapper around the
config_setAPI.
Quoted 9 lines
Quoted 6 lines
Signed-off-by Matthieu Moy <[email protected]>Signed-off-by Tanay Abhra <[email protected]>Documentation/technical/api-config.txt | 137 +++++++++++++++++ cache.h | 30 ++++ config.c | 264 +++++++++++++++++++++++++++++++++ 3 files changed, 431 insertions(+)(you broke the patch by removing the ---)
Yikes, sorry about that.
Quoted 4 lines
Quoted 4 lines
config.c +4
-0+static void git_config_check_init (void ) +{ +if (the_config_set.hash_initialized) +return ;
Quoted 9 lines
config.c int git_config(config_fn_t fn, void *data) · line 1413
Quoted 3 lines
config.c +3
-0+git_configset_init (&the_config_set); +git_config (config_set_callback, &the_config_set); +}So, you're now ignoring the return value of
git_config. What is the rationale for this? In particular, why did you reject the "die" possibility (I understood that you were inclined to take this option, so I'm curious why you changed your mind).
The errors (non accessible, non existent files etc) were already being caught by git_config_early(). Since git_config() only returns positive values except the weird race case you mentioned, I thought the die confused the reader of the patch more than it provided error checking. I also tried myself simulating the race condition but failed. All the callers of git_config() also ignore the return value, so I ended up ignoring the return value myself.
Quoted 11 lines
OTOH, you're transmitting the return value without dying here:
config.c +4
-0+int git_configset_add_file (struct config_set *cs,const char *filename) +{ +return git_config_from_file (config_set_callback, filename, cs); +}and I think this one is correct, as we cannot tell in advance how serious an error would be for any callers. And we do test this (I think we can improve a bit, I'll send a fixup patch).
After reading the commit log that you mentioned and some previous ones before that I surmised that the official slant was to silently ignore nonexistent files. Though an access_or_warn() check was placed on most of the files like git attributes, since non accessible file errors may be a user configuration error. So, I decided to ignore the return value.
But I do think that an access_or_warn() check should be put on git config --file and git_configset_add_file since other parts of git follow it. What do you think about it, still I will send followup patch correcting the git config --file condition where it silently ignores the file access error and continues?
Re: [PATCH v9r2 1/2] add `config_set` API for caching config-like files
Quoted 16 lines · Tanay Abhra
Quoted 14 lines · Matthieu Moy
config.c int git_config(config_fn_t fn, void *data) · line 1413
Quoted 7 lines · Tanay Abhra
config.c +7
-0+static void git_config_check_init (void ) +{ +if (the_config_set.hash_initialized) +return ; +git_configset_init (&the_config_set); +git_config (config_set_callback, &the_config_set); +}So, you're now ignoring the return value of
git_config. What is the rationale for this? In particular, why did you reject the "die" possibility (I understood that you were inclined to take this option, so I'm curious why you changed your mind).
Quoted 6 lines
The errors (non accessible, non existent files etc) were already being caught by
git_config_early(). Sincegit_config()only returns positive values except the weird race case you mentioned, I thought the die confused the reader of the patch more than it provided error checking. I also tried myself simulating the race condition but failed. All the callers ofgit_config()also ignore the return value, so I ended up ignoring the return value myself.
OK. My preference would be to die, but your argument makes sense.
Quoted 4 lines
But I do think that an
access_or_warn()check should be put on git config --file andgit_configset_add_filesince other parts of git follow it. What do you think about it, still I will send followup patch correcting the git config --file condition where it silently ignores the file access error and continues?
I think it should be done, but should not be your priority (i.e. it's good to do it, but only if the patch is trivial enough).
Signature · 2 lines
Matthieu Moy http://www-verimag.imag.fr/~moy/
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH v9r2 1/2] add `config_set` API for caching config-like files
Quoted 11 lines · Tanay Abhra
I have a question about renaming
git_config()togit_config_raw(). I was working on a patch and I am getting stuck here.
It fails in t1001, t1020 & t4018. The reason for the last test failure is unknown right now. For the first two, it boils down to this,git init --bare bare-ancestor-aliased.git && cd bare-ancestor-aliased.git && echo "[alias] aliasedinit = init" >>config && mkdir plain-nested && cd plain-nested && git aliasedinit
Git tries to read .git/config relative to the current directory, and tries to resolve aliases from it. The problem is: if one tries to do this from a subdirectory inside the repo, .git/config is not the right path, and the alias lookup fails.
I'll investigate more later.
Signature · 2 lines
Matthieu Moy http://www-verimag.imag.fr/~moy/
Re: [PATCH v9r2 1/2] add `config_set` API for caching config-like files
Quoted 14 lines · Matthieu Moy
Quoted 11 lines · Tanay Abhra
I have a question about renaming
git_config()togit_config_raw(). I was working on a patch and I am getting stuck here.
It fails in t1001, t1020 & t4018. The reason for the last test failure is unknown right now. For the first two, it boils down to this,git init --bare bare-ancestor-aliased.git && cd bare-ancestor-aliased.git && echo "[alias] aliasedinit = init" >>config && mkdir plain-nested && cd plain-nested && git aliasedinit
Quoted 7 lines
Git tries to read .git/config relative to the current directory, and tries to resolve aliases from it. The problem is: if one tries to do this from a subdirectory inside the repo, .git/config is not the right path, and the alias lookup fails.
I'll investigate more later.
Hmn, this does the trick,
-- 8< --
cache.h +1 -0
@@ -1272,6 +1272,7 @@ extern int git_config_from_buf(config_fn_t fn, const char *name, const char *buf, size_t len, void *data);extern void git_config_push_parameter (const char *text);extern int git_config_from_parameters (config_fn_t fn,void *data); +extern int git_config_raw (config_fn_t fn,void *);extern int git_config (config_fn_t fn,void *); extern int git_config_with_options(config_fn_t fn, void *, struct git_config_source *config_source,
config.c +27 -2
@@ -1222,11 +1222,36 @@ int git_config_with_options(config_fn_t fn, void *data,return ret; } -int git_config (config_fn_t fn,void *data) +extern int git_config_raw (config_fn_t fn,void *data) {return git_config_with_options (fn, data,NULL ,1 ); } +int git_configset_iter (struct config_set *cs,config_fn_t fn,void *data) +{ +int i; +struct string_list *strptr; +struct config_set_element *entry; +struct hashmap_iter iter; +hashmap_iter_init (&cs->config_hash, &iter); +while ((entry =hashmap_iter_next (&iter))) { + strptr = &entry->value_list; +for (i =0 ; i < strptr->nr; i++) { +if (fn (entry->key, strptr->items[i].string, data) <0 ) +die ("bad config file" ); + } + } +return 0 ; +} + +static void git_config_check_init (void ); + +int git_config (config_fn_t fn,void *data) +{ +git_config_check_init (); +return git_configset_iter (&the_config_set, fn, data); +} +static struct config_set_element *configset_find_element (struct config_set *cs,const char *key) {struct config_set_element k;
@@ -1409,7 +1434,7 @@ static void git_config_check_init(void)if (the_config_set.hash_initialized)return ;git_configset_init (&the_config_set); -git_config (config_hash_callback, &the_config_set); +git_config_raw (config_hash_callback, &the_config_set); }void git_config_clear (void )
pager.c +1 -1
@@ -177,7 +177,7 @@ int check_pager_config(const char *cmd) c.cmd = cmd; c.want =-1 ; c.value =NULL ; -git_config (pager_command_config, &c); +git_config_raw (pager_command_config, &c);if (c.value) pager_program = c.value;return c.want;
-- 8< --
The offending part is in git.c, line number 540:
static void execv_dashed_external(const char **argv)
{
struct strbuf cmd = STRBUF_INIT;
const char *tmp;
int status;
if (use_pager == -1)
use_pager = check_pager_config(argv[0]);
----------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^
commit_pager_choice();which cause git_config() to be called before handle_alias can call it in git.c line no 587,
static int run_argv(int *argcp, const char ***argv)
{
int done_alias = 0;
while (1) {
/* See if it's a builtin */
handle_builtin(*argcp, *argv);
/* .. then try the external ones */
execv_dashed_external(*argv);
/* calls git_config() first, skips the .git/config file*/
/* It could be an alias -- this works around the insanity
* of overriding "git log" with "git show" by having
* alias.log = show
*/
if (done_alias)
break;
save_env();
if (!handle_alias(argcp, argv))
/* does setup_git_directory_gently() before calling git_config() */
break;
done_alias = 1;
}
return done_alias;
}I am searching for a more elegant solution to this problem. Thanks,
Tanay Abhra.
Re: [PATCH v9r2 1/2] add `config_set` API for caching config-like files
Quoted 1 line · Tanay Abhra
I am searching for a more elegant solution to this problem.
The "efficient" (not sure about elegant) solution would be to keep one configset per file, and re-parse only the files needed.
I find the solution I posted in the other thread relatively "elegant":
invalidate the config cache when things change.
The last test failure seems more tricky. This (dirty) patch fixes the failure:
builtin/diff.c +3 -1
@@ -255,6 +255,8 @@ static int builtin_diff_files(struct rev_info *revs, int argc, const char **argvreturn run_diff_files (revs, options); } +int git_config_raw (config_fn_t fn,void *data); +int cmd_diff (int argc,const char **argv,const char *prefix) {int i;
@@ -317,7 +319,7 @@ int cmd_diff(int argc, const char **argv, const char *prefix)if (!no_index)gitmodules_config (); -git_config (git_diff_ui_config,NULL ); +git_config_raw (git_diff_ui_config,NULL );init_revisions (&rev, prefix);
But this one does not:
builtin/diff.c +2 -0
@@ -317,7 +317,9 @@ int cmd_diff(int argc, const char **argv, const char *prefix)if (!no_index)gitmodules_config (); +git_config_clear ();git_config (git_diff_ui_config,NULL ); +git_config_clear ();init_revisions (&rev, prefix);
So it's not just a matter of invalid cache not cleared, it's a real difference between git_config() and git_config_raw(). The guilty part is probably userdiff_config(const char *k, const char *v) in userdiff.c, that parses the xfuncname config option.
Signature · 2 lines
Matthieu Moy http://www-verimag.imag.fr/~moy/
Re: [PATCH v9r2 1/2] add `config_set` API for caching config-like files
In reply to Matthieu MoyQuoted 14 lines · Matthieu Moy
Quoted 11 lines · Tanay Abhra
I have a question about renaming
git_config()togit_config_raw(). I was working on a patch and I am getting stuck here.
It fails in t1001, t1020 & t4018. The reason for the last test failure is unknown right now. For the first two, it boils down to this,git init --bare bare-ancestor-aliased.git && cd bare-ancestor-aliased.git && echo "[alias] aliasedinit = init" >>config && mkdir plain-nested && cd plain-nested && git aliasedinit
Quoted 6 lines
Git tries to read .git/config relative to the current directory, and tries to resolve aliases from it. The problem is: if one tries to do this from a subdirectory inside the repo, .git/config is not the right path, and the alias lookup fails.
I'll investigate more later.
This fixes the first two tests (it should be squashed into your PATCH 1 regardless of the rename git_config -> git_config_raw):
commit 42315d10e21a1273b73671a3f8c9f7640c4feb44 (HEAD, config-v9)
Author: Matthieu Moy <[email protected]> Date: Thu Jul 17 13:12:21 2014 +0200
clear the config cache in setup_git_dir
setup.c +9 -0
@@ -625,6 +625,15 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)int one_filesystem =1 ;/* +* We may have read an incomplete configuration before +* setting-up the git directory. If so, clear the cache so +* that the next queries to the configuration reload complete +* configuration (including the per-repo config file that we +* ignored previously). +*/ +git_config_clear (); + +/* * Let's assume that we are in a git repository. * If it turns out later that we are somewhere else, the value will be * updated accordingly.
Signature · 2 lines
Matthieu Moy http://www-verimag.imag.fr/~moy/
Re: [PATCH v9r2 1/2] add `config_set` API for caching config-like files
Quoted 37 lines · Matthieu Moy
Quoted 14 lines · Matthieu Moy
Quoted 11 lines · Tanay Abhra
I have a question about renaming
git_config()togit_config_raw(). I was working on a patch and I am getting stuck here.
It fails in t1001, t1020 & t4018. The reason for the last test failure is unknown right now. For the first two, it boils down to this,git init --bare bare-ancestor-aliased.git && cd bare-ancestor-aliased.git && echo "[alias] aliasedinit = init" >>config && mkdir plain-nested && cd plain-nested && git aliasedinitQuoted 6 lines
Git tries to read .git/config relative to the current directory, and tries to resolve aliases from it. The problem is: if one tries to do this from a subdirectory inside the repo, .git/config is not the right path, and the alias lookup fails.
I'll investigate more later.
This fixes the first two tests (it should be squashed into your PATCH 1 regardless of the rename
git_config->git_config_raw):commit
42315d10e21a1273b73671a3f8c9f7640c4feb44(HEAD, config-v9)Author: Matthieu Moy <[email protected]> Date: Thu Jul 17 13:12:21 2014 +0200clear the config cache in
setup_git_dirsetup.c +0
-0
setup.c static const char *setup_git_directory_gently_1(int *nongit_ok) · line 634
Quoted 10 lines
setup.c +7
-0@@ -625,6 +625,15 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)int one_filesystem =1 ;/* +* We may have read an incomplete configuration before +* setting-up the git directory. If so, clear the cache so +* that the next queries to the configuration reload complete +* configuration (including the per-repo config file that we +* ignored previously). +*/ +git_config_clear ();
Very sensible.
Quoted 4 lines
setup.c +1
-0+/* * Let's assume that we are in a git repository. * If it turns out later that we are somewhere else, the value will be * updated accordingly.
PATCH 2/2 test-config: add tests for the config_set API
In reply to Matthieu MoyExpose the config_set C API as a set of simple commands in order to facilitate testing. Add tests for the config_set API as well as for git_config_get_*() family for the usual config files.
--- .gitignore | 1 + Makefile | 1 + t/t1308-config-set.sh | 200 ++++++++++++++++++++++++++++++++++++++++++++++++++ test-config.c | 140 +++++++++++++++++++++++++++++++++++ 4 files changed, 342 insertions(+) create mode 100755 t/t1308-config-set.sh create mode 100644 test-config.c
.gitignore +1 -0
@@ -178,6 +178,7 @@ /gitweb/static/gitweb.min.* /test-chmtime /test-ctype +/test-config /test-date /test-delta /test-dump-cache-tree
Makefile +1 -0
@@ -549,6 +549,7 @@ PROGRAMS += $(patsubst %.o,git-%$X,$(PROGRAM_OBJS)) TEST_PROGRAMS_NEED_X += test-chmtime TEST_PROGRAMS_NEED_X += test-ctype +TEST_PROGRAMS_NEED_X += test-config TEST_PROGRAMS_NEED_X += test-date TEST_PROGRAMS_NEED_X += test-delta TEST_PROGRAMS_NEED_X += test-dump-cache-tree
t/t1308-config-set.sh new file +200 -0
200 lines not on this page · raw message
test-config.c new file +140 -0
140 lines not on this page · raw message
Signature · 1 line
1.9.0.GIT