Skip to contentSkip to conversation

CLD: Introduce the "New CLD" API

PATCH Applied Pete Zaitcev 2 participants 17 messages Feb 7, 2010 – Feb 19, 2010 Library code

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 CLD: Introduce the "New CLD" API

The "traditional" CLD API is too difficult to program. In particular, switching from a rigid "Group" policy to arbitrary paths in existing clients in Chunk and tabled turned out to be next to impossible. The issue is due to the fundamental nature of the API as based on very fine-grained events.

So, in the interests of clients, introduce the "new" API. Its basic definition is in <ncld.h>. It presents a filesystem-like interface, extended with event callbacks for coarse events, such as an end of CLD session.

The patch converts all in-tree clients from cldc_xxx to ncld_xxx and discards some of code that became unused (test/util.c).

---
 include/Makefile.am    |    2 
 include/ncld.h         |   88 ++
 lib/cldc-dns.c         |   92 +-
 lib/cldc.c             |  856 ++++++++++++++++++++++++++
 test/Makefile.am       |    5 
 test/it-works.c        |  125 ---
 test/load-file-event.c |  244 +------
 test/lock-file-event.c |  269 +-------
 test/save-file-event.c |  245 -------
 test/test.h            |    5 
 test/util.c            |   79 --
 tools/cldcli.c         | 1249 ++++++++++-----------------------------
 12 files changed, 1433 insertions(+), 1826 deletions(-)
include/Makefile.am +1 -1
index 533f51d..cb82261 100644
@@ -1,5 +1,5 @@
 
 EXTRA_DIST = cld-private.h
 
-include_HEADERS = cldc.h hail_log.h cld_common.h
+include_HEADERS = cldc.h hail_log.h cld_common.h ncld.h
 
include/ncld.h new file +88 -0

88 lines not on this page · raw message

lib/cldc-dns.c +49 -43

191 lines not on this page · raw message

lib/cldc.c +854 -2

880 lines not on this page · raw message

test/Makefile.am +0 -5
index ff4d5c7..9e66458 100644
@@ -26,15 +26,10 @@ check_PROGRAMS		= it-works \
 			  save-file-event load-file-event lock-file-event
 
 TESTLDADD		= ../lib/libcldc.la	\
-			  libtest.a		\
 			  @GLIB_LIBS@ @CRYPTO_LIBS@
 it_works_LDADD		= $(TESTLDADD)
 save_file_event_LDADD	= $(TESTLDADD)
 load_file_event_LDADD	= $(TESTLDADD)
 lock_file_event_LDADD	= $(TESTLDADD)
 
-noinst_LIBRARIES	= libtest.a
-
-libtest_a_SOURCES	= util.c
-
 TESTS_ENVIRONMENT=top_srcdir=$(top_srcdir)
test/it-works.c +15 -110

142 lines not on this page · raw message

test/load-file-event.c +33 -211

279 lines not on this page · raw message

test/lock-file-event.c +37 -232

304 lines not on this page · raw message

test/save-file-event.c +28 -217

265 lines not on this page · raw message

test/test.h +0 -5
index a70609d..abd9284 100644
@@ -1,9 +1,6 @@
 #ifndef _CLD_TEST_H_
 #define _CLD_TEST_H_
 
-#include <stdbool.h>
-#include <cldc.h>
-
 #define TESTSTR          "longertestdata\n"
 #define TESTLEN  (sizeof("longertestdata\n")-1)
 
@@ -20,6 +17,4 @@
 
 #define TEST_PORTFILE_CLD	"cld.port"
 
-extern void test_loop(struct cld_timer_list *tlist, struct cldc_udp *udp);
-
 #endif
test/util.c deleted +0 -79

79 lines not on this page · raw message

tools/cldcli.c +328 -921

736 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Part of the diff is not shown here. View the raw message.

#1 lore raw
Quoted 11 lines · Pete Zaitcev

The "traditional" CLD API is too difficult to program. In particular, switching from a rigid "Group" policy to arbitrary paths in existing clients in Chunk and tabled turned out to be next to impossible. The issue is due to the fundamental nature of the API as based on very fine-grained events.

So, in the interests of clients, introduce the "new" API. Its basic definition is in<ncld.h>. It presents a filesystem-like interface, extended with event callbacks for coarse events, such as an end of CLD session.

Quoted 4 lines

The patch converts all in-tree clients from cldc_xxx to ncld_xxx and discards some of code that became unused (test/util.c).

At a quick glance, looks good. A few nits like the "if (log)" stuff and indenting struct field names jump out, but nothing major. I'll give it a thorough review and apply on Monday...

Jeff

Applied
#2 lore raw
In reply to Pete Zaitcev
Quoted 16 lines · Pete Zaitcev

The "traditional" CLD API is too difficult to program. In particular, switching from a rigid "Group" policy to arbitrary paths in existing clients in Chunk and tabled turned out to be next to impossible. The issue is due to the fundamental nature of the API as based on very fine-grained events.

So, in the interests of clients, introduce the "new" API. Its basic definition is in<ncld.h>. It presents a filesystem-like interface, extended with event callbacks for coarse events, such as an end of CLD session.

The patch converts all in-tree clients from cldc_xxx to ncld_xxx and discards some of code that became unused (test/util.c).

Quoted 14 lines
---
  include/Makefile.am    |    2
  include/ncld.h         |   88 ++
  lib/cldc-dns.c         |   92 +-
  lib/cldc.c             |  856 ++++++++++++++++++++++++++
  test/Makefile.am       |    5
  test/it-works.c        |  125 ---
  test/load-file-event.c |  244 +------
  test/lock-file-event.c |  269 +-------
  test/save-file-event.c |  245 -------
  test/test.h            |    5
  test/util.c            |   79 --
  tools/cldcli.c         | 1249 ++++++++++-----------------------------
  12 files changed, 1433 insertions(+), 1826 deletions(-)

applied, with changes:

- some cosmetic reformatting, to match existing code

- log param in cldc-dns should not be made conditional as an undocumented side effect of ncld

#3 lore raw
Quoted 2 lines · On Mon, 08 Feb 2010 07:32:56 -0500 Jeff Garzik <[email protected]> wrote:

- log param in cldc-dns should not be made conditional as an undocumented side effect of ncld

I thought it was compatible with the old clients, they can continue to pass the log in an argument as they did before.

BTW, I noticed you reworked the tests, which is fine, but could you also rename lock-file-event to lock-file? The idea was to have two
versions: one with libevent and one without, but that never happened.

-- Pete

#4 lore raw
Quoted 3 lines · Pete Zaitcev

BTW, I noticed you reworked the tests, which is fine, but could you also rename lock-file-event to lock-file? The idea was to have two
versions: one with libevent and one without, but that never happened.

Easy enough... done.

Jeff

#5 lore raw
In reply to Jeff Garzik
Quoted 2 lines · On Mon, 08 Feb 2010 07:32:56 -0500 Jeff Garzik <[email protected]> wrote:

- log param in cldc-dns should not be made conditional as an undocumented side effect of ncld

I pulled, and it appears that you merged a code that crashes at the first error message. We need something like this:

lib/cldc.c +11 -2
index 7c30064..beb1e22 100644
@@ -98,6 +98,15 @@ static void cldc_errlog(int prio, const char *fmt, ...)
 	va_end(ap);
 }
 
+static void ncld_nolog(int prio, const char *fmt, ...)
+{
+	;
+}
+
+static struct hail_log ncld_log = {
+	.func	= ncld_nolog
+};
+
 static int ack_seqid(struct cldc_session *sess, uint64_t seqid_le)
 {
 	XDR xdrs;
@@ -1297,7 +1306,7 @@ static int ncld_getsrv(char **hostp, unsigned short *portp)
 		return errno;
 	hostb[hostsz-1] = 0;
 
-	if (cldc_getaddr(&host_list, hostb, NULL))
+	if (cldc_getaddr(&host_list, hostb, &ncld_log))
 		return 1001;
 
 	/*
@@ -1490,7 +1499,7 @@ static struct cldc_ops ncld_ops = {
 	.timer_ctl	= ncld_p_timer_ctl,
 	.pkt_send	= ncld_p_pkt_send,
 	.event		= ncld_p_event,
-	.errlog		= NULL,
+	.errlog		= ncld_nolog,
 };
 
 static int ncld_new_sess(struct cldc_call_opts *copts, enum cle_err_codes errc)

Stubbing of ncld_ops is not related to the crash, but just for consistency.

-- Pete

#6 lore raw
Quoted 8 lines · Pete Zaitcev
Quoted 2 lines · On Mon, 08 Feb 2010 07:32:56 -0500 Jeff Garzik<[email protected]> wrote:

- log param in cldc-dns should not be made conditional as an undocumented side effect of ncld

I pulled, and it appears that you merged a code that crashes at the first error message. We need something like this:

So it successfully exposed locations that failed to propagate logging support. :)

Quoted 21 lines
lib/cldc.c +9 -0
index 7c30064..beb1e22 100644
@@ -98,6 +98,15 @@ static void cldc_errlog(int prio, const char *fmt, ...)
  	va_end(ap);
  }
 
+static void ncld_nolog(int prio, const char *fmt, ...)
+{
+	;
+}
+
+static struct hail_log ncld_log = {
+	.func	= ncld_nolog
+};
+
  static int ack_seqid(struct cldc_session *sess, uint64_t seqid_le)
  {
  	XDR xdrs;

lib/cldc.c static int ncld_getsrv(char **hostp, unsigned short *portp) · line 1311

Quoted 7 lines
lib/cldc.c +1 -1
@@ -1297,7 +1306,7 @@ static int ncld_getsrv(char **hostp, unsigned short *portp)
  		return errno;
  	hostb[hostsz-1] = 0;
 
-	if (cldc_getaddr(&host_list, hostb, NULL))
+	if (cldc_getaddr(&host_list, hostb,&ncld_log))
  		return 1001;
 

Logging pointer should be supplied by the caller... that is the larger bug. Needing a no-op log function is just an indication of that.

Jeff

#7 lore raw
Quoted 6 lines · On Mon, 08 Feb 2010 21:06:17 -0500 Jeff Garzik <[email protected]> wrote:

lib/cldc.c static int ncld_getsrv(char **hostp, unsigned short *portp) · line 1310

Quoted 3 lines
lib/cldc.c +1 -1
-	if (cldc_getaddr(&host_list, hostb, NULL))
+	if (cldc_getaddr(&host_list, hostb, &ncld_log))
  		return 1001;

Logging pointer should be supplied by the caller... that is the larger bug. Needing a no-op log function is just an indication of that.

I did it before, but then every client, without exception, needed to include <stdarg.h> and provide the log. And what for? This is STUPID and proof is in the pudding: your getsrvinfo does not do anything of the sort. Might as well replace cldc_getadd with getsrvinfo and then everyone becomes happy.

-- Pete

#8 lore raw
Quoted 10 lines · Pete Zaitcev
Quoted 6 lines · On Mon, 08 Feb 2010 21:06:17 -0500 Jeff Garzik<[email protected]> wrote:

lib/cldc.c static int ncld_getsrv(char **hostp, unsigned short *portp) · line 1310

Quoted 3 lines
lib/cldc.c +1 -1
-	if (cldc_getaddr(&host_list, hostb, NULL))
+	if (cldc_getaddr(&host_list, hostb,&ncld_log))
   		return 1001;

Logging pointer should be supplied by the caller... that is the larger bug. Needing a no-op log function is just an indication of that.

Quoted 4 lines

I did it before, but then every client, without exception, needed to include<stdarg.h> and provide the log. And what for? This is STUPID and proof is in the pudding: your getsrvinfo does not do anything of the sort.

Possibly true -- but it is worse to introduce internal inconsistencies.
It is far better to remove debugging statements completely than to
create inconsistent usage.

Quoted 2 lines

Might as well replace cldc_getadd with getsrvinfo
and then everyone becomes happy.

Seems a good fit :)

Jeff

#9 lore raw
Quoted 7 lines · On Mon, 08 Feb 2010 23:55:03 -0500 Jeff Garzik <[email protected]> wrote:
Quoted 7 lines
Quoted 6 lines

lib/cldc.c static int ncld_getsrv(char **hostp, unsigned short *portp) · line 1310

Quoted 3 lines
lib/cldc.c +1 -1
-	if (cldc_getaddr(&host_list, hostb, NULL))
+	if (cldc_getaddr(&host_list, hostb,&ncld_log))
   		return 1001;

Logging pointer should be supplied by the caller... that is the larger bug. Needing a no-op log function is just an indication of that.

Quoted 8 lines
Quoted 4 lines

I did it before, but then every client, without exception, needed to include<stdarg.h> and provide the log. And what for? This is STUPID and proof is in the pudding: your getsrvinfo does not do anything of the sort.

Possibly true -- but it is worse to introduce internal inconsistencies.
It is far better to remove debugging statements completely than to
create inconsistent usage.

Hmm, you may be right about that.

Quoted 4 lines
Quoted 2 lines

Might as well replace cldc_getadd with getsrvinfo
and then everyone becomes happy.

Seems a good fit :)

How far along were you with getsrvinfo? I only have the header that you sent to the list just before Christmas.

-- Pete

#10 lore raw
Quoted 21 lines · Pete Zaitcev
Quoted 7 lines · On Mon, 08 Feb 2010 23:55:03 -0500 Jeff Garzik<[email protected]> wrote:
Quoted 7 lines
Quoted 6 lines

lib/cldc.c static int ncld_getsrv(char **hostp, unsigned short *portp) · line 1310

Quoted 3 lines
lib/cldc.c +1 -1
-	if (cldc_getaddr(&host_list, hostb, NULL))
+	if (cldc_getaddr(&host_list, hostb,&ncld_log))
    		return 1001;

Logging pointer should be supplied by the caller... that is the larger bug. Needing a no-op log function is just an indication of that.

Quoted 8 lines
Quoted 4 lines

I did it before, but then every client, without exception, needed
to include<stdarg.h> and provide the log. And what for? This is
STUPID and proof is in the pudding: your getsrvinfo does not do anything of the sort.

Possibly true -- but it is worse to introduce internal inconsistencies.
It is far better to remove debugging statements completely than to
create inconsistent usage.

Hmm, you may be right about that.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

Might as well replace cldc_getadd with getsrvinfo
and then everyone becomes happy.

Seems a good fit :)

How far along were you with getsrvinfo? I only have the header that you sent to the list just before Christmas.

Coding complete, but needed testing. Probably works, since it came from working code. See attached.

Jeff

#ifndef __GETSRVINFO_H__
#define __GETSRVINFO_H__

#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>

struct srvinfo;

enum srvinfo_error_codes {
ESI_NONE = 0, /* no error */
ESI_CORRUPT = 1, /* server returned bad data */
ESI_FAIL = 2, /* server returned permanent failure */
ESI_AGAIN = 3, /* server returned temporary failure;
* try again later.*/
ESI_OOM = 4, /* internal memory alloc failed */
ESI_INVAL = 5, /* invalid argument(s) */
};

enum srvinfo_flags {
	FSI_NO_ADDR		= (1U << 0),  /* skip host->addrs lookup */
};

struct srvinfo {
unsigned int si_prio; /* SRV priority */
unsigned int si_weight; /* SRV weight */
char *si_target; /* SRV target domainname */
unsigned short si_port; /* SRV port */

struct addrinfo *si_addr; /* addresses returned
* from getaddrinfo(3) lookup
* on si_target */
struct srvinfo *si_next; /* next srvinfo in result list */
};

extern int getsrvinfo(const char *service_name, const char *domain_name,

		      const struct addrinfo *hints,
		      unsigned int flags, struct srvinfo **res);

extern void freesrvinfo(struct srvinfo *res);
extern const char *gsi_strerror(int errcode);

#endif /* __GETSRVINFO_H__ */

/*
* Copyright 2009 Red Hat, Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; see the file COPYING. If not, write to
* the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA. *
*/

#include <sys/types.h>
#include <sys/socket.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <netinet/in.h>
#include <arpa/nameser.h>
#include <netdb.h>
#include <resolv.h>
#include "getsrvinfo.h"

#define __must_be_array(a) \
	(__builtin_types_compatible_p(typeof(a), typeof(&a[0])))

#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]) + __must_be_array(arr))

enum {
	gsi_max_dom_name_sz		= 64,
	gsi_dns_buf_sz			= 1024,
};

static int fill_srvinfo(struct srvinfo *si,
const struct addrinfo *hints, unsigned int gsi_flags, unsigned int priority, unsigned int weight, unsigned int port, unsigned int nlen, const char *name)

{
	char portstr[11];
	char *hostname;
	struct addrinfo *res0 = NULL;
	int rc = 0;

	sprintf(portstr, "%u", port);

	hostname = malloc(nlen + 1);
	if (!hostname)
		return -ENOMEM;

	memcpy(hostname, name, nlen);
	hostname[nlen] = 0;

	if (!(gsi_flags & FSI_NO_ADDR)) {
		rc = getaddrinfo(hostname, portstr, hints, &res0);

		/* because we do not wish failure of one lookup
		 * to cause failure of the entire operation,
		 * we simply record the lack of an address (NULL)
		 * and move on.
		 */
		if (rc)
			res0 = NULL;
	}

	si->si_prio = priority;
	si->si_weight = weight;
	si->si_target = hostname;
	si->si_port = port;

	si->si_addr = res0;
	si->si_next = NULL;

	return 0;
}

void freesrvinfo(struct srvinfo *res)

{
	while (res) {
		struct srvinfo *tmp;
		
		tmp = res;
		res = res->si_next;

		if (tmp->si_addr)
			freeaddrinfo(tmp->si_addr);
		free(tmp->si_target);
		free(tmp);
	}
}

int getsrvinfo(const char *service_name, const char *domain_name,
const struct addrinfo *hints, unsigned int gsi_flags, struct srvinfo **res_out)

{
	unsigned char resp[gsi_dns_buf_sz];
	int rlen;
	ns_msg nsb;
	ns_rr rrb;
	int rrlen;
	char hostb[gsi_max_dom_name_sz];
	struct srvinfo *si, *res = NULL, *res_last = NULL;
	const unsigned char *p;
	int rc, gsi_rc, i;

	if (!domain_name || !res_out)
		return ESI_INVAL;

	*res_out = NULL;

	/* we concatencate service_name and domain_name as a helpful
	 * service for the caller, because it is very common
	 * that service_name is either completely static, or at least
	 * stored in a separate variable from domain_name.
	 */
	if (service_name) {
		char *name;
		size_t name_len;
		int has_dot;

		has_dot = (service_name[strlen(service_name) - 1] == '.');
		name_len = strlen(service_name) + strlen(domain_name) +
			   (has_dot ? 0 : 1) + 1;

		name = malloc(name_len);
		if (!name)
			return ESI_OOM;

		snprintf(name, name_len, "%s%s%s", service_name,
			     has_dot ? "" : ".",
			     domain_name);
			
		rc = res_search(name, ns_c_in, ns_t_srv, resp, sizeof(resp));

		free(name);
	} else {
		rc = res_search(domain_name, ns_c_in, ns_t_srv,
				resp, sizeof(resp));
	}

	/* parse resolver return value */
	if (rc < 0) {
		switch (h_errno) {
		case TRY_AGAIN:
			return ESI_AGAIN;
		case HOST_NOT_FOUND:
		case NO_DATA:
		case NO_RECOVERY:
		default:
			return ESI_FAIL;
		}
	}
	rlen = rc;

	if (rlen == 0)
		return ESI_FAIL;

	/* set up DNS result parse */
	if (ns_initparse(resp, rlen, &nsb) < 0)
		return ESI_CORRUPT;

	/* iterate through each answer.  Because DNS packets may
	 * be truncated, we do not signal an error on
	 * short-length faults found during packet parsing
	 */
	for (i = 0; i < ns_msg_count(nsb, ns_s_an); i++) {
		rc = ns_parserr(&nsb, ns_s_an, i, &rrb);
		if (rc < 0)
			continue;

		if (ns_rr_class(rrb) != ns_c_in)
			continue;

		switch (ns_rr_type(rrb)) {
		case ns_t_srv:
			rrlen = ns_rr_rdlen(rrb);
			if (rrlen < 8) {	/* 2+2+2 and 2 for host */
				break;
			}
			p = ns_rr_rdata(rrb);
			rc = dn_expand(resp, resp+rlen, p+6,
				       hostb, gsi_max_dom_name_sz);
			if (rc < 0) {
				break;
			}
			if (rc < 2) {
				break;
			}

			si = malloc(sizeof(*si));
			if (!si) {
				gsi_rc = ESI_OOM;
				goto err_out;
			}

			if (fill_srvinfo(si, hints, gsi_flags,
					 ns_get16(p+0),
					 ns_get16(p+2),
					 ns_get16(p+4),
					 rc, hostb)) {
				free(si);
				gsi_rc = ESI_OOM;
				goto err_out;
			}

			/* if first item, set BOL-ptr */
			if (!res)
				res = si;

			/* append to EOL */
			if (res_last)
				res_last->si_next = si;
			res_last = si;

			break;

		case ns_t_cname:	/* impossible, but ... ? */
		default:
			break;
		}
	}

	*res_out = res;
	return ESI_NONE;

err_out:

	freesrvinfo(res);
	return gsi_rc;
}

static const char *gsi_error_str[] = {
[ESI_NONE] = "no error",
[ESI_CORRUPT] = "server returned bad data",
[ESI_FAIL] = "server returned permanent failure",
[ESI_AGAIN] = "server returned temporary failure",
[ESI_OOM] = "internal memory alloc failed",
[ESI_INVAL] = "invalid argument(s)",
};

const char *gsi_strerror(int errcode)

{
	if (errcode < 0 || errcode >= ARRAY_SIZE(gsi_error_str))
		return NULL;
	
	return gsi_error_str[errcode];
}
#11 lore raw
In reply to Pete Zaitcev
Quoted 11 lines · Pete Zaitcev
tools/cldcli.c +11 -0
+	case CREQ_MKDIR:
+		rc = 0;
+		fhp = ncld_open(nsp, creq.path,
+				COM_DIRECTORY | COM_CREATE | COM_EXCL,&error,
+				0, NULL, NULL);
+		if (fhp)
+			ncld_close(fhp);
+		else
+			rc = error;
+		break;
+

[...]

Quoted 5 lines
tools/cldcli.c +2 -3
-	}
-	if (tcode != TC_OK) {
-		fprintf(stderr, "thread startup failed\n");
+	nsp = ncld_sess_open(dr->host, dr->port,&error, sess_event, NULL,
+			     "cldcli", "cldcli");

tools/cldcli.c int main (int argc, char *argv[]) · line 724

Quoted 10 lines
tools/cldcli.c +8 -0
+	if (!nsp) {
+		if (error<  1000) {
+			fprintf(stderr, TAG ": cannot open CLD session: %s\n",
+				strerror(error));
+		} else {
+			fprintf(stderr, TAG ": cannot open CLD session: %d\n",
+				error);
+		}
  		return 1;
  	}

A point of style: the 'p' suffix is discouraged. It is too much like the much-maligned Hungarian notation, which is odious in part because it duplicates information found in the type definition. In this case, 'p' duplicates the indication that it is a pointer.

The return value from ncld_sess_open() should be named 'ns' or 'sess' or similar. The return value from ncld_open() should be named 'fh' or similar.

Jeff

#12 lore raw
Quoted 1 line · On Sat, 13 Feb 2010 19:52:18 -0500 Jeff Garzik <[email protected]> wrote:

A point of style: the 'p' suffix is discouraged.

I'll batch-rename it when I get better.

-- Pete

#13 lore raw
Quoted 6 lines · Pete Zaitcev
Quoted 1 line · On Sat, 13 Feb 2010 19:52:18 -0500 Jeff Garzik<[email protected]> wrote:

A point of style: the 'p' suffix is discouraged.

I'll batch-rename it when I get better.

Get well soon...

Since it sounds like there will be no conflicting patches, I went ahead and fixed things up myself, using my trusty sar (search-and-replace) script. I find sar quite useful, because it (a) matches and replaces on word boundaries, and (b) it performs in-situ updates, unlike most other Unix tools.

Attached are both sar, and the cld commit it helped produce.

Jeff

#!/usr/bin/perl -w

use strict;

my ($search, $replace, $fn, @data, $mod);
my $modified = 0;

die "usage: sar search-text replace-text files..."

	unless $search = shift;
die "usage: sar search-text replace-text files..."
	unless $replace = shift;
foreach $fn (@ARGV) {
	@data = ();
	$mod = 0;

	open(F, $fn) or die "$fn: $!\n";
	while (<F>) {
		if ($mod) {
			s/\b$search\b/$replace/go;
		}
		elsif (/\b$search\b/o) {
			$mod = 1;
			s/\b$search\b/$replace/go;
		}
		push(@data, $_);
	}
	close(F);

	if ($mod) {
		open(F, ">$fn") or die "$fn: $!\n";
		print F @data;
		close(F);
		$modified++;
	}
}

printf "%d files modified.\n", $modified;
exit(0);

commit 1c7d8de3e44e641651f766d6863780343ecaa5a8

Author: Jeff Garzik <[email protected]>
Date:   Wed Feb 17 20:14:04 2010 -0500

Cosmetic renaming related to ncld API.

s/fhp/fh/
s/ses/sess/
s/nsp/nsess/

And use a more standard way of calling callbacks:

- if (nsp->event)
- (*nsp->event)(nsp->event_arg, what);
+ if (nsess->event)
+ nsess->event(nsess->event_arg, what);

include/ncld.h +3 -3
index 4664299..dbf126a 100644
@@ -47,7 +47,7 @@ struct ncld_sess {
 };
 
 struct ncld_fh {
-	struct ncld_sess	*ses;
+	struct ncld_sess	*sess;
 	struct cldc_fh		*fh;	/* FIXME cldc_open2 take direct & */
 	bool			is_open;
 	int			errc;
@@ -76,8 +76,8 @@ extern struct ncld_sess *ncld_sess_open(const char *host, int port,
 extern struct ncld_fh *ncld_open(struct ncld_sess *s, const char *fname,
 	unsigned int mode, int *error, unsigned int events,
 	void (*event)(void *, unsigned int), void *ev_arg);
-extern int ncld_del(struct ncld_sess *nsp, const char *fname);
-extern struct ncld_read *ncld_get(struct ncld_fh *fhp, int *error);
+extern int ncld_del(struct ncld_sess *nsess, const char *fname);
+extern struct ncld_read *ncld_get(struct ncld_fh *fh, int *error);
 extern struct ncld_read *ncld_get_meta(struct ncld_fh *fh, int *error);
 extern void ncld_read_free(struct ncld_read *rp);
 extern int ncld_write(struct ncld_fh *, const void *data, long len);
lib/cldc.c +220 -220

906 lines not on this page · raw message

test/basic-io.c +18 -18
index 2565280..31bfcab 100644
@@ -35,59 +35,59 @@
 
 static int test_write(int port)
 {
-	struct ncld_sess *nsp;
-	struct ncld_fh *fhp;
+	struct ncld_sess *nsess;
+	struct ncld_fh *fh;
 	int error;
 
-	nsp = ncld_sess_open(TEST_HOST, port, &error, NULL, NULL,
+	nsess = ncld_sess_open(TEST_HOST, port, &error, NULL, NULL,
 			     TEST_USER, TEST_USER_KEY);
-	if (!nsp) {
+	if (!nsess) {
 		fprintf(stderr, "ncld_sess_open(host %s port %u) failed: %d\n",
 			TEST_HOST, port, error);
 		exit(1);
 	}
 
-	fhp = ncld_open(nsp, TFNAME, COM_WRITE | COM_CREATE,
+	fh = ncld_open(nsess, TFNAME, COM_WRITE | COM_CREATE,
 			&error, 0, NULL, NULL);
-	if (!fhp) {
+	if (!fh) {
 		fprintf(stderr, "ncld_open(%s) failed: %d\n", TFNAME, error);
 		exit(1);
 	}
 
-	error = ncld_write(fhp, TESTSTR, TESTLEN);
+	error = ncld_write(fh, TESTSTR, TESTLEN);
 	if (error) {
 		fprintf(stderr, "ncld_write failed: %d\n", error);
 		exit(1);
 	}
 
 	/* These two are perfect places to hang or crash, so don't just exit. */
-	ncld_close(fhp);
-	ncld_sess_close(nsp);
+	ncld_close(fh);
+	ncld_sess_close(nsess);
 	return 0;
 }
 
 static int test_read(int port)
 {
-	struct ncld_sess *nsp;
-	struct ncld_fh *fhp;
+	struct ncld_sess *nsess;
+	struct ncld_fh *fh;
 	struct ncld_read *rp;
 	int error;
 
-	nsp = ncld_sess_open(TEST_HOST, port, &error, NULL, NULL,
+	nsess = ncld_sess_open(TEST_HOST, port, &error, NULL, NULL,
 			     TEST_USER, TEST_USER_KEY);
-	if (!nsp) {
+	if (!nsess) {
 		fprintf(stderr, "ncld_sess_open(host %s port %u) failed: %d\n",
 			TEST_HOST, port, error);
 		exit(1);
 	}
 
-	fhp = ncld_open(nsp, TFNAME, COM_READ, &error, 0, NULL, NULL);
-	if (!fhp) {
+	fh = ncld_open(nsess, TFNAME, COM_READ, &error, 0, NULL, NULL);
+	if (!fh) {
 		fprintf(stderr, "ncld_open(%s) failed: %d\n", TFNAME, error);
 		exit(1);
 	}
 
-	rp = ncld_get(fhp, &error);
+	rp = ncld_get(fh, &error);
 	if (!rp) {
 		fprintf(stderr, "ncld_get failed: %d\n", error);
 		exit(1);
@@ -106,8 +106,8 @@ static int test_read(int port)
 	ncld_read_free(rp);
 
 	/* These two are perfect places to hang or crash, so don't just exit. */
-	ncld_close(fhp);
-	ncld_sess_close(nsp);
+	ncld_close(fh);
+	ncld_sess_close(nsess);
 	return 0;
 }
 
test/basic-session.c +4 -4
index d8f9ec1..6caf046 100644
@@ -31,7 +31,7 @@
 
 int main (int argc, char *argv[])
 {
-	struct ncld_sess *nsp;
+	struct ncld_sess *nsess;
 	int error;
 	int port;
 
@@ -44,15 +44,15 @@ int main (int argc, char *argv[])
 	if (port == 0)
 		return -1;
 
-	nsp = ncld_sess_open(TEST_HOST, port, &error, NULL, NULL,
+	nsess = ncld_sess_open(TEST_HOST, port, &error, NULL, NULL,
 			     TEST_USER, TEST_USER_KEY);
-	if (!nsp) {
+	if (!nsess) {
 		fprintf(stderr, "ncld_sess_open(host %s port %u) failed: %d\n",
 			TEST_HOST, port, error);
 		exit(1);
 	}
 
-	ncld_sess_close(nsp);
+	ncld_sess_close(nsess);
 	return 0;
 }
 
test/lock-file.c +11 -11
index 64e6020..d0ea9c1 100644
@@ -33,8 +33,8 @@
 
 int main (int argc, char *argv[])
 {
-	struct ncld_sess *nsp;
-	struct ncld_fh *fhp;
+	struct ncld_sess *nsess;
+	struct ncld_fh *fh;
 	int port;
 	struct timespec tm;
 	int error;
@@ -49,28 +49,28 @@ int main (int argc, char *argv[])
 	if (port == 0)
 		return -1;
 
-	nsp = ncld_sess_open(TEST_HOST, port, &error, NULL, NULL,
+	nsess = ncld_sess_open(TEST_HOST, port, &error, NULL, NULL,
 			     TEST_USER, TEST_USER_KEY);
-	if (!nsp) {
+	if (!nsess) {
 		fprintf(stderr, "ncld_sess_open(host %s port %u) failed: %d\n",
 			TEST_HOST, port, error);
 		exit(1);
 	}
 
-	fhp = ncld_open(nsp, TLNAME, COM_WRITE | COM_LOCK | COM_CREATE,
+	fh = ncld_open(nsess, TLNAME, COM_WRITE | COM_LOCK | COM_CREATE,
 			&error, 0, NULL, NULL);
-	if (!fhp) {
+	if (!fh) {
 		fprintf(stderr, "ncld_open(%s) failed: %d\n", TLNAME, error);
 		exit(1);
 	}
 
-	rc = ncld_write(fhp, LOCKSTR, LOCKLEN);
+	rc = ncld_write(fh, LOCKSTR, LOCKLEN);
 	if (rc) {
 		fprintf(stderr, "ncld_write failed: %d\n", rc);
 		exit(1);
 	}
 
-	rc = ncld_trylock(fhp);
+	rc = ncld_trylock(fh);
 	if (rc) {
 		fprintf(stderr, "ncld_trylock failed: %d\n", rc);
 		exit(1);
@@ -82,15 +82,15 @@ int main (int argc, char *argv[])
 	tm.tv_nsec = 0;
 	nanosleep(&tm, NULL);
 
-	rc = ncld_unlock(fhp);
+	rc = ncld_unlock(fh);
 	if (rc) {
 		fprintf(stderr, "ncld_unlock failed: %d\n", rc);
 		exit(1);
 	}
 
 	/* These two are perfect places to hang or crash, so don't just exit. */
-	ncld_close(fhp);
-	ncld_sess_close(nsp);
+	ncld_close(fh);
+	ncld_sess_close(nsess);
 	return 0;
 }
 
tools/cldcli.c +45 -45

276 lines not on this page · raw message

#14 lore raw
Quoted 9 lines · On Wed, 17 Feb 2010 20:18:52 -0500 Jeff Garzik <[email protected]> wrote:
Quoted 3 lines
Quoted 1 line

A point of style: the 'p' suffix is discouraged.

I'll batch-rename it when I get better.

Since it sounds like there will be no conflicting patches, I went ahead and fixed things up myself, using my trusty sar (search-and-replace) script. I find sar quite useful, because it (a) matches and replaces on word boundaries, and (b) it performs in-situ updates, unlike most other Unix tools.

I imagine the numbers of '>>>>>>' clauses I have to fix after the next "git pull". Oh well.

-- Pete

#15 lore raw
Quoted 7 lines · Pete Zaitcev
Quoted 3 lines · On Wed, 17 Feb 2010 20:18:52 -0500 Jeff Garzik<[email protected]> wrote:
Quoted 3 lines
Quoted 1 line

A point of style: the 'p' suffix is discouraged.

I'll batch-rename it when I get better.

Quoted 8 lines
Quoted 5 lines

Since it sounds like there will be no conflicting patches, I went ahead and fixed things up myself, using my trusty sar (search-and-replace) script. I find sar quite useful, because it (a) matches and replaces on word boundaries, and (b) it performs in-situ updates, unlike most other Unix tools.

I imagine the numbers of '>>>>>>' clauses I have to fix after the next "git pull". Oh well.

If you perform the substitutions on your local working tree using the sar script, the diff between git HEAD and your local working tree should provide something manageable.

That's how to I split up Colin's XDR patch into multiple patches... repeatedly diff between HEAD and XDR-everything trees, apply a subset of the total diff, a single logical change. lather rinse repeat.

Jeff

#16 lore raw
Quoted 6 lines · On Thu, 18 Feb 2010 21:41:35 -0500 Jeff Garzik <[email protected]> wrote:
Quoted 2 lines

I imagine the numbers of '>>>>>>' clauses I have to fix after the next "git pull". Oh well.

If you perform the substitutions on your local working tree using the sar script, the diff between git HEAD and your local working tree should provide something manageable.

Yep, I paniced too early. One :%s/\<nsp\>/nsess/g was sufficient.

-- Pete