Skip to contentSkip to conversation

Kernel Access to Ftrace Instances.

RFC In review Divya Indi 2 participants 8 messages Jun 12, 2019 – Jun 19, 2019 Tracing

lore
Share & tools
lore

Divya Indi

author
#0 lore raw

Hi,
Please review the patches that follow -

[PATCH 1/3] tracing: Relevant changes for kernel access to Ftrace instances. [PATCH 2/3] tracing: Adding additional NULL checks.
[PATCH 3/3] tracing: Add 2 new funcs. for kernel access to Ftrace instances.

This is v2 for the series with changes made to Patch 3/3 to address review comments.

The new changes ensure that a trace array created by trace_array_create or accessed via trace_array_lookup cannot be freed if still in use.

Let me know if you have any questions/concerns/suggestions.

Thanks,
Divya

author
#1 lore raw

PATCH 1/3 tracing: Relevant changes for kernel access to Ftrace instances.

For commit f45d1225adb0 ("tracing: Kernel access to Ftrace instances") Adding the following changes to ensure other kernel components can use these functions -
1) Remove static keyword for newly exported fn - ftrace_set_clr_event.
2) Add the req functions to header file include/linux/trace_events.h.

---
 include/linux/trace_events.h | 6 ++++++
 kernel/trace/trace_events.c  | 2 +-
 2 files changed, 7 insertions(+), 1 deletion(-)
include/linux/trace_events.h +6 -0
index 8a62731..d7b7d85 100644
@@ -539,6 +539,12 @@ extern int trace_define_field(struct trace_event_call *call, const char *type,
 
 #define is_signed_type(type)	(((type)(-1)) < (type)1)
 
+void trace_printk_init_buffers(void);
+int trace_array_printk(struct trace_array *tr, unsigned long ip,
+		const char *fmt, ...);
+struct trace_array *trace_array_create(const char *name);
+int trace_array_destroy(struct trace_array *tr);
+int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set);
 int trace_set_clr_event(const char *system, const char *event, int set);
 
 /*
kernel/trace/trace_events.c +1 -1
index 0ce3db6..b6b4618 100644
@@ -795,7 +795,7 @@ static int __ftrace_set_clr_event(struct trace_array *tr, const char *match,
 	return ret;
 }
 
-static int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set)
+int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set)
 {
 	char *event = NULL, *sub = NULL, *match;
 	int ret;
Signature · 1 line
1.8.3.1
author
#2 lore raw

PATCH 2/3 tracing: Adding additional NULL checks.

commit f45d1225adb0 ("tracing: Kernel access to Ftrace instances") exported certain functions providing access to Ftrace instances from other kernel components.
Adding some additional NULL checks to ensure safe usage by the users.

---
 kernel/trace/trace.c        | 3 +++
 kernel/trace/trace_events.c | 2 ++
 2 files changed, 5 insertions(+)
kernel/trace/trace.c +3 -0
index 1c80521..a60dc13 100644
@@ -3205,6 +3205,9 @@ int trace_array_printk(struct trace_array *tr,
 	if (!(global_trace.trace_flags & TRACE_ITER_PRINTK))
 		return 0;
 
+	if (!tr)
+		return -EINVAL;
+
 	va_start(ap, fmt);
 	ret = trace_array_vprintk(tr, ip, fmt, ap);
 	va_end(ap);
kernel/trace/trace_events.c +2 -0
index b6b4618..445b059 100644
@@ -800,6 +800,8 @@ int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set)
 	char *event = NULL, *sub = NULL, *match;
 	int ret;
 
+	if (!tr)
+		return -ENODEV;
 	/*
 	 * The buf format can be <subsystem>:<event-name>
 	 *  *:<event-name> means any event by that name.
Signature · 1 line
1.8.3.1
author
#3 lore raw

PATCH 3/3 tracing: Add 2 new funcs. for kernel access to Ftrace instances.

Adding 2 new functions -
1) trace_array_lookup : Look up and return a trace array, given its name.
2) trace_array_set_clr_event : Enable/disable event recording to the given trace array.

Newly added functions trace_array_lookup and trace_array_create also need to increment the reference counter associated with the trace array they return. This is to ensure the trace array does not get freed while in use by the newly introduced APIs.
The reference ctr is decremented in the trace_array_destroy.

---
 include/linux/trace_events.h |  3 +++
 kernel/trace/trace.c         | 30 +++++++++++++++++++++++++++++-
 kernel/trace/trace_events.c  | 22 ++++++++++++++++++++++
 3 files changed, 54 insertions(+), 1 deletion(-)
include/linux/trace_events.h +3 -0
index d7b7d85..0cc99a8 100644
@@ -545,7 +545,10 @@ int trace_array_printk(struct trace_array *tr, unsigned long ip,
 struct trace_array *trace_array_create(const char *name);
 int trace_array_destroy(struct trace_array *tr);
 int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set);
+struct trace_array *trace_array_lookup(const char *name);
 int trace_set_clr_event(const char *system, const char *event, int set);
+int trace_array_set_clr_event(struct trace_array *tr, const char *system,
+		const char *event, int set);
 
 /*
  * The double __builtin_constant_p is because gcc will give us an error
kernel/trace/trace.c +29 -1
index a60dc13..fb70ccc 100644
@@ -8364,6 +8364,8 @@ struct trace_array *trace_array_create(const char *name)
 
 	list_add(&tr->list, &ftrace_trace_arrays);
 
+	tr->ref++;
+
 	mutex_unlock(&trace_types_lock);
 	mutex_unlock(&event_mutex);
 
@@ -8385,7 +8387,14 @@ struct trace_array *trace_array_create(const char *name)
 
 static int instance_mkdir(const char *name)
 {
-	return PTR_ERR_OR_ZERO(trace_array_create(name));
+	struct trace_array *tr;
+
+	tr = trace_array_create(name);
+	if (IS_ERR(tr))
+		return PTR_ERR(tr);
+	trace_array_put(tr);
+
+	return 0;
 }
 
 static int __remove_instance(struct trace_array *tr)
@@ -8434,6 +8443,7 @@ int trace_array_destroy(struct trace_array *tr)
 	mutex_lock(&event_mutex);
 	mutex_lock(&trace_types_lock);
 
+	tr->ref--;
 	ret = __remove_instance(tr);
 
 	mutex_unlock(&trace_types_lock);
@@ -8465,6 +8475,24 @@ static int instance_rmdir(const char *name)
 	return ret;
 }
 
+struct trace_array *trace_array_lookup(const char *name)
+{
+	struct trace_array *tr;
+
+	mutex_lock(&trace_types_lock);
+
+	list_for_each_entry(tr, &ftrace_trace_arrays, list) {
+		if (tr->name && strcmp(tr->name, name) == 0) {
+			tr->ref++;
+			mutex_unlock(&trace_types_lock);
+			return tr;
+		}
+	}
+	mutex_unlock(&trace_types_lock);
+	return NULL;
+}
+EXPORT_SYMBOL_GPL(trace_array_lookup);
+
 static __init void create_trace_instances(struct dentry *d_tracer)
 {
 	trace_instance_dir = tracefs_create_instance_dir("instances", d_tracer,
kernel/trace/trace_events.c +22 -0
index 445b059..c126d2c 100644
@@ -859,6 +859,28 @@ int trace_set_clr_event(const char *system, const char *event, int set)
 }
 EXPORT_SYMBOL_GPL(trace_set_clr_event);
 
+/**
+ * trace_array_set_clr_event - enable or disable an event for a trace array
+ * @system: system name to match (NULL for any system)
+ * @event: event name to match (NULL for all events, within system)
+ * @set: 1 to enable, 0 to disable
+ *
+ * This is a way for other parts of the kernel to enable or disable
+ * event recording to instances.
+ *
+ * Returns 0 on success, -EINVAL if the parameters do not match any
+ * registered events.
+ */
+int trace_array_set_clr_event(struct trace_array *tr, const char *system,
+		const char *event, int set)
+{
+	if (!tr)
+		return -ENODEV;
+
+	return __ftrace_set_clr_event(tr, NULL, system, event, set);
+}
+EXPORT_SYMBOL_GPL(trace_array_set_clr_event);
+
 /* 128 should be much more than enough */
 #define EVENT_BUF_SIZE		127
 
Signature · 1 line
1.8.3.1
maintainer
#4 lore raw

Re: [PATCH 3/3] tracing: Add 2 new funcs. for kernel access to Ftrace instances.

Quoted 39 lines · On Wed, 12 Jun 2019 09:34:19 -0700 Divya Indi <[email protected]> wrote:

Adding 2 new functions -
1) trace_array_lookup : Look up and return a trace array, given its name.
2) trace_array_set_clr_event : Enable/disable event recording to the given trace array.

Newly added functions trace_array_lookup and trace_array_create also need to increment the reference counter associated with the trace array they return. This is to ensure the trace array does not get freed while in use by the newly introduced APIs.
The reference ctr is decremented in the trace_array_destroy.

---
 include/linux/trace_events.h |  3 +++
 kernel/trace/trace.c         | 30 +++++++++++++++++++++++++++++-
 kernel/trace/trace_events.c  | 22 ++++++++++++++++++++++
 3 files changed, 54 insertions(+), 1 deletion(-)
include/linux/trace_events.h +3 -0
index d7b7d85..0cc99a8 100644
@@ -545,7 +545,10 @@ int trace_array_printk(struct trace_array *tr, unsigned long ip,
 struct trace_array *trace_array_create(const char *name);
 int trace_array_destroy(struct trace_array *tr);
 int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set);
+struct trace_array *trace_array_lookup(const char *name);
 int trace_set_clr_event(const char *system, const char *event, int set);
+int trace_array_set_clr_event(struct trace_array *tr, const char *system,
+		const char *event, int set);
 
 /*
  * The double __builtin_constant_p is because gcc will give us an error
kernel/trace/trace.c +0 -0
index a60dc13..fb70ccc 100644

kernel/trace/trace.c struct trace_array *trace_array_create(const char *name) · line 8368

Quoted 5 lines
kernel/trace/trace.c +2 -0
@@ -8364,6 +8364,8 @@ struct trace_array *trace_array_create(const char *name)
 
 	list_add(&tr->list, &ftrace_trace_arrays);
 
+	tr->ref++;
+

Needs a comment for why the ref is incremented.

Quoted 4 lines
kernel/trace/trace.c +0 -0
 	mutex_unlock(&trace_types_lock);
 	mutex_unlock(&event_mutex);
 

kernel/trace/trace.c struct trace_array *trace_array_create(const char *name) · line 8395

Quoted 10 lines
kernel/trace/trace.c +6 -1
@@ -8385,7 +8387,14 @@ struct trace_array *trace_array_create(const char *name)
 
 static int instance_mkdir(const char *name)
 {
-	return PTR_ERR_OR_ZERO(trace_array_create(name));
+	struct trace_array *tr;
+
+	tr = trace_array_create(name);
+	if (IS_ERR(tr))
+		return PTR_ERR(tr);
+	trace_array_put(tr);

Need a comment to why we need to do a put here.

Quoted 14 lines
kernel/trace/trace.c +3 -0
+
+	return 0;
 }
 
 static int __remove_instance(struct trace_array *tr)
@@ -8434,6 +8443,7 @@ int trace_array_destroy(struct trace_array *tr)
 	mutex_lock(&event_mutex);
 	mutex_lock(&trace_types_lock);
 
+	tr->ref--;
 	ret = __remove_instance(tr);
 
 	mutex_unlock(&trace_types_lock);

kernel/trace/trace.c static int instance_rmdir(const char *name) · line 8477

Quoted 3 lines
kernel/trace/trace.c +0 -0
@@ -8465,6 +8475,24 @@ static int instance_rmdir(const char *name)
 	return ret;
 }
 

Need a kerneldoc heading here, that also states that if a trace_array is returned, it requires a call to trace_array_put().

Quoted 36 lines
kernel/trace/trace.c +18 -0
+struct trace_array *trace_array_lookup(const char *name)
+{
+	struct trace_array *tr;
+
+	mutex_lock(&trace_types_lock);
+
+	list_for_each_entry(tr, &ftrace_trace_arrays, list) {
+		if (tr->name && strcmp(tr->name, name) == 0) {
+			tr->ref++;
+			mutex_unlock(&trace_types_lock);
+			return tr;
+		}
+	}
+	mutex_unlock(&trace_types_lock);
+	return NULL;
+}
+EXPORT_SYMBOL_GPL(trace_array_lookup);
+
 static __init void create_trace_instances(struct dentry *d_tracer)
 {
 	trace_instance_dir = tracefs_create_instance_dir("instances", d_tracer,
kernel/trace/trace_events.c +7 -0
index 445b059..c126d2c 100644
@@ -859,6 +859,28 @@ int trace_set_clr_event(const char *system, const char *event, int set)
 }
 EXPORT_SYMBOL_GPL(trace_set_clr_event);
 
+/**
+ * trace_array_set_clr_event - enable or disable an event for a trace array
+ * @system: system name to match (NULL for any system)
+ * @event: event name to match (NULL for all events, within system)
+ * @set: 1 to enable, 0 to disable
+ *
+ * This is a way for other parts of the kernel to enable or disable

kernel/trace/trace_events.c int trace_set_clr_event(const char *system, const char *event, int set) · line 878

Quoted 10 lines
kernel/trace/trace_events.c +10 -0
+ * event recording to instances.
+ *
+ * Returns 0 on success, -EINVAL if the parameters do not match any
+ * registered events.
+ */
+int trace_array_set_clr_event(struct trace_array *tr, const char *system,
+		const char *event, int set)
+{
+	if (!tr)
+		return -ENODEV;

If we have this, should we get rid of the external use of ftrace_set_clr_event()?

-- Steve

Quoted 8 lines
kernel/trace/trace_events.c +5 -0
+
+	return __ftrace_set_clr_event(tr, NULL, system, event, set);
+}
+EXPORT_SYMBOL_GPL(trace_array_set_clr_event);
+
 /* 128 should be much more than enough */
 #define EVENT_BUF_SIZE		127
 
author
#5 lore raw

Re: [PATCH 3/3] tracing: Add 2 new funcs. for kernel access to Ftrace instances.

Hi Steven,

Thanks for reviewing the patch. I have made a note of all the suggested changes, comment requirements.

Will send them out soon as v2.

(comments inline)

Quoted 127 lines · Steven Rostedt
Quoted 39 lines · On Wed, 12 Jun 2019 09:34:19 -0700 Divya Indi <[email protected]> wrote:

Adding 2 new functions -
1) trace_array_lookup : Look up and return a trace array, given its name.
2) trace_array_set_clr_event : Enable/disable event recording to the given trace array.

Newly added functions trace_array_lookup and trace_array_create also need to increment the reference counter associated with the trace array they return. This is to ensure the trace array does not get freed while in use by the newly introduced APIs.
The reference ctr is decremented in the trace_array_destroy.

---
  include/linux/trace_events.h |  3 +++
  kernel/trace/trace.c         | 30 +++++++++++++++++++++++++++++-
  kernel/trace/trace_events.c  | 22 ++++++++++++++++++++++
  3 files changed, 54 insertions(+), 1 deletion(-)
include/linux/trace_events.h +3 -0
index d7b7d85..0cc99a8 100644
@@ -545,7 +545,10 @@ int trace_array_printk(struct trace_array *tr, unsigned long ip,
  struct trace_array *trace_array_create(const char *name);
  int trace_array_destroy(struct trace_array *tr);
  int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set);
+struct trace_array *trace_array_lookup(const char *name);
  int trace_set_clr_event(const char *system, const char *event, int set);
+int trace_array_set_clr_event(struct trace_array *tr, const char *system,
+		const char *event, int set);
 
  /*
   * The double __builtin_constant_p is because gcc will give us an error
kernel/trace/trace.c +0 -0
index a60dc13..fb70ccc 100644

kernel/trace/trace.c struct trace_array *trace_array_create(const char *name) · line 8368

Quoted 5 lines
kernel/trace/trace.c +2 -0
@@ -8364,6 +8364,8 @@ struct trace_array *trace_array_create(const char *name)
 
  	list_add(&tr->list, &ftrace_trace_arrays);
 
+	tr->ref++;
+

Needs a comment for why the ref is incremented.

Quoted 4 lines
kernel/trace/trace.c +0 -0
  	mutex_unlock(&trace_types_lock);
  	mutex_unlock(&event_mutex);
 

kernel/trace/trace.c struct trace_array *trace_array_create(const char *name) · line 8395

Quoted 10 lines
kernel/trace/trace.c +6 -1
@@ -8385,7 +8387,14 @@ struct trace_array *trace_array_create(const char *name)
 
  static int instance_mkdir(const char *name)
  {
-	return PTR_ERR_OR_ZERO(trace_array_create(name));
+	struct trace_array *tr;
+
+	tr = trace_array_create(name);
+	if (IS_ERR(tr))
+		return PTR_ERR(tr);
+	trace_array_put(tr);

Need a comment to why we need to do a put here.

Quoted 14 lines
kernel/trace/trace.c +3 -0
+
+	return 0;
  }
 
  static int __remove_instance(struct trace_array *tr)
@@ -8434,6 +8443,7 @@ int trace_array_destroy(struct trace_array *tr)
  	mutex_lock(&event_mutex);
  	mutex_lock(&trace_types_lock);
 
+	tr->ref--;
  	ret = __remove_instance(tr);
 
  	mutex_unlock(&trace_types_lock);

kernel/trace/trace.c static int instance_rmdir(const char *name) · line 8477

Quoted 3 lines
kernel/trace/trace.c +0 -0
@@ -8465,6 +8475,24 @@ static int instance_rmdir(const char *name)
  	return ret;
  }
 

Need a kerneldoc heading here, that also states that if a trace_array is returned, it requires a call to trace_array_put().

Quoted 40 lines
kernel/trace/trace.c +18 -0
+struct trace_array *trace_array_lookup(const char *name)
+{
+	struct trace_array *tr;
+
+	mutex_lock(&trace_types_lock);
+
+	list_for_each_entry(tr, &ftrace_trace_arrays, list) {
+		if (tr->name && strcmp(tr->name, name) == 0) {
+			tr->ref++;
+			mutex_unlock(&trace_types_lock);
+			return tr;
+		}
+	}
+	mutex_unlock(&trace_types_lock);
+	return NULL;
+}
+EXPORT_SYMBOL_GPL(trace_array_lookup);
+
  static __init void create_trace_instances(struct dentry *d_tracer)
  {
  	trace_instance_dir = tracefs_create_instance_dir("instances", d_tracer,
kernel/trace/trace_events.c +11 -0
index 445b059..c126d2c 100644
@@ -859,6 +859,28 @@ int trace_set_clr_event(const char *system, const char *event, int set)
  }
  EXPORT_SYMBOL_GPL(trace_set_clr_event);
 
+/**
+ * trace_array_set_clr_event - enable or disable an event for a trace array
+ * @system: system name to match (NULL for any system)
+ * @event: event name to match (NULL for all events, within system)
+ * @set: 1 to enable, 0 to disable
+ *
+ * This is a way for other parts of the kernel to enable or disable
+ * event recording to instances.
+ *
+ * Returns 0 on success, -EINVAL if the parameters do not match any
+ * registered events.
Quoted 8 lines

kernel/trace/trace_events.c int trace_set_clr_event(const char *system, const char *event, int set) · line 878

Quoted 6 lines
kernel/trace/trace_events.c +6 -0
+ */
+int trace_array_set_clr_event(struct trace_array *tr, const char *system,
+		const char *event, int set)
+{
+	if (!tr)
+		return -ENODEV;

If we have this, should we get rid of the external use of ftrace_set_clr_event()?

I think so too. External use of trace_array_set_clr_event is more convenient and intuitive. Will make the change.

Thanks,

Divya

Quoted 11 lines

-- Steve

Quoted 8 lines
kernel/trace/trace_events.c +5 -0
+
+	return __ftrace_set_clr_event(tr, NULL, system, event, set);
+}
+EXPORT_SYMBOL_GPL(trace_array_set_clr_event);
+
  /* 128 should be much more than enough */
  #define EVENT_BUF_SIZE		127
 
maintainer
#6 lore raw

Re: [PATCH 2/3] tracing: Adding additional NULL checks.

In reply to Divya Indi
Quoted 3 lines · On Wed, 12 Jun 2019 09:34:18 -0700 Divya Indi <[email protected]> wrote:

commit f45d1225adb0 ("tracing: Kernel access to Ftrace instances") exported certain functions providing access to Ftrace instances from other kernel components.

I'm fine with the patch, the above statement is hard to understand.

-- Steve

Quoted 35 lines

Adding some additional NULL checks to ensure safe usage by the users.

---
 kernel/trace/trace.c        | 3 +++
 kernel/trace/trace_events.c | 2 ++
 2 files changed, 5 insertions(+)
kernel/trace/trace.c +3 -0
index 1c80521..a60dc13 100644
@@ -3205,6 +3205,9 @@ int trace_array_printk(struct trace_array *tr,
 	if (!(global_trace.trace_flags & TRACE_ITER_PRINTK))
 		return 0;
 
+	if (!tr)
+		return -EINVAL;
+
 	va_start(ap, fmt);
 	ret = trace_array_vprintk(tr, ip, fmt, ap);
 	va_end(ap);
kernel/trace/trace_events.c +2 -0
index b6b4618..445b059 100644
@@ -800,6 +800,8 @@ int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set)
 	char *event = NULL, *sub = NULL, *match;
 	int ret;
 
+	if (!tr)
+		return -ENODEV;
 	/*
 	 * The buf format can be <subsystem>:<event-name>
 	 *  *:<event-name> means any event by that name.
maintainer
#7 lore raw

Re: [PATCH 1/3] tracing: Relevant changes for kernel access to Ftrace instances.

In reply to Divya Indi

On Wed, 12 Jun 2019 09:34:17 -0700
Divya Indi <[email protected]> wrote:

Hi Divya,

First, make sure the patches are all replied to the cover patch [0/3]. That is, patch 1, 2 and 3 should all be in reply to [0/3] to see a all the patches at the same level, and that makes replies easy to stand out.

Having patch 2 a reply to patch 1 and patch 3 a reply to patch 2, makes it hard to see what comments are for what patches.

Quoted 5 lines

For commit f45d1225adb0 ("tracing: Kernel access to Ftrace instances") Adding the following changes to ensure other kernel components can use these functions -
1) Remove static keyword for newly exported fn - ftrace_set_clr_event.
2) Add the req functions to header file include/linux/trace_events.h.

The above change log is hard to parse. The "Adding" looks to be a start of a new sentence, and what changes and what components can use these functions?

Also avoid shorten words ("fn", "req") that just makes trying to figure out what is being said that more confusing.

Quoted 12 lines
---
 include/linux/trace_events.h | 6 ++++++
 kernel/trace/trace_events.c  | 2 +-
 2 files changed, 7 insertions(+), 1 deletion(-)
include/linux/trace_events.h +0 -0
index 8a62731..d7b7d85 100644

include/linux/trace_events.h extern int trace_define_field(struct trace_event_call *call, const char *type, · line 547

Quoted 9 lines
include/linux/trace_events.h +6 -0
@@ -539,6 +539,12 @@ extern int trace_define_field(struct trace_event_call *call, const char *type,
 
 #define is_signed_type(type)	(((type)(-1)) < (type)1)
 
+void trace_printk_init_buffers(void);
+int trace_array_printk(struct trace_array *tr, unsigned long ip,
+		const char *fmt, ...);
+struct trace_array *trace_array_create(const char *name);
+int trace_array_destroy(struct trace_array *tr);
+int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set);

These function are declared elsewhere. If we are adding them here, then they should be removed from the other places.

But trace_events.h is not the proper place to put these. The trace_events.h file is only for code used in the TRACE_EVENT() macros.

The proper file is include/linux/trace.h

-- Steve

Quoted 16 lines
include/linux/trace_events.h +0 -0
 int trace_set_clr_event(const char *system, const char *event, int set);
 
 /*
kernel/trace/trace_events.c +1 -1
index 0ce3db6..b6b4618 100644
@@ -795,7 +795,7 @@ static int __ftrace_set_clr_event(struct trace_array *tr, const char *match,
 	return ret;
 }
 
-static int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set)
+int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set)
 {
 	char *event = NULL, *sub = NULL, *match;
 	int ret;