Skip to contentSkip to conversation

Attribute timeout handling and wrapping u32 jiffies

PATCH In review Fabio Olive Leite 3 participants 11 messages Jul 25, 2007 – Aug 27, 2007

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Attribute timeout handling and wrapping u32 jiffies

Hi all,

I would like to discuss the idea that the current checks for attribute timeout using time_after are inadequate for 32bit architectures, since time_after works correctly only when the two timestamps being compared are within 2^31 jiffies of each other. The signed overflow caused by comparing values more than 2^31 jiffies apart will flip the result, causing incorrect assumptions of validity.

2^31 jiffies is a fairly large period of time (~25 days) when compared to the lifetime of most kernel data structures, but for long lived NFS mounts that can sit idle for months (think that for some reason autofs cannot be used), it is easy to compare inode attribute timestamps with very disparate or even bogus values (as in when jiffies have wrapped many times, where the comparison doesn't even make sense).

Currently the code tests for attribute timeout by simply adding the desired amount of jiffies to the stored timestamp and comparing that with the current timestamp of obtained attribute data with time_after. This is incorrect, as it returns true for the desired timeout period and another full 2^31 range of jiffies.

I would like to propose a different way for testing for timeouts: we first make sure the timestamps are comparable, meaning the "current" timestamp is checked for being after the "stored" timestamp (if it is before we assume it has wrapped), and after that we check if the difference between them is greater than the desired timeout/validity period.

This test does not cast the values to signed, and instead of comparing two possibly unrelated points in time, it actually tests if the timestamp wrapped, and if the period of time between the two timestamps is greater than the period represented by the timeout value. It seems to make more sense this way.

In testing with artificial jumps (several small jumps, not one big crank) of the jiffies I was able to reproduce a problem found in a server with very long lived NFS mounts, where attributes would not be refreshed even after touching files and directories in the server:

Initial uptime:
03:42:01 up 6 min, 0 users, load average: 0.01, 0.12, 0.07

NFS volume is mounted and time is advanced:
03:38:09 up 25 days, 2 min, 0 users, load average: 1.22, 1.05, 1.08

# ls -l /local/A/foo/bar /nfs/A/foo/bar
-rw-r--r--  1 root root 0 Dec 17 03:38 /local/A/foo/bar
-rw-r--r--  1 root root 0 Nov 22 00:36 /nfs/A/foo/bar
# touch /local/A/foo/bar
# ls -l /local/A/foo/bar /nfs/A/foo/bar
-rw-r--r--  1 root root 0 Dec 17 03:47 /local/A/foo/bar
-rw-r--r--  1 root root 0 Nov 22 00:36 /nfs/A/foo/bar

We can see the local mtime is updated, but the NFS mount still shows the old value. The patch below makes it work:

Initial setup...
07:11:02 up 25 days, 1 min, 0 users, load average: 0.15, 0.03, 0.04

# ls -l /local/A/foo/bar /nfs/A/foo/bar
-rw-r--r--  1 root root 0 Jan 11 07:11 /local/A/foo/bar
-rw-r--r--  1 root root 0 Jan 11 07:11 /nfs/A/foo/bar
# touch /local/A/foo/bar
# ls -l /local/A/foo/bar /nfs/A/foo/bar
-rw-r--r--  1 root root 0 Jan 11 07:14 /local/A/foo/bar
-rw-r--r--  1 root root 0 Jan 11 07:14 /nfs/A/foo/bar

I'd love to hear comments and criticism. Even if this is not the correct way to fix it, I'm fairly sure we can agree that there's the need for a fix and a proper fix can be found.

fs/nfs/dir.c +1 -1
index ea97408..8e85710 100644
@@ -1852,7 +1852,7 @@ int nfs_access_get_cached(struct inode *inode, struct=
 rpc_cred *cred, struct nfs
 	cache =3D nfs_access_search_rbtree(inode, cred);
 	if (cache =3D=3D NULL)
 		goto out;
-	if (time_after(jiffies, cache->jiffies + NFS_ATTRTIMEO(inode)))
+	if (timeout_or_wrap(jiffies, cache->jiffies, NFS_ATTRTIMEO(inode)))
 		goto out_stale;
 	res->jiffies =3D cache->jiffies;

res->cred =3D cache->cred;

fs/nfs/inode.c +1 -1
index bca6cdc..f611f95 100644
@@ -662,7 +662,7 @@ int nfs_attribute_timeout(struct inode *inode)
 =
 
 	if (nfs_have_delegation(inode, FMODE_READ))
 		return 0;
-	return time_after(jiffies, nfsi->read_cache_jiffies+nfsi->attrtimeo);
+	return timeout_or_wrap(jiffies, nfsi->read_cache_jiffies, nfsi->attrtimeo=

);
}
=

/**
@@ -1061,7 +1061,7 @@ static int nfs_update_inode(struct inode *inode, stru= ct nfs_fattr *fattr)
nfs_inc_stats(inode, NFSIOS_ATTRINVALIDATE); nfsi->attrtimeo =3D NFS_MINATTRTIMEO(inode); nfsi->attrtimeo_timestamp =3D now;
- } else if (time_after(now, nfsi->attrtimeo_timestamp+nfsi->attrtimeo)) {
+ } else if (timeout_or_wrap(now, nfsi->attrtimeo_timestamp, nfsi->attrtime=
o)) {
if ((nfsi->attrtimeo <<=3D 1) > NFS_MAXATTRTIMEO(inode))
nfsi->attrtimeo =3D NFS_MAXATTRTIMEO(inode);
nfsi->attrtimeo_timestamp =3D now;

include/linux/jiffies.h +5 -0
index c080f61..ffda7cb 100644
@@ -115,6 +115,11 @@ static inline u64 get_jiffies_64(void)
 	 ((long)(a) - (long)(b) >=3D 0))
 #define time_before_eq(a,b)	time_after_eq(b,a)
 =
 
+#define timeout_or_wrap(a,b,c)	\
+	(typecheck(unsigned long, a) && \
+	 typecheck(unsigned long, b) && \
+	 ((a) < (b) || (a) - (b) > (c)))
+
 /* Same as above, but does so with platform independent 64bit types.
  * These must be used when utilizing jiffies_64 (i.e. return value of

* get_jiffies_64() */

Cheers!
F=E1bio
-- =

ex sed lex awk yacc, e pluribus unix, amem

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#1 lore raw
Quoted 116 lines · Fabio Olive Leite

Hi all,

I would like to discuss the idea that the current checks for attribute timeout using time_after are inadequate for 32bit architectures, since time_after works correctly only when the two timestamps being compared are within 2^31 jiffies of each other. The signed overflow caused by comparing values more than 2^31 jiffies apart will flip the result, causing incorrect assumptions of validity.

2^31 jiffies is a fairly large period of time (~25 days) when compared to the lifetime of most kernel data structures, but for long lived NFS mounts that can sit idle for months (think that for some reason autofs cannot be used), it is easy to compare inode attribute timestamps with very disparate or even bogus values (as in when jiffies have wrapped many times, where the comparison doesn't even make sense).

Currently the code tests for attribute timeout by simply adding the desired amount of jiffies to the stored timestamp and comparing that with the current timestamp of obtained attribute data with time_after. This is incorrect, as it returns true for the desired timeout period and another full 2^31 range of jiffies.

I would like to propose a different way for testing for timeouts: we first make sure the timestamps are comparable, meaning the "current" timestamp is checked for being after the "stored" timestamp (if it is before we assume it has wrapped), and after that we check if the difference between them is greater than the desired timeout/validity period.

This test does not cast the values to signed, and instead of comparing two possibly unrelated points in time, it actually tests if the timestamp wrapped, and if the period of time between the two timestamps is greater than the period represented by the timeout value. It seems to make more sense this way.

In testing with artificial jumps (several small jumps, not one big crank) of the jiffies I was able to reproduce a problem found in a server with very long lived NFS mounts, where attributes would not be refreshed even after touching files and directories in the server:

Initial uptime:
03:42:01 up 6 min, 0 users, load average: 0.01, 0.12, 0.07

NFS volume is mounted and time is advanced:
03:38:09 up 25 days, 2 min, 0 users, load average: 1.22, 1.05, 1.08

# ls -l /local/A/foo/bar /nfs/A/foo/bar
(quoted diff) +0 -2
-rw-r--r--  1 root root 0 Dec 17 03:38 /local/A/foo/bar
-rw-r--r--  1 root root 0 Nov 22 00:36 /nfs/A/foo/bar
# touch /local/A/foo/bar
# ls -l /local/A/foo/bar /nfs/A/foo/bar
(quoted diff) +0 -2
-rw-r--r--  1 root root 0 Dec 17 03:47 /local/A/foo/bar
-rw-r--r--  1 root root 0 Nov 22 00:36 /nfs/A/foo/bar

We can see the local mtime is updated, but the NFS mount still shows the old value. The patch below makes it work:

Initial setup...
07:11:02 up 25 days, 1 min, 0 users, load average: 0.15, 0.03, 0.04

# ls -l /local/A/foo/bar /nfs/A/foo/bar
(quoted diff) +0 -2
-rw-r--r--  1 root root 0 Jan 11 07:11 /local/A/foo/bar
-rw-r--r--  1 root root 0 Jan 11 07:11 /nfs/A/foo/bar
# touch /local/A/foo/bar
# ls -l /local/A/foo/bar /nfs/A/foo/bar
(quoted diff) +0 -2
-rw-r--r--  1 root root 0 Jan 11 07:14 /local/A/foo/bar
-rw-r--r--  1 root root 0 Jan 11 07:14 /nfs/A/foo/bar

I'd love to hear comments and criticism. Even if this is not the correct way to fix it, I'm fairly sure we can agree that there's the need for a fix and a proper fix can be found.

fs/nfs/dir.c +1 -1
index ea97408..8e85710 100644
@@ -1852,7 +1852,7 @@ int nfs_access_get_cached(struct inode *inode, struct rpc_cred *cred, struct nfs
 	cache = nfs_access_search_rbtree(inode, cred);
 	if (cache == NULL)
 		goto out;
-	if (time_after(jiffies, cache->jiffies + NFS_ATTRTIMEO(inode)))
+	if (timeout_or_wrap(jiffies, cache->jiffies, NFS_ATTRTIMEO(inode)))
 		goto out_stale;
 	res->jiffies = cache->jiffies;
 	res->cred = cache->cred;
fs/nfs/inode.c +2 -2
index bca6cdc..f611f95 100644
@@ -662,7 +662,7 @@ int nfs_attribute_timeout(struct inode *inode)
 
 	if (nfs_have_delegation(inode, FMODE_READ))
 		return 0;
-	return time_after(jiffies, nfsi->read_cache_jiffies+nfsi->attrtimeo);
+	return timeout_or_wrap(jiffies, nfsi->read_cache_jiffies, nfsi->attrtimeo);
 }
 
 /**
@@ -1061,7 +1061,7 @@ static int nfs_update_inode(struct inode *inode, struct nfs_fattr *fattr)
 		nfs_inc_stats(inode, NFSIOS_ATTRINVALIDATE);
 		nfsi->attrtimeo = NFS_MINATTRTIMEO(inode);
 		nfsi->attrtimeo_timestamp = now;
-	} else if (time_after(now, nfsi->attrtimeo_timestamp+nfsi->attrtimeo)) {
+	} else if (timeout_or_wrap(now, nfsi->attrtimeo_timestamp, nfsi->attrtimeo)) {
 		if ((nfsi->attrtimeo <<= 1) > NFS_MAXATTRTIMEO(inode))
 			nfsi->attrtimeo = NFS_MAXATTRTIMEO(inode);
 		nfsi->attrtimeo_timestamp = now;
include/linux/jiffies.h +0 -0
index c080f61..ffda7cb 100644

include/linux/jiffies.h static inline u64 get_jiffies_64(void) · line 122

Quoted 8 lines
include/linux/jiffies.h +5 -0
@@ -115,6 +115,11 @@ static inline u64 get_jiffies_64(void)
 	 ((long)(a) - (long)(b) >= 0))
 #define time_before_eq(a,b)	time_after_eq(b,a)
 
+#define timeout_or_wrap(a,b,c)	\
+	(typecheck(unsigned long, a) && \
+	 typecheck(unsigned long, b) && \
+	 ((a) < (b) || (a) - (b) > (c)))
+

Ugly name. Besides, the above macro causes problems at the wraparound boundary when (b) may indeed be < (a), but we're still in the allowed range. There is also no guarantee that (c) is unsigned. How about

#define time_in_range(a,b,c) \
	(time_after_eq(a,b) && \
	 time_before_eq(a,c))

instead?

Trond

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#2 lore raw
Quoted 14 lines · Trond Myklebust

include/linux/jiffies.h static inline u64 get_jiffies_64(void) · line 123

Quoted 7 lines

=

include/linux/jiffies.h +5 -0
+#define timeout_or_wrap(a,b,c)	\
+	(typecheck(unsigned long, a) && \
+	 typecheck(unsigned long, b) && \
+	 ((a) < (b) || (a) - (b) > (c)))
+

=

Ugly name. Besides, the above macro causes problems at the wraparound boundary when (b) may indeed be < (a), but we're still in the allowed range. There is also no guarantee that (c) is unsigned. How about =

Quoted 6 lines
#define time_in_range(a,b,c) \
	(time_after_eq(a,b) && \
	 time_before_eq(a,c))

=

instead?

Hmmm (a) is the "current" jiffies, (b) is the "stored" jiffies, and both are always unsigned long. But (c) is the timeout constant, and I thought better not to assume typing info there, as a small integer fits in many types.

I think that in the second check you mean this:

#define time_in_range(a,b,c) \
	(time_after_eq(a,b) && \
	 time_before_eq(a,b+c))

I'll see about testing this new macro in my test box and will report back soon.

Thanks,
F=E1bio
-- =

ex sed lex awk yacc, e pluribus unix, amem

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#3 lore raw
Quoted 23 lines · Fabio Olive Leite
Quoted 11 lines · Trond Myklebust

include/linux/jiffies.h static inline u64 get_jiffies_64(void) · line 123

Quoted 6 lines
include/linux/jiffies.h +5 -0
+#define timeout_or_wrap(a,b,c)	\
+	(typecheck(unsigned long, a) && \
+	 typecheck(unsigned long, b) && \
+	 ((a) < (b) || (a) - (b) > (c)))
+

Ugly name. Besides, the above macro causes problems at the wraparound boundary when (b) may indeed be < (a), but we're still in the allowed range. There is also no guarantee that (c) is unsigned. How about

Quoted 5 lines
#define time_in_range(a,b,c) \
	(time_after_eq(a,b) && \
	 time_before_eq(a,c))

instead?

Hmmm (a) is the "current" jiffies, (b) is the "stored" jiffies, and both are always unsigned long. But (c) is the timeout constant, and I thought better not to assume typing info there, as a small integer fits in many types.

Quoted 5 lines

I think that in the second check you mean this:

#define time_in_range(a,b,c) \
	(time_after_eq(a,b) && \
	 time_before_eq(a,b+c))

No. IMO [b;c] should be the range for the generic function, and it should be up to the NFS code to do the 'stored jiffies + constant timeout' calculation. You will note that time_in_range() does in any case return true if you are inside the valid time range, so you will usually want to do

if (!time_in_range())...

in order to do the timeout_or_wrap() test.

Trond

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#4 lore raw
Quoted 5 lines · Trond Myklebust
Quoted 4 lines

=

I think that in the second check you mean this:
=

Quoted 8 lines
Quoted 3 lines
#define time_in_range(a,b,c) \
	(time_after_eq(a,b) && \
	 time_before_eq(a,b+c))

=

No. IMO [b;c] should be the range for the generic function, and it should be up to the NFS code to do the 'stored jiffies + constant timeout' calculation.

OK. That makes sense, since we then only care about jiffy values and can assume typing.

Quoted 9 lines

You will note that time_in_range() does in any
case return true if you are inside the valid time range, so you will usually want to do
=

if (!time_in_range())...
=

in order to do the timeout_or_wrap() test.

Indeed, and the code also looks more readable.

Let's see how it behaves.

Regards,
F=E1bio
-- =

ex sed lex awk yacc, e pluribus unix, amem

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#5 lore raw
In reply to Trond Myklebust
Quoted 9 lines · Trond Myklebust

=

No. IMO [b;c] should be the range for the generic function, and it should be up to the NFS code to do the 'stored jiffies + constant timeout' calculation. You will note that time_in_range() does in any case return true if you are inside the valid time range, so you will usually want to do
=

Quoted 4 lines

if (!time_in_range())...
=

in order to do the timeout_or_wrap() test.

Tried it out and it behaves correctly. Thanks for your suggestions. The patch below is the final version, and fixes the attribute timeout issue with wrapping jiffies.

fs/nfs/dir.c +1 -1
index ea97408..54d3926 100644
@@ -1852,7 +1852,7 @@ int nfs_access_get_cached(struct inode *inode, struct=
 rpc_cred *cred, struct nfs
 	cache =3D nfs_access_search_rbtree(inode, cred);
 	if (cache =3D=3D NULL)
 		goto out;
-	if (time_after(jiffies, cache->jiffies + NFS_ATTRTIMEO(inode)))
+	if (!time_in_range(jiffies, cache->jiffies, cache->jiffies + NFS_ATTRTIME=

O(inode)))
goto out_stale;
res->jiffies =3D cache->jiffies;
res->cred =3D cache->cred;

fs/nfs/inode.c +1 -1
index bca6cdc..89c3e91 100644
@@ -662,7 +662,7 @@ int nfs_attribute_timeout(struct inode *inode)
 =
 
 	if (nfs_have_delegation(inode, FMODE_READ))
 		return 0;
-	return time_after(jiffies, nfsi->read_cache_jiffies+nfsi->attrtimeo);
+	return !time_in_range(jiffies, nfsi->read_cache_jiffies, nfsi->read_cache=

_jiffies + nfsi->attrtimeo);
}
=

/**
@@ -1061,7 +1061,7 @@ static int nfs_update_inode(struct inode *inode, stru= ct nfs_fattr *fattr)
nfs_inc_stats(inode, NFSIOS_ATTRINVALIDATE); nfsi->attrtimeo =3D NFS_MINATTRTIMEO(inode); nfsi->attrtimeo_timestamp =3D now;
- } else if (time_after(now, nfsi->attrtimeo_timestamp+nfsi->attrtimeo)) {
+ } else if (!time_in_range(now, nfsi->attrtimeo_timestamp, nfsi->attrtimeo=
_timestamp + nfsi->attrtimeo)) {
if ((nfsi->attrtimeo <<=3D 1) > NFS_MAXATTRTIMEO(inode))
nfsi->attrtimeo =3D NFS_MAXATTRTIMEO(inode);
nfsi->attrtimeo_timestamp =3D now;

include/linux/jiffies.h +4 -0
index c080f61..f1c87ad 100644
@@ -115,6 +115,10 @@ static inline u64 get_jiffies_64(void)
 	 ((long)(a) - (long)(b) >=3D 0))
 #define time_before_eq(a,b)	time_after_eq(b,a)
 =
 
+#define time_in_range(a,b,c) \
+	(time_after_eq(a,b) && \
+	 time_before_eq(a,c))
+
 /* Same as above, but does so with platform independent 64bit types.
  * These must be used when utilizing jiffies_64 (i.e. return value of

* get_jiffies_64() */

Cheers!
F=E1bio
-- =

ex sed lex awk yacc, e pluribus unix, amem

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#6 lore raw

Hi Trond,

Quoted 8 lines · Fabio Olive Leite

=

Tried it out and it behaves correctly. Thanks for your suggestions. The patch below is the final version, and fixes the attribute timeout issue with wrapping jiffies.
=

...

Have you had a chance to review/commit this patch? Should I do any more testing on it?

Thanks,
F=E1bio
-- =

ex sed lex awk yacc, e pluribus unix, amem

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#7 lore raw
Quoted 4 lines · Fabio Olive Leite

Hi Trond,

Quoted 1 line · Fabio Olive Leite
Quoted 8 lines
Quoted 5 lines

Tried it out and it behaves correctly. Thanks for your suggestions. The patch below is the final version, and fixes the attribute timeout issue with wrapping jiffies.

...

Have you had a chance to review/commit this patch? Should I do any more testing on it?

It looked good when I laster reviewed it, but I haven't yet tested it myself. Please send me a reminder if you don't see it appear in the next NFS_ALL release (which should be within a week).

Cheers
Trond

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#8 lore raw

Hi Trond,

Quoted 5 lines · Trond Myklebust

=

It looked good when I laster reviewed it, but I haven't yet tested it myself. Please send me a reminder if you don't see it appear in the next NFS_ALL release (which should be within a week).

It was commited to the nfs-2.6.git repo, but I haven't seen it get upstream yet. Were there any issues I could work on to speed up this commit?

Thanks!
F=E1bio
-- =

ex sed lex awk yacc, e pluribus unix, amem

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#9 lore raw
Quoted 11 lines · Fabio Olive Leite

Hi Trond,

Quoted 4 lines · Trond Myklebust

It looked good when I laster reviewed it, but I haven't yet tested it myself. Please send me a reminder if you don't see it appear in the next NFS_ALL release (which should be within a week).

It was commited to the nfs-2.6.git repo, but I haven't seen it get upstream yet. Were there any issues I could work on to speed up this commit?

Normally if it's not fairly serious (causing an oops, for example) it'll wait for the next merge window (after the final 2.6.23 comes out, probably next month some time).

--b.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#10 lore raw
In reply to Fabio Olive Leite

T24gV2VkLCAyMDA3LTA4LTIyIGF0IDE1OjI1IC0wMzAwLCBGYWJpbyBPbGl2ZSBMZWl0ZSB3cm90 ZToKPiBIaSBUcm9uZCwKPiAKPiBPbiBUdWUsIEp1bCAzMSwgMjAwNyBhdCAwMzo0NzozM1BNIC0w NDAwLCBUcm9uZCBNeWtsZWJ1c3Qgd3JvdGU6Cj4gPiAKPiA+IEl0IGxvb2tlZCBnb29kIHdoZW4g SSBsYXN0ZXIgcmV2aWV3ZWQgaXQsIGJ1dCBJIGhhdmVuJ3QgeWV0IHRlc3RlZAo+ID4gaXQgbXlz ZWxmLiBQbGVhc2Ugc2VuZCBtZSBhIHJlbWluZGVyIGlmIHlvdSBkb24ndCBzZWUgaXQgYXBwZWFy IGluCj4gPiB0aGUgbmV4dCBORlNfQUxMIHJlbGVhc2UgKHdoaWNoIHNob3VsZCBiZSB3aXRoaW4g YSB3ZWVrKS4KPiAKPiBJdCB3YXMgY29tbWl0ZWQgdG8gdGhlIG5mcy0yLjYuZ2l0IHJlcG8sIGJ1 dCBJIGhhdmVuJ3Qgc2VlbiBpdCBnZXQKPiB1cHN0cmVhbSB5ZXQuIFdlcmUgdGhlcmUgYW55IGlz c3VlcyBJIGNvdWxkIHdvcmsgb24gdG8gc3BlZWQgdXAgdGhpcwo+IGNvbW1pdD8KPiAKPiBUaGFu a3MhCj4gRsOhYmlvCgpJdCBkb2VzIG5vdCBhcHBlYXIgdG8gYmUgYSBjcml0aWNhbCBmaXgsIG9y IGlmIGl0IGlzIHRoZW4gSSBuZWVkIGEKcmVmZXJlbmNlIHRvIGEgY2FzZSB3aGVyZSBhbiBPb3Bz IG9yIGNvcnJ1cHRpb24gd2FzIHNlZW4gc28gdGhhdCBJIGNhbgpqdXN0aWZ5IG1lcmdpbmcgaXQg bm93IHRvIExpbnVzIGFuZCBBbmRyZXcuCgpGYWlsaW5nIHRoYXQgSSBoYXZlIHF1ZXVlZCBpdCBm b3IgdGhlIDIuNi4yNCBtZXJnZSB3aW5kb3cuCgpDaGVlcnMKICBUcm9uZAoKCi0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0KVGhpcyBTRi5uZXQgZW1haWwgaXMgc3BvbnNvcmVkIGJ5OiBTcGx1bmsgSW5jLgpTdGls bCBncmVwcGluZyB0aHJvdWdoIGxvZyBmaWxlcyB0byBmaW5kIHByb2JsZW1zPyAgU3RvcC4KTm93 IFNlYXJjaCBsb2cgZXZlbnRzIGFuZCBjb25maWd1cmF0aW9uIGZpbGVzIHVzaW5nIEFKQVggYW5k IGEgYnJvd3Nlci4KRG93bmxvYWQgeW91ciBGUkVFIGNvcHkgb2YgU3BsdW5rIG5vdyA+PiAgaHR0 cDovL2dldC5zcGx1bmsuY29tLwpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fXwpORlMgbWFpbGxpc3QgIC0gIE5GU0BsaXN0cy5zb3VyY2Vmb3JnZS5uZXQKaHR0 cHM6Ly9saXN0cy5zb3VyY2Vmb3JnZS5uZXQvbGlzdHMvbGlzdGluZm8vbmZzCg==