Skip to contentSkip to conversation

Versioning file system

DISCUSSION Bodo Eggert 8 participants 17 messages Jun 18, 2007 – Jun 19, 2007 7 messages not in the archive VFS

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#7 lore raw
Quoted 5 lines · alan

I just wish that people would learn from the mistakes of others. The MacOS is a prime example of why you do not want to use a forked filesystem, yet some people still seem to think it is a good idea. (Forked filesystems tend to be fragile and do not play well with non-forked filesystems.)

What's the conceptual difference between forks and extended user attributes?

Signature · 4 lines
"Unix policy is to not stop root from doing stupid things because
that would also stop him from doing clever things." - Andi Kleen

"It's such a fine line between stupid and clever" - Derek Smalls
#8 lore raw
Quoted 9 lines · Bodo Eggert
Quoted 5 lines · alan

I just wish that people would learn from the mistakes of others. The MacOS is a prime example of why you do not want to use a forked filesystem, yet some people still seem to think it is a good idea. (Forked filesystems tend to be fragile and do not play well with non-forked filesystems.)

What's the conceptual difference between forks and extended user attributes?

Forks tend to contain more than just extended attributes. They contain all sorts of other meta-data including icons, descriptions, author information, copyright data, and whatever else can be shoveled into them by the author/user.

Signature · 4 lines
"ANSI C says access to the padding fields of a struct is undefined.
ANSI C also says that struct assignment is a memcpy. Therefore struct
assignment in ANSI C is a violation of ANSI C..."
                                  - Alan Cox
#9 lore raw
Quoted 10 lines · alan
Quoted 8 lines · Bodo Eggert
Quoted 5 lines · alan

I just wish that people would learn from the mistakes of others. The MacOS is a prime example of why you do not want to use a forked filesystem, yet some people still seem to think it is a good idea. (Forked filesystems tend to be fragile and do not play well with non-forked filesystems.)

Quoted 7 lines
Quoted 2 lines

What's the conceptual difference between forks and extended user attributes?

Forks tend to contain more than just extended attributes. They contain all sorts of other meta-data including icons, descriptions, author information, copyright data, and whatever else can be shoveled into them by the author/user.

And that makes them different from extended attributes, how?

Both of these really are nothing but ad hocky syntactic sugar for directories, sometimes combined with in-filesystem support for small data items.

-hpa

#10 lore raw
Quoted 1 line · H. Peter Anvin

And that makes them different from extended attributes, how?

Streams on systems that support them allow lseek and are accessed by fd's. EA's are always a blob of data, read/written in their entirity.

Jeremy.

#11 lore raw
In reply to H. Peter Anvin
Quoted 5 lines · H. Peter Anvin

And that makes them different from extended attributes, how?

Both of these really are nothing but ad hocky syntactic sugar for directories, sometimes combined with in-filesystem support for small data items.

There's a good discussion of the issues involved in my LCA 2006 presentation.... which doesn't seem to be on the LCA 2006 site. Hrm. I'll have to ask that this be fixed. In any case, here it is:

http://thunk.org/tytso/forkdepot.odp

- Ted

#12 lore raw
Quoted 7 lines · Theodore Tso
Quoted 5 lines · H. Peter Anvin

And that makes them different from extended attributes, how?

Both of these really are nothing but ad hocky syntactic sugar for directories, sometimes combined with in-filesystem support for small data items.

Quoted 5 lines

There's a good discussion of the issues involved in my LCA 2006 presentation.... which doesn't seem to be on the LCA 2006 site. Hrm. I'll have to ask that this be fixed. In any case, here it is:

http://thunk.org/tytso/forkdepot.odp

The main difference appears to be the potential size. Both extended attributes and forks allow for extra data that I neither want or need. But once the extra space is large enough to hide a rootkit in, it becomes a security problem instead of just something pointless.

Pointless here means that _I_ don't see the point. Maybe there are valid uses for extended attributes. If there are, noone has explained them to me yet.

Jörn

Signature · 7 lines
They laughed at Galileo.  They laughed at Copernicus.  They laughed at
Columbus. But remember, they also laughed at Bozo the Clown.
-- unknown
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#13 lore raw
Quoted 4 lines · Jörn Engel

Pointless here means that _I_ don't see the point. Maybe there are valid uses for extended attributes. If there are, noone has explained them to me yet.

Samba uses them to store DOS'ism's that you don't want in your POSIX filesystem.

Jeremy.
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#14 lore raw
In reply to Jörn Engel
Quoted 14 lines · Jörn Engel
Quoted 7 lines · Theodore Tso
Quoted 5 lines · H. Peter Anvin

And that makes them different from extended attributes, how?

Both of these really are nothing but ad hocky syntactic sugar for directories, sometimes combined with in-filesystem support for small data items.

Quoted 5 lines

There's a good discussion of the issues involved in my LCA 2006 presentation.... which doesn't seem to be on the LCA 2006 site. Hrm. I'll have to ask that this be fixed. In any case, here it is:

http://thunk.org/tytso/forkdepot.odp

Quoted 8 lines

The main difference appears to be the potential size. Both extended attributes and forks allow for extra data that I neither want or need. But once the extra space is large enough to hide a rootkit in, it becomes a security problem instead of just something pointless.

Pointless here means that _I_ don't see the point. Maybe there are valid uses for extended attributes. If there are, noone has explained them to me yet.

Most of the extended attribute systems I have seen have been a set of flags. "If this bit is set, the user can do thus to this object." Sometimes it is a named attribute that is attached to the object.

Forks tend to be "this blob of data is attached to this object".

With forks, the choices tend to be a lot more arbitrary.

Signature · 4 lines
"ANSI C says access to the padding fields of a struct is undefined.
ANSI C also says that struct assignment is a memcpy. Therefore struct
assignment in ANSI C is a violation of ANSI C..."
                                  - Alan Cox
#15 lore raw
In reply to Jörn Engel
Quoted 3 lines · Jörn Engel

Pointless here means that _I_ don't see the point. Maybe there are valid uses for extended attributes. If there are, noone has explained them to me yet.

The users of extended attributes that I've dealt with are ACL support and SELinux. These both use extended attributes under the covers. It's just not immediately obvious if you aren't looking.

Brad Boyer
[email protected]

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#16 lore raw
Quoted 8 lines · Brad Boyer
Quoted 3 lines · Jörn Engel

Pointless here means that _I_ don't see the point. Maybe there are valid uses for extended attributes. If there are, noone has explained them to me yet.

The users of extended attributes that I've dealt with are ACL support and SELinux. These both use extended attributes under the covers. It's just not immediately obvious if you aren't looking.

Yeah, extended attributes are typically used for exactly that:
"attributes" like labels, permissions, encoding, cached file-type, DOS/Windows/Mac metadata, etc. Sometimes people suggest sticking icons in there, but that's probably a bad idea. At most stick an "icon label" attribute which refers to a file "/usr/share/icons/ by_attr/$ICON_LABEL.png". If you're trying to put more than 256 bytes of data in an extended attribute then you're probably doing something wrong. They're very good for cached attributes (like file- type) where you don't care if the data is lost by "tar", and they're reasonable for security-related attributes where you don't want attribute-unaware programs trying to save and restore them (like SELinux labels).

Cheers,
Kyle Moffett

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#17 lore raw
In reply to Jörn Engel
Quoted 4 lines · Jörn Engel

The main difference appears to be the potential size. Both extended attributes and forks allow for extra data that I neither want or need. But once the extra space is large enough to hide a rootkit in, it becomes a security problem instead of just something pointless.

The other difference is that you can't execute an extended attribute.

You can store kvm/qemu, a complete virtualization enviroment, shared libraries, and other executables all inside a forks inside a file, and then execute programs/rootkit out of said file fork(s).

As I mentioned in my LCA presentation, one system administrator refused to upgrade beyond Solaris 8 because he thought forks were good for nothing but letting system crackers hide rootkits that wouldn't be detected by programs like tripwire. The question then is why in the world would we want to replicate Sun's mistakes?

- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#18 lore raw
In reply to Theodore Tso
Quoted 7 lines · Theodore Tso
Quoted 5 lines · H. Peter Anvin

And that makes them different from extended attributes, how?

Both of these really are nothing but ad hocky syntactic sugar for directories, sometimes combined with in-filesystem support for small data items.

Quoted 5 lines

There's a good discussion of the issues involved in my LCA 2006 presentation.... which doesn't seem to be on the LCA 2006 site. Hrm. I'll have to ask that this be fixed. In any case, here it is:

http://thunk.org/tytso/forkdepot.odp

Did you ever code up forkdepot ? Just wondering ?

Just because I now agree with you that streams are a bad idea doesn't mean the pressure to support them in some way in Samba has gone away :-).

Jeremy.

#19 lore raw
Quoted 3 lines · Jeremy Allison

Just because I now agree with you that streams are a bad idea doesn't mean the pressure to support them in some way in Samba has gone away :-).

Having dealt with Stream's support[1] in the past, I can assure you it is a bad idea. ]:>

[1] http://www.stream.com/

Signature · 4 lines
"ANSI C says access to the padding fields of a struct is undefined.
ANSI C also says that struct assignment is a memcpy. Therefore struct
assignment in ANSI C is a violation of ANSI C..."
                                  - Alan Cox
#20 lore raw
In reply to Jeremy Allison
Quoted 1 line · Jeremy Allison

Did you ever code up forkdepot ? Just wondering ?

There is a partial implementation lieing around somewhere, but there were a number of problems we ran into that were discussed in the slidedeck. Basically, if the only program accessing the files containing forks was the Samba program calling forkdepot library, it worked fine. But if there were other programs (or NFS servers) that were potentially deleting files, moving files around, the things fell apart fairly quickly.

Quoted 3 lines

Just because I now agree with you that streams are a bad idea doesn't mean the pressure to support them in some way in Samba has gone away :-).

What, even with Winfs delaying Microsoft Longwait by years before finally being flushed? :-)

- Ted

#21 lore raw
Quoted 8 lines · Theodore Tso

There is a partial implementation lieing around somewhere, but there were a number of problems we ran into that were discussed in the slidedeck. Basically, if the only program accessing the files containing forks was the Samba program calling forkdepot library, it worked fine. But if there were other programs (or NFS servers) that were potentially deleting files, moving files around, the things fell apart fairly quickly.

I'd be happy with a Samba-only implementation for Appliance vendors.

Quoted 2 lines

What, even with Winfs delaying Microsoft Longwait by years before finally being flushed? :-)

I'm not talking WinFS, I'm talking streams..... Streams are already being used (mainly by malware writers of course - but hey, don't you want full compatibility ? :-).

Jeremy.

#22 lore raw
Quoted 5 lines · Jeremy Allison

I'm not talking WinFS, I'm talking streams..... Streams are already being used (mainly by malware writers of course - but hey, don't you want full compatibility ? :-).

Reminds me of the Linux Journal (I believe?) article which did viruses-on-Wine compatibility testing.

-hpa

#23 lore raw
In reply to H. Peter Anvin
Quoted 14 lines · H. Peter Anvin
Quoted 13 lines · alan
Quoted 10 lines · Bodo Eggert
Quoted 5 lines · alan

I just wish that people would learn from the mistakes of others. The MacOS is a prime example of why you do not want to use a forked filesystem, yet some people still seem to think it is a good idea. (Forked filesystems tend to be fragile and do not play well with non-forked filesystems.)

What's the conceptual difference between forks and extended user attributes?

Quoted 6 lines
Quoted 4 lines

Forks tend to contain more than just extended attributes. They contain all sorts of other meta-data including icons, descriptions, author information, copyright data, and whatever else can be shoveled into them by the author/user.

And that makes them different from extended attributes, how?

The amount of crap. Both seem to become a collection bin for "stuff we need to describe this object". Forks seem to get more piled on, but they are effectively the same thing.

Quoted 3 lines

Both of these really are nothing but ad hocky syntactic sugar for directories, sometimes combined with in-filesystem support for small data items.

And both tend to break when you go to a file system that does not support them.

Signature · 4 lines
"ANSI C says access to the padding fields of a struct is undefined.
ANSI C also says that struct assignment is a memcpy. Therefore struct
assignment in ANSI C is a violation of ANSI C..."
                                  - Alan Cox