Skip to contentSkip to conversation

Learning ext2 fs

DISCUSSION Paulo da Silva 3 participants 5 messages Oct 27, 2005 – Oct 28, 2005

lore
Share & tools
lore
#0 lore raw

I am reading the ext2 fs code. One of my purposes is to save the original data of a file to another file just before it is changed by write/mmap/whatever. Because of mmap (any other reasons?) I thought of doing this at "ext2-writepage" or/and "ext2-writepages".

Is this the right place?
Is there a lower level where I can read/write blocks of data from/to hd instead of full pages?

How do I tell the really file data from other data?

I traced these functions but I only got "ext2-writepages" to be called. "ext2-writepage" was never called using the programs
I wrote to test this. When is "ext2-writepage" called?

Thanks for any help.
Any readings advice is also welcome.

#1 lore raw
Quoted 8 lines · Paulo da Silva

I am reading the ext2 fs code. One of my purposes is to save the original data of a file to another file just before it is changed by write/mmap/whatever. Because of mmap (any other reasons?) I thought of doing this at "ext2-writepage" or/and "ext2-writepages".

Is this the right place?

ya, AFAIK ext2-writepage/s and ext2-readpage/s are the functions actually dealing with the file data and you can also see generic_file_write/read assigned to the fields in ext2_file_operations variable because they also deals with the file data ...

Quoted 3 lines

Is there a lower level where I can read/write blocks of data from/to hd instead of full pages?

I think at the lower-level you can deal with the block-layer but at this layer AFAIK you won't be able to distinguish between file data and file system data/structures because you will be dealing with just the blocks/chunks of data for block devices.

Quoted 2 lines

How do I tell the really file data from other data?

the functions readpage/writepage deals with the file data only not with the file-system data/structures. Here I am considering by saying other data you mean file-system data/structures

Quoted 5 lines

I traced these functions but I only got "ext2-writepages" to be called. "ext2-writepage" was never called using the programs
I wrote to test this. When is "ext2-writepage" called?

I think it doesn't matter that writepage or writepages are called because both almost do the same thing and the writepages version deals with more data and in your case the data may be big enough to handle by the writepages.

Quoted 3 lines

Thanks for any help.
Any readings advice is also welcome.

I think you can first get some detailed understanding of VFS layer and then you can easily understand the code of the file-system like EXT2 !

--
Fawad Lateef

#2 lore raw
Quoted 10 lines · Fawad Lateef
Quoted 7 lines · Paulo da Silva

I am reading the ext2 fs code. One of my purposes is to save the original data of a file to another file just before it is changed by write/mmap/whatever. Because of mmap (any other reasons?) I thought of doing this at "ext2-writepage" or/and "ext2-writepages".

Quoted 10 lines
Quoted 3 lines

Is this the right place?

ya, AFAIK ext2-writepage/s and ext2-readpage/s are the functions actually dealing with the file data and you can also see generic_file_write/read assigned to the fields in ext2_file_operations variable because they also deals with the file data ...

Yes, I thought to hook generic_file_write/read for my purpose until I found mmap does not use them! Then I went down to writepage(s).

Quoted 8 lines
Quoted 7 lines

Is there a lower level where I can read/write blocks of data from/to hd instead of full pages?

Quoted 6 lines

I think at the lower-level you can deal with the block-layer but at this layer AFAIK you won't be able to distinguish between file data and file system data/structures because you will be dealing with just the blocks/chunks of data for block devices.

Yes, but it could be useful for another purpose I have in mind.
So far, I have found sb_read that allows for blocks reading using a buffer_head. Does all block readings use it? What about writes?

Quoted 14 lines
Quoted 6 lines

How do I tell the really file data from other data?

the functions readpage/writepage deals with the file data only not with the file-system data/structures. Here I am considering by saying other data you mean file-system data/structures

May I conclude that all pages refer to data from/to the file? I am not sure here, because I noticed that writepages gets called when I unmounted the device!

Quoted 8 lines
Quoted 7 lines

I traced these functions but I only got "ext2-writepages" to be called. "ext2-writepage" was never called using the programs
I wrote to test this. When is "ext2-writepage" called?

Quoted 7 lines

I think it doesn't matter that writepage or writepages are called because both almost do the same thing and the writepages version deals with more data and in your case the data may be big enough to handle by the writepages.

Not at all. I just used a python program to create a file with less than 1kb and then I wrote a few bytes mixing "write" and "mmap". That's why I found strange writepage not being called. Only writepages.

Quoted 11 lines
Quoted 5 lines

Thanks for any help.
Any readings advice is also welcome.

I think you can first get some detailed understanding of VFS layer and then you can easily understand the code of the file-system like EXT2 !

Well, so far I have only a superficial understanding of VFS, but I feel relatively comfortable with it.
The problems I found are more related to the lower levels. How to "say" to write the data to the storage, how to handle the file data structures. For example, in this case, I have to reread the original data from the file, write it to another file, and then commit the pages to disc as requested to the writepage(s) functions. But to reread them, I have to save the "about to be written pages".
There is no literature that I know of, about this ... and understanding the code is a hard work.

It would be nice if someone could write information about FSs. I could find lots of information about VFS level, even a small fs example with only 1 file. Unfortunately it works in memory and makes no access to the disc. So,
the lower level remains obscure ...

Thank you.
Paulo

#3 lore raw
Quoted 17 lines · Paulo da Silva
Quoted 9 lines · Fawad Lateef
Quoted 5 lines · Paulo da Silva

Is there a lower level where I can read/write blocks of data from/to hd instead of full pages?

Quoted 6 lines

I think at the lower-level you can deal with the block-layer but at this layer AFAIK you won't be able to distinguish between file data and file system data/structures because you will be dealing with just the blocks/chunks of data for block devices.

Quoted 6 lines

Yes, but it could be useful for another purpose I have in mind.
So far, I have found sb_read that allows for blocks reading using a buffer_head. Does all block readings use it? What about writes?

sb_bread is used to read the blocks containing file-system data not the files, and for writing file-system data blocks back to storage mark_buffer_dirty and sync_dirty_buffer functions are used as you can see in the function ext2_sync_super (I am not FS expert, rather just worked on it for understanding because i am going to use file-system's stuff in my current project). sb_bread is actually reading the data from the storage directly means with-out going through the buffer_cache layer.

Quoted 14 lines
Quoted 14 lines
Quoted 6 lines

How do I tell the really file data from other data?

the functions readpage/writepage deals with the file data only not with the file-system data/structures. Here I am considering by saying other data you mean file-system data/structures

Quoted 4 lines

May I conclude that all pages refer to data from/to the file? I am not sure here, because I noticed that writepages gets called when I unmounted the device!

I am pretty sure that pages here will be refering to the file data, and as far as unmounting case is concern, synching of the file data might be done so you are noticing them called by the VFS layer for writing the un-synched data of file to storage.

Quoted 13 lines
Quoted 6 lines
Quoted 5 lines

I traced these functions but I only got "ext2-writepages" to be called. "ext2-writepage" was never called using the programs
I wrote to test this. When is "ext2-writepage" called?

Quoted 7 lines

I think it doesn't matter that writepage or writepages are called because both almost do the same thing and the writepages version deals with more data and in your case the data may be big enough to handle by the writepages.

Quoted 5 lines

Not at all. I just used a python program to create a file with less than 1kb and then I wrote a few bytes mixing "write" and "mmap". That's why I found strange writepage not being called. Only writepages.

I can't say much about this, but I think writepages are called by VFS layer might be with nr_pages=1, so that a single page can be written with small data ! Have you checked the nr_pages field when it is called ?

Quoted 7 lines

It would be nice if someone could write information about FSs. I could find lots of information about VFS level, even a small fs example with only 1 file. Unfortunately it works in memory and makes no access to the disc. So,
the lower level remains obscure ...

AFAIR there are books which contain chapters explaining the file-systems like EXT2/3 and you can also search for "Simple File System" or "sfs" on the google for the code of small file-system which works upto 64MB storage but on the real device not on the memory !

--
Fawad Lateef

#4 lore raw
In reply to Paulo da Silva
Quoted 9 lines · Paulo da Silva

I am reading the ext2 fs code. One of my purposes is to save the original data of a file to another file just before it is changed by write/mmap/whatever. Because of mmap (any other reasons?) I thought of doing this at "ext2-writepage" or/and "ext2-writepages".

Is this the right place?

Maybe, but the error handling will drive you insane. My approach was to copy on open, not on write. See
http://wohnheim.fh-wedel.de/~joern/cowlink/

Jörn

Signature · 2 lines
Happiness isn't having what you want, it's wanting what you have.
-- unknown