Accessing odd last partition sector (was: [Linux-NTFS-Dev] mkntfs dirty volume marking)
A message that is not in the archive (sent off-list or not yet received): 001601c41db7$aa0a02e0$0100000a@p667
Quoted 5 lines · fledely
TODO.ntfsprogs conatins the following TODO item under mkntfs:
- We don't know what the real last sector is, thus we mark the volume
dirty and the subsequent chkdsk (which will happen on reboot into Windows automatically) recreates the backup boot sector if the Linux kernel lied to us about the number of sectors.
ntfsresize, ntfsclone and others have the same problem due to this kernel limitation.
Quoted 3 lines
Now that we know about the extended/legacy BIOS geometry, and
assuming kernel version 2.6.5+ Is it possible to finally get fixed, or totally unrelated?
I'm afraid unrelated. There are two main issues two consider if you want to fix this:
1) You must know the exact, _real_ partition size.
2) You must be able to access it.
Firstly, I don't think EDD tells anything about the partitions but maybe I'm wrong, I didn't have time to check it out.
Secondly the kernel doesn't always allow access to the last sector via the partition (longstanding kernel bug). We could access it via the full device (e.g. /dev/hda, etc) but that's quite messy, error-prone and this should be fixed in the _kernel_.
Other people are also disturbed by this for a long time,
- they can't make a full partition backup (dd)
- just like NTFS, the new partitioning format, GPT, also stores backup data in the last sectors.
There were attempts to fix this in 2.6 but I don't know if it was completed. Maybe somebody knows or have the time to investigate it?
Szaka
Quoted 5 lines · Szakacsits Szabolcs
Quoted 5 lines
TODO.ntfsprogs conatins the following TODO item under mkntfs:
- We don't know what the real last sector is, thus we mark the volume
dirty and the subsequent chkdsk (which will happen on reboot into Windows automatically) recreates the backup boot sector if the Linux kernel lied to us about the number of sectors.
The ioctl BLKGETSIZE64 will tell you the size (in bytes) of a block device.
Andries
Quoted 9 lines · Andries Brouwer
Quoted 5 lines · Szakacsits Szabolcs
Quoted 5 lines
TODO.ntfsprogs conatins the following TODO item under mkntfs:
- We don't know what the real last sector is, thus we mark the volume
dirty and the subsequent chkdsk (which will happen on reboot into Windows automatically) recreates the backup boot sector if the Linux kernel lied to us about the number of sectors.The ioctl BLKGETSIZE64 will tell you the size (in bytes) of a block device.
So will lseek() to SEEK_END, actually (both 2.4 and 2.6). And yes, last sector _is_ accessible for dd(1) et.al.
Quoted 9 lines · On Sat, 10 Apr 2004 [email protected] wrote:
Quoted 8 lines · Andries Brouwer
Quoted 5 lines · Szakacsits Szabolcs
Quoted 5 lines
TODO.ntfsprogs conatins the following TODO item under mkntfs:
- We don't know what the real last sector is, thus we mark the volume
dirty and the subsequent chkdsk (which will happen on reboot into Windows automatically) recreates the backup boot sector if the Linux kernel lied to us about the number of sectors.
Quoted 4 lines
Quoted 1 line
The ioctl BLKGETSIZE64 will tell you the size (in bytes) of a block device.
So will
lseek()toSEEK_END, actually (both 2.4 and 2.6). And yes, last sector _is_ accessible for dd(1) et.al.
In 2.6? Not for 2.4 when I tried (it wasn't the latest 2.4 kernel).
Szaka
Quoted 11 lines · Szakacsits Szabolcs
Quoted 9 lines · On Sat, 10 Apr 2004 [email protected] wrote:
Quoted 8 lines · Andries Brouwer
Quoted 5 lines · Szakacsits Szabolcs
Quoted 5 lines
TODO.ntfsprogs conatins the following TODO item under mkntfs:
- We don't know what the real last sector is, thus we mark the volume
dirty and the subsequent chkdsk (which will happen on reboot into Windows automatically) recreates the backup boot sector if the Linux kernel lied to us about the number of sectors.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
The ioctl BLKGETSIZE64 will tell you the size (in bytes) of a block device.
So will
lseek()toSEEK_END, actually (both 2.4 and 2.6). And yes, last sector _is_ accessible for dd(1) et.al.In 2.6? Not for 2.4 when I tried (it wasn't the latest 2.4 kernel).
2.4 logics around block size handling is broken; we probably could backport that series of patches, though. 2.6 simply sets block size to GCD(page size, device size), so we don't have to worry about all that crap.
Quoted 3 lines · On Sat, 10 Apr 2004 [email protected] wrote:
2.4 logics around block size handling is broken; we probably could backport that series of patches, though. 2.6 simply sets block size to
GCD(page size, device size), so we don't have to worry about all that crap.
That's great!
Just one question, in the most common cases the block size ends up between 512 and 4096 bytes. Depending on how this block size used, it can have a significant impact on performance (e.g. 512 vs 4096). Is this true or is it used to be performance independent?
Szaka
Quoted 4 lines · Szakacsits Szabolcs
Just one question, in the most common cases the block size ends up between 512 and 4096 bytes. Depending on how this block size used, it can have a significant impact on performance (e.g. 512 vs 4096). Is this true or is it used to be performance independent?
Resulting requests are immediately merged anyway. Yes, we get more bio sitting on top of the merged request; however, it's heavily IO-dominated and I would be surprised if you really saw any noticable overhead in that situation.
Quoted 11 lines · On Sat, 10 Apr 2004 [email protected] wrote:
Quoted 4 lines · Szakacsits Szabolcs
Just one question, in the most common cases the block size ends up between 512 and 4096 bytes. Depending on how this block size used, it can have a significant impact on performance (e.g. 512 vs 4096). Is this true or is it used to be performance independent?
Resulting requests are immediately merged anyway. Yes, we get more bio sitting on top of the merged request; however, it's heavily IO-dominated and I would be surprised if you really saw any noticable overhead in that situation.
Thanks, I'll test it in the near future unless somebody does it earlier.
I have my test stuff but I'm interested of you could suggest specific ones that might exhibit/trigger the overhead if it exists at all.
Szaka
Quoted 9 lines · On Sat, 10 Apr 2004 [email protected] wrote:
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines
Quoted 6 lines
TODO.ntfsprogs conatins the following TODO item under mkntfs:
- We don't know what the real last sector is, thus we mark the volume
dirty and the subsequent chkdsk (which will happen on reboot into Windows automatically) recreates the backup boot sector if the Linux kernel lied to us about the number of sectors.The ioctl BLKGETSIZE64 will tell you the size (in bytes) of a block device.
Unless kernel 2.4.1[567] used (they return the size in sectors) or no ioctl conflict with the unofficial but used BLKSETLASTSECT (the issue was also summarized at http://lwn.net/2001/0906/kernel.php3). The last one could (did?) corrupt NTFS (NTFS keeps metadata there) when one tried to get the device size by BLKGETSIZE64 ...
Unfortunately not many softwares get BLKGETSIZE64 right, but at least the latest fdisk (2.12a) and e2fsprogs (1.36-WIP) are ok AFAIS, although they workaround these issues differently.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
So will
lseek()toSEEK_END, actually (both 2.4 and 2.6). And yes, last sector _is_ accessible for dd(1) et.al.
I checked these for 2.4.25 and 2.6.5. In 2.4 the last odd sector is visible only by BLKGETSIZE64 and BLKGETSIZE, otherwise it can't be accessed, as we agreed later on.
But 2.6 is ok in all cases, size is seen correctly by BLKGETSIZE64, BLKGETSIZE and accessible by lseek.
I don't know how intrusive, risky would be the backport but I suspect it's not worth.
Thank you for clarifying the issue,
Szaka