IA64 ino_t incorrectly sized?
Hi there,
I've been doing some large filesystem testing on XFS, and noticed that several of our regression tests are failing. One particular problem area is xfsdump and its use of the getdents64 interface. I believe I've traced the problem back to the definition of ino_t (aka __kernel_ino_t) as an unsigned int in the platform-specific headers, and I suspect this should instead be an unsigned long on IA64. Here's the rationale:
- all of IA64 userspace (glibc) allows for 64 bit inode numbers; see /usr/include/bits/types.h in particular, and __ino_t which is used in the stat and getdents64 syscalls.
- when we come into the kernel via sys_getdents64, we call into filesystem specific code via fs/readdir.c::vfs_readdir and the readdir file operation.
- this passes a filldir_t routine into the fs, which the fs uses to fill in the getdents buffer for each directory entry.
- filldir64 from sys_getdents64 makes use of the linux_dirent64 structure internally (with a u64 d_ino) and is passed an ino_t as the inode number.
- so, when we fill in each directory entry we're silently chopping off the high 32 bits of the inode number (casting from the XFS 64 bit inode to the ino_t argument of the filldir routines), which causes problems for xfsdump as it is looking at the inode numbers in the getdents buffer returned from the kernel.
- I can see no reason to have this restriction on IA64, because userspace does allow for 64 bit inode numbers.
Does anyone know why the IA64 platform-specific ino_t definition is an int and not a long? Patch below fixes this problem for me but I wonder if there will be side-effects I haven't considered (i.e. was there a reason for making this 32 bits originally?). If not, could the IA64 maintainers push this patch around to the official kernel trees for me? (pretty please)
many thanks.
Signature · 14 lines
Nathan --- /usr/tmp/TmpDir.16879-0/linux/include/asm-ia64/posix_types.h_1.1 Wed Sep 17 16:04:09 2003 +++ linux/include/asm-ia64/posix_types.h Wed Sep 17 11:29:30 2003 @@ -11,7 +11,7 @@ */ typedef unsigned int __kernel_dev_t; -typedef unsigned int __kernel_ino_t; +typedef unsigned long __kernel_ino_t; typedef unsigned int __kernel_mode_t; typedef unsigned int __kernel_nlink_t; typedef long __kernel_off_t;
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Nathan" = Nathan Scott <[email protected]> writes:
Nathan> Does anyone know why the IA64 platform-specific ino_t Nathan> definition is an int and not a long? Patch below fixes this Nathan> problem for me but I wonder if there will be side-effects I Nathan> haven't considered (i.e. was there a reason for making this 32 Nathan> bits originally?). If not, could the IA64 maintainers push Nathan> this patch around to the official kernel trees for me? Nathan> (pretty please)
Hi Nathan,
I am actually surprised it's still a 32 bit int in the kernel. I deliberately used 64 bit types in glibc so it could be done right. Must have slipped on fixing the kernel for this one.
David?
Cheers,
Jes
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines · Jes Sorensen
"Nathan" = Nathan Scott <[email protected]> writes:
Nathan> Does anyone know why the IA64 platform-specific ino_t Nathan> definition is an int and not a long? Patch below fixes this Nathan> problem for me but I wonder if there will be side-effects I Nathan> haven't considered (i.e. was there a reason for making this Nathan> 32 bits originally?). If not, could the IA64 maintainers Nathan> push this patch around to the official kernel trees for me? Nathan> (pretty please)
Jes> Hi Nathan,
Jes> I am actually surprised it's still a 32 bit int in the Jes> kernel. I deliberately used 64 bit types in glibc so it could Jes> be done right. Must have slipped on fixing the kernel for this Jes> one.
Jes> David?
Extending ino_t to 64 bits came up last October [1]. AFAIK, nobody bothered to investigate & send a patch, so things didn't change since then.
--david
Quoted 20 lines · David Mosberger
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines · Jes Sorensen
"Nathan" = Nathan Scott <[email protected]> writes:
Nathan> Does anyone know why the IA64 platform-specific
ino_tNathan> definition is an int and not a long? Patch below fixes this Nathan> problem for me but I wonder if there will be side-effects I Nathan> haven't considered (i.e. was there a reason for making this Nathan> 32 bits originally?). If not, could the IA64 maintainers Nathan> push this patch around to the official kernel trees for me? Nathan> (pretty please)Jes> Hi Nathan,
Jes> I am actually surprised it's still a 32 bit int in the Jes> kernel. I deliberately used 64 bit types in glibc so it could Jes> be done right. Must have slipped on fixing the kernel for this Jes> one.
Jes> David?
Quoted 8 lines
Extending
ino_tto 64 bits came up last October [1]. AFAIK, nobody bothered to investigate & send a patch, so things didn't change since then.--david
I notice a big batch of IA64 changes has just gone into 2.6-test6, but this change seems to be missing. Is it in someones queue for next time or do I need to describe the problem more clearly?
The investigation and the patch I sent are available here[2].
cheers.
Signature · 3 lines
Nathan [2] http://www.gelato.unsw.edu.au/linux-ia64/0309/6681.html
Quoted 9 lines
Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line · Nathan Scott
Extending
ino_tto 64 bits came up last October [1]. AFAIK,
nobody bothered to investigate & send a patch, so things didn't change since then.--david
Nathan> I notice a big batch of IA64 changes has just gone into Nathan> 2.6-test6, but this change seems to be missing. Is it in Nathan> someones queue for next time or do I need to describe the Nathan> problem more clearly?
Nathan> The investigation and the patch I sent are available Nathan> here[2].
Nathan> cheers.
Nathan> -- Nathan
Nathan> [2] http://www.gelato.unsw.edu.au/linux-ia64/0309/6681.html
The mail you're referring to talks about getdents64() only. What I didn't see is an argument why this is the _only_ user visible interface that might be affected. Are there any other interfaces (syscalls, /proc/whatever, etc.) that may directly or indirectly be affected? If not and if the change has been run through a reasonable test-suite (LTP?) without ill effects, I'm certainly OK with the change.
--david
PS: It would help to get this resolved quickly, e.g., before test8.
hi David,
Quoted 6 lines · David Mosberger
The mail you're referring to talks about
getdents64()only. What I didn't see is an argument why this is the _only_ user visible interface that might be affected. Are there any other interfaces (syscalls,/proc/whatever, etc.) that may directly or indirectly be affected?
The problem is not the interfaces - they tend to use "long" directly and will be unaffected. When I started running our tests with large inode numbers, for the most part everything just worked. Thats because, as Jes said, the IA64 userspace (libc, etc) is already 64 bit clean, and inode numbers are a long (directly, not via ino_t) in much of the kernel.
The problem is the ia64 kernel sometimes chops off the high 32 bits of an inode number for no good reason. Note that ino_t is not the kernels ubiquitous inode number representation, its only used for a subset of the internal interfaces and not for kernel/user interfaces directly.
e.g. the ia64 include/asm-ia64/stat.h uses long for the inode number field, and the struct inode i_ino is a long also. So, for the most part everything works as is - there are corner cases (i.e. getdents) that this fixes though, as I explained. It does not change any external interfaces (if it did they'd be broken already, but I haven't seen any evidence of that in my testing).
Quoted 3 lines
If not and if the change has been run through a reasonable test-suite (LTP?) without ill effects, I'm certainly OK with the change.
With this fix, the XFS test suite passes on filesystems with all inode numbers forced into the >2^32 range. This includes a large amount of filesystem/vfs stress covering all fs ops.
Without the fix, getdents and stat can report different inode numbers for the same file, even though both structures have sufficient bits for the correct inode number.
cheers.
ps: all of the above holds for 2.4 as well, the same patch will apply cleanly there.
Signature · 1 line
Nathan
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Nathan Scott
Nathan> The problem is not the interfaces - they tend to use "long" Nathan> directly and will be unaffected. When I started running our Nathan> tests with large inode numbers, for the most part everything Nathan> just worked.
Argh, of course it's about interfaces. We don't want to break something unknowingly because of this change. Saying that "for the most part everything just worked" isn't exactly reassuring!
Nathan> e.g. the ia64 include/asm-ia64/stat.h uses long for the Nathan> inode number field, and the struct inode i_ino is a long Nathan> also. So, for the most part everything works as is - there Nathan> are corner cases (i.e. getdents) that this fixes though, as Nathan> I explained. It does not change any external interfaces (if Nathan> it did they'd be broken already, but I haven't seen any Nathan> evidence of that in my testing).
Are you saying that you reviewed the entire syscall interface and did a reasonable review of anything else (e.g., /proc) that might expose ino_t in one way or another?
--david
Quoted 9 lines · David Mosberger
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Nathan Scott
Nathan> The problem is not the interfaces - they tend to use "long" Nathan> directly and will be unaffected. When I started running our Nathan> tests with large inode numbers, for the most part everything Nathan> just worked.
Argh, of course it's about interfaces. We don't want to break something unknowingly because of this change.
Yes, I understand that.
Quoted 2 lines
Saying that "for the
most part everything just worked" isn't exactly reassuring!
You have misunderstood and/or I have not been clear enough.
"When I started running tests"... i.e. before I fixed this issue in asm-ia64/posix_types.h ... "for the most part everything just worked"... i.e. most stuff worked before, but not the thing I fixed; but now it is fixed, and all our tests pass now, and I know of no remaining issues.
I did not review every possible interface for use of ino_t, no. I suppose that needs to be done, I don't have that kind of time available just now.
cheers.
Signature · 1 line
Nathan
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Nathan Scott
Nathan> I did not review every possible interface for use of ino_t, Nathan> no. I suppose that needs to be done, I don't have that kind Nathan> of time available just now.
Well, then don't be surprised if it doesn't get fixed!
--david
Quoted 7 lines · David Mosberger
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Nathan Scott
Nathan> I did not review every possible interface for use of
ino_t, Nathan> no. I suppose that needs to be done, I don't have that kind Nathan> of time available just now.Well, then don't be surprised if it doesn't get fixed!
s/fixed/merged/
I'm not surprised, and I can understand your position. I'll come back to do an audit when I can, if noone beats me to it in the meantime.
cheers.
Signature · 1 line
Nathan
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Nathan Scott
Quoted 1 line · David Mosberger
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Nathan Scott
Nathan> I did not review every possible interface for use of ino_t, Nathan> no. I suppose that needs to be done, I don't have that kind Nathan> of time available just now.
Quoted 1 line
Quoted 1 line
Well, then don't be surprised if it doesn't get fixed!
Nathan> s/fixed/merged/
Nathan> I'm not surprised, and I can understand your position. I'll Nathan> come back to do an audit when I can, if noone beats me to it Nathan> in the meantime.
I doubt anyone will beat you to it. I don't think the audit should take a huge amount of time. Perhaps half to a full day, just to make sure we're not missing something subtle (such as ioctl()s or something like that).
Also, note that the time window of making this change for 2.6 is closing quickly. Once 2.6.0 is out, it would be 2.7 material.
--david
Quoted 8 lines · David Mosberger
Nathan> I'm not surprised, and I can understand your position. I'll Nathan> come back to do an audit when I can, if noone beats me to it Nathan> in the meantime.
I doubt anyone will beat you to it. I don't think the audit should take a huge amount of time. Perhaps half to a full day, just to make sure we're not missing something subtle (such as ioctl()s or something like that).
I've started the audit, but I'm not finished yet & gotta run (short vacation, sorry wife says no laptop allowed). I think I have found two possible issues, I will send out an updated patch next week.
cheers.
Signature · 1 line
Nathan
hi David,
[ Below are updated patches. I've added Christoph and Andrew onto the discussion because the patches add __ia64__ ifdef's into core kernel headers, and they've shown me cleaner ways to do this sort of thing in the past. ]
Quoted 9 lines · David Mosberger
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Nathan Scott
Nathan> e.g. the ia64
include/asm-ia64/stat.huses long for the Nathan> inode number field, and the struct inodei_inois a long Nathan> also. So, for the most part everything works as is - there Nathan> are corner cases (i.e. getdents) that this fixes though, as Nathan> I explained. It does not change any external interfaces (if Nathan> it did they'd be broken already, but I haven't seen any Nathan> evidence of that in my testing).
Quoted 3 lines
Are you saying that you reviewed the entire syscall interface and did a reasonable review of anything else (e.g., /proc) that might expose
ino_tin one way or another?
So, my brave assertion that no code would be exporting a type called "__kernel_ino_t" turns out to be incorrect (/me hangs head in shame) as there are two cases I see where this type is exported to userspace.
Here's some more details about what I've reviewed and found. If I've overlooked any areas let me know.
system call interface -- I examined the 2.4 IA64 system call table and each of the structures passed across it in detail. This revealed that the ustat and NFS system calls pass around binary structures with __kernel_ino_t fields (see my updated patches). I then diff'd the 2.4 and 2.6 asm-ia64/unistd.h and reviewed each of the new syscalls - there are no new 2.6 interfaces that deal with an ino_t.
procfs interfaces -- the one procfs interface I found dealing with inode numbers is the /proc/PID/maps file. In 2.4, this is done in fs/proc/array.c::proc_pid_maps_get_line and in 2.6 its done in fs/proc/task_mmu.c::show_map. In both places, the ino is copied out of (struct inode)->i_ino into a local unsigned long and then formatted via "%lu". Hence, ino_t doesn't come into the picture here and we are not exposed to __kernel_ino_t changes in either kernel.
ioctl interfaces -- there are several places in arch/ia64/ia32/ ia32_ioctl.c and sys_ia32.c where we copy an ino_t into an unsigned int...
sys_ia32.c::filldir32 sys_ia32.c::fillonedir32 ia32_ioctl.c::put_dirent32
These would change from copying 'unsigned int'->'unsigned int' into 'unsigned long'->'unsigned int'. It turns out that this is already being done in other places, so I think its OK; e.g. sys_ia32.c::putstat copies a kbuf->st_ino (unsigned long, from the IA64 version of struct stat) into an ubuf->st_ino (uint).
From reviewing the code with Keith Owens, it looks like this copies the first 4 bytes (sizeof the specified field in the user buffer) and we've "got lucky" here due to our endianness (otherwise we would get all zeroes). Not sure if this was by luck or by design, but obviously it works as is.
Anyway, here's the two patches (2.6 & 2.4). The 2.4 version no longer applies cleanly to 2.6 due to other kernel changes, and the 2.6 kernel deprecates one of the 2.4 NFS structures.
I'd appreciate any suggestions about a cleaner way to do the common types.h and nfsd/syscall.h header changes.
cheers.
Signature · 1 line
Nathan
===========================================================================linux/include/asm-ia64/posix_types.h
===========================================================================
include/asm-ia64/posix_types.h renamed from usr/tmp/TmpDir.2032422-0/linux/include/asm-ia64/posix_types.h_1.4 +1 -1
@@ -10,7 +10,7 @@* David Mosberger-Tang <[email protected]> */ -typedef unsigned int __kernel_ino_t ; +typedef unsigned long __kernel_ino_t ;typedef unsigned int __kernel_mode_t ;typedef unsigned int __kernel_nlink_t ;typedef long __kernel_off_t ;
===========================================================================linux/include/linux/nfsd/syscall.h
===========================================================================
include/linux/nfsd/syscall.h renamed from usr/tmp/TmpDir.2032422-0/linux/include/linux/nfsd/syscall.h_1.11 +4 -0
@@ -60,7 +60,11 @@char ex_client[NFSCLNT_IDMAX +1 ];char ex_path[NFS_MAXPATHLEN +1 ];__kernel_old_dev_t ex_dev; +#ifdef __ia64__ +unsigned int ex_ino;/* ABI preservation */ +#else __kernel_ino_t ex_ino; +#endif int ex_flags;__kernel_uid_t ex_anon_uid;__kernel_gid_t ex_anon_gid;
===========================================================================linux/include/linux/types.h
===========================================================================
include/linux/types.h renamed from usr/tmp/TmpDir.2032422-0/linux/include/linux/types.h_1.14 +4 -0
@@ -150,7 +150,11 @@struct ustat {__kernel_daddr_t f_tfree; +#ifdef __ia64__ +unsigned int f_tinode;/* ABI preservation */ +#else __kernel_ino_t f_tinode; +#endif char f_fname[6 ];char f_fpack[6 ]; };
===========================================================================linux/include/asm-ia64/posix_types.h
===========================================================================
include/asm-ia64/posix_types.h renamed from usr/tmp/TmpDir.14060-0/linux/include/asm-ia64/posix_types.h_1.1 +1 -1
@@ -11,7 +11,7 @@*/ typedef unsigned int __kernel_dev_t ; -typedef unsigned int __kernel_ino_t ; +typedef unsigned long __kernel_ino_t ;typedef unsigned int __kernel_mode_t ;typedef unsigned int __kernel_nlink_t ;typedef long __kernel_off_t ;
===========================================================================linux/include/linux/nfsd/syscall.h
===========================================================================
include/linux/nfsd/syscall.h renamed from usr/tmp/TmpDir.14060-0/linux/include/linux/nfsd/syscall.h_1.6 +8 -0
@@ -60,7 +60,11 @@char ex_client[NFSCLNT_IDMAX +1 ];char ex_path[NFS_MAXPATHLEN +1 ];__kernel_dev_t ex_dev; +#ifdef __ia64__ +unsigned int ex_ino;/* ABI preservation */ +#else __kernel_ino_t ex_ino; +#endif int ex_flags;__kernel_uid_t ex_anon_uid;__kernel_gid_t ex_anon_gid;
@@ -81,7 +85,11 @@struct nfsctl_fhparm {struct sockaddr gf_addr;__kernel_dev_t gf_dev; +#ifdef __ia64__ +unsigned int gf_ino;/* ABI preservation */ +#else __kernel_ino_t gf_ino; +#endif int gf_version; };
===========================================================================linux/include/linux/types.h
===========================================================================
include/linux/types.h renamed from usr/tmp/TmpDir.14060-0/linux/include/linux/types.h_1.9 +4 -0
@@ -129,7 +129,11 @@struct ustat {__kernel_daddr_t f_tfree; +#ifdef __ia64__ +unsigned int f_tinode;/* ABI preservation */ +#else __kernel_ino_t f_tinode; +#endif char f_fname[6 ];char f_fpack[6 ]; };
Quoted 7 lines · Nathan Scott
Anyway, here's the two patches (2.6 & 2.4). The 2.4 version no longer applies cleanly to 2.6 due to other kernel changes, and the 2.6 kernel deprecates one of the 2.4 NFS structures.
I'd appreciate any suggestions about a cleaner way to do the common
types.handnfsd/syscall.hheader changes.
Can't think of anything clever really.
Hi Nathan,
Thanks for doing the thorough analysis!
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Nathan Scott
Nathan> system call interface -- I examined the 2.4 IA64 system call Nathan> table and each of the structures passed across it in detail. Nathan> This revealed that the ustat and NFS system calls pass around Nathan> binary structures with __kernel_ino_t fields (see my updated Nathan> patches). I then diff'd the 2.4 and 2.6 asm-ia64/unistd.h Nathan> and reviewed each of the new syscalls - there are no new 2.6 Nathan> interfaces that deal with an ino_t.
Those are nasty. I suppose your patch works, but wouldn't it mean that NFS-export and/or ustat() of XFS file systems would fail?
ustat() we could handle pretty easily, by introducing a new syscall number which uses the "long" ino_t. Then we just write a small wrapper for the old ustat() syscall.
NFS is nastier (ugly multiplexed syscalls...). I suppose it could be handled by incrementing NFSCTL_VERSION, but I'm not sure of all the implications of doing that (probably not what we want). Actually, I think we could handle NFS the same way as ustat(): change the ino_t to long and then have a wrapper for the old nfsctl() syscall which passes through everything except for some nfsctl_export munging.
Do you want to try this? If you do, you could use syscall numbers 1259 and 1260. I don't expect/hope that other syscalls will be added this late in the game.
--david
Quoted 2 lines · David Mosberger
NFS is nastier (ugly multiplexed syscalls...).
Well I think that whole interface is deprecated anyway and the new nfsutils doesn't use it - it uses exportfs instead.
You'd need to check with Neil, but if I'm right you could just say "use current nfsutils on ia64".
Quoted 3 lines · David Mosberger
Hi Nathan,
Thanks for doing the thorough analysis!
No problem. Thanks for the prodding. ;-)
Quoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Nathan Scott
Nathan> system call interface -- I examined the 2.4 IA64 system call Nathan> table and each of the structures passed across it in detail. Nathan> This revealed that the ustat and NFS system calls pass around Nathan> binary structures with
__kernel_ino_tfields (see my updated Nathan> patches). I then diff'd the 2.4 and 2.6asm-ia64/unistd.hNathan> and reviewed each of the new syscalls - there are no new 2.6 Nathan> interfaces that deal with anino_t.
Quoted 2 lines
Those are nasty. I suppose your patch works, but wouldn't it mean that NFS-export and/or
ustat()of XFS file systems would fail?
It turns out that neither is a problem for us in practice.
In the case of ustat(2) ...
ino_t f_tinode; /* Number of free inodes */
is meaningless on those filesystems (like XFS) which don't allocate a fixed set of inodes at mkfs time. It's only an "ino_t" for hysterical raisins too (the count of free inodes? != an inode number!) - I notice IRIX defines this in exactly the same way, I guess this came from SVR4 verbatim. I'm hard pressed finding an application that uses this, and I was quite surprised to find it in Linux at all.
For the NFS case - I asked one of the local NFS gurus to look over the changes yesterday, and he tells me that field is only used to hold the root inode number of a filesystem. So, for XFS (and I'd imagine most other filesystems too) this is never going to be a problem - the root never has a large inode number in XFS because it's allocated at mkfs time and always from the first allocation group (where inode numbers are small).
Quoted 3 lines
...
1259 and 1260. I don't expect/hope that other syscalls will be added this late in the game.
No, nor do I. I don't think we should go that far though, certainly it seems all our needs in XFS will be met without adding these calls.
cheers.
Signature · 1 line
Nathan
Quoted 11 lines · Andrew Morton
Quoted 7 lines · Nathan Scott
Anyway, here's the two patches (2.6 & 2.4). The 2.4 version no longer applies cleanly to 2.6 due to other kernel changes, and the 2.6 kernel deprecates one of the 2.4 NFS structures.
I'd appreciate any suggestions about a cleaner way to do the common
types.handnfsd/syscall.hheader changes.Can't think of anything clever really.
One alternate approach would be to introduce __kernel_ustatino_t and __kernel_nfsdino_t into every architecture's posix_types.h (defining them as __kernel_ino_t everywhere but IA64, where they would be unsigned int). This would do away with any conditional code in headers, but I'm not sure its the right approach at this stage?
cheers.
Signature · 1 line
Nathan
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Nathan Scott
Nathan> It turns out that neither is a problem for us in practice.
Sounds find to me, then. Except, I'd replace #ifdef __ia64__ with #ifdef CONFIG_IA64, so you're relying (less) on compiler magic.
--david
Quoted 7 lines · Nathan Scott
One alternate approach would be to introduce
__kernel_ustatino_tand__kernel_nfsdino_tinto every architecture'sposix_types.h(defining them as__kernel_ino_teverywhere but IA64, where they would be unsigned int). This would do away with any conditional code in headers, but I'm not sure its the right approach at this stage?
Sounds overly elaborate to me. The patch which you had looked liveable-with.
Quoted 7 lines · David Mosberger
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Nathan Scott
Nathan> It turns out that neither is a problem for us in practice.
Sounds find to me, then. Except, I'd replace #ifdef
__ia64__with #ifdefCONFIG_IA64, so you're relying (less) on compiler magic.
Oh. I had avoided that because it requires any sources including these headers to have already included linux/config.h, which they may not be doing. linux/types.h is included by userspace code too, I believe, so may be an issue there too.
cheers.
Signature · 1 line
Nathan
Review
Quoted 6 lines
(quoted diff) +4
-0__kernel_old_dev_t ex_dev; +#ifdef __ia64__ + unsigned int ex_ino; /* ABI preservation */ +#else __kernel_ino_t ex_ino; +#endif
Please don't make it #ifdef __ia64__, but some other define that other architectures who may want to move the 64bit ino_t too can just set.
-Andi
Quoted 2 lines · Nathan Scott
I'd appreciate any suggestions about a cleaner way to do the common
types.handnfsd/syscall.hheader changes.
First question: What about other LP64 arches besides IA64. IMHO having differen't ino_t on different LP64 arches is a rather bad idea.
AQnd if we go the ino_t is unsigned long route for all 64bit arches the ifdefs would also magically disappear..
Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Nathan Scott
I'd appreciate any suggestions about a cleaner way to do the common
types.handnfsd/syscall.hheader changes.First question: What about other LP64 arches besides IA64. IMHO having differen't
ino_ton different LP64 arches is a rather bad idea.
All architectures typedef __kernel_ino_t to unsigned long except alpha, ia64 and s390x. I guess ia64 just copied the wrong architecture at the beginning.
Quoted 2 lines
AQnd if we go the
ino_tis unsigned long route for all 64bit arches the ifdefs would also magically disappear..
Sounds like a good plan.
Signature · 3 lines
"It's not Hollywood. War is real, war is primarily not about defeat or victory, it is about death. I've seen thousands and thousands of dead bodies. Do you think I want to have an academic debate on this subject?" -- Robert Fisk
Quoted 11 lines · Matthew Wilcox
Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Nathan Scott
I'd appreciate any suggestions about a cleaner way to do the common
types.handnfsd/syscall.hheader changes.First question: What about other LP64 arches besides IA64. IMHO having differen't
ino_ton different LP64 arches is a rather bad idea.All architectures typedef
__kernel_ino_tto unsigned long except alpha, ia64 and s390x. I guess ia64 just copied the wrong architecture at the beginning.
So we should probably change introduce __HAVE_BROKEN_INO_T for alpha and s390x? Even better those should get new versions of the relevant syscalls, I expect at least s390x to want them one day.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Nathan Scott
Nathan> On Tue, Oct 14, 2003 at 11:21:09PM -0700, David Mosberger
Quoted 2 lines · Nathan> wrote:
Quoted 2 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Wed, 15 Oct 2003 16:06:02 +1000, Nathan Scott
<[email protected]> said:
Nathan> It turns out that neither is a problem for us in practice.
Quoted 3 lines
Quoted 3 lines
Sounds find to me, then. Except, I'd replace #ifdef
__ia64__
with #ifdefCONFIG_IA64, so you're relying (less) on compiler magic.
Nathan> Oh. I had avoided that because it requires any sources Nathan> including these headers to have already included Nathan> linux/config.h, which they may not be doing. linux/types.h Nathan> is included by userspace code too, I believe, so may be an Nathan> issue there too.
OK.
Upon further investigation, I found that glibc defines its own "struct ustat" and, guess what, it already declare __ino_t as unsigned long:
(gdb) ptype struct ustat
type = struct ustat {
__daddr_t f_tfree;
__ino_t f_tinode;
char f_fname[6];
char f_fpack[6]; }
(gdb) ptype __ino_t
type = long unsigned int
So there is no need to have that ugly #ifdef for struct ustat.
I'm not 100% sure yet what to do about NFS.
--david
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Andrew Morton
Quoted 1 line · Andrew> David Mosberger <[email protected]> wrote:
Quoted 1 line
NFS is nastier (ugly multiplexed syscalls...).
Andrew> Well I think that whole interface is deprecated anyway and Andrew> the new nfsutils doesn't use it - it uses exportfs instead.
Andrew> You'd need to check with Neil, but if I'm right you could Andrew> just say "use current nfsutils on ia64".
Ugh, looks like there may be some problems with exportfs itself? I see:
static struct dentry *get_object(struct super_block *sb, void *vobjp)
{
__u32 *objp = vobjp;
unsigned long ino = objp[0];
__u32 generation = objp[1];
return export_iget(sb, ino, generation);
}So it looks to me like exportfs supports only 32-bit ino_t?
(Neil, to give you some background: we'd like to change ino_t on ia64 from 32 to 64 bits and found that the only potential ABI issue is due to NFS; "struct nfsctl_export" is definitely an issue, but perhaps we can live with that. I'm less certain about any issues exportfs might have.)
--david
Nathan,
Based on the previous discussions, I applied your patch to change ino_t to "unsigned long". This fixes ustat() (for glibc) and breaks nfsctl_export, but with reasonably recent nfsutils, that broken call won't be used and exportfs seems to work fine (at least I was able to mount an NFS filesystem from the patched box just fine).
Thanks,
--david
Quoted 24 lines · On Wednesday October 15, [email protected] wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Andrew Morton
Quoted 1 line · Andrew> David Mosberger <[email protected]> wrote:
Quoted 1 line
NFS is nastier (ugly multiplexed syscalls...).
Andrew> Well I think that whole interface is deprecated anyway and Andrew> the new nfsutils doesn't use it - it uses exportfs instead.
Andrew> You'd need to check with Neil, but if I'm right you could Andrew> just say "use current nfsutils on ia64".
Ugh, looks like there may be some problems with exportfs itself? I see:
static struct dentry *get_object(struct
super_block*sb, void *vobjp){ __u32 *objp = vobjp; unsigned long ino = objp[0]; __u32 generation = objp[1]; return export_iget(sb, ino, generation); }
Quoted 1 line
So it looks to me like exportfs supports only 32-bit
ino_t?
Looks can be deceiving (especially when those /* */ operators haven't been used correctly).
get_object is part of the legacy support in exportfs, where legacy really means ext2/ext3.
Any filesystem that actually used 64bit inode numbers would need to define it's own export_operations->get_dentry function which can interpret the file handle however it likes, including using 64 bits of it for an inode number (though such a filesystem probably wouldn't work with NFSv2, but that's OK).
Quoted 6 lines
(Neil, to give you some background: we'd like to change
ino_ton ia64 from 32 to 64 bits and found that the only potential ABI issue is due to NFS; "structnfsctl_export" is definitely an issue, but perhaps we can live with that. I'm less certain about any issues exportfs might have.)
ex_ino in nfsctl_export is not actually used, so all that is needed is to make sure user_space and kernel_space agree on the side of the field.
Maybe a __kernel_old_ino_t or soemthing.
But I would be quite happy to #ifdef out that system call for ia64 if that was preferred.
So in short: there is no big problem, and the small problem is largely
one of aesthetics.
NeilBrown
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line · Neil Brown
So it looks to me like exportfs supports only 32-bit
ino_t?
Neil> Looks can be deceiving (especially when those /* */ operators Neil> haven't been used correctly).
Neil> get_object is part of the legacy support in exportfs, where Neil> legacy really means ext2/ext3. Any filesystem that actually Neil> used 64bit inode numbers would need to define it's own Neil> export_operations->get_dentry function which can interpret the Neil> file handle however it likes, including using 64 bits of it Neil> for an inode number (though such a filesystem probably Neil> wouldn't work with NFSv2, but that's OK).
Ah, that sounds good then.
Quoted 5 lines
Quoted 5 lines
(Neil, to give you some background: we'd like to change
ino_ton
ia64 from 32 to 64 bits and found that the only potential ABI issue is due to NFS; "structnfsctl_export" is definitely an issue, but perhaps we can live with that. I'm less certain about any issues exportfs might have.)
Neil> ex_ino in nfsctl_export is not actually used, so all that is Neil> needed is to make sure user_space and kernel_space agree on Neil> the side of the field. Maybe a __kernel_old_ino_t or Neil> soemthing. But I would be quite happy to #ifdef out that Neil> system call for ia64 if that was preferred.
Neil> So in short: there is no big problem, and the small problem is Neil> largely one of aesthetics.
OK, perhaps for now you could use Nathan's #ifdef __ia64__? Something along the lines of __kernel_old_ino_t certainly would be cleaner, if other arches need something similar.
--daviad
Hi all,
Quoted 12 lines · David Mosberger
Quoted 5 lines
Quoted 5 lines
(Neil, to give you some background: we'd like to change
ino_ton
ia64 from 32 to 64 bits and found that the only potential ABI issue is due to NFS; "structnfsctl_export" is definitely an issue, but perhaps we can live with that. I'm less certain about any issues exportfs might have.)Neil>
ex_inoinnfsctl_exportis not actually used, so all that is Neil> needed is to make sureuser_spaceandkernel_spaceagree on Neil> the side of the field. Maybe a__kernel_old_ino_tor Neil> soemthing. But I would be quite happy to #ifdef out that Neil> system call for ia64 if that was preferred.
Quoted 7 lines
Neil> So in short: there is no big problem, and the small problem is Neil> largely one of aesthetics.
OK, perhaps for now you could use Nathan's #ifdef
__ia64__? Something along the lines of__kernel_old_ino_tcertainly would be cleaner, if other arches need something similar.
What's your preference here Neil? I'll make a patch to create __kernel_old_ino_t for each arch and nfsd/syscall.h if you like that approach, else these patches seem to be all we need for ABI compatibility now.
thanks.
Signature · 1 line
Nathan
===========================================================================linux/include/asm-ia64/posix_types.h
===========================================================================
include/asm-ia64/posix_types.h renamed from usr/tmp/TmpDir.2032422-0/linux/include/asm-ia64/posix_types.h_1.4 +1 -1
@@ -10,7 +10,7 @@* David Mosberger-Tang <[email protected]> */ -typedef unsigned int __kernel_ino_t ; +typedef unsigned long __kernel_ino_t ;typedef unsigned int __kernel_mode_t ;typedef unsigned int __kernel_nlink_t ;typedef long __kernel_off_t ;
===========================================================================linux/include/linux/nfsd/syscall.h
===========================================================================
include/linux/nfsd/syscall.h renamed from usr/tmp/TmpDir.2032422-0/linux/include/linux/nfsd/syscall.h_1.11 +4 -0
@@ -60,7 +60,11 @@char ex_client[NFSCLNT_IDMAX +1 ];char ex_path[NFS_MAXPATHLEN +1 ];__kernel_old_dev_t ex_dev; +#ifdef __ia64__ +unsigned int ex_ino;/* ABI preservation */ +#else __kernel_ino_t ex_ino; +#endif int ex_flags;__kernel_uid_t ex_anon_uid;__kernel_gid_t ex_anon_gid;
===========================================================================linux/include/asm-ia64/posix_types.h
===========================================================================
include/asm-ia64/posix_types.h renamed from usr/tmp/TmpDir.14060-0/linux/include/asm-ia64/posix_types.h_1.1 +1 -1
@@ -11,7 +11,7 @@*/ typedef unsigned int __kernel_dev_t ; -typedef unsigned int __kernel_ino_t ; +typedef unsigned long __kernel_ino_t ;typedef unsigned int __kernel_mode_t ;typedef unsigned int __kernel_nlink_t ;typedef long __kernel_off_t ;
===========================================================================linux/include/linux/nfsd/syscall.h
===========================================================================
include/linux/nfsd/syscall.h renamed from usr/tmp/TmpDir.14060-0/linux/include/linux/nfsd/syscall.h_1.6 +8 -0
@@ -60,7 +60,11 @@char ex_client[NFSCLNT_IDMAX +1 ];char ex_path[NFS_MAXPATHLEN +1 ];__kernel_dev_t ex_dev; +#ifdef __ia64__ +unsigned int ex_ino;/* ABI preservation */ +#else __kernel_ino_t ex_ino; +#endif int ex_flags;__kernel_uid_t ex_anon_uid;__kernel_gid_t ex_anon_gid;
@@ -81,7 +85,11 @@struct nfsctl_fhparm {struct sockaddr gf_addr;__kernel_dev_t gf_dev; +#ifdef __ia64__ +unsigned int gf_ino;/* ABI preservation */ +#else __kernel_ino_t gf_ino; +#endif int gf_version; };
Quoted 8 lines · On Friday October 17, [email protected] wrote:
Quoted 1 line
What's your preference here Neil? I'll make a patch to create
__kernel_old_ino_tfor each arch andnfsd/syscall.hif you like that approach, else these patches seem to be all we need for ABI compatibility now.thanks.
I don't really want __kernel_old_ino_t as it would clutter every arch just for nfsd, and just for 2.6.
I think I would like
#ifdef __ia64__
typedef unsigned int __nfsd_ino_t;
#else
typedef __kernel_ino_t __nfsd_ino_t;
#endif
at the top of syscall.h, and then use __nfsd_ino_t where needed. This would keep all the 'clutter' inside nfsd, and in one place, and I can remove it all when I discard that system call in 2.7.
If other arch's want to change their __kernel_ino_t, then we just need to add a make a change within that #if/#endif
NeilBrown
hi Neil,
Quoted 2 lines · Neil Brown
I don't really want
__kernel_old_ino_tas it would clutter every arch just for nfsd, and just for 2.6.
OK sounds good, although we'd like these changes in 2.4 too (eventually). Do these patches look acceptable?
thanks.
Signature · 1 line
Nathan
include/linux/nfsd/syscall.h renamed from usr/tmp/TmpDir.1899229-0/linux/include/linux/nfsd/syscall.h_1.11 +11 -1
@@ -22,6 +22,16 @@#include <linux/nfsd/auth.h> /* +* ABI compatibility type definition for the IA64 __kernel_ino_t. +* This can be removed along with the nfsctl_export syscall (2.7). +*/ +#ifdef __ia64__ +typedef unsigned int __nfsd_ino_t ; +#else +typedef __kernel_ino_t __nfsd_ino_t ; +#endif + +/* * Version of the syscall interface */ #define NFSCTL_VERSION 0x0201
@@ -60,7 +70,7 @@char ex_client[NFSCLNT_IDMAX +1 ];char ex_path[NFS_MAXPATHLEN +1 ];__kernel_old_dev_t ex_dev; -__kernel_ino_t ex_ino; +__nfsd_ino_t ex_ino;int ex_flags;__kernel_uid_t ex_anon_uid;__kernel_gid_t ex_anon_gid;
include/linux/nfsd/syscall.h renamed from usr/tmp/TmpDir.1959-0/linux/include/linux/nfsd/syscall.h_1.6 +13 -2
@@ -22,6 +22,17 @@#include <linux/nfsd/auth.h> /* +* ABI compatibility type definition for the IA64 __kernel_ino_t. +* This can be removed along with the nfsctl_export syscall (2.7) +* and the nfsctl_fhparm syscall (2.5). +*/ +#ifdef __ia64__ +typedef unsigned int __nfsd_ino_t ; +#else +typedef __kernel_ino_t __nfsd_ino_t ; +#endif + +/* * Version of the syscall interface */ #define NFSCTL_VERSION 0x0201
@@ -60,7 +71,7 @@char ex_client[NFSCLNT_IDMAX +1 ];char ex_path[NFS_MAXPATHLEN +1 ];__kernel_dev_t ex_dev; -__kernel_ino_t ex_ino; +__nfsd_ino_t ex_ino;int ex_flags;__kernel_uid_t ex_anon_uid;__kernel_gid_t ex_anon_gid;
@@ -81,7 +92,7 @@struct nfsctl_fhparm {struct sockaddr gf_addr;__kernel_dev_t gf_dev; -__kernel_ino_t gf_ino; +__nfsd_ino_t gf_ino;int gf_version; };
Quoted 9 lines · On Friday October 17, [email protected] wrote:
hi Neil,
Quoted 2 lines · Neil Brown
I don't really want
__kernel_old_ino_tas it would clutter every arch just for nfsd, and just for 2.6.OK sounds good, although we'd like these changes in 2.4 too (eventually). Do these patches look acceptable?
Yes, I am happy with those.
NeilBrown
Quoted 70 lines
thanks.
Signature · 1 line
Nathaninclude/linux/nfsd/syscall.h renamed from usr/tmp/TmpDir.1899229-0/linux/include/linux/nfsd/syscall.h_1.11 +11
-1@@ -22,6 +22,16 @@#include <linux/nfsd/auth.h> /* +* ABI compatibility type definition for the IA64 __kernel_ino_t. +* This can be removed along with the nfsctl_export syscall (2.7). +*/ +#ifdef __ia64__ +typedef unsigned int __nfsd_ino_t ; +#else +typedef __kernel_ino_t __nfsd_ino_t ; +#endif + +/* * Version of the syscall interface */ #define NFSCTL_VERSION 0x0201@@ -60,7 +70,7 @@char ex_client[NFSCLNT_IDMAX +1 ];char ex_path[NFS_MAXPATHLEN +1 ];__kernel_old_dev_t ex_dev; -__kernel_ino_t ex_ino; +__nfsd_ino_t ex_ino;int ex_flags;__kernel_uid_t ex_anon_uid;__kernel_gid_t ex_anon_gid;include/linux/nfsd/syscall.h renamed from usr/tmp/TmpDir.1959-0/linux/include/linux/nfsd/syscall.h_1.6 +13
-2@@ -22,6 +22,17 @@#include <linux/nfsd/auth.h> /* +* ABI compatibility type definition for the IA64 __kernel_ino_t. +* This can be removed along with the nfsctl_export syscall (2.7) +* and the nfsctl_fhparm syscall (2.5). +*/ +#ifdef __ia64__ +typedef unsigned int __nfsd_ino_t ; +#else +typedef __kernel_ino_t __nfsd_ino_t ; +#endif + +/* * Version of the syscall interface */ #define NFSCTL_VERSION 0x0201@@ -60,7 +71,7 @@char ex_client[NFSCLNT_IDMAX +1 ];char ex_path[NFS_MAXPATHLEN +1 ];__kernel_dev_t ex_dev; -__kernel_ino_t ex_ino; +__nfsd_ino_t ex_ino;int ex_flags;__kernel_uid_t ex_anon_uid;__kernel_gid_t ex_anon_gid;@@ -81,7 +92,7 @@struct nfsctl_fhparm {struct sockaddr gf_addr;__kernel_dev_t gf_dev; -__kernel_ino_t gf_ino; +__nfsd_ino_t gf_ino;int gf_version; };