Skip to contentSkip to conversation

64 bit driver 32 bit app ioctl

RFCLUE2 William D Waddington 4 participants 6 messages Mar 23, 2006 – Mar 24, 2006

lore
Share & tools
lore
#0 lore raw

Apologies for dashing this off without the proper homework. My customer is out of country doing an installation, and didn't test this configuration first :(

Customer is running RHEL3 on a 64 bit PC. Running the 64 bit kernel and my 64 bit driver. They are calling the driver from their 32 bit app. The driver supports a whole mess of ioctls.

It seems that the kernel is trapping the 32-bit ioctl call and returning an error to the app w/out calling the driver. It looks like register_ioctl32_conversion() can convice the kernel that the driver can handle 32-bit calls, but it has to be called for each ioctl cmd (??)

Putting aside (please) discussion of whether the kernel should presume to hijack private ioctls, and whether I should be using the ioctl interface at all (compatibility with app interface going back to 2.0 and SunOS) is there some way to make _one_ register call to indicate that all my cmds are safe, or maybe an alternate ioctl entry point that the kernel won't trap?

Yours in desperation,
Bill

Signature · 7 lines
--------------------------------------------
William D Waddington
Bainbridge Island, WA, USA
[email protected]
--------------------------------------------
"Even bugs...are unexpected signposts on
the long road of creativity..." - Ken Burtch
#1 lore raw
Quoted 4 lines · William D Waddington

Apologies for dashing this off without the proper homework. My customer is out of country doing an installation, and didn't test this configuration first :(

Quoted 8 lines

Customer is running RHEL3 on a 64 bit PC. Running the 64 bit kernel and my 64 bit driver. They are calling the driver from their 32 bit app. The driver supports a whole mess of ioctls.

It seems that the kernel is trapping the 32-bit ioctl call and returning an error to the app w/out calling the driver. It looks like register_ioctl32_conversion() can convice the kernel that the driver can handle 32-bit calls, but it has to be called for each ioctl cmd (??)

In these old pre-compat_ioctl kernels you have to register each ioctl command individually. Yes that sucks. Live with it.

Quoted 6 lines

Putting aside (please) discussion of whether the kernel should presume to hijack private ioctls, and whether I should be using the ioctl interface at all (compatibility with app interface going back to 2.0 and SunOS) is there some way to make _one_ register call to indicate that all my cmds are safe, or maybe an alternate ioctl entry point that the kernel won't trap?

Not as long as you're stuck with old 2.4 kernels. 2.6 kernels since 2.6.11-rc2 allow you to set up a single ->compat_ioctl() method, but not even RHEL4 has that yet.

#2 lore raw
Quoted 24 lines · Mikael Pettersson
Quoted 4 lines · William D Waddington

Apologies for dashing this off without the proper homework. My customer is out of country doing an installation, and didn't test this configuration first :(

Quoted 8 lines

Customer is running RHEL3 on a 64 bit PC. Running the 64 bit kernel and my 64 bit driver. They are calling the driver from their 32 bit app. The driver supports a whole mess of ioctls.

It seems that the kernel is trapping the 32-bit ioctl call and returning an error to the app w/out calling the driver. It looks like register_ioctl32_conversion() can convice the kernel that the driver can handle 32-bit calls, but it has to be called for each ioctl cmd (??)

In these old pre-compat_ioctl kernels you have to register each ioctl command individually. Yes that sucks. Live with it.

Quoted 6 lines

Putting aside (please) discussion of whether the kernel should presume to hijack private ioctls, and whether I should be using the ioctl interface at all (compatibility with app interface going back to 2.0 and SunOS) is there some way to make _one_ register call to indicate that all my cmds are safe, or maybe an alternate ioctl entry point that the kernel won't trap?

Quoted 3 lines

Not as long as you're stuck with old 2.4 kernels. 2.6 kernels since 2.6.11-rc2 allow you to set up a single ->compat_ioctl() method, but not even RHEL4 has that yet.

Thanks,

It's working OK in my test cases: FC1/64 and the customer's RHEL3. I just #include <asm/ioctl32.h> and register all my ioctls. Ugh.

The location of ioctl32.h seems to move from 2.4 kernel to kernel (and distro to distro??). Any suggestion how to include in a universal way and how to detect all the appropriate 64 bit configs for conditional inclusion w/a 2.4 kernel? I detest #ifdef'd code but I guess I have to do that too, or just keep one version around for this specific case :(

Thanks again,
Bill
--------------------------------------------
William D Waddington
Bainbridge Island, WA, USA
[email protected]
--------------------------------------------
"Even bugs...are unexpected signposts on
the long road of creativity..." - Ken Burtch

#3 lore raw
In reply to William D Waddington
Quoted 4 lines · William D Waddington

Apologies for dashing this off without the proper homework. My customer is out of country doing an installation, and didn't test this configuration first :(

Quoted 8 lines

Customer is running RHEL3 on a 64 bit PC. Running the 64 bit kernel and my 64 bit driver. They are calling the driver from their 32 bit app. The driver supports a whole mess of ioctls.

It seems that the kernel is trapping the 32-bit ioctl call and returning an error to the app w/out calling the driver. It looks like register_ioctl32_conversion() can convice the kernel that the driver can handle 32-bit calls, but it has to be called for each ioctl cmd (??)

you forgot to attach you code btw or post the url to it..

#4 lore raw
Quoted 6 lines · Arjan van de Ven

On Thu, 2006-03-23 at 07:06 -0800, William D Waddington wrote:

[...]
you forgot to attach you code btw or post the url to it..

is that your new .signature?

:)

Signature · 1 line
error compiling committee.c: too many arguments to function
#5 lore raw
In reply to Arjan van de Ven
Quoted 10 lines · Arjan van de Ven
Quoted 8 lines · William D Waddington

Apologies for dashing this off without the proper homework. My customer is out of country doing an installation, and didn't test this configuration first :(

Customer is running RHEL3 on a 64 bit PC. Running the 64 bit kernel and my 64 bit driver. They are calling the driver from their 32 bit app. The driver supports a whole mess of ioctls.

Quoted 7 lines
Quoted 4 lines

It seems that the kernel is trapping the 32-bit ioctl call and returning an error to the app w/out calling the driver. It looks like register_ioctl32_conversion() can convice the kernel that the driver can handle 32-bit calls, but it has to be called for each ioctl cmd (??)

you forgot to attach you code btw or post the url to it..

No I didn't :) It's just too ugly for public view. And I notice it needs some other fix-ups like fixed width data types in the ioctl routine...

Thanks to all for the info. I've got some typing to do.

Bill