Dealing with 2038
I'd like to discuss some thoughts on how to address the 2038 issues on 32bit architectures. This is important, as vendors are still producing lots of 32bit hardware, which may very well have 24+ year lifespans (think industrial control applications, security systems). NetBSD and OpenBSD have recently broken their ABI, converted their time_t to long long, to properly address this. So I'd like to discuss thoughts on how Linux can do similar despite our no-breaking-userspace rules, after all, one way or another (almost) all of Linux's 32bit architectures are terminally broken past 2038.
Thomas (who I don't think can attend due to other plans) and I have had some small talks about this and we have different initial preferences on how to go about solving things, so I'd like to present the pros and cons of the current options we're stewing on, open the discussion up to other ideas, and see if there's a consensus on which way to go.
thanks
-john
Quoted 10 lines · John Stultz
I'd like to discuss some thoughts on how to address the 2038 issues on 32bit architectures. This is important, as vendors are still producing lots of 32bit hardware, which may very well have 24+ year lifespans (think industrial control applications, security systems). NetBSD and OpenBSD have recently broken their ABI, converted their
time_tto long long, to properly address this. So I'd like to discuss thoughts on how Linux can do similar despite our no-breaking-userspace rules, after all, one way or another (almost) all of Linux's 32bit architectures are terminally broken past 2038.
Quoted 5 lines
Thomas (who I don't think can attend due to other plans) and I have had some small talks about this and we have different initial preferences on how to go about solving things, so I'd like to present the pros and cons of the current options we're stewing on, open the discussion up to other ideas, and see if there's a consensus on which way to go.
I'm interested in this discussion.
It would be nice to try to do something about the seconds, nanoseconds representation in clock_gettime at the same time. One way or another, a lot of programs will continue to expect to see seconds and nanoseconds, but it's simpler and faster to be able to feed userspace a nanosecond count.
Also, leap seconds :(
--Andy
Quoted 9 lines · John Stultz
I'd like to discuss some thoughts on how to address the 2038 issues on 32bit architectures. This is important, as vendors are still producing lots of 32bit hardware, which may very well have 24+ year lifespans (think industrial control applications, security systems). NetBSD and OpenBSD have recently broken their ABI, converted their
time_tto long long, to properly address this. So I'd like to discuss thoughts on how Linux can do similar despite our no-breaking-userspace rules, after all, one way or another (almost) all of Linux's 32bit architectures are terminally broken past 2038.
I would be interested in this, not just because of time_t itself, but as a general pattern for "how can we transition away from an old and broken ABI". Whether by introducing new system calls, new personalities, seccomp filters, or other mechanisms, we should have some ways to start such transitions and to smooth them out. Sure, we never break userspace, but that just means we need an appropriate CONFIG_OLD_AND_BUSTED option for as long as people still need the old ABI.
- Josh Triplett
Quoted 11 lines · On Mon, May 5, 2014 at 1:53 PM, <[email protected]> wrote:
Quoted 9 lines · John Stultz
I'd like to discuss some thoughts on how to address the 2038 issues on 32bit architectures. This is important, as vendors are still producing lots of 32bit hardware, which may very well have 24+ year lifespans (think industrial control applications, security systems). NetBSD and OpenBSD have recently broken their ABI, converted their
time_tto long long, to properly address this. So I'd like to discuss thoughts on how Linux can do similar despite our no-breaking-userspace rules, after all, one way or another (almost) all of Linux's 32bit architectures are terminally broken past 2038.
Quoted 8 lines
I would be interested in this, not just because of
time_titself, but as a general pattern for "how can we transition away from an old and broken ABI". Whether by introducing new system calls, new personalities, seccomp filters, or other mechanisms, we should have some ways to start such transitions and to smooth them out. Sure, we never break userspace, but that just means we need an appropriateCONFIG_OLD_AND_BUSTEDoption for as long as people still need the old ABI.
I encountered this with CONFIG_COMPAT_VDSO, and it would be nice if there were a clear, mostly-applicable rule for how this should work.
The current situation is that CONFIG_COMPAT_VDSO's name is silly, and it's 'default n', but the name is stuck so that old configurations keep being able to run old software.
--Andy
Quoted 4 lines · John Stultz
I'd like to discuss some thoughts on how to address the 2038 issues on 32bit architectures. This is important, as vendors are still producing lots of 32bit hardware, which may very well have 24+ year lifespans (think industrial control applications, security systems).
This is definitely true for some of our products, so we're aware of this issue and concerned about it.
Quoted 15 lines
NetBSD and
OpenBSD have recently broken their ABI, converted theirtime_tto long long, to properly address this. So I'd like to discuss thoughts on how Linux can do similar despite our no-breaking-userspace rules, after all, one way or another (almost) all of Linux's 32bit architectures are terminally broken past 2038.Thomas (who I don't think can attend due to other plans) and I have had some small talks about this and we have different initial preferences on how to go about solving things, so I'd like to present the pros and cons of the current options we're stewing on, open the discussion up to other ideas, and see if there's a consensus on which way to go.
thanks
-john
Quoted 10 lines · On 05/05/2014 01:53 PM, [email protected] wrote:
I would be interested in this, not just because of
time_titself, but as a general pattern for "how can we transition away from an old and broken ABI". Whether by introducing new system calls, new personalities, seccomp filters, or other mechanisms, we should have some ways to start such transitions and to smooth them out. Sure, we never break userspace, but that just means we need an appropriateCONFIG_OLD_AND_BUSTEDoption for as long as people still need the old ABI.
There is absolutely nothing new here... we have dealt with these kinds of transitions for most of Linux' existence.
However, time_t is a particularly nontrivial issue, because it is not just a matter of changing the kernel ABI but a *lot* of user space ABIs also contain this type. The kernel/glibc interfaces are pretty well set up to handle this properly these days, but I have very little hope that all the user space libraries will properly handle having two versions of a bunch of APIs with different version numbers.
On the other hand, a *limited* form of compatibility could be had by changing the windowing by changing int32_t (1901-2038) to uint32_t (1970-2106). This is similar to how a lot of legacy applications were "fixed" for the Y2K problem.
That has its own share of problems, in particular legacy applications might end up thinking that 2039 comes long before 2037, but there is to some degree a choice of plague or cholera, as we say in Swedish.
-hpa
Quoted 15 lines · H. Peter Anvin
Quoted 10 lines · On 05/05/2014 01:53 PM, [email protected] wrote:
I would be interested in this, not just because of
time_titself, but as a general pattern for "how can we transition away from an old and broken ABI". Whether by introducing new system calls, new personalities, seccomp filters, or other mechanisms, we should have some ways to start such transitions and to smooth them out. Sure, we never break userspace, but that just means we need an appropriateCONFIG_OLD_AND_BUSTEDoption for as long as people still need the old ABI.There is absolutely nothing new here... we have dealt with these kinds of transitions for most of Linux' existence.
Quoted 6 lines
However,
time_tis a particularly nontrivial issue, because it is not just a matter of changing the kernel ABI but a *lot* of user space ABIs also contain this type. The kernel/glibc interfaces are pretty well set up to handle this properly these days, but I have very little hope that all the user space libraries will properly handle having two versions of a bunch of APIs with different version numbers.
64-bit off_t support seems like a similar issue; all the kernel interfaces support 64-bit off_t, and glibc has #defines for 64-bit off_t, but a large number of userspace programs don't bother to define them.
That seems like the right path to transition time_t as well.
- Josh Triplett
Quoted 7 lines · Josh Triplett
64-bit
off_tsupport seems like a similar issue; all the kernel interfaces support 64-bitoff_t, and glibc has #defines for 64-bitoff_t, but a large number of userspace programs don't bother to define them.That seems like the right path to transition
time_tas well.
It's actually quite different, because there off_t isn't used in userspace libraries or data structures nearly as much as time_t. It's not that terrible if you have some libraries which are compiled with 64-bit loff_t (note the different type) and some userspace programs which aren't. But the problems if some libraries want to use a 64-bit time_t and some programs are still only able to support a 32-bit time_t are much more trickly.
I suspect the only real way of dealing with the problem for 32-bit architectures is to consider doing a major version bump for all shared libraries, and for a while, 32-bit distributions might have to ship two versions of all of the more popular shared libraries.
It might be easier for x86 to simply say, "suck it up", if you care, you should use x32 ABI. However, I believe the original concern was that there might still be 32-bit x86 chips still out there in 2038, and x32 won't solve that problem, unfortunately.
OTOH, if the main issue is "industrial control and security systems" as stated in the original problem description,, I very much doubt they would need GNOME shared libraries. :-)
- Ted
Quoted 4 lines · Theodore Ts'o
I suspect the only real way of dealing with the problem for 32-bit architectures is to consider doing a major version bump for all shared libraries, and for a while, 32-bit distributions might have to ship two versions of all of the more popular shared libraries.
...
Quoted 3 lines
OTOH, if the main issue is "industrial control and security systems" as stated in the original problem description,, I very much doubt they would need GNOME shared libraries. :-)
Well, there's also things like in-car-infotainment systems (which would annoy future car collectors should they stop working), which may require GUI toolkits. But I suspect the most problematic cases will be embedded systems in the literal sense of "embedded in the wall".
But the point being that I think we'll have to consider all existing 32bit applications as broken after 2038 (of course there may be some which don't use time a t all, or if they do only use relative offsets - but as a general interface, the existing Linux 32bit ABI is broken). So if we do something like an ABI break, while providing a compat personality for existing applications, duplicated libraries would be necessary, but that extra cost might help motivate vendors to get their applications ported to the new ABI.
And of course, while rebuilding for the new ABI might be one of the easier ways to address things, there's no magic bullet to fix all the places where applications might be casting or storing time_ts as longs either internally or on fixed wire or on-disk formats. (So folks who have been planning their retirement around lucrative Y2K-like contracting work, will still have things to do!).
thanks
-john
Quoted 6 lines · On Tuesday 06 May 2014 10:53:57 John Stultz wrote:
Quoted 4 lines · Theodore Ts'o
I suspect the only real way of dealing with the problem for 32-bit architectures is to consider doing a major version bump for all shared libraries, and for a while, 32-bit distributions might have to ship two versions of all of the more popular shared libraries.
...
Quoted 8 lines
Quoted 3 lines
OTOH, if the main issue is "industrial control and security systems" as stated in the original problem description,, I very much doubt they would need GNOME shared libraries. :-)
Well, there's also things like in-car-infotainment systems (which would annoy future car collectors should they stop working), which may require GUI toolkits. But I suspect the most problematic cases will be embedded systems in the literal sense of "embedded in the wall".
Medical equipment (and I assume military, which is often in the same category) is probably a big thing. I remember we had a big discussion about a product at a former employer when a customer asked for support beyond 2038. I think we ended up saying that the hardware is fine but no supported distro would handle this.
Arnd
Quoted 5 lines · Arnd Bergmann
Medical equipment (and I assume military, which is often in the same category) is probably a big thing. I remember we had a big discussion about a product at a former employer when a customer asked for support beyond 2038. I think we ended up saying that the hardware is fine but no supported distro would handle this.
The question for the kernel summit is what sort of solutions can we suggest where we as kernel developers could help provide a solution for this problem?
If the answer is "the kernel is fine (or could be trivially made fine)" but the problems are all at the glibc and/or distro level, then it's a problem, but I'm not sure we'd be able to make progress on solving it in this venue.
- Ted
Quoted 7 lines · Theodore Ts'o
Quoted 5 lines · Arnd Bergmann
Medical equipment (and I assume military, which is often in the same category) is probably a big thing. I remember we had a big discussion about a product at a former employer when a customer asked for support beyond 2038. I think we ended up saying that the hardware is fine but no supported distro would handle this.
Quoted 8 lines
The question for the kernel summit is what sort of solutions can we suggest where we as kernel developers could help provide a solution for this problem?
If the answer is "the kernel is fine (or could be trivially made fine)" but the problems are all at the glibc and/or distro level, then it's a problem, but I'm not sure we'd be able to make progress on solving it in this venue.
I don't think it's entirely trivial to make the kernel fine. Two topics that *do* make sense for Kernel Summit:
- How do we fix existing ABIs while making it minimally painful to transition existing userspace to the new ABI? Should we just introduce a replacement for every system call that includes a time_t and deprecate the old ones, or should we introduce some more systematic migration mechanism?
- Can the kernel do anything to make the transition easier for libc or other userspace programs?
One possible approach to that second point that might be worth exploring independently, though I'd expect it to prove controversial:
- Could we move some of the userspace compatibility code out of the kernel and into low-level userspace shipped with the kernel?
- Josh Triplett
Quoted 9 lines · On Tue, May 06, 2014 at 01:33:37PM -0700, [email protected] wrote:
I don't think it's entirely trivial to make the kernel fine. Two topics that *do* make sense for Kernel Summit:
- How do we fix existing ABIs while making it minimally painful to transition existing userspace to the new ABI? Should we just introduce a replacement for every system call that includes a
time_tand deprecate the old ones, or should we introduce some more systematic migration mechanism?
It might be useful if someone were to do a bit of homework first. How many system calls (and there will be some ioctl's too, but system calls will probably be the hard part) that are using a 32-bit time_t (including struct timeval and struct timespec) or clock_t? Can someone produce a list so we know how many interfaces we're talking about here?
Quoted 2 lines
- Can the kernel do anything to make the transition easier for libc or other userspace programs?
My experience is that mailing list discussions (like this one) are better at this kind of brain-storming than the actual face-to-face summit meeting. Summit meetings are good for weighing tradeoffs between different possible solutions, but in general, things go much more smoothly if someone has done some homework and outlined a few possible proposals (even if they are strawman proposals) to prime the discussion.
So I'm not just picking on this topic, but in general --- if we have topics of the form, "we should bell the cat", the program committee will be more likely to consider it a good use of everyone's time if there is evidence that some potential technologies for applying the bell to the cat (and how to handle minor problems like the cat's teeth and claws) have been considered raised ahead of time. (And of course, if someone sends a patch making the whole discussion moot, that's even better. :-)
Cheers,
- Ted
Quoted 6 lines · Theodore Ts'o
Quoted 5 lines · Arnd Bergmann
Medical equipment (and I assume military, which is often in the same category) is probably a big thing. I remember we had a big discussion about a product at a former employer when a customer asked for support beyond 2038. I think we ended up saying that the hardware is fine but no supported distro would handle this.
Quoted 9 lines
The question for the kernel summit is what sort of solutions can we suggest where we as kernel developers could help provide a solution for this problem?
If the answer is "the kernel is fine (or could be trivially made fine)" but the problems are all at the glibc and/or distro level, then it's a problem, but I'm not sure we'd be able to make progress on solving it in this venue.
Hi Ted,
At minimum, the collective kernel voice has a great soapbox to stand on, it would not make sense to waste that, and no doubt there will always be more that could be done on the kernel side. For example, the kernel has a definitive vantage point to witness all 32 bit timestamps coming in and going out with the sign bit set. Perhaps some logging mechanism would be helpful in setting up audit systems to use in sandbox testing and triage on Jan 1, 2038, with a view to clawing back the sign bit.
Regards,
Daniel
Quoted 3 lines
and going out with the sign bit set. Perhaps some logging mechanism would be helpful in setting up audit systems to use in sandbox testing and triage on Jan 1, 2038, with a view to clawing back the sign bit.
We can't wait till Jan 2038 to commit a change - programs want to work with dates in the future - allegedly the first Y2038 bugs began appearing in 2008 when banks issuing 30 year mortgages were unable to print the correct date for final payment.
-Tony
Quoted 9 lines · Theodore Ts'o
Quoted 8 lines · On Tue, May 06, 2014 at 01:33:37PM -0700, [email protected] wrote:
I don't think it's entirely trivial to make the kernel fine. Two topics that *do* make sense for Kernel Summit:
- How do we fix existing ABIs while making it minimally painful to transition existing userspace to the new ABI? Should we just introduce a replacement for every system call that includes a
time_tand deprecate the old ones, or should we introduce some more systematic migration mechanism?
Quoted 6 lines
It might be useful if someone were to do a bit of homework first. How many system calls (and there will be some ioctl's too, but system calls will probably be the hard part) that are using a 32-bit
time_t(including struct timeval and struct timespec) orclock_t? Can someone produce a list so we know how many interfaces we're talking about here?
Sure. If its preferred I can try to get some mail on the list outlining my and Thomas' proposals w/ various pros and cons to the list prior to the discussion. And I can make a point to provide specific data on the # of syscall and ioctl structures that embed time_t (or embed structures which contain time_t), etc.
Briefly, my proposal (and forgive me for maybe letting my bias creep in here a bit. I was hoping to work with Thomas to have a more neutral presentation by Kernel Summit, so this summary is a little rough and probably not completely fair to all sides) is to introduce a new ABI for 32bit architectures where time_t is defined as a long long. We could continue supporting the existing ABI via a personality/compat interface.
Pros:
* This would allow applications to be updated by being recompiled, letting the compiler provide warnings where time_ts are cast down to a long, etc.
* This follows in the path that the BSDs have gone, which allows us to leverage their work, where they have already ensured applications they package are fixed.
Cons:
* Having yet another new compat layer, and all the various compat structs just for the new time_t size.
* Distros wanting to support existing ABI would need to have compat libraries as well (much as they do for i386 or x32 on x86_64) - Although this extra cost helps motivate folks to migrate away from the "old" ABI, so might not completely be negative.
* Of course, if we do introduce a new ABI w/ my proposal, there may be other non-time_t related ABI changes folks would like to make on 32bit systems. So there could be lots of "riders" to the change which could make the transition more complex.
Thomas (at least as far as my understanding last we spoke) would prefer not to introduce a whole new ABI, but to introduce new syscalls in addition existing syscalls which would provide something like ktime_t (u64 nanosecond value) as main time type. Eventually deprecating all the syscalls that use time_t.
Pros:
* Various folks have wanted a u64 nanosecond interface as converting back and forth from timespecs has overhead in userspace, so this solves that issue as well.
* Avoids hard ABI break for slow transition.
Cons:
* There are lots of syscalls and ioctls that use time_t (sometimes deeply embedded in structures) for things like timeouts, etc, so creating new syscalls for all of those will be substantial effort.
* We'd also have to generate the same syscalls on 64bit systems, where it would be of minimal benefit.
* All applications would have to be manually converted from time_t interfaces to the new u64 interfaces (not just a recompile)
* Auditing that all the interfaces have been replaced and getting applications to move away from them would be more difficult.
And of course, there is the variant of my proposal which Peter already raised, which is to preserve time_t as 32bits, but define it as a unsigned long. Basically doing a "quiet" ABI break.
Pros:
* No size changes, no need for compat interfaces
* Kernel already considers time values prior to 1970 invalid, so we could easily reuse them from the kernel side.
Cons:
* I worry this would lead to lots of subtle breakage in applications. Its less easy to audit to see if things are fixed or not.
* Applications would still need to make changes if they want to continue to support time values prior to 1970 (likely using 64bit values internally anyway).
* Have to decide if actually change the ABI to an unsigned long or not for newly compiled applications.
Quoted 11 lines
Quoted 3 lines
- Can the kernel do anything to make the transition easier for libc or other userspace programs?
My experience is that mailing list discussions (like this one) are better at this kind of brain-storming than the actual face-to-face summit meeting. Summit meetings are good for weighing tradeoffs between different possible solutions, but in general, things go much more smoothly if someone has done some homework and outlined a few possible proposals (even if they are strawman proposals) to prime the discussion.
Quoted 8 lines
So I'm not just picking on this topic, but in general --- if we have topics of the form, "we should bell the cat", the program committee will be more likely to consider it a good use of everyone's time if there is evidence that some potential technologies for applying the bell to the cat (and how to handle minor problems like the cat's teeth and claws) have been considered raised ahead of time. (And of course, if someone sends a patch making the whole discussion moot, that's even better. :-)
Right. So clearly I don't have all the details ready right this moment, but I hope providing more detail above gives you a better sense of the current proposals and some confidence that I'll try to do my homework and have some discussions on lkml before the Kernel Summit discussion.
thanks
-john
Quoted 8 lines · Tony
Quoted 3 lines
and going out with the sign bit set. Perhaps some logging mechanism would be helpful in setting up audit systems to use in sandbox testing and triage on Jan 1, 2038, with a view to clawing back the sign bit.
We can't wait till Jan 2038 to commit a change - programs want to work with dates in the future - allegedly the first Y2038 bugs began appearing in 2008 when banks issuing 30 year mortgages were unable to print the correct date for final payment.
-Tony
To clarify, sandbox testing can and should start any time. The "triage" part attempts to answer the question: what about bad behavior that somehow slips through the net and manifests only on the day of reckoning? The idea is to provide admins the ability to monitor their systems and spot issues at source as opposed to letting things go silently wrong and potentially cause problems far downstream. Another way of saying it: what way is there to be sure that the problematic interfaces are no longer being used, other than monitoring them? It's clear this is the responsible thing to do, what is less clear is whether the task could benefit from kernel help. Say, CONFIG_Y2038_PARANOIA.
Regards,
Daniel
Quoted 6 lines · John Stultz
Sure. If its preferred I can try to get some mail on the list outlining my and Thomas' proposals w/ various pros and cons to the list prior to the discussion. And I can make a point to provide specific data on the # of syscall and ioctl structures that embed
time_t(or embed structures which containtime_t), etc.
It also might be useful to also consider which interfaces are mediated by glibc, and which aren't (i.e., ioctl's). We might be able to separate what might be doable by negotaitons between glibc and the kernel, and between applications and the shared library ecosystem, which would allow a soft/slow cutover, but with all or most of the compatibility in glibc.
Quoted 5 lines
Thomas (at least as far as my understanding last we spoke) would prefer not to introduce a whole new ABI, but to introduce new syscalls in addition existing syscalls which would provide something like
ktime_t(u64 nanosecond value) as main time type. Eventually deprecating all the syscalls that usetime_t.
One of the questions to evaluate this proposal is how many syscalls this would take. And if we the goal is to "avoid a hard ABI break", then that also means doing something like the Large File Support hack (i.e., open64, read64, etc.) which is application visible. Right? But the problem is unless you get all applications to use these non-standard, non-POSIX interfaces --- or you need to get them to use a magic #define, ala the LFS --- and good luck getting all applications to do make that change, or to get distributions to modify their build scrpits to include that --- and then it's equivalent to a hard ABI break, since it means time_t changes size.
Quoted 4 lines
Right. So clearly I don't have all the details ready right this moment, but I hope providing more detail above gives you a better sense of the current proposals and some confidence that I'll try to do my homework and have some discussions on lkml before the Kernel Summit discussion.
Something else to consider is whether we would need to have some glibc folks present at the kernel summit if we want to discuss it there, and whether we should also think about also planning some discussions at the Plumber's Conference --- because there will certainly be userspace impacts that will need to be taken into account.
- Ted
Quoted 5 lines · On Tue, 6 May 2014 22:07:43 -0400 Theodore Ts'o <[email protected]> wrote:
Something else to consider is whether we would need to have some glibc folks present at the kernel summit if we want to discuss it there, and whether we should also think about also planning some discussions at the Plumber's Conference --- because there will certainly be userspace impacts that will need to be taken into account.
In fact, a year 2038 miniconf at Plumbers seems like a good idea if the right people can be pulled into it.
jon
Quoted 22 lines · Theodore Ts'o
Quoted 7 lines · Josh Triplett
64-bit
off_tsupport seems like a similar issue; all the kernel interfaces support 64-bitoff_t, and glibc has #defines for 64-bitoff_t, but a large number of userspace programs don't bother to define them.That seems like the right path to transition
time_tas well.It's actually quite different, because there
off_tisn't used in userspace libraries or data structures nearly as much astime_t. It's not that terrible if you have some libraries which are compiled with 64-bitloff_t(note the different type) and some userspace programs which aren't. But the problems if some libraries want to use a 64-bittime_tand some programs are still only able to support a 32-bittime_tare much more trickly.I suspect the only real way of dealing with the problem for 32-bit architectures is to consider doing a major version bump for all shared libraries, and for a while, 32-bit distributions might have to ship two versions of all of the more popular shared libraries.
Quoted 4 lines
It might be easier for x86 to simply say, "suck it up", if you care, you should use x32 ABI. However, I believe the original concern was that there might still be 32-bit x86 chips still out there in 2038, and x32 won't solve that problem, unfortunately.
And not just x86. There will be a very long tail of 32-bit embedded platforms that will continue to be produced in huge quantities because they are cheap. Tim's cereal box freebie Linux machine isn't far fetched.
g.
Quoted 10 lines · Theodore Ts'o
Quoted 4 lines · John Stultz
Right. So clearly I don't have all the details ready right this moment, but I hope providing more detail above gives you a better sense of the current proposals and some confidence that I'll try to do my homework and have some discussions on lkml before the Kernel Summit discussion.
Something else to consider is whether we would need to have some glibc folks present at the kernel summit if we want to discuss it there, and whether we should also think about also planning some discussions at the Plumber's Conference --- because there will certainly be userspace impacts that will need to be taken into account.
Do you have a sense of who might be a good glibc contact for this sort of thing?
And yes, When I ran into Thomas at lsf-mm, we talked about splitting the duty here, with me bringing it up at Kernel Summit as Thomas cannot attend, and since I cannot attend Plumbers this year having Thomas covering it there.
thanks
-john
On Tue, 2014-05-06 at 22:07 -0400, Theodore Ts'o wrote:
[...]
Quoted 8 lines
One of the questions to evaluate this proposal is how many syscalls this would take. And if we the goal is to "avoid a hard ABI break", then that also means doing something like the Large File Support hack (i.e., open64, read64, etc.) which is application visible. Right? But the problem is unless you get all applications to use these non-standard, non-POSIX interfaces --- or you need to get them to use a magic #define, ala the LFS --- and good luck getting all applications to do make that change,
LFS is far from universally supported by applications, 17 years after it was standardised. In fact, many applications recently regressed due to a broken test for LFS in autoconf <https://bugs.debian.org/742780>. It doesn't seem like a good example to follow.
Quoted 3 lines
or to get distributions to modify
their build scrpits to include that --- and then it's equivalent to a hard ABI break, since it meanstime_tchanges size.
[...]
However this is done, almost every library that includes time_t in its API will change ABI. I say 'almost' because glibc will probably use symbol versioning or mangling to maintain binary compatibility, but most library maintainers won't go to that trouble.
Ben.
Signature · 3 lines
Ben Hutchings The obvious mathematical breakthrough [to break modern encryption] would be development of an easy way to factor large prime numbers. - Bill Gates
Quoted 3 lines · John Stultz
Do you have a sense of who might be a good glibc contact for this sort of thing?
I'm not that familiar with the internals of the glibc project post Uli, but I'd suggest asking Roland McGrath and DaveM to start.
Quoted 4 lines
And yes, When I ran into Thomas at lsf-mm, we talked about splitting the duty here, with me bringing it up at Kernel Summit as Thomas cannot attend, and since I cannot attend Plumbers this year having Thomas covering it there.
Excellent!
- Ted
Quoted 5 lines · Ben Hutchings
LFS is far from universally supported by applications, 17 years after it was standardised. In fact, many applications recently regressed due to a broken test for LFS in autoconf <https://bugs.debian.org/742780>. It doesn't seem like a good example to follow.
Yes, that was my point.
Quoted 4 lines
However this is done, almost every library that includes
time_tin its API will change ABI. I say 'almost' because glibc will probably use symbol versioning or mangling to maintain binary compatibility, but most library maintainers won't go to that trouble.
Agreed. This is why I'm not sure anything other than a hard ABI break is realistic. Yes, it's incredibly painful, and the distro's will probably be very unhappy, but I suspect the alternatives are worse. The only real question is do we start trying to deal with the pain now, or in 2020, or in 2030, or 2035, or even worse, in 2037....
Given what what I saw with Y2K, if I was going to participate in a betting pool on the question, I'd probably put my money down for 2035 or so. :-/
- Ted
Quoted 8 lines · Josh Triplett
64-bit
off_tsupport seems like a similar issue; all the kernel interfaces support 64-bitoff_t, and glibc has #defines for 64-bitoff_t, but a large number of userspace programs don't bother to define them.That seems like the right path to transition
time_tas well.
The transition to a 64-bit off_t was still enormously complex, which involved defining entirely new types and APIs which were then optionally #defined to the traditional names. Hence the _FILE_OFFSET_BITS opt-in.
For time_t it is even worse.
-hpa
Quoted 20 lines · On Friday 09 May 2014 11:10:43 Theodore Ts'o wrote:
Quoted 5 lines · Ben Hutchings
LFS is far from universally supported by applications, 17 years after it was standardised. In fact, many applications recently regressed due to a broken test for LFS in autoconf <https://bugs.debian.org/742780>. It doesn't seem like a good example to follow.
Yes, that was my point.
Quoted 4 lines
However this is done, almost every library that includes
time_tin its API will change ABI. I say 'almost' because glibc will probably use symbol versioning or mangling to maintain binary compatibility, but most library maintainers won't go to that trouble.Agreed. This is why I'm not sure anything other than a hard ABI break is realistic. Yes, it's incredibly painful, and the distro's will probably be very unhappy, but I suspect the alternatives are worse. The only real question is do we start trying to deal with the pain now, or in 2020, or in 2030, or 2035, or even worse, in 2037....
Quoted 3 lines
Given what what I saw with Y2K, if I was going to participate in a betting pool on the question, I'd probably put my money down for 2035 or so. :-/
I think an important distinction is that the majority of systems that will be seriously affected are embedded machines, which run a custom user space anyway.
x86-32 PCs and end-user distros are going to be largely extinct in a couple of years and replaced by x64-64 or arm64 depending on who you ask, and arm32 Android phones are going to be replaced with arm64 hardware shortly after, or they see an ABI break before then anyway.
The typical embedded machines don't even use glibc, and they cross-build everything from source.
While I'm sure there will be the odd case where people run into the 2038 problem on existing software, we can for now solve a lot of problems just by getting the kernel interfaces in place and making it as easy as possible to build a user space that uses the new types.
Arnd
Quoted 29 lines · Arnd Bergmann
Quoted 20 lines · On Friday 09 May 2014 11:10:43 Theodore Ts'o wrote:
Quoted 5 lines · Ben Hutchings
LFS is far from universally supported by applications, 17 years after it was standardised. In fact, many applications recently regressed due to a broken test for LFS in autoconf <https://bugs.debian.org/742780>. It doesn't seem like a good example to follow.
Yes, that was my point.
Quoted 4 lines
However this is done, almost every library that includes
time_tin its API will change ABI. I say 'almost' because glibc will probably use symbol versioning or mangling to maintain binary compatibility, but most library maintainers won't go to that trouble.Agreed. This is why I'm not sure anything other than a hard ABI break is realistic. Yes, it's incredibly painful, and the distro's will probably be very unhappy, but I suspect the alternatives are worse. The only real question is do we start trying to deal with the pain now, or in 2020, or in 2030, or 2035, or even worse, in 2037....
Quoted 3 lines
Given what what I saw with Y2K, if I was going to participate in a betting pool on the question, I'd probably put my money down for 2035 or so. :-/
I think an important distinction is that the majority of systems that will be seriously affected are embedded machines, which run a custom user space anyway.
Quoted 7 lines
x86-32 PCs and end-user distros are going to be largely extinct in a couple of years and replaced by x64-64 or arm64 depending on who you ask, and arm32 Android phones are going to be replaced with arm64 hardware shortly after, or they see an ABI break before then anyway.
The typical embedded machines don't even use glibc, and they cross-build everything from source.
In particular, even systems that want some of the properties of 32-bit on 64-bit hardware can use x32; the concern is with new systems that don't support 64-bit at all. Hence why we need to solve the problem *today*, so that the devices we're building in the next few years will survive 2038.
- Josh Triplett
Quoted 13 lines · Josh Triplett
Quoted 5 lines
I think an important distinction is that the majority of systems that will be seriously affected are embedded machines, which run a custom user space anyway.
Quoted 7 lines
x86-32 PCs and end-user distros are going to be largely extinct in a couple of years and replaced by x64-64 or arm64 depending on who you ask, and arm32 Android phones are going to be replaced with arm64 hardware shortly after, or they see an ABI break before then anyway.
The typical embedded machines don't even use glibc, and they cross-build everything from source.
Quoted 6 lines
In particular, even systems that want some of the properties of 32-bit on 64-bit hardware can use x32; the concern is with new systems that don't support 64-bit at all. Hence why we need to solve the problem *today*, so that the devices we're building in the next few years will survive 2038.
I used to think 32-bit devices would be extinct by the mid-2020s. It is now obvious that not only will that be wrong, it will be wrong in the most dramatic way possible... simply because all the places where we currently have $0.25 8-bit microcontrollers running trivial operating systems we'll have $0.25 32-bit microcontrollers and a fair chunk of them will run Linux. As we're getting to the point where the most expensive part of the microcontroller is the package, there is simply no reason to not have a powerful CPU with a real OS and minimize the amount of time spent programming the damned thing.
Not to mention that the Internet of Things is going to mean many of them are going to want to be Internet-connected.
-hpa
Here's a suggestion. I don't know whether it's at all practical. What if we added a mode in which all structs passed into and out of the kernel use the 64-bit variant *even for 32-bit code*? Think of it as the opposite of x32.
This has a big benefit: there's no whole new ABI to maintain. The downside is that every single use of unsigned long on the uapi side will have to change to __u64, etc. This might be doable automatically. Another downside is that some structs may be laid out differently in the 32-bit C ABI as compared to the 64-bit C ABI, even given the same primitive type sizes.
Pointers would be tricky. Hmm. Could we get gcc and clang to add __attribute__((64_bit_layout)) if needed?
--Andy
Quoted 31 lines · H. Peter Anvin
Quoted 13 lines · Josh Triplett
Quoted 5 lines
I think an important distinction is that the majority of systems that will be seriously affected are embedded machines, which run a custom user space anyway.
Quoted 7 lines
x86-32 PCs and end-user distros are going to be largely extinct in a couple of years and replaced by x64-64 or arm64 depending on who you ask, and arm32 Android phones are going to be replaced with arm64 hardware shortly after, or they see an ABI break before then anyway.
The typical embedded machines don't even use glibc, and they cross-build everything from source.Quoted 6 lines
In particular, even systems that want some of the properties of 32-bit on 64-bit hardware can use x32; the concern is with new systems that don't support 64-bit at all. Hence why we need to solve the problem *today*, so that the devices we're building in the next few years will survive 2038.
I used to think 32-bit devices would be extinct by the mid-2020s. It is now obvious that not only will that be wrong, it will be wrong in the most dramatic way possible... simply because all the places where we currently have $0.25 8-bit microcontrollers running trivial operating systems we'll have $0.25 32-bit microcontrollers and a fair chunk of them will run Linux. As we're getting to the point where the most expensive part of the microcontroller is the package, there is simply no reason to not have a powerful CPU with a real OS and minimize the amount of time spent programming the damned thing.
Quoted 2 lines
Not to mention that the Internet of Things is going to mean many of them are going to want to be Internet-connected.
Yeah. And there's a good chance that some of them will still be kind of doing something in 2038.
Rafael
Quoted 31 lines · H. Peter Anvin
Quoted 13 lines · Josh Triplett
Quoted 5 lines
I think an important distinction is that the majority of systems that will be seriously affected are embedded machines, which run a custom user space anyway.
Quoted 7 lines
x86-32 PCs and end-user distros are going to be largely extinct in a couple of years and replaced by x64-64 or arm64 depending on who you ask, and arm32 Android phones are going to be replaced with arm64 hardware shortly after, or they see an ABI break before then anyway.
The typical embedded machines don't even use glibc, and they cross-build everything from source.Quoted 6 lines
In particular, even systems that want some of the properties of 32-bit on 64-bit hardware can use x32; the concern is with new systems that don't support 64-bit at all. Hence why we need to solve the problem *today*, so that the devices we're building in the next few years will survive 2038.
I used to think 32-bit devices would be extinct by the mid-2020s. It is now obvious that not only will that be wrong, it will be wrong in the most dramatic way possible... simply because all the places where we currently have $0.25 8-bit microcontrollers running trivial operating systems we'll have $0.25 32-bit microcontrollers and a fair chunk of them will run Linux. As we're getting to the point where the most expensive part of the microcontroller is the package, there is simply no reason to not have a powerful CPU with a real OS and minimize the amount of time spent programming the damned thing.
Quoted 2 lines
Not to mention that the Internet of Things is going to mean many of them are going to want to be Internet-connected.
And yet still, once deployed will never get updated. :-(
g.
Quoted 3 lines · Grant Likely
And yet still, once deployed will never get updated. :-(
That is a whole separate problem. I'm not quite convinced that it will work that way... maybe ubiquitous upgrades will be a solved problem by then.
-hpa
Quoted 7 lines · H. Peter Anvin
Quoted 3 lines · Grant Likely
And yet still, once deployed will never get updated. :-(
That is a whole separate problem. I'm not quite convinced that it will work that way... maybe ubiquitous upgrades will be a solved problem by then.
With the infrastructure to support verification, which brings in key management. Otherwise, it'll be an easy attack surface. I'm not optimistic...
The alternative is to make the devices so trivially disposable that outdated/vulnerable devices are rejected from the network. The user then orders a replacement.
thx,
Jason.