What to do?

Hi all.

A few months back, I posted TuxOnIce for something like the third time, seeking to see if we could make some progress on getting it merged. The outcome of that was an agreement between Rafael and I that we'd work on getting TuxOnIce functionality (if not the actual code) merged bit by bit, improving swsusp rather than doing an outright merge.

Unfortunately, as the intervening months have shown, Rafael is busy with other things and so am I. The few patches I did prepare and post have had no review, and I haven't had a chance to build on or rework them since. (I'd actually like them not to be merged, because I've realised that there are more foundational changes that need to go in first.)

Given that this has been the outcome so far, I see no reason to imagine that we're going to make any serious progress any time soon.

I'd therefore like to ask whether there's anyone out there who could do one or more of:

1) Step up to the plate and help improve swsusp, without relying on any help from me but with my blessing if they choose to copy wholesale bits of TuxOnIce;
2) Take over maintaining TuxOnIce from me;
3) Some other way forward that hasn't occurred to me.

Thoughts, input, flames, whatever else welcome. A lack of deadly silence would be especially appreciated :)

Quoted 2 lines · Nigel Cunningham

Hi all.

Hi,

Quoted 6 lines

A few months back, I posted TuxOnIce for something like the third time, seeking to see if we could make some progress on getting it merged. The outcome of that was an agreement between Rafael and I that we'd work on getting TuxOnIce functionality (if not the actual code) merged bit by bit, improving swsusp rather than doing an outright merge.

Quoted 9 lines

Unfortunately, as the intervening months have shown, Rafael is busy with other things and so am I. The few patches I did prepare and post have had no review, and I haven't had a chance to build on or rework them since. (I'd actually like them not to be merged, because I've realised that there are more foundational changes that need to go in first.)

Given that this has been the outcome so far, I see no reason to imagine that we're going to make any serious progress any time soon.

That unfortunately is probably right.

The problem for my part is that new things are still appearing that I need to take care of rather urgently.

In fact the user space part of (u)swsusp also has been suffering from the lack of attention recently, so to speak.

Quoted 12 lines

I'd therefore like to ask whether there's anyone out there who could do one or more of:

1) Step up to the plate and help improve swsusp, without relying on any help from me but with my blessing if they choose to copy wholesale bits of TuxOnIce;
2) Take over maintaining TuxOnIce from me;
3) Some other way forward that hasn't occurred to me.

Thoughts, input, flames, whatever else welcome. A lack of deadly silence would be especially appreciated :)

Well, the code is out there. If there's time to try to get it in, I think it's worth doing.

Best,
Rafael

Quoted 2 lines · Nigel Cunningham

Hi all.

Hi,

Quoted 6 lines

A few months back, I posted TuxOnIce for something like the third time, seeking to see if we could make some progress on getting it merged. The outcome of that was an agreement between Rafael and I that we'd work on getting TuxOnIce functionality (if not the actual code) merged bit by bit, improving swsusp rather than doing an outright merge.

Quoted 9 lines

Unfortunately, as the intervening months have shown, Rafael is busy with other things and so am I. The few patches I did prepare and post have had no review, and I haven't had a chance to build on or rework them since. (I'd actually like them not to be merged, because I've realised that there are more foundational changes that need to go in first.)

Given that this has been the outcome so far, I see no reason to imagine that we're going to make any serious progress any time soon.

That unfortunately is probably right.

The problem for my part is that new things are still appearing that I need to take care of rather urgently.

In fact the user space part of (u)swsusp also has been suffering from the lack of attention recently, so to speak.

Quoted 12 lines

I'd therefore like to ask whether there's anyone out there who could do one or more of:

1) Step up to the plate and help improve swsusp, without relying on any help from me but with my blessing if they choose to copy wholesale bits of TuxOnIce;
2) Take over maintaining TuxOnIce from me;
3) Some other way forward that hasn't occurred to me.

Thoughts, input, flames, whatever else welcome. A lack of deadly silence would be especially appreciated :)

Well, the code is out there. If there's time to try to get it in, I think it's worth doing.

Best,
Rafael

Well I have no time to jump into the details of the code but I can test whatever you want :)

Well I have no time to jump into the details of the code but I can test whatever you want :)

Hi all.

A few months back, I posted TuxOnIce for something like the third time, seeking to see if we could make some progress on getting it merged. The outcome of that was an agreement between Rafael and I that we'd work on getting TuxOnIce functionality (if not the actual code) merged bit by bit, improving swsusp rather than doing an outright merge.

Unfortunately, as the intervening months have shown, Rafael is busy with other things and so am I. The few patches I did prepare and post have had no review, and I haven't had a chance to build on or rework them since. (I'd actually like them not to be merged, because I've realised that there are more foundational changes that need to go in first.)

Given that this has been the outcome so far, I see no reason to imagine that we're going to make any serious progress any time soon.

I'd therefore like to ask whether there's anyone out there who could do one or more of:

1) Step up to the plate and help improve swsusp, without relying on any help from me but with my blessing if they choose to copy wholesale bits of TuxOnIce;
2) Take over maintaining TuxOnIce from me;
3) Some other way forward that hasn't occurred to me.

Thoughts, input, flames, whatever else welcome. A lack of deadly silence would be especially appreciated :)

I'd therefore like to ask whether there's anyone out there who could do one or more of:

1) Step up to the plate and help improve swsusp, without relying on any help from me but with my blessing if they choose to copy wholesale bits of TuxOnIce;

I'd therefore like to ask whether there's anyone out there who could do one or more of:

1) Step up to the plate and help improve swsusp, without relying on any help from me but with my blessing if they choose to copy wholesale bits of TuxOnIce;

Quoted 6 lines · Nigel Cunningham

I'd therefore like to ask whether there's anyone out there who could do one or more of:

1) Step up to the plate and help improve swsusp, without relying on any help from me but with my blessing if they choose to copy wholesale bits of TuxOnIce;

Hi, I think I can help here.

Is there any TODO or similar resources?

Quoted 8 lines · Jiri Slaby

Quoted 6 lines · Nigel Cunningham

I'd therefore like to ask whether there's anyone out there who could do one or more of:

1) Step up to the plate and help improve swsusp, without relying on any help from me but with my blessing if they choose to copy wholesale bits of TuxOnIce;

Hi, I think I can help here.

Great, thanks!

Quoted 1 line

Is there any TODO or similar resources?

Not that I know of, but I think we should start with the non-controversial things, like image compression etc.

Hi.

Quoted 11 lines · Rafael J. Wysocki

Quoted 8 lines · Jiri Slaby

Quoted 6 lines · Nigel Cunningham

I'd therefore like to ask whether there's anyone out there who could do one or more of:

1) Step up to the plate and help improve swsusp, without relying on any help from me but with my blessing if they choose to copy wholesale bits of TuxOnIce;

Hi, I think I can help here.

Great, thanks!

Yes, thanks for volunteering, Jiri!

We should start by asking where you're at as far as knowledge of swsusp and TuxOnIce goes. I know you've been around the TuxOnIce lists a bit, but don't know how much you know about C programming or the inards of the kernel, swsusp or TuxOnIce (ie how much help do you need to get up to speed?).

Quoted 4 lines

Quoted 1 line

Is there any TODO or similar resources?

Not that I know of, but I think we should start with the non-controversial things, like image compression etc.

No, I have a bit of a mental to-do list, but haven't written anything down yet. Here's a quick, high-level list (some of these things are big):

- Rework swap allocation (as per below) and freeing.
- Rework i/o to use bmapping and lay the foundations for use multiple block devices.
- Add modular design (will simplify the following additions)
- Add multithreaded I/O
- Add compression support
- Add support for multiple swap devices
- Add support for generic files (file allocator)
- Do storage allocation etc ('image preparation') prior to the atomic copy instead of afterwards, making the only remaining variable how much ram drivers' pm routines need (and that might not be an issue by then).
- Add userspace helper support (storage manager/userui).

I'd really like to start with adding the modular design - it will make things a lot cleaner, but I think we need to do some disentanglement first (TuxOnIce has all the support for compression in one file that simply isn't compiled if we don't want compression support. It would be good to get swsusp set up like that). This is why I have above the initial thought of handling doing the swap & compression stuff first.

When I prepared those compression patches earlier, it became apparent that we really need to begin with reworking how the swap is allocated, remembered and stored in the image header.

The current swsusp scheme of having a page of swap addresses preceding the pages that are written doesn't work well for compression, because when rereading the image, you can only do readahead in batches of that size. Everything will have to grind to a halt while we wait for the next 'index' page.

In TuxOnIce, we allocate all of the swap at the start of preparing to hibernate, which has other advantages I won't go into now. The pages allocated are stored as extents in memory. For doing the actual I/O, we bmap the swap that's allocated right after allocating it, storing the sector numbers in extents as well (remembering the block sizes that are relevant as well). We then use them directly in writing the image, storing major and minor numbers, block sizes and the extents in the image header for use at resume time.

I'd suggest that the place to begin would be converting swsusp to use this sort of scheme.

Regarding the non-controversial vs controversial thing, let's not worry about that. We should do things in the order that's logical, sensible and lays the foundation for the improvements we're planning to have follow. If an improvement is really an improvement, it shouldn't be a big problem to show that it has technical merits that make it worthwhile. As always in Linux, if someone wants to nak something without good reason, they'll just end up getting themselves ignored.

Quoted 2 lines · Nigel Cunningham

We should start by asking where you're at as far as knowledge of swsusp and TuxOnIce goes.

Quoted 4 lines

I know you've been around the TuxOnIce lists a bit,
but don't know how much you know about C programming or the inards of the kernel, swsusp or TuxOnIce (ie how much help do you need to get up to speed?).

Well, if you start from something that people don't like, it may be difficult to push the rest.

Still, I think Jiri is more than capable of handling that, so more or less everything that is a clear improvement and doesn't introduce regressions of any kind will probably work for me.

Quoted 2 lines · Nigel Cunningham

We should start by asking where you're at as far as knowledge of swsusp and TuxOnIce goes.

Quoted 4 lines

I know you've been around the TuxOnIce lists a bit,
but don't know how much you know about C programming or the inards of the kernel, swsusp or TuxOnIce (ie how much help do you need to get up to speed?).

Well, if you start from something that people don't like, it may be difficult to push the rest.

Still, I think Jiri is more than capable of handling that, so more or less everything that is a clear improvement and doesn't introduce regressions of any kind will probably work for me.

Hi.

Quoted 11 lines · Rafael J. Wysocki

Quoted 8 lines · Jiri Slaby

Quoted 6 lines · Nigel Cunningham

I'd therefore like to ask whether there's anyone out there who could do one or more of:

1) Step up to the plate and help improve swsusp, without relying on any help from me but with my blessing if they choose to copy wholesale bits of TuxOnIce;

Hi, I think I can help here.

Great, thanks!

Yes, thanks for volunteering, Jiri!

We should start by asking where you're at as far as knowledge of swsusp and TuxOnIce goes. I know you've been around the TuxOnIce lists a bit, but don't know how much you know about C programming or the inards of the kernel, swsusp or TuxOnIce (ie how much help do you need to get up to speed?).

Quoted 4 lines

Quoted 1 line

Is there any TODO or similar resources?

Not that I know of, but I think we should start with the non-controversial things, like image compression etc.

No, I have a bit of a mental to-do list, but haven't written anything down yet. Here's a quick, high-level list (some of these things are big):

- Rework swap allocation (as per below) and freeing.
- Rework i/o to use bmapping and lay the foundations for use multiple block devices.
- Add modular design (will simplify the following additions)
- Add multithreaded I/O
- Add compression support
- Add support for multiple swap devices
- Add support for generic files (file allocator)
- Do storage allocation etc ('image preparation') prior to the atomic copy instead of afterwards, making the only remaining variable how much ram drivers' pm routines need (and that might not be an issue by then).
- Add userspace helper support (storage manager/userui).

I'd really like to start with adding the modular design - it will make things a lot cleaner, but I think we need to do some disentanglement first (TuxOnIce has all the support for compression in one file that simply isn't compiled if we don't want compression support. It would be good to get swsusp set up like that). This is why I have above the initial thought of handling doing the swap & compression stuff first.

When I prepared those compression patches earlier, it became apparent that we really need to begin with reworking how the swap is allocated, remembered and stored in the image header.

The current swsusp scheme of having a page of swap addresses preceding the pages that are written doesn't work well for compression, because when rereading the image, you can only do readahead in batches of that size. Everything will have to grind to a halt while we wait for the next 'index' page.

In TuxOnIce, we allocate all of the swap at the start of preparing to hibernate, which has other advantages I won't go into now. The pages allocated are stored as extents in memory. For doing the actual I/O, we bmap the swap that's allocated right after allocating it, storing the sector numbers in extents as well (remembering the block sizes that are relevant as well). We then use them directly in writing the image, storing major and minor numbers, block sizes and the extents in the image header for use at resume time.

I'd suggest that the place to begin would be converting swsusp to use this sort of scheme.

Regarding the non-controversial vs controversial thing, let's not worry about that. We should do things in the order that's logical, sensible and lays the foundation for the improvements we're planning to have follow. If an improvement is really an improvement, it shouldn't be a big problem to show that it has technical merits that make it worthwhile. As always in Linux, if someone wants to nak something without good reason, they'll just end up getting themselves ignored.

Quoted 8 lines · Jiri Slaby

Quoted 6 lines · Nigel Cunningham

I'd therefore like to ask whether there's anyone out there who could do one or more of:

1) Step up to the plate and help improve swsusp, without relying on any help from me but with my blessing if they choose to copy wholesale bits of TuxOnIce;

Hi, I think I can help here.

Great, thanks!

Quoted 1 line

Is there any TODO or similar resources?

Not that I know of, but I think we should start with the non-controversial things, like image compression etc.

Quoted 6 lines · Nigel Cunningham

I'd therefore like to ask whether there's anyone out there who could do one or more of:

1) Step up to the plate and help improve swsusp, without relying on any help from me but with my blessing if they choose to copy wholesale bits of TuxOnIce;

Hi, I think I can help here.

Is there any TODO or similar resources?