Skip to contentSkip to conversation

How to access "upper" directory of overlayfs programatically

DISCUSSION Jonathan Busby 2 participants 2 messages Sep 14, 2018 1 message not in the archive

lore
Share & tools
lore
#1 lore raw
In reply to a message not in the archive

Oops... forgot that I was replying to you personally :)

-------- Forwarded Message --------
Subject: Re: How to access "upper" directory of overlayfs programatically
Date: Thu, 13 Sep 2018 13:01:16 -0500
From: Jonathan Busby <[email protected]>
To: Amir Goldstein <[email protected]>

Quoted 44 lines · Jonathan Busby
Quoted 1 line · Amir Goldstein

[changing over to overlayfs list]

Thanks for this! :D I had been searching for an overlayfs mailing list but it seems I overlooked unionfs.

Quoted 23 lines · Jonathan Busby
Quoted 21 lines

Hello. I'm new to this list and I don't know if this is the right forum for my question, but, I've asked this same question on many forums and never received an answer -- please don't flame me.

I have a Debian based Linux live system which is running out of RAM and was booted without the "persistent" boot option, so the capser-rw filesystem was never created.

I wish to gain access to the "upper" directory of the overlayfs filesystem in the casper init scripts initramfs "/cow/upper" filesystem so that I may programatically save the filesystem, hopefully without having to re-implement cp.

I know how to commit the changes of a Debian based live CD session to a storage device, but only if the live session in question was booted with the "persistent" option.

I've looked at the overlayfs source code but I still can't figure out how to accomplish what I want.

Quoted 8 lines
Quoted 2 lines

I'm sorry if this is stupid question -- hopefully it's not.

I don't think the programmable API you think of exists, but maybe you can get access to upper dir as admin user.

Please share the init scripts responsible for the mounts at live CD boot and the content of /proc/self/mountinfo

The contents of /proc/self/mountinfo are can be found here :

https://pastebin.com/raw/9uQdJ66w

Quoted 12 lines

The init scripts responsible for the live CD mounts can be found here :

https://www.dropbox.com/s/j4zqsn8avxrv6zf/initrd-scripts.tar.gz?dl=0

Thanks,

Jonathan

Quoted 4 lines

Thanks,
Amir.

I forgot to mention that the init scripts that handle mounting the overlayfs chroot are located in the /scripts/casper directory, and the main script is just called "casper".

Regards,

Jonathan

#2 lore raw
Quoted 65 lines · Jonathan Busby

-------- Forwarded Message --------
Subject: Re: How to access "upper" directory of overlayfs programatically
Date: Thu, 13 Sep 2018 13:01:16 -0500
From: Jonathan Busby <[email protected]>
To: Amir Goldstein <[email protected]>

Quoted 53 lines · Jonathan Busby
Quoted 1 line · Amir Goldstein

[changing over to overlayfs list]

Thanks for this! :D I had been searching for an overlayfs mailing list but it seems I overlooked unionfs.

Quoted 23 lines · Jonathan Busby
Quoted 21 lines

Hello. I'm new to this list and I don't know if this is the right forum for my question, but, I've asked this same question on many forums and never received an answer -- please don't flame me.

I have a Debian based Linux live system which is running out of RAM and was booted without the "persistent" boot option, so the capser-rw filesystem was never created.

I wish to gain access to the "upper" directory of the overlayfs filesystem in the casper init scripts initramfs "/cow/upper" filesystem so that I may programatically save the filesystem, hopefully without having to re-implement cp.

I know how to commit the changes of a Debian based live CD session to a storage device, but only if the live session in question was booted with the "persistent" option.

I've looked at the overlayfs source code but I still can't figure out how to accomplish what I want.

Quoted 8 lines
Quoted 2 lines

I'm sorry if this is stupid question -- hopefully it's not.

I don't think the programmable API you think of exists, but maybe you can get access to upper dir as admin user.

Please share the init scripts responsible for the mounts at live CD boot and the content of /proc/self/mountinfo

The contents of /proc/self/mountinfo are can be found here :

https://pastebin.com/raw/9uQdJ66w

The init scripts responsible for the live CD mounts can be found here :

https://www.dropbox.com/s/j4zqsn8avxrv6zf/initrd-scripts.tar.gz?dl=0

Thanks,

Jonathan

Quoted 1 line
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

Thanks,
Amir.

I forgot to mention that the init scripts that handle mounting the overlayfs chroot are located in the /scripts/casper directory, and the main script is just called "casper".

Jonathan,

Not sure this is what you were looking for, but one thing you could try is to iterate the directories of root fs (similar to find / -xdev) executing stat() on files inside directories.

Depending on your kernel version, the findings might vary, but I would expect all values of st_dev to be identical (this is what makes find -xdev stop on fs boundary).

For non directories, the value of st_dev is an indication to the layer (at least in your specific non-same-fs combination of upper/lower).

If you create a new file, it will be in upper, so you can use its st_dev as the identifier of the upper layer. All files with the same st_dev are either newly created files or modified files, so you can copy them to backup.

Directories will not carry a similar indication if they are modified or new. This method will not give you the list of deleted files (whiteout in upper).

Thanks,
Amir.