Return-Path: Received: from mail-qk0-f196.google.com ([209.85.220.196]:36931 "EHLO mail-qk0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751381AbeEaEyP (ORCPT ); Thu, 31 May 2018 00:54:15 -0400 Received: by mail-qk0-f196.google.com with SMTP id j12-v6so12930467qkk.4 for ; Wed, 30 May 2018 21:54:14 -0700 (PDT) Received: from [192.168.1.237] (pool-72-66-26-176.washdc.fios.verizon.net. [72.66.26.176]) by smtp.googlemail.com with ESMTPSA id 31-v6sm1950073qtq.80.2018.05.30.21.54.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 30 May 2018 21:54:13 -0700 (PDT) From: Sean Anderson Subject: [Q] Unaligned filesystem data To: linux-fsdevel@vger.kernel.org Message-ID: <66ea8863-8f00-6b41-2534-d4e25711d6bd@gmail.com> Date: Thu, 31 May 2018 00:54:12 -0400 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="NDsLHGtScGlX0ep1avKRqDbHsnKMDjeaj" Sender: linux-fsdevel-owner@vger.kernel.org List-ID: This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --NDsLHGtScGlX0ep1avKRqDbHsnKMDjeaj Content-Type: multipart/mixed; boundary="x63jRgaqmspv1ihDnnbf5l8aSqfwE2q76"; protected-headers="v1" From: Sean Anderson To: linux-fsdevel@vger.kernel.org Message-ID: <66ea8863-8f00-6b41-2534-d4e25711d6bd@gmail.com> Subject: [Q] Unaligned filesystem data --x63jRgaqmspv1ihDnnbf5l8aSqfwE2q76 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable I'm implementing the lean filesystem for fun, and I've run into an issue with how the data is stored. The fs is broadly similar to ext2, however instead of having an inode table, it stores the inode in the first 176 octets of the first sector the file (in this fs, "sector" always refers to a block of 512 octets). The file data itself starts immediately after the inode. The helper functions generic_file_read_iter, mpage_readpages and friends assume that file data begins on block boundaries. I can partially get around this limitation by, for example, messing with iocb->ki_pos before calling generic_file_read_iter. However, the end of the file is then truncated because the file length is 176 bytes short. This is also an issue where the last page may not get mapped in mpage_readpages. The obvious (and hacky) solution is to modify these functions to add the size of the inode to their calls to i_size_read. This would require continually merging in upstream changes to these files. Alternatively, my implementation could lie about the actual length of files. The existing helper functions could be used here, with minor modification for offsets. This would probably not play well with userspace software, as it would yield surprising results for different system calls (stat(fd, statbuf); statbuf->st_size !=3D lseek(fd, 0L, SEEK_END)). One other option would be to use the rest of the inode block for other information and always start the beginning of file data at the beginning of a block. The standard already allows this space to be used for xattrs. However, this would go mostly unused on many systems without SELinux. Other options could include inline duplicate file data, as in ext4 and btrfs or additional extent information. While this is probably the most pragmatic solution, it is also unsatisfying, as (to my knowledge) there is no technical reason this sort of filesystem cannot be implemented in linux. Are there any other approaches I have missed? I saw there was a proposed patch to introduce callbacks to mpage_readpage[s], but this doesn't seem appropriate for the problem at hand, as it isn't length-preserving. My current implementation is at . At the moment I have implemented part of the first option I discussed in src/file.c. I'll be happy to answer any questions you might have. --Sean Anderson --x63jRgaqmspv1ihDnnbf5l8aSqfwE2q76-- --NDsLHGtScGlX0ep1avKRqDbHsnKMDjeaj Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQGTBAEBCgB9FiEEkGEdW86NSNID6GAoPuiP7LShEG4FAlsPf/RfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDkw NjExRDVCQ0U4RDQ4RDIwM0U4NjAyODNFRTg4RkVDQjRBMTEwNkUACgkQPuiP7LSh EG7/BQf/dNBVybi4J6YZ00xMa4INiLud5Hh1iKFiyFL2Rs4uW6sGytc8Q4rkIfIi HRPF/Hji/233i9VdjSl+Xt44tUC5bns7UusHTtMiMiL3B/xlXjSk4WD24LItXHu4 Dv++6BG089w44XazFRjBkgSZhlUCF3OyEfNzZRp8qbdVIf162pekTv1MrAKE7gn6 l1xtvnqIYzgjeyXESaFHDWTM41bjIeaH8LzkcgXKoIOzMMebwqC2geWfKX4J9vjp tf8q53y/O03n3cHdyM1lFCYjDPD73BKgCaCli1WV5DLXYxEVid44/QDwHuWEjCIQ Tqm8vPB8j4emPFboQIvff+qzew+NBA== =JelE -----END PGP SIGNATURE----- --NDsLHGtScGlX0ep1avKRqDbHsnKMDjeaj--