fsverity.rst 37 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806
  1. .. SPDX-License-Identifier: GPL-2.0
  2. .. _fsverity:
  3. =======================================================
  4. fs-verity: read-only file-based authenticity protection
  5. =======================================================
  6. Introduction
  7. ============
  8. fs-verity (``fs/verity/``) is a support layer that filesystems can
  9. hook into to support transparent integrity and authenticity protection
  10. of read-only files. Currently, it is supported by the ext4 and f2fs
  11. filesystems. Like fscrypt, not too much filesystem-specific code is
  12. needed to support fs-verity.
  13. fs-verity is similar to `dm-verity
  14. <https://www.kernel.org/doc/Documentation/device-mapper/verity.txt>`_
  15. but works on files rather than block devices. On regular files on
  16. filesystems supporting fs-verity, userspace can execute an ioctl that
  17. causes the filesystem to build a Merkle tree for the file and persist
  18. it to a filesystem-specific location associated with the file.
  19. After this, the file is made readonly, and all reads from the file are
  20. automatically verified against the file's Merkle tree. Reads of any
  21. corrupted data, including mmap reads, will fail.
  22. Userspace can use another ioctl to retrieve the root hash (actually
  23. the "fs-verity file digest", which is a hash that includes the Merkle
  24. tree root hash) that fs-verity is enforcing for the file. This ioctl
  25. executes in constant time, regardless of the file size.
  26. fs-verity is essentially a way to hash a file in constant time,
  27. subject to the caveat that reads which would violate the hash will
  28. fail at runtime.
  29. Use cases
  30. =========
  31. By itself, the base fs-verity feature only provides integrity
  32. protection, i.e. detection of accidental (non-malicious) corruption.
  33. However, because fs-verity makes retrieving the file hash extremely
  34. efficient, it's primarily meant to be used as a tool to support
  35. authentication (detection of malicious modifications) or auditing
  36. (logging file hashes before use).
  37. Trusted userspace code (e.g. operating system code running on a
  38. read-only partition that is itself authenticated by dm-verity) can
  39. authenticate the contents of an fs-verity file by using the
  40. `FS_IOC_MEASURE_VERITY`_ ioctl to retrieve its hash, then verifying a
  41. digital signature of it.
  42. A standard file hash could be used instead of fs-verity. However,
  43. this is inefficient if the file is large and only a small portion may
  44. be accessed. This is often the case for Android application package
  45. (APK) files, for example. These typically contain many translations,
  46. classes, and other resources that are infrequently or even never
  47. accessed on a particular device. It would be slow and wasteful to
  48. read and hash the entire file before starting the application.
  49. Unlike an ahead-of-time hash, fs-verity also re-verifies data each
  50. time it's paged in. This ensures that malicious disk firmware can't
  51. undetectably change the contents of the file at runtime.
  52. fs-verity does not replace or obsolete dm-verity. dm-verity should
  53. still be used on read-only filesystems. fs-verity is for files that
  54. must live on a read-write filesystem because they are independently
  55. updated and potentially user-installed, so dm-verity cannot be used.
  56. The base fs-verity feature is a hashing mechanism only; actually
  57. authenticating the files is up to userspace. However, to meet some
  58. users' needs, fs-verity optionally supports a simple signature
  59. verification mechanism where users can configure the kernel to require
  60. that all fs-verity files be signed by a key loaded into a keyring; see
  61. `Built-in signature verification`_. Support for fs-verity file hashes
  62. in IMA (Integrity Measurement Architecture) policies is also planned.
  63. User API
  64. ========
  65. FS_IOC_ENABLE_VERITY
  66. --------------------
  67. The FS_IOC_ENABLE_VERITY ioctl enables fs-verity on a file. It takes
  68. in a pointer to a struct fsverity_enable_arg, defined as
  69. follows::
  70. struct fsverity_enable_arg {
  71. __u32 version;
  72. __u32 hash_algorithm;
  73. __u32 block_size;
  74. __u32 salt_size;
  75. __u64 salt_ptr;
  76. __u32 sig_size;
  77. __u32 __reserved1;
  78. __u64 sig_ptr;
  79. __u64 __reserved2[11];
  80. };
  81. This structure contains the parameters of the Merkle tree to build for
  82. the file, and optionally contains a signature. It must be initialized
  83. as follows:
  84. - ``version`` must be 1.
  85. - ``hash_algorithm`` must be the identifier for the hash algorithm to
  86. use for the Merkle tree, such as FS_VERITY_HASH_ALG_SHA256. See
  87. ``include/uapi/linux/fsverity.h`` for the list of possible values.
  88. - ``block_size`` must be the Merkle tree block size. Currently, this
  89. must be equal to the system page size, which is usually 4096 bytes.
  90. Other sizes may be supported in the future. This value is not
  91. necessarily the same as the filesystem block size.
  92. - ``salt_size`` is the size of the salt in bytes, or 0 if no salt is
  93. provided. The salt is a value that is prepended to every hashed
  94. block; it can be used to personalize the hashing for a particular
  95. file or device. Currently the maximum salt size is 32 bytes.
  96. - ``salt_ptr`` is the pointer to the salt, or NULL if no salt is
  97. provided.
  98. - ``sig_size`` is the size of the signature in bytes, or 0 if no
  99. signature is provided. Currently the signature is (somewhat
  100. arbitrarily) limited to 16128 bytes. See `Built-in signature
  101. verification`_ for more information.
  102. - ``sig_ptr`` is the pointer to the signature, or NULL if no
  103. signature is provided.
  104. - All reserved fields must be zeroed.
  105. FS_IOC_ENABLE_VERITY causes the filesystem to build a Merkle tree for
  106. the file and persist it to a filesystem-specific location associated
  107. with the file, then mark the file as a verity file. This ioctl may
  108. take a long time to execute on large files, and it is interruptible by
  109. fatal signals.
  110. FS_IOC_ENABLE_VERITY checks for write access to the inode. However,
  111. it must be executed on an O_RDONLY file descriptor and no processes
  112. can have the file open for writing. Attempts to open the file for
  113. writing while this ioctl is executing will fail with ETXTBSY. (This
  114. is necessary to guarantee that no writable file descriptors will exist
  115. after verity is enabled, and to guarantee that the file's contents are
  116. stable while the Merkle tree is being built over it.)
  117. On success, FS_IOC_ENABLE_VERITY returns 0, and the file becomes a
  118. verity file. On failure (including the case of interruption by a
  119. fatal signal), no changes are made to the file.
  120. FS_IOC_ENABLE_VERITY can fail with the following errors:
  121. - ``EACCES``: the process does not have write access to the file
  122. - ``EBADMSG``: the signature is malformed
  123. - ``EBUSY``: this ioctl is already running on the file
  124. - ``EEXIST``: the file already has verity enabled
  125. - ``EFAULT``: the caller provided inaccessible memory
  126. - ``EINTR``: the operation was interrupted by a fatal signal
  127. - ``EINVAL``: unsupported version, hash algorithm, or block size; or
  128. reserved bits are set; or the file descriptor refers to neither a
  129. regular file nor a directory.
  130. - ``EISDIR``: the file descriptor refers to a directory
  131. - ``EKEYREJECTED``: the signature doesn't match the file
  132. - ``EMSGSIZE``: the salt or signature is too long
  133. - ``ENOKEY``: the fs-verity keyring doesn't contain the certificate
  134. needed to verify the signature
  135. - ``ENOPKG``: fs-verity recognizes the hash algorithm, but it's not
  136. available in the kernel's crypto API as currently configured (e.g.
  137. for SHA-512, missing CONFIG_CRYPTO_SHA512).
  138. - ``ENOTTY``: this type of filesystem does not implement fs-verity
  139. - ``EOPNOTSUPP``: the kernel was not configured with fs-verity
  140. support; or the filesystem superblock has not had the 'verity'
  141. feature enabled on it; or the filesystem does not support fs-verity
  142. on this file. (See `Filesystem support`_.)
  143. - ``EPERM``: the file is append-only; or, a signature is required and
  144. one was not provided.
  145. - ``EROFS``: the filesystem is read-only
  146. - ``ETXTBSY``: someone has the file open for writing. This can be the
  147. caller's file descriptor, another open file descriptor, or the file
  148. reference held by a writable memory map.
  149. FS_IOC_MEASURE_VERITY
  150. ---------------------
  151. The FS_IOC_MEASURE_VERITY ioctl retrieves the digest of a verity file.
  152. The fs-verity file digest is a cryptographic digest that identifies
  153. the file contents that are being enforced on reads; it is computed via
  154. a Merkle tree and is different from a traditional full-file digest.
  155. This ioctl takes in a pointer to a variable-length structure::
  156. struct fsverity_digest {
  157. __u16 digest_algorithm;
  158. __u16 digest_size; /* input/output */
  159. __u8 digest[];
  160. };
  161. ``digest_size`` is an input/output field. On input, it must be
  162. initialized to the number of bytes allocated for the variable-length
  163. ``digest`` field.
  164. On success, 0 is returned and the kernel fills in the structure as
  165. follows:
  166. - ``digest_algorithm`` will be the hash algorithm used for the file
  167. digest. It will match ``fsverity_enable_arg::hash_algorithm``.
  168. - ``digest_size`` will be the size of the digest in bytes, e.g. 32
  169. for SHA-256. (This can be redundant with ``digest_algorithm``.)
  170. - ``digest`` will be the actual bytes of the digest.
  171. FS_IOC_MEASURE_VERITY is guaranteed to execute in constant time,
  172. regardless of the size of the file.
  173. FS_IOC_MEASURE_VERITY can fail with the following errors:
  174. - ``EFAULT``: the caller provided inaccessible memory
  175. - ``ENODATA``: the file is not a verity file
  176. - ``ENOTTY``: this type of filesystem does not implement fs-verity
  177. - ``EOPNOTSUPP``: the kernel was not configured with fs-verity
  178. support, or the filesystem superblock has not had the 'verity'
  179. feature enabled on it. (See `Filesystem support`_.)
  180. - ``EOVERFLOW``: the digest is longer than the specified
  181. ``digest_size`` bytes. Try providing a larger buffer.
  182. FS_IOC_READ_VERITY_METADATA
  183. ---------------------------
  184. The FS_IOC_READ_VERITY_METADATA ioctl reads verity metadata from a
  185. verity file. This ioctl is available since Linux v5.12.
  186. This ioctl allows writing a server program that takes a verity file
  187. and serves it to a client program, such that the client can do its own
  188. fs-verity compatible verification of the file. This only makes sense
  189. if the client doesn't trust the server and if the server needs to
  190. provide the storage for the client.
  191. This is a fairly specialized use case, and most fs-verity users won't
  192. need this ioctl.
  193. This ioctl takes in a pointer to the following structure::
  194. #define FS_VERITY_METADATA_TYPE_MERKLE_TREE 1
  195. #define FS_VERITY_METADATA_TYPE_DESCRIPTOR 2
  196. #define FS_VERITY_METADATA_TYPE_SIGNATURE 3
  197. struct fsverity_read_metadata_arg {
  198. __u64 metadata_type;
  199. __u64 offset;
  200. __u64 length;
  201. __u64 buf_ptr;
  202. __u64 __reserved;
  203. };
  204. ``metadata_type`` specifies the type of metadata to read:
  205. - ``FS_VERITY_METADATA_TYPE_MERKLE_TREE`` reads the blocks of the
  206. Merkle tree. The blocks are returned in order from the root level
  207. to the leaf level. Within each level, the blocks are returned in
  208. the same order that their hashes are themselves hashed.
  209. See `Merkle tree`_ for more information.
  210. - ``FS_VERITY_METADATA_TYPE_DESCRIPTOR`` reads the fs-verity
  211. descriptor. See `fs-verity descriptor`_.
  212. - ``FS_VERITY_METADATA_TYPE_SIGNATURE`` reads the signature which was
  213. passed to FS_IOC_ENABLE_VERITY, if any. See `Built-in signature
  214. verification`_.
  215. The semantics are similar to those of ``pread()``. ``offset``
  216. specifies the offset in bytes into the metadata item to read from, and
  217. ``length`` specifies the maximum number of bytes to read from the
  218. metadata item. ``buf_ptr`` is the pointer to the buffer to read into,
  219. cast to a 64-bit integer. ``__reserved`` must be 0. On success, the
  220. number of bytes read is returned. 0 is returned at the end of the
  221. metadata item. The returned length may be less than ``length``, for
  222. example if the ioctl is interrupted.
  223. The metadata returned by FS_IOC_READ_VERITY_METADATA isn't guaranteed
  224. to be authenticated against the file digest that would be returned by
  225. `FS_IOC_MEASURE_VERITY`_, as the metadata is expected to be used to
  226. implement fs-verity compatible verification anyway (though absent a
  227. malicious disk, the metadata will indeed match). E.g. to implement
  228. this ioctl, the filesystem is allowed to just read the Merkle tree
  229. blocks from disk without actually verifying the path to the root node.
  230. FS_IOC_READ_VERITY_METADATA can fail with the following errors:
  231. - ``EFAULT``: the caller provided inaccessible memory
  232. - ``EINTR``: the ioctl was interrupted before any data was read
  233. - ``EINVAL``: reserved fields were set, or ``offset + length``
  234. overflowed
  235. - ``ENODATA``: the file is not a verity file, or
  236. FS_VERITY_METADATA_TYPE_SIGNATURE was requested but the file doesn't
  237. have a built-in signature
  238. - ``ENOTTY``: this type of filesystem does not implement fs-verity, or
  239. this ioctl is not yet implemented on it
  240. - ``EOPNOTSUPP``: the kernel was not configured with fs-verity
  241. support, or the filesystem superblock has not had the 'verity'
  242. feature enabled on it. (See `Filesystem support`_.)
  243. FS_IOC_GETFLAGS
  244. ---------------
  245. The existing ioctl FS_IOC_GETFLAGS (which isn't specific to fs-verity)
  246. can also be used to check whether a file has fs-verity enabled or not.
  247. To do so, check for FS_VERITY_FL (0x00100000) in the returned flags.
  248. The verity flag is not settable via FS_IOC_SETFLAGS. You must use
  249. FS_IOC_ENABLE_VERITY instead, since parameters must be provided.
  250. statx
  251. -----
  252. Since Linux v5.5, the statx() system call sets STATX_ATTR_VERITY if
  253. the file has fs-verity enabled. This can perform better than
  254. FS_IOC_GETFLAGS and FS_IOC_MEASURE_VERITY because it doesn't require
  255. opening the file, and opening verity files can be expensive.
  256. Accessing verity files
  257. ======================
  258. Applications can transparently access a verity file just like a
  259. non-verity one, with the following exceptions:
  260. - Verity files are readonly. They cannot be opened for writing or
  261. truncate()d, even if the file mode bits allow it. Attempts to do
  262. one of these things will fail with EPERM. However, changes to
  263. metadata such as owner, mode, timestamps, and xattrs are still
  264. allowed, since these are not measured by fs-verity. Verity files
  265. can also still be renamed, deleted, and linked to.
  266. - Direct I/O is not supported on verity files. Attempts to use direct
  267. I/O on such files will fall back to buffered I/O.
  268. - DAX (Direct Access) is not supported on verity files, because this
  269. would circumvent the data verification.
  270. - Reads of data that doesn't match the verity Merkle tree will fail
  271. with EIO (for read()) or SIGBUS (for mmap() reads).
  272. - If the sysctl "fs.verity.require_signatures" is set to 1 and the
  273. file is not signed by a key in the fs-verity keyring, then opening
  274. the file will fail. See `Built-in signature verification`_.
  275. Direct access to the Merkle tree is not supported. Therefore, if a
  276. verity file is copied, or is backed up and restored, then it will lose
  277. its "verity"-ness. fs-verity is primarily meant for files like
  278. executables that are managed by a package manager.
  279. File digest computation
  280. =======================
  281. This section describes how fs-verity hashes the file contents using a
  282. Merkle tree to produce the digest which cryptographically identifies
  283. the file contents. This algorithm is the same for all filesystems
  284. that support fs-verity.
  285. Userspace only needs to be aware of this algorithm if it needs to
  286. compute fs-verity file digests itself, e.g. in order to sign files.
  287. .. _fsverity_merkle_tree:
  288. Merkle tree
  289. -----------
  290. The file contents is divided into blocks, where the block size is
  291. configurable but is usually 4096 bytes. The end of the last block is
  292. zero-padded if needed. Each block is then hashed, producing the first
  293. level of hashes. Then, the hashes in this first level are grouped
  294. into 'blocksize'-byte blocks (zero-padding the ends as needed) and
  295. these blocks are hashed, producing the second level of hashes. This
  296. proceeds up the tree until only a single block remains. The hash of
  297. this block is the "Merkle tree root hash".
  298. If the file fits in one block and is nonempty, then the "Merkle tree
  299. root hash" is simply the hash of the single data block. If the file
  300. is empty, then the "Merkle tree root hash" is all zeroes.
  301. The "blocks" here are not necessarily the same as "filesystem blocks".
  302. If a salt was specified, then it's zero-padded to the closest multiple
  303. of the input size of the hash algorithm's compression function, e.g.
  304. 64 bytes for SHA-256 or 128 bytes for SHA-512. The padded salt is
  305. prepended to every data or Merkle tree block that is hashed.
  306. The purpose of the block padding is to cause every hash to be taken
  307. over the same amount of data, which simplifies the implementation and
  308. keeps open more possibilities for hardware acceleration. The purpose
  309. of the salt padding is to make the salting "free" when the salted hash
  310. state is precomputed, then imported for each hash.
  311. Example: in the recommended configuration of SHA-256 and 4K blocks,
  312. 128 hash values fit in each block. Thus, each level of the Merkle
  313. tree is approximately 128 times smaller than the previous, and for
  314. large files the Merkle tree's size converges to approximately 1/127 of
  315. the original file size. However, for small files, the padding is
  316. significant, making the space overhead proportionally more.
  317. .. _fsverity_descriptor:
  318. fs-verity descriptor
  319. --------------------
  320. By itself, the Merkle tree root hash is ambiguous. For example, it
  321. can't a distinguish a large file from a small second file whose data
  322. is exactly the top-level hash block of the first file. Ambiguities
  323. also arise from the convention of padding to the next block boundary.
  324. To solve this problem, the fs-verity file digest is actually computed
  325. as a hash of the following structure, which contains the Merkle tree
  326. root hash as well as other fields such as the file size::
  327. struct fsverity_descriptor {
  328. __u8 version; /* must be 1 */
  329. __u8 hash_algorithm; /* Merkle tree hash algorithm */
  330. __u8 log_blocksize; /* log2 of size of data and tree blocks */
  331. __u8 salt_size; /* size of salt in bytes; 0 if none */
  332. __le32 __reserved_0x04; /* must be 0 */
  333. __le64 data_size; /* size of file the Merkle tree is built over */
  334. __u8 root_hash[64]; /* Merkle tree root hash */
  335. __u8 salt[32]; /* salt prepended to each hashed block */
  336. __u8 __reserved[144]; /* must be 0's */
  337. };
  338. Built-in signature verification
  339. ===============================
  340. With CONFIG_FS_VERITY_BUILTIN_SIGNATURES=y, fs-verity supports putting
  341. a portion of an authentication policy (see `Use cases`_) in the
  342. kernel. Specifically, it adds support for:
  343. 1. At fs-verity module initialization time, a keyring ".fs-verity" is
  344. created. The root user can add trusted X.509 certificates to this
  345. keyring using the add_key() system call, then (when done)
  346. optionally use keyctl_restrict_keyring() to prevent additional
  347. certificates from being added.
  348. 2. `FS_IOC_ENABLE_VERITY`_ accepts a pointer to a PKCS#7 formatted
  349. detached signature in DER format of the file's fs-verity digest.
  350. On success, this signature is persisted alongside the Merkle tree.
  351. Then, any time the file is opened, the kernel will verify the
  352. file's actual digest against this signature, using the certificates
  353. in the ".fs-verity" keyring.
  354. 3. A new sysctl "fs.verity.require_signatures" is made available.
  355. When set to 1, the kernel requires that all verity files have a
  356. correctly signed digest as described in (2).
  357. fs-verity file digests must be signed in the following format, which
  358. is similar to the structure used by `FS_IOC_MEASURE_VERITY`_::
  359. struct fsverity_formatted_digest {
  360. char magic[8]; /* must be "FSVerity" */
  361. __le16 digest_algorithm;
  362. __le16 digest_size;
  363. __u8 digest[];
  364. };
  365. fs-verity's built-in signature verification support is meant as a
  366. relatively simple mechanism that can be used to provide some level of
  367. authenticity protection for verity files, as an alternative to doing
  368. the signature verification in userspace or using IMA-appraisal.
  369. However, with this mechanism, userspace programs still need to check
  370. that the verity bit is set, and there is no protection against verity
  371. files being swapped around.
  372. Filesystem support
  373. ==================
  374. fs-verity is currently supported by the ext4 and f2fs filesystems.
  375. The CONFIG_FS_VERITY kconfig option must be enabled to use fs-verity
  376. on either filesystem.
  377. ``include/linux/fsverity.h`` declares the interface between the
  378. ``fs/verity/`` support layer and filesystems. Briefly, filesystems
  379. must provide an ``fsverity_operations`` structure that provides
  380. methods to read and write the verity metadata to a filesystem-specific
  381. location, including the Merkle tree blocks and
  382. ``fsverity_descriptor``. Filesystems must also call functions in
  383. ``fs/verity/`` at certain times, such as when a file is opened or when
  384. pages have been read into the pagecache. (See `Verifying data`_.)
  385. ext4
  386. ----
  387. ext4 supports fs-verity since Linux v5.4 and e2fsprogs v1.45.2.
  388. To create verity files on an ext4 filesystem, the filesystem must have
  389. been formatted with ``-O verity`` or had ``tune2fs -O verity`` run on
  390. it. "verity" is an RO_COMPAT filesystem feature, so once set, old
  391. kernels will only be able to mount the filesystem readonly, and old
  392. versions of e2fsck will be unable to check the filesystem. Moreover,
  393. currently ext4 only supports mounting a filesystem with the "verity"
  394. feature when its block size is equal to PAGE_SIZE (often 4096 bytes).
  395. ext4 sets the EXT4_VERITY_FL on-disk inode flag on verity files. It
  396. can only be set by `FS_IOC_ENABLE_VERITY`_, and it cannot be cleared.
  397. ext4 also supports encryption, which can be used simultaneously with
  398. fs-verity. In this case, the plaintext data is verified rather than
  399. the ciphertext. This is necessary in order to make the fs-verity file
  400. digest meaningful, since every file is encrypted differently.
  401. ext4 stores the verity metadata (Merkle tree and fsverity_descriptor)
  402. past the end of the file, starting at the first 64K boundary beyond
  403. i_size. This approach works because (a) verity files are readonly,
  404. and (b) pages fully beyond i_size aren't visible to userspace but can
  405. be read/written internally by ext4 with only some relatively small
  406. changes to ext4. This approach avoids having to depend on the
  407. EA_INODE feature and on rearchitecturing ext4's xattr support to
  408. support paging multi-gigabyte xattrs into memory, and to support
  409. encrypting xattrs. Note that the verity metadata *must* be encrypted
  410. when the file is, since it contains hashes of the plaintext data.
  411. Currently, ext4 verity only supports the case where the Merkle tree
  412. block size, filesystem block size, and page size are all the same. It
  413. also only supports extent-based files.
  414. f2fs
  415. ----
  416. f2fs supports fs-verity since Linux v5.4 and f2fs-tools v1.11.0.
  417. To create verity files on an f2fs filesystem, the filesystem must have
  418. been formatted with ``-O verity``.
  419. f2fs sets the FADVISE_VERITY_BIT on-disk inode flag on verity files.
  420. It can only be set by `FS_IOC_ENABLE_VERITY`_, and it cannot be
  421. cleared.
  422. Like ext4, f2fs stores the verity metadata (Merkle tree and
  423. fsverity_descriptor) past the end of the file, starting at the first
  424. 64K boundary beyond i_size. See explanation for ext4 above.
  425. Moreover, f2fs supports at most 4096 bytes of xattr entries per inode
  426. which wouldn't be enough for even a single Merkle tree block.
  427. Currently, f2fs verity only supports a Merkle tree block size of 4096.
  428. Also, f2fs doesn't support enabling verity on files that currently
  429. have atomic or volatile writes pending.
  430. Implementation details
  431. ======================
  432. Verifying data
  433. --------------
  434. fs-verity ensures that all reads of a verity file's data are verified,
  435. regardless of which syscall is used to do the read (e.g. mmap(),
  436. read(), pread()) and regardless of whether it's the first read or a
  437. later read (unless the later read can return cached data that was
  438. already verified). Below, we describe how filesystems implement this.
  439. Pagecache
  440. ~~~~~~~~~
  441. For filesystems using Linux's pagecache, the ``->readpage()`` and
  442. ``->readpages()`` methods must be modified to verify pages before they
  443. are marked Uptodate. Merely hooking ``->read_iter()`` would be
  444. insufficient, since ``->read_iter()`` is not used for memory maps.
  445. Therefore, fs/verity/ provides a function fsverity_verify_page() which
  446. verifies a page that has been read into the pagecache of a verity
  447. inode, but is still locked and not Uptodate, so it's not yet readable
  448. by userspace. As needed to do the verification,
  449. fsverity_verify_page() will call back into the filesystem to read
  450. Merkle tree pages via fsverity_operations::read_merkle_tree_page().
  451. fsverity_verify_page() returns false if verification failed; in this
  452. case, the filesystem must not set the page Uptodate. Following this,
  453. as per the usual Linux pagecache behavior, attempts by userspace to
  454. read() from the part of the file containing the page will fail with
  455. EIO, and accesses to the page within a memory map will raise SIGBUS.
  456. fsverity_verify_page() currently only supports the case where the
  457. Merkle tree block size is equal to PAGE_SIZE (often 4096 bytes).
  458. In principle, fsverity_verify_page() verifies the entire path in the
  459. Merkle tree from the data page to the root hash. However, for
  460. efficiency the filesystem may cache the hash pages. Therefore,
  461. fsverity_verify_page() only ascends the tree reading hash pages until
  462. an already-verified hash page is seen, as indicated by the PageChecked
  463. bit being set. It then verifies the path to that page.
  464. This optimization, which is also used by dm-verity, results in
  465. excellent sequential read performance. This is because usually (e.g.
  466. 127 in 128 times for 4K blocks and SHA-256) the hash page from the
  467. bottom level of the tree will already be cached and checked from
  468. reading a previous data page. However, random reads perform worse.
  469. Block device based filesystems
  470. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  471. Block device based filesystems (e.g. ext4 and f2fs) in Linux also use
  472. the pagecache, so the above subsection applies too. However, they
  473. also usually read many pages from a file at once, grouped into a
  474. structure called a "bio". To make it easier for these types of
  475. filesystems to support fs-verity, fs/verity/ also provides a function
  476. fsverity_verify_bio() which verifies all pages in a bio.
  477. ext4 and f2fs also support encryption. If a verity file is also
  478. encrypted, the pages must be decrypted before being verified. To
  479. support this, these filesystems allocate a "post-read context" for
  480. each bio and store it in ``->bi_private``::
  481. struct bio_post_read_ctx {
  482. struct bio *bio;
  483. struct work_struct work;
  484. unsigned int cur_step;
  485. unsigned int enabled_steps;
  486. };
  487. ``enabled_steps`` is a bitmask that specifies whether decryption,
  488. verity, or both is enabled. After the bio completes, for each needed
  489. postprocessing step the filesystem enqueues the bio_post_read_ctx on a
  490. workqueue, and then the workqueue work does the decryption or
  491. verification. Finally, pages where no decryption or verity error
  492. occurred are marked Uptodate, and the pages are unlocked.
  493. Files on ext4 and f2fs may contain holes. Normally, ``->readpages()``
  494. simply zeroes holes and sets the corresponding pages Uptodate; no bios
  495. are issued. To prevent this case from bypassing fs-verity, these
  496. filesystems use fsverity_verify_page() to verify hole pages.
  497. ext4 and f2fs disable direct I/O on verity files, since otherwise
  498. direct I/O would bypass fs-verity. (They also do the same for
  499. encrypted files.)
  500. Userspace utility
  501. =================
  502. This document focuses on the kernel, but a userspace utility for
  503. fs-verity can be found at:
  504. https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/fsverity-utils.git
  505. See the README.md file in the fsverity-utils source tree for details,
  506. including examples of setting up fs-verity protected files.
  507. Tests
  508. =====
  509. To test fs-verity, use xfstests. For example, using `kvm-xfstests
  510. <https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md>`_::
  511. kvm-xfstests -c ext4,f2fs -g verity
  512. FAQ
  513. ===
  514. This section answers frequently asked questions about fs-verity that
  515. weren't already directly answered in other parts of this document.
  516. :Q: Why isn't fs-verity part of IMA?
  517. :A: fs-verity and IMA (Integrity Measurement Architecture) have
  518. different focuses. fs-verity is a filesystem-level mechanism for
  519. hashing individual files using a Merkle tree. In contrast, IMA
  520. specifies a system-wide policy that specifies which files are
  521. hashed and what to do with those hashes, such as log them,
  522. authenticate them, or add them to a measurement list.
  523. IMA is planned to support the fs-verity hashing mechanism as an
  524. alternative to doing full file hashes, for people who want the
  525. performance and security benefits of the Merkle tree based hash.
  526. But it doesn't make sense to force all uses of fs-verity to be
  527. through IMA. As a standalone filesystem feature, fs-verity
  528. already meets many users' needs, and it's testable like other
  529. filesystem features e.g. with xfstests.
  530. :Q: Isn't fs-verity useless because the attacker can just modify the
  531. hashes in the Merkle tree, which is stored on-disk?
  532. :A: To verify the authenticity of an fs-verity file you must verify
  533. the authenticity of the "fs-verity file digest", which
  534. incorporates the root hash of the Merkle tree. See `Use cases`_.
  535. :Q: Isn't fs-verity useless because the attacker can just replace a
  536. verity file with a non-verity one?
  537. :A: See `Use cases`_. In the initial use case, it's really trusted
  538. userspace code that authenticates the files; fs-verity is just a
  539. tool to do this job efficiently and securely. The trusted
  540. userspace code will consider non-verity files to be inauthentic.
  541. :Q: Why does the Merkle tree need to be stored on-disk? Couldn't you
  542. store just the root hash?
  543. :A: If the Merkle tree wasn't stored on-disk, then you'd have to
  544. compute the entire tree when the file is first accessed, even if
  545. just one byte is being read. This is a fundamental consequence of
  546. how Merkle tree hashing works. To verify a leaf node, you need to
  547. verify the whole path to the root hash, including the root node
  548. (the thing which the root hash is a hash of). But if the root
  549. node isn't stored on-disk, you have to compute it by hashing its
  550. children, and so on until you've actually hashed the entire file.
  551. That defeats most of the point of doing a Merkle tree-based hash,
  552. since if you have to hash the whole file ahead of time anyway,
  553. then you could simply do sha256(file) instead. That would be much
  554. simpler, and a bit faster too.
  555. It's true that an in-memory Merkle tree could still provide the
  556. advantage of verification on every read rather than just on the
  557. first read. However, it would be inefficient because every time a
  558. hash page gets evicted (you can't pin the entire Merkle tree into
  559. memory, since it may be very large), in order to restore it you
  560. again need to hash everything below it in the tree. This again
  561. defeats most of the point of doing a Merkle tree-based hash, since
  562. a single block read could trigger re-hashing gigabytes of data.
  563. :Q: But couldn't you store just the leaf nodes and compute the rest?
  564. :A: See previous answer; this really just moves up one level, since
  565. one could alternatively interpret the data blocks as being the
  566. leaf nodes of the Merkle tree. It's true that the tree can be
  567. computed much faster if the leaf level is stored rather than just
  568. the data, but that's only because each level is less than 1% the
  569. size of the level below (assuming the recommended settings of
  570. SHA-256 and 4K blocks). For the exact same reason, by storing
  571. "just the leaf nodes" you'd already be storing over 99% of the
  572. tree, so you might as well simply store the whole tree.
  573. :Q: Can the Merkle tree be built ahead of time, e.g. distributed as
  574. part of a package that is installed to many computers?
  575. :A: This isn't currently supported. It was part of the original
  576. design, but was removed to simplify the kernel UAPI and because it
  577. wasn't a critical use case. Files are usually installed once and
  578. used many times, and cryptographic hashing is somewhat fast on
  579. most modern processors.
  580. :Q: Why doesn't fs-verity support writes?
  581. :A: Write support would be very difficult and would require a
  582. completely different design, so it's well outside the scope of
  583. fs-verity. Write support would require:
  584. - A way to maintain consistency between the data and hashes,
  585. including all levels of hashes, since corruption after a crash
  586. (especially of potentially the entire file!) is unacceptable.
  587. The main options for solving this are data journalling,
  588. copy-on-write, and log-structured volume. But it's very hard to
  589. retrofit existing filesystems with new consistency mechanisms.
  590. Data journalling is available on ext4, but is very slow.
  591. - Rebuilding the Merkle tree after every write, which would be
  592. extremely inefficient. Alternatively, a different authenticated
  593. dictionary structure such as an "authenticated skiplist" could
  594. be used. However, this would be far more complex.
  595. Compare it to dm-verity vs. dm-integrity. dm-verity is very
  596. simple: the kernel just verifies read-only data against a
  597. read-only Merkle tree. In contrast, dm-integrity supports writes
  598. but is slow, is much more complex, and doesn't actually support
  599. full-device authentication since it authenticates each sector
  600. independently, i.e. there is no "root hash". It doesn't really
  601. make sense for the same device-mapper target to support these two
  602. very different cases; the same applies to fs-verity.
  603. :Q: Since verity files are immutable, why isn't the immutable bit set?
  604. :A: The existing "immutable" bit (FS_IMMUTABLE_FL) already has a
  605. specific set of semantics which not only make the file contents
  606. read-only, but also prevent the file from being deleted, renamed,
  607. linked to, or having its owner or mode changed. These extra
  608. properties are unwanted for fs-verity, so reusing the immutable
  609. bit isn't appropriate.
  610. :Q: Why does the API use ioctls instead of setxattr() and getxattr()?
  611. :A: Abusing the xattr interface for basically arbitrary syscalls is
  612. heavily frowned upon by most of the Linux filesystem developers.
  613. An xattr should really just be an xattr on-disk, not an API to
  614. e.g. magically trigger construction of a Merkle tree.
  615. :Q: Does fs-verity support remote filesystems?
  616. :A: Only ext4 and f2fs support is implemented currently, but in
  617. principle any filesystem that can store per-file verity metadata
  618. can support fs-verity, regardless of whether it's local or remote.
  619. Some filesystems may have fewer options of where to store the
  620. verity metadata; one possibility is to store it past the end of
  621. the file and "hide" it from userspace by manipulating i_size. The
  622. data verification functions provided by ``fs/verity/`` also assume
  623. that the filesystem uses the Linux pagecache, but both local and
  624. remote filesystems normally do so.
  625. :Q: Why is anything filesystem-specific at all? Shouldn't fs-verity
  626. be implemented entirely at the VFS level?
  627. :A: There are many reasons why this is not possible or would be very
  628. difficult, including the following:
  629. - To prevent bypassing verification, pages must not be marked
  630. Uptodate until they've been verified. Currently, each
  631. filesystem is responsible for marking pages Uptodate via
  632. ``->readpages()``. Therefore, currently it's not possible for
  633. the VFS to do the verification on its own. Changing this would
  634. require significant changes to the VFS and all filesystems.
  635. - It would require defining a filesystem-independent way to store
  636. the verity metadata. Extended attributes don't work for this
  637. because (a) the Merkle tree may be gigabytes, but many
  638. filesystems assume that all xattrs fit into a single 4K
  639. filesystem block, and (b) ext4 and f2fs encryption doesn't
  640. encrypt xattrs, yet the Merkle tree *must* be encrypted when the
  641. file contents are, because it stores hashes of the plaintext
  642. file contents.
  643. So the verity metadata would have to be stored in an actual
  644. file. Using a separate file would be very ugly, since the
  645. metadata is fundamentally part of the file to be protected, and
  646. it could cause problems where users could delete the real file
  647. but not the metadata file or vice versa. On the other hand,
  648. having it be in the same file would break applications unless
  649. filesystems' notion of i_size were divorced from the VFS's,
  650. which would be complex and require changes to all filesystems.
  651. - It's desirable that FS_IOC_ENABLE_VERITY uses the filesystem's
  652. transaction mechanism so that either the file ends up with
  653. verity enabled, or no changes were made. Allowing intermediate
  654. states to occur after a crash may cause problems.