erofs.rst 11 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240
  1. .. SPDX-License-Identifier: GPL-2.0
  2. ======================================
  3. Enhanced Read-Only File System - EROFS
  4. ======================================
  5. Overview
  6. ========
  7. EROFS file-system stands for Enhanced Read-Only File System. Different
  8. from other read-only file systems, it aims to be designed for flexibility,
  9. scalability, but be kept simple and high performance.
  10. It is designed as a better filesystem solution for the following scenarios:
  11. - read-only storage media or
  12. - part of a fully trusted read-only solution, which means it needs to be
  13. immutable and bit-for-bit identical to the official golden image for
  14. their releases due to security and other considerations and
  15. - hope to save some extra storage space with guaranteed end-to-end performance
  16. by using reduced metadata and transparent file compression, especially
  17. for those embedded devices with limited memory (ex, smartphone);
  18. Here is the main features of EROFS:
  19. - Little endian on-disk design;
  20. - Currently 4KB block size (nobh) and therefore maximum 16TB address space;
  21. - Metadata & data could be mixed by design;
  22. - 2 inode versions for different requirements:
  23. ===================== ============ =====================================
  24. compact (v1) extended (v2)
  25. ===================== ============ =====================================
  26. Inode metadata size 32 bytes 64 bytes
  27. Max file size 4 GB 16 EB (also limited by max. vol size)
  28. Max uids/gids 65536 4294967296
  29. File change time no yes (64 + 32-bit timestamp)
  30. Max hardlinks 65536 4294967296
  31. Metadata reserved 4 bytes 14 bytes
  32. ===================== ============ =====================================
  33. - Support extended attributes (xattrs) as an option;
  34. - Support xattr inline and tail-end data inline for all files;
  35. - Support POSIX.1e ACLs by using xattrs;
  36. - Support transparent file compression as an option:
  37. LZ4 algorithm with 4 KB fixed-sized output compression for high performance.
  38. The following git tree provides the file system user-space tools under
  39. development (ex, formatting tool mkfs.erofs):
  40. - git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git
  41. Bugs and patches are welcome, please kindly help us and send to the following
  42. linux-erofs mailing list:
  43. - linux-erofs mailing list <linux-erofs@lists.ozlabs.org>
  44. Mount options
  45. =============
  46. =================== =========================================================
  47. (no)user_xattr Setup Extended User Attributes. Note: xattr is enabled
  48. by default if CONFIG_EROFS_FS_XATTR is selected.
  49. (no)acl Setup POSIX Access Control List. Note: acl is enabled
  50. by default if CONFIG_EROFS_FS_POSIX_ACL is selected.
  51. cache_strategy=%s Select a strategy for cached decompression from now on:
  52. ========== =============================================
  53. disabled In-place I/O decompression only;
  54. readahead Cache the last incomplete compressed physical
  55. cluster for further reading. It still does
  56. in-place I/O decompression for the rest
  57. compressed physical clusters;
  58. readaround Cache the both ends of incomplete compressed
  59. physical clusters for further reading.
  60. It still does in-place I/O decompression
  61. for the rest compressed physical clusters.
  62. ========== =============================================
  63. =================== =========================================================
  64. On-disk details
  65. ===============
  66. Summary
  67. -------
  68. Different from other read-only file systems, an EROFS volume is designed
  69. to be as simple as possible::
  70. |-> aligned with the block size
  71. ____________________________________________________________
  72. | |SB| | ... | Metadata | ... | Data | Metadata | ... | Data |
  73. |_|__|_|_____|__________|_____|______|__________|_____|______|
  74. 0 +1K
  75. All data areas should be aligned with the block size, but metadata areas
  76. may not. All metadatas can be now observed in two different spaces (views):
  77. 1. Inode metadata space
  78. Each valid inode should be aligned with an inode slot, which is a fixed
  79. value (32 bytes) and designed to be kept in line with compact inode size.
  80. Each inode can be directly found with the following formula:
  81. inode offset = meta_blkaddr * block_size + 32 * nid
  82. ::
  83. |-> aligned with 8B
  84. |-> followed closely
  85. + meta_blkaddr blocks |-> another slot
  86. _____________________________________________________________________
  87. | ... | inode | xattrs | extents | data inline | ... | inode ...
  88. |________|_______|(optional)|(optional)|__(optional)_|_____|__________
  89. |-> aligned with the inode slot size
  90. . .
  91. . .
  92. . .
  93. . .
  94. . .
  95. . .
  96. .____________________________________________________|-> aligned with 4B
  97. | xattr_ibody_header | shared xattrs | inline xattrs |
  98. |____________________|_______________|_______________|
  99. |-> 12 bytes <-|->x * 4 bytes<-| .
  100. . . .
  101. . . .
  102. . . .
  103. ._______________________________.______________________.
  104. | id | id | id | id | ... | id | ent | ... | ent| ... |
  105. |____|____|____|____|______|____|_____|_____|____|_____|
  106. |-> aligned with 4B
  107. |-> aligned with 4B
  108. Inode could be 32 or 64 bytes, which can be distinguished from a common
  109. field which all inode versions have -- i_format::
  110. __________________ __________________
  111. | i_format | | i_format |
  112. |__________________| |__________________|
  113. | ... | | ... |
  114. | | | |
  115. |__________________| 32 bytes | |
  116. | |
  117. |__________________| 64 bytes
  118. Xattrs, extents, data inline are followed by the corresponding inode with
  119. proper alignment, and they could be optional for different data mappings.
  120. _currently_ total 4 valid data mappings are supported:
  121. == ====================================================================
  122. 0 flat file data without data inline (no extent);
  123. 1 fixed-sized output data compression (with non-compacted indexes);
  124. 2 flat file data with tail packing data inline (no extent);
  125. 3 fixed-sized output data compression (with compacted indexes, v5.3+).
  126. == ====================================================================
  127. The size of the optional xattrs is indicated by i_xattr_count in inode
  128. header. Large xattrs or xattrs shared by many different files can be
  129. stored in shared xattrs metadata rather than inlined right after inode.
  130. 2. Shared xattrs metadata space
  131. Shared xattrs space is similar to the above inode space, started with
  132. a specific block indicated by xattr_blkaddr, organized one by one with
  133. proper align.
  134. Each share xattr can also be directly found by the following formula:
  135. xattr offset = xattr_blkaddr * block_size + 4 * xattr_id
  136. ::
  137. |-> aligned by 4 bytes
  138. + xattr_blkaddr blocks |-> aligned with 4 bytes
  139. _________________________________________________________________________
  140. | ... | xattr_entry | xattr data | ... | xattr_entry | xattr data ...
  141. |________|_____________|_____________|_____|______________|_______________
  142. Directories
  143. -----------
  144. All directories are now organized in a compact on-disk format. Note that
  145. each directory block is divided into index and name areas in order to support
  146. random file lookup, and all directory entries are _strictly_ recorded in
  147. alphabetical order in order to support improved prefix binary search
  148. algorithm (could refer to the related source code).
  149. ::
  150. ___________________________
  151. / |
  152. / ______________|________________
  153. / / | nameoff1 | nameoffN-1
  154. ____________.______________._______________v________________v__________
  155. | dirent | dirent | ... | dirent | filename | filename | ... | filename |
  156. |___.0___|____1___|_____|___N-1__|____0_____|____1_____|_____|___N-1____|
  157. \ ^
  158. \ | * could have
  159. \ | trailing '\0'
  160. \________________________| nameoff0
  161. Directory block
  162. Note that apart from the offset of the first filename, nameoff0 also indicates
  163. the total number of directory entries in this block since it is no need to
  164. introduce another on-disk field at all.
  165. Compression
  166. -----------
  167. Currently, EROFS supports 4KB fixed-sized output transparent file compression,
  168. as illustrated below::
  169. |---- Variant-Length Extent ----|-------- VLE --------|----- VLE -----
  170. clusterofs clusterofs clusterofs
  171. | | | logical data
  172. _________v_______________________________v_____________________v_______________
  173. ... | . | | . | | . | ...
  174. ____|____.________|_____________|________.____|_____________|__.__________|____
  175. |-> cluster <-|-> cluster <-|-> cluster <-|-> cluster <-|-> cluster <-|
  176. size size size size size
  177. . . . .
  178. . . . .
  179. . . . .
  180. _______._____________._____________._____________._____________________
  181. ... | | | | ... physical data
  182. _______|_____________|_____________|_____________|_____________________
  183. |-> cluster <-|-> cluster <-|-> cluster <-|
  184. size size size
  185. Currently each on-disk physical cluster can contain 4KB (un)compressed data
  186. at most. For each logical cluster, there is a corresponding on-disk index to
  187. describe its cluster type, physical cluster address, etc.
  188. See "struct z_erofs_vle_decompressed_index" in erofs_fs.h for more details.