nilfs2.rst 11 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286
  1. .. SPDX-License-Identifier: GPL-2.0
  2. ======
  3. NILFS2
  4. ======
  5. NILFS2 is a log-structured file system (LFS) supporting continuous
  6. snapshotting. In addition to versioning capability of the entire file
  7. system, users can even restore files mistakenly overwritten or
  8. destroyed just a few seconds ago. Since NILFS2 can keep consistency
  9. like conventional LFS, it achieves quick recovery after system
  10. crashes.
  11. NILFS2 creates a number of checkpoints every few seconds or per
  12. synchronous write basis (unless there is no change). Users can select
  13. significant versions among continuously created checkpoints, and can
  14. change them into snapshots which will be preserved until they are
  15. changed back to checkpoints.
  16. There is no limit on the number of snapshots until the volume gets
  17. full. Each snapshot is mountable as a read-only file system
  18. concurrently with its writable mount, and this feature is convenient
  19. for online backup.
  20. The userland tools are included in nilfs-utils package, which is
  21. available from the following download page. At least "mkfs.nilfs2",
  22. "mount.nilfs2", "umount.nilfs2", and "nilfs_cleanerd" (so called
  23. cleaner or garbage collector) are required. Details on the tools are
  24. described in the man pages included in the package.
  25. :Project web page: https://nilfs.sourceforge.io/
  26. :Download page: https://nilfs.sourceforge.io/en/download.html
  27. :List info: http://vger.kernel.org/vger-lists.html#linux-nilfs
  28. Caveats
  29. =======
  30. Features which NILFS2 does not support yet:
  31. - atime
  32. - extended attributes
  33. - POSIX ACLs
  34. - quotas
  35. - fsck
  36. - defragmentation
  37. Mount options
  38. =============
  39. NILFS2 supports the following mount options:
  40. (*) == default
  41. ======================= =======================================================
  42. barrier(*) This enables/disables the use of write barriers. This
  43. nobarrier requires an IO stack which can support barriers, and
  44. if nilfs gets an error on a barrier write, it will
  45. disable again with a warning.
  46. errors=continue Keep going on a filesystem error.
  47. errors=remount-ro(*) Remount the filesystem read-only on an error.
  48. errors=panic Panic and halt the machine if an error occurs.
  49. cp=n Specify the checkpoint-number of the snapshot to be
  50. mounted. Checkpoints and snapshots are listed by lscp
  51. user command. Only the checkpoints marked as snapshot
  52. are mountable with this option. Snapshot is read-only,
  53. so a read-only mount option must be specified together.
  54. order=relaxed(*) Apply relaxed order semantics that allows modified data
  55. blocks to be written to disk without making a
  56. checkpoint if no metadata update is going. This mode
  57. is equivalent to the ordered data mode of the ext3
  58. filesystem except for the updates on data blocks still
  59. conserve atomicity. This will improve synchronous
  60. write performance for overwriting.
  61. order=strict Apply strict in-order semantics that preserves sequence
  62. of all file operations including overwriting of data
  63. blocks. That means, it is guaranteed that no
  64. overtaking of events occurs in the recovered file
  65. system after a crash.
  66. norecovery Disable recovery of the filesystem on mount.
  67. This disables every write access on the device for
  68. read-only mounts or snapshots. This option will fail
  69. for r/w mounts on an unclean volume.
  70. discard This enables/disables the use of discard/TRIM commands.
  71. nodiscard(*) The discard/TRIM commands are sent to the underlying
  72. block device when blocks are freed. This is useful
  73. for SSD devices and sparse/thinly-provisioned LUNs.
  74. ======================= =======================================================
  75. Ioctls
  76. ======
  77. There is some NILFS2 specific functionality which can be accessed by applications
  78. through the system call interfaces. The list of all NILFS2 specific ioctls are
  79. shown in the table below.
  80. Table of NILFS2 specific ioctls:
  81. ============================== ===============================================
  82. Ioctl Description
  83. ============================== ===============================================
  84. NILFS_IOCTL_CHANGE_CPMODE Change mode of given checkpoint between
  85. checkpoint and snapshot state. This ioctl is
  86. used in chcp and mkcp utilities.
  87. NILFS_IOCTL_DELETE_CHECKPOINT Remove checkpoint from NILFS2 file system.
  88. This ioctl is used in rmcp utility.
  89. NILFS_IOCTL_GET_CPINFO Return info about requested checkpoints. This
  90. ioctl is used in lscp utility and by
  91. nilfs_cleanerd daemon.
  92. NILFS_IOCTL_GET_CPSTAT Return checkpoints statistics. This ioctl is
  93. used by lscp, rmcp utilities and by
  94. nilfs_cleanerd daemon.
  95. NILFS_IOCTL_GET_SUINFO Return segment usage info about requested
  96. segments. This ioctl is used in lssu,
  97. nilfs_resize utilities and by nilfs_cleanerd
  98. daemon.
  99. NILFS_IOCTL_SET_SUINFO Modify segment usage info of requested
  100. segments. This ioctl is used by
  101. nilfs_cleanerd daemon to skip unnecessary
  102. cleaning operation of segments and reduce
  103. performance penalty or wear of flash device
  104. due to redundant move of in-use blocks.
  105. NILFS_IOCTL_GET_SUSTAT Return segment usage statistics. This ioctl
  106. is used in lssu, nilfs_resize utilities and
  107. by nilfs_cleanerd daemon.
  108. NILFS_IOCTL_GET_VINFO Return information on virtual block addresses.
  109. This ioctl is used by nilfs_cleanerd daemon.
  110. NILFS_IOCTL_GET_BDESCS Return information about descriptors of disk
  111. block numbers. This ioctl is used by
  112. nilfs_cleanerd daemon.
  113. NILFS_IOCTL_CLEAN_SEGMENTS Do garbage collection operation in the
  114. environment of requested parameters from
  115. userspace. This ioctl is used by
  116. nilfs_cleanerd daemon.
  117. NILFS_IOCTL_SYNC Make a checkpoint. This ioctl is used in
  118. mkcp utility.
  119. NILFS_IOCTL_RESIZE Resize NILFS2 volume. This ioctl is used
  120. by nilfs_resize utility.
  121. NILFS_IOCTL_SET_ALLOC_RANGE Define lower limit of segments in bytes and
  122. upper limit of segments in bytes. This ioctl
  123. is used by nilfs_resize utility.
  124. ============================== ===============================================
  125. NILFS2 usage
  126. ============
  127. To use nilfs2 as a local file system, simply::
  128. # mkfs -t nilfs2 /dev/block_device
  129. # mount -t nilfs2 /dev/block_device /dir
  130. This will also invoke the cleaner through the mount helper program
  131. (mount.nilfs2).
  132. Checkpoints and snapshots are managed by the following commands.
  133. Their manpages are included in the nilfs-utils package above.
  134. ==== ===========================================================
  135. lscp list checkpoints or snapshots.
  136. mkcp make a checkpoint or a snapshot.
  137. chcp change an existing checkpoint to a snapshot or vice versa.
  138. rmcp invalidate specified checkpoint(s).
  139. ==== ===========================================================
  140. To mount a snapshot::
  141. # mount -t nilfs2 -r -o cp=<cno> /dev/block_device /snap_dir
  142. where <cno> is the checkpoint number of the snapshot.
  143. To unmount the NILFS2 mount point or snapshot, simply::
  144. # umount /dir
  145. Then, the cleaner daemon is automatically shut down by the umount
  146. helper program (umount.nilfs2).
  147. Disk format
  148. ===========
  149. A nilfs2 volume is equally divided into a number of segments except
  150. for the super block (SB) and segment #0. A segment is the container
  151. of logs. Each log is composed of summary information blocks, payload
  152. blocks, and an optional super root block (SR)::
  153. ______________________________________________________
  154. | |SB| | Segment | Segment | Segment | ... | Segment | |
  155. |_|__|_|____0____|____1____|____2____|_____|____N____|_|
  156. 0 +1K +4K +8M +16M +24M +(8MB x N)
  157. . . (Typical offsets for 4KB-block)
  158. . .
  159. .______________________.
  160. | log | log |... | log |
  161. |__1__|__2__|____|__m__|
  162. . .
  163. . .
  164. . .
  165. .______________________________.
  166. | Summary | Payload blocks |SR|
  167. |_blocks__|_________________|__|
  168. The payload blocks are organized per file, and each file consists of
  169. data blocks and B-tree node blocks::
  170. |<--- File-A --->|<--- File-B --->|
  171. _______________________________________________________________
  172. | Data blocks | B-tree blocks | Data blocks | B-tree blocks | ...
  173. _|_____________|_______________|_____________|_______________|_
  174. Since only the modified blocks are written in the log, it may have
  175. files without data blocks or B-tree node blocks.
  176. The organization of the blocks is recorded in the summary information
  177. blocks, which contains a header structure (nilfs_segment_summary), per
  178. file structures (nilfs_finfo), and per block structures (nilfs_binfo)::
  179. _________________________________________________________________________
  180. | Summary | finfo | binfo | ... | binfo | finfo | binfo | ... | binfo |...
  181. |_blocks__|___A___|_(A,1)_|_____|(A,Na)_|___B___|_(B,1)_|_____|(B,Nb)_|___
  182. The logs include regular files, directory files, symbolic link files
  183. and several meta data files. The mata data files are the files used
  184. to maintain file system meta data. The current version of NILFS2 uses
  185. the following meta data files::
  186. 1) Inode file (ifile) -- Stores on-disk inodes
  187. 2) Checkpoint file (cpfile) -- Stores checkpoints
  188. 3) Segment usage file (sufile) -- Stores allocation state of segments
  189. 4) Data address translation file -- Maps virtual block numbers to usual
  190. (DAT) block numbers. This file serves to
  191. make on-disk blocks relocatable.
  192. The following figure shows a typical organization of the logs::
  193. _________________________________________________________________________
  194. | Summary | regular file | file | ... | ifile | cpfile | sufile | DAT |SR|
  195. |_blocks__|_or_directory_|_______|_____|_______|________|________|_____|__|
  196. To stride over segment boundaries, this sequence of files may be split
  197. into multiple logs. The sequence of logs that should be treated as
  198. logically one log, is delimited with flags marked in the segment
  199. summary. The recovery code of nilfs2 looks this boundary information
  200. to ensure atomicity of updates.
  201. The super root block is inserted for every checkpoints. It includes
  202. three special inodes, inodes for the DAT, cpfile, and sufile. Inodes
  203. of regular files, directories, symlinks and other special files, are
  204. included in the ifile. The inode of ifile itself is included in the
  205. corresponding checkpoint entry in the cpfile. Thus, the hierarchy
  206. among NILFS2 files can be depicted as follows::
  207. Super block (SB)
  208. |
  209. v
  210. Super root block (the latest cno=xx)
  211. |-- DAT
  212. |-- sufile
  213. `-- cpfile
  214. |-- ifile (cno=c1)
  215. |-- ifile (cno=c2) ---- file (ino=i1)
  216. : : |-- file (ino=i2)
  217. `-- ifile (cno=xx) |-- file (ino=i3)
  218. : :
  219. `-- file (ino=yy)
  220. ( regular file, directory, or symlink )
  221. For detail on the format of each file, please see nilfs2_ondisk.h
  222. located at include/uapi/linux directory.
  223. There are no patents or other intellectual property that we protect
  224. with regard to the design of NILFS2. It is allowed to replicate the
  225. design in hopes that other operating systems could share (mount, read,
  226. write, etc.) data stored in this format.