lzma-sdk.txt 13 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357
  1. LZMA SDK 18.05
  2. --------------
  3. LZMA SDK provides the documentation, samples, header files,
  4. libraries, and tools you need to develop applications that
  5. use 7z / LZMA / LZMA2 / XZ compression.
  6. LZMA is an improved version of famous LZ77 compression algorithm.
  7. It was improved in way of maximum increasing of compression ratio,
  8. keeping high decompression speed and low memory requirements for
  9. decompressing.
  10. LZMA2 is a LZMA based compression method. LZMA2 provides better
  11. multithreading support for compression than LZMA and some other improvements.
  12. 7z is a file format for data compression and file archiving.
  13. 7z is a main file format for 7-Zip compression program (www.7-zip.org).
  14. 7z format supports different compression methods: LZMA, LZMA2 and others.
  15. 7z also supports AES-256 based encryption.
  16. XZ is a file format for data compression that uses LZMA2 compression.
  17. XZ format provides additional features: SHA/CRC check, filters for
  18. improved compression ratio, splitting to blocks and streams,
  19. LICENSE
  20. -------
  21. LZMA SDK is written and placed in the public domain by Igor Pavlov.
  22. Some code in LZMA SDK is based on public domain code from another developers:
  23. 1) PPMd var.H (2001): Dmitry Shkarin
  24. 2) SHA-256: Wei Dai (Crypto++ library)
  25. Anyone is free to copy, modify, publish, use, compile, sell, or distribute the
  26. original LZMA SDK code, either in source code form or as a compiled binary, for
  27. any purpose, commercial or non-commercial, and by any means.
  28. LZMA SDK code is compatible with open source licenses, for example, you can
  29. include it to GNU GPL or GNU LGPL code.
  30. LZMA SDK Contents
  31. -----------------
  32. Source code:
  33. - C / C++ / C# / Java - LZMA compression and decompression
  34. - C / C++ - LZMA2 compression and decompression
  35. - C / C++ - XZ compression and decompression
  36. - C - 7z decompression
  37. - C++ - 7z compression and decompression
  38. - C - small SFXs for installers (7z decompression)
  39. - C++ - SFXs and SFXs for installers (7z decompression)
  40. Precompiled binaries:
  41. - console programs for lzma / 7z / xz compression and decompression
  42. - SFX modules for installers.
  43. UNIX/Linux version
  44. ------------------
  45. To compile C++ version of file->file LZMA encoding, go to directory
  46. CPP/7zip/Bundles/LzmaCon
  47. and call make to recompile it:
  48. make -f makefile.gcc clean all
  49. In some UNIX/Linux versions you must compile LZMA with static libraries.
  50. To compile with static libraries, you can use
  51. LIB = -lm -static
  52. Also you can use p7zip (port of 7-Zip for POSIX systems like Unix or Linux):
  53. http://p7zip.sourceforge.net/
  54. Files
  55. -----
  56. DOC/7zC.txt - 7z ANSI-C Decoder description
  57. DOC/7zFormat.txt - 7z Format description
  58. DOC/installer.txt - information about 7-Zip for installers
  59. DOC/lzma.txt - LZMA compression description
  60. DOC/lzma-sdk.txt - LZMA SDK description (this file)
  61. DOC/lzma-history.txt - history of LZMA SDK
  62. DOC/lzma-specification.txt - Specification of LZMA
  63. DOC/Methods.txt - Compression method IDs for .7z
  64. bin/installer/ - example script to create installer that uses SFX module,
  65. bin/7zdec.exe - simplified 7z archive decoder
  66. bin/7zr.exe - 7-Zip console program (reduced version)
  67. bin/x64/7zr.exe - 7-Zip console program (reduced version) (x64 version)
  68. bin/lzma.exe - file->file LZMA encoder/decoder for Windows
  69. bin/7zS2.sfx - small SFX module for installers (GUI version)
  70. bin/7zS2con.sfx - small SFX module for installers (Console version)
  71. bin/7zSD.sfx - SFX module for installers.
  72. 7zDec.exe
  73. ---------
  74. 7zDec.exe is simplified 7z archive decoder.
  75. It supports only LZMA, LZMA2, and PPMd methods.
  76. 7zDec decodes whole solid block from 7z archive to RAM.
  77. The RAM consumption can be high.
  78. Source code structure
  79. ---------------------
  80. Asm/ - asm files (optimized code for CRC calculation and Intel-AES encryption)
  81. C/ - C files (compression / decompression and other)
  82. Util/
  83. 7z - 7z decoder program (decoding 7z files)
  84. Lzma - LZMA program (file->file LZMA encoder/decoder).
  85. LzmaLib - LZMA library (.DLL for Windows)
  86. SfxSetup - small SFX module for installers
  87. CPP/ -- CPP files
  88. Common - common files for C++ projects
  89. Windows - common files for Windows related code
  90. 7zip - files related to 7-Zip
  91. Archive - files related to archiving
  92. Common - common files for archive handling
  93. 7z - 7z C++ Encoder/Decoder
  94. Bundles - Modules that are bundles of other modules (files)
  95. Alone7z - 7zr.exe: Standalone 7-Zip console program (reduced version)
  96. Format7zExtractR - 7zxr.dll: Reduced version of 7z DLL: extracting from 7z/LZMA/BCJ/BCJ2.
  97. Format7zR - 7zr.dll: Reduced version of 7z DLL: extracting/compressing to 7z/LZMA/BCJ/BCJ2
  98. LzmaCon - lzma.exe: LZMA compression/decompression
  99. LzmaSpec - example code for LZMA Specification
  100. SFXCon - 7zCon.sfx: Console 7z SFX module
  101. SFXSetup - 7zS.sfx: 7z SFX module for installers
  102. SFXWin - 7z.sfx: GUI 7z SFX module
  103. Common - common files for 7-Zip
  104. Compress - files for compression/decompression
  105. Crypto - files for encryption / decompression
  106. UI - User Interface files
  107. Client7z - Test application for 7za.dll, 7zr.dll, 7zxr.dll
  108. Common - Common UI files
  109. Console - Code for console program (7z.exe)
  110. Explorer - Some code from 7-Zip Shell extension
  111. FileManager - Some GUI code from 7-Zip File Manager
  112. GUI - Some GUI code from 7-Zip
  113. CS/ - C# files
  114. 7zip
  115. Common - some common files for 7-Zip
  116. Compress - files related to compression/decompression
  117. LZ - files related to LZ (Lempel-Ziv) compression algorithm
  118. LZMA - LZMA compression/decompression
  119. LzmaAlone - file->file LZMA compression/decompression
  120. RangeCoder - Range Coder (special code of compression/decompression)
  121. Java/ - Java files
  122. SevenZip
  123. Compression - files related to compression/decompression
  124. LZ - files related to LZ (Lempel-Ziv) compression algorithm
  125. LZMA - LZMA compression/decompression
  126. RangeCoder - Range Coder (special code of compression/decompression)
  127. Note:
  128. Asm / C / C++ source code of LZMA SDK is part of 7-Zip's source code.
  129. 7-Zip's source code can be downloaded from 7-Zip's SourceForge page:
  130. http://sourceforge.net/projects/sevenzip/
  131. LZMA features
  132. -------------
  133. - Variable dictionary size (up to 1 GB)
  134. - Estimated compressing speed: about 2 MB/s on 2 GHz CPU
  135. - Estimated decompressing speed:
  136. - 20-30 MB/s on modern 2 GHz cpu
  137. - 1-2 MB/s on 200 MHz simple RISC cpu: (ARM, MIPS, PowerPC)
  138. - Small memory requirements for decompressing (16 KB + DictionarySize)
  139. - Small code size for decompressing: 5-8 KB
  140. LZMA decoder uses only integer operations and can be
  141. implemented in any modern 32-bit CPU (or on 16-bit CPU with some conditions).
  142. Some critical operations that affect the speed of LZMA decompression:
  143. 1) 32*16 bit integer multiply
  144. 2) Mispredicted branches (penalty mostly depends from pipeline length)
  145. 3) 32-bit shift and arithmetic operations
  146. The speed of LZMA decompressing mostly depends from CPU speed.
  147. Memory speed has no big meaning. But if your CPU has small data cache,
  148. overall weight of memory speed will slightly increase.
  149. How To Use
  150. ----------
  151. Using LZMA encoder/decoder executable
  152. --------------------------------------
  153. Usage: LZMA <e|d> inputFile outputFile [<switches>...]
  154. e: encode file
  155. d: decode file
  156. b: Benchmark. There are two tests: compressing and decompressing
  157. with LZMA method. Benchmark shows rating in MIPS (million
  158. instructions per second). Rating value is calculated from
  159. measured speed and it is normalized with Intel's Core 2 results.
  160. Also Benchmark checks possible hardware errors (RAM
  161. errors in most cases). Benchmark uses these settings:
  162. (-a1, -d21, -fb32, -mfbt4). You can change only -d parameter.
  163. Also you can change the number of iterations. Example for 30 iterations:
  164. LZMA b 30
  165. Default number of iterations is 10.
  166. <Switches>
  167. -a{N}: set compression mode 0 = fast, 1 = normal
  168. default: 1 (normal)
  169. d{N}: Sets Dictionary size - [0, 30], default: 23 (8MB)
  170. The maximum value for dictionary size is 1 GB = 2^30 bytes.
  171. Dictionary size is calculated as DictionarySize = 2^N bytes.
  172. For decompressing file compressed by LZMA method with dictionary
  173. size D = 2^N you need about D bytes of memory (RAM).
  174. -fb{N}: set number of fast bytes - [5, 273], default: 128
  175. Usually big number gives a little bit better compression ratio
  176. and slower compression process.
  177. -lc{N}: set number of literal context bits - [0, 8], default: 3
  178. Sometimes lc=4 gives gain for big files.
  179. -lp{N}: set number of literal pos bits - [0, 4], default: 0
  180. lp switch is intended for periodical data when period is
  181. equal 2^N. For example, for 32-bit (4 bytes)
  182. periodical data you can use lp=2. Often it's better to set lc0,
  183. if you change lp switch.
  184. -pb{N}: set number of pos bits - [0, 4], default: 2
  185. pb switch is intended for periodical data
  186. when period is equal 2^N.
  187. -mf{MF_ID}: set Match Finder. Default: bt4.
  188. Algorithms from hc* group doesn't provide good compression
  189. ratio, but they often works pretty fast in combination with
  190. fast mode (-a0).
  191. Memory requirements depend from dictionary size
  192. (parameter "d" in table below).
  193. MF_ID Memory Description
  194. bt2 d * 9.5 + 4MB Binary Tree with 2 bytes hashing.
  195. bt3 d * 11.5 + 4MB Binary Tree with 3 bytes hashing.
  196. bt4 d * 11.5 + 4MB Binary Tree with 4 bytes hashing.
  197. hc4 d * 7.5 + 4MB Hash Chain with 4 bytes hashing.
  198. -eos: write End Of Stream marker. By default LZMA doesn't write
  199. eos marker, since LZMA decoder knows uncompressed size
  200. stored in .lzma file header.
  201. -si: Read data from stdin (it will write End Of Stream marker).
  202. -so: Write data to stdout
  203. Examples:
  204. 1) LZMA e file.bin file.lzma -d16 -lc0
  205. compresses file.bin to file.lzma with 64 KB dictionary (2^16=64K)
  206. and 0 literal context bits. -lc0 allows to reduce memory requirements
  207. for decompression.
  208. 2) LZMA e file.bin file.lzma -lc0 -lp2
  209. compresses file.bin to file.lzma with settings suitable
  210. for 32-bit periodical data (for example, ARM or MIPS code).
  211. 3) LZMA d file.lzma file.bin
  212. decompresses file.lzma to file.bin.
  213. Compression ratio hints
  214. -----------------------
  215. Recommendations
  216. ---------------
  217. To increase the compression ratio for LZMA compressing it's desirable
  218. to have aligned data (if it's possible) and also it's desirable to locate
  219. data in such order, where code is grouped in one place and data is
  220. grouped in other place (it's better than such mixing: code, data, code,
  221. data, ...).
  222. Filters
  223. -------
  224. You can increase the compression ratio for some data types, using
  225. special filters before compressing. For example, it's possible to
  226. increase the compression ratio on 5-10% for code for those CPU ISAs:
  227. x86, IA-64, ARM, ARM-Thumb, PowerPC, SPARC.
  228. You can find C source code of such filters in C/Bra*.* files
  229. You can check the compression ratio gain of these filters with such
  230. 7-Zip commands (example for ARM code):
  231. No filter:
  232. 7z a a1.7z a.bin -m0=lzma
  233. With filter for little-endian ARM code:
  234. 7z a a2.7z a.bin -m0=arm -m1=lzma
  235. It works in such manner:
  236. Compressing = Filter_encoding + LZMA_encoding
  237. Decompressing = LZMA_decoding + Filter_decoding
  238. Compressing and decompressing speed of such filters is very high,
  239. so it will not increase decompressing time too much.
  240. Moreover, it reduces decompression time for LZMA_decoding,
  241. since compression ratio with filtering is higher.
  242. These filters convert CALL (calling procedure) instructions
  243. from relative offsets to absolute addresses, so such data becomes more
  244. compressible.
  245. For some ISAs (for example, for MIPS) it's impossible to get gain from such filter.
  246. ---
  247. http://www.7-zip.org
  248. http://www.7-zip.org/sdk.html
  249. http://www.7-zip.org/support.html