README.rst 8.5 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213
  1. Subzero - Fast code generator for PNaCl bitcode
  2. ===============================================
  3. Design
  4. ------
  5. See the accompanying DESIGN.rst file for a more detailed technical overview of
  6. Subzero.
  7. Building
  8. --------
  9. Subzero is set up to be built within the Native Client tree. Follow the
  10. `Developing PNaCl
  11. <https://sites.google.com/a/chromium.org/dev/nativeclient/pnacl/developing-pnacl>`_
  12. instructions, in particular the section on building PNaCl sources. This will
  13. prepare the necessary external headers and libraries that Subzero needs.
  14. Checking out the Native Client project also gets the pre-built clang and LLVM
  15. tools in ``native_client/../third_party/llvm-build/Release+Asserts/bin`` which
  16. are used for building Subzero.
  17. The Subzero source is in ``native_client/toolchain_build/src/subzero``. From
  18. within that directory, ``git checkout master && git pull`` to get the latest
  19. version of Subzero source code.
  20. The Makefile is designed to be used as part of the higher level LLVM build
  21. system. To build manually, use the ``Makefile.standalone``. There are several
  22. build configurations from the command line::
  23. make -f Makefile.standalone
  24. make -f Makefile.standalone DEBUG=1
  25. make -f Makefile.standalone NOASSERT=1
  26. make -f Makefile.standalone DEBUG=1 NOASSERT=1
  27. make -f Makefile.standalone MINIMAL=1
  28. make -f Makefile.standalone ASAN=1
  29. make -f Makefile.standalone TSAN=1
  30. ``DEBUG=1`` builds without optimizations and is good when running the translator
  31. inside a debugger. ``NOASSERT=1`` disables assertions and is the preferred
  32. configuration for performance testing the translator. ``MINIMAL=1`` attempts to
  33. minimize the size of the translator by compiling out everything unnecessary.
  34. ``ASAN=1`` enables AddressSanitizer, and ``TSAN=1`` enables ThreadSanitizer.
  35. The result of the ``make`` command is the target ``pnacl-sz`` in the current
  36. directory.
  37. Building within LLVM trunk
  38. --------------------------
  39. Subzero can also be built from within a standard LLVM trunk checkout. Here is
  40. an example of how it can be checked out and built::
  41. mkdir llvm-git
  42. cd llvm-git
  43. git clone http://llvm.org/git/llvm.git
  44. cd llvm/projects/
  45. git clone https://chromium.googlesource.com/native_client/pnacl-subzero
  46. cd ../..
  47. mkdir build
  48. cd build
  49. cmake -G Ninja ../llvm/
  50. ninja
  51. ./bin/pnacl-sz -version
  52. This creates a default build of ``pnacl-sz``; currently any options such as
  53. ``DEBUG=1`` or ``MINIMAL=1`` have to be added manually.
  54. ``pnacl-sz``
  55. ------------
  56. The ``pnacl-sz`` program parses a pexe or an LLVM bitcode file and translates it
  57. into ICE (Subzero's intermediate representation). It then invokes the ICE
  58. translate method to lower it to target-specific machine code, optionally dumping
  59. the intermediate representation at various stages of the translation.
  60. The program can be run as follows::
  61. ../pnacl-sz ./path/to/<file>.pexe
  62. ../pnacl-sz ./tests_lit/pnacl-sz_tests/<file>.ll
  63. At this time, ``pnacl-sz`` accepts a number of arguments, including the
  64. following:
  65. ``-help`` -- Show available arguments and possible values. (Note: this
  66. unfortunately also pulls in some LLVM-specific options that are reported but
  67. that Subzero doesn't use.)
  68. ``-notranslate`` -- Suppress the ICE translation phase, which is useful if
  69. ICE is missing some support.
  70. ``-target=<TARGET>`` -- Set the target architecture. The default is x8632.
  71. Future targets include x8664, arm32, and arm64.
  72. ``-filetype=obj|asm|iasm`` -- Select the output file type. ``obj`` is a
  73. native ELF file, ``asm`` is a textual assembly file, and ``iasm`` is a
  74. low-level textual assembly file demonstrating the integrated assembler.
  75. ``-O<LEVEL>`` -- Set the optimization level. Valid levels are ``2``, ``1``,
  76. ``0``, ``-1``, and ``m1``. Levels ``-1`` and ``m1`` are synonyms, and
  77. represent the minimum optimization and worst code quality, but fastest code
  78. generation.
  79. ``-verbose=<list>`` -- Set verbosity flags. This argument allows a
  80. comma-separated list of values. The default is ``none``, and the value
  81. ``inst,pred`` will roughly match the .ll bitcode file. Of particular use
  82. are ``all``, ``most``, and ``none``.
  83. ``-o <FILE>`` -- Set the assembly output file name. Default is stdout.
  84. ``-log <FILE>`` -- Set the file name for diagnostic output (whose level is
  85. controlled by ``-verbose``). Default is stdout.
  86. ``-timing`` -- Dump some pass timing information after translating the input
  87. file.
  88. Running the test suite
  89. ----------------------
  90. Subzero uses the LLVM ``lit`` testing tool for part of its test suite, which
  91. lives in ``tests_lit``. To execute the test suite, first build Subzero, and then
  92. run::
  93. make -f Makefile.standalone check-lit
  94. There is also a suite of cross tests in the ``crosstest`` directory. A cross
  95. test takes a test bitcode file implementing some unit tests, and translates it
  96. twice, once with Subzero and once with LLVM's known-good ``llc`` translator.
  97. The Subzero-translated symbols are specially mangled to avoid multiple
  98. definition errors from the linker. Both translated versions are linked together
  99. with a driver program that calls each version of each unit test with a variety
  100. of interesting inputs and compares the results for equality. The cross tests
  101. are currently invoked by running::
  102. make -f Makefile.standalone check-xtest
  103. Similar, there is a suite of unit tests::
  104. make -f Makefile.standalone check-unit
  105. A convenient way to run the lit, cross, and unit tests is::
  106. make -f Makefile.standalone check
  107. Assembling ``pnacl-sz`` output as needed
  108. ----------------------------------------
  109. ``pnacl-sz`` can now produce a native ELF binary using ``-filetype=obj``.
  110. ``pnacl-sz`` can also produce textual assembly code in a structure suitable for
  111. input to ``llvm-mc``, using ``-filetype=asm`` or ``-filetype=iasm``. An object
  112. file can then be produced using the command::
  113. llvm-mc -triple=i686 -filetype=obj -o=MyObj.o
  114. Building a translated binary
  115. ----------------------------
  116. There is a helper script, ``pydir/szbuild.py``, that translates a finalized pexe
  117. into a fully linked executable. Run it with ``-help`` for extensive
  118. documentation.
  119. By default, ``szbuild.py`` builds an executable using only Subzero translation,
  120. but it can also be used to produce hybrid Subzero/``llc`` binaries (``llc`` is
  121. the name of the LLVM translator) for bisection-based debugging. In bisection
  122. debugging mode, the pexe is translated using both Subzero and ``llc``, and the
  123. resulting object files are combined into a single executable using symbol
  124. weakening and other linker tricks to control which Subzero symbols and which
  125. ``llc`` symbols take precedence. This is controlled by the ``-include`` and
  126. ``-exclude`` arguments. These can be used to rapidly find a single function
  127. that Subzero translates incorrectly leading to incorrect output.
  128. There is another helper script, ``pydir/szbuild_spec2k.py``, that runs
  129. ``szbuild.py`` on one or more components of the Spec2K suite. This assumes that
  130. Spec2K is set up in the usual place in the Native Client tree, and the finalized
  131. pexe files have been built. (Note: for working with Spec2K and other pexes,
  132. it's helpful to finalize the pexe using ``--no-strip-syms``, to preserve the
  133. original function and global variable names.)
  134. Status
  135. ------
  136. Subzero currently fully supports the x86-32 architecture, for both native and
  137. Native Client sandboxing modes. The x86-64 architecture is also supported in
  138. native mode only, and only for the x32 flavor due to the fact that pointers and
  139. 32-bit integers are indistinguishable in PNaCl bitcode. Sandboxing support for
  140. x86-64 is in progress. ARM and MIPS support is in progress. Two optimization
  141. levels, ``-Om1`` and ``-O2``, are implemented.
  142. The ``-Om1`` configuration is designed to be the simplest and fastest possible,
  143. with a minimal set of passes and transformations.
  144. * Simple Phi lowering before target lowering, by generating temporaries and
  145. adding assignments to the end of predecessor blocks.
  146. * Simple register allocation limited to pre-colored or infinite-weight
  147. Variables.
  148. The ``-O2`` configuration is designed to use all optimizations available and
  149. produce the best code.
  150. * Address mode inference to leverage the complex x86 addressing modes.
  151. * Compare/branch fusing based on liveness/last-use analysis.
  152. * Global, linear-scan register allocation.
  153. * Advanced phi lowering after target lowering and global register allocation,
  154. via edge splitting, topological sorting of the parallel moves, and final local
  155. register allocation.
  156. * Stack slot coalescing to reduce frame size.
  157. * Branch optimization to reduce the number of branches to the following block.