binman.rst 44 KB

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154
  1. .. SPDX-License-Identifier: GPL-2.0+
  2. .. Copyright (c) 2016 Google, Inc
  3. Introduction
  4. ============
  5. Firmware often consists of several components which must be packaged together.
  6. For example, we may have SPL, U-Boot, a device tree and an environment area
  7. grouped together and placed in MMC flash. When the system starts, it must be
  8. able to find these pieces.
  9. Building firmware should be separate from packaging it. Many of the complexities
  10. of modern firmware build systems come from trying to do both at once. With
  11. binman, you build all the pieces that are needed, using whatever assortment of
  12. projects and build systems are needed, then use binman to stitch everything
  13. together.
  14. What it does
  15. ------------
  16. Binman reads your board's device tree and finds a node which describes the
  17. required image layout. It uses this to work out what to place where.
  18. Binman provides a mechanism for building images, from simple SPL + U-Boot
  19. combinations, to more complex arrangements with many parts. It also allows
  20. users to inspect images, extract and replace binaries within them, repacking if
  21. needed.
  22. Features
  23. --------
  24. Apart from basic padding, alignment and positioning features, Binman supports
  25. hierarchical images, compression, hashing and dealing with the binary blobs
  26. which are a sad trend in open-source firmware at present.
  27. Executable binaries can access the location of other binaries in an image by
  28. using special linker symbols (zero-overhead but somewhat limited) or by reading
  29. the devicetree description of the image.
  30. Binman is designed primarily for use with U-Boot and associated binaries such
  31. as ARM Trusted Firmware, but it is suitable for use with other projects, such
  32. as Zephyr. Binman also provides facilities useful in Chromium OS, such as CBFS,
  33. vblocks and and the like.
  34. Binman provides a way to process binaries before they are included, by adding a
  35. Python plug-in.
  36. Binman is intended for use with U-Boot but is designed to be general enough
  37. to be useful in other image-packaging situations.
  38. Motivation
  39. ----------
  40. As mentioned above, packaging of firmware is quite a different task from
  41. building the various parts. In many cases the various binaries which go into
  42. the image come from separate build systems. For example, ARM Trusted Firmware
  43. is used on ARMv8 devices but is not built in the U-Boot tree. If a Linux kernel
  44. is included in the firmware image, it is built elsewhere.
  45. It is of course possible to add more and more build rules to the U-Boot
  46. build system to cover these cases. It can shell out to other Makefiles and
  47. build scripts. But it seems better to create a clear divide between building
  48. software and packaging it.
  49. At present this is handled by manual instructions, different for each board,
  50. on how to create images that will boot. By turning these instructions into a
  51. standard format, we can support making valid images for any board without
  52. manual effort, lots of READMEs, etc.
  53. Benefits:
  54. - Each binary can have its own build system and tool chain without creating
  55. any dependencies between them
  56. - Avoids the need for a single-shot build: individual parts can be updated
  57. and brought in as needed
  58. - Provides for a standard image description available in the build and at
  59. run-time
  60. - SoC-specific image-signing tools can be accommodated
  61. - Avoids cluttering the U-Boot build system with image-building code
  62. - The image description is automatically available at run-time in U-Boot,
  63. SPL. It can be made available to other software also
  64. - The image description is easily readable (it's a text file in device-tree
  65. format) and permits flexible packing of binaries
  66. Terminology
  67. -----------
  68. Binman uses the following terms:
  69. - image - an output file containing a firmware image
  70. - binary - an input binary that goes into the image
  71. Relationship to FIT
  72. -------------------
  73. FIT is U-Boot's official image format. It supports multiple binaries with
  74. load / execution addresses, compression. It also supports verification
  75. through hashing and RSA signatures.
  76. FIT was originally designed to support booting a Linux kernel (with an
  77. optional ramdisk) and device tree chosen from various options in the FIT.
  78. Now that U-Boot supports configuration via device tree, it is possible to
  79. load U-Boot from a FIT, with the device tree chosen by SPL.
  80. Binman considers FIT to be one of the binaries it can place in the image.
  81. Where possible it is best to put as much as possible in the FIT, with binman
  82. used to deal with cases not covered by FIT. Examples include initial
  83. execution (since FIT itself does not have an executable header) and dealing
  84. with device boundaries, such as the read-only/read-write separation in SPI
  85. flash.
  86. For U-Boot, binman should not be used to create ad-hoc images in place of
  87. FIT.
  88. Relationship to mkimage
  89. -----------------------
  90. The mkimage tool provides a means to create a FIT. Traditionally it has
  91. needed an image description file: a device tree, like binman, but in a
  92. different format. More recently it has started to support a '-f auto' mode
  93. which can generate that automatically.
  94. More relevant to binman, mkimage also permits creation of many SoC-specific
  95. image types. These can be listed by running 'mkimage -T list'. Examples
  96. include 'rksd', the Rockchip SD/MMC boot format. The mkimage tool is often
  97. called from the U-Boot build system for this reason.
  98. Binman considers the output files created by mkimage to be binary blobs
  99. which it can place in an image. Binman does not replace the mkimage tool or
  100. this purpose. It would be possible in some situations to create a new entry
  101. type for the images in mkimage, but this would not add functionality. It
  102. seems better to use the mkimage tool to generate binaries and avoid blurring
  103. the boundaries between building input files (mkimage) and packaging then
  104. into a final image (binman).
  105. Using binman
  106. ============
  107. Example use of binman in U-Boot
  108. -------------------------------
  109. Binman aims to replace some of the ad-hoc image creation in the U-Boot
  110. build system.
  111. Consider sunxi. It has the following steps:
  112. #. It uses a custom mksunxiboot tool to build an SPL image called
  113. sunxi-spl.bin. This should probably move into mkimage.
  114. #. It uses mkimage to package U-Boot into a legacy image file (so that it can
  115. hold the load and execution address) called u-boot.img.
  116. #. It builds a final output image called u-boot-sunxi-with-spl.bin which
  117. consists of sunxi-spl.bin, some padding and u-boot.img.
  118. Binman is intended to replace the last step. The U-Boot build system builds
  119. u-boot.bin and sunxi-spl.bin. Binman can then take over creation of
  120. sunxi-spl.bin (by calling mksunxiboot, or hopefully one day mkimage). In any
  121. case, it would then create the image from the component parts.
  122. This simplifies the U-Boot Makefile somewhat, since various pieces of logic
  123. can be replaced by a call to binman.
  124. Example use of binman for x86
  125. -----------------------------
  126. In most cases x86 images have a lot of binary blobs, 'black-box' code
  127. provided by Intel which must be run for the platform to work. Typically
  128. these blobs are not relocatable and must be placed at fixed areas in the
  129. firmware image.
  130. Currently this is handled by ifdtool, which places microcode, FSP, MRC, VGA
  131. BIOS, reference code and Intel ME binaries into a u-boot.rom file.
  132. Binman is intended to replace all of this, with ifdtool left to handle only
  133. the configuration of the Intel-format descriptor.
  134. Running binman
  135. --------------
  136. First install prerequisites, e.g::
  137. sudo apt-get install python-pyelftools python3-pyelftools lzma-alone \
  138. liblz4-tool
  139. Type::
  140. binman build -b <board_name>
  141. to build an image for a board. The board name is the same name used when
  142. configuring U-Boot (e.g. for sandbox_defconfig the board name is 'sandbox').
  143. Binman assumes that the input files for the build are in ../b/<board_name>.
  144. Or you can specify this explicitly::
  145. binman build -I <build_path>
  146. where <build_path> is the build directory containing the output of the U-Boot
  147. build.
  148. (Future work will make this more configurable)
  149. In either case, binman picks up the device tree file (u-boot.dtb) and looks
  150. for its instructions in the 'binman' node.
  151. Binman has a few other options which you can see by running 'binman -h'.
  152. Enabling binman for a board
  153. ---------------------------
  154. At present binman is invoked from a rule in the main Makefile. You should be
  155. able to enable CONFIG_BINMAN to enable this rule.
  156. The output file is typically named image.bin and is located in the output
  157. directory. If input files are needed to you add these to INPUTS-y either in the
  158. main Makefile or in a config.mk file in your arch subdirectory.
  159. Once binman is executed it will pick up its instructions from a device-tree
  160. file, typically <soc>-u-boot.dtsi, where <soc> is your CONFIG_SYS_SOC value.
  161. You can use other, more specific CONFIG options - see 'Automatic .dtsi
  162. inclusion' below.
  163. Access to binman entry offsets at run time (symbols)
  164. ----------------------------------------------------
  165. Binman assembles images and determines where each entry is placed in the image.
  166. This information may be useful to U-Boot at run time. For example, in SPL it
  167. is useful to be able to find the location of U-Boot so that it can be executed
  168. when SPL is finished.
  169. Binman allows you to declare symbols in the SPL image which are filled in
  170. with their correct values during the build. For example::
  171. binman_sym_declare(ulong, u_boot_any, image_pos);
  172. declares a ulong value which will be assigned to the image-pos of any U-Boot
  173. image (u-boot.bin, u-boot.img, u-boot-nodtb.bin) that is present in the image.
  174. You can access this value with something like::
  175. ulong u_boot_offset = binman_sym(ulong, u_boot_any, image_pos);
  176. Thus u_boot_offset will be set to the image-pos of U-Boot in memory, assuming
  177. that the whole image has been loaded, or is available in flash. You can then
  178. jump to that address to start U-Boot.
  179. At present this feature is only supported in SPL and TPL. In principle it is
  180. possible to fill in such symbols in U-Boot proper, as well, but a future C
  181. library is planned for this instead, to read from the device tree.
  182. As well as image-pos, it is possible to read the size of an entry and its
  183. offset (which is the start position of the entry within its parent).
  184. A small technical note: Binman automatically adds the base address of the image
  185. (i.e. __image_copy_start) to the value of the image-pos symbol, so that when the
  186. image is loaded to its linked address, the value will be correct and actually
  187. point into the image.
  188. For example, say SPL is at the start of the image and linked to start at address
  189. 80108000. If U-Boot's image-pos is 0x8000 then binman will write an image-pos
  190. for U-Boot of 80110000 into the SPL binary, since it assumes the image is loaded
  191. to 80108000, with SPL at 80108000 and U-Boot at 80110000.
  192. For x86 devices (with the end-at-4gb property) this base address is not added
  193. since it is assumed that images are XIP and the offsets already include the
  194. address.
  195. Access to binman entry offsets at run time (fdt)
  196. ------------------------------------------------
  197. Binman can update the U-Boot FDT to include the final position and size of
  198. each entry in the images it processes. The option to enable this is -u and it
  199. causes binman to make sure that the 'offset', 'image-pos' and 'size' properties
  200. are set correctly for every entry. Since it is not necessary to specify these in
  201. the image definition, binman calculates the final values and writes these to
  202. the device tree. These can be used by U-Boot at run-time to find the location
  203. of each entry.
  204. Alternatively, an FDT map entry can be used to add a special FDT containing
  205. just the information about the image. This is preceded by a magic string so can
  206. be located anywhere in the image. An image header (typically at the start or end
  207. of the image) can be used to point to the FDT map. See fdtmap and image-header
  208. entries for more information.
  209. Map files
  210. ---------
  211. The -m option causes binman to output a .map file for each image that it
  212. generates. This shows the offset and size of each entry. For example::
  213. Offset Size Name
  214. 00000000 00000028 main-section
  215. 00000000 00000010 section@0
  216. 00000000 00000004 u-boot
  217. 00000010 00000010 section@1
  218. 00000000 00000004 u-boot
  219. This shows a hierarchical image with two sections, each with a single entry. The
  220. offsets of the sections are absolute hex byte offsets within the image. The
  221. offsets of the entries are relative to their respective sections. The size of
  222. each entry is also shown, in bytes (hex). The indentation shows the entries
  223. nested inside their sections.
  224. Passing command-line arguments to entries
  225. -----------------------------------------
  226. Sometimes it is useful to pass binman the value of an entry property from the
  227. command line. For example some entries need access to files and it is not
  228. always convenient to put these filenames in the image definition (device tree).
  229. The-a option supports this::
  230. -a<prop>=<value>
  231. where::
  232. <prop> is the property to set
  233. <value> is the value to set it to
  234. Not all properties can be provided this way. Only some entries support it,
  235. typically for filenames.
  236. Image description format
  237. ========================
  238. The binman node is called 'binman'. An example image description is shown
  239. below::
  240. binman {
  241. filename = "u-boot-sunxi-with-spl.bin";
  242. pad-byte = <0xff>;
  243. blob {
  244. filename = "spl/sunxi-spl.bin";
  245. };
  246. u-boot {
  247. offset = <CONFIG_SPL_PAD_TO>;
  248. };
  249. };
  250. This requests binman to create an image file called u-boot-sunxi-with-spl.bin
  251. consisting of a specially formatted SPL (spl/sunxi-spl.bin, built by the
  252. normal U-Boot Makefile), some 0xff padding, and a U-Boot legacy image. The
  253. padding comes from the fact that the second binary is placed at
  254. CONFIG_SPL_PAD_TO. If that line were omitted then the U-Boot binary would
  255. immediately follow the SPL binary.
  256. The binman node describes an image. The sub-nodes describe entries in the
  257. image. Each entry represents a region within the overall image. The name of
  258. the entry (blob, u-boot) tells binman what to put there. For 'blob' we must
  259. provide a filename. For 'u-boot', binman knows that this means 'u-boot.bin'.
  260. Entries are normally placed into the image sequentially, one after the other.
  261. The image size is the total size of all entries. As you can see, you can
  262. specify the start offset of an entry using the 'offset' property.
  263. Note that due to a device tree requirement, all entries must have a unique
  264. name. If you want to put the same binary in the image multiple times, you can
  265. use any unique name, with the 'type' property providing the type.
  266. The attributes supported for entries are described below.
  267. offset:
  268. This sets the offset of an entry within the image or section containing
  269. it. The first byte of the image is normally at offset 0. If 'offset' is
  270. not provided, binman sets it to the end of the previous region, or the
  271. start of the image's entry area (normally 0) if there is no previous
  272. region.
  273. align:
  274. This sets the alignment of the entry. The entry offset is adjusted
  275. so that the entry starts on an aligned boundary within the containing
  276. section or image. For example 'align = <16>' means that the entry will
  277. start on a 16-byte boundary. This may mean that padding is added before
  278. the entry. The padding is part of the containing section but is not
  279. included in the entry, meaning that an empty space may be created before
  280. the entry starts. Alignment should be a power of 2. If 'align' is not
  281. provided, no alignment is performed.
  282. size:
  283. This sets the size of the entry. The contents will be padded out to
  284. this size. If this is not provided, it will be set to the size of the
  285. contents.
  286. pad-before:
  287. Padding before the contents of the entry. Normally this is 0, meaning
  288. that the contents start at the beginning of the entry. This can be used
  289. to offset the entry contents a little. While this does not affect the
  290. contents of the entry within binman itself (the padding is performed
  291. only when its parent section is assembled), the end result will be that
  292. the entry starts with the padding bytes, so may grow. Defaults to 0.
  293. pad-after:
  294. Padding after the contents of the entry. Normally this is 0, meaning
  295. that the entry ends at the last byte of content (unless adjusted by
  296. other properties). This allows room to be created in the image for
  297. this entry to expand later. While this does not affect the contents of
  298. the entry within binman itself (the padding is performed only when its
  299. parent section is assembled), the end result will be that the entry ends
  300. with the padding bytes, so may grow. Defaults to 0.
  301. align-size:
  302. This sets the alignment of the entry size. For example, to ensure
  303. that the size of an entry is a multiple of 64 bytes, set this to 64.
  304. While this does not affect the contents of the entry within binman
  305. itself (the padding is performed only when its parent section is
  306. assembled), the end result is that the entry ends with the padding
  307. bytes, so may grow. If 'align-size' is not provided, no alignment is
  308. performed.
  309. align-end:
  310. This sets the alignment of the end of an entry with respect to the
  311. containing section. Some entries require that they end on an alignment
  312. boundary, regardless of where they start. This does not move the start
  313. of the entry, so the contents of the entry will still start at the
  314. beginning. But there may be padding at the end. While this does not
  315. affect the contents of the entry within binman itself (the padding is
  316. performed only when its parent section is assembled), the end result
  317. is that the entry ends with the padding bytes, so may grow.
  318. If 'align-end' is not provided, no alignment is performed.
  319. filename:
  320. For 'blob' types this provides the filename containing the binary to
  321. put into the entry. If binman knows about the entry type (like
  322. u-boot-bin), then there is no need to specify this.
  323. type:
  324. Sets the type of an entry. This defaults to the entry name, but it is
  325. possible to use any name, and then add (for example) 'type = "u-boot"'
  326. to specify the type.
  327. offset-unset:
  328. Indicates that the offset of this entry should not be set by placing
  329. it immediately after the entry before. Instead, is set by another
  330. entry which knows where this entry should go. When this boolean
  331. property is present, binman will give an error if another entry does
  332. not set the offset (with the GetOffsets() method).
  333. image-pos:
  334. This cannot be set on entry (or at least it is ignored if it is), but
  335. with the -u option, binman will set it to the absolute image position
  336. for each entry. This makes it easy to find out exactly where the entry
  337. ended up in the image, regardless of parent sections, etc.
  338. expand-size:
  339. Expand the size of this entry to fit available space. This space is only
  340. limited by the size of the image/section and the position of the next
  341. entry.
  342. compress:
  343. Sets the compression algortihm to use (for blobs only). See the entry
  344. documentation for details.
  345. missing-msg:
  346. Sets the tag of the message to show if this entry is missing. This is
  347. used for external blobs. When they are missing it is helpful to show
  348. information about what needs to be fixed. See missing-blob-help for the
  349. message for each tag.
  350. The attributes supported for images and sections are described below. Several
  351. are similar to those for entries.
  352. size:
  353. Sets the image size in bytes, for example 'size = <0x100000>' for a
  354. 1MB image.
  355. offset:
  356. This is similar to 'offset' in entries, setting the offset of a section
  357. within the image or section containing it. The first byte of the section
  358. is normally at offset 0. If 'offset' is not provided, binman sets it to
  359. the end of the previous region, or the start of the image's entry area
  360. (normally 0) if there is no previous region.
  361. align-size:
  362. This sets the alignment of the image size. For example, to ensure
  363. that the image ends on a 512-byte boundary, use 'align-size = <512>'.
  364. If 'align-size' is not provided, no alignment is performed.
  365. pad-before:
  366. This sets the padding before the image entries. The first entry will
  367. be positioned after the padding. This defaults to 0.
  368. pad-after:
  369. This sets the padding after the image entries. The padding will be
  370. placed after the last entry. This defaults to 0.
  371. pad-byte:
  372. This specifies the pad byte to use when padding in the image. It
  373. defaults to 0. To use 0xff, you would add 'pad-byte = <0xff>'.
  374. filename:
  375. This specifies the image filename. It defaults to 'image.bin'.
  376. sort-by-offset:
  377. This causes binman to reorder the entries as needed to make sure they
  378. are in increasing positional order. This can be used when your entry
  379. order may not match the positional order. A common situation is where
  380. the 'offset' properties are set by CONFIG options, so their ordering is
  381. not known a priori.
  382. This is a boolean property so needs no value. To enable it, add a
  383. line 'sort-by-offset;' to your description.
  384. multiple-images:
  385. Normally only a single image is generated. To create more than one
  386. image, put this property in the binman node. For example, this will
  387. create image1.bin containing u-boot.bin, and image2.bin containing
  388. both spl/u-boot-spl.bin and u-boot.bin::
  389. binman {
  390. multiple-images;
  391. image1 {
  392. u-boot {
  393. };
  394. };
  395. image2 {
  396. spl {
  397. };
  398. u-boot {
  399. };
  400. };
  401. };
  402. end-at-4gb:
  403. For x86 machines the ROM offsets start just before 4GB and extend
  404. up so that the image finished at the 4GB boundary. This boolean
  405. option can be enabled to support this. The image size must be
  406. provided so that binman knows when the image should start. For an
  407. 8MB ROM, the offset of the first entry would be 0xfff80000 with
  408. this option, instead of 0 without this option.
  409. skip-at-start:
  410. This property specifies the entry offset of the first entry.
  411. For PowerPC mpc85xx based CPU, CONFIG_SYS_TEXT_BASE is the entry
  412. offset of the first entry. It can be 0xeff40000 or 0xfff40000 for
  413. nor flash boot, 0x201000 for sd boot etc.
  414. 'end-at-4gb' property is not applicable where CONFIG_SYS_TEXT_BASE +
  415. Image size != 4gb.
  416. Examples of the above options can be found in the tests. See the
  417. tools/binman/test directory.
  418. It is possible to have the same binary appear multiple times in the image,
  419. either by using a unit number suffix (u-boot@0, u-boot@1) or by using a
  420. different name for each and specifying the type with the 'type' attribute.
  421. Sections and hierachical images
  422. -------------------------------
  423. Sometimes it is convenient to split an image into several pieces, each of which
  424. contains its own set of binaries. An example is a flash device where part of
  425. the image is read-only and part is read-write. We can set up sections for each
  426. of these, and place binaries in them independently. The image is still produced
  427. as a single output file.
  428. This feature provides a way of creating hierarchical images. For example here
  429. is an example image with two copies of U-Boot. One is read-only (ro), intended
  430. to be written only in the factory. Another is read-write (rw), so that it can be
  431. upgraded in the field. The sizes are fixed so that the ro/rw boundary is known
  432. and can be programmed::
  433. binman {
  434. section@0 {
  435. read-only;
  436. name-prefix = "ro-";
  437. size = <0x100000>;
  438. u-boot {
  439. };
  440. };
  441. section@1 {
  442. name-prefix = "rw-";
  443. size = <0x100000>;
  444. u-boot {
  445. };
  446. };
  447. };
  448. This image could be placed into a SPI flash chip, with the protection boundary
  449. set at 1MB.
  450. A few special properties are provided for sections:
  451. read-only:
  452. Indicates that this section is read-only. This has no impact on binman's
  453. operation, but his property can be read at run time.
  454. name-prefix:
  455. This string is prepended to all the names of the binaries in the
  456. section. In the example above, the 'u-boot' binaries which actually be
  457. renamed to 'ro-u-boot' and 'rw-u-boot'. This can be useful to
  458. distinguish binaries with otherwise identical names.
  459. Image Properties
  460. ----------------
  461. Image nodes act like sections but also have a few extra properties:
  462. filename:
  463. Output filename for the image. This defaults to image.bin (or in the
  464. case of multiple images <nodename>.bin where <nodename> is the name of
  465. the image node.
  466. allow-repack:
  467. Create an image that can be repacked. With this option it is possible
  468. to change anything in the image after it is created, including updating
  469. the position and size of image components. By default this is not
  470. permitted since it is not possibly to know whether this might violate a
  471. constraint in the image description. For example, if a section has to
  472. increase in size to hold a larger binary, that might cause the section
  473. to fall out of its allow region (e.g. read-only portion of flash).
  474. Adding this property causes the original offset and size values in the
  475. image description to be stored in the FDT and fdtmap.
  476. Hashing Entries
  477. ---------------
  478. It is possible to ask binman to hash the contents of an entry and write that
  479. value back to the device-tree node. For example::
  480. binman {
  481. u-boot {
  482. hash {
  483. algo = "sha256";
  484. };
  485. };
  486. };
  487. Here, a new 'value' property will be written to the 'hash' node containing
  488. the hash of the 'u-boot' entry. Only SHA256 is supported at present. Whole
  489. sections can be hased if desired, by adding the 'hash' node to the section.
  490. The has value can be chcked at runtime by hashing the data actually read and
  491. comparing this has to the value in the device tree.
  492. Expanded entries
  493. ----------------
  494. Binman automatically replaces 'u-boot' with an expanded version of that, i.e.
  495. 'u-boot-expanded'. This means that when you write::
  496. u-boot {
  497. };
  498. you actually get::
  499. u-boot {
  500. type = "u-boot-expanded';
  501. };
  502. which in turn expands to::
  503. u-boot {
  504. type = "section";
  505. u-boot-nodtb {
  506. };
  507. u-boot-dtb {
  508. };
  509. };
  510. U-Boot's various phase binaries actually comprise two or three pieces.
  511. For example, u-boot.bin has the executable followed by a devicetree.
  512. With binman we want to be able to update that devicetree with full image
  513. information so that it is accessible to the executable. This is tricky
  514. if it is not clear where the devicetree starts.
  515. The above feature ensures that the devicetree is clearly separated from the
  516. U-Boot executable and can be updated separately by binman as needed. It can be
  517. disabled with the --no-expanded flag if required.
  518. The same applies for u-boot-spl and u-boot-spl. In those cases, the expansion
  519. includes the BSS padding, so for example::
  520. spl {
  521. type = "u-boot-spl"
  522. };
  523. you actually get::
  524. spl {
  525. type = "u-boot-expanded';
  526. };
  527. which in turn expands to::
  528. spl {
  529. type = "section";
  530. u-boot-spl-nodtb {
  531. };
  532. u-boot-spl-bss-pad {
  533. };
  534. u-boot-spl-dtb {
  535. };
  536. };
  537. Of course we should not expand SPL if it has no devicetree. Also if the BSS
  538. padding is not needed (because BSS is in RAM as with CONFIG_SPL_SEPARATE_BSS),
  539. the 'u-boot-spl-bss-pad' subnode should not be created. The use of the expaned
  540. entry type is controlled by the UseExpanded() method. In the SPL case it checks
  541. the 'spl-dtb' entry arg, which is 'y' or '1' if SPL has a devicetree.
  542. For the BSS case, a 'spl-bss-pad' entry arg controls whether it is present. All
  543. entry args are provided by the U-Boot Makefile.
  544. Compression
  545. -----------
  546. Binman support compression for 'blob' entries (those of type 'blob' and
  547. derivatives). To enable this for an entry, add a 'compress' property::
  548. blob {
  549. filename = "datafile";
  550. compress = "lz4";
  551. };
  552. The entry will then contain the compressed data, using the 'lz4' compression
  553. algorithm. Currently this is the only one that is supported. The uncompressed
  554. size is written to the node in an 'uncomp-size' property, if -u is used.
  555. Compression is also supported for sections. In that case the entire section is
  556. compressed in one block, including all its contents. This means that accessing
  557. an entry from the section required decompressing the entire section. Also, the
  558. size of a section indicates the space that it consumes in its parent section
  559. (and typically the image). With compression, the section may contain more data,
  560. and the uncomp-size property indicates that, as above. The contents of the
  561. section is compressed first, before any padding is added. This ensures that the
  562. padding itself is not compressed, which would be a waste of time.
  563. Automatic .dtsi inclusion
  564. -------------------------
  565. It is sometimes inconvenient to add a 'binman' node to the .dts file for each
  566. board. This can be done by using #include to bring in a common file. Another
  567. approach supported by the U-Boot build system is to automatically include
  568. a common header. You can then put the binman node (and anything else that is
  569. specific to U-Boot, such as u-boot,dm-pre-reloc properies) in that header
  570. file.
  571. Binman will search for the following files in arch/<arch>/dts::
  572. <dts>-u-boot.dtsi where <dts> is the base name of the .dts file
  573. <CONFIG_SYS_SOC>-u-boot.dtsi
  574. <CONFIG_SYS_CPU>-u-boot.dtsi
  575. <CONFIG_SYS_VENDOR>-u-boot.dtsi
  576. u-boot.dtsi
  577. U-Boot will only use the first one that it finds. If you need to include a
  578. more general file you can do that from the more specific file using #include.
  579. If you are having trouble figuring out what is going on, you can uncomment
  580. the 'warning' line in scripts/Makefile.lib to see what it has found::
  581. # Uncomment for debugging
  582. # This shows all the files that were considered and the one that we chose.
  583. # u_boot_dtsi_options_debug = $(u_boot_dtsi_options_raw)
  584. Entry Documentation
  585. ===================
  586. For details on the various entry types supported by binman and how to use them,
  587. see entries.rst which is generated from the source code using:
  588. binman entry-docs >tools/binman/entries.rst
  589. .. toctree::
  590. :maxdepth: 2
  591. entries
  592. Managing images
  593. ===============
  594. Listing images
  595. --------------
  596. It is possible to list the entries in an existing firmware image created by
  597. binman, provided that there is an 'fdtmap' entry in the image. For example::
  598. $ binman ls -i image.bin
  599. Name Image-pos Size Entry-type Offset Uncomp-size
  600. ----------------------------------------------------------------------
  601. main-section c00 section 0
  602. u-boot 0 4 u-boot 0
  603. section 5fc section 4
  604. cbfs 100 400 cbfs 0
  605. u-boot 138 4 u-boot 38
  606. u-boot-dtb 180 108 u-boot-dtb 80 3b5
  607. u-boot-dtb 500 1ff u-boot-dtb 400 3b5
  608. fdtmap 6fc 381 fdtmap 6fc
  609. image-header bf8 8 image-header bf8
  610. This shows the hierarchy of the image, the position, size and type of each
  611. entry, the offset of each entry within its parent and the uncompressed size if
  612. the entry is compressed.
  613. It is also possible to list just some files in an image, e.g.::
  614. $ binman ls -i image.bin section/cbfs
  615. Name Image-pos Size Entry-type Offset Uncomp-size
  616. --------------------------------------------------------------------
  617. cbfs 100 400 cbfs 0
  618. u-boot 138 4 u-boot 38
  619. u-boot-dtb 180 108 u-boot-dtb 80 3b5
  620. or with wildcards::
  621. $ binman ls -i image.bin "*cb*" "*head*"
  622. Name Image-pos Size Entry-type Offset Uncomp-size
  623. ----------------------------------------------------------------------
  624. cbfs 100 400 cbfs 0
  625. u-boot 138 4 u-boot 38
  626. u-boot-dtb 180 108 u-boot-dtb 80 3b5
  627. image-header bf8 8 image-header bf8
  628. Extracting files from images
  629. ----------------------------
  630. You can extract files from an existing firmware image created by binman,
  631. provided that there is an 'fdtmap' entry in the image. For example::
  632. $ binman extract -i image.bin section/cbfs/u-boot
  633. which will write the uncompressed contents of that entry to the file 'u-boot' in
  634. the current directory. You can also extract to a particular file, in this case
  635. u-boot.bin::
  636. $ binman extract -i image.bin section/cbfs/u-boot -f u-boot.bin
  637. It is possible to extract all files into a destination directory, which will
  638. put files in subdirectories matching the entry hierarchy::
  639. $ binman extract -i image.bin -O outdir
  640. or just a selection::
  641. $ binman extract -i image.bin "*u-boot*" -O outdir
  642. Replacing files in an image
  643. ---------------------------
  644. You can replace files in an existing firmware image created by binman, provided
  645. that there is an 'fdtmap' entry in the image. For example:
  646. $ binman replace -i image.bin section/cbfs/u-boot
  647. which will write the contents of the file 'u-boot' from the current directory
  648. to the that entry, compressing if necessary. If the entry size changes, you must
  649. add the 'allow-repack' property to the original image before generating it (see
  650. above), otherwise you will get an error.
  651. You can also use a particular file, in this case u-boot.bin::
  652. $ binman replace -i image.bin section/cbfs/u-boot -f u-boot.bin
  653. It is possible to replace all files from a source directory which uses the same
  654. hierarchy as the entries::
  655. $ binman replace -i image.bin -I indir
  656. Files that are missing will generate a warning.
  657. You can also replace just a selection of entries::
  658. $ binman replace -i image.bin "*u-boot*" -I indir
  659. Logging
  660. -------
  661. Binman normally operates silently unless there is an error, in which case it
  662. just displays the error. The -D/--debug option can be used to create a full
  663. backtrace when errors occur. You can use BINMAN_DEBUG=1 when building to select
  664. this.
  665. Internally binman logs some output while it is running. This can be displayed
  666. by increasing the -v/--verbosity from the default of 1:
  667. 0: silent
  668. 1: warnings only
  669. 2: notices (important messages)
  670. 3: info about major operations
  671. 4: detailed information about each operation
  672. 5: debug (all output)
  673. You can use BINMAN_VERBOSE=5 (for example) when building to select this.
  674. Technical details
  675. =================
  676. Order of image creation
  677. -----------------------
  678. Image creation proceeds in the following order, for each entry in the image.
  679. 1. AddMissingProperties() - binman can add calculated values to the device
  680. tree as part of its processing, for example the offset and size of each
  681. entry. This method adds any properties associated with this, expanding the
  682. device tree as needed. These properties can have placeholder values which are
  683. set later by SetCalculatedProperties(). By that stage the size of sections
  684. cannot be changed (since it would cause the images to need to be repacked),
  685. but the correct values can be inserted.
  686. 2. ProcessFdt() - process the device tree information as required by the
  687. particular entry. This may involve adding or deleting properties. If the
  688. processing is complete, this method should return True. If the processing
  689. cannot complete because it needs the ProcessFdt() method of another entry to
  690. run first, this method should return False, in which case it will be called
  691. again later.
  692. 3. GetEntryContents() - the contents of each entry are obtained, normally by
  693. reading from a file. This calls the Entry.ObtainContents() to read the
  694. contents. The default version of Entry.ObtainContents() calls
  695. Entry.GetDefaultFilename() and then reads that file. So a common mechanism
  696. to select a file to read is to override that function in the subclass. The
  697. functions must return True when they have read the contents. Binman will
  698. retry calling the functions a few times if False is returned, allowing
  699. dependencies between the contents of different entries.
  700. 4. GetEntryOffsets() - calls Entry.GetOffsets() for each entry. This can
  701. return a dict containing entries that need updating. The key should be the
  702. entry name and the value is a tuple (offset, size). This allows an entry to
  703. provide the offset and size for other entries. The default implementation
  704. of GetEntryOffsets() returns {}.
  705. 5. PackEntries() - calls Entry.Pack() which figures out the offset and
  706. size of an entry. The 'current' image offset is passed in, and the function
  707. returns the offset immediately after the entry being packed. The default
  708. implementation of Pack() is usually sufficient.
  709. Note: for sections, this also checks that the entries do not overlap, nor extend
  710. outside the section. If the section does not have a defined size, the size is
  711. set large enough to hold all the entries.
  712. 6. SetImagePos() - sets the image position of every entry. This is the absolute
  713. position 'image-pos', as opposed to 'offset' which is relative to the containing
  714. section. This must be done after all offsets are known, which is why it is quite
  715. late in the ordering.
  716. 7. SetCalculatedProperties() - update any calculated properties in the device
  717. tree. This sets the correct 'offset' and 'size' vaues, for example.
  718. 8. ProcessEntryContents() - this calls Entry.ProcessContents() on each entry.
  719. The default implementatoin does nothing. This can be overriden to adjust the
  720. contents of an entry in some way. For example, it would be possible to create
  721. an entry containing a hash of the contents of some other entries. At this
  722. stage the offset and size of entries should not be adjusted unless absolutely
  723. necessary, since it requires a repack (going back to PackEntries()).
  724. 9. ResetForPack() - if the ProcessEntryContents() step failed, in that an entry
  725. has changed its size, then there is no alternative but to go back to step 5 and
  726. try again, repacking the entries with the updated size. ResetForPack() removes
  727. the fixed offset/size values added by binman, so that the packing can start from
  728. scratch.
  729. 10. WriteSymbols() - write the value of symbols into the U-Boot SPL binary.
  730. See 'Access to binman entry offsets at run time' below for a description of
  731. what happens in this stage.
  732. 11. BuildImage() - builds the image and writes it to a file
  733. 12. WriteMap() - writes a text file containing a map of the image. This is the
  734. final step.
  735. External tools
  736. --------------
  737. Binman can make use of external command-line tools to handle processing of
  738. entry contents or to generate entry contents. These tools are executed using
  739. the 'tools' module's Run() method. The tools generally must exist on the PATH,
  740. but the --toolpath option can be used to specify additional search paths to
  741. use. This option can be specified multiple times to add more than one path.
  742. For some compile tools binman will use the versions specified by commonly-used
  743. environment variables like CC and HOSTCC for the C compiler, based on whether
  744. the tool's output will be used for the target or for the host machine. If those
  745. aren't given, it will also try to derive target-specific versions from the
  746. CROSS_COMPILE environment variable during a cross-compilation.
  747. Code coverage
  748. -------------
  749. Binman is a critical tool and is designed to be very testable. Entry
  750. implementations target 100% test coverage. Run 'binman test -T' to check this.
  751. To enable Python test coverage on Debian-type distributions (e.g. Ubuntu)::
  752. $ sudo apt-get install python-coverage python3-coverage python-pytest
  753. Concurrent tests
  754. ----------------
  755. Binman tries to run tests concurrently. This means that the tests make use of
  756. all available CPUs to run.
  757. To enable this::
  758. $ sudo apt-get install python-subunit python3-subunit
  759. Use '-P 1' to disable this. It is automatically disabled when code coverage is
  760. being used (-T) since they are incompatible.
  761. Debugging tests
  762. ---------------
  763. Sometimes when debugging tests it is useful to keep the input and output
  764. directories so they can be examined later. Use -X or --test-preserve-dirs for
  765. this.
  766. Running tests on non-x86 architectures
  767. --------------------------------------
  768. Binman's tests have been written under the assumption that they'll be run on a
  769. x86-like host and there hasn't been an attempt to make them portable yet.
  770. However, it's possible to run the tests by cross-compiling to x86.
  771. To install an x86 cross-compiler on Debian-type distributions (e.g. Ubuntu)::
  772. $ sudo apt-get install gcc-x86-64-linux-gnu
  773. Then, you can run the tests under cross-compilation::
  774. $ CROSS_COMPILE=x86_64-linux-gnu- binman test -T
  775. You can also use gcc-i686-linux-gnu similar to the above.
  776. Writing new entries and debugging
  777. ---------------------------------
  778. The behaviour of entries is defined by the Entry class. All other entries are
  779. a subclass of this. An important subclass is Entry_blob which takes binary
  780. data from a file and places it in the entry. In fact most entry types are
  781. subclasses of Entry_blob.
  782. Each entry type is a separate file in the tools/binman/etype directory. Each
  783. file contains a class called Entry_<type> where <type> is the entry type.
  784. New entry types can be supported by adding new files in that directory.
  785. These will automatically be detected by binman when needed.
  786. Entry properties are documented in entry.py. The entry subclasses are free
  787. to change the values of properties to support special behaviour. For example,
  788. when Entry_blob loads a file, it sets content_size to the size of the file.
  789. Entry classes can adjust other entries. For example, an entry that knows
  790. where other entries should be positioned can set up those entries' offsets
  791. so they don't need to be set in the binman decription. It can also adjust
  792. entry contents.
  793. Most of the time such essoteric behaviour is not needed, but it can be
  794. essential for complex images.
  795. If you need to specify a particular device-tree compiler to use, you can define
  796. the DTC environment variable. This can be useful when the system dtc is too
  797. old.
  798. To enable a full backtrace and other debugging features in binman, pass
  799. BINMAN_DEBUG=1 to your build::
  800. make qemu-x86_defconfig
  801. make BINMAN_DEBUG=1
  802. To enable verbose logging from binman, base BINMAN_VERBOSE to your build, which
  803. adds a -v<level> option to the call to binman::
  804. make qemu-x86_defconfig
  805. make BINMAN_VERBOSE=5
  806. History / Credits
  807. -----------------
  808. Binman takes a lot of inspiration from a Chrome OS tool called
  809. 'cros_bundle_firmware', which I wrote some years ago. That tool was based on
  810. a reasonably simple and sound design but has expanded greatly over the
  811. years. In particular its handling of x86 images is convoluted.
  812. Quite a few lessons have been learned which are hopefully applied here.
  813. Design notes
  814. ------------
  815. On the face of it, a tool to create firmware images should be fairly simple:
  816. just find all the input binaries and place them at the right place in the
  817. image. The difficulty comes from the wide variety of input types (simple
  818. flat binaries containing code, packaged data with various headers), packing
  819. requirments (alignment, spacing, device boundaries) and other required
  820. features such as hierarchical images.
  821. The design challenge is to make it easy to create simple images, while
  822. allowing the more complex cases to be supported. For example, for most
  823. images we don't much care exactly where each binary ends up, so we should
  824. not have to specify that unnecessarily.
  825. New entry types should aim to provide simple usage where possible. If new
  826. core features are needed, they can be added in the Entry base class.
  827. To do
  828. -----
  829. Some ideas:
  830. - Use of-platdata to make the information available to code that is unable
  831. to use device tree (such as a very small SPL image). For now, limited info is
  832. available via linker symbols
  833. - Allow easy building of images by specifying just the board name
  834. - Support building an image for a board (-b) more completely, with a
  835. configurable build directory
  836. - Detect invalid properties in nodes
  837. - Sort the fdtmap by offset
  838. - Output temporary files to a different directory
  839. --
  840. Simon Glass <sjg@chromium.org>
  841. 7/7/2016