README.rst 38 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841
  1. .. contents:: Table of contents
  2. :backlinks: none
  3. News
  4. ----
  5. Dependency loop with recent linux-next kernels
  6. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  7. To fix issues with dependency loops on recent linux-next kernels, apply `this
  8. patch <https://www.spinics.net/lists/linux-kbuild/msg23455.html>`_. Hopefully,
  9. it will be in ``linux-next`` soon.
  10. ``windows-curses`` is no longer automatically installed on Windows
  11. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  12. Starting with Kconfiglib 13.0.0, the `windows-curses
  13. <https://github.com/zephyrproject-rtos/windows-curses>`__ package is no longer
  14. automatically installed on Windows, and needs to be installed manually for the
  15. terminal ``menuconfig`` to work.
  16. This fixes installation of Kconfiglib on MSYS2, which is not compatible with
  17. ``windows-curses``. See `this issue
  18. <https://github.com/ulfalizer/Kconfiglib/issues/77>`__.
  19. The ``menuconfig`` now shows a hint re. installing ``windows-curses`` when the
  20. ``curses`` module can't be imported on Windows.
  21. Sorry if this change caused problems!
  22. Overview
  23. --------
  24. Kconfiglib is a `Kconfig
  25. <https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-language.rst>`__
  26. implementation in Python 2/3. It started out as a helper library, but now has a
  27. enough functionality to also work well as a standalone Kconfig implementation
  28. (including `terminal and GUI menuconfig interfaces <Menuconfig interfaces_>`_
  29. and `Kconfig extensions`_).
  30. The entire library is contained in `kconfiglib.py
  31. <https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_. The
  32. bundled scripts are implemented on top of it. Implementing your own scripts
  33. should be relatively easy, if needed.
  34. Kconfiglib is used exclusively by e.g. the `Zephyr
  35. <https://www.zephyrproject.org/>`__, `esp-idf
  36. <https://github.com/espressif/esp-idf>`__, and `ACRN
  37. <https://projectacrn.org/>`__ projects. It is also used for many small helper
  38. scripts in various projects.
  39. Since Kconfiglib is based around a library, it can be used e.g. to generate a
  40. `Kconfig cross-reference
  41. <https://docs.zephyrproject.org/latest/reference/kconfig/index.html>`_, using
  42. the same robust Kconfig parser used for other Kconfig tools, instead of brittle
  43. ad-hoc parsing. The documentation generation script can be found `here
  44. <https://github.com/zephyrproject-rtos/zephyr/blob/master/doc/scripts/genrest.py>`__.
  45. Kconfiglib implements the recently added `Kconfig preprocessor
  46. <https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-macro-language.rst>`__.
  47. For backwards compatibility, environment variables can be referenced both as
  48. ``$(FOO)`` (the new syntax) and as ``$FOO`` (the old syntax). The old syntax is
  49. deprecated, but will probably be supported for a long time, as it's needed to
  50. stay compatible with older Linux kernels. The major version will be increased
  51. if support is ever dropped. Using the old syntax with an undefined environment
  52. variable keeps the string as is.
  53. Note: See `this issue <https://github.com/ulfalizer/Kconfiglib/issues/47>`__ if
  54. you run into a "macro expanded to blank string" error with kernel 4.18+.
  55. See `this page
  56. <https://docs.zephyrproject.org/latest/guides/kconfig/tips.html>`__ for some
  57. Kconfig tips and best practices.
  58. Installation
  59. ------------
  60. Installation with pip
  61. ~~~~~~~~~~~~~~~~~~~~~
  62. Kconfiglib is available on `PyPI <https://pypi.python.org/pypi/kconfiglib/>`_ and can be
  63. installed with e.g.
  64. .. code::
  65. $ pip(3) install kconfiglib
  66. Microsoft Windows is supported.
  67. The ``pip`` installation will give you both the base library and the following
  68. executables. All but two (``genconfig`` and ``setconfig``) mirror functionality
  69. available in the C tools.
  70. - `menuconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/menuconfig.py>`_
  71. - `guiconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/guiconfig.py>`_
  72. - `oldconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/oldconfig.py>`_
  73. - `olddefconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/olddefconfig.py>`_
  74. - `savedefconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/savedefconfig.py>`_
  75. - `defconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/defconfig.py>`_
  76. - `alldefconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/alldefconfig.py>`_
  77. - `allnoconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/allnoconfig.py>`_
  78. - `allmodconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/allmodconfig.py>`_
  79. - `allyesconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/allyesconfig.py>`_
  80. - `listnewconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/listnewconfig.py>`_
  81. - `genconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/genconfig.py>`_
  82. - `setconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/setconfig.py>`_
  83. ``genconfig`` is intended to be run at build time. It generates a C header from
  84. the configuration and (optionally) information that can be used to rebuild only
  85. files that reference Kconfig symbols that have changed value.
  86. Starting with Kconfiglib version 12.2.0, all utilities are compatible with both
  87. Python 2 and Python 3. Previously, ``menuconfig.py`` only ran under Python 3
  88. (i.e., it's now more backwards compatible than before).
  89. **Note:** If you install Kconfiglib with ``pip``'s ``--user`` flag, make sure
  90. that your ``PATH`` includes the directory where the executables end up. You can
  91. list the installed files with ``pip(3) show -f kconfiglib``.
  92. All releases have a corresponding tag in the git repository, e.g. ``v14.1.0``
  93. (the latest version).
  94. `Semantic versioning <http://semver.org/>`_ is used. There's been ten small
  95. changes to the behavior of the API, a Windows packaging change, and a hashbang
  96. change to use ``python3``
  97. (`1 <https://github.com/ulfalizer/Kconfiglib/commit/e8b4ecb6ff6ccc1c7be0818314fbccda2ef2b2ee>`_,
  98. `2 <https://github.com/ulfalizer/Kconfiglib/commit/db633015a4d7b0ba1e882f665e191f350932b2af>`_,
  99. `3 <https://github.com/ulfalizer/Kconfiglib/commit/8983f7eb297dd614faf0beee3129559bc8ba338e>`_,
  100. `4 <https://github.com/ulfalizer/Kconfiglib/commit/cbf32e29a130d22bc734b7778e6304ac9df2a3e8>`_,
  101. `5 <https://github.com/ulfalizer/Kconfiglib/commit/eb6c21a9b33a2d6e2bed9882d4f930d0cab2f03b>`_,
  102. `6 <https://github.com/ulfalizer/Kconfiglib/commit/c19fc11355b13d75d97286402c7a933fb23d3b70>`_,
  103. `7 <https://github.com/ulfalizer/Kconfiglib/commit/7a428aa415606820a44291f475248b08e3952c4b>`_,
  104. `8 <https://github.com/ulfalizer/Kconfiglib/commit/f247ddf618ad29718e5efd3e69f8baf75d4d347b>`_,
  105. `9 <https://github.com/ulfalizer/Kconfiglib/commit/4fed39d9271ceb68be4157ab3f96a45b94f77dc0>`_,
  106. `10 <https://github.com/ulfalizer/Kconfiglib/commit/55bc8c380869ea663092212e8fe388ad7abae596>`_,
  107. `Windows packaging change <https://github.com/ulfalizer/Kconfiglib/commit/21b4c1e3b6e2867b9a0788d21a358f6b1f581d86>`_,
  108. `Python 3 hashbang change <https://github.com/ulfalizer/Kconfiglib/commit/9e0a8d29fa76adcb3f27bb2e20f16fefc2a8591e>`_),
  109. which is why the major version is at 14 rather than 2. I do major version bumps
  110. for all behavior changes, even tiny ones, and most of these were fixes for baby
  111. issues in the early days of the Kconfiglib 2 API.
  112. Manual installation
  113. ~~~~~~~~~~~~~~~~~~~
  114. Just drop ``kconfiglib.py`` and the scripts you want somewhere. There are no
  115. third-party dependencies, but the terminal ``menuconfig`` won't work on Windows
  116. unless a package like `windows-curses
  117. <https://github.com/zephyrproject-rtos/windows-curses>`__ is installed.
  118. Installation for the Linux kernel
  119. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  120. See the module docstring at the top of `kconfiglib.py <https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_.
  121. Python version compatibility (2.7/3.2+)
  122. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  123. Kconfiglib and all utilities run under both Python 2.7 and Python 3.2 and
  124. later. The code mostly uses basic Python features and has no third-party
  125. dependencies, so keeping it backwards-compatible is pretty low effort.
  126. The 3.2 requirement comes from ``argparse``. ``format()`` with unnumbered
  127. ``{}`` is used as well.
  128. A recent Python 3 version is recommended if you have a choice, as it'll give
  129. you better Unicode handling.
  130. Getting started
  131. ---------------
  132. 1. `Install <Installation_>`_ the library and the utilities.
  133. 2. Write `Kconfig
  134. <https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-language.rst>`__
  135. files that describe the available configuration options. See `this page
  136. <https://docs.zephyrproject.org/latest/guides/kconfig/tips.html>`__ for some
  137. general Kconfig advice.
  138. 3. Generate an initial configuration with e.g. ``menuconfig``/``guiconfig`` or
  139. ``alldefconfig``. The configuration is saved as ``.config`` by default.
  140. For more advanced projects, the ``defconfig`` utility can be used to
  141. generate the initial configuration from an existing configuration file.
  142. Usually, this existing configuration file would be a minimal configuration
  143. file, as generated by e.g. ``savedefconfig``.
  144. 4. Run ``genconfig`` to generate a header file. By default, it is saved as
  145. ``config.h``.
  146. Normally, ``genconfig`` would be run automatically as part of the build.
  147. Before writing a header file or other configuration output, Kconfiglib
  148. compares the old contents of the file against the new contents. If there's
  149. no change, the write is skipped. This avoids updating file metadata like the
  150. modification time, and might save work depending on your build setup.
  151. Adding new configuration output formats should be relatively straightforward.
  152. See the implementation of ``write_config()`` in `kconfiglib.py
  153. <https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_.
  154. The documentation for the ``Symbol.config_string`` property has some tips as
  155. well.
  156. 5. To update an old ``.config`` file after the Kconfig files have changed (e.g.
  157. to add new options), run ``oldconfig`` (prompts for values for new options)
  158. or ``olddefconfig`` (gives new options their default value). Entering the
  159. ``menuconfig`` or ``guiconfig`` interface and saving the configuration will
  160. also update it (the configuration interfaces always prompt for saving
  161. on exit if it would modify the contents of the ``.config`` file).
  162. Due to Kconfig semantics, simply loading an old ``.config`` file performs an
  163. implicit ``olddefconfig``, so building will normally not be affected by
  164. having an outdated configuration.
  165. Whenever ``.config`` is overwritten, the previous version of the file is saved
  166. to ``.config.old`` (or, more generally, to ``$KCONFIG_CONFIG.old``).
  167. Using ``.config`` files as Make input
  168. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  169. ``.config`` files use Make syntax and can be included directly in Makefiles to
  170. read configuration values from there. This is why ``n``-valued
  171. ``bool``/``tristate`` values are written out as ``# CONFIG_FOO is not set`` (a
  172. Make comment) in ``.config``, allowing them to be tested with ``ifdef`` in
  173. Make.
  174. If you make use of this, you might want to pass ``--config-out <filename>`` to
  175. ``genconfig`` and include the configuration file it generates instead of
  176. including ``.config`` directly. This has the advantage that the generated
  177. configuration file will always be a "full" configuration file, even if
  178. ``.config`` is outdated. Otherwise, it might be necessary to run
  179. ``old(def)config`` or ``menuconfig``/``guiconfig`` before rebuilding with an
  180. outdated ``.config``.
  181. If you use ``--sync-deps`` to generate incremental build information, you can
  182. include ``deps/auto.conf`` instead, which is also a full configuration file.
  183. Useful helper macros
  184. ~~~~~~~~~~~~~~~~~~~~
  185. The `include/linux/kconfig.h
  186. <https://github.com/torvalds/linux/blob/master/include/linux/kconfig.h>`_
  187. header in the Linux kernel defines some useful helper macros for testing
  188. Kconfig configuration values.
  189. ``IS_ENABLED()`` is generally useful, allowing configuration values to be
  190. tested in ``if`` statements with no runtime overhead.
  191. Incremental building
  192. ~~~~~~~~~~~~~~~~~~~~
  193. See the docstring for ``Kconfig.sync_deps()`` in `kconfiglib.py
  194. <https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_ for hints
  195. on implementing incremental builds (rebuilding just source files that reference
  196. changed configuration values).
  197. Running the ``scripts/basic/fixdep.c`` tool from the kernel on the output of
  198. ``gcc -MD <source file>`` might give you an idea of how it all fits together.
  199. Library documentation
  200. ---------------------
  201. Kconfiglib comes with extensive documentation in the form of docstrings. To view it, run e.g.
  202. the following command:
  203. .. code:: sh
  204. $ pydoc(3) kconfiglib
  205. For HTML output, add ``-w``:
  206. .. code:: sh
  207. $ pydoc(3) -w kconfiglib
  208. This will also work after installing Kconfiglib with ``pip(3)``.
  209. Documentation for other modules can be viewed in the same way (though a plain
  210. ``--help`` will work when they're run as executables):
  211. .. code:: sh
  212. $ pydoc(3) menuconfig/guiconfig/...
  213. A good starting point for learning the library is to read the module docstring
  214. (which you could also just read directly at the beginning of `kconfiglib.py
  215. <https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_). It
  216. gives an introduction to symbol values, the menu tree, and expressions.
  217. After reading the module docstring, a good next step is to read the ``Kconfig``
  218. class documentation, and then the documentation for the ``Symbol``, ``Choice``,
  219. and ``MenuNode`` classes.
  220. Please tell me if something is unclear or can be explained better.
  221. Library features
  222. ----------------
  223. Kconfiglib can do the following, among other things:
  224. - **Programmatically get and set symbol values**
  225. See `allnoconfig.py
  226. <https://github.com/ulfalizer/Kconfiglib/blob/master/allnoconfig.py>`_ and
  227. `allyesconfig.py
  228. <https://github.com/ulfalizer/Kconfiglib/blob/master/allyesconfig.py>`_,
  229. which are automatically verified to produce identical output to the standard
  230. ``make allnoconfig`` and ``make allyesconfig``.
  231. - **Read and write .config and defconfig files**
  232. The generated ``.config`` and ``defconfig`` (minimal configuration) files are
  233. character-for-character identical to what the C implementation would generate
  234. (except for the header comment). The test suite relies on this, as it
  235. compares the generated files.
  236. - **Write C headers**
  237. The generated headers use the same format as ``include/generated/autoconf.h``
  238. from the Linux kernel. Output for symbols appears in the order that they're
  239. defined, unlike in the C tools (where the order depends on the hash table
  240. implementation).
  241. - **Implement incremental builds**
  242. This uses the same scheme as the ``include/config`` directory in the kernel:
  243. Symbols are translated into files that are touched when the symbol's value
  244. changes between builds, which can be used to avoid having to do a full
  245. rebuild whenever the configuration is changed.
  246. See the ``sync_deps()`` function for more information.
  247. - **Inspect symbols**
  248. Printing a symbol or other item (which calls ``__str__()``) returns its
  249. definition in Kconfig format. This also works for symbols defined in multiple
  250. locations.
  251. A helpful ``__repr__()`` is on all objects too.
  252. All ``__str__()`` and ``__repr__()`` methods are deliberately implemented
  253. with just public APIs, so all symbol information can be fetched separately as
  254. well.
  255. - **Inspect expressions**
  256. Expressions use a simple tuple-based format that can be processed manually
  257. if needed. Expression printing and evaluation functions are provided,
  258. implemented with public APIs.
  259. - **Inspect the menu tree**
  260. The underlying menu tree is exposed, including submenus created implicitly
  261. from symbols depending on preceding symbols. This can be used e.g. to
  262. implement menuconfig-like functionality.
  263. See `menuconfig.py
  264. <https://github.com/ulfalizer/Kconfiglib/blob/master/menuconfig.py>`_/`guiconfig.py
  265. <https://github.com/ulfalizer/Kconfiglib/blob/master/guiconfig.py>`_ and the
  266. minimalistic `menuconfig_example.py
  267. <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/menuconfig_example.py>`_
  268. example.
  269. Kconfig extensions
  270. ~~~~~~~~~~~~~~~~~~
  271. The following Kconfig extensions are available:
  272. - ``source`` supports glob patterns and includes each matching file. A pattern
  273. is required to match at least one file.
  274. A separate ``osource`` statement is available for cases where it's okay for
  275. the pattern to match no files (in which case ``osource`` turns into a no-op).
  276. - A relative ``source`` statement (``rsource``) is available, where file paths
  277. are specified relative to the directory of the current Kconfig file. An
  278. ``orsource`` statement is available as well, analogous to ``osource``.
  279. - Preprocessor user functions can be defined in Python, which makes it simple
  280. to integrate information from existing Python tools into Kconfig (e.g. to
  281. have Kconfig symbols depend on hardware information stored in some other
  282. format).
  283. See the *Kconfig extensions* section in the
  284. `kconfiglib.py <https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_
  285. module docstring for more information.
  286. - ``def_int``, ``def_hex``, and ``def_string`` are available in addition to
  287. ``def_bool`` and ``def_tristate``, allowing ``int``, ``hex``, and ``string``
  288. symbols to be given a type and a default at the same time.
  289. These can be useful in projects that make use of symbols defined in multiple
  290. locations, and remove some Kconfig inconsistency.
  291. - Environment variables are expanded directly in e.g. ``source`` and
  292. ``mainmenu`` statements, meaning ``option env`` symbols are redundant.
  293. This is the standard behavior with the new `Kconfig preprocessor
  294. <https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-macro-language.rst>`__,
  295. which Kconfiglib implements.
  296. ``option env`` symbols are accepted but ignored, which leads the caveat that
  297. they must have the same name as the environment variables they reference
  298. (Kconfiglib warns if the names differ). This keeps Kconfiglib compatible with
  299. older Linux kernels, where the name of the ``option env`` symbol always
  300. matched the environment variable. Compatibility with older Linux kernels is
  301. the main reason ``option env`` is still supported.
  302. The C tools have dropped support for ``option env``.
  303. - Two extra optional warnings can be enabled by setting environment variables,
  304. covering cases that are easily missed when making changes to Kconfig files:
  305. * ``KCONFIG_WARN_UNDEF``: If set to ``y``, warnings will be generated for all
  306. references to undefined symbols within Kconfig files. The only gotcha is
  307. that all hex literals must be prefixed with ``0x`` or ``0X``, to make it
  308. possible to distinguish them from symbol references.
  309. Some projects (e.g. the Linux kernel) use multiple Kconfig trees with many
  310. shared Kconfig files, leading to some safe undefined symbol references.
  311. ``KCONFIG_WARN_UNDEF`` is useful in projects that only have a single
  312. Kconfig tree though.
  313. ``KCONFIG_STRICT`` is an older alias for this environment variable,
  314. supported for backwards compatibility.
  315. * ``KCONFIG_WARN_UNDEF_ASSIGN``: If set to ``y``, warnings will be generated
  316. for all assignments to undefined symbols within ``.config`` files. By
  317. default, no such warnings are generated.
  318. This warning can also be enabled/disabled by setting
  319. ``Kconfig.warn_assign_undef`` to ``True``/``False``.
  320. Other features
  321. --------------
  322. - **Single-file implementation**
  323. The entire library is contained in `kconfiglib.py
  324. <https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_.
  325. The tools implemented on top of it are one file each.
  326. - **Robust and highly compatible with the C Kconfig tools**
  327.  The `test suite <https://github.com/ulfalizer/Kconfiglib/blob/master/testsuite.py>`_
  328. automatically compares output from Kconfiglib and the C tools
  329. by diffing the generated ``.config`` files for the real kernel Kconfig and
  330. defconfig files, for all ARCHes.
  331. This currently involves comparing the output for 36 ARCHes and 498 defconfig
  332. files (or over 18000 ARCH/defconfig combinations in "obsessive" test suite
  333. mode). All tests are expected to pass.
  334. A comprehensive suite of selftests is included as well.
  335. - **Not horribly slow despite being a pure Python implementation**
  336. The `allyesconfig.py
  337. <https://github.com/ulfalizer/Kconfiglib/blob/master/allyesconfig.py>`_
  338. script currently runs in about 1.3 seconds on the Linux kernel on a Core i7
  339. 2600K (with a warm file cache), including the ``make`` overhead from ``make
  340. scriptconfig``. Note that the Linux kernel Kconfigs are absolutely massive
  341. (over 14k symbols for x86) compared to most projects, and also have overhead
  342. from running shell commands via the Kconfig preprocessor.
  343. Kconfiglib is especially speedy in cases where multiple ``.config`` files
  344. need to be processed, because the ``Kconfig`` files will only need to be parsed
  345. once.
  346. For long-running jobs, `PyPy <https://pypy.org/>`_ gives a big performance
  347. boost. CPython is faster for short-running jobs as PyPy needs some time to
  348. warm up.
  349. Kconfiglib also works well with the
  350. `multiprocessing <https://docs.python.org/3/library/multiprocessing.html>`_
  351. module. No global state is kept.
  352. - **Generates more warnings than the C implementation**
  353. Generates the same warnings as the C implementation, plus additional ones.
  354. Also detects dependency and ``source`` loops.
  355. All warnings point out the location(s) in the ``Kconfig`` files where a
  356. symbol is defined, where applicable.
  357. - **Unicode support**
  358. Unicode characters in string literals in ``Kconfig`` and ``.config`` files are
  359. correctly handled. This support mostly comes for free from Python.
  360. - **Windows support**
  361. Nothing Linux-specific is used. Universal newlines mode is used for both
  362. Python 2 and Python 3.
  363. The `Zephyr <https://www.zephyrproject.org/>`_ project uses Kconfiglib to
  364. generate ``.config`` files and C headers on Linux as well as Windows.
  365. - **Internals that (mostly) mirror the C implementation**
  366. While being simpler to understand and tweak.
  367. Menuconfig interfaces
  368. ---------------------
  369. Three configuration interfaces are currently available:
  370. - `menuconfig.py <https://github.com/ulfalizer/Kconfiglib/blob/master/menuconfig.py>`_
  371. is a terminal-based configuration interface implemented using the standard
  372. Python ``curses`` module. ``xconfig`` features like showing invisible symbols and
  373. showing symbol names are included, and it's possible to jump directly to a symbol
  374. in the menu tree (even if it's currently invisible).
  375. .. image:: https://raw.githubusercontent.com/ulfalizer/Kconfiglib/screenshots/screenshots/menuconfig.gif
  376. *There is now also a show-help mode that shows the help text of the currently
  377. selected symbol in the help window at the bottom.*
  378. Starting with Kconfiglib 12.2.0, ``menuconfig.py`` runs under both Python 2
  379. and Python 3 (previously, it only ran under Python 3, so this was a
  380. backport). Running it under Python 3 provides better support for Unicode text
  381. entry (``get_wch()`` is not available in the ``curses`` module on Python 2).
  382. There are no third-party dependencies on \*nix. On Windows,
  383. the ``curses`` modules is not available by default, but support
  384. can be added by installing the ``windows-curses`` package:
  385. .. code-block:: shell
  386. $ pip install windows-curses
  387. This uses wheels built from `this repository
  388. <https://github.com/zephyrproject-rtos/windows-curses>`_, which is in turn
  389. based on Christoph Gohlke's `Python Extension Packages for Windows
  390. <https://www.lfd.uci.edu/~gohlke/pythonlibs/#curses>`_.
  391. See the docstring at the top of `menuconfig.py
  392. <https://github.com/ulfalizer/Kconfiglib/blob/master/menuconfig.py>`_ for
  393. more information about the terminal menuconfig implementation.
  394. - `guiconfig.py
  395. <https://github.com/ulfalizer/Kconfiglib/blob/master/guiconfig.py>`_ is a
  396. graphical configuration interface written in `Tkinter
  397. <https://docs.python.org/3/library/tkinter.html>`_. Like ``menuconfig.py``,
  398. it supports showing all symbols (with invisible symbols in red) and jumping
  399. directly to symbols. Symbol values can also be changed directly from the
  400. jump-to dialog.
  401. When single-menu mode is enabled, a single menu is shown at a time, like in
  402. the terminal menuconfig. Only this mode distinguishes between symbols defined
  403. with ``config`` and symbols defined with ``menuconfig``.
  404. ``guiconfig.py`` has been tested on X11, Windows, and macOS, and is
  405. compatible with both Python 2 and Python 3.
  406. Despite being part of the Python standard library, ``tkinter`` often isn't
  407. included by default in Python installations on Linux. These commands will
  408. install it on a few different distributions:
  409. - Ubuntu: ``sudo apt install python-tk``/``sudo apt install python3-tk``
  410. - Fedora: ``dnf install python2-tkinter``/``dnf install python3-tkinter``
  411. - Arch: ``sudo pacman -S tk``
  412. - Clear Linux: ``sudo swupd bundle-add python3-tcl``
  413. Screenshot below, with show-all mode enabled and the jump-to dialog open:
  414. .. image:: https://raw.githubusercontent.com/ulfalizer/Kconfiglib/screenshots/screenshots/guiconfig.png
  415. To avoid having to carry around a bunch of GIFs, the image data is embedded
  416. in ``guiconfig.py``. To use separate GIF files instead, change
  417. ``_USE_EMBEDDED_IMAGES`` to ``False`` in ``guiconfig.py``. The image files
  418. can be found in the `screenshots
  419. <https://github.com/ulfalizer/Kconfiglib/tree/screenshots/guiconfig>`_
  420. branch.
  421. I did my best with the images, but some are definitely only art adjacent.
  422. Touch-ups are welcome. :)
  423. - `pymenuconfig <https://github.com/RomaVis/pymenuconfig>`_, built by `RomaVis
  424. <https://github.com/RomaVis>`_, is an older portable Python 2/3 TkInter
  425. menuconfig implementation.
  426. Screenshot below:
  427. .. image:: https://raw.githubusercontent.com/RomaVis/pymenuconfig/master/screenshot.PNG
  428. While working on the terminal menuconfig implementation, I added a few APIs
  429. to Kconfiglib that turned out to be handy. ``pymenuconfig`` predates
  430. ``menuconfig.py`` and ``guiconfig.py``, and so didn't have them available.
  431. Blame me for any workarounds.
  432. Examples
  433. --------
  434. Example scripts
  435. ~~~~~~~~~~~~~~~
  436. The `examples/ <https://github.com/ulfalizer/Kconfiglib/blob/master/examples>`_ directory contains some simple example scripts. Among these are the following ones. Make sure you run them with the latest version of Kconfiglib, as they might make use of newly added features.
  437. - `eval_expr.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/eval_expr.py>`_ evaluates an expression in the context of a configuration.
  438. - `find_symbol.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/find_symbol.py>`_ searches through expressions to find references to a symbol, also printing a "backtrace" with parents for each reference found.
  439. - `help_grep.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/help_grep.py>`_ searches for a string in all help texts.
  440. - `print_tree.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/print_tree.py>`_ prints a tree of all configuration items.
  441. - `print_config_tree.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/print_config_tree.py>`_ is similar to ``print_tree.py``, but dumps the tree as it would appear in ``menuconfig``, including values. This can be handy for visually diffing between ``.config`` files and different versions of ``Kconfig`` files.
  442. - `list_undefined.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/list_undefined.py>`_ finds references to symbols that are not defined by any architecture in the Linux kernel.
  443. - `merge_config.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/merge_config.py>`_ merges configuration fragments to produce a complete .config, similarly to ``scripts/kconfig/merge_config.sh`` from the kernel.
  444. - `menuconfig_example.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/menuconfig_example.py>`_ implements a configuration interface that uses notation similar to ``make menuconfig``. It's deliberately kept as simple as possible to demonstrate just the core concepts.
  445. Real-world examples
  446. ~~~~~~~~~~~~~~~~~~~
  447. - `kconfig.py
  448. <https://github.com/zephyrproject-rtos/zephyr/blob/master/scripts/kconfig/kconfig.py>`_
  449. from the `Zephyr <https://www.zephyrproject.org/>`_ project handles
  450. ``.config`` and header file generation, also doing configuration fragment
  451. merging
  452. - `genrest.py
  453. <https://github.com/zephyrproject-rtos/zephyr/blob/master/doc/scripts/genrest.py>`_
  454. generates a Kconfig symbol cross-reference, which can be viewed `here
  455. <http://docs.zephyrproject.org/reference/kconfig/index.html>`__
  456. - `CMake and IDE integration
  457. <https://github.com/espressif/esp-idf/tree/master/tools/kconfig_new>`_ from
  458. the ESP-IDF project, via a configuration server program.
  459. - `A script for turning on USB-related options
  460. <https://github.com/google/syzkaller/blob/master/dashboard/config/kconfiglib-merge-usb-configs.py>`_,
  461. from the `syzkaller <https://github.com/google/syzkaller>`_ project.
  462. - `Various automated checks
  463. <https://github.com/zephyrproject-rtos/ci-tools/blob/master/scripts/check_compliance.py>`_,
  464. including a check for references to undefined Kconfig symbols in source code.
  465. See the ``KconfigCheck`` class.
  466. - `Various utilities
  467. <https://github.com/projectacrn/acrn-hypervisor/tree/master/scripts/kconfig>`_
  468. from the `ACRN <https://projectacrn.org/>`_ project
  469. These use the older Kconfiglib 1 API, which was clunkier and not as general
  470. (functions instead of properties, no direct access to the menu structure or
  471. properties, uglier ``__str__()`` output):
  472. - `genboardscfg.py <http://git.denx.de/?p=u-boot.git;a=blob;f=tools/genboardscfg.py;hb=HEAD>`_ from `Das U-Boot <http://www.denx.de/wiki/U-Boot>`_ generates some sort of legacy board database by pulling information from a newly added Kconfig-based configuration system (as far as I understand it :).
  473. - `gen-manual-lists.py <https://git.busybox.net/buildroot/tree/support/scripts/gen-manual-lists.py?id=5676a2deea896f38123b99781da0a612865adeb0>`_ generated listings for an appendix in the `Buildroot <https://buildroot.org>`_ manual. (The listing has since been removed.)
  474. - `gen_kconfig_doc.py <https://github.com/espressif/esp-idf/blob/master/docs/gen-kconfig-doc.py>`_ from the `esp-idf <https://github.com/espressif/esp-idf>`_ project generates documentation from Kconfig files.
  475. - `SConf <https://github.com/CoryXie/SConf>`_ builds an interactive configuration interface (like ``menuconfig``) on top of Kconfiglib, for use e.g. with `SCons <scons.org>`_.
  476. - `kconfig-diff.py <https://gist.github.com/dubiousjim/5638961>`_ -- a script by `dubiousjim <https://github.com/dubiousjim>`_ that compares kernel configurations.
  477. - Originally, Kconfiglib was used in chapter 4 of my `master's thesis <http://liu.diva-portal.org/smash/get/diva2:473038/FULLTEXT01.pdf>`_ to automatically generate a "minimal" kernel for a given system. Parts of it bother me a bit now, but that's how it goes with old work.
  478. Sample ``make iscriptconfig`` session
  479. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  480. The following log should give some idea of the functionality available in the API:
  481. .. code-block::
  482. $ make iscriptconfig
  483. A Kconfig instance 'kconf' for the architecture x86 has been created.
  484. >>> kconf # Calls Kconfig.__repr__()
  485. <configuration with 13711 symbols, main menu prompt "Linux/x86 4.14.0-rc7 Kernel Configuration", srctree ".", config symbol prefix "CONFIG_", warnings enabled, undef. symbol assignment warnings disabled>
  486. >>> kconf.mainmenu_text # Expanded main menu text
  487. 'Linux/x86 4.14.0-rc7 Kernel Configuration'
  488. >>> kconf.top_node # The implicit top-level menu
  489. <menu node for menu, prompt "Linux/x86 4.14.0-rc7 Kernel Configuration" (visibility y), deps y, 'visible if' deps y, has child, Kconfig:5>
  490. >>> kconf.top_node.list # First child menu node
  491. <menu node for symbol SRCARCH, deps y, has next, Kconfig:7>
  492. >>> print(kconf.top_node.list) # Calls MenuNode.__str__()
  493. config SRCARCH
  494. string
  495. option env="SRCARCH"
  496. default "x86"
  497. >>> sym = kconf.top_node.list.next.item # Item contained in next menu node
  498. >>> print(sym) # Calls Symbol.__str__()
  499. config 64BIT
  500. bool "64-bit kernel" if ARCH = "x86"
  501. default ARCH != "i386"
  502. help
  503. Say yes to build a 64-bit kernel - formerly known as x86_64
  504. Say no to build a 32-bit kernel - formerly known as i386
  505. >>> sym # Calls Symbol.__repr__()
  506. <symbol 64BIT, bool, "64-bit kernel", value y, visibility y, direct deps y, arch/x86/Kconfig:2>
  507. >>> sym.assignable # Currently assignable values (0, 1, 2 = n, m, y)
  508. (0, 2)
  509. >>> sym.set_value(0) # Set it to n
  510. True
  511. >>> sym.tri_value # Check the new value
  512. 0
  513. >>> sym = kconf.syms["X86_MPPARSE"] # Look up symbol by name
  514. >>> print(sym)
  515. config X86_MPPARSE
  516. bool "Enable MPS table" if (ACPI || SFI) && X86_LOCAL_APIC
  517. default y if X86_LOCAL_APIC
  518. help
  519. For old smp systems that do not have proper acpi support. Newer systems
  520. (esp with 64bit cpus) with acpi support, MADT and DSDT will override it
  521. >>> default = sym.defaults[0] # Fetch its first default
  522. >>> sym = default[1] # Fetch the default's condition (just a Symbol here)
  523. >>> print(sym)
  524. config X86_LOCAL_APIC
  525. bool
  526. default y
  527. select IRQ_DOMAIN_HIERARCHY
  528. select PCI_MSI_IRQ_DOMAIN if PCI_MSI
  529. depends on X86_64 || SMP || X86_32_NON_STANDARD || X86_UP_APIC || PCI_MSI
  530. >>> sym.nodes # Show the MenuNode(s) associated with it
  531. [<menu node for symbol X86_LOCAL_APIC, deps n, has next, arch/x86/Kconfig:1015>]
  532. >>> kconfiglib.expr_str(sym.defaults[0][1]) # Print the default's condition
  533. 'X86_64 || SMP || X86_32_NON_STANDARD || X86_UP_APIC || PCI_MSI'
  534. >>> kconfiglib.expr_value(sym.defaults[0][1]) # Evaluate it (0 = n)
  535. 0
  536. >>> kconf.syms["64BIT"].set_value(2)
  537. True
  538. >>> kconfiglib.expr_value(sym.defaults[0][1]) # Evaluate it again (2 = y)
  539. 2
  540. >>> kconf.write_config("myconfig") # Save a .config
  541. >>> ^D
  542. $ cat myconfig
  543. # Generated by Kconfiglib (https://github.com/ulfalizer/Kconfiglib)
  544. CONFIG_64BIT=y
  545. CONFIG_X86_64=y
  546. CONFIG_X86=y
  547. CONFIG_INSTRUCTION_DECODER=y
  548. CONFIG_OUTPUT_FORMAT="elf64-x86-64"
  549. CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig"
  550. CONFIG_LOCKDEP_SUPPORT=y
  551. CONFIG_STACKTRACE_SUPPORT=y
  552. CONFIG_MMU=y
  553. ...
  554. Test suite
  555. ----------
  556. The test suite is run with
  557. .. code::
  558. $ python(3) Kconfiglib/testsuite.py
  559. `pypy <https://pypy.org/>`_ works too, and is much speedier for everything except ``allnoconfig.py``/``allnoconfig_simpler.py``/``allyesconfig.py``, where it doesn't have time to warm up since
  560. the scripts are run via ``make scriptconfig``.
  561. The test suite must be run from the top-level kernel directory. It requires that the
  562. Kconfiglib git repository has been cloned into it and that the makefile patch has been applied.
  563. To get rid of warnings generated for the kernel ``Kconfig`` files, add ``2>/dev/null`` to the command to
  564. discard ``stderr``.
  565. **NOTE: Forgetting to apply the Makefile patch will cause some tests that compare generated configurations to fail**
  566. **NOTE: The test suite overwrites .config in the kernel root, so make sure to back it up.**
  567. The test suite consists of a set of selftests and a set of compatibility tests that
  568. compare configurations generated by Kconfiglib with
  569. configurations generated by the C tools, for a number of cases. See
  570. `testsuite.py <https://github.com/ulfalizer/Kconfiglib/blob/master/testsuite.py>`_
  571. for the available options.
  572. The `tests/reltest <https://github.com/ulfalizer/Kconfiglib/blob/master/tests/reltest>`_ script runs the test suite
  573. and all the example scripts for both Python 2 and Python 3, verifying that everything works.
  574. Rarely, the output from the C tools is changed slightly (most recently due to a
  575. `change <https://www.spinics.net/lists/linux-kbuild/msg17074.html>`_ I added).
  576. If you get test suite failures, try running the test suite again against the
  577. `linux-next tree <https://www.kernel.org/doc/man-pages/linux-next.html>`_,
  578. which has all the latest changes. I will make it clear if any
  579. non-backwards-compatible changes appear.
  580. A lot of time is spent waiting around for ``make`` and the C utilities (which need to reparse all the
  581. Kconfig files for each defconfig test). Adding some multiprocessing to the test suite would make sense
  582. too.
  583. Notes
  584. -----
  585. * This is version 2 of Kconfiglib, which is not backwards-compatible with
  586. Kconfiglib 1. A summary of changes between Kconfiglib 1 and Kconfiglib
  587. 2 can be found `here
  588. <https://github.com/ulfalizer/Kconfiglib/blob/screenshots/kconfiglib-2-changes.txt>`__.
  589. * I sometimes see people add custom output formats, which is pretty
  590. straightforward to do (see the implementations of ``write_autoconf()`` and
  591. ``write_config()`` for a template, and also the documentation of the
  592. ``Symbol.config_string`` property). If you come up with something you think
  593. might be useful to other people, I'm happy to take it in upstream. Batteries
  594. included and all that.
  595. * Kconfiglib assumes the modules symbol is ``MODULES``, which is backwards-compatible.
  596. A warning is printed by default if ``option modules`` is set on some other symbol.
  597. Let me know if you need proper ``option modules`` support. It wouldn't be that
  598. hard to add.
  599. Thanks
  600. ------
  601. - To `RomaVis <https://github.com/RomaVis>`_, for making
  602. `pymenuconfig <https://github.com/RomaVis/pymenuconfig>`_ and suggesting
  603. the ``rsource`` keyword.
  604. - To `Mitja Horvat <https://github.com/pinkfluid>`_, for adding support
  605. for user-defined styles to the terminal menuconfig.
  606. - To `Philip Craig <https://github.com/philipc>`_ for adding
  607. support for the ``allnoconfig_y`` option and fixing an obscure issue
  608. with ``comment``\s inside ``choice``\s (that didn't affect correctness but
  609. made outputs differ). ``allnoconfig_y`` is used to force certain symbols
  610. to ``y`` during ``make allnoconfig`` to improve coverage.
  611. License
  612. -------
  613. See `LICENSE.txt <https://github.com/ulfalizer/Kconfiglib/blob/master/LICENSE.txt>`_. SPDX license identifiers are used in the
  614. source code.