intel_idle.rst 15 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268
  1. .. SPDX-License-Identifier: GPL-2.0
  2. .. include:: <isonum.txt>
  3. ==============================================
  4. ``intel_idle`` CPU Idle Time Management Driver
  5. ==============================================
  6. :Copyright: |copy| 2020 Intel Corporation
  7. :Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
  8. General Information
  9. ===================
  10. ``intel_idle`` is a part of the
  11. :doc:`CPU idle time management subsystem <cpuidle>` in the Linux kernel
  12. (``CPUIdle``). It is the default CPU idle time management driver for the
  13. Nehalem and later generations of Intel processors, but the level of support for
  14. a particular processor model in it depends on whether or not it recognizes that
  15. processor model and may also depend on information coming from the platform
  16. firmware. [To understand ``intel_idle`` it is necessary to know how ``CPUIdle``
  17. works in general, so this is the time to get familiar with :doc:`cpuidle` if you
  18. have not done that yet.]
  19. ``intel_idle`` uses the ``MWAIT`` instruction to inform the processor that the
  20. logical CPU executing it is idle and so it may be possible to put some of the
  21. processor's functional blocks into low-power states. That instruction takes two
  22. arguments (passed in the ``EAX`` and ``ECX`` registers of the target CPU), the
  23. first of which, referred to as a *hint*, can be used by the processor to
  24. determine what can be done (for details refer to Intel Software Developer’s
  25. Manual [1]_). Accordingly, ``intel_idle`` refuses to work with processors in
  26. which the support for the ``MWAIT`` instruction has been disabled (for example,
  27. via the platform firmware configuration menu) or which do not support that
  28. instruction at all.
  29. ``intel_idle`` is not modular, so it cannot be unloaded, which means that the
  30. only way to pass early-configuration-time parameters to it is via the kernel
  31. command line.
  32. .. _intel-idle-enumeration-of-states:
  33. Enumeration of Idle States
  34. ==========================
  35. Each ``MWAIT`` hint value is interpreted by the processor as a license to
  36. reconfigure itself in a certain way in order to save energy. The processor
  37. configurations (with reduced power draw) resulting from that are referred to
  38. as C-states (in the ACPI terminology) or idle states. The list of meaningful
  39. ``MWAIT`` hint values and idle states (i.e. low-power configurations of the
  40. processor) corresponding to them depends on the processor model and it may also
  41. depend on the configuration of the platform.
  42. In order to create a list of available idle states required by the ``CPUIdle``
  43. subsystem (see :ref:`idle-states-representation` in :doc:`cpuidle`),
  44. ``intel_idle`` can use two sources of information: static tables of idle states
  45. for different processor models included in the driver itself and the ACPI tables
  46. of the system. The former are always used if the processor model at hand is
  47. recognized by ``intel_idle`` and the latter are used if that is required for
  48. the given processor model (which is the case for all server processor models
  49. recognized by ``intel_idle``) or if the processor model is not recognized.
  50. [There is a module parameter that can be used to make the driver use the ACPI
  51. tables with any processor model recognized by it; see
  52. `below <intel-idle-parameters_>`_.]
  53. If the ACPI tables are going to be used for building the list of available idle
  54. states, ``intel_idle`` first looks for a ``_CST`` object under one of the ACPI
  55. objects corresponding to the CPUs in the system (refer to the ACPI specification
  56. [2]_ for the description of ``_CST`` and its output package). Because the
  57. ``CPUIdle`` subsystem expects that the list of idle states supplied by the
  58. driver will be suitable for all of the CPUs handled by it and ``intel_idle`` is
  59. registered as the ``CPUIdle`` driver for all of the CPUs in the system, the
  60. driver looks for the first ``_CST`` object returning at least one valid idle
  61. state description and such that all of the idle states included in its return
  62. package are of the FFH (Functional Fixed Hardware) type, which means that the
  63. ``MWAIT`` instruction is expected to be used to tell the processor that it can
  64. enter one of them. The return package of that ``_CST`` is then assumed to be
  65. applicable to all of the other CPUs in the system and the idle state
  66. descriptions extracted from it are stored in a preliminary list of idle states
  67. coming from the ACPI tables. [This step is skipped if ``intel_idle`` is
  68. configured to ignore the ACPI tables; see `below <intel-idle-parameters_>`_.]
  69. Next, the first (index 0) entry in the list of available idle states is
  70. initialized to represent a "polling idle state" (a pseudo-idle state in which
  71. the target CPU continuously fetches and executes instructions), and the
  72. subsequent (real) idle state entries are populated as follows.
  73. If the processor model at hand is recognized by ``intel_idle``, there is a
  74. (static) table of idle state descriptions for it in the driver. In that case,
  75. the "internal" table is the primary source of information on idle states and the
  76. information from it is copied to the final list of available idle states. If
  77. using the ACPI tables for the enumeration of idle states is not required
  78. (depending on the processor model), all of the listed idle state are enabled by
  79. default (so all of them will be taken into consideration by ``CPUIdle``
  80. governors during CPU idle state selection). Otherwise, some of the listed idle
  81. states may not be enabled by default if there are no matching entries in the
  82. preliminary list of idle states coming from the ACPI tables. In that case user
  83. space still can enable them later (on a per-CPU basis) with the help of
  84. the ``disable`` idle state attribute in ``sysfs`` (see
  85. :ref:`idle-states-representation` in :doc:`cpuidle`). This basically means that
  86. the idle states "known" to the driver may not be enabled by default if they have
  87. not been exposed by the platform firmware (through the ACPI tables).
  88. If the given processor model is not recognized by ``intel_idle``, but it
  89. supports ``MWAIT``, the preliminary list of idle states coming from the ACPI
  90. tables is used for building the final list that will be supplied to the
  91. ``CPUIdle`` core during driver registration. For each idle state in that list,
  92. the description, ``MWAIT`` hint and exit latency are copied to the corresponding
  93. entry in the final list of idle states. The name of the idle state represented
  94. by it (to be returned by the ``name`` idle state attribute in ``sysfs``) is
  95. "CX_ACPI", where X is the index of that idle state in the final list (note that
  96. the minimum value of X is 1, because 0 is reserved for the "polling" state), and
  97. its target residency is based on the exit latency value. Specifically, for
  98. C1-type idle states the exit latency value is also used as the target residency
  99. (for compatibility with the majority of the "internal" tables of idle states for
  100. various processor models recognized by ``intel_idle``) and for the other idle
  101. state types (C2 and C3) the target residency value is 3 times the exit latency
  102. (again, that is because it reflects the target residency to exit latency ratio
  103. in the majority of cases for the processor models recognized by ``intel_idle``).
  104. All of the idle states in the final list are enabled by default in this case.
  105. .. _intel-idle-initialization:
  106. Initialization
  107. ==============
  108. The initialization of ``intel_idle`` starts with checking if the kernel command
  109. line options forbid the use of the ``MWAIT`` instruction. If that is the case,
  110. an error code is returned right away.
  111. The next step is to check whether or not the processor model is known to the
  112. driver, which determines the idle states enumeration method (see
  113. `above <intel-idle-enumeration-of-states_>`_), and whether or not the processor
  114. supports ``MWAIT`` (the initialization fails if that is not the case). Then,
  115. the ``MWAIT`` support in the processor is enumerated through ``CPUID`` and the
  116. driver initialization fails if the level of support is not as expected (for
  117. example, if the total number of ``MWAIT`` substates returned is 0).
  118. Next, if the driver is not configured to ignore the ACPI tables (see
  119. `below <intel-idle-parameters_>`_), the idle states information provided by the
  120. platform firmware is extracted from them.
  121. Then, ``CPUIdle`` device objects are allocated for all CPUs and the list of
  122. available idle states is created as explained
  123. `above <intel-idle-enumeration-of-states_>`_.
  124. Finally, ``intel_idle`` is registered with the help of cpuidle_register_driver()
  125. as the ``CPUIdle`` driver for all CPUs in the system and a CPU online callback
  126. for configuring individual CPUs is registered via cpuhp_setup_state(), which
  127. (among other things) causes the callback routine to be invoked for all of the
  128. CPUs present in the system at that time (each CPU executes its own instance of
  129. the callback routine). That routine registers a ``CPUIdle`` device for the CPU
  130. running it (which enables the ``CPUIdle`` subsystem to operate that CPU) and
  131. optionally performs some CPU-specific initialization actions that may be
  132. required for the given processor model.
  133. .. _intel-idle-parameters:
  134. Kernel Command Line Options and Module Parameters
  135. =================================================
  136. The *x86* architecture support code recognizes three kernel command line
  137. options related to CPU idle time management: ``idle=poll``, ``idle=halt``,
  138. and ``idle=nomwait``. If any of them is present in the kernel command line, the
  139. ``MWAIT`` instruction is not allowed to be used, so the initialization of
  140. ``intel_idle`` will fail.
  141. Apart from that there are four module parameters recognized by ``intel_idle``
  142. itself that can be set via the kernel command line (they cannot be updated via
  143. sysfs, so that is the only way to change their values).
  144. The ``max_cstate`` parameter value is the maximum idle state index in the list
  145. of idle states supplied to the ``CPUIdle`` core during the registration of the
  146. driver. It is also the maximum number of regular (non-polling) idle states that
  147. can be used by ``intel_idle``, so the enumeration of idle states is terminated
  148. after finding that number of usable idle states (the other idle states that
  149. potentially might have been used if ``max_cstate`` had been greater are not
  150. taken into consideration at all). Setting ``max_cstate`` can prevent
  151. ``intel_idle`` from exposing idle states that are regarded as "too deep" for
  152. some reason to the ``CPUIdle`` core, but it does so by making them effectively
  153. invisible until the system is shut down and started again which may not always
  154. be desirable. In practice, it is only really necessary to do that if the idle
  155. states in question cannot be enabled during system startup, because in the
  156. working state of the system the CPU power management quality of service (PM
  157. QoS) feature can be used to prevent ``CPUIdle`` from touching those idle states
  158. even if they have been enumerated (see :ref:`cpu-pm-qos` in :doc:`cpuidle`).
  159. Setting ``max_cstate`` to 0 causes the ``intel_idle`` initialization to fail.
  160. The ``no_acpi`` and ``use_acpi`` module parameters (recognized by ``intel_idle``
  161. if the kernel has been configured with ACPI support) can be set to make the
  162. driver ignore the system's ACPI tables entirely or use them for all of the
  163. recognized processor models, respectively (they both are unset by default and
  164. ``use_acpi`` has no effect if ``no_acpi`` is set).
  165. The value of the ``states_off`` module parameter (0 by default) represents a
  166. list of idle states to be disabled by default in the form of a bitmask.
  167. Namely, the positions of the bits that are set in the ``states_off`` value are
  168. the indices of idle states to be disabled by default (as reflected by the names
  169. of the corresponding idle state directories in ``sysfs``, :file:`state0`,
  170. :file:`state1` ... :file:`state<i>` ..., where ``<i>`` is the index of the given
  171. idle state; see :ref:`idle-states-representation` in :doc:`cpuidle`).
  172. For example, if ``states_off`` is equal to 3, the driver will disable idle
  173. states 0 and 1 by default, and if it is equal to 8, idle state 3 will be
  174. disabled by default and so on (bit positions beyond the maximum idle state index
  175. are ignored).
  176. The idle states disabled this way can be enabled (on a per-CPU basis) from user
  177. space via ``sysfs``.
  178. .. _intel-idle-core-and-package-idle-states:
  179. Core and Package Levels of Idle States
  180. ======================================
  181. Typically, in a processor supporting the ``MWAIT`` instruction there are (at
  182. least) two levels of idle states (or C-states). One level, referred to as
  183. "core C-states", covers individual cores in the processor, whereas the other
  184. level, referred to as "package C-states", covers the entire processor package
  185. and it may also involve other components of the system (GPUs, memory
  186. controllers, I/O hubs etc.).
  187. Some of the ``MWAIT`` hint values allow the processor to use core C-states only
  188. (most importantly, that is the case for the ``MWAIT`` hint value corresponding
  189. to the ``C1`` idle state), but the majority of them give it a license to put
  190. the target core (i.e. the core containing the logical CPU executing ``MWAIT``
  191. with the given hint value) into a specific core C-state and then (if possible)
  192. to enter a specific package C-state at the deeper level. For example, the
  193. ``MWAIT`` hint value representing the ``C3`` idle state allows the processor to
  194. put the target core into the low-power state referred to as "core ``C3``" (or
  195. ``CC3``), which happens if all of the logical CPUs (SMT siblings) in that core
  196. have executed ``MWAIT`` with the ``C3`` hint value (or with a hint value
  197. representing a deeper idle state), and in addition to that (in the majority of
  198. cases) it gives the processor a license to put the entire package (possibly
  199. including some non-CPU components such as a GPU or a memory controller) into the
  200. low-power state referred to as "package ``C3``" (or ``PC3``), which happens if
  201. all of the cores have gone into the ``CC3`` state and (possibly) some additional
  202. conditions are satisfied (for instance, if the GPU is covered by ``PC3``, it may
  203. be required to be in a certain GPU-specific low-power state for ``PC3`` to be
  204. reachable).
  205. As a rule, there is no simple way to make the processor use core C-states only
  206. if the conditions for entering the corresponding package C-states are met, so
  207. the logical CPU executing ``MWAIT`` with a hint value that is not core-level
  208. only (like for ``C1``) must always assume that this may cause the processor to
  209. enter a package C-state. [That is why the exit latency and target residency
  210. values corresponding to the majority of ``MWAIT`` hint values in the "internal"
  211. tables of idle states in ``intel_idle`` reflect the properties of package
  212. C-states.] If using package C-states is not desirable at all, either
  213. :ref:`PM QoS <cpu-pm-qos>` or the ``max_cstate`` module parameter of
  214. ``intel_idle`` described `above <intel-idle-parameters_>`_ must be used to
  215. restrict the range of permissible idle states to the ones with core-level only
  216. ``MWAIT`` hint values (like ``C1``).
  217. References
  218. ==========
  219. .. [1] *Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 2B*,
  220. https://www.intel.com/content/www/us/en/architecture-and-technology/64-ia-32-architectures-software-developer-vol-2b-manual.html
  221. .. [2] *Advanced Configuration and Power Interface (ACPI) Specification*,
  222. https://uefi.org/specifications