pmu.rst 9.7 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255
  1. ===================================
  2. Supporting PMUs on RISC-V platforms
  3. ===================================
  4. Alan Kao <alankao@andestech.com>, Mar 2018
  5. Introduction
  6. ------------
  7. As of this writing, perf_event-related features mentioned in The RISC-V ISA
  8. Privileged Version 1.10 are as follows:
  9. (please check the manual for more details)
  10. * [m|s]counteren
  11. * mcycle[h], cycle[h]
  12. * minstret[h], instret[h]
  13. * mhpeventx, mhpcounterx[h]
  14. With such function set only, porting perf would require a lot of work, due to
  15. the lack of the following general architectural performance monitoring features:
  16. * Enabling/Disabling counters
  17. Counters are just free-running all the time in our case.
  18. * Interrupt caused by counter overflow
  19. No such feature in the spec.
  20. * Interrupt indicator
  21. It is not possible to have many interrupt ports for all counters, so an
  22. interrupt indicator is required for software to tell which counter has
  23. just overflowed.
  24. * Writing to counters
  25. There will be an SBI to support this since the kernel cannot modify the
  26. counters [1]. Alternatively, some vendor considers to implement
  27. hardware-extension for M-S-U model machines to write counters directly.
  28. This document aims to provide developers a quick guide on supporting their
  29. PMUs in the kernel. The following sections briefly explain perf' mechanism
  30. and todos.
  31. You may check previous discussions here [1][2]. Also, it might be helpful
  32. to check the appendix for related kernel structures.
  33. 1. Initialization
  34. -----------------
  35. *riscv_pmu* is a global pointer of type *struct riscv_pmu*, which contains
  36. various methods according to perf's internal convention and PMU-specific
  37. parameters. One should declare such instance to represent the PMU. By default,
  38. *riscv_pmu* points to a constant structure *riscv_base_pmu*, which has very
  39. basic support to a baseline QEMU model.
  40. Then he/she can either assign the instance's pointer to *riscv_pmu* so that
  41. the minimal and already-implemented logic can be leveraged, or invent his/her
  42. own *riscv_init_platform_pmu* implementation.
  43. In other words, existing sources of *riscv_base_pmu* merely provide a
  44. reference implementation. Developers can flexibly decide how many parts they
  45. can leverage, and in the most extreme case, they can customize every function
  46. according to their needs.
  47. 2. Event Initialization
  48. -----------------------
  49. When a user launches a perf command to monitor some events, it is first
  50. interpreted by the userspace perf tool into multiple *perf_event_open*
  51. system calls, and then each of them calls to the body of *event_init*
  52. member function that was assigned in the previous step. In *riscv_base_pmu*'s
  53. case, it is *riscv_event_init*.
  54. The main purpose of this function is to translate the event provided by user
  55. into bitmap, so that HW-related control registers or counters can directly be
  56. manipulated. The translation is based on the mappings and methods provided in
  57. *riscv_pmu*.
  58. Note that some features can be done in this stage as well:
  59. (1) interrupt setting, which is stated in the next section;
  60. (2) privilege level setting (user space only, kernel space only, both);
  61. (3) destructor setting. Normally it is sufficient to apply *riscv_destroy_event*;
  62. (4) tweaks for non-sampling events, which will be utilized by functions such as
  63. *perf_adjust_period*, usually something like the follows::
  64. if (!is_sampling_event(event)) {
  65. hwc->sample_period = x86_pmu.max_period;
  66. hwc->last_period = hwc->sample_period;
  67. local64_set(&hwc->period_left, hwc->sample_period);
  68. }
  69. In the case of *riscv_base_pmu*, only (3) is provided for now.
  70. 3. Interrupt
  71. ------------
  72. 3.1. Interrupt Initialization
  73. This often occurs at the beginning of the *event_init* method. In common
  74. practice, this should be a code segment like::
  75. int x86_reserve_hardware(void)
  76. {
  77. int err = 0;
  78. if (!atomic_inc_not_zero(&pmc_refcount)) {
  79. mutex_lock(&pmc_reserve_mutex);
  80. if (atomic_read(&pmc_refcount) == 0) {
  81. if (!reserve_pmc_hardware())
  82. err = -EBUSY;
  83. else
  84. reserve_ds_buffers();
  85. }
  86. if (!err)
  87. atomic_inc(&pmc_refcount);
  88. mutex_unlock(&pmc_reserve_mutex);
  89. }
  90. return err;
  91. }
  92. And the magic is in *reserve_pmc_hardware*, which usually does atomic
  93. operations to make implemented IRQ accessible from some global function pointer.
  94. *release_pmc_hardware* serves the opposite purpose, and it is used in event
  95. destructors mentioned in previous section.
  96. (Note: From the implementations in all the architectures, the *reserve/release*
  97. pair are always IRQ settings, so the *pmc_hardware* seems somehow misleading.
  98. It does NOT deal with the binding between an event and a physical counter,
  99. which will be introduced in the next section.)
  100. 3.2. IRQ Structure
  101. Basically, a IRQ runs the following pseudo code::
  102. for each hardware counter that triggered this overflow
  103. get the event of this counter
  104. // following two steps are defined as *read()*,
  105. // check the section Reading/Writing Counters for details.
  106. count the delta value since previous interrupt
  107. update the event->count (# event occurs) by adding delta, and
  108. event->hw.period_left by subtracting delta
  109. if the event overflows
  110. sample data
  111. set the counter appropriately for the next overflow
  112. if the event overflows again
  113. too frequently, throttle this event
  114. fi
  115. fi
  116. end for
  117. However as of this writing, none of the RISC-V implementations have designed an
  118. interrupt for perf, so the details are to be completed in the future.
  119. 4. Reading/Writing Counters
  120. ---------------------------
  121. They seem symmetric but perf treats them quite differently. For reading, there
  122. is a *read* interface in *struct pmu*, but it serves more than just reading.
  123. According to the context, the *read* function not only reads the content of the
  124. counter (event->count), but also updates the left period to the next interrupt
  125. (event->hw.period_left).
  126. But the core of perf does not need direct write to counters. Writing counters
  127. is hidden behind the abstraction of 1) *pmu->start*, literally start counting so one
  128. has to set the counter to a good value for the next interrupt; 2) inside the IRQ
  129. it should set the counter to the same resonable value.
  130. Reading is not a problem in RISC-V but writing would need some effort, since
  131. counters are not allowed to be written by S-mode.
  132. 5. add()/del()/start()/stop()
  133. -----------------------------
  134. Basic idea: add()/del() adds/deletes events to/from a PMU, and start()/stop()
  135. starts/stop the counter of some event in the PMU. All of them take the same
  136. arguments: *struct perf_event *event* and *int flag*.
  137. Consider perf as a state machine, then you will find that these functions serve
  138. as the state transition process between those states.
  139. Three states (event->hw.state) are defined:
  140. * PERF_HES_STOPPED: the counter is stopped
  141. * PERF_HES_UPTODATE: the event->count is up-to-date
  142. * PERF_HES_ARCH: arch-dependent usage ... we don't need this for now
  143. A normal flow of these state transitions are as follows:
  144. * A user launches a perf event, resulting in calling to *event_init*.
  145. * When being context-switched in, *add* is called by the perf core, with a flag
  146. PERF_EF_START, which means that the event should be started after it is added.
  147. At this stage, a general event is bound to a physical counter, if any.
  148. The state changes to PERF_HES_STOPPED and PERF_HES_UPTODATE, because it is now
  149. stopped, and the (software) event count does not need updating.
  150. - *start* is then called, and the counter is enabled.
  151. With flag PERF_EF_RELOAD, it writes an appropriate value to the counter (check
  152. previous section for detail).
  153. Nothing is written if the flag does not contain PERF_EF_RELOAD.
  154. The state now is reset to none, because it is neither stopped nor updated
  155. (the counting already started)
  156. * When being context-switched out, *del* is called. It then checks out all the
  157. events in the PMU and calls *stop* to update their counts.
  158. - *stop* is called by *del*
  159. and the perf core with flag PERF_EF_UPDATE, and it often shares the same
  160. subroutine as *read* with the same logic.
  161. The state changes to PERF_HES_STOPPED and PERF_HES_UPTODATE, again.
  162. - Life cycle of these two pairs: *add* and *del* are called repeatedly as
  163. tasks switch in-and-out; *start* and *stop* is also called when the perf core
  164. needs a quick stop-and-start, for instance, when the interrupt period is being
  165. adjusted.
  166. Current implementation is sufficient for now and can be easily extended to
  167. features in the future.
  168. A. Related Structures
  169. ---------------------
  170. * struct pmu: include/linux/perf_event.h
  171. * struct riscv_pmu: arch/riscv/include/asm/perf_event.h
  172. Both structures are designed to be read-only.
  173. *struct pmu* defines some function pointer interfaces, and most of them take
  174. *struct perf_event* as a main argument, dealing with perf events according to
  175. perf's internal state machine (check kernel/events/core.c for details).
  176. *struct riscv_pmu* defines PMU-specific parameters. The naming follows the
  177. convention of all other architectures.
  178. * struct perf_event: include/linux/perf_event.h
  179. * struct hw_perf_event
  180. The generic structure that represents perf events, and the hardware-related
  181. details.
  182. * struct riscv_hw_events: arch/riscv/include/asm/perf_event.h
  183. The structure that holds the status of events, has two fixed members:
  184. the number of events and the array of the events.
  185. References
  186. ----------
  187. [1] https://github.com/riscv/riscv-linux/pull/124
  188. [2] https://groups.google.com/a/groups.riscv.org/forum/#!topic/sw-dev/f19TmCNP6yA