0022-RISC-V-Rework-branch-costing-model-for-if-conversion.patch 15 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312
  1. From e1e277b418b3ccdd062e81444aee59c72c8c574f Mon Sep 17 00:00:00 2001
  2. From: "Maciej W. Rozycki" <macro@embecosm.com>
  3. Date: Wed, 22 Nov 2023 01:18:25 +0000
  4. Subject: [PATCH 22/30] RISC-V: Rework branch costing model for if-conversion
  5. The generic branch costing model for if-conversion assumes a fixed cost
  6. of COSTS_N_INSNS (2) for a conditional branch, and that one half of that
  7. cost comes from a preceding condition-set instruction, such as with
  8. MODE_CC targets, and then the other half of that cost is for the actual
  9. branch instruction. This is hardcoded for `if_info.original_cost' in
  10. `noce_find_if_block' and regardless of the cost set for branches via
  11. BRANCH_COST.
  12. Then `default_max_noce_ifcvt_seq_cost' instructs if-conversion to prefer
  13. a branchless sequence as costly as high as triple the BRANCH_COST value
  14. set. This is apparently to make up for the inability to accurately
  15. guess the branch penalty.
  16. Consequently for the BRANCH_COST of 3 we commonly set for tuning,
  17. if-conversion will consider branchless sequences costing 3 * 3 - 2 = 7
  18. instruction units more than a corresponding branch sequence. For the
  19. BRANCH_COST of 4 such as with `sifive-7-series' tuning this is even
  20. worse, at 3 * 4 - 2 = 10. Effectively it means a branchless sequence
  21. will always be chosen if available, even a very inefficient one.
  22. Rework the branch costing model to better match our architecture,
  23. observing in particular that we have no preparatory instructions for
  24. branches so that the cost of a branch is naked BRANCH_COST plus any
  25. extra overhead the processing of a branch's source RTX might incur.
  26. Provide TARGET_INSN_COST and TARGET_MAX_NOCE_IFCVT_SEQ_COST handlers
  27. than that return suitable cost based on BRANCH_COST. The latter hook
  28. usually returns a value that is lower than the cost of the corresponding
  29. branched sequence. This is because we don't really want to produce a
  30. branchless sequence that is more expensive than the original branched
  31. sequence. If this turns out too conservative for some corner case, then
  32. this choice might be revisited.
  33. Then we don't want to fiddle with `noce_find_if_block' without a lot of
  34. cross-target verification, so add TARGET_NOCE_CONVERSION_PROFITABLE_P
  35. defined such that it subtracts the fixed COSTS_N_INSNS (2) cost from the
  36. cost of the original branched sequence supplied and instead adds actual
  37. branch cost calculated from the conditional branch instruction used. It
  38. is then further tweaked according to simple analysis of the replacement
  39. branchless sequence produced so as to cancel the cost of an extraneous
  40. zero extend operation produced by `noce_try_store_flag_mask' as observed
  41. with gcc/testsuite/gcc.target/riscv/pr105314.c.
  42. Tweak the testsuite accordingly and set `-mbranch-cost=' explicitly for
  43. the relevant cases so that the expected if-conversion transformation is
  44. made regardless of the default BRANCH_COST value of tuning in effect.
  45. Some of these settings will be lowered later on as deficiencies in
  46. branchless sequence generation have been fixed that lower their cost
  47. calculated by if-conversion.
  48. gcc/
  49. * config/riscv/riscv.cc (riscv_insn_cost): New function.
  50. (riscv_max_noce_ifcvt_seq_cost): Likewise.
  51. (riscv_noce_conversion_profitable_p): Likewise.
  52. (TARGET_INSN_COST): New macro.
  53. (TARGET_MAX_NOCE_IFCVT_SEQ_COST): New macro.
  54. (TARGET_NOCE_CONVERSION_PROFITABLE_P): New macro.
  55. gcc/testsuite/
  56. * gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_imm.c:
  57. Explicitly set the branch cost.
  58. * gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_reg.c:
  59. Likewise.
  60. * gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_reg_reg.c:
  61. Likewise.
  62. * gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_imm.c:
  63. Likewise.
  64. * gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_reg.c:
  65. Likewise.
  66. * gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_reg_reg.c:
  67. Likewise.
  68. ---
  69. gcc/config/riscv/riscv.cc | 120 ++++++++++++++++++
  70. ...tiveSemantics_compare_imm_return_imm_imm.c | 4 +-
  71. ...tiveSemantics_compare_imm_return_imm_reg.c | 4 +-
  72. ...tiveSemantics_compare_imm_return_reg_reg.c | 4 +-
  73. ...tiveSemantics_compare_reg_return_imm_imm.c | 4 +-
  74. ...tiveSemantics_compare_reg_return_imm_reg.c | 4 +-
  75. ...tiveSemantics_compare_reg_return_reg_reg.c | 4 +-
  76. 7 files changed, 132 insertions(+), 12 deletions(-)
  77. diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
  78. index c11223475eb..0a240d81cb3 100644
  79. --- a/gcc/config/riscv/riscv.cc
  80. +++ b/gcc/config/riscv/riscv.cc
  81. @@ -43,6 +43,7 @@ along with GCC; see the file COPYING3. If not see
  82. #include "calls.h"
  83. #include "function.h"
  84. #include "explow.h"
  85. +#include "ifcvt.h"
  86. #include "memmodel.h"
  87. #include "emit-rtl.h"
  88. #include "reload.h"
  89. @@ -2768,6 +2769,118 @@ riscv_address_cost (rtx addr, machine_mode mode,
  90. return riscv_address_insns (addr, mode, false);
  91. }
  92. +/* Implement TARGET_INSN_COST. We factor in the branch cost in the cost
  93. + calculation for conditional branches: one unit is considered the cost
  94. + of microarchitecture-dependent actual branch execution and therefore
  95. + multiplied by BRANCH_COST and any remaining units are considered fixed
  96. + branch overhead. */
  97. +
  98. +static int
  99. +riscv_insn_cost (rtx_insn *insn, bool speed)
  100. +{
  101. + rtx x = PATTERN (insn);
  102. + int cost = pattern_cost (x, speed);
  103. +
  104. + if (JUMP_P (insn)
  105. + && GET_CODE (x) == SET
  106. + && GET_CODE (SET_DEST (x)) == PC
  107. + && GET_CODE (SET_SRC (x)) == IF_THEN_ELSE)
  108. + cost += COSTS_N_INSNS (BRANCH_COST (speed, false) - 1);
  109. + return cost;
  110. +}
  111. +
  112. +/* Implement TARGET_MAX_NOCE_IFCVT_SEQ_COST. Like the default implementation,
  113. + but we consider cost units of branch instructions equal to cost units of
  114. + other instructions. */
  115. +
  116. +static unsigned int
  117. +riscv_max_noce_ifcvt_seq_cost (edge e)
  118. +{
  119. + bool predictable_p = predictable_edge_p (e);
  120. +
  121. + if (predictable_p)
  122. + {
  123. + if (OPTION_SET_P (param_max_rtl_if_conversion_predictable_cost))
  124. + return param_max_rtl_if_conversion_predictable_cost;
  125. + }
  126. + else
  127. + {
  128. + if (OPTION_SET_P (param_max_rtl_if_conversion_unpredictable_cost))
  129. + return param_max_rtl_if_conversion_unpredictable_cost;
  130. + }
  131. +
  132. + return COSTS_N_INSNS (BRANCH_COST (true, predictable_p));
  133. +}
  134. +
  135. +/* Implement TARGET_NOCE_CONVERSION_PROFITABLE_P. We replace the cost of a
  136. + conditional branch assumed by `noce_find_if_block' at `COSTS_N_INSNS (2)'
  137. + by our actual conditional branch cost, observing that our branches test
  138. + conditions directly, so there is no preparatory extra condition-set
  139. + instruction. */
  140. +
  141. +static bool
  142. +riscv_noce_conversion_profitable_p (rtx_insn *seq,
  143. + struct noce_if_info *if_info)
  144. +{
  145. + struct noce_if_info riscv_if_info = *if_info;
  146. +
  147. + riscv_if_info.original_cost -= COSTS_N_INSNS (2);
  148. + riscv_if_info.original_cost += insn_cost (if_info->jump, if_info->speed_p);
  149. +
  150. + /* Hack alert! When `noce_try_store_flag_mask' uses `cstore<mode>4'
  151. + to emit a conditional set operation on DImode output it comes up
  152. + with a sequence such as:
  153. +
  154. + (insn 26 0 27 (set (reg:SI 140)
  155. + (eq:SI (reg/v:DI 137 [ c ])
  156. + (const_int 0 [0]))) 302 {*seq_zero_disi}
  157. + (nil))
  158. + (insn 27 26 28 (set (reg:DI 139)
  159. + (zero_extend:DI (reg:SI 140))) 116 {*zero_extendsidi2_internal}
  160. + (nil))
  161. +
  162. + because our `cstore<mode>4' pattern expands to an insn that gives
  163. + a SImode output. The output of conditional set is 0 or 1 boolean,
  164. + so it is valid for input in any scalar integer mode and therefore
  165. + combine later folds the zero extend operation into an equivalent
  166. + conditional set operation that produces a DImode output, however
  167. + this redundant zero extend operation counts towards the cost of
  168. + the replacement sequence. Compensate for that by incrementing the
  169. + cost of the original sequence as well as the maximum sequence cost
  170. + accordingly. */
  171. + rtx last_dest = NULL_RTX;
  172. + for (rtx_insn *insn = seq; insn; insn = NEXT_INSN (insn))
  173. + {
  174. + if (!NONDEBUG_INSN_P (insn))
  175. + continue;
  176. +
  177. + rtx x = PATTERN (insn);
  178. + if (NONJUMP_INSN_P (insn)
  179. + && GET_CODE (x) == SET)
  180. + {
  181. + rtx src = SET_SRC (x);
  182. + if (last_dest != NULL_RTX
  183. + && GET_CODE (src) == ZERO_EXTEND
  184. + && REG_P (XEXP (src, 0))
  185. + && REGNO (XEXP (src, 0)) == REGNO (last_dest))
  186. + {
  187. + riscv_if_info.original_cost += COSTS_N_INSNS (1);
  188. + riscv_if_info.max_seq_cost += COSTS_N_INSNS (1);
  189. + }
  190. + last_dest = NULL_RTX;
  191. + rtx dest = SET_DEST (x);
  192. + if (COMPARISON_P (src)
  193. + && REG_P (dest)
  194. + && GET_MODE (dest) == SImode)
  195. + last_dest = dest;
  196. + }
  197. + else
  198. + last_dest = NULL_RTX;
  199. + }
  200. +
  201. + return default_noce_conversion_profitable_p (seq, &riscv_if_info);
  202. +}
  203. +
  204. /* Return one word of double-word value OP. HIGH_P is true to select the
  205. high part or false to select the low part. */
  206. @@ -7449,6 +7562,13 @@ riscv_lshift_subword (machine_mode mode, rtx value, rtx shift,
  207. #define TARGET_RTX_COSTS riscv_rtx_costs
  208. #undef TARGET_ADDRESS_COST
  209. #define TARGET_ADDRESS_COST riscv_address_cost
  210. +#undef TARGET_INSN_COST
  211. +#define TARGET_INSN_COST riscv_insn_cost
  212. +
  213. +#undef TARGET_MAX_NOCE_IFCVT_SEQ_COST
  214. +#define TARGET_MAX_NOCE_IFCVT_SEQ_COST riscv_max_noce_ifcvt_seq_cost
  215. +#undef TARGET_NOCE_CONVERSION_PROFITABLE_P
  216. +#define TARGET_NOCE_CONVERSION_PROFITABLE_P riscv_noce_conversion_profitable_p
  217. #undef TARGET_ASM_FILE_START
  218. #define TARGET_ASM_FILE_START riscv_file_start
  219. diff --git a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_imm.c b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_imm.c
  220. index e806f6f0807..d6d3f013e9b 100644
  221. --- a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_imm.c
  222. +++ b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_imm.c
  223. @@ -1,6 +1,6 @@
  224. /* { dg-do compile } */
  225. -/* { dg-options "-march=rv64gc_zicond -mabi=lp64d" { target { rv64 } } } */
  226. -/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f" { target { rv32 } } } */
  227. +/* { dg-options "-march=rv64gc_zicond -mabi=lp64d -mbranch-cost=4" { target { rv64 } } } */
  228. +/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f -mbranch-cost=4" { target { rv32 } } } */
  229. /* { dg-skip-if "" { *-*-* } {"-O0" "-Og" "-Os" "-Oz"} } */
  230. long primitiveSemantics_compare_imm_return_imm_imm_00(long a, long b) {
  231. diff --git a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_reg.c b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_reg.c
  232. index f976d608a03..4c14e892bac 100644
  233. --- a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_reg.c
  234. +++ b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_reg.c
  235. @@ -1,6 +1,6 @@
  236. /* { dg-do compile } */
  237. -/* { dg-options "-march=rv64gc_zicond -mabi=lp64d" { target { rv64 } } } */
  238. -/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f" { target { rv32 } } } */
  239. +/* { dg-options "-march=rv64gc_zicond -mabi=lp64d -mbranch-cost=4" { target { rv64 } } } */
  240. +/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f -mbranch-cost=4" { target { rv32 } } } */
  241. /* { dg-skip-if "" { *-*-* } {"-O0" "-Og" "-Os" "-Oz"} } */
  242. long primitiveSemantics_compare_imm_return_imm_reg_00(long a, long b) {
  243. diff --git a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_reg_reg.c b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_reg_reg.c
  244. index 90e91192373..e95318301bf 100644
  245. --- a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_reg_reg.c
  246. +++ b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_reg_reg.c
  247. @@ -1,6 +1,6 @@
  248. /* { dg-do compile } */
  249. -/* { dg-options "-march=rv64gc_zicond -mabi=lp64d" { target { rv64 } } } */
  250. -/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f" { target { rv32 } } } */
  251. +/* { dg-options "-march=rv64gc_zicond -mabi=lp64d -mbranch-cost=5" { target { rv64 } } } */
  252. +/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f -mbranch-cost=5" { target { rv32 } } } */
  253. /* { dg-skip-if "" { *-*-* } {"-O0" "-Og" "-Os" "-Oz"} } */
  254. long primitiveSemantics_compare_imm_return_reg_reg_00(long a, long b, long c) {
  255. diff --git a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_imm.c b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_imm.c
  256. index 8ad97abc320..cafdf79537d 100644
  257. --- a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_imm.c
  258. +++ b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_imm.c
  259. @@ -1,6 +1,6 @@
  260. /* { dg-do compile } */
  261. -/* { dg-options "-march=rv64gc_zicond -mabi=lp64d" { target { rv64 } } } */
  262. -/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f" { target { rv32 } } } */
  263. +/* { dg-options "-march=rv64gc_zicond -mabi=lp64d -mbranch-cost=4" { target { rv64 } } } */
  264. +/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f -mbranch-cost=4" { target { rv32 } } } */
  265. /* { dg-skip-if "" { *-*-* } {"-O0" "-Og" "-Os" "-Oz"} } */
  266. long primitiveSemantics_compare_reg_return_imm_imm_00(long a, long b, long c) {
  267. diff --git a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_reg.c b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_reg.c
  268. index 5199ba71ef4..dda9a58c388 100644
  269. --- a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_reg.c
  270. +++ b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_reg.c
  271. @@ -1,6 +1,6 @@
  272. /* { dg-do compile } */
  273. -/* { dg-options "-march=rv64gc_zicond -mabi=lp64d" { target { rv64 } } } */
  274. -/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f" { target { rv32 } } } */
  275. +/* { dg-options "-march=rv64gc_zicond -mabi=lp64d -mbranch-cost=4" { target { rv64 } } } */
  276. +/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f -mbranch-cost=4" { target { rv32 } } } */
  277. /* { dg-skip-if "" { *-*-* } {"-O0" "-Og" "-Os" "-Oz"} } */
  278. long primitiveSemantics_compare_reg_return_imm_reg_00(long a, long b, long c) {
  279. diff --git a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_reg_reg.c b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_reg_reg.c
  280. index eecb95688f4..0c2db3ca59d 100644
  281. --- a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_reg_reg.c
  282. +++ b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_reg_reg.c
  283. @@ -1,6 +1,6 @@
  284. /* { dg-do compile } */
  285. -/* { dg-options "-march=rv64gc_zicond -mabi=lp64d" { target { rv64 } } } */
  286. -/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f" { target { rv32 } } } */
  287. +/* { dg-options "-march=rv64gc_zicond -mabi=lp64d -mbranch-cost=5" { target { rv64 } } } */
  288. +/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f -mbranch-cost=5" { target { rv32 } } } */
  289. /* { dg-skip-if "" { *-*-* } {"-O0" "-Og" "-Os" "-Oz"} } */
  290. long primitiveSemantics_compare_reg_return_reg_reg_00(long a, long b, long c,
  291. --
  292. 2.25.1