123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312 |
- From e1e277b418b3ccdd062e81444aee59c72c8c574f Mon Sep 17 00:00:00 2001
- From: "Maciej W. Rozycki" <macro@embecosm.com>
- Date: Wed, 22 Nov 2023 01:18:25 +0000
- Subject: [PATCH 22/30] RISC-V: Rework branch costing model for if-conversion
- The generic branch costing model for if-conversion assumes a fixed cost
- of COSTS_N_INSNS (2) for a conditional branch, and that one half of that
- cost comes from a preceding condition-set instruction, such as with
- MODE_CC targets, and then the other half of that cost is for the actual
- branch instruction. This is hardcoded for `if_info.original_cost' in
- `noce_find_if_block' and regardless of the cost set for branches via
- BRANCH_COST.
- Then `default_max_noce_ifcvt_seq_cost' instructs if-conversion to prefer
- a branchless sequence as costly as high as triple the BRANCH_COST value
- set. This is apparently to make up for the inability to accurately
- guess the branch penalty.
- Consequently for the BRANCH_COST of 3 we commonly set for tuning,
- if-conversion will consider branchless sequences costing 3 * 3 - 2 = 7
- instruction units more than a corresponding branch sequence. For the
- BRANCH_COST of 4 such as with `sifive-7-series' tuning this is even
- worse, at 3 * 4 - 2 = 10. Effectively it means a branchless sequence
- will always be chosen if available, even a very inefficient one.
- Rework the branch costing model to better match our architecture,
- observing in particular that we have no preparatory instructions for
- branches so that the cost of a branch is naked BRANCH_COST plus any
- extra overhead the processing of a branch's source RTX might incur.
- Provide TARGET_INSN_COST and TARGET_MAX_NOCE_IFCVT_SEQ_COST handlers
- than that return suitable cost based on BRANCH_COST. The latter hook
- usually returns a value that is lower than the cost of the corresponding
- branched sequence. This is because we don't really want to produce a
- branchless sequence that is more expensive than the original branched
- sequence. If this turns out too conservative for some corner case, then
- this choice might be revisited.
- Then we don't want to fiddle with `noce_find_if_block' without a lot of
- cross-target verification, so add TARGET_NOCE_CONVERSION_PROFITABLE_P
- defined such that it subtracts the fixed COSTS_N_INSNS (2) cost from the
- cost of the original branched sequence supplied and instead adds actual
- branch cost calculated from the conditional branch instruction used. It
- is then further tweaked according to simple analysis of the replacement
- branchless sequence produced so as to cancel the cost of an extraneous
- zero extend operation produced by `noce_try_store_flag_mask' as observed
- with gcc/testsuite/gcc.target/riscv/pr105314.c.
- Tweak the testsuite accordingly and set `-mbranch-cost=' explicitly for
- the relevant cases so that the expected if-conversion transformation is
- made regardless of the default BRANCH_COST value of tuning in effect.
- Some of these settings will be lowered later on as deficiencies in
- branchless sequence generation have been fixed that lower their cost
- calculated by if-conversion.
- gcc/
- * config/riscv/riscv.cc (riscv_insn_cost): New function.
- (riscv_max_noce_ifcvt_seq_cost): Likewise.
- (riscv_noce_conversion_profitable_p): Likewise.
- (TARGET_INSN_COST): New macro.
- (TARGET_MAX_NOCE_IFCVT_SEQ_COST): New macro.
- (TARGET_NOCE_CONVERSION_PROFITABLE_P): New macro.
- gcc/testsuite/
- * gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_imm.c:
- Explicitly set the branch cost.
- * gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_reg.c:
- Likewise.
- * gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_reg_reg.c:
- Likewise.
- * gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_imm.c:
- Likewise.
- * gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_reg.c:
- Likewise.
- * gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_reg_reg.c:
- Likewise.
- ---
- gcc/config/riscv/riscv.cc | 120 ++++++++++++++++++
- ...tiveSemantics_compare_imm_return_imm_imm.c | 4 +-
- ...tiveSemantics_compare_imm_return_imm_reg.c | 4 +-
- ...tiveSemantics_compare_imm_return_reg_reg.c | 4 +-
- ...tiveSemantics_compare_reg_return_imm_imm.c | 4 +-
- ...tiveSemantics_compare_reg_return_imm_reg.c | 4 +-
- ...tiveSemantics_compare_reg_return_reg_reg.c | 4 +-
- 7 files changed, 132 insertions(+), 12 deletions(-)
- diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
- index c11223475eb..0a240d81cb3 100644
- --- a/gcc/config/riscv/riscv.cc
- +++ b/gcc/config/riscv/riscv.cc
- @@ -43,6 +43,7 @@ along with GCC; see the file COPYING3. If not see
- #include "calls.h"
- #include "function.h"
- #include "explow.h"
- +#include "ifcvt.h"
- #include "memmodel.h"
- #include "emit-rtl.h"
- #include "reload.h"
- @@ -2768,6 +2769,118 @@ riscv_address_cost (rtx addr, machine_mode mode,
- return riscv_address_insns (addr, mode, false);
- }
-
- +/* Implement TARGET_INSN_COST. We factor in the branch cost in the cost
- + calculation for conditional branches: one unit is considered the cost
- + of microarchitecture-dependent actual branch execution and therefore
- + multiplied by BRANCH_COST and any remaining units are considered fixed
- + branch overhead. */
- +
- +static int
- +riscv_insn_cost (rtx_insn *insn, bool speed)
- +{
- + rtx x = PATTERN (insn);
- + int cost = pattern_cost (x, speed);
- +
- + if (JUMP_P (insn)
- + && GET_CODE (x) == SET
- + && GET_CODE (SET_DEST (x)) == PC
- + && GET_CODE (SET_SRC (x)) == IF_THEN_ELSE)
- + cost += COSTS_N_INSNS (BRANCH_COST (speed, false) - 1);
- + return cost;
- +}
- +
- +/* Implement TARGET_MAX_NOCE_IFCVT_SEQ_COST. Like the default implementation,
- + but we consider cost units of branch instructions equal to cost units of
- + other instructions. */
- +
- +static unsigned int
- +riscv_max_noce_ifcvt_seq_cost (edge e)
- +{
- + bool predictable_p = predictable_edge_p (e);
- +
- + if (predictable_p)
- + {
- + if (OPTION_SET_P (param_max_rtl_if_conversion_predictable_cost))
- + return param_max_rtl_if_conversion_predictable_cost;
- + }
- + else
- + {
- + if (OPTION_SET_P (param_max_rtl_if_conversion_unpredictable_cost))
- + return param_max_rtl_if_conversion_unpredictable_cost;
- + }
- +
- + return COSTS_N_INSNS (BRANCH_COST (true, predictable_p));
- +}
- +
- +/* Implement TARGET_NOCE_CONVERSION_PROFITABLE_P. We replace the cost of a
- + conditional branch assumed by `noce_find_if_block' at `COSTS_N_INSNS (2)'
- + by our actual conditional branch cost, observing that our branches test
- + conditions directly, so there is no preparatory extra condition-set
- + instruction. */
- +
- +static bool
- +riscv_noce_conversion_profitable_p (rtx_insn *seq,
- + struct noce_if_info *if_info)
- +{
- + struct noce_if_info riscv_if_info = *if_info;
- +
- + riscv_if_info.original_cost -= COSTS_N_INSNS (2);
- + riscv_if_info.original_cost += insn_cost (if_info->jump, if_info->speed_p);
- +
- + /* Hack alert! When `noce_try_store_flag_mask' uses `cstore<mode>4'
- + to emit a conditional set operation on DImode output it comes up
- + with a sequence such as:
- +
- + (insn 26 0 27 (set (reg:SI 140)
- + (eq:SI (reg/v:DI 137 [ c ])
- + (const_int 0 [0]))) 302 {*seq_zero_disi}
- + (nil))
- + (insn 27 26 28 (set (reg:DI 139)
- + (zero_extend:DI (reg:SI 140))) 116 {*zero_extendsidi2_internal}
- + (nil))
- +
- + because our `cstore<mode>4' pattern expands to an insn that gives
- + a SImode output. The output of conditional set is 0 or 1 boolean,
- + so it is valid for input in any scalar integer mode and therefore
- + combine later folds the zero extend operation into an equivalent
- + conditional set operation that produces a DImode output, however
- + this redundant zero extend operation counts towards the cost of
- + the replacement sequence. Compensate for that by incrementing the
- + cost of the original sequence as well as the maximum sequence cost
- + accordingly. */
- + rtx last_dest = NULL_RTX;
- + for (rtx_insn *insn = seq; insn; insn = NEXT_INSN (insn))
- + {
- + if (!NONDEBUG_INSN_P (insn))
- + continue;
- +
- + rtx x = PATTERN (insn);
- + if (NONJUMP_INSN_P (insn)
- + && GET_CODE (x) == SET)
- + {
- + rtx src = SET_SRC (x);
- + if (last_dest != NULL_RTX
- + && GET_CODE (src) == ZERO_EXTEND
- + && REG_P (XEXP (src, 0))
- + && REGNO (XEXP (src, 0)) == REGNO (last_dest))
- + {
- + riscv_if_info.original_cost += COSTS_N_INSNS (1);
- + riscv_if_info.max_seq_cost += COSTS_N_INSNS (1);
- + }
- + last_dest = NULL_RTX;
- + rtx dest = SET_DEST (x);
- + if (COMPARISON_P (src)
- + && REG_P (dest)
- + && GET_MODE (dest) == SImode)
- + last_dest = dest;
- + }
- + else
- + last_dest = NULL_RTX;
- + }
- +
- + return default_noce_conversion_profitable_p (seq, &riscv_if_info);
- +}
- +
- /* Return one word of double-word value OP. HIGH_P is true to select the
- high part or false to select the low part. */
-
- @@ -7449,6 +7562,13 @@ riscv_lshift_subword (machine_mode mode, rtx value, rtx shift,
- #define TARGET_RTX_COSTS riscv_rtx_costs
- #undef TARGET_ADDRESS_COST
- #define TARGET_ADDRESS_COST riscv_address_cost
- +#undef TARGET_INSN_COST
- +#define TARGET_INSN_COST riscv_insn_cost
- +
- +#undef TARGET_MAX_NOCE_IFCVT_SEQ_COST
- +#define TARGET_MAX_NOCE_IFCVT_SEQ_COST riscv_max_noce_ifcvt_seq_cost
- +#undef TARGET_NOCE_CONVERSION_PROFITABLE_P
- +#define TARGET_NOCE_CONVERSION_PROFITABLE_P riscv_noce_conversion_profitable_p
-
- #undef TARGET_ASM_FILE_START
- #define TARGET_ASM_FILE_START riscv_file_start
- diff --git a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_imm.c b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_imm.c
- index e806f6f0807..d6d3f013e9b 100644
- --- a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_imm.c
- +++ b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_imm.c
- @@ -1,6 +1,6 @@
- /* { dg-do compile } */
- -/* { dg-options "-march=rv64gc_zicond -mabi=lp64d" { target { rv64 } } } */
- -/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f" { target { rv32 } } } */
- +/* { dg-options "-march=rv64gc_zicond -mabi=lp64d -mbranch-cost=4" { target { rv64 } } } */
- +/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f -mbranch-cost=4" { target { rv32 } } } */
- /* { dg-skip-if "" { *-*-* } {"-O0" "-Og" "-Os" "-Oz"} } */
-
- long primitiveSemantics_compare_imm_return_imm_imm_00(long a, long b) {
- diff --git a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_reg.c b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_reg.c
- index f976d608a03..4c14e892bac 100644
- --- a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_reg.c
- +++ b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_imm_reg.c
- @@ -1,6 +1,6 @@
- /* { dg-do compile } */
- -/* { dg-options "-march=rv64gc_zicond -mabi=lp64d" { target { rv64 } } } */
- -/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f" { target { rv32 } } } */
- +/* { dg-options "-march=rv64gc_zicond -mabi=lp64d -mbranch-cost=4" { target { rv64 } } } */
- +/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f -mbranch-cost=4" { target { rv32 } } } */
- /* { dg-skip-if "" { *-*-* } {"-O0" "-Og" "-Os" "-Oz"} } */
-
- long primitiveSemantics_compare_imm_return_imm_reg_00(long a, long b) {
- diff --git a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_reg_reg.c b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_reg_reg.c
- index 90e91192373..e95318301bf 100644
- --- a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_reg_reg.c
- +++ b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_imm_return_reg_reg.c
- @@ -1,6 +1,6 @@
- /* { dg-do compile } */
- -/* { dg-options "-march=rv64gc_zicond -mabi=lp64d" { target { rv64 } } } */
- -/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f" { target { rv32 } } } */
- +/* { dg-options "-march=rv64gc_zicond -mabi=lp64d -mbranch-cost=5" { target { rv64 } } } */
- +/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f -mbranch-cost=5" { target { rv32 } } } */
- /* { dg-skip-if "" { *-*-* } {"-O0" "-Og" "-Os" "-Oz"} } */
-
- long primitiveSemantics_compare_imm_return_reg_reg_00(long a, long b, long c) {
- diff --git a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_imm.c b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_imm.c
- index 8ad97abc320..cafdf79537d 100644
- --- a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_imm.c
- +++ b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_imm.c
- @@ -1,6 +1,6 @@
- /* { dg-do compile } */
- -/* { dg-options "-march=rv64gc_zicond -mabi=lp64d" { target { rv64 } } } */
- -/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f" { target { rv32 } } } */
- +/* { dg-options "-march=rv64gc_zicond -mabi=lp64d -mbranch-cost=4" { target { rv64 } } } */
- +/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f -mbranch-cost=4" { target { rv32 } } } */
- /* { dg-skip-if "" { *-*-* } {"-O0" "-Og" "-Os" "-Oz"} } */
-
- long primitiveSemantics_compare_reg_return_imm_imm_00(long a, long b, long c) {
- diff --git a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_reg.c b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_reg.c
- index 5199ba71ef4..dda9a58c388 100644
- --- a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_reg.c
- +++ b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_imm_reg.c
- @@ -1,6 +1,6 @@
- /* { dg-do compile } */
- -/* { dg-options "-march=rv64gc_zicond -mabi=lp64d" { target { rv64 } } } */
- -/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f" { target { rv32 } } } */
- +/* { dg-options "-march=rv64gc_zicond -mabi=lp64d -mbranch-cost=4" { target { rv64 } } } */
- +/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f -mbranch-cost=4" { target { rv32 } } } */
- /* { dg-skip-if "" { *-*-* } {"-O0" "-Og" "-Os" "-Oz"} } */
-
- long primitiveSemantics_compare_reg_return_imm_reg_00(long a, long b, long c) {
- diff --git a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_reg_reg.c b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_reg_reg.c
- index eecb95688f4..0c2db3ca59d 100644
- --- a/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_reg_reg.c
- +++ b/gcc/testsuite/gcc.target/riscv/zicond-primitiveSemantics_compare_reg_return_reg_reg.c
- @@ -1,6 +1,6 @@
- /* { dg-do compile } */
- -/* { dg-options "-march=rv64gc_zicond -mabi=lp64d" { target { rv64 } } } */
- -/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f" { target { rv32 } } } */
- +/* { dg-options "-march=rv64gc_zicond -mabi=lp64d -mbranch-cost=5" { target { rv64 } } } */
- +/* { dg-options "-march=rv32gc_zicond -mabi=ilp32f -mbranch-cost=5" { target { rv32 } } } */
- /* { dg-skip-if "" { *-*-* } {"-O0" "-Og" "-Os" "-Oz"} } */
-
- long primitiveSemantics_compare_reg_return_reg_reg_00(long a, long b, long c,
- --
- 2.25.1
|