123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184 |
- .In
- .SH
- A. MEASUREMENTS
- .SH
- A.1. \*(OQThe bottom line\*(CQ
- .PP
- Although examples often are most illustrative, the cruel world out there is
- usually more interested in everyday performance figures. To satisfy those
- people too, we will present a series of measurements on our code expander
- taken from (close to) real life situations. These include measurements
- of compile and run times of different programs,
- compiled with different compilers.
- .SH
- A.2. Compile time measurements
- .PP
- Figure A.2.1 shows compile-time measurements for typical C code:
- the dhrystone benchmark\(dg
- .[ [
- dhrystone
- .]].
- .FS
- \(dg To be certain that we only tested the compiler and not the quality of
- the code in the library, we have added our own version of
- \fIstrcmp\fR and \fIstrcpy\fR and have not used the ones present in the
- library.
- .FE
- The numbers represent the duration of each separate pass of the compiler.
- The numbers at the end of each bar represent the total duration of the
- compilation process. As with all measurements in this chapter, the
- quoted time or duration is the sum of user and system time in seconds.
- .PS
- copy "pics/compile_bars"
- .PE
- .DS
- .IP cem: 6
- C to EM frontend
- .IP opt:
- EM peep-hole optimizer
- .IP be:
- EM to assembler backend
- .IP cpp:
- Sun's C preprocessor
- .IP ccom:
- Sun's C compiler
- .IP iropt:
- Sun's optimizer
- .IP cg:
- Sun's code generator
- .IP as:
- Sun's assembler
- .IP ld:
- Sun's linker
- .ce 1
- \fIFigure A.2.1: compile-time measurements.\fR
- .DE
- .sp
- .PP
- A close examination of the first two bars in fig A.2.1 shows that the maximum
- achievable compile-time
- gain compared to \fIcc\fR is about 50% for medium-sized
- programs.\(dd
- .FS
- \(dd (cpp+ccom+as+ld)/(cem+as+ld) = 1.53
- .FE
- For small programs the gain will be less, due to the almost constant
- start-up time of each pass in the compilation process. Only a
- built-in assembler may increase this number up to
- 180% in the ideal case that the optimizer, backend and assembler
- would run in zero time. Speed-ups of 5 to 10 times as mentioned in
- .[ [
- fast portable compilers
- .]]
- are therefore not possible on the Sun-4 family. This is also due to
- Sun's implementation of saving and restoring register windows. With
- the current implementation in which only a single window is saved
- or restored on a register-window overflow, it is very time consuming
- when programs have highly dynamic stack use
- due to procedure calls (as is often the case with compilers).
- .PP
- Although we are currently a little slower than \fIcc\fR, it is hard to
- blame this on our backend. Optimizing the backend so that it would run
- twice as fast would only reduce the total compilation process by
- a mere 14%.
- .PP
- Finally it is nice to see that our push/pop-optimization,
- initially designed to generate faster code, has also increased the
- compilation speed. (see also figures A.4.1 and A.4.2.)
- .SH
- A.3. Run time performance
- .PP
- Figure A.3.1 shows the run-time performance of different compilers.
- All results are normalized, where the best available compiler (Sun's
- compiler with full optimization) is represented by 1.0 on our scale.
- .PS
- copy "pics/run-time_bars"
- .PE
- .ce 1
- \fIFigure A.3.1: run time performance.\fR
- .sp 1
- .PP
- The fact that our compiler behaves rather poorly compared to Sun's
- compiler is due to the fact that the dhrystone benchmark uses
- relatively many subroutine calls; all of which have to be 'emulated'
- by our backend.
- .SH
- A.4. Overall performance
- .LP
- In the next two figures we will show the combined run and compile time
- performance of 'our' compiler (the ACK C frontend and our backend)
- compared to Sun's C compiler. Figure A.4.1 shows the results from
- measurements on the dhrystone benchmark.
- .G1
- frame invis left solid bot solid
- label left "run time" "(in \(*msec/dhrystone)"
- label bot "compile time (in sec)"
- coord x 0,21 y 0,610
- ticks left out from 0 to 600 by 200
- ticks bot out from 0 to 20 by 5
- "\(bu" at 3.5, 1000000/1700
- "ack w/o opt" ljust at 3.5 + 1, 1000000/1700
- "\(bu" at 2.8, 1000000/8770
- "ack with opt" below at 2.8 + 0.1, 1000000/8770
- "\(bu" at 16.0, 1000000/10434
- "ack -O4" above at 16.0, 1000000/10434
- "\(bu" at 2.3, 1000000/7270
- "\fIcc\fR" above at 2.3, 1000000/7270
- "\(bu" at 9.0, 1000000/12500
- "\fIcc -O4\fR" above at 9.0, 1000000/12500
- "\(bu" at 5.9, 1000000/15250
- "\fIcc -O\fR" below at 5.9, 1000000/15250
- .G2
- .ce 1
- \fIFigure A.4.1: overall performance on dhrystones.
- .sp 1
- .LP
- Fortunately for us, dhrystones are not all there is. The following
- figure shows the same measurements as the previous one, except
- this time we took a benchmark that uses no subroutines: an implementation
- of Eratosthenes' sieve:
- .G1
- frame invis left solid bot solid
- label left "run time" "for one run" "(in sec)" left .6
- label bot "compile time (in sec)"
- coord x 0,11 y 0,21
- ticks bot out from 0 to 10 by 5
- ticks left out from 0 to 20 by 5
- "\(bu" at 2.5, 17.28
- "ack w/o opt" above at 2.5, 17.28
- "\(bu" at 1.6, 2.93
- "ack with opt" above at 1.6, 2.93
- "\(bu" at 9.4, 2.26
- "ack -O4" above at 9.4, 2.26
- "\(bu" at 1.5, 7.43
- "\fIcc\fR" above at 1.5, 7.43
- "\(bu" at 2.7, 2.02
- "\fIcc -O4\fR" ljust at 1.9, 1.2
- "\(bu" at 2.6, 2.10
- "\fIcc -O\fR" ljust at 3.1,2.5
- .G2
- .ce 1
- \fIFigure A.4.2: overall performance on Eratosthenes' sieve.
- .sp 1
- .PP
- Although the above figures speak for themselves, a small comment
- may be in place. At first it is clear that our compiler is neither
- faster than \fIcc\fR, nor produces faster code than \fIcc -O4\fR. It should
- also be noted however, that we do produce better code than \fIcc\fR
- at only a very small additional cost.
- It is also worth noticing that push-pop optimization
- increases run-time speed as well as compile speed.
- The first seems rather obvious,
- since optimized code is
- faster code, but the increase in compile speed may come as a surprise.
- The main reason is that the \fIas\fR+\fIld\fR time depends largely on the
- amount of generated code, which in general
- depends on the efficiency of the code.
- Push-pop optimization removes a lot of useless instructions which
- would otherwise
- have found their way through to the assembler and the loader.
- Useless instructions inserted in an early stage in the compilation
- process will slow down every following stage, so elimination of useless
- instructions in an early stage, even when it requires a little computational
- overhead, can often be beneficial to the overall compilation speed.
- .bp
|