123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753 |
- .\" $Id$
- .ll 72n
- .wh 0 hd
- .wh 60 fo
- .de hd
- 'sp 5
- ..
- .de fo
- 'bp
- ..
- .tr ~
- . PARAGRAPH
- .de PP
- .sp
- ..
- . CHAPTER
- .de CH
- .br
- .ne 15
- .sp 3
- .in 0
- \\fB\\$1\\fR
- .in 5
- .PP
- ..
- . SUBCHAPTER
- .de SH
- .br
- .ne 10
- .sp
- .in 5
- \\fB\\$1\\fR
- .in 10
- .PP
- ..
- . INDENT START
- .de IS
- .sp
- .in +5
- ..
- . INDENT END
- .de IE
- .in -5
- .sp
- ..
- . DOUBLE INDENT START
- .de DS
- .sp
- .in +5
- .ll -5
- ..
- . DOUBLE INDENT END
- .de DE
- .ll +5
- .in -5
- .sp
- ..
- . EQUATION START
- .de EQ
- .sp
- .nf
- ..
- . EQUATION END
- .de EN
- .fi
- .sp
- ..
- . TEST
- .de TT
- .ti -5
- Test~\\$1:~
- .br
- ..
- . IMPLEMENTATION 1
- .de I1
- .br
- Implementation~1:
- ..
- . IMPLEMENTATION 2
- .de I2
- .br
- Implementation~2:
- ..
- .de CS
- .br
- ~-~\\
- ..
- .br
- .fi
- .sp 5
- .ce
- \fBPascal Validation Suite Report\fR
- .CH "Pascal processor identification"
- The ACK-Pascal compiler produces code for an EM machine
- as defined in [1].
- It is up to the implementor of the EM machine whether errors like
- integer overflow, undefined operand and range bound error are recognized or not.
- Therefore it depends on the EM machine implementation whether these errors
- are recognized in Pascal programs or not.
- The validation suite results of all known implementations are given.
- .PP
- There does not (yet) exist a hardware EM machine.
- Therefore, EM programs must be interpreted, or translated into
- instructions for a target machine.
- The following implementations currently exist:
- .IS
- .I1
- an interpreter running on a PDP-11 (using UNIX).
- The normal mode of operation for this interpreter is to check
- for undefined integers, overflow, range errors etc.
- .sp
- .I2
- a translator into PDP-11 instructions (using UNIX).
- Less checks are performed than in the interpreter, because the translator
- is intended to speed up the execution of well-debugged programs.
- .IE
- .CH "Test Conditions"
- Tester: E.G. Keizer
- .br
- Date: October 1983
- .br
- Validation Suite version: 3.0
- .PP
- The final test run is made with a slightly
- modified validation suite.
- .SH "Erroneous programs"
- Some test did not conform to the standard proposal of February 1979.
- It is this version of the standard proposal that is used
- by the authors of the validation suite.
- .IS
- .TT 6.6.3.7-4
- The semicolon between high and integer on line 17 is replaced
- by a colon.
- .sp
- .TT 6.7.2.2-13
- The div operator on line 14 replaced by mod.
- .CH "Conformance tests"
- Number of tests passed = 150
- .br
- Number of tests failed = 6
- .SH "Details of failed tests"
- .IS
- .TT 6.1.2-1
- Character sequences starting with the 8 characters 'procedur'
- or 'function' are
- erroneously classified as the word-symbols 'procedure' and 'function'.
- .sp
- .TT 6.1.3-2
- Identifiers identical in the first eight characters, but
- differing in ninth or higher numbered characters are treated as
- identical.
- .sp
- .TT 6.5.1-1
- ACK-Pascal requires all formal program parameters to be
- declared with type \fIfile\fP.
- .sp
- .TT 6.6.6.5-1
- Gives run-time error eof seen at call to eoln.
- A have a hunch that this is a error in the suit.
- .sp
- .TT 6.6.4.1-1
- Redefining the names of some standard procedures leads to incorrect
- behaviour of the runtime system.
- In this case it crashes without a sensible error message.
- .sp
- .TT 6.9.3.5.1-1
- This test can not be translated by our compiler because two
- non-identical variables are used in the same block with the same first eight
- characters.
- The test passed after replacement of one of those names.
- .IE
- .CH "Deviance tests"
- Number of deviations correctly detected = 120
- .br
- Number of tests not detecting deviations = 20
- .SH "Details of deviations"
- The following tests are compiled without a proper error
- indication although they do
- not conform to the standard.
- .IS
- .TT 6.1.6-5
- ACK-Pascal allows labels in the range 0..32767.
- A warning is produced when testing for deviations from the
- standard.
- .sp
- .TT 6.1.8-5
- A missing space between a number and a word symbol is not
- detected.
- .sp
- .TT 6.2.2-8
- .TT 6.3-6
- .TT 6.4.1-3
- .TT 6.6.1-3
- .TT 6.6.1-4
- Undetected scope error. The scope of an identifier should start at the
- beginning of the block in which it is declared.
- In the ACK-Pascal compiler the scope starts just after the declaration,
- however.
- .sp
- .TT 6.4.3.3-7
- The values of fields from one variant are accessible from
- another variant.
- The correlation is exact.
- .sp
- .TT 6.6.3.3-4
- The passing as a variable parameter of the selector of a
- variant part is not detected.
- A runtime error is produced because the variant selector is not
- initialized.
- .sp
- .TT 6.8.2.4-2
- .TT 6.8.2.4-3
- .TT 6.8.2.4-4
- .TT 6.8.2.4-5
- .TT 6.8.2.4-6
- The ACK-Pascal compiler does not restrict the places from where
- a jump to a label by means of a goto-statement is allowed.
- .sp
- .TT 6.8.3.9-5
- .TT 6.8.3.9-6
- .TT 6.8.3.9-7
- .TT 6.8.3.9-16
- There are no errors produced for assignments to a variable
- in use as control-variable of a for-statement.
- .TT 6.8.3.9-8
- .TT 6.8.3.9-9
- Use of a controlled variable after leaving the loop without
- intervening initialization is not detected.
- .IE
- .CH "Error handling"
- The results depend on the EM implementation.
- .sp
- Number of errors correctly detected =
- .in +5
- .I1
- 32
- .I2
- 17
- .in -5
- Number of errors not detected =
- .in +5
- .I1
- 21
- .I2
- 36
- .in -5
- Number of errors incorrectly detected =
- .in +5
- .I1
- 2
- .I2
- 2
- .in -5
- .SH "Details of errors not detected"
- The following test fails because the ACK-Pascal compiler only
- generates a warning that does not prevent to run the tests.
- .IS
- .TT 6.6.2-8
- A warning is produced if there is no assignment to a function-identifier.
- .IE
- With this test the ACK-Pascal compiler issues an error message for a legal
- construct not directly related to the error to be detected.
- .IS
- .TT 6.5.5-2
- Program does not compile.
- Buffer variable of text file is not allowed as variable
- parameter.
- .IE
- The following errors are not detected at all.
- .IS
- .TT 6.2.1-11
- .I2
- The use of an undefined integer is not caught as an error.
- .sp
- .TT 6.4.3.3-10
- .TT 6.4.3.3-11
- .TT 6.4.3.3-12
- .TT 6.4.3.3-13
- The notion of 'current variant' is not implemented, not even if a tagfield
- is present.
- .sp
- .TT 6.4.5-15
- .TT 6.4.6-9
- .TT 6.4.6-10
- .TT 6.4.6-11
- .TT 6.5.3.2-2
- .I2
- Subrange bounds are not checked.
- .sp
- .TT 6.4.6-12
- .TT 6.4.6-13
- .TT 6.7.2.4-4
- If the base-type of a set is a subrange, then the set elements are not checked
- against the bounds of the subrange.
- Only the host-type of this subrange-type is relevant for ACK-Pascal.
- .sp
- .TT 6.5.4-1
- .I2
- Nil pointers are not detected.
- .sp
- .TT 6.5.4-2
- .I2
- Undefined pointers are not detected.
- .sp
- .TT 6.5.5-3
- Changing the file position while the window is in use as actual variable
- parameter or as an element of the record variable list of a with-statement
- is not detected.
- .sp
- .TT 6.6.2-9
- An undefined function result is not detected,
- because it is never used in an expression.
- .sp
- .TT 6.6.5.3-6
- .TT 6.6.5.3-7
- Disposing a variable while it is in use as actual variable parameter or
- as an element of the record variable list of a with-statement is not detected.
- .sp
- .TT 6.6.5.3-8
- .TT 6.6.5.3-9
- .TT 6.6.5.3-10
- It is not detected that a record variable, created with the variant form
- of new, is used as an operand in an expression or as the variable in an
- assignment or as an actual value parameter.
- .sp
- .TT 6.6.5.3-11
- Use of a variable that is not reinitialized after a dispose is
- not detected.
- .sp
- .TT 6.6.6.4-4
- .TT 6.6.6.4-5
- .TT 6.6.6.4-7
- .I2
- There are no range checks for pred, succ and chr.
- .sp
- .TT 6.6.6.5-6
- ACK-Pascal considers a rewrite of a file as a defining
- occurence.
- .sp
- .TT 6.7.2.2-8
- .TT 6.7.2.2-9
- .TT 6.7.2.2-10
- .TT 6.7.2.2-12
- .I2
- Division by 0 or integer overflow is not detected.
- .sp
- .TT 6.8.3.9-18
- The use of the some control variable in two nested for
- statements in not detected.
- .sp
- .TT 6.8.3.9-19
- Access of a control variable after leaving the loop results in
- the final-value, although an error should be produced.
- .sp
- .TT 6.9.3.2-3
- The program stops with a file not open error.
- The rewrite before the write is missing in the program.
- .sp
- .TT 6.9.3.2-4
- .TT 6.9.3.2-5
- Illegal FracDigits values are not detected.
- .CH "Implementation dependence"
- Number of tests run = 14
- .br
- Number of tests incorrectly handled = 0
- .SH "Details of implementation dependence"
- .IS
- .TT 6.1.9-5
- Alternate comment delimiters are implemented
- .sp
- .TT 6.1.9-6
- The equivalent symbols @ for ^, (. for [ and .) for ] are not
- implemented.
- .sp
- .TT 6.4.2.2-10
- Maxint = 32767
- .sp
- .TT 6.4.3.4-5
- Only elements with non-negative ordinal value are allowed in sets.
- .sp
- .TT 6.6.6.1-1
- Standard procedures and functions are not allowed as parameters.
- .sp
- .TT 6.6.6.2-11
- Details of the machine characteristics regarding real numbers:
- .IS
- .nf
- beta = 2
- t = 56
- rnd = 1
- ngrd = 0
- machep = -56
- negep = -56
- iexp = 8
- minexp = -128
- maxexp = 127
- eps = 1.387779e-17
- epsneg = 1.387779e-17
- xmin = 2.938736e-39
- xmax = 1.701412e+38
- .fi
- .IE
- .sp
- .TT 6.7.2.3-3
- .TT 6.7.2.3-4
- All operands of boolean expressions are evaluated.
- .sp
- .TT 6.8.2.2-1
- .TT 6.8.2.2-2
- The expression in an assignment statement is evaluated
- before the variable selection if this involves pointer
- dereferencing or array indexing.
- .sp
- .TT 6.8.2.3-2
- Actual parameters are evaluated in reverse order.
- .sp
- .TT 6.9.3.2-6
- The default width for integer, Boolean and real are 6, 5 and 13.
- .sp
- .TT 6.9.3.5.1-2
- The number of digits written in an exponent is 2.
- .sp
- .TT 6.9.3.6-1
- The representations of true and false are (~true) and (false).
- The parenthesis serve to indicate width.
- .IE
- .CH "Quality measurement"
- Number of tests run = 60
- .br
- Number of tests handled incorrectly = 1
- .SH "Results of tests"
- Several test perform operations on reals on indicate the error
- introduced by these operations.
- For each of these tests the following two quality measures are extracted:
- .sp
- .in +5
- maxRE:~~maximum relative error
- .br
- rmsRE:~~root-mean-square relative error
- .in -5
- .sp 2
- .IS
- .TT 1.2-1
- .I1
- 25 thousand Whetstone instructions per second.
- .I2
- 169 thousand Whetstone instructions per second.
- .sp
- .TT 1.2-2
- The value of (TRUEACC-ACC)*2^56/100000 is 1.4 .
- This is well within the bounds specified in [3].
- .br
- The GAMM measure is:
- .I1
- 238 microseconds
- .I2
- 26.3 microseconds.
- .sp
- .TT 1.2-3
- The number of procedure calls calculated in this test exceeds
- the maximum integer value.
- The program stops indicating overflow.
- .sp
- .TT 6.1.3-3
- The number of significant characters for identifiers is 8.
- .sp
- .TT 6.1.5-8
- There is no maximum to the line length.
- .sp
- .TT 6.1.5-9
- The error message "too many digits" is given for numbers larger
- than maxint.
- .sp
- .TT 6.1.5-10
- .TT 6.1.5-11
- .TT 6.1.5-12
- Normal values are allowed for real constants and variables.
- .sp
- .TT 6.1.7-14
- A reasonably large number of strings is allowed.
- .sp
- .TT 6.1.8-6
- No warning is given for possibly unclosed comments.
- .sp
- .TT 6.2.1-12
- .TT 6.2.1-13
- .TT 6.2.1-14
- .TT 6.2.1-15
- .TT 6.5.1-2
- Large lists of declarations are possible in each block.
- .sp
- .TT 6.4.3.2-6
- An 'array[integer] of' is not allowed.
- .sp
- .TT 6.4.3.2-7
- .TT 6.4.3.2-8
- Large values are allowed for arrays and indices.
- .sp
- .TT 6.4.3.3-14
- Large amounts of case-constant values are allowed in variants.
- .sp
- .TT 6.4.3.3-15
- Large amounts of record sections can appear in the fixed part of
- a record.
- .sp
- .TT 6.4.3.3-16
- Large amounts of variants are allowed in a record.
- .TT 6.4.3.4-4
- Size and speed of Warshall's algorithm depend on the
- implementation of EM:
- .IS
- .I1
- .br
- size: 122 bytes
- .br
- speed: 5.2 seconds
- .sp
- .I2
- .br
- size: 196 bytes
- .br
- speed: 0.7 seconds
- .IE
- .TT 6.5.3.2-3
- Deep nesting of array indices is allowed.
- .sp
- .TT 6.5.3.2-4
- .TT 6.5.3.2-5
- Arrays can have at least 8 dimensions.
- .sp
- .TT 6.6.1-8
- Deep static nesting of procedure is allowed.
- .sp
- .TT 6.6.3.1-6
- Large amounts of formal parameters are allowed.
- .sp
- .TT 6.6.5.3-12
- Dispose is fully implemented.
- .sp
- .TT 6.6.6.2-6
- Test sqrt(x): no errors.
- The error is within acceptable bounds.
- .in +5
- maxRE:~~2~**~-55.50
- .br
- rmsRE:~~2~**~-57.53
- .in -5
- .sp
- .TT 6.6.6.2-7
- Test arctan(x): may cause underflow or overflow errors.
- The error is within acceptable bounds.
- .in +5
- .br
- maxRE:~~2~**~-55.00
- .br
- rmsRE:~~2~**~-56.36
- .in -5
- .sp
- .TT 6.6.6.2-8
- Test exp(x): may cause underflow or overflow errors.
- The error is not within acceptable bounds.
- .in +5
- maxRE:~~2~**~-50.03
- .br
- rmsRE:~~2~**~-51.03
- .in -5
- .sp
- .TT 6.6.6.2-9
- Test sin(x): may cause underflow errors.
- The error is not within acceptable bounds.
- .in +5
- maxRE:~~2~**~-38.20
- .br
- rmsRE:~~2~**~-43.68
- .in -5
- .sp
- Test cos(x): may cause underflow errors.
- The error is not within acceptable bounds.
- .in +5
- maxRE:~~2~**~-41.33
- .br
- rmsRE:~~2~**~-46.62
- .in -5
- .sp
- .TT 6.6.6.2-10
- Test ln(x):
- The error is not within acceptable bounds.
- .in +5
- maxRE:~~2~**~-54.05
- .br
- rmsRE:~~2~**~-55.77
- .in -5
- .sp
- .TT 6.7.1-3
- .TT 6.7.1-4
- .TT 6.7.1-5
- Complex nested expressions are allowed.
- .sp
- .TT 6.7.2.2-14
- Test real division:
- The error is within acceptable bounds.
- .in +5
- maxRE:~~0
- .br
- rmsRE:~~0
- .in -5
- .sp
- .TT 6.7.2.2-15
- Operations of reals in the integer range are exact.
- .sp
- .TT 6.7.3-1
- .TT 6.8.3.2-1
- .TT 6.8.3.4-2
- .TT 6.8.3.5-15
- .TT 6.8.3.7-4
- .TT 6.8.3.8-3
- .TT 6.8.3.9-20
- .TT 6.8.3.10-7
- Static deep nesting of function calls,
- compound statements, if statements, case statements, repeat
- loops, while loops, for loops and with statements is possible.
- .sp
- .TT 6.8.3.2-2
- Large amounts of statements are allowed in a compound
- statement.
- .sp
- .TT 6.8.3.5-12
- The compiler requires case constants to be compatible with
- the case selector.
- .sp
- .TT 6.8.3.5-13
- .TT 6.8.3.5-14
- Large case statements are possible.
- .sp
- .TT 6.9-2
- Recursive IO on the same file is well-behaved.
- .sp
- .TT 6.9.1-6
- The reading of real values from a text file is done with
- sufficient accuracy.
- .in +5
- maxRE:~~2~**~-54.61
- .br
- rmsRE:~~2~**~-56.32
- .in -5
- .sp
- .TT 6.9.1-7
- .TT 6.9.2-2
- .TT 6.9.3-3
- .TT 6.9.4-2
- Read, readln, write and writeln may have large amounts of
- parameters.
- .sp
- .TT 6.9.1-8
- The loss of precision for reals written on a text file and read
- back is:
- .in +5
- maxRE:~~2~**~-53.95
- .br
- rmsRE:~~2~**~-55.90
- .in -5
- .sp
- .TT 6.9.3-2
- File IO buffers without trailing marker are correctly flushed.
- .sp
- .TT 6.9.3.5.2-2
- Reals are written with sufficient accuracy.
- .in +5
- maxRE:~~0
- .br
- rmsRE:~~0
- .in -5
- .IE
- .CH "Level 1 conformance tests"
- Number of test passed = 4
- .br
- Number of tests failed = 1
- .SH "Details of failed tests"
- .IS
- .TT 6.6.3.7-4
- An expression indicated by parenthesis whose
- value is a conformant array is not allowed.
- .IE
- .CH "Level 1 deviance tests"
- Number of deviations correctly detected = 4
- .br
- Number of tests not detecting deviations = 0
- .IE
- .CH "Level 1 error handling"
- The results depend on the EM implementation.
- .sp
- Number of errors correctly detected =
- .in +5
- .I1
- 1
- .I2
- 0
- .in -5
- Number of errors not detected =
- .in +5
- .I1
- 0
- .I2
- 1
- .in -5
- .SH "Details of errors not detected"
- .IS
- .TT 6.6.3.7-9
- .I2
- Subrange bounds are not checked.
- .IE
- .CH "Level 1 quality measurement"
- Number of tests run = 1
- .SH "Results of test"
- .IS
- .TT 6.6.3.7-10
- Large conformant arrays are allowed.
- .IE
- .CH "Extensions"
- Number of tests run = 3
- .SH Details of test failed
- .IS
- .TT 6.1.9-7
- The alternative relational operators are not allowed.
- .sp
- .TT 6.1.9-8
- The alternative symbols for colon, semicolon and assignment are
- not allowed.
- .sp
- .TT 6.8.3.5-16
- The otherwise selector in case statements is not allowed.
- .IE
- .CH "References"
- .ti -5
- [1]~~\
- A.S.Tanenbaum, E.G.Keizer, J.W.Stevenson, Hans van Staveren,
- "Description of a machine architecture for use with block structured
- languages",
- Informatica rapport IR-81.
- .ti -5
- [2]~~\
- ISO standard proposal ISO/TC97/SC5-N462, dated February 1979.
- The same proposal, in slightly modified form, can be found in:
- A.M.Addyman e.a., "A draft description of Pascal",
- Software, practice and experience, May 1979.
- An improved version, received March 1980,
- is followed as much as possible for the
- current ACK-Pascal.
- .ti -5
- [3]~~\
- B. A. Wichman and J du Croz,
- A program to calculate the GAMM measure, Computer Journal,
- November 1979.
|