CHANGES_FROM_133.txt 89 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999100010011002100310041005100610071008100910101011101210131014101510161017101810191020102110221023102410251026102710281029103010311032103310341035103610371038103910401041104210431044104510461047104810491050105110521053105410551056105710581059106010611062106310641065106610671068106910701071107210731074107510761077107810791080108110821083108410851086108710881089109010911092109310941095109610971098109911001101110211031104110511061107110811091110111111121113111411151116111711181119112011211122112311241125112611271128112911301131113211331134113511361137113811391140114111421143114411451146114711481149115011511152115311541155115611571158115911601161116211631164116511661167116811691170117111721173117411751176117711781179118011811182118311841185118611871188118911901191119211931194119511961197119811991200120112021203120412051206120712081209121012111212121312141215121612171218121912201221122212231224122512261227122812291230123112321233123412351236123712381239124012411242124312441245124612471248124912501251125212531254125512561257125812591260126112621263126412651266126712681269127012711272127312741275127612771278127912801281128212831284128512861287128812891290129112921293129412951296129712981299130013011302130313041305130613071308130913101311131213131314131513161317131813191320132113221323132413251326132713281329133013311332133313341335133613371338133913401341134213431344134513461347134813491350135113521353135413551356135713581359136013611362136313641365136613671368136913701371137213731374137513761377137813791380138113821383138413851386138713881389139013911392139313941395139613971398139914001401140214031404140514061407140814091410141114121413141414151416141714181419142014211422142314241425142614271428142914301431143214331434143514361437143814391440144114421443144414451446144714481449145014511452145314541455145614571458145914601461146214631464146514661467146814691470147114721473147414751476147714781479148014811482148314841485148614871488148914901491149214931494149514961497149814991500150115021503150415051506150715081509151015111512151315141515151615171518151915201521152215231524152515261527152815291530153115321533153415351536153715381539154015411542154315441545154615471548154915501551155215531554155515561557155815591560156115621563156415651566156715681569157015711572157315741575157615771578157915801581158215831584158515861587158815891590159115921593159415951596159715981599160016011602160316041605160616071608160916101611161216131614161516161617161816191620162116221623162416251626162716281629163016311632163316341635163616371638163916401641164216431644164516461647164816491650165116521653165416551656165716581659166016611662166316641665166616671668166916701671167216731674167516761677167816791680168116821683168416851686168716881689169016911692169316941695169616971698169917001701170217031704170517061707170817091710171117121713171417151716171717181719172017211722172317241725172617271728172917301731173217331734173517361737173817391740174117421743174417451746174717481749175017511752175317541755175617571758175917601761176217631764176517661767176817691770177117721773177417751776177717781779178017811782178317841785178617871788178917901791179217931794179517961797179817991800180118021803180418051806180718081809181018111812181318141815181618171818181918201821182218231824182518261827182818291830183118321833183418351836183718381839184018411842184318441845184618471848184918501851185218531854185518561857185818591860186118621863186418651866186718681869187018711872187318741875187618771878187918801881188218831884188518861887188818891890189118921893189418951896189718981899190019011902190319041905190619071908190919101911191219131914191519161917191819191920192119221923192419251926192719281929193019311932193319341935193619371938193919401941194219431944194519461947194819491950195119521953195419551956195719581959196019611962196319641965196619671968196919701971197219731974197519761977197819791980198119821983198419851986198719881989199019911992199319941995199619971998199920002001200220032004200520062007200820092010201120122013201420152016201720182019202020212022202320242025202620272028202920302031203220332034203520362037203820392040204120422043204420452046204720482049205020512052205320542055205620572058205920602061206220632064206520662067206820692070207120722073207420752076207720782079208020812082208320842085208620872088208920902091209220932094209520962097209820992100210121022103210421052106210721082109211021112112211321142115211621172118211921202121212221232124212521262127212821292130213121322133213421352136213721382139214021412142214321442145214621472148214921502151215221532154215521562157215821592160216121622163216421652166216721682169217021712172217321742175217621772178217921802181218221832184218521862187218821892190219121922193219421952196219721982199220022012202220322042205220622072208220922102211221222132214221522162217221822192220222122222223222422252226222722282229223022312232223322342235223622372238223922402241224222432244224522462247224822492250225122522253225422552256225722582259226022612262226322642265226622672268226922702271227222732274227522762277227822792280228122822283228422852286228722882289229022912292229322942295229622972298229923002301230223032304230523062307230823092310231123122313231423152316231723182319232023212322232323242325232623272328232923302331233223332334233523362337233823392340234123422343234423452346234723482349235023512352235323542355235623572358235923602361236223632364236523662367236823692370237123722373237423752376237723782379238023812382238323842385238623872388238923902391239223932394239523962397239823992400240124022403240424052406240724082409241024112412241324142415241624172418241924202421242224232424242524262427242824292430243124322433243424352436243724382439244024412442244324442445244624472448
  1. =======================================================================
  2. List of Implemented Fixes and Changes for Maintenance Releases of PCCTS
  3. =======================================================================
  4. DISCLAIMER
  5. The software and these notes are provided "as is". They may include
  6. typographical or technical errors and their authors disclaims all
  7. liability of any kind or nature for damages due to error, fault,
  8. defect, or deficiency regardless of cause. All warranties of any
  9. kind, either express or implied, including, but not limited to, the
  10. implied warranties of merchantability and fitness for a particular
  11. purpose are disclaimed.
  12. -------------------------------------------------------
  13. Note: Items #153 to #1 are now in a separate file named
  14. CHANGES_FROM_133_BEFORE_MR13.txt
  15. -------------------------------------------------------
  16. #312. (Changed in MR33) Bug caused by change #299.
  17. In change #299 a warning message was suppressed when there was
  18. no LT(1) in a semantic predicate and max(k,ck) was 1. The
  19. changed caused the code which set a default predicate depth for
  20. the semantic predicate to be left as 0 rather than set to 1.
  21. This manifested as an error at line #1559 of mrhost.c
  22. Reported by Peter Dulimov.
  23. #311. (Changed in MR33) Added sorcer/lib to Makefile.
  24. Reported by Dale Martin.
  25. #310. (Changed in MR32) In C mode zzastPush was spelled zzastpush in one case.
  26. Reported by Jean-Claude Durand
  27. #309. (Changed in MR32) Renamed baseName because of VMS name conflict
  28. Renamed baseName to pcctsBaseName to avoid library name conflict with
  29. VMS library routine. Reported by Jean-François PIÉRONNE.
  30. #308. (Changed in MR32) Used "template" as name of formal in C routine
  31. In astlib.h routine ast_scan a formal was named "template". This caused
  32. problems when the C code was compiled with a C++ compiler. Reported by
  33. Sabyasachi Dey.
  34. #307. (Changed in MR31) Compiler dependent bug in function prototype generation
  35. The code which generated function prototypes contained a bug which
  36. was compiler/optimization dependent. Under some circumstance an
  37. extra character would be included in portions of a function prototype.
  38. Reported by David Cook.
  39. #306. (Changed in MR30) Validating predicate following a token
  40. A validating predicate which immediately followed a token match
  41. consumed the token after the predicate rather than before. Prior
  42. to this fix (in the following example) isValidTimeScaleValue() in
  43. the predicate would test the text for TIMESCALE rather than for
  44. NUMBER:
  45. time_scale :
  46. TIMESCALE
  47. <<isValidTimeScaleValue(LT(1)->getText())>>?
  48. ts:NUMBER
  49. ( us:MICROSECOND << tVal = ...>>
  50. | ns:NANOSECOND << tVal = ... >>
  51. )
  52. Reported by Adalbert Perbandt.
  53. #305. (Changed in MR30) Alternatives with guess blocks inside (...)* blocks.
  54. In MR14 change #175 fixed a bug in the prediction expressions for guess
  55. blocks which were of the form (alpha)? beta. Unfortunately, this
  56. resulted in a new bug as exemplified by the example below, which computed
  57. the first set for r as {B} rather than {B C}:
  58. r : ( (A)? B
  59. | C
  60. )*
  61. This example doesn't make any sense as A is not a prefix of B, but it
  62. illustrates the problem. This bug did not appear for:
  63. r : ( (A)?
  64. | C
  65. )*
  66. because it does not use the (alpha)? beta form.
  67. Item #175 fixed an asymmetry in ambiguity messages for the following
  68. constructs which appear to have identical ambiguities (between repeating
  69. the loop vs. exiting the loop). MR30 retains this fix, but the implementation
  70. is slightly different.
  71. r_star : ( (A B)? )* A ;
  72. r_plus : ( (A B)? )+ A ;
  73. Reported by Arpad Beszedes (beszedes inf.u-szeged.hu).
  74. #304. (Changed in MR30) Crash when mismatch between output value counts.
  75. For a rule such as:
  76. r1 : r2>[i,j];
  77. r2 >[int i, int j] : A;
  78. If there were extra actuals for the reference to rule r2 from rule r1
  79. there antlr would crash. This bug was introduced by change #276.
  80. Reported by Sinan Karasu.
  81. #303. (Changed in MR30) DLGLexerBase::replchar
  82. DLGLexerBase::replchar and the C mode routine zzreplchar did not work
  83. properly when the new character was 0.
  84. Reported with fix by Philippe Laporte
  85. #302. (Changed in MR28) Fix significant problems in initial release of MR27.
  86. #301. (Changed in MR27) Default tab stops set to 2 spaces.
  87. To have antlr generate true tabs rather than spaces, use "antlr -tab 0".
  88. To generate 4 spaces per tab stop use "antlr -tab 4"
  89. #300. (Changed in MR27)
  90. Consider the following methods of constructing an AST from ID:
  91. rule1!
  92. : id:ID << #0 = #[id]; >> ;
  93. rule2!
  94. : id:ID << #0 = #id; >> ;
  95. rule3
  96. : ID ;
  97. rule4
  98. : id:ID << #0 = #id; >> ;
  99. For rule_2, the AST corresponding to id would always be NULL. This
  100. is because the user explicitly suppressed AST construction using the
  101. "!" operator on the rule. In MR27 the use of an AST expression
  102. such as #id overrides the "!" operator and forces construction of
  103. the AST.
  104. This fix does not apply to C mode ASTs when the ASTs are referenced
  105. using numbers rather than symbols.
  106. For C mode, this requires that the (optional) function/macro zzmk_ast
  107. be defined. This functions copies information from an attribute into
  108. a previously allocated AST.
  109. Reported by Jan Langer (jan langernetz.de)
  110. #299. (Changed in MR27) Don't warn if k=1 and semantic predicate missing LT(i)
  111. If a semantic does not have a reference to LT(i) or (C mode LATEXT(i))
  112. then pccts doesn't know how many lookahead tokens to use for context.
  113. However, if max(k,ck) is 1 then there is really only one choice and
  114. the warning is unnecessary.
  115. #298. (Changed in MR27) Removed "register" for lastpos in dlgauto.c zzgettok
  116. #297. (Changed in MR27) Incorrect prototypes when used with classic C
  117. There were a number of errors in function headers when antlr was
  118. built with compilers that do not have __STDC__ or __cplusplus set.
  119. The functions which have variable length argument lists now use
  120. PCCTS_USE_STDARG rather than __USE_PROTOTYPES__ to determine
  121. whether to use stdargs or varargs.
  122. #296. (Changed in MR27) Complex return types in rules.
  123. The following return type was not properly handled when
  124. unpacking a struct with containing multiple return values:
  125. rule > [int i, IIR_Bool (IIR_Decl::*constraint)()] : ...
  126. Instead of using "constraint", the program got lost and used
  127. an empty string.
  128. Reported by P.A. Wilsey.
  129. #295. (Changed in MR27) Extra ";" following zzGUESS_DONE sometimes.
  130. Certain constructs with guess blocks in MR23 led to extra ";"
  131. preceding the "else" clause of an "if".
  132. Reported by P.A. Wilsey.
  133. #294. (Changed in MR27) Infinite loop in antlr for nested blocks
  134. An oversight in detecting an empty alternative sometimes led
  135. to an infinite loop in antlr when it encountered a rule with
  136. nested blocks and guess blocks.
  137. Reported by P.A. Wilsey.
  138. #293. (Changed in MR27) Sorcerer optimization of _t->type()
  139. Sorcerer generated code may contain many calls to _t->type() in a
  140. single statement. This change introduces a temporary variable
  141. to eliminate unnecesary function calls.
  142. Change implemented by Tom Molteno (tim videoscript.com).
  143. #292. (Changed in MR27)
  144. WARNING: Item #267 changes the signature of methods in the AST class.
  145. **** Be sure to revise your AST functions of the same name ***
  146. #291. (Changed in MR24)
  147. Fix to serious code generation error in MR23 for (...)+ block.
  148. #290. (Changed in MR23)
  149. Item #247 describes a change in the way {...} blocks handled
  150. an error. Consider:
  151. r1 : {A} b ;
  152. b : B;
  153. with input "C".
  154. Prior to change #247, the error would resemble "expected B -
  155. found C". This is correct but incomplete, and therefore
  156. misleading. In #247 it was changed to "expected A, B - found
  157. C". This was fine, except for users of parser exception
  158. handling because the exception was generated in the epilogue
  159. for {...} block rather than in rule b. This made it difficult
  160. for users of parser exception handling because B was not
  161. expected in that context. Those not using parser exception
  162. handling didn't notice the difference.
  163. The current change restores the behavior prior to #247 when
  164. parser exceptions are present, but retains the revised behavior
  165. otherwise. This change should be visible only when exceptions
  166. are in use and only for {...} blocks and sub-blocks of the form
  167. (something|something | something | epsilon) where epsilon represents
  168. an empty production and it is the last alternative of a sub-block.
  169. In contrast, (something | epsilon | something) should generate the
  170. same code as before, even when exceptions are used.
  171. Reported by Philippe Laporte (philippe at transvirtual.com).
  172. #289. (Changed in MR23) Bug in matching complement of a #tokclass
  173. Prior to MR23 when a #tokclass was matched in both its complemented form
  174. and uncomplemented form, the bit set generated for its first use was used
  175. for both cases. However, the prediction expression was correctly computed
  176. in both cases. This meant that the second case would never be matched
  177. because, for the second appearance, the prediction expression and the
  178. set to be matched would be complements of each other.
  179. Consider:
  180. #token A "a"
  181. #token B "b"
  182. #token C "c"
  183. #tokclass AB {A B}
  184. r1 : AB /* alt 1x */
  185. | ~AB /* alt 1y */
  186. ;
  187. Prior to MR23, this resulted in alternative 1y being unreachable. Had it
  188. been written:
  189. r2 : ~AB /* alt 2x */
  190. : AB /* alt 2y */
  191. then alternative 2y would have become unreachable.
  192. This bug was only for the case of complemented #tokclass. For complemented
  193. #token the proper code was generated.
  194. #288. (Changed in MR23) #errclass not restricted to choice points
  195. The #errclass directive is supposed to allow a programmer to define
  196. print strings which should appear in syntax error messages as a replacement
  197. for some combinations of tokens. For instance:
  198. #errclass Operator {PLUS MINUS TIMES DIVIDE}
  199. If a syntax message includes all four of these tokens, and there is no
  200. "better" choice of error class, the word "Operator" will be used rather
  201. than a list of the four token names.
  202. Prior to MR23 the #errclass definitions were used only at choice points
  203. (which call the FAIL macro). In other cases where there was no choice
  204. (e.g. where a single token or token class were matched) the #errclass
  205. information was not used.
  206. With MR23 the #errclass declarations are used for syntax error messages
  207. when matching a #tokclass, a wildcard (i.e. "*"), or the complement of a
  208. #token or #tokclass (e.g. ~Operator).
  209. Please note that #errclass may now be defined using #tokclass names
  210. (see Item #284).
  211. Reported by Philip A. Wilsey.
  212. #287. (Changed in MR23) Print name for #tokclass
  213. Item #148 describes how to give a print name to a #token so that,for
  214. example, #token ID could have the expression "identifier" in syntax
  215. error messages. This has been extended to #tokclass:
  216. #token ID("identifier") "[a-zA-Z]+"
  217. #tokclass Primitive("primitive type")
  218. {INT, FLOAT, CHAR, FLOAT, DOUBLE, BOOL}
  219. This is really a cosmetic change, since #tokclass names do not appear
  220. in any error messages.
  221. #286. (Changed in MR23) Makefile change to use of cd
  222. In cases where a pccts subdirectory name matched a directory identified
  223. in a $CDPATH environment variable the build would fail. All makefile
  224. cd commands have been changed from "cd xyz" to "cd ./xyz" in order
  225. to avoid this problem.
  226. #285. (Changed in MR23) Check for null pointers in some dlg structures
  227. An invalid regular expression can cause dlg to build an invalid
  228. structure to represent the regular expression even while it issues
  229. error messages. Additional pointer checks were added.
  230. Reported by Robert Sherry.
  231. #284. (Changed in MR23) Allow #tokclass in #errclass definitions
  232. Previously, a #tokclass reference in the definition of an
  233. #errclass was not handled properly. Instead of being expanded
  234. into the set of tokens represented by the #tokclass it was
  235. treated somewhat like an #errclass. However, in a later phase
  236. when all #errclass were expanded into the corresponding tokens
  237. the #tokclass reference was not expanded (because it wasn't an
  238. #errclass). In effect the reference was ignored.
  239. This has been fixed.
  240. Problem reported by Mike Dimmick (mike dimmick.demon.co.uk).
  241. #283. (Changed in MR23) Option -tmake invoke's parser's tmake
  242. When the string #(...) appears in an action antlr replaces it with
  243. a call to ASTBase::tmake(...) to construct an AST. It is sometimes
  244. useful to change the tmake routine so that it has access to information
  245. in the parser - something which is not possible with a static method
  246. in an application where they may be multiple parsers active.
  247. The antlr option -tmake replaces the call to ASTBase::tmake with a call
  248. to a user supplied tmake routine.
  249. #282. (Changed in MR23) Initialization error for DBG_REFCOUNTTOKEN
  250. When the pre-processor symbol DBG_REFCOUNTTOKEN is defined
  251. incorrect code is generated to initialize ANTLRRefCountToken::ctor and
  252. dtor.
  253. Fix reported by Sven Kuehn (sven sevenkuehn.de).
  254. #281. (Changed in MR23) Addition of -noctor option for Sorcerer
  255. Added a -noctor option to suppress generation of the blank ctor
  256. for users who wish to define their own ctor.
  257. Contributed by Jan Langer (jan langernetz.de).
  258. #280. (Changed in MR23) Syntax error message for EOF token
  259. The EOF token now receives special treatment in syntax error messages
  260. because there is no text matched by the eof token. The token name
  261. of the eof token is used unless it is "@" - in which case the string
  262. "<eof>" is used.
  263. Problem reported by Erwin Achermann (erwin.achermann switzerland.org).
  264. #279. (Changed in MR23) Exception groups
  265. There was a bug in the way that exception groups were attached to
  266. alternatives which caused problems when there was a block contained
  267. in an alternative. For instance, in the following rule;
  268. statement : IF S { ELSE S }
  269. exception ....
  270. ;
  271. the exception would be attached to the {...} block instead of the
  272. entire alternative because it was attached, in error, to the last
  273. alternative instead of the last OPEN alternative.
  274. Reported by Ty Mordane (tymordane hotmail.com).
  275. #278. (Changed in MR23) makefile changes
  276. Contributed by Tomasz Babczynski (faster lab05-7.ict.pwr.wroc.pl).
  277. The -cfile option is not absolutely needed: when extension of
  278. source file is one of the well-known C/C++ extensions it is
  279. treated as C/C++ source
  280. The gnu make defines the CXX variable as the default C++ compiler
  281. name, so I added a line to copy this (if defined) to the CCC var.
  282. Added a -sor option: after it any -class command defines the class
  283. name for sorcerer, not for ANTLR. A file extended with .sor is
  284. treated as sorcerer input. Because sorcerer can be called multiple
  285. times, -sor option can be repeated. Any files and classes (one class
  286. per group) after each -sor makes one tree parser.
  287. Not implemented:
  288. 1. Generate dependences for user c/c++ files.
  289. 2. Support for -sor in c mode not.
  290. I have left the old genmk program in the directory as genmk_old.c.
  291. #277. (Changed in MR23) Change in macro for failed semantic predicates
  292. In the past, a semantic predicate that failed generated a call to
  293. the macro zzfailed_pred:
  294. #ifndef zzfailed_pred
  295. #define zzfailed_pred(_p) \
  296. if (guessing) { \
  297. zzGUESS_FAIL; \
  298. } else { \
  299. something(_p)
  300. }
  301. #endif
  302. If a user wished to use the failed action option for semantic predicates:
  303. rule : <<my_predicate>>? [my_fail_action] A
  304. | ...
  305. the code for my_fail_action would have to contain logic for handling
  306. the guess part of the zzfailed_pred macro. The user should not have
  307. to be aware of the guess logic in writing the fail action.
  308. The zzfailed_pred has been rewritten to have three arguments:
  309. arg 1: the stringized predicate of the semantic predicate
  310. arg 2: 0 => there is no user-defined fail action
  311. 1 => there is a user-defined fail action
  312. arg 3: the user-defined fail action (if defined)
  313. otherwise a no-operation
  314. The zzfailed_pred macro is now defined as:
  315. #ifndef zzfailed_pred
  316. #define zzfailed_pred(_p,_hasuseraction,_useraction) \
  317. if (guessing) { \
  318. zzGUESS_FAIL; \
  319. } else { \
  320. zzfailed_pred_action(_p,_hasuseraction,_useraction) \
  321. }
  322. #endif
  323. With zzfailed_pred_action defined as:
  324. #ifndef zzfailed_pred_action
  325. #define zzfailed_pred_action(_p,_hasuseraction,_useraction) \
  326. if (_hasUserAction) { _useraction } else { failedSemanticPredicate(_p); }
  327. #endif
  328. In C++ mode failedSemanticPredicate() is a virtual function.
  329. In C mode the default action is a fprintf statement.
  330. Suggested by Erwin Achermann (erwin.achermann switzerland.org).
  331. #276. (Changed in MR23) Addition of return value initialization syntax
  332. In an attempt to reduce the problems caused by the PURIFY macro I have
  333. added new syntax for initializing the return value of rules and the
  334. antlr option "-nopurify".
  335. A rule with a single return argument:
  336. r1 > [Foo f = expr] :
  337. now generates code that resembles:
  338. Foo r1(void) {
  339. Foo _retv = expr;
  340. ...
  341. }
  342. A rule with more than one return argument:
  343. r2 > [Foo f = expr1, Bar b = expr2 ] :
  344. generates code that resembles:
  345. struct _rv1 {
  346. Foo f;
  347. Bar b;
  348. }
  349. _rv1 r2(void) {
  350. struct _rv1 _retv;
  351. _retv.f = expr1;
  352. _retv.b = expr2;
  353. ...
  354. }
  355. C++ style comments appearing in the initialization list may cause problems.
  356. #275. (Changed in MR23) Addition of -nopurify option to antlr
  357. A long time ago the PURIFY macro was introduced to initialize
  358. return value arguments and get rid of annoying messages from program
  359. that checked for uninitialized variables.
  360. This has caused significant annoyance for C++ users that had
  361. classes with virtual functions or non-trivial constructors because
  362. it would zero the object, including the pointer to the virtual
  363. function table. This could be defeated by redefining
  364. the PURIFY macro to be empty, but it was a constant surprise to
  365. new C++ users of pccts.
  366. I would like to remove it, but I fear that some existing programs
  367. depend on it and would break. My temporary solution is to add
  368. an antlr option -nopurify which disables generation of the PURIFY
  369. macro call.
  370. The PURIFY macro should be avoided in favor of the new syntax
  371. for initializing return arguments described in item #275.
  372. To avoid name clash, the PURIFY macro has been renamed PCCTS_PURIFY.
  373. #274. (Changed in MR23) DLexer.cpp renamed to DLexer.h
  374. (Changed in MR23) ATokPtr.cpp renamed to ATokPtrImpl.h
  375. These two files had .cpp extensions but acted like .h files because
  376. there were included in other files. This caused problems for many IDE.
  377. I have renamed them. The ATokPtrImpl.h was necessary because there was
  378. already an ATokPtr.h.
  379. #273. (Changed in MR23) Default win32 library changed to multi-threaded DLL
  380. The model used for building the Win32 debug and release libraries has changed
  381. to multi-threaded DLL.
  382. To make this change in your MSVC 6 project:
  383. Project -> Settings
  384. Select the C++ tab in the right pane of the dialog box
  385. Select "Category: Code Generation"
  386. Under "Use run-time library" select one of the following:
  387. Multi-threaded DLL
  388. Debug Multi-threaded DLL
  389. Suggested by Bill Menees (bill.menees gogallagher.com)
  390. #272. (Changed in MR23) Failed semantic predicate reported via virtual function
  391. In the past, a failed semantic predicated reported the problem via a
  392. macro which used fprintf(). The macro now expands into a call on
  393. the virtual function ANTLRParser::failedSemanticPredicate().
  394. #271. (Changed in MR23) Warning for LT(i), LATEXT(i) in token match actions
  395. An bug (or at least an oddity) is that a reference to LT(1), LA(1),
  396. or LATEXT(1) in an action which immediately follows a token match
  397. in a rule refers to the token matched, not the token which is in
  398. the lookahead buffer. Consider:
  399. r : abc <<action alpha>> D <<action beta>> E;
  400. In this case LT(1) in action alpha will refer to the next token in
  401. the lookahead buffer ("D"), but LT(1) in action beta will refer to
  402. the token matched by D - the preceding token.
  403. A warning has been added for users about this when an action
  404. following a token match contains a reference to LT(1), LA(1), or LATEXT(1).
  405. This behavior should be changed, but it appears in too many programs
  406. now. Another problem, perhaps more significant, is that the obvious
  407. fix (moving the consume() call to before the action) could change the
  408. order in which input is requested and output appears in existing programs.
  409. This problem was reported, along with a fix by Benjamin Mandel
  410. (beny sd.co.il). However, I felt that changing the behavior was too
  411. dangerous for existing code.
  412. #270. (Changed in MR23) Removed static objects from PCCTSAST.cpp
  413. There were some statically allocated objects in PCCTSAST.cpp
  414. These were changed to non-static.
  415. #269. (Changed in MR23) dlg output for initializing static array
  416. The output from dlg contains a construct similar to the
  417. following:
  418. struct XXX {
  419. static const int size;
  420. static int array1[5];
  421. };
  422. const int XXX::size = 4;
  423. int XXX::array1[size+1];
  424. The problem is that although the expression "size+1" used in
  425. the definition of array1 is equal to 5 (the expression used to
  426. declare array), it is not considered equivalent by some compilers.
  427. Reported with fix by Volker H. Simonis (simonis informatik.uni-tuebingen.de)
  428. #268. (Changed in MR23) syn() routine output when k > 1
  429. The syn() routine is supposed to print out the text of the
  430. token causing the syntax error. It appears that it always
  431. used the text from the first lookahead token rather than the
  432. appropriate one. The appropriate one is computed by comparing
  433. the token codes of lookahead token i (for i = 1 to k) with
  434. the FIRST(i) set.
  435. This has been corrected in ANTLRParser::syn().
  436. Reported by Bill Menees (bill.menees gogallagher.com)
  437. #267. (Changed in MR23) AST traversal functions client data argument
  438. The AST traversal functions now take an extra (optional) parameter
  439. which can point to client data:
  440. preorder_action(void* pData = NULL)
  441. preorder_before_action(void* pData = NULL)
  442. preorder_after_action(void* pData = NULL)
  443. **** Warning: this changes the AST signature. ***
  444. **** Be sure to revise your AST functions of the same name ***
  445. Bill Menees (bill.menees gogallagher.com)
  446. #266. (Changed in MR23) virtual function printMessage()
  447. Bill Menees (bill.menees gogallagher.com) has completed the
  448. tedious tasks of replacing all calls to fprintf() with calls
  449. to the virtual function printMessage(). For classes which
  450. have a pointer to the parser it forwards the printMessage()
  451. call to the parser's printMessage() routine.
  452. This should make it significantly easier to redirect pccts
  453. error and warning messages.
  454. #265. (Changed in MR23) Remove "labase++" in C++ mode
  455. In C++ mode labase++ is called when a token is matched.
  456. It appears that labase is not used in C++ mode at all, so
  457. this code has been commented out.
  458. #264. (Changed in MR23) Complete rewrite of ParserBlackBox.h
  459. The parser black box (PBlackBox.h) was completely rewritten
  460. by Chris Uzdavinis (chris atdesk.com) to improve its robustness.
  461. #263. (Changed in MR23) -preamble and -preamble_first rescinded
  462. Changes for item #253 have been rescinded.
  463. #262. (Changed in MR23) Crash with -alpha option during traceback
  464. Under some circumstances a -alpha traceback was started at the
  465. "wrong" time. As a result, internal data structures were not
  466. initialized.
  467. Reported by Arpad Beszedes (beszedes inf.u-szeged.hu).
  468. #261. (Changed in MR23) Defer token fetch for C++ mode
  469. Item #216 has been revised to indicate that use of the defer fetch
  470. option (ZZDEFER_FETCH) requires dlg option -i.
  471. #260. (MR22) Raise default lex buffer size from 8,000 to 32,000 bytes.
  472. ZZLEXBUFSIZE is the size (in bytes) of the buffer used by dlg
  473. generated lexers. The default value has been raised to 32,000 and
  474. the value used by antlr, dlg, and sorcerer has also been raised to
  475. 32,000.
  476. #259. (MR22) Default function arguments in C++ mode.
  477. If a rule is declared:
  478. rr [int i = 0] : ....
  479. then the declaration generated by pccts resembles:
  480. void rr(int i = 0);
  481. however, the definition must omit the default argument:
  482. void rr(int i) {...}
  483. In the past the default value was not omitted. In MR22
  484. the generated code resembles:
  485. void rr(int i /* = 0 */ ) {...}
  486. Implemented by Volker H. Simonis (simonis informatik.uni-tuebingen.de)
  487. Note: In MR23 this was changed so that nested C style comments
  488. ("/* ... */") would not cause problems.
  489. #258. (MR22) Using a base class for your parser
  490. In item #102 (MR10) the class statement was extended to allow one
  491. to specify a base class other than ANTLRParser for the generated
  492. parser. It turned out that this was less than useful because
  493. the constructor still specified ANTLRParser as the base class.
  494. The class statement now uses the first identifier appearing after
  495. the ":" as the name of the base class. For example:
  496. class MyParser : public FooParser {
  497. Generates in MyParser.h:
  498. class MyParser : public FooParser {
  499. Generates in MyParser.cpp something that resembles:
  500. MyParser::MyParser(ANTLRTokenBuffer *input) :
  501. FooParser(input,1,0,0,4)
  502. {
  503. token_tbl = _token_tbl;
  504. traceOptionValueDefault=1; // MR10 turn trace ON
  505. }
  506. The base class constructor must have a signature similar to
  507. that of ANTLRParser.
  508. #257. (MR21a) Removed dlg statement that -i has no effect in C++ mode.
  509. This was incorrect.
  510. #256. (MR21a) Malformed syntax graph causes crash after error message.
  511. In the past, certain kinds of errors in the very first grammar
  512. element could cause the construction of a malformed graph
  513. representing the grammar. This would eventually result in a
  514. fatal internal error. The code has been changed to be more
  515. resistant to this particular error.
  516. #255. (MR21a) ParserBlackBox(FILE* f)
  517. This constructor set openByBlackBox to the wrong value.
  518. Reported by Kees Bakker (kees_bakker tasking.nl).
  519. #254. (MR21a) Reporting syntax error at end-of-file
  520. When there was a syntax error at the end-of-file the syntax
  521. error routine would substitute "<eof>" for the programmer's
  522. end-of-file symbol. This substitution is now done only when
  523. the programmer does not define his own end-of-file symbol
  524. or the symbol begins with the character "@".
  525. Reported by Kees Bakker (kees_bakker tasking.nl).
  526. #253. (MR21) Generation of block preamble (-preamble and -preamble_first)
  527. *** This change was rescinded by item #263 ***
  528. The antlr option -preamble causes antlr to insert the code
  529. BLOCK_PREAMBLE at the start of each rule and block. It does
  530. not insert code before rules references, token references, or
  531. actions. By properly defining the macro BLOCK_PREAMBLE the
  532. user can generate code which is specific to the start of blocks.
  533. The antlr option -preamble_first is similar, but inserts the
  534. code BLOCK_PREAMBLE_FIRST(PreambleFirst_123) where the symbol
  535. PreambleFirst_123 is equivalent to the first set defined by
  536. the #FirstSetSymbol described in Item #248.
  537. I have not investigated how these options interact with guess
  538. mode (syntactic predicates).
  539. #252. (MR21) Check for null pointer in trace routine
  540. When some trace options are used when the parser is generated
  541. without the trace enabled, the current rule name may be a
  542. NULL pointer. A guard was added to check for this in
  543. restoreState.
  544. Reported by Douglas E. Forester (dougf projtech.com).
  545. #251. (MR21) Changes to #define zzTRACE_RULES
  546. The macro zzTRACE_RULES was being use to pass information to
  547. AParser.h. If this preprocessor symbol was not properly
  548. set the first time AParser.h was #included, the declaration
  549. of zzTRACEdata would be omitted (it is used by the -gd option).
  550. Subsequent #includes of AParser.h would be skipped because of
  551. the #ifdef guard, so the declaration of zzTracePrevRuleName would
  552. never be made. The result was that proper compilation was very
  553. order dependent.
  554. The declaration of zzTRACEdata was made unconditional and the
  555. problem of removing unused declarations will be left to optimizers.
  556. Diagnosed by Douglas E. Forester (dougf projtech.com).
  557. #250. (MR21) Option for EXPERIMENTAL change to error sets for blocks
  558. The antlr option -mrblkerr turns on an experimental feature
  559. which is supposed to provide more accurate syntax error messages
  560. for k=1, ck=1 grammars. When used with k>1 or ck>1 grammars the
  561. behavior should be no worse than the current behavior.
  562. There is no problem with the matching of elements or the computation
  563. of prediction expressions in pccts. The task is only one of listing
  564. the most appropriate tokens in the error message. The error sets used
  565. in pccts error messages are approximations of the exact error set when
  566. optional elements in (...)* or (...)+ are involved. While entirely
  567. correct, the error messages are sometimes not 100% accurate.
  568. There is also a minor philosophical issue. For example, suppose the
  569. grammar expects the token to be an optional A followed by Z, and it
  570. is X. X, of course, is neither A nor Z, so an error message is appropriate.
  571. Is it appropriate to say "Expected Z" ? It is correct, it is accurate,
  572. but it is not complete.
  573. When k>1 or ck>1 the problem of providing the exactly correct
  574. list of tokens for the syntax error messages ends up becoming
  575. equivalent to evaluating the prediction expression for the
  576. alternatives twice. However, for k=1 ck=1 grammars the prediction
  577. expression can be computed easily and evaluated cheaply, so I
  578. decided to try implementing it to satisfy a particular application.
  579. This application uses the error set in an interactive command language
  580. to provide prompts which list the alternatives available at that
  581. point in the parser. The user can then enter additional tokens to
  582. complete the command line. To do this required more accurate error
  583. sets then previously provided by pccts.
  584. In some cases the default pccts behavior may lead to more robust error
  585. recovery or clearer error messages then having the exact set of tokens.
  586. This is because (a) features like -ge allow the use of symbolic names for
  587. certain sets of tokens, so having extra tokens may simply obscure things
  588. and (b) the error set is use to resynchronize the parser, so a good
  589. choice is sometimes more important than having the exact set.
  590. Consider the following example:
  591. Note: All examples code has been abbreviated
  592. to the absolute minimum in order to make the
  593. examples concise.
  594. star1 : (A)* Z;
  595. The generated code resembles:
  596. old new (with -mrblkerr)
  597. --//----------- --------------------
  598. for (;;) { for (;;) {
  599. match(A); match(A);
  600. } }
  601. match(Z); if (! A and ! Z) then
  602. FAIL(...{A,Z}...);
  603. }
  604. match(Z);
  605. With input X
  606. old message: Found X, expected Z
  607. new message: Found X, expected A, Z
  608. For the example:
  609. star2 : (A|B)* Z;
  610. old new (with -mrblkerr)
  611. ------------- --------------------
  612. for (;;) { for (;;) {
  613. if (!A and !B) break; if (!A and !B) break;
  614. if (...) { if (...) {
  615. <same ...> <same ...>
  616. } }
  617. else { else {
  618. FAIL(...{A,B,Z}...) FAIL(...{A,B}...);
  619. } }
  620. } }
  621. match(B); if (! A and ! B and !Z) then
  622. FAIL(...{A,B,Z}...);
  623. }
  624. match(B);
  625. With input X
  626. old message: Found X, expected Z
  627. new message: Found X, expected A, B, Z
  628. With input A X
  629. old message: Found X, expected Z
  630. new message: Found X, expected A, B, Z
  631. This includes the choice of looping back to the
  632. star block.
  633. The code for plus blocks:
  634. plus1 : (A)+ Z;
  635. The generated code resembles:
  636. old new (with -mrblkerr)
  637. ------------- --------------------
  638. do { do {
  639. match(A); match(A);
  640. } while (A) } while (A)
  641. match(Z); if (! A and ! Z) then
  642. FAIL(...{A,Z}...);
  643. }
  644. match(Z);
  645. With input A X
  646. old message: Found X, expected Z
  647. new message: Found X, expected A, Z
  648. This includes the choice of looping back to the
  649. plus block.
  650. For the example:
  651. plus2 : (A|B)+ Z;
  652. old new (with -mrblkerr)
  653. ------------- --------------------
  654. do { do {
  655. if (A) { <same>
  656. match(A); <same>
  657. } else if (B) { <same>
  658. match(B); <same>
  659. } else { <same>
  660. if (cnt > 1) break; <same>
  661. FAIL(...{A,B,Z}...) FAIL(...{A,B}...);
  662. } }
  663. cnt++; <same>
  664. } }
  665. match(Z); if (! A and ! B and !Z) then
  666. FAIL(...{A,B,Z}...);
  667. }
  668. match(B);
  669. With input X
  670. old message: Found X, expected A, B, Z
  671. new message: Found X, expected A, B
  672. With input A X
  673. old message: Found X, expected Z
  674. new message: Found X, expected A, B, Z
  675. This includes the choice of looping back to the
  676. star block.
  677. #249. (MR21) Changes for DEC/VMS systems
  678. Jean-François Piéronne (jfp altavista.net) has updated some
  679. VMS related command files and fixed some minor problems related
  680. to building pccts under the DEC/VMS operating system. For DEC/VMS
  681. users the most important differences are:
  682. a. Revised makefile.vms
  683. b. Revised genMMS for genrating VMS style makefiles.
  684. #248. (MR21) Generate symbol for first set of an alternative
  685. pccts can generate a symbol which represents the tokens which may
  686. appear at the start of a block:
  687. rr : #FirstSetSymbol(rr_FirstSet) ( Foo | Bar ) ;
  688. This will generate the symbol rr_FirstSet of type SetWordType with
  689. elements Foo and Bar set. The bits can be tested using code similar
  690. to the following:
  691. if (set_el(Foo, &rr_FirstSet)) { ...
  692. This can be combined with the C array zztokens[] or the C++ routine
  693. tokenName() to get the print name of the token in the first set.
  694. The size of the set is given by the newly added enum SET_SIZE, a
  695. protected member of the generated parser's class. The number of
  696. elements in the generated set will not be exactly equal to the
  697. value of SET_SIZE because of synthetic tokens created by #tokclass,
  698. #errclass, the -ge option, and meta-tokens such as epsilon, and
  699. end-of-file.
  700. The #FirstSetSymbol must appear immediately before a block
  701. such as (...)+, (...)*, and {...}, and (...). It may not appear
  702. immediately before a token, a rule reference, or action. However
  703. a token or rule reference can be enclosed in a (...) in order to
  704. make the use of #pragma FirstSetSymbol legal.
  705. rr_bad : #FirstSetSymbol(rr_bad_FirstSet) Foo; // Illegal
  706. rr_ok : #FirstSetSymbol(rr_ok_FirstSet) (Foo); // Legal
  707. Do not confuse FirstSetSymbol sets with the sets used for testing
  708. lookahead. The sets used for FirstSetSymbol have one element per bit,
  709. so the number of bytes is approximately the largest token number
  710. divided by 8. The sets used for testing lookahead store 8 lookahead
  711. sets per byte, so the length of the array is approximately the largest
  712. token number.
  713. If there is demand, a similar routine for follow sets can be added.
  714. #247. (MR21) Misleading error message on syntax error for optional elements.
  715. ===================================================
  716. The behavior has been revised when parser exception
  717. handling is used. See Item #290
  718. ===================================================
  719. Prior to MR21, tokens which were optional did not appear in syntax
  720. error messages if the block which immediately followed detected a
  721. syntax error.
  722. Consider the following grammar which accepts Number, Word, and Other:
  723. rr : {Number} Word;
  724. For this rule the code resembles:
  725. if (LA(1) == Number) {
  726. match(Number);
  727. consume();
  728. }
  729. match(Word);
  730. Prior to MR21, the error message for input "$ a" would be:
  731. line 1: syntax error at "$" missing Word
  732. With MR21 the message will be:
  733. line 1: syntax error at "$" expecting Word, Number.
  734. The generate code resembles:
  735. if ( (LA(1)==Number) ) {
  736. zzmatch(Number);
  737. consume();
  738. }
  739. else {
  740. if ( (LA(1)==Word) ) {
  741. /* nothing */
  742. }
  743. else {
  744. FAIL(... message for both Number and Word ...);
  745. }
  746. }
  747. match(Word);
  748. The code generated for optional blocks in MR21 is slightly longer
  749. than the previous versions, but it should give better error messages.
  750. The code generated for:
  751. { a | b | c }
  752. should now be *identical* to:
  753. ( a | b | c | )
  754. which was not the case prior to MR21.
  755. Reported by Sue Marvin (sue siara.com).
  756. #246. (Changed in MR21) Use of $(MAKE) for calls to make
  757. Calls to make from the makefiles were replaced with $(MAKE)
  758. because of problems when using gmake.
  759. Reported with fix by Sunil K.Vallamkonda (sunil siara.com).
  760. #245. (Changed in MR21) Changes to genmk
  761. The following command line options have been added to genmk:
  762. -cfiles ...
  763. To add a user's C or C++ files into makefile automatically.
  764. The list of files must be enclosed in apostrophes. This
  765. option may be specified multiple times.
  766. -compiler ...
  767. The name of the compiler to use for $(CCC) or $(CC). The
  768. default in C++ mode is "CC". The default in C mode is "cc".
  769. -pccts_path ...
  770. The value for $(PCCTS), the pccts directory. The default
  771. is /usr/local/pccts.
  772. Contributed by Tomasz Babczynski (t.babczynski ict.pwr.wroc.pl).
  773. #244. (Changed in MR21) Rename variable "not" in antlr.g
  774. When antlr.g is compiled with a C++ compiler, a variable named
  775. "not" causes problems. Reported by Sinan Karasu
  776. (sinan.karasu boeing.com).
  777. #243 (Changed in MR21) Replace recursion with iteration in zzfree_ast
  778. Another refinement to zzfree_ast in ast.c to limit recursion.
  779. NAKAJIMA Mutsuki (muc isr.co.jp).
  780. #242. (Changed in MR21) LineInfoFormatStr
  781. Added an #ifndef/#endif around LineInfoFormatStr in pcctscfg.h.
  782. #241. (Changed in MR21) Changed macro PURIFY to a no-op
  783. ***********************
  784. *** NOT IMPLEMENTED ***
  785. ***********************
  786. The PURIFY macro was changed to a no-op because it was causing
  787. problems when passing C++ objects.
  788. The old definition:
  789. #define PURIFY(r,s) memset((char *) &(r),'\\0',(s));
  790. The new definition:
  791. #define PURIFY(r,s) /* nothing */
  792. #endif
  793. #240. (Changed in MR21) sorcerer/h/sorcerer.h _MATCH and _MATCHRANGE
  794. Added test for NULL token pointer.
  795. Suggested by Peter Keller (keller ebi.ac.uk)
  796. #239. (Changed in MR21) C++ mode AParser::traceGuessFail
  797. If tracing is turned on when the code has been generated
  798. without trace code, a failed guess generates a trace report
  799. even though there are no other trace reports. This
  800. make the behavior consistent with other parts of the
  801. trace system.
  802. Reported by David Wigg (wiggjd sbu.ac.uk).
  803. #238. (Changed in MR21) Namespace version #include files
  804. Changed reference from CStdio to cstdio (and other
  805. #include file names) in the namespace version of pccts.
  806. Should have known better.
  807. #237. (Changed in MR21) ParserBlackBox(FILE*)
  808. In the past, ParserBlackBox would close the FILE in the dtor
  809. even though it was not opened by ParserBlackBox. The problem
  810. is that there were two constructors, one which accepted a file
  811. name and did an fopen, the other which accepted a FILE and did
  812. not do an fopen. There is now an extra member variable which
  813. remembers whether ParserBlackBox did the open or not.
  814. Suggested by Mike Percy (mpercy scires.com).
  815. #236. (Changed in MR21) tmake now reports down pointer problem
  816. When ASTBase::tmake attempts to update the down pointer of
  817. an AST it checks to see if the down pointer is NULL. If it
  818. is not NULL it does not do the update and returns NULL.
  819. An attempt to update the down pointer is almost always a
  820. result of a user error. This can lead to difficult to find
  821. problems during tree construction.
  822. With this change, the routine calls a virtual function
  823. reportOverwriteOfDownPointer() which calls panic to
  824. report the problem. Users who want the old behavior can
  825. redefined the virtual function in their AST class.
  826. Suggested by Sinan Karasu (sinan.karasu boeing.com)
  827. #235. (Changed in MR21) Made ANTLRParser::resynch() virtual
  828. Suggested by Jerry Evans (jerry swsl.co.uk).
  829. #234. (Changed in MR21) Implicit int for function return value
  830. ATokenBuffer:bufferSize() did not specify a type for the
  831. return value.
  832. Reported by Hai Vo-Ba (hai fc.hp.com).
  833. #233. (Changed in MR20) Converted to MSVC 6.0
  834. Due to external circumstances I have had to convert to MSVC 6.0
  835. The MSVC 5.0 project files (.dsw and .dsp) have been retained as
  836. xxx50.dsp and xxx50.dsw. The MSVC 6.0 files are named xxx60.dsp
  837. and xxx60.dsw (where xxx is the related to the directory/project).
  838. #232. (Changed in MR20) Make setwd bit vectors protected in parser.h
  839. The access for the setwd array in the parser header was not
  840. specified. As a result, it would depend on the code which
  841. preceded it. In MR20 it will always have access "protected".
  842. Reported by Piotr Eljasiak (eljasiak zt.gdansk.tpsa.pl).
  843. #231. (Changed in MR20) Error in token buffer debug code.
  844. When token buffer debugging is selected via the pre-processor
  845. symbol DEBUG_TOKENBUFFER there is an erroneous check in
  846. AParser.cpp:
  847. #ifdef DEBUG_TOKENBUFFER
  848. if (i >= inputTokens->bufferSize() ||
  849. inputTokens->minTokens() < LLk ) /* MR20 Was "<=" */
  850. ...
  851. #endif
  852. Reported by David Wigg (wiggjd sbu.ac.uk).
  853. #230. (Changed in MR20) Fixed problem with #define for -gd option
  854. There was an error in setting zzTRACE_RULES for the -gd (trace) option.
  855. Reported by Gary Funck (gary intrepid.com).
  856. #229. (Changed in MR20) Additional "const" for literals
  857. "const" was added to the token name literal table.
  858. "const" was added to some panic() and similar routine
  859. #228. (Changed in MR20) dlg crashes on "()"
  860. The following token definition will cause DLG to crash.
  861. #token "()"
  862. When there is a syntax error in a regular expression
  863. many of the dlg routines return a structure which has
  864. null pointers. When this is accessed by callers it
  865. generates the crash.
  866. I have attempted to fix the more common cases.
  867. Reported by Mengue Olivier (dolmen bigfoot.com).
  868. #227. (Changed in MR20) Array overwrite
  869. Steveh Hand (sassth unx.sas.com) reported a problem which
  870. was traced to a temporary array which was not properly
  871. resized for deeply nested blocks. This has been fixed.
  872. #226. (Changed in MR20) -pedantic conformance
  873. G. Hobbelt (i_a mbh.org) and THM made many, many minor
  874. changes to create prototypes for all the functions and
  875. bring antlr, dlg, and sorcerer into conformance with
  876. the gcc -pedantic option.
  877. This may require uses to add pccts/h/pcctscfg.h to some
  878. files or makefiles in order to have __USE_PROTOS defined.
  879. #225 (Changed in MR20) AST stack adjustment in C mode
  880. The fix in #214 for AST stack adjustment in C mode missed
  881. some cases.
  882. Reported with fix by Ger Hobbelt (i_a mbh.org).
  883. #224 (Changed in MR20) LL(1) and LL(2) with #pragma approx
  884. This may take a record for the oldest, most trival, lexical
  885. error in pccts. The regular expressions for LL(1) and LL(2)
  886. lacked an escape for the left and right parenthesis.
  887. Reported by Ger Hobbelt (i_a mbh.org).
  888. #223 (Changed in MR20) Addition of IBM_VISUAL_AGE directory
  889. Build files for antlr, dlg, and sorcerer under IBM Visual Age
  890. have been contributed by Anton Sergeev (ags mlc.ru). They have
  891. been placed in the pccts/IBM_VISUAL_AGE directory.
  892. #222 (Changed in MR20) Replace __STDC__ with __USE_PROTOS
  893. Most occurrences of __STDC__ replaced with __USE_PROTOS due to
  894. complaints from several users.
  895. #221 (Changed in MR20) Added #include for DLexerBase.h to PBlackBox.
  896. Added #include for DLexerBase.h to PBlackBox.
  897. #220 (Changed in MR19) strcat arguments reversed in #pred parse
  898. The arguments to strcat are reversed when creating a print
  899. name for a hash table entry for use with #pred feature.
  900. Problem diagnosed and fix reported by Scott Harrington
  901. (seh4 ix.netcom.com).
  902. #219. (Changed in MR19) C Mode routine zzfree_ast
  903. Changes to reduce use of recursion for AST trees with only right
  904. links or only left links in the C mode routine zzfree_ast.
  905. Implemented by SAKAI Kiyotaka (ksakai isr.co.jp).
  906. #218. (Changed in MR19) Changes to support unsigned char in C mode
  907. Changes to antlr.h and err.h to fix omissions in use of zzchar_t
  908. Implemented by SAKAI Kiyotaka (ksakai isr.co.jp).
  909. #217. (Changed in MR19) Error message when dlg -i and -CC options selected
  910. *** This change was rescinded by item #257 ***
  911. The parsers generated by pccts in C++ mode are not able to support the
  912. interactive lexer option (except, perhaps, when using the deferred fetch
  913. parser option.(Item #216).
  914. DLG now warns when both -i and -CC are selected.
  915. This warning was suggested by David Venditti (07751870267-0001 t-online.de).
  916. #216. (Changed in MR19) Defer token fetch for C++ mode
  917. Implemented by Volker H. Simonis (simonis informatik.uni-tuebingen.de)
  918. Normally, pccts keeps the lookahead token buffer completely filled.
  919. This requires max(k,ck) tokens of lookahead. For some applications
  920. this can cause deadlock problems. For example, there may be cases
  921. when the parser can't tell when the input has been completely consumed
  922. until the parse is complete, but the parse can't be completed because
  923. the input routines are waiting for additional tokens to fill the
  924. lookahead buffer.
  925. When the ANTLRParser class is built with the pre-processor option
  926. ZZDEFER_FETCH defined, the fetch of new tokens by consume() is deferred
  927. until LA(i) or LT(i) is called.
  928. To test whether this option has been built into the ANTLRParser class
  929. use "isDeferFetchEnabled()".
  930. Using the -gd trace option with the default tracein() and traceout()
  931. routines will defeat the effort to defer the fetch because the
  932. trace routines print out information about the lookahead token at
  933. the start of the rule.
  934. Because the tracein and traceout routines are virtual it is
  935. easy to redefine them in your parser:
  936. class MyParser {
  937. <<
  938. virtual void tracein(ANTLRChar * ruleName)
  939. { fprintf(stderr,"Entering: %s\n", ruleName); }
  940. virtual void traceout(ANTLRChar * ruleName)
  941. { fprintf(stderr,"Leaving: %s\n", ruleName); }
  942. >>
  943. The originals for those routines are pccts/h/AParser.cpp
  944. This requires use of the dlg option -i (interactive lexer).
  945. This is implemented only for C++ mode.
  946. This is experimental. The interaction with guess mode (syntactic
  947. predicates)is not known.
  948. #215. (Changed in MR19) Addition of reset() to DLGLexerBase
  949. There was no obvious way to reset the lexer for reuse. The
  950. reset() method now does this.
  951. Suggested by David Venditti (07751870267-0001 t-online.de).
  952. #214. (Changed in MR19) C mode: Adjust AST stack pointer at exit
  953. In C mode the AST stack pointer needs to be reset if there will
  954. be multiple calls to the ANTLRx macros.
  955. Reported with fix by Paul D. Smith (psmith baynetworks.com).
  956. #213. (Changed in MR18) Fatal error with -mrhoistk (k>1 hoisting)
  957. When rearranging code I forgot to un-comment a critical line of
  958. code that handles hoisting of predicates with k>1 lookahead. This
  959. is now fixed.
  960. Reported by Reinier van den Born (reinier vnet.ibm.com).
  961. #212. (Changed in MR17) Mac related changes by Kenji Tanaka
  962. Kenji Tanaka (kentar osa.att.ne.jp) has made a number of changes for
  963. Macintosh users.
  964. a. The following Macintosh MPW files aid in installing pccts on Mac:
  965. pccts/MPW_Read_Me
  966. pccts/install68K.mpw
  967. pccts/installPPC.mpw
  968. pccts/antlr/antlr.r
  969. pccts/antlr/antlr68K.make
  970. pccts/antlr/antlrPPC.make
  971. pccts/dlg/dlg.r
  972. pccts/dlg/dlg68K.make
  973. pccts/dlg/dlgPPC.make
  974. pccts/sorcerer/sor.r
  975. pccts/sorcerer/sor68K.make
  976. pccts/sorcerer/sorPPC.make
  977. They completely replace the previous Mac installation files.
  978. b. The most significant is a change in the MAC_FILE_CREATOR symbol
  979. in pcctscfg.h:
  980. old: #define MAC_FILE_CREATOR 'MMCC' /* Metrowerks C/C++ Text files */
  981. new: #define MAC_FILE_CREATOR 'CWIE' /* Metrowerks C/C++ Text files */
  982. c. Added calls to special_fopen_actions() where necessary.
  983. #211. (Changed in MR16a) C++ style comment in dlg
  984. This has been fixed.
  985. #210. (Changed in MR16a) Sor accepts \r\n, \r, or \n for end-of-line
  986. A user requested that Sorcerer be changed to accept other forms
  987. of end-of-line.
  988. #209. (Changed in MR16) Name of files changed.
  989. Old: CHANGES_FROM_1.33
  990. New: CHANGES_FROM_133.txt
  991. Old: KNOWN_PROBLEMS
  992. New: KNOWN_PROBLEMS.txt
  993. #208. (Changed in MR16) Change in use of pccts #include files
  994. There were problems with MS DevStudio when mixing Sorcerer and
  995. PCCTS in the same source file. The problem is caused by the
  996. redefinition of setjmp in the MS header file setjmp.h. In
  997. setjmp.h the pre-processor symbol setjmp was redefined to be
  998. _setjmp. A later effort to execute #include <setjmp.h> resulted
  999. in an effort to #include <_setjmp.h>. I'm not sure whether this
  1000. is a bug or a feature. In any case, I decided to fix it by
  1001. avoiding the use of pre-processor symbols in #include statements
  1002. altogether. This has the added benefit of making pre-compiled
  1003. headers work again.
  1004. I've replaced statements:
  1005. old: #include PCCTS_SETJMP_H
  1006. new: #include "pccts_setjmp.h"
  1007. Where pccts_setjmp.h contains:
  1008. #ifndef __PCCTS_SETJMP_H__
  1009. #define __PCCTS_SETJMP_H__
  1010. #ifdef PCCTS_USE_NAMESPACE_STD
  1011. #include <Csetjmp>
  1012. #else
  1013. #include <setjmp.h>
  1014. #endif
  1015. #endif
  1016. A similar change has been made for other standard header files
  1017. required by pccts and sorcerer: stdlib.h, stdarg.h, stdio.h, etc.
  1018. Reported by Jeff Vincent (JVincent novell.com) and Dale Davis
  1019. (DalDavis spectrace.com).
  1020. #207. (Changed in MR16) dlg reports an invalid range for: [\0x00-\0xff]
  1021. -----------------------------------------------------------------
  1022. Note from MR23: This fix does not work. I am investigating why.
  1023. -----------------------------------------------------------------
  1024. dlg will report that this is an invalid range.
  1025. Diagnosed by Piotr Eljasiak (eljasiak no-spam.zt.gdansk.tpsa.pl):
  1026. I think this problem is not specific to unsigned chars
  1027. because dlg reports no error for the range [\0x00-\0xfe].
  1028. I've found that information on range is kept in field
  1029. letter (unsigned char) of Attrib struct. Unfortunately
  1030. the letter value internally is for some reasons increased
  1031. by 1, so \0xff is represented here as 0.
  1032. That's why dlg complains about the range [\0x00-\0xff] in
  1033. dlg_p.g:
  1034. if ($$.letter > $2.letter) {
  1035. error("invalid range ", zzline);
  1036. }
  1037. The fix is:
  1038. if ($$.letter > $2.letter && 255 != $$2.letter) {
  1039. error("invalid range ", zzline);
  1040. }
  1041. #206. (Changed in MR16) Free zzFAILtext in ANTLRParser destructor
  1042. The ANTLRParser destructor now frees zzFAILtext.
  1043. Problem and fix reported by Manfred Kogler (km cast.uni-linz.ac.at).
  1044. #205. (Changed in MR16) DLGStringReset argument now const
  1045. Changed: void DLGStringReset(DLGChar *s) {...}
  1046. To: void DLGStringReset(const DLGChar *s) {...}
  1047. Suggested by Dale Davis (daldavis spectrace.com)
  1048. #204. (Changed in MR15a) Change __WATCOM__ to __WATCOMC__ in pcctscfg.h
  1049. Reported by Oleg Dashevskii (olegdash my-dejanews.com).
  1050. #203. (Changed in MR15) Addition of sorcerer to distribution kit
  1051. I have finally caved in to popular demand. The pccts 1.33mr15
  1052. kit will include sorcerer. The separate sorcerer kit will be
  1053. discontinued.
  1054. #202. (Changed) in MR15) Organization of MS Dev Studio Projects in Kit
  1055. Previously there was one workspace that contained projects for
  1056. all three parts of pccts: antlr, dlg, and sorcerer. Now each
  1057. part (and directory) has its own workspace/project and there
  1058. is an additional workspace/project to build a library from the
  1059. .cpp files in the pccts/h directory.
  1060. The library build will create pccts_debug.lib or pccts_release.lib
  1061. according to the configuration selected.
  1062. If you don't want to build pccts 1.33MR15 you can download a
  1063. ready-to-run kit for win32 from http://www.polhode.com/win32.zip.
  1064. The ready-to-run for win32 includes executables, a pre-built static
  1065. library for the .cpp files in the pccts/h directory, and a sample
  1066. application
  1067. You will need to define the environment variable PCCTS to point to
  1068. the root of the pccts directory hierarchy.
  1069. #201. (Changed in MR15) Several fixes by K.J. Cummings (cummings peritus.com)
  1070. Generation of SETJMP rather than SETJMP_H in gen.c.
  1071. (Sor B19) Declaration of ref_vars_inits for ref_var_inits in
  1072. pccts/sorcerer/sorcerer.h.
  1073. #200. (Changed in MR15) Remove operator=() in AToken.h
  1074. User reported that WatCom couldn't handle use of
  1075. explicit operator =(). Replace with equivalent
  1076. using cast operator.
  1077. #199. (Changed in MR15) Don't allow use of empty #tokclass
  1078. Change antlr.g to disallow empty #tokclass sets.
  1079. Reported by Manfred Kogler (km cast.uni-linz.ac.at).
  1080. #198. Revised ANSI C grammar due to efforts by Manuel Kessler
  1081. Manuel Kessler (mlkessler cip.physik.uni-wuerzburg.de)
  1082. Allow trailing ... in function parameter lists.
  1083. Add bit fields.
  1084. Allow old-style function declarations.
  1085. Support cv-qualified pointers.
  1086. Better checking of combinations of type specifiers.
  1087. Release of memory for local symbols on scope exit.
  1088. Allow input file name on command line as well as by redirection.
  1089. and other miscellaneous tweaks.
  1090. This is not part of the pccts distribution kit. It must be
  1091. downloaded separately from:
  1092. http://www.polhode.com/ansi_mr15.zip
  1093. #197. (Changed in MR14) Resetting the lookahead buffer of the parser
  1094. Explanation and fix by Sinan Karasu (sinan.karasu boeing.com)
  1095. Consider the code used to prime the lookahead buffer LA(i)
  1096. of the parser when init() is called:
  1097. void
  1098. ANTLRParser::
  1099. prime_lookahead()
  1100. {
  1101. int i;
  1102. for(i=1;i<=LLk; i++) consume();
  1103. dirty=0;
  1104. //lap = 0; // MR14 - Sinan Karasu (sinan.karusu boeing.com)
  1105. //labase = 0; // MR14
  1106. labase=lap; // MR14
  1107. }
  1108. When the parser is instantiated, lap=0,labase=0 is set.
  1109. The "for" loop runs LLk times. In consume(), lap = lap +1 (mod LLk) is
  1110. computed. Therefore, lap(before the loop) == lap (after the loop).
  1111. Now the only problem comes in when one does an init() of the parser
  1112. after an Eof has been seen. At that time, lap could be non zero.
  1113. Assume it was lap==1. Now we do a prime_lookahead(). If LLk is 2,
  1114. then
  1115. consume()
  1116. {
  1117. NLA = inputTokens->getToken()->getType();
  1118. dirty--;
  1119. lap = (lap+1)&(LLk-1);
  1120. }
  1121. or expanding NLA,
  1122. token_type[lap&(LLk-1)]) = inputTokens->getToken()->getType();
  1123. dirty--;
  1124. lap = (lap+1)&(LLk-1);
  1125. so now we prime locations 1 and 2. In prime_lookahead it used to set
  1126. lap=0 and labase=0. Now, the next token will be read from location 0,
  1127. NOT 1 as it should have been.
  1128. This was never caught before, because if a parser is just instantiated,
  1129. then lap and labase are 0, the offending assignment lines are
  1130. basically no-ops, since the for loop wraps around back to 0.
  1131. #196. (Changed in MR14) Problems with "(alpha)? beta" guess
  1132. Consider the following syntactic predicate in a grammar
  1133. with 2 tokens of lookahead (k=2 or ck=2):
  1134. rule : ( alpha )? beta ;
  1135. alpha : S t ;
  1136. t : T U
  1137. | T
  1138. ;
  1139. beta : S t Z ;
  1140. When antlr computes the prediction expression with one token
  1141. of lookahead for alts 1 and 2 of rule t it finds an ambiguity.
  1142. Because the grammar has a lookahead of 2 it tries to compute
  1143. two tokens of lookahead for alts 1 and 2 of t. Alt 1 clearly
  1144. has a lookahead of (T U). Alt 2 is one token long so antlr
  1145. tries to compute the follow set of alt 2, which means finding
  1146. the things which can follow rule t in the context of (alpha)?.
  1147. This cannot be computed, because alpha is only part of a rule,
  1148. and antlr can't tell what part of beta is matched by alpha and
  1149. what part remains to be matched. Thus it impossible for antlr
  1150. to properly determine the follow set of rule t.
  1151. Prior to 1.33MR14, the follow of (alpha)? was computed as
  1152. FIRST(beta) as a result of the internal representation of
  1153. guess blocks.
  1154. With MR14 the follow set will be the empty set for that context.
  1155. Normally, one expects a rule appearing in a guess block to also
  1156. appear elsewhere. When the follow context for this other use
  1157. is "ored" with the empty set, the context from the other use
  1158. results, and a reasonable follow context results. However if
  1159. there is *no* other use of the rule, or it is used in a different
  1160. manner then the follow context will be inaccurate - it was
  1161. inaccurate even before MR14, but it will be inaccurate in a
  1162. different way.
  1163. For the example given earlier, a reasonable way to rewrite the
  1164. grammar:
  1165. rule : ( alpha )? beta
  1166. alpha : S t ;
  1167. t : T U
  1168. | T
  1169. ;
  1170. beta : alpha Z ;
  1171. If there are no other uses of the rule appearing in the guess
  1172. block it will generate a test for EOF - a workaround for
  1173. representing a null set in the lookahead tests.
  1174. If you encounter such a problem you can use the -alpha option
  1175. to get additional information:
  1176. line 2: error: not possible to compute follow set for alpha
  1177. in an "(alpha)? beta" block.
  1178. With the antlr -alpha command line option the following information
  1179. is inserted into the generated file:
  1180. #if 0
  1181. Trace of references leading to attempt to compute the follow set of
  1182. alpha in an "(alpha)? beta" block. It is not possible for antlr to
  1183. compute this follow set because it is not known what part of beta has
  1184. already been matched by alpha and what part remains to be matched.
  1185. Rules which make use of the incorrect follow set will also be incorrect
  1186. 1 #token T alpha/2 line 7 brief.g
  1187. 2 end alpha alpha/3 line 8 brief.g
  1188. 2 end (...)? block at start/1 line 2 brief.g
  1189. #endif
  1190. At the moment, with the -alpha option selected the program marks
  1191. any rules which appear in the trace back chain (above) as rules with
  1192. possible problems computing follow set.
  1193. Reported by Greg Knapen (gregory.knapen bell.ca).
  1194. #195. (Changed in MR14) #line directive not at column 1
  1195. Under certain circumstances a predicate test could generate
  1196. a #line directive which was not at column 1.
  1197. Reported with fix by David Kågedal (davidk lysator.liu.se)
  1198. (http://www.lysator.liu.se/~davidk/).
  1199. #194. (Changed in MR14) (C Mode only) Demand lookahead with #tokclass
  1200. In C mode with the demand lookahead option there is a bug in the
  1201. code which handles matches for #tokclass (zzsetmatch and
  1202. zzsetmatch_wsig).
  1203. The bug causes the lookahead pointer to get out of synchronization
  1204. with the current token pointer.
  1205. The problem was reported with a fix by Ger Hobbelt (hobbelt axa.nl).
  1206. #193. (Changed in MR14) Use of PCCTS_USE_NAMESPACE_STD
  1207. The pcctscfg.h now contains the following definitions:
  1208. #ifdef PCCTS_USE_NAMESPACE_STD
  1209. #define PCCTS_STDIO_H <Cstdio>
  1210. #define PCCTS_STDLIB_H <Cstdlib>
  1211. #define PCCTS_STDARG_H <Cstdarg>
  1212. #define PCCTS_SETJMP_H <Csetjmp>
  1213. #define PCCTS_STRING_H <Cstring>
  1214. #define PCCTS_ASSERT_H <Cassert>
  1215. #define PCCTS_ISTREAM_H <istream>
  1216. #define PCCTS_IOSTREAM_H <iostream>
  1217. #define PCCTS_NAMESPACE_STD namespace std {}; using namespace std;
  1218. #else
  1219. #define PCCTS_STDIO_H <stdio.h>
  1220. #define PCCTS_STDLIB_H <stdlib.h>
  1221. #define PCCTS_STDARG_H <stdarg.h>
  1222. #define PCCTS_SETJMP_H <setjmp.h>
  1223. #define PCCTS_STRING_H <string.h>
  1224. #define PCCTS_ASSERT_H <assert.h>
  1225. #define PCCTS_ISTREAM_H <istream.h>
  1226. #define PCCTS_IOSTREAM_H <iostream.h>
  1227. #define PCCTS_NAMESPACE_STD
  1228. #endif
  1229. The runtime support in pccts/h uses these pre-processor symbols
  1230. consistently.
  1231. Also, antlr and dlg have been changed to generate code which uses
  1232. these pre-processor symbols rather than having the names of the
  1233. #include files hard-coded in the generated code.
  1234. This required the addition of "#include pcctscfg.h" to a number of
  1235. files in pccts/h.
  1236. It appears that this sometimes causes problems for MSVC 5 in
  1237. combination with the "automatic" option for pre-compiled headers.
  1238. In such cases disable the "automatic" pre-compiled headers option.
  1239. Suggested by Hubert Holin (Hubert.Holin Bigfoot.com).
  1240. #192. (Changed in MR14) Change setText() to accept "const ANTLRChar *"
  1241. Changed ANTLRToken::setText(ANTLRChar *) to setText(const ANTLRChar *).
  1242. This allows literal strings to be used to initialize tokens. Since
  1243. the usual token implementation (ANTLRCommonToken) makes a copy of the
  1244. input string, this was an unnecessary limitation.
  1245. Suggested by Bob McWhirter (bob netwrench.com).
  1246. #191. (Changed in MR14) HP/UX aCC compiler compatibility problem
  1247. Needed to explicitly declare zzINF_DEF_TOKEN_BUFFER_SIZE and
  1248. zzINF_BUFFER_TOKEN_CHUNK_SIZE as ints in pccts/h/AParser.cpp.
  1249. Reported by David Cook (dcook bmc.com).
  1250. #190. (Changed in MR14) IBM OS/2 CSet compiler compatibility problem
  1251. Name conflict with "_cs" in pccts/h/ATokenBuffer.cpp
  1252. Reported by David Cook (dcook bmc.com).
  1253. #189. (Changed in MR14) -gxt switch in C mode
  1254. The -gxt switch in C mode didn't work because of incorrect
  1255. initialization.
  1256. Reported by Sinan Karasu (sinan boeing.com).
  1257. #188. (Changed in MR14) Added pccts/h/DLG_stream_input.h
  1258. This is a DLG stream class based on C++ istreams.
  1259. Contributed by Hubert Holin (Hubert.Holin Bigfoot.com).
  1260. #187. (Changed in MR14) Rename config.h to pcctscfg.h
  1261. The PCCTS configuration file has been renamed from config.h to
  1262. pcctscfg.h. The problem with the original name is that it led
  1263. to name collisions when pccts parsers were combined with other
  1264. software.
  1265. All of the runtime support routines in pccts/h/* have been
  1266. changed to use the new name. Existing software can continue
  1267. to use pccts/h/config.h. The contents of pccts/h/config.h is
  1268. now just "#include "pcctscfg.h".
  1269. I don't have a record of the user who suggested this.
  1270. #186. (Changed in MR14) Pre-processor symbol DllExportPCCTS class modifier
  1271. Classes in the C++ runtime support routines are now declared:
  1272. class DllExportPCCTS className ....
  1273. By default, the pre-processor symbol is defined as the empty
  1274. string. This if for use by MSVC++ users to create DLL classes.
  1275. Suggested by Manfred Kogler (km cast.uni-linz.ac.at).
  1276. #185. (Changed in MR14) Option to not use PCCTS_AST base class for ASTBase
  1277. Normally, the ASTBase class is derived from PCCTS_AST which contains
  1278. functions useful to Sorcerer. If these are not necessary then the
  1279. user can define the pre-processor symbol "PCCTS_NOT_USING_SOR" which
  1280. will cause the ASTBase class to replace references to PCCTS_AST with
  1281. references to ASTBase where necessary.
  1282. The class ASTDoublyLinkedBase will contain a pure virtual function
  1283. shallowCopy() that was formerly defined in class PCCTS_AST.
  1284. Suggested by Bob McWhirter (bob netwrench.com).
  1285. #184. (Changed in MR14) Grammars with no tokens generate invalid tokens.h
  1286. Reported by Hubert Holin (Hubert.Holin bigfoot.com).
  1287. #183. (Changed in MR14) -f to specify file with names of grammar files
  1288. In DEC/VMS it is difficult to specify very long command lines.
  1289. The -f option allows one to place the names of the grammar files
  1290. in a data file in order to bypass limitations of the DEC/VMS
  1291. command language interpreter.
  1292. Addition supplied by Bernard Giroud (b_giroud decus.ch).
  1293. #182. (Changed in MR14) Output directory option for DEC/VMS
  1294. Fix some problems with the -o option under DEC/VMS.
  1295. Fix supplied by Bernard Giroud (b_giroud decus.ch).
  1296. #181. (Changed in MR14) Allow chars > 127 in DLGStringInput::nextChar()
  1297. Changed DLGStringInput to cast the character using (unsigned char)
  1298. so that languages with character codes greater than 127 work
  1299. without changes.
  1300. Suggested by Manfred Kogler (km cast.uni-linz.ac.at).
  1301. #180. (Added in MR14) ANTLRParser::getEofToken()
  1302. Added "ANTLRToken ANTLRParser::getEofToken() const" to match the
  1303. setEofToken routine.
  1304. Requested by Manfred Kogler (km cast.uni-linz.ac.at).
  1305. #179. (Fixed in MR14) Memory leak for BufFileInput subclass of DLGInputStream
  1306. The BufFileInput class described in Item #142 neglected to release
  1307. the allocated buffer when an instance was destroyed.
  1308. Reported by Manfred Kogler (km cast.uni-linz.ac.at).
  1309. #178. (Fixed in MR14) Bug in "(alpha)? beta" guess blocks first sets
  1310. In 1.33 vanilla, and all maintenance releases prior to MR14
  1311. there is a bug in the handling of guess blocks which use the
  1312. "long" form:
  1313. (alpha)? beta
  1314. inside a (...)*, (...)+, or {...} block.
  1315. This problem does *not* apply to the case where beta is omitted
  1316. or when the syntactic predicate is on the leading edge of an
  1317. alternative.
  1318. The problem is that both alpha and beta are stored in the
  1319. syntax diagram, and that some analysis routines would fail
  1320. to skip the alpha portion when it was not on the leading edge.
  1321. Consider the following grammar with -ck 2:
  1322. r : ( (A)? B )* C D
  1323. | A B /* forces -ck 2 computation for old antlr */
  1324. /* reports ambig for alts 1 & 2 */
  1325. | B C /* forces -ck 2 computation for new antlr */
  1326. /* reports ambig for alts 1 & 3 */
  1327. ;
  1328. The prediction expression for the first alternative should be
  1329. LA(1)={B C} LA(2)={B C D}, but previous versions of antlr
  1330. would compute the prediction expression as LA(1)={A C} LA(2)={B D}
  1331. Reported by Arpad Beszedes (beszedes inf.u-szeged.hu) who provided
  1332. a very clear example of the problem and identified the probable cause.
  1333. #177. (Changed in MR14) #tokdefs and #token with regular expression
  1334. In MR13 the change described by Item #162 caused an existing
  1335. feature of antlr to fail. Prior to the change it was possible
  1336. to give regular expression definitions and actions to tokens
  1337. which were defined via the #tokdefs directive.
  1338. This now works again.
  1339. Reported by Manfred Kogler (km cast.uni-linz.ac.at).
  1340. #176. (Changed in MR14) Support for #line in antlr source code
  1341. Note: this was implemented by Arpad Beszedes (beszedes inf.u-szeged.hu).
  1342. In 1.33MR14 it is possible for a pre-processor to generate #line
  1343. directives in the antlr source and have those line numbers and file
  1344. names used in antlr error messages and in the #line directives
  1345. generated by antlr.
  1346. The #line directive may appear in the following forms:
  1347. #line ll "sss" xx xx ...
  1348. where ll represents a line number, "sss" represents the name of a file
  1349. enclosed in quotation marks, and xxx are arbitrary integers.
  1350. The following form (without "line") is not supported at the moment:
  1351. # ll "sss" xx xx ...
  1352. The result:
  1353. zzline
  1354. is replaced with ll from the # or #line directive
  1355. FileStr[CurFile]
  1356. is updated with the contents of the string (if any)
  1357. following the line number
  1358. Note
  1359. ----
  1360. The file-name string following the line number can be a complete
  1361. name with a directory-path. Antlr generates the output files from
  1362. the input file name (by replacing the extension from the file-name
  1363. with .c or .cpp).
  1364. If the input file (or the file-name from the line-info) contains
  1365. a path:
  1366. "../grammar.g"
  1367. the generated source code will be placed in "../grammar.cpp" (i.e.
  1368. in the parent directory). This is inconvenient in some cases
  1369. (even the -o switch can not be used) so the path information is
  1370. removed from the #line directive. Thus, if the line-info was
  1371. #line 2 "../grammar.g"
  1372. then the current file-name will become "grammar.g"
  1373. In this way, the generated source code according to the grammar file
  1374. will always be in the current directory, except when the -o switch
  1375. is used.
  1376. #175. (Changed in MR14) Bug when guess block appears at start of (...)*
  1377. In 1.33 vanilla and all maintenance releases prior to 1.33MR14
  1378. there is a bug when a guess block appears at the start of a (...)+.
  1379. Consider the following k=1 (ck=1) grammar:
  1380. rule :
  1381. ( (STAR)? ZIP )* ID ;
  1382. Prior to 1.33MR14, the generated code resembled:
  1383. ...
  1384. zzGUESS_BLOCK
  1385. while ( 1 ) {
  1386. if ( ! LA(1)==STAR) break;
  1387. zzGUESS
  1388. if ( !zzrv ) {
  1389. zzmatch(STAR);
  1390. zzCONSUME;
  1391. zzGUESS_DONE
  1392. zzmatch(ZIP);
  1393. zzCONSUME;
  1394. ...
  1395. Note that the routine uses STAR for the prediction expression
  1396. rather than ZIP. With 1.33MR14 the generated code resembles:
  1397. ...
  1398. while ( 1 ) {
  1399. if ( ! LA(1)==ZIP) break;
  1400. ...
  1401. This problem existed only with (...)* blocks and was caused
  1402. by the slightly more complicated graph which represents (...)*
  1403. blocks. This caused the analysis routine to compute the first
  1404. set for the alpha part of the "(alpha)? beta" rather than the
  1405. beta part.
  1406. Both (...)+ and {...} blocks handled the guess block correctly.
  1407. Reported by Arpad Beszedes (beszedes inf.u-szeged.hu) who provided
  1408. a very clear example of the problem and identified the probable cause.
  1409. #174. (Changed in MR14) Bug when action precedes syntactic predicate
  1410. In 1.33 vanilla, and all maintenance releases prior to MR14,
  1411. there was a bug when a syntactic predicate was immediately
  1412. preceded by an action. Consider the following -ck 2 grammar:
  1413. rule :
  1414. <<int i;>>
  1415. (alpha)? beta C
  1416. | A B
  1417. ;
  1418. alpha : A ;
  1419. beta : A B;
  1420. Prior to MR14, the code generated for the first alternative
  1421. resembled:
  1422. ...
  1423. zzGUESS
  1424. if ( !zzrv && LA(1)==A && LA(2)==A) {
  1425. alpha();
  1426. zzGUESS_DONE
  1427. beta();
  1428. zzmatch(C);
  1429. zzCONSUME;
  1430. } else {
  1431. ...
  1432. The prediction expression (i.e. LA(1)==A && LA(2)==A) is clearly
  1433. wrong because LA(2) should be matched to B (first[2] of beta is {B}).
  1434. With 1.33MR14 the prediction expression is:
  1435. ...
  1436. if ( !zzrv && LA(1)==A && LA(2)==B) {
  1437. alpha();
  1438. zzGUESS_DONE
  1439. beta();
  1440. zzmatch(C);
  1441. zzCONSUME;
  1442. } else {
  1443. ...
  1444. This will only affect users in which alpha is shorter than
  1445. than max(k,ck) and there is an action immediately preceding
  1446. the syntactic predicate.
  1447. This problem was reported by reported by Arpad Beszedes
  1448. (beszedes inf.u-szeged.hu) who provided a very clear example
  1449. of the problem and identified the presence of the init-action
  1450. as the likely culprit.
  1451. #173. (Changed in MR13a) -glms for Microsoft style filenames with -gl
  1452. With the -gl option antlr generates #line directives using the
  1453. exact name of the input files specified on the command line.
  1454. An oddity of the Microsoft C and C++ compilers is that they
  1455. don't accept file names in #line directives containing "\"
  1456. even though these are names from the native file system.
  1457. With -glms option, the "\" in file names appearing in #line
  1458. directives is replaced with a "/" in order to conform to
  1459. Microsoft compiler requirements.
  1460. Reported by Erwin Achermann (erwin.achermann switzerland.org).
  1461. #172. (Changed in MR13) \r\n in antlr source counted as one line
  1462. Some MS software uses \r\n to indicate a new line. Antlr
  1463. now recognizes this in counting lines.
  1464. Reported by Edward L. Hepler (elh ece.vill.edu).
  1465. #171. (Changed in MR13) #tokclass L..U now allowed
  1466. The following is now allowed:
  1467. #tokclass ABC { A..B C }
  1468. Reported by Dave Watola (dwatola amtsun.jpl.nasa.gov)
  1469. #170. (Changed in MR13) Suppression for predicates with lookahead depth >1
  1470. In MR12 the capability for suppression of predicates with lookahead
  1471. depth=1 was introduced. With MR13 this had been extended to
  1472. predicates with lookahead depth > 1 and released for use by users
  1473. on an experimental basis.
  1474. Consider the following grammar with -ck 2 and the predicate in rule
  1475. "a" with depth 2:
  1476. r1 : (ab)* "@"
  1477. ;
  1478. ab : a
  1479. | b
  1480. ;
  1481. a : (A B)? => <<p(LATEXT(2))>>? A B C
  1482. ;
  1483. b : A B C
  1484. ;
  1485. Normally, the predicate would be hoisted into rule r1 in order to
  1486. determine whether to call rule "ab". However it should *not* be
  1487. hoisted because, even if p is false, there is a valid alternative
  1488. in rule b. With "-mrhoistk on" the predicate will be suppressed.
  1489. If "-info p" command line option is present the following information
  1490. will appear in the generated code:
  1491. while ( (LA(1)==A)
  1492. #if 0
  1493. Part (or all) of predicate with depth > 1 suppressed by alternative
  1494. without predicate
  1495. pred << p(LATEXT(2))>>?
  1496. depth=k=2 ("=>" guard) rule a line 8 t1.g
  1497. tree context:
  1498. (root = A
  1499. B
  1500. )
  1501. The token sequence which is suppressed: ( A B )
  1502. The sequence of references which generate that sequence of tokens:
  1503. 1 to ab r1/1 line 1 t1.g
  1504. 2 ab ab/1 line 4 t1.g
  1505. 3 to b ab/2 line 5 t1.g
  1506. 4 b b/1 line 11 t1.g
  1507. 5 #token A b/1 line 11 t1.g
  1508. 6 #token B b/1 line 11 t1.g
  1509. #endif
  1510. A slightly more complicated example:
  1511. r1 : (ab)* "@"
  1512. ;
  1513. ab : a
  1514. | b
  1515. ;
  1516. a : (A B)? => <<p(LATEXT(2))>>? (A B | D E)
  1517. ;
  1518. b : <<q(LATEXT(2))>>? D E
  1519. ;
  1520. In this case, the sequence (D E) in rule "a" which lies behind
  1521. the guard is used to suppress the predicate with context (D E)
  1522. in rule b.
  1523. while ( (LA(1)==A || LA(1)==D)
  1524. #if 0
  1525. Part (or all) of predicate with depth > 1 suppressed by alternative
  1526. without predicate
  1527. pred << q(LATEXT(2))>>?
  1528. depth=k=2 rule b line 11 t2.g
  1529. tree context:
  1530. (root = D
  1531. E
  1532. )
  1533. The token sequence which is suppressed: ( D E )
  1534. The sequence of references which generate that sequence of tokens:
  1535. 1 to ab r1/1 line 1 t2.g
  1536. 2 ab ab/1 line 4 t2.g
  1537. 3 to a ab/1 line 4 t2.g
  1538. 4 a a/1 line 8 t2.g
  1539. 5 #token D a/1 line 8 t2.g
  1540. 6 #token E a/1 line 8 t2.g
  1541. #endif
  1542. &&
  1543. #if 0
  1544. pred << p(LATEXT(2))>>?
  1545. depth=k=2 ("=>" guard) rule a line 8 t2.g
  1546. tree context:
  1547. (root = A
  1548. B
  1549. )
  1550. #endif
  1551. (! ( LA(1)==A && LA(2)==B ) || p(LATEXT(2)) ) {
  1552. ab();
  1553. ...
  1554. #169. (Changed in MR13) Predicate test optimization for depth=1 predicates
  1555. When the MR12 generated a test of a predicate which had depth 1
  1556. it would use the depth >1 routines, resulting in correct but
  1557. inefficient behavior. In MR13, a bit test is used.
  1558. #168. (Changed in MR13) Token expressions in context guards
  1559. The token expressions appearing in context guards such as:
  1560. (A B)? => <<test(LT(1))>>? someRule
  1561. are computed during an early phase of antlr processing. As
  1562. a result, prior to MR13, complex expressions such as:
  1563. ~B
  1564. L..U
  1565. ~L..U
  1566. TokClassName
  1567. ~TokClassName
  1568. were not computed properly. This resulted in incorrect
  1569. context being computed for such expressions.
  1570. In MR13 these context guards are verified for proper semantics
  1571. in the initial phase and then re-evaluated after complex token
  1572. expressions have been computed in order to produce the correct
  1573. behavior.
  1574. Reported by Arpad Beszedes (beszedes inf.u-szeged.hu).
  1575. #167. (Changed in MR13) ~L..U
  1576. Prior to MR13, the complement of a token range was
  1577. not properly computed.
  1578. #166. (Changed in MR13) token expression L..U
  1579. The token U was represented as an unsigned char, restricting
  1580. the use of L..U to cases where U was assigned a token number
  1581. less than 256. This is corrected in MR13.
  1582. #165. (Changed in MR13) option -newAST
  1583. To create ASTs from an ANTLRTokenPtr antlr usually calls
  1584. "new AST(ANTLRTokenPtr)". This option generates a call
  1585. to "newAST(ANTLRTokenPtr)" instead. This allows a user
  1586. to define a parser member function to create an AST object.
  1587. Similar changes for ASTBase::tmake and ASTBase::link were not
  1588. thought necessary since they do not create AST objects, only
  1589. use existing ones.
  1590. #164. (Changed in MR13) Unused variable _astp
  1591. For many compilations, we have lived with warnings about
  1592. the unused variable _astp. It turns out that this variable
  1593. can *never* be used because the code which references it was
  1594. commented out.
  1595. This investigation was sparked by a note from Erwin Achermann
  1596. (erwin.achermann switzerland.org).
  1597. #163. (Changed in MR13) Incorrect makefiles for testcpp examples
  1598. All the examples in pccts/testcpp/* had incorrect definitions
  1599. in the makefiles for the symbol "CCC". Instead of CCC=CC they
  1600. had CC=$(CCC).
  1601. There was an additional problem in testcpp/1/test.g due to the
  1602. change in ANTLRToken::getText() to a const member function
  1603. (Item #137).
  1604. Reported by Maurice Mass (maas cuci.nl).
  1605. #162. (Changed in MR13) Combining #token with #tokdefs
  1606. When it became possible to change the print-name of a
  1607. #token (Item #148) it became useful to give a #token
  1608. statement whose only purpose was to giving a print name
  1609. to the #token. Prior to this change this could not be
  1610. combined with the #tokdefs feature.
  1611. #161. (Changed in MR13) Switch -gxt inhibits generation of tokens.h
  1612. #160. (Changed in MR13) Omissions in list of names for remap.h
  1613. When a user selects the -gp option antlr creates a list
  1614. of macros in remap.h to rename some of the standard
  1615. antlr routines from zzXXX to userprefixXXX.
  1616. There were number of omissions from the remap.h name
  1617. list related to the new trace facility. This was reported,
  1618. along with a fix, by Bernie Solomon (bernard ug.eds.com).
  1619. #159. (Changed in MR13) Violations of classic C rules
  1620. There were a number of violations of classic C style in
  1621. the distribution kit. This was reported, along with fixes,
  1622. by Bernie Solomon (bernard ug.eds.com).
  1623. #158. (Changed in MR13) #header causes problem for pre-processors
  1624. A user who runs the C pre-processor on antlr source suggested
  1625. that another syntax be allowed. With MR13 such directives
  1626. such as #header, #pragma, etc. may be written as "\#header",
  1627. "\#pragma", etc. For escaping pre-processor directives inside
  1628. a #header use something like the following:
  1629. \#header
  1630. <<
  1631. \#include <stdio.h>
  1632. >>
  1633. #157. (Fixed in MR13) empty error sets for rules with infinite recursion
  1634. When the first set for a rule cannot be computed due to infinite
  1635. left recursion and it is the only alternative for a block then
  1636. the error set for the block would be empty. This would result
  1637. in a fatal error.
  1638. Reported by Darin Creason (creason genedax.com)
  1639. #156. (Changed in MR13) DLGLexerBase::getToken() now public
  1640. #155. (Changed in MR13) Context behind predicates can suppress
  1641. With -mrhoist enabled the context behind a guarded predicate can
  1642. be used to suppress other predicates. Consider the following grammar:
  1643. r0 : (r1)+;
  1644. r1 : rp
  1645. | rq
  1646. ;
  1647. rp : <<p LATEXT(1)>>? B ;
  1648. rq : (A)? => <<q LATEXT(1)>>? (A|B);
  1649. In earlier versions both predicates "p" and "q" would be hoisted into
  1650. rule r0. With MR12c predicate p is suppressed because the context which
  1651. follows predicate q includes "B" which can "cover" predicate "p". In
  1652. other words, in trying to decide in r0 whether to call r1, it doesn't
  1653. really matter whether p is false or true because, either way, there is
  1654. a valid choice within r1.
  1655. #154. (Changed in MR13) Making hoist suppression explicit using <<nohoist>>
  1656. A common error, even among experienced pccts users, is to code
  1657. an init-action to inhibit hoisting rather than a leading action.
  1658. An init-action does not inhibit hoisting.
  1659. This was coded:
  1660. rule1 : <<;>> rule2
  1661. This is what was meant:
  1662. rule1 : <<;>> <<;>> rule2
  1663. With MR13, the user can code:
  1664. rule1 : <<;>> <<nohoist>> rule2
  1665. The following will give an error message:
  1666. rule1 : <<nohoist>> rule2
  1667. If the <<nohoist>> appears as an init-action rather than a leading
  1668. action an error message is issued. The meaning of an init-action
  1669. containing "nohoist" is unclear: does it apply to just one
  1670. alternative or to all alternatives ?
  1671. -------------------------------------------------------
  1672. Note: Items #153 to #1 are now in a separate file named
  1673. CHANGES_FROM_133_BEFORE_MR13.txt
  1674. -------------------------------------------------------