C# DebugToken Manager Problem

[skfcz]

Hello All,

I generated a C# parser with debug tokenmanager enabled and get a problem with the generated code

The exception is

System.IndexOutOfRangeException : Der Index war außerhalb des Arraybereichs.
bei prostep.aveva.vol.SimpleCharStream.getBeginLine() in C:\home\cz\OPS\opdm-aveva-val-parser\aveva-vol-csharp\src\CharStream.cs:Zeile 238.
bei prostep.aveva.vol.VOLParserTokenManager.getNextToken() in C:\home\cz\OPS\opdm-aveva-val-parser\aveva-vol-csharp\src\VOLParserTokenManager.cs:Zeile 394.
bei prostep.aveva.vol.VOLParser.jj_scan_token(Int32 kind) in C:\home\cz\OPS\opdm-aveva-val-parser\aveva-vol-csharp\src\VOLParser.cs:Zeile 1517.
....

and the generated code

for (;;) {
// First see if we have any input at all.
try {
curChar = input_stream.BeginToken();
} catch(System.Exception) {
if (lexStateNames.Length > 1) {
System.Console.Error.Write("<" + lexStateNames[curLexState] + "> ");
}
System.Console.Error.WriteLine("Reached EOF at " +
input_stream.getBeginLine() + ":" +
input_stream.getBeginColumn());

At the input_stream is out of range and throws the IndexOutOfRangeException exception.
My proposal ist to change the template for the EOF case.

[kaikalur]

Are you using javacc8?

[kaikalur]

But also we know there are some issues with debug token manager :( You can try using the JavaCCInterpreter to debug it better.

[MarcMazas]

I've been able to reproduce the problem. I'll try to find its origin.