Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Jonathan Revusky
    @revusky
    javac org/parsers/pylexer/PythonLexer.java
    As for the rest of what we were discussing, you know, there really isn't any great need for any eclipse plugin or whatever. Actually, at this stage, you can just use any text editor you are comfortable with and the JDK. Quite arguably, anything beyond that could just be a distraction. (I mean at this sort of beginner stage...)
    Jonathan Revusky
    @revusky

    Oh, and as for this...

    @revusky do you happen to have a mechanism to restrict the parser to the latest javacc21 syntax which rejects the older syntax?
    No, not currently. But for a long time I have been intending to write a "lint" utility that runs over grammar files and replaces the legacy syntax with the newer streamlined syntax. But I haven't done that yet.

    wyjjoi
    @wyjjoi
    @revusky Please explain where the PythonLexer you used is located.
    I have searched the tree downloaded over the weekend and there is no mention of an org/parsers.
    Vinay Sajip
    @vsajip
    The org.parsers.python directory is created when the lexer/parser are generated.
    I mean org/parsers/python
    wyjjoi
    @wyjjoi
    @vsajip I did an 'ant; followed by 'ant test' and now I see the org tree. Thanks
    wyjjoi
    @wyjjoi

    I have been experimenting with grammars based on
    javacc21/examples/tutorial/arithmetic2 as, for my own purposes, I have
    a need for arithmetic expressions. The direcroy contains a .javacc file
    and a free standing ArithmeticTest.java which controls
    the input from standardin, parses it, dumps the AST and calculates the result.

    $ javac *.java
    $ java ArithmeticTest
    2.+4.
    Dumping the AST...
    AdditiveExpression
    2
    +
    4
    The result is: 6.0

    $ java ArithmeticTest
    2.1+4.1
    Dumping the AST...
    AdditiveExpression
    2.1
    +
    4.1
    The result is: 6.19999999999999

    Both runs give the correct answer but the decimal points are missing i
    n the tree of the first run. The tokens in the first AST omit the decimal points.

    The following are the output of a grep of the javacc file.
    $ grep NUMBER *.javacc
    <NUMBER : (["0"-"9"])+ ("."(["0"-"9"])+)?>
    (<NUMBER> | ParentheticalExpression())
    (<NUMBER> | ParentheticalExpression())
    Would someone please explain for me how this works as I can't explain
    why the first example above doesn't cause a syntax error because the
    decimal points are not followed by 'one or more' digits.

    I can see that the hooked expression takes care of ( neither "." norany digits )
    but where do the solitary points end up?.

    Thanks

    Jonathan Revusky
    @revusky
    Uff, you seem to have run into a bug. And it seems to be present since nearly 2 months ago. Basically, it's skipping over invalid tokens. That is the correct behavior in fault-tolerant mode, but if you haven't set FAULT_TOLERANT=true, then it should not be skipping over them. Hang on, I'll have this fixed in a little while.
    wyjjoi
    @wyjjoi
    @revusky Thanks for the info. The problem isn't holding me up at the moment so no rush. I will search for some info on FAULT_TOLERABNT.
    Jonathan Revusky
    @revusky
    It's all fixed up now. Actually, I modernized that example. That was something very ancient that I wrote for a tutorial that I was writing a long time ago, like 2008, and now it can be written much more cleanly. So, if you just do a git pull you'll have the new version. And it should work correctly. Give it a try. Thanks.
    wyjjoi
    @wyjjoi
    @revusky I eventually discovered that the git pull required no further parameters! I worked apparently OK. On checking the arithmetic2 directory I found that the ArithmeticTest,java file was missing. I carried on the generator which worked successfuly but of course I couldn't run the ArithmeticTest class so I ran the ArithmeticParser class which gave a large number of error messages starting as follows::
    $ java ArithmeticParser.java 1+2
    ArithmeticParser.java:16: error: cannot find symbol
    public class ArithmeticParser implements ArithmeticConstants {
    ^
    symbol: class ArithmeticConstants
    ArithmeticParser.java:17: error: cannot find symbol
    static public void main(String[] args) throws ParseException {
    ^
    symbol: class ParseException
    location: class ArithmeticParser
    ArithmeticParser.java:28: error: cannot find symbol
    Token lastConsumedToken;
    ^
    symbol: class Token
    location: class ArithmeticParser
    ArithmeticParser.java:29: error: cannot find symbol
    private TokenType nextTokenType;
    ^
    symbol: class TokenType
    location: class ArithmeticParser
    ArithmeticParser.java:30: error: cannot find symbol
    Do you think the missing java file might be the cause of this?
    wyjjoi
    @wyjjoi
    @revusky When I include the ArithmeticTest.java the program runs but shows the same behaviour the 1+2. results in 3 rather than an error. BTW I am using jar javacc-1642690736424-full.jar
    Jonathan Revusky
    @revusky
    You should use the latest javacc-full.jar. Also, the ArithmeticTest.java is no longer needed. You just run the test with java ArithmeticParser
    Just delete all the .java and .class files in the directory.
    Then it's just java -jar javacc-full.jar Arithmetic.javacc and javac *.java and then java ArithmeticParser
    wyjjoi
    @wyjjoi
    @revusky I thought that javacc-1642690736424-full.jar was the latest jar I had down loaded. It appeared on my desktop when I used a javacc-full.jar which told me that it was downloading a newer one which I used from then onward. Now that I compare using cmp and ls I find a difference
    $ ls -l ../inserts02/*.jar
    -rwxrwxrwx 1 jones jones 1796178 Jan 18 15:16 ../inserts02/javacc-1642690736424-full.jar
    -rwxrwxrwx 1 jones jones 1806668 Jan 20 14:58 ../inserts02/javacc-full.jar
    I have no idea what I did to get the new jar but when I use it:
    FAULT_TOLERANT =false; results in a syntax error (At Last!)
    wyjjoi
    @wyjjoi
    BUT setting FAULT_TOLERANT=true; results in compilation errors in AritmeticParser.java to do with while (lastConsumedToken.getType()!=EOF) {
    ^
    symbol: variable EOF
    Please tell me the procedure to download the latest full jar?
    wyjjoi
    @wyjjoi
    Please tell me the procedure to build a full.jar from the Source Tree pull 'ed from githead
    Without any setting I now get the desired syntax error. So now the only anomaly is that setting an explicit false results in a few compilation errors.
    Jonathan Revusky
    @revusky
    The latest (or about the latest) jarfile should be downloadable at https://javacc.com/download/javacc-full.jar
    You can also build it if you have the code checked out. Make sure to pick up the latest changes with git pull
    And then just ant full-jar
    That's from the root directory.
    Jonathan Revusky
    @revusky
    Oh, by the way, I just put a better version of the arithmetic example up. It is in a different directory, examples/arithmetic, and the older one is now deleted.
    wyjjoi
    @wyjjoi
    @revusky I have looked through the new Arithmetic 's ex1 and ex2. They look slick to me and I will move my own little mods into a copy of them. Thanks.
    wyjjoi
    @wyjjoi
    Does anybody know of an example dealing with a TOKENIZER/ LEXER that is simpler than the Python Example in the main tree. (The python case deals with indents and dedents ands split= lines.)
    Jonathan Revusky
    @revusky
    I just looked and that is currently the only example of using the TOKEN_HOOK machinery. Other places were using it but then I mostly didn't need it any more because I introduced the whole context-sensitive tokenization thing which handles the most typical use of TOKEN_HOOK. So the only place I need to use it now is for Python. But basically, what is it that you want to do?
    wyjjoi
    @wyjjoi
    I have two problems that I think are related. I have based a small grammar on the example Arithmetic with an extra INCLUDE file. Most of a test statement is parsed ok but the tail isn't and I cannot see why not! Part of the problem is, I believe, that I have not connected up a lexer with a TOKEN_HOOK. At the moment I am following the Doc in putting the most simple T_Hook in.
    Could we start with a look at the additions I have made to your Arithmetic example/
    ? May I send you the little files somehow? email ?l
    wyjjoi
    @wyjjoi
    Feed "JavaCC 21 – JavacCC for the 21st Century" has been de-activated@revusky
    @revusky I have been unable to find a support email for Javacc21 thought I know I have seen one in passing over the last couple of days
    wyjjoi
    @wyjjoi
    @revusky A chat request popped up from you but disappeared before I could click
    Jonathan Revusky
    @revusky
    I recently just enabled the Discussions feature on Github. You can attach files there, it seems. I haven't used this at all yet. But we could try it. https://github.com/javacc21/javacc21/discussions
    But, really, rather than attaching files and asking me to sort it out for you, it would be best to try to formulate a very clear, focused sort of question, to clarify some issue you're having trouble with. But, anyway, maybe try that Discussions feature. And we'll see how that goes.
    wyjjoi
    @wyjjoi
    @revusky I really am not attempting to dump this on you. I find that the documentation is quite spread out!! I have at last succeeded in getting a Lexer generated so I can make some progress in understanding the details.
    wyjjoi
    @wyjjoi
    I have tried the above URL and again tried to send you a tar file. AT last an error message saying it does not support tar files but wants a zip.
    wyjjoi
    @wyjjoi
    @revusky It now reports that it has created a zip file on javacc21/javacc21. unfortunately my wife is telling that our evening meal is ready so I have to pause. Since I don't know whether you have received my other communications One important message gave a test string which will cause the error that I cannot fathom : "W A rtag[3]+4: tws " (without the " chars)
    At this point I should mention that I can work only three days per week on this project, Tue Wed and Thur. I will check in 40 minutes and hope that you are not waiting for anything from me. Thanks for putting up with this chaotic exchange.
    The zip file is revusky.zip
    wyjjoi
    @wyjjoi
    wyjjoi
    @wyjjoi
    @revusky Apologies for all the confusion yesterday. I eventually found the problem I had yesterday to be caused by the order of the token definitions.