Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Noah Santschi-Cooney
@Strum355
Depending on the stability of the parser and the symbol solver, our needs for maintenance may only include the addressing of any pull requests rather than active contributions
Danny van Bruggen
@matozoid
The parser has not caused significant issues for years now - although it needs to be updated for every new Java version. The symbol solver could use a rewrite, but as @ftomassetti says: it has been used in large projects, so it is useful right now.
Noah Santschi-Cooney
@Strum355
Contributing to add support for new constructs would probably be easier to greenlight than contributing to fix bugs, so hopefully the eval period goes well :slight_smile:
mdproctor
@mdproctor
did anyone here look into javacc21? https://github.com/javacc21/javacc21
mdproctor
@mdproctor
going through javacc21 now. nesting lookahead, much better general syntax. java13 support. improved jjtree. generating more readable code and supports "include" for modularity (last is great for language developers).
jean pierre Lerbscher
@jlerbsc
Hi @mdproctor there is already an issue on this subject but i don't know what is the conclusion.
javaparser/javaparser#2659
mdproctor
@mdproctor
it has includes modularity, so already sounds great for language developers looking to do extensions to java :)
mdproctor
@mdproctor
"Long term, the goal is to switch to using ANTLR anyway - so any meaningful efforts would be in that direction."
for some reason, no one maintain the Java antlr grammar.... so it's way behind.
that's gonna be an impedeminent to antlr adoption.
Federico Tomassetti
@ftomassetti
I strongly discourages anyone to look into javacc21: the project is led by a person who came and insulted in a violent way all the members of the JP community and has been banned from participating. He also forked JP out of spite. I never encountered someone who behaved in such a way, in a totally unprovoked manner
He also forked JavaCC and called JavaCC21, registering also domain related to both JavaCC and JavaParser. Please be careful with any interaction with that person
Danny van Bruggen
@matozoid
Ban the guy, he's poison.
Nicolas Delsaux
@Riduidel
Hey JavaParser community!
I'm trying to write some kind of code to read Java source into sequence diagrams backed by C4 model components.
And I'm having a hard time parsing the following code block
            .map(code -> code.getType())
            .map(code -> namesToClasses.get(code))
            .forEach(representation -> 
                representation.accept( new SequenceDiagramGenerator(
                        new File(destination, component.getCanonicalName().substring(1).replace('.', '.')),
                        classesToComponents,
                        this))
                    );
It seems like JavaParser visits as first element the forEach element, which is the last one of the call chain
Then it visits the two map(...) operations
And the question is ... why is the last method visited first ?
Danny van Bruggen
@matozoid
I bet because the whole thing is a nested method call expression, and the method gets visited before the target!
Anyway, you can get insight on how the AST is constructed with @MysterAitch 's inspector: https://plugins.jetbrains.com/plugin/14245-javaparser-ast-inspector
and feel free to look at the source of the visitor, which defines the visit order.
Chinese News Board
@ralphcnb_twitter

Hello, I am the HTML Parsing Guy. I just want to say that the last time I was here, User @MysterAitch and myself had a bit of a tizzie, so I was hoping to try one more time to retrieve an answer about Java Parser usage. Mostly, I was just trying to talk - since where I live (Dallas, Texas), I don't ever meet other software developers. It gets pretty boring around here, and redirecting the Java Parser community into "My Code" was not my goal, though I think it seemed that way given the writings I was sending to this Gitter Page.

Because I am in Dallas, I get VERY BORED by looking at Code. There aren't software developers, much, if at all. So I posted all kinds ofstuff - thinking people were going to "hang out" in this Gitter Chat Room. Anyway...

I started as a Foreign News Translation programmer (中文, and Español). It turned into an HTML Parsing Project. I have a full blown HTML Parse, Modify and Retrieve Tool that is very advanced at this point. It is the most advanced I have seen in the Java World - better than Python's Beautiful Soup which uses DOM Tree's. C# has a good tool, but I am simply illiterate in C# since Java is so closely related to it. The first thing I did after developing this tool was to parse JavaDoc Generated HTML Documentation Pages for Java Code. As a result, I eventually started using the parsed HTML to include Code Hiliting in the JavaDoc Pages as seen here:

http://developer.torello.directory/JavaHTML/Version%201/1.6/javadoc/index.html?overview-summary.html

In order to insert Hilited Java Source Method Bodies and Constructor Bodies into a JavaDoc Generated HTML Page, I needed to use JavaParser to retrieve the method bodies using the LexicalPreservingPrinter - as seen by these lines below. This is the intended purpose of the code I have written that uses JavaParser. Once I have retrieved a method body (or constructor body) as a String, I send that body to an external Code Hiliter / Syntax Hiliter (not of my design at all, and not using Java Parser) to HiLite the code, and include it the JavaDoc HTML pages that I use for my HTML Library.

// Methods (MethodDeclaration)

(md.getBody().isPresent()
    ?   LexicalPreservingPrinter.print
           (LexicalPreservingPrinter.setup(md.getBody().get()))
    :   null)

// Constructors (ConstructorDeclaration)

LexicalPreservingPrinter.print(LexicalPreservingPrinter.setup(cd.getBody()))

// Fields (VariableDeclarator, FieldDeclaration)

vd.getInitializer().isPresent()
    ? LexicalPreservingPrinter.print(LexicalPreservingPrinter.setup(vd.getInitializer().get()))
    : null;

I got this to work back last December - literally within a week or two of trying JavaParser. Again, these String's are HiLited using an external "Code Documentation HiLiting Tool" on the Internet. The processing isn't even done on the host computer, but rather by a Server elsewhere on Internet. I also retrieve Parameter Names, Parameters Types, and Return Types, and Modifier Lists, and Throws Exceptions, but each and every one of these JavaParser invokations are for retrieving a java.lang.String ONLY. The actual references to the JavaParser Abstract Syntax Tree nodes are fully discarded once I have retrieved all these datum as Java String's. Below is a link to my Hanlder:

http://developer.torello.directory/JavaHTML/Version%201/1.6/javadoc/Torello/HTML/Tools/JavaDoc/JavaSourceCodeFile.HANDLER.html

HERE IS MY QUESTION:

There is a lot of "Memory Leak" whenever I use the class StaticJavaParser. I hope to explain (quickly) that my Java HTML Parsing utility CANNOT BE the source of the major memory leak in my code. I can run an HTML Scrape that parses web-pages for hours and hours, and watch as Web-Pages are downloaded into String's, and parsed in Vector's of HTMLNode's. I have a largely "static" library, where NO INTERNAL STATE is maintained by my code. This means, whenever I call the Java Garbage Collector - 100% of the Parsed HTML Vector's of HTMLNode are fully removed from memory, and the Java Heap is (easily) emptied by an invokation of System.gc(). I print the memory available on the heap to check this often with my NewsSite Scrapers. See a log to watch Memory Consumption using Vector<HTMLNode>:

http://developer.torello.directory/JavaHTML/Version%201/1.6/javadoc/Torello/HTML/Tools/NewsSite/GovCN-ScrapeArticles.html

HOWEVER: When I use class StaticJavaParser to parse a java Source File (".java") file into memory, somewhere and somehow there are cyclic references being maintained on the Java Heap by the class StaticJavaParser. I say this because I have written a build script that uses my HTML Parser for Parsing JavaDoc Generated HTML Pages, and these HTML Pages are updated via the JavaParser class StaticJavaParser to retrieve Method Bodies, Constructor Bodies, and Field Declarations. As I run my Build Script to process well over 300 class (110,000 lines of code and documentation), the amount of memory on the Java Heap gets larger and larger and larger. Whenever I invoke System.gc(), the JRE gets slower and slower and slower. My Build Script takes almost 15 minutes to run sometimes, and a huge percentage of the time it is the System.gc() invokation that is causing it to run so slowly.

When class StaticJavaParser builds an Abstract Syntax Tree from a Java Source File - how many references to the AST nodes are maintained by pointers inside the class? If I invoke the following line of code:

CompilationUnit cu = StaticJavaParser.parse(javaSrcFileAsString);
VoidVisitor<?> visit = new HANDLER(ret, javaSrcFileName);

// Call JavaParser, it will use the GETTER class as the "Event Handler"
visit.visit(cu, null);

// Feeble minded attempt to clear out the AST tree, quickly.
// NOTE: This line is not doing anything right now...
NULL_THE_TREE(cu);

Is there any easy way to Garbage Collect an AST Tree after it has been used to retrieve String Information once it has been parsed? Here is my "feeble minded attempt" to clear the AST Tree which is not working:

    private static void NULL_THE_TREE(Node n)
    {
        java.util.List<Node>                    l = n.getChildNodes();
        java.util.stream.Stream.Builder<Node>   b = java.util.stream.Stream.builder();

        l.forEach((Node nChild) -> b.accept(nChild));

        b.build().forEach((Node nChild) -> 
            { try { nChild.removeForced(); NULL_THE_TREE(nChild); } catch (Exception e) { } }
        );
    }

Sorry for the length of this question, I am all ears for answers though... Is there a better version of NULL_THE_TREE(Node n) or perhaps NULL_THE_TREE(CompilationUnit c), so that the Java Garbage Collector can remove the nodes for one source file and move on to the next, without completely restarting my Build Script for each Java Source Code File? Again, I have well over 300 source files. Many of them over a thousand lines long (with Documentation)

Chinese News Board
@ralphcnb_twitter
You may look at my Build Script Log right which prints the memory available / memory used by the JRE. It is not the amount of memory that is really the problem, it is that since there are (obviously) large AST objects that the Garbage Collector is unable to clean that causes it to get slower and slower and slower. Whenever System.gc() is invoked, it will hang as it searches (likely) tens of thousands or hundreds of thousands of Node references that it is unable to remove...
jean pierre Lerbscher
@jlerbsc
Hi @ralphcnb_twitter Do you use LexicalPreservingPrinter? If yes, can you execute a test without LexicalPreservingPrinter?
Chinese News Board
@ralphcnb_twitter
? Don't get it... The other printer's aren't capable of retrieving the Java String's needed by the utility / tool.
Again, I need to null the pointers or - eliminate the AST's from The Java Heap so that the Java Garbage Collector (a.k.a. System.gc()) can work its magic. I don't want to call it a Java Parser Memory Leak yet - because I cannot tell whether or not the class StaticJavaParser has a just cause in retaining the AST's after it has parsed '.java' source files...
Chinese News Board
@ralphcnb_twitter
Let me know... 🙏
jean pierre Lerbscher
@jlerbsc
First you don't have to setup LexicalPreservingPrinter more than once per CompilationUnit. Secondly you can tune memory heap to make the gc footprint smaller and thirdly you can use toString() to print every statement. LexicalPreservingPrinter keep track of the original syntaxe in each node which uses more memory. I hope these tips can help you.
Chinese News Board
@ralphcnb_twitter
Are you trying to say that the AST Nodes that are being retained in memory / on the Java Heap - permanently - are in fact being retained by the class LexicalPreservingPrinter AND NOT by the class StaticJavaParser? If so, how might one enforce that those AST's are nulled (set to null) so that they may be Garbage Collected ?
Chinese News Board
@ralphcnb_twitter
I was sort of hoping that Van Bruggen / Matazoid was out there... He seems to know JP the best on Gitter. Hey! Danny van Bruggen if you happen to read this, I'm trying to identify whether there is a way to clear the Heap of all AST's or Nodes after I am done using them. My Build Script uses (quite well, actually) Java Parser to parse well over 300 source code '.java' files - and all I can tell is that when the class Visitor / Handler I use "goes out of scope" for one '.java' source code file and moves on to the next one - all of the AST Nodes are retained in memory and cannot be garbage collected (from the last source-code file) I would just like to know if there were a way to 'null' their references in whichever JP class these references are being retrained.
Chinese News Board
@ralphcnb_twitter
Um... And here is the VoidVisitorAdapter class I use, here's The JavaDoc for it... Here are the other classes the retrieve the java.lang.String that I need for my HTML Documentation Tool.... Here is Callable, (JavaDoc), and 'Method' (JavaDoc), and 'Constructor' (JavaDoc), and 'Field' (JavaDoc)
Nuwan Sanjeewa Abeysiriwardana
@nuwansa
how do i include java sdk libraries to the symbol resolver. I'm getting following exception.
UnsolvedSymbolException{context='null', name='List', cause='null'}
jean pierre Lerbscher
@jlerbsc
Hi @nuwansa You have to initialize JavaSymbolSolver with ReflectionTypeSolver.
Nuwan Sanjeewa Abeysiriwardana
@nuwansa
@jlerbsc thanks
Déaglán Connolly Bree
@Deaglancb_gitlab
Hi everyone. I came across some behaviour but I'm not sure if it's a bug or if I'm missing something (RE how Lexical Preservation is intended to be used). Trying to addExtendedType to a class, I get an unsupported operation if there isn't a space before the opening brace of the class (which I want to add the extends to) with lexical preservation enabled. So public class Customer{} won't work, but if I change it to public class Customer {}I can add an extends no problem. The problem also goes away if I don't set the configuration of the parser to enable lexical preservation, and if I don't setup the LexicalPreservingPrinter for each Compilation Unit. If I'm not being clear I can post more details in the issues on Github, might be easier.
jean pierre Lerbscher
@jlerbsc
Hi @Deaglancb_gitlab Could you open an issue and provide a test case please. Thanks.
Déaglán Connolly Bree
@Deaglancb_gitlab
Done, cheers!
marcosvasousa
@marcosvasousa

I have classe bellow, so I would like to have output

OUTPUT:

WebServiceXpto

method_1

call_1

call_2

String x = "end";

method_2

call_2

String x = "end";

Maybe on an map having list of method as key and list os all call stack.

CLASSE:
public class WebServiceXpto {

public void method_1() {
call_1()
}

public void method_2() {
call_2()
}

public void call_1() {
call_2()
}

public void call_2() {
String x = "end";
}

}

jean pierre Lerbscher
@jlerbsc
Hi @marcosvasousa I think you have to implement your own visitor (which can inherit from VoidVisitorAdapter) and redefine visit() methods for ClassOrInterfaceDeclaration, MethodDeclaration, MethodCallExpr... to print your own output format.
marcosvasousa
@marcosvasousa

Hi @jlerbsc I tried but did't found solution. I din't found I way to have all children of MethodCallExpr or recursive have it
My MethodCallCheck extends VoidVisitorAdapter<Void>
public void visit(MethodCallExpr methodCallExpr, Void arg) {
super.visit(methodCallExpr, arg);
}

This is my MethodCheck extends VoidVisitorAdapter<Void>
@Override
public void visit(MethodDeclaration methodDeclaration, Void arg) {
super.visit(methodDeclaration, arg);

    LOG.info("> MethodDeclaration: " + methodDeclaration.getNameAsString());

    NodeList<Statement> nodeList = methodDeclaration.getBody().get().getStatements();
    if (nodeList != null) {
        startCheck = false;

        for (Statement statement : nodeList) {
            // LOG.info("> Statement: " + statement + " - " + statement.isExpressionStmt());

            VoidVisitor<Void> methodCallCheck = new MethodCallCheck(packageNameBase, classNameBase, methodDeclaration.getNameAsString());
                statement.accept(methodCallCheck, arg);
        }
    }
}
jean pierre Lerbscher
@jlerbsc
Hi @marcosvasousa What you have represented in your example is an internal stack trace of the class but i think you want to extend it to all the classes of a project. I don't have time to implement your algorithm but what I recommend at first glance is to use the plain capacity of the visitor pattern and use the second argument to propagate an execution context which keep track of current class declaration, method declaration, method call expression for each class and use graph to modelize each relationship... Personnally i use this technique to implement a java class/package/module relationship in 3D representation (example http://www.jperf.com/jp/). Unfortunately i can't help you more without spending a lot more time on it, but I think you need to look further into these solutions or even study others if this one does not allow you to achieve your objective.
makbro
@makbro
Good evening, everyone! If someone would be so kind as to help me setup javaparser into NetBeans I would appreciate it. I want Netbeans to playback some feedback sounds whenever the parser detects an error as the user types code. How would I go about to do that? If there is already a video or resource explaining how to do that let me know.
And if there is someone I can contact directly that can help me, please let me know.
Danny van Bruggen
@matozoid
@makbro I think you want to look into creating Netbeans plugins for that - Netbeans already parses the code and shows the errors, so there must be a parser and some handling in there that you might be able to plug into. This project, JavaParser, is not in use in Netbeans.
makbro
@makbro
@matozoid I will do that. Thank you!