.map(code -> code.getType())
.map(code -> namesToClasses.get(code))
.forEach(representation ->
representation.accept( new SequenceDiagramGenerator(
new File(destination, component.getCanonicalName().substring(1).replace('.', '.')),
classesToComponents,
this))
);
forEach
element, which is the last one of the call chain
map(...)
operations
Hello, I am the HTML Parsing Guy. I just want to say that the last time I was here, User @MysterAitch and myself had a bit of a tizzie, so I was hoping to try one more time to retrieve an answer about Java Parser usage. Mostly, I was just trying to talk - since where I live (Dallas, Texas), I don't ever meet other software developers. It gets pretty boring around here, and redirecting the Java Parser community into "My Code" was not my goal, though I think it seemed that way given the writings I was sending to this Gitter Page.
Because I am in Dallas, I get VERY BORED by looking at Code. There aren't software developers, much, if at all. So I posted all kinds ofstuff - thinking people were going to "hang out" in this Gitter Chat Room. Anyway...
I started as a Foreign News Translation programmer (中文, and Español). It turned into an HTML Parsing Project. I have a full blown HTML Parse, Modify
and Retrieve Tool
that is very advanced at this point. It is the most advanced I have seen in the Java World - better than Python's Beautiful Soup which uses DOM Tree's. C#
has a good tool, but I am simply illiterate in C#
since Java
is so closely related to it. The first thing I did after developing this tool was to parse JavaDoc
Generated HTML Documentation Pages for Java Code. As a result, I eventually started using the parsed HTML to include Code Hiliting in the JavaDoc
Pages as seen here:
http://developer.torello.directory/JavaHTML/Version%201/1.6/javadoc/index.html?overview-summary.html
In order to insert Hilited Java Source
Method Bodies and Constructor Bodies into a JavaDoc Generated HTML Page, I needed to use JavaParser to retrieve the method bodies using the LexicalPreservingPrinter
- as seen by these lines below. This is the intended purpose of the code I have written that uses JavaParser. Once I have retrieved a method body (or constructor body) as a String
, I send that body to an external Code Hiliter / Syntax Hiliter (not of my design at all, and not using Java Parser) to HiLite the code, and include it the JavaDoc HTML pages that I use for my HTML Library
.
// Methods (MethodDeclaration)
(md.getBody().isPresent()
? LexicalPreservingPrinter.print
(LexicalPreservingPrinter.setup(md.getBody().get()))
: null)
// Constructors (ConstructorDeclaration)
LexicalPreservingPrinter.print(LexicalPreservingPrinter.setup(cd.getBody()))
// Fields (VariableDeclarator, FieldDeclaration)
vd.getInitializer().isPresent()
? LexicalPreservingPrinter.print(LexicalPreservingPrinter.setup(vd.getInitializer().get()))
: null;
I got this to work back last December - literally within a week or two of trying JavaParser. Again, these String's
are HiLited using an external "Code Documentation HiLiting Tool" on the Internet. The processing isn't even done on the host computer, but rather by a Server elsewhere on Internet. I also retrieve Parameter Names, Parameters Types, and Return Types, and Modifier Lists, and Throws Exceptions, but each and every one of these JavaParser invokations are for retrieving a java.lang.String
ONLY. The actual references to the JavaParser Abstract Syntax Tree
nodes are fully discarded once I have retrieved all these datum as Java String's
. Below is a link to my Hanlder
:
HERE IS MY QUESTION:
There is a lot of "Memory Leak" whenever I use the class StaticJavaParser
. I hope to explain (quickly) that my Java HTML Parsing utility CANNOT BE the source of the major memory leak in my code. I can run an HTML Scrape that parses web-pages for hours and hours, and watch as Web-Pages are downloaded into String's
, and parsed in Vector's
of HTMLNode's
. I have a largely "static" library, where NO INTERNAL STATE is maintained by my code. This means, whenever I call the Java Garbage Collector
- 100% of the Parsed HTML Vector's of HTMLNode
are fully removed from memory, and the Java Heap is (easily) emptied by an invokation of System.gc()
. I print the memory available on the heap to check this often with my NewsSite Scrapers. See a log to watch Memory Consumption using Vector<HTMLNode>
:
HOWEVER: When I use class StaticJavaParser
to parse a java Source File (".java") file into memory, somewhere and somehow there are cyclic references being maintained on the Java Heap by the class StaticJavaParser
. I say this because I have written a build script that uses my HTML Parser for Parsing JavaDoc Generated HTML Pages, and these HTML Pages are updated via the JavaParser class StaticJavaParser
to retrieve Method Bodies, Constructor Bodies, and Field Declarations. As I run my Build Script to process well over 300 class (110,000 lines of code and documentation), the amount of memory on the Java Heap
gets larger and larger and larger. Whenever I invoke System.gc()
, the JRE gets slower and slower and slower. My Build Script takes almost 15 minutes to run sometimes, and a huge percentage of the time it is the System.gc()
invokation that is causing it to run so slowly.
When class StaticJavaParser
builds an Abstract Syntax Tree
from a Java Source File - how many references to the AST
nodes are maintained by pointers inside the class? If I invoke the following line of code:
CompilationUnit cu = StaticJavaParser.parse(javaSrcFileAsString);
VoidVisitor<?> visit = new HANDLER(ret, javaSrcFileName);
// Call JavaParser, it will use the GETTER class as the "Event Handler"
visit.visit(cu, null);
// Feeble minded attempt to clear out the AST tree, quickly.
// NOTE: This line is not doing anything right now...
NULL_THE_TREE(cu);
Is there any easy way to Garbage Collect an AST Tree
after it has been used to retrieve String
Information once it has been parsed? Here is my "feeble minded attempt" to clear the AST Tree
which is not working:
private static void NULL_THE_TREE(Node n)
{
java.util.List<Node> l = n.getChildNodes();
java.util.stream.Stream.Builder<Node> b = java.util.stream.Stream.builder();
l.forEach((Node nChild) -> b.accept(nChild));
b.build().forEach((Node nChild) ->
{ try { nChild.removeForced(); NULL_THE_TREE(nChild); } catch (Exception e) { } }
);
}
Sorry for the length of this question, I am all ears for answers though... Is there a better version of NULL_THE_TREE(Node n)
or perhaps NULL_THE_TREE(CompilationUnit c)
, so that the Java Garbage Collector can remove the nodes for one source file and move on to the next, without completely restarting my Build Script for each Java Source Code File? Again, I have well over 300 source files. Many of them over a thousand lines long (with Documentation)
AST
objects that the Garbage Collector is unable to clean that causes it to get slower and slower and slower. Whenever System.gc()
is invoked, it will hang as it searches (likely) tens of thousands or hundreds of thousands of Node
references that it is unable to remove...
AST's
from The Java Heap so that the Java Garbage Collector
(a.k.a. System.gc()
) can work its magic. I don't want to call it a Java Parser Memory Leak yet - because I cannot tell whether or not the class StaticJavaParser
has a just cause in retaining the AST's
after it has parsed '.java' source files
...
AST Nodes
that are being retained in memory / on the Java Heap - permanently - are in fact being retained by the class LexicalPreservingPrinter
AND NOT by the class StaticJavaParser
? If so, how might one enforce that those AST's
are nulled (set to null
) so that they may be Garbage Collected ?
Gitter
. Hey! Danny van Bruggen if you happen to read this, I'm trying to identify whether there is a way to clear the Heap of all AST's
or Nodes
after I am done using them. My Build Script uses (quite well, actually) Java Parser to parse well over 300 source code '.java'
files - and all I can tell is that when the class Visitor
/ Handler I use "goes out of scope" for one '.java'
source code file and moves on to the next one - all of the AST Nodes
are retained in memory and cannot be garbage collected (from the last source-code file) I would just like to know if there were a way to 'null'
their references in whichever JP class
these references are being retrained.
VoidVisitorAdapter
class I use, here's The JavaDoc for it... Here are the other classes the retrieve the java.lang.String
that I need for my HTML Documentation Tool.... Here is Callable
, (JavaDoc), and 'Method' (JavaDoc), and 'Constructor' (JavaDoc), and 'Field' (JavaDoc)
public class Customer{}
won't work, but if I change it to public class Customer {}
I can add an extends no problem. The problem also goes away if I don't set the configuration of the parser to enable lexical preservation, and if I don't setup the LexicalPreservingPrinter for each Compilation Unit. If I'm not being clear I can post more details in the issues on Github, might be easier.
I have classe bellow, so I would like to have output
OUTPUT:
WebServiceXpto
method_1
call_1
call_2
String x = "end";
method_2
call_2
String x = "end";
Maybe on an map having list of method as key and list os all call stack.
CLASSE:
public class WebServiceXpto {
public void method_1() {
call_1()
}
public void method_2() {
call_2()
}
public void call_1() {
call_2()
}
public void call_2() {
String x = "end";
}
}
Hi @jlerbsc I tried but did't found solution. I din't found I way to have all children of MethodCallExpr or recursive have it
My MethodCallCheck extends VoidVisitorAdapter<Void>
public void visit(MethodCallExpr methodCallExpr, Void arg) {
super.visit(methodCallExpr, arg);
}
This is my MethodCheck extends VoidVisitorAdapter<Void>
@Override
public void visit(MethodDeclaration methodDeclaration, Void arg) {
super.visit(methodDeclaration, arg);
LOG.info("> MethodDeclaration: " + methodDeclaration.getNameAsString());
NodeList<Statement> nodeList = methodDeclaration.getBody().get().getStatements();
if (nodeList != null) {
startCheck = false;
for (Statement statement : nodeList) {
// LOG.info("> Statement: " + statement + " - " + statement.isExpressionStmt());
VoidVisitor<Void> methodCallCheck = new MethodCallCheck(packageNameBase, classNameBase, methodDeclaration.getNameAsString());
statement.accept(methodCallCheck, arg);
}
}
}
AST
after I am done with it? I have nary a clue where to start looking for that. I load AST's
by asking the class StaticJavaParser
for a CompilationUnit cu
, and then have a straightforward VoidVisitor
retrieve several String's
from the AST
... Is there any simple way to clear the AST
out of the Java Heap ("memory") after my build code moves on from one source file to the next source-file ? Would you like to help ?? (Specifically, I mean System.gc()
doesn't get the AST's
out of memory !!
Memory Analyzer
(you brought it up, before)... Umm... Do you know how in C/C++
you have to say free(ptr)
or dealloc(ptr)
(I haven't done C/C++
in 20 years)... But in Java as long as you say objReference = null
you are guaranteed that the Object objReference
(whatever it was) will be cleared from the Heap by an invokation of System.gc()
(because there are no references to it) ... How in the world can I make sure the AST
pointers that are being retained by ether LexicalPreservingPrinter
or StaticJavaParser
are set to null, so that System.gc()
will clean the heap out after my code is through with an AST
? I've got to clean that out in my build process - when the build moves from one source-file AST
and one to the next source-file AST