by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Roger Howell
@MysterAitch
@laleye -- it should be fairly straightforward to do, but I don't have much of a clue how to do it!
laleye
@laleye
@MysterAitch Ok, thanks
MarcosCosmos
@MarcosCosmos
nah I don't think they are version issues although actually hold on
private Date currentDate = new Date();
    private String strDate;
    private SimpleDateFormat formatador = new SimpleDateFormat("dd/MM/yyyy");

    /* Pega data currente e atribui a uma string */
    strDate = formatador.format(currentDate);
com.github.javaparser.ParseProblemException: (line 16,col 5) Parse error. Found "=", expected one of "enum" "exports" "module" "open" "opens" "provides" "requires" "strictfp" "to" "transitive" "uses" "with" "yield" <IDENTIFIER>
This seems like valid code am I missing something?
(line 16 is the last line in the snippet)
oh it's in the static context without well, being in the static context nvm
MarcosCosmos
@MarcosCosmos
oh goodie I have OOM errors encoding all this. Welp, I can technically flush nodes to file every compilation unit, as long as I don't run into memory issues with the attributes I should be okay?
Roger Howell
@MysterAitch
Are you doing an resolution?
*any
MarcosCosmos
@MarcosCosmos
resolution?
as in symbol resolution?
not yet
Roger Howell
@MysterAitch
Symbol resolution -- resolving usages of methods/variables to their declarations
Yes
As far as I know, parsing is completely independent and caches can be flushed
It only becomes more important when needing to cache things between compilation units (i.e. symbol resolution)
If @matozoid or @ftomassetti or others say different, it is likely best to trust what they :)
MarcosCosmos
@MarcosCosmos
Yeah I'm just trying to load 300mb+ of parsed code into memory at once - I realised that I only need to hold attributes between compilation units though which should be near primitive classes and not contain any backrefs so that should work out
How do I flush the caches anyway?
I could just replace the parser object each time but
Oh wait I use the static one
so that will flush
once I stop holding nodes in my vars anyway
Roger Howell
@MysterAitch
There was an issue about memory usage and caches -- the discussions there might be more useful than my speculations
MarcosCosmos
@MarcosCosmos
cool cool
ok I think that kept the memory down really well but I guess I'll know in like 10 minutes when it approaches where it was before it died last time.
MarcosCosmos
@MarcosCosmos
I'm actually really glad I'm encoding ASTs into simple trees of integers now though, anything more would probably be impossible to use memory wise
oh my god the android library is huge it's already encoded over 500 files and is only up to c alphabetically
Also I hope parse order is fully deterministic or else I'm probably kind of fucked for now lol
Roger Howell
@MysterAitch
Do you have numbers you could share re: memory usage?
MarcosCosmos
@MarcosCosmos
so now that I started flushing, usage reported in my system has only grown by 3% (of 32gb) so far in this run
and includes me having a big text file listing all the file paths processed so far open
More than 1800 units have been processed now
Roger Howell
@MysterAitch
MarcosCosmos
@MarcosCosmos
cheers
MarcosCosmos
@MarcosCosmos
okay, 57274215 AST nodes+attributes successfully processed with no memory dramas.
Roger Howell
@MysterAitch
Cool, glad you managed to get the small test done.. when will you start the full sized test started?
:wink:
MarcosCosmos
@MarcosCosmos
Well you see, the encoded file is 300mb larger than the input code corpus due to well, needing a unique integer for each of those nodes to appear exactly twice in plain text, plus seperator characters for things, so the need task is to read that file and find out how big it actually is in memory with a cpp vector for every node etc
xD
if it's less than like 100mb I should be apply my whole algo on my home machine, which is faster than the university grid AFAIK for single threaded stuff (which mine is for reasons of me not having time to thread it)
MarcosCosmos
@MarcosCosmos
Welp I did the math and it's like minimum 1gb but that's probably workable.
Yi Zhang
@yzhang90
Hi, I have a project that uses java parser 1.5. Is it possible to license it under Apache license?
Danny van Bruggen
@matozoid
Hi! The licensing changed later, so officially no ( javaparser/javaparser#144 ). Does it use a lot of the API? Maybe I'll try to upgrade it.
laleye
@laleye
Hi
I'm just starting to learn javaparser...I would want to know if it's possible to parse a snippet code like (StringBuilder sb = new StringBuilder();)
?
Roger Howell
@MysterAitch
Hi @laleye -- have you seen the examples on the homepage of https://javaparser.org/ ?
@laleye the (good news) short answer is "yes" :smile:
@laleye there are also LOTS of examples available in the book too -- there's a link at the bottom of the web page iirc... Donations to the book authors (not me!) are very welcome, but it can be downloaded for free too :)