by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Apr 28 12:05
    pushed to delors/opal
    Merged in florian_kuebler/readmemarkdown-edited-online-with-bitbuc-1578400042961 (pull request #534) Updated Readme.markdown with deprecation/github migration statement Approved-by: errt <helm@cs.tu-darmstadt.de> (compare)
  • Apr 28 12:05
    pushed to delors/opal
    0 commits
  • Jan 07 12:28
    pushed to delors/opal
    Updated Readme.markdown with deprecation/github migration statement (compare)
  • Oct 10 2019 13:54
    pushed to delors/opal
    A few minor adaptions (compare)
  • Oct 10 2019 13:23
    pushed to delors/opal
    Formatting... (compare)
  • Oct 10 2019 13:16
    pushed to delors/opal
    Measure performace on 5 runs (compare)
  • Oct 07 2019 11:35
    pushed to delors/opal
    Adapt to changes in OPAL implementation (compare)
  • Aug 30 2019 11:52
    pushed to delors/opal
    fixed an issue related to the removal of dependers from a final EPK state (compare)
  • Aug 30 2019 09:09
    pushed to delors/opal
    3 commits
  • Aug 30 2019 09:04
    pushed to delors/opal
    Don't schedule PointsTo Analysis for other CGs (compare)
  • Aug 29 2019 14:32
    pushed to delors/opal
    2 commits
  • Aug 29 2019 14:27
    pushed to delors/opal
    3 commits
  • Aug 29 2019 14:09
    pushed to delors/opal
    2 commits
  • Aug 29 2019 13:36
    pushed to delors/opal
    2 commits
  • Aug 29 2019 13:18
    pushed to delors/opal
    10 commits
  • Aug 29 2019 13:16
    pushed to delors/opal
    More code duplication but should be slightly faster (compare)
  • Aug 29 2019 13:06
    pushed to delors/opal
    Prevent useless partial results (compare)
  • Aug 28 2019 15:04
    pushed to delors/opal
    improved the performance of LongTrieSet by specializing the grow function (compare)
  • Aug 27 2019 13:47
    pushed to delors/opal
    4 commits
  • Aug 27 2019 13:24
    pushed to delors/opal
    Commit just for testing purposes (compare)
mariotrageser
@mariotrageser
Are Demos and Validate the new projects, which replace the two others?
Michael Eichberg
@Delors
Not exactly
Demos exists “forever"
OPAL-Validate => Validate
mariotrageser
@mariotrageser
I meant Tools and OPAL-Developer Tools. So „OPAL-Developer Tools“ should also no longer exist?
Michael Eichberg
@Delors
OPAL-DeveloperTools => Tools
Yes
mariotrageser
@mariotrageser
I had to remove them manually, but now everything works. Thank you
Javor Nikolov
@bubbazz
there is a way to get the current program counter from the classes "SimpleBranchInstruction" and "LOOKUPSWITCH".
Michael Eichberg
@Delors
No - this information is generally only implicitly available. If you use Code’s iterate and foreach methods you are always provided with the current pc. Does this help? If not, can you provide us with a short sample?
ewontfix
@ewontfix
Is there an easy way to get all primitive expressions (expressions which don't contain subexpressions) which occur in a TAC Stmt? Stmt.forallSubExpression seems to be a bit overkill for this use case.
Michael Eichberg
@Delors
In general, the TAC code is flat; i.e., a statement’s expressions are either Vars or Consts - that’s it. (OPAL has the ability to represent nested (sub) expressions, but this feature is reserved for future usage related to decompling code.) Hence, Stmt.forallSubExpression combined with a simple match over the two mentioned types of expressions should be sufficient in most to all cases. Does this help?
ewontfix
@ewontfix
Yes, thanks. My Code already works with Stmt.forallSubExpressions. It just takes somewhat longer than expected to execute and I guessed unnecessary exploring of a lot of intermediate expressions might be the cause. But if expressions are already flat, there is probably another cause.
Michael Eichberg
@Delors
One of the most frequent sources of performance isssues that I see when people use OPAL is the “naive” usage of Scala collections.
For example: val x = Set(..); x.filter(…).map(..).toList creates two intermediate sets before the list is finally created. Rewriting such code to avoid the creation of intermediate collections (e.g., by using foldLeft and withFilter) can lead to very significant performance improvements. But, this is only a wild guess given that I don’t know your code at all...
ewontfix
@ewontfix
To be more specific: I'm working on exercise 4 in APSA. My Analysis works for the provided AllocationFreeness class. But each time I run it against the whole JDK (rt.jar) it fails with an OutOFMemoryError after roughly 7 minutes. I stripped down my analysis step by step. Now I have reached a minimal bare-bones version, in which the analyze method immediately returns a Result. Still if I run this analysis against the JDK it fails with the same error. This is the complete code of my stripped down analyze method:
def analyze(method: DeclaredMethod): PropertyComputationResult = {
    if (!method.hasSingleDefinedMethod)
      return Result(method, MethodWithAllocations);

    if (method.definedMethod.isNative)
      return Result(method, MethodWithAllocations);

    return Result(method, MethodWithAllocations);
  }
ewontfix
@ewontfix
Here's the full diff against the provided template for this version:
44,45c44,45
<     allocationFreeMethods.map(_.e).mkString("allocation free: ", "\nallocation free: ", "\n")+
<     methodsWithAllocations.map(_.e).mkString("with allocations: ", "\nwith allocations: ", "\n")+
---
> //    allocationFreeMethods.map(_.e).mkString("allocation free: ", "\nallocation free: ", "\n")+
> //    methodsWithAllocations.map(_.e).mkString("with allocations: ", "\nwith allocations: ", "\n")+
65c65
<     ???
---
>     return Result(method, MethodWithAllocations);
Am I correct that at least this really minimal code should work?
roterEmil
@roterEmil
I had the same problem, that the analysis works on the given AllocationFreeness class but gets an OutOFMemoryError when running it against the JDK. When running the given template without my analysis against the JDK it also gets this error.
Michael Eichberg
@Delors
When you want to analyze the JdK you should at least allocate 6gb to the analysis! Depending on the concrete Analysis even much more memory may be required
David Kretzmer
@david-k_gitlab
Hi, I don't quite understand how to specify the entry points. In particular, what is the meaning of the Fact that must be associated with a DeclaredMethod in entryPoints?
Florian Kübler
@florian-kuebler
Without spoiling the solution:
The IFDS algorithm is a graph reachability problem, and the map of entrypoints specifices the starting points for the graph exploration.
More specific that are the Methods and Facts that are true before analyzing the methods. If these facts do not need to store any specific information, tautology facts are used.
Does that help?
David Kretzmer
@david-k_gitlab
Thanks, in principal I understand this, I'm just a bit confused that we can specify only one Fact per DeclaredMethod, and so I don't really know how I would provide initial facts for e.g. the parameters.
Florian Kübler
@florian-kuebler
You should could consider the tautology fact in taint analysis as something expressing: This is not tainted, but this statement/method is reachable and a taint could be generated from here.
However, later on you should also propagate taints into parameters, and you are not restricted to one Fact per DeclaredMethod when propagating facts.
David Kretzmer
@david-k_gitlab
Thanks for trying to help me understand! Currently, I think my main confusion is that, if I understand the original IDFS paper correctly, then the tautology fact (represented by a bold zero in the paper) is more of an implementation detail of the IDFS algorithm and not something that you would use directly in the dataflow functions. So why would I need to specify a tautology fact when I only work with dataflow functions?
(I think this question is rather general and doesn't require spoiling the solution, but correct me if I'm wrong.)
Dominik Helm
@errt
It is not just an implementation detail. Your analysis needs some kind of starting condition and as, at least in OPAL, you can only start at methods' starts, you need some fact that is valid at that point. Now if your analysis tries e.g. to track parameters of public methods, you can use these, but if you are searching for taint sources that might be inside a method, you need that tautological fact.
David Kretzmer
@david-k_gitlab
Okay, if you think about it as a graph reachability problem, then this makes sense to me. But when thinking about dataflow functions (and this seems to be the level of abstraction of AbstractIFDSAnalysis) then I don't see why a tautological fact is needed. I probably haven't really understood how this all works, so I will do some more reading and thinking and then ask a more concrete question if needed. Thanks!
Mehdi
@ashkboos

Hi- I am using:

     <artifactId>bytecode-representation_2.11</artifactId>
     <version>0.8.15</version>

as a maven dependency in a java application. It's working on tests, IDE and jar file with a specified classpath, but while running my application as a runnable jar-with-dependencies it throws this error No configuration setting found for key 'org.opalj.analysisMode'. Please let me know if anyone knows how to fix this.

Marcel Bruch
@MarcelBruch
Hi, I wanted to give Opal a spin (after years) :-) The website says that the latest version is 3.0.0 and can be downloaded from mvn central. However, the latest version returned by https://search.maven.org/search?q=de.opal-project is 2.0.1. What's the latest version? :-) Thx!
Florian Kübler
@florian-kuebler
@MarcelBruch Sorry for the late answer. Unfortunately, there is no 3.0 release on Maven central yet. The most recent version (which is indeed 3.0.) is only a Snapshot version and available in the sonatype snapshot repo.
We are right now working hard on doing an official release on maven central, but this will probable not happen within this month. We try to keep you all updated about the progress. Also note, that we moved the repository to GitHub (https://github.com/stg-tud/opal).
Florian Kübler
@florian-kuebler

@ashkboos I am very sorry, that you message got lost. The issue that you describe, is connected with OPAL’s configuration framework (we use TypeSafe Config here). All projects provide default configurations in the reference.conffiles, which seem to be not found in your runnable jar.

We do not actively support 0.8.15 anymore, and can not easily provide a fix for this. I’d suggest to either move to 2.0 or the 3.0 Snapshot. If the issue still occurr, it would be nice if you could create an issue (with steps to reproduce it) on GitHub.

Mehdi
@ashkboos
@florian-kuebler thank you so much for the answer. Is it possible to release the 3.0 on maven? also, I think there is a problem with the documentation on the OPAL website I can not open the scala-doc.
Florian Kübler
@florian-kuebler

@ashkboos We are currently working hard on a new release (which will be available on maven). As soon as the release goes out, we will post this here.

Also thanks for the issue with the scala-doc. We will have a look and fix this asap.

Florian Kübler
@florian-kuebler
The scala-doc issue should be fixed now.
Mehdi
@ashkboos
Thanks a lot.
Mehdi
@ashkboos
Hi, I want to extract the Invocation instruction of each edge from OPAL 1.0.0 call graphs in java. I use the method "CallGraph().calls(Method)" to get the edges and then I search among the caller's instructions for the callee's class, name, and descriptor. Is there a better(easier) way to do so? I couldn't find by searching.
Florian Kübler
@florian-kuebler
Hi @ashkboos, unfortunately, OPAL’s call graphs have been subject to massive and frequent change (including several complete reimplementations) with the last releases. In the current Implementation, there is a callGraph.callersOf(method), that returns a Traversable[(DeclaredMethod, Int, Boolean)], where the Int corresponds the program counter of the instruction, that leads to the call and the Boolean indecates, if this call is “direct”, i.e. results from a invocation instruction, or “indirect”, e.g. resulting from an reflective call. Having, the Bytecode of a method, you can then directly use the programcounter to lookup the instruction. As I am not very familliar with the “old” API, I am not sure, whether the program counter was also given here.
Mehdi
@ashkboos
@florian-kuebler, Thanks. Yes, you are right in the old API, the callee has a key(PC?) that I used as the index of the instruction array of the caller and it seems it's working.
Florian Kübler
@florian-kuebler
@ashkboos good to hear that. If there is anything still unclear, just keep us posted.
roterEmil
@roterEmil
[FPCF] Hi, is there a chance to create two different propertystores in one ProjectAnalysisApplication in order to compare them.
Florian Kübler
@florian-kuebler
Hi, you can use project.recreate to get a copy of the project, and then use project.get(PropertyStoreKey). Within the recreate method, you can also specify keys, that should not be recreated (e.g. if you want to have the same DeclaredMethods)
To specify the propertyStore:
project.getOrCreateProjectInformationKeyInitializationData(
            PropertyStoreKey,
            (context: List[PropertyStoreContext[AnyRef]]) ⇒ {
                implicit val lg: LogContext = project.logContext
                org.opalj.fpcf.par.PKECPropertyStore(context: _*)
                //org.opalj.fpcf.seq.PKESequentialPropertyStore(context: _*)
            }
        )
Mehdi
@ashkboos
Hello, I am trying to replicate the results of Judge paper. I want to get the maximum number of sound call graphs (102) reported in the Judge paper from JCG test cases using OPAL version 3. I used the Sonatype snapshot and develop branch of the OPAL repository. The maximum number of sound call graphs that I created was 57. I tried different configs like library analysis and application analysis, analyzing the JDK, and not analyzing it. Anyway, I don't know what should be the classpath for each test case. Would you please help me to reach a higher number? Am I missing something?
Michael Reif
@mreif
Hi @ashkboos, that's unfortunate. Can you please say more about the version you used? Is it OPAL's current develop branch or the commit that's stated in the paper? Since OPAL is still under development, especially w.r.t. to call graphs, and how they are built it might be the case that Judge's OPAL adapter is out-of-date. I'll look into this. In the meanwhile, feel free to checkout our the docker container https://hub.docker.com/repository/docker/mreif/jcg
Mehdi
@ashkboos
@mreif Thanks for the response, I have tried both current develop branch and version 3 snapshots on sonatype (https://oss.sonatype.org/content/repositories/snapshots/de/opal-project/), both worked but can not get the exact number of sound CGs with them. I have also built the stated OPAL commit in the paper, although OPAL was built successfully, I have faced errors like object cg is not a member of package org.opalj.br.fpcf.properties while building the JCG over afterward. I will also take a look at the docker container, thanks.
Vincent Paul
@Vangle_gitlab
Hi, https://www.opal-project.de/tools/bugpicker/index.php is broken... I emailed eichberg(at)informatik.tu-darmstadt.de as mentioned on https://www.opal-project.de/ but the email address seems to be not working...
Dominik Helm
@errt
Hi @Vangle_gitlab, thank you for pointing this out, we will have a look at why the link is dead. Also, as you correctly noted, the email address is outdated as well and we will change this.
6 replies
Mehdi
@ashkboos

Hi, I am trying to generate call graphs using the latest commits of the develop branch. I am using OPAL as a maven library in a java application. I first load the project Project.apply(file) and then get the call graph project.get(RTACallGraphKey$.MODULE$). But the number of reachable methods in call graphs differs in different runs. However, for each call graph, it always returns only two specific numbers. I guess it's because of OPA or CPA analysis mode, probably it sometimes does OPA and some times CPA. I have tried the followings to fix it but didn't work:

  • Project.apply(files) plus adding a reference.conf in the resources specifying the InitialEntryPointKey
  • Project.apply(application files, library files)
  • Project.apply(files, logContext, config) config is loaded with library entrypoint finder or application.

I really appreciate any help.

2 replies