Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
    Fabian Yamaguchi
    Simon Koch
    yup sehe ich
    Simon Koch
    @fabsx00 ich bin gerade dabei die File Nodes einzubauen. Dabei sind mir ein paar Fragen gekommen, die ich so gestern nicht auf dem Schirm hatte:
    (1) mit neo4j konnte man sich die Graphen im Browser anzeigen lassen um Teile des Graphen zu erforschen. Gibt es für die overflowdb eine vergleichbare Möglichkeit?
    (2) Es ist keine Kante gelistet mit der man Ordnerstrukturen representieren kann? Ist dieses Feature gedropped worden seit dem alten Phpjoern?
    (3) Ich würde gerne auch direkt anfangen Tests mit meinem Quellcode zu schreiben um nichts zu pushen, dass nicht funktioniert. Soll ich selber die Testordnerstruktur anlegen anhand von fuzzyc2cpg oder ist dies Teil des Code den du noch hochladen wirst?
    Fabian Yamaguchi
    Let me answer in English, so that everyone here + future users have a better chance of understanding what's going on :) (1) overflowdb doesn't have a graph visualizer, but we've typically just dumped dot. For AST, CFG, and DDG, I've added plotting utilities over time, e.g., cfg.method.plotDotAst will plot the AST for you (2) yes, we dropped directory nodes. (3) I added some boilerplate yesterday and I added the test framework but haven't created a sample test yet. We'll need to create a small test fixture but that will depend on whether you're using fastparse or something else.
    Simon Koch
    (1) ok, so basic plotting is available but it is not as interactive?
    (3) I think it is going to be fastparse. I am starting to get the hang of fastparse and am already able to parse a minor subset of possible output files. This is the point where I would like to be able to write some tests as interactive testing using the repl proofs rather cumbersome at the moment.
    Niko Schmidt
    @simkoc (1) I would recommend you to check the list below Interactive viewers. on https://graphviz.gitlab.io/resources/ :)
    Simon Koch
    @fabsx00 I have implemented basic creation of file nodes. Furthermore I wrote a basic parser for the php bytecode dumps and we are now able to parse and theoretically process said dumps. However, this is the point where I am stuck as I am not entirely sure I understand the CPG specs concerning method bodies, i.e., how do I represent simple statements (send_val) or complex ones (assignments) that are not calls? There seems to be no statement node.
    The basic php program we are already able to parse is in ./resources/unittesting/trivial-php.php.

    hi, i'm trying to load CPGs directly using the code in the codepropertygraph repo (w/o joern). i checked regenerate-test-cpgs.sh for how it creates the java CPGs and adapted the command so it would create a C CPG. i used one of the joern examples:

    $ ./fuzzyc2cpg.sh /home/christoph/.applications/x42/c/ -o x42.bin.zip


    SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
    SLF4J: Defaulting to no-operation (NOP) logger implementation
    SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

    this created a file 16 kB in size. i then tried to feed it into semanticcpg:

    $ sbt semanticcpg/console
    import io.shiftleft.codepropertygraph.Cpg
    import io.shiftleft.semanticcpg.language._
    val cpg = io.shiftleft.codepropertygraph.cpgloading.CpgLoader.load("./x42.bin.zip")

    however, it fails:

    java.util.zip.ZipError: zip END header not found
      at com.sun.nio.zipfs.ZipFileSystem.zerror(ZipFileSystem.java:1651)
      at com.sun.nio.zipfs.ZipFileSystem.findEND(ZipFileSystem.java:1040)
      at com.sun.nio.zipfs.ZipFileSystem.initCEN(ZipFileSystem.java:1049)
      at com.sun.nio.zipfs.ZipFileSystem.<init>(ZipFileSystem.java:130)
      at com.sun.nio.zipfs.ZipFileSystemProvider.newFileSystem(ZipFileSystemProvider.java:139)
      at java.nio.file.FileSystems.newFileSystem(FileSystems.java:390)
      at io.shiftleft.codepropertygraph.cpgloading.ZipArchive.<init>(ZipArchive.scala:12)
      at io.shiftleft.codepropertygraph.cpgloading.ProtoCpgLoader$.$anonfun$loadFromProtoZip$2(ProtoCpgLoader.scala:28)
      at io.shiftleft.codepropertygraph.cpgloading.ProtoCpgLoader$.$anonfun$loadFromProtoZip$2$adapted(ProtoCpgLoader.scala:26)
      at scala.util.Using$Manager.scala$util$Using$Manager$$manage(Using.scala:159)
      at scala.util.Using$Manager$.apply(Using.scala:212)
      at io.shiftleft.codepropertygraph.cpgloading.ProtoCpgLoader$.$anonfun$loadFromProtoZip$1(ProtoCpgLoader.scala:26)
      at io.shiftleft.codepropertygraph.cpgloading.ProtoCpgLoader$.measureAndReport(ProtoCpgLoader.scala:82)
      at io.shiftleft.codepropertygraph.cpgloading.ProtoCpgLoader$.loadFromProtoZip(ProtoCpgLoader.scala:24)
      at io.shiftleft.codepropertygraph.cpgloading.CpgLoader.load(CpgLoader.scala:93)
      at io.shiftleft.codepropertygraph.cpgloading.CpgLoader$.load(CpgLoader.scala:23)
      ... 36 elided

    i tried zipping the file manually and read through the code that reads the CPG but i cannot figure out how to open this file


    i now looked at joern instead and managed to load a CPG

    $ sbt fuzzyc2cpg/console
    import io.shiftleft.fuzzyc2cpg.FuzzyC2Cpg

    which created a file named cpg.bin.zip

    $ sbt semanticcpg/console
    import io.shiftleft.codepropertygraph.cpgloading.CpgLoaderConfig
    val filename = "./cpg2.bin.zip"
    val odbConfig = overflowdb.Config.withDefaults().withStorageLocation(filename)
    val config = CpgLoaderConfig().withOverflowConfig(odbConfig).doNotCreateIndexesOnLoad
    val cpg = io.shiftleft.codepropertygraph.cpgloading.CpgLoader.loadFromOverflowDb(config)

    but i do not understand why the method described above in my question does not work

    sorry, that file name includes a typo and i cannot edit the message anymore. "./cpg2.bin.zip" should be "./cpg.bin.zip"