Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 22 18:43
    gitoleg opened #1036
  • Jan 22 18:41
    gitoleg opened #1035
  • Jan 22 18:40
    gitoleg opened #1034
  • Jan 22 18:40
    gitoleg opened #1033
  • Jan 22 18:38
    gitoleg opened #1032
  • Jan 21 22:30
    gitoleg labeled #957
  • Jan 21 22:30
    gitoleg review_requested #957
  • Jan 21 20:35
    gitoleg synchronize #957
  • Jan 06 12:38
    ivg opened #1031
  • Dec 17 2019 19:29
    ivg assigned #1030
  • Dec 17 2019 19:29
    ivg opened #1030
  • Dec 16 2019 17:46

    ivg on master

    adds the list of supported arch… (compare)

  • Dec 16 2019 14:37

    ivg on switch-to-omake

    (compare)

  • Dec 13 2019 23:04

    ivg on switch-to-omake

    fixes library dependencies The… adds OMakefiles They describe … cleans up the main Makefile and 7 more (compare)

  • Dec 10 2019 19:54
    gitoleg synchronize #957
  • Dec 10 2019 18:41
    gitoleg synchronize #957
  • Dec 10 2019 16:12
    gitoleg synchronize #957
  • Dec 09 2019 22:31
    ivg closed #1029
  • Dec 09 2019 21:32
    mjambon opened #1029
  • Dec 06 2019 22:15
    ivg synchronize #1028
Enkelmann
@Enkelmann
As I had the pleasure of learning Ocaml just to learn BAP, I have to disagree. Compared to other languages Ocaml is not easy to learn. Especially if you don't have an experienced Ocaml programmer sitting next to you and telling you how to get sensible error messages out of the compiler.
But speaking of other languages: Do the Rust bindings for BAP still exist? Last time I looked, they did not seem to be actively maintained and right now I cannot find them anymore.
Ivan Gotovchits
@ivg

Well, if you compare with languages that have dynamic types, then yes. For example, Python lets you right total garbage that looks like a Python program and only in runtime tells you that it is not well-formed. In that sense, OCaml is much more strict, and won't accept your program unless it type checks.

If you will compare with statically typed languages, then I don't believe that C++ type errors are any better. Or , even, rust errors.
Especially now, when error messages got improved with each new release of OCaml. And we also have merlin, so you do not postpone error checking until it is hard to understand. With merlin is much easier now. You type new code and if merlin highlights it, then something is wrong with your code. Usually, I don't even bother myself reading the error message. It's like with the spellchecker, once it highlights you see that the word is wrong.

But even today, some people are trying to use OCaml without Merlin - and this brings us back to the infrastructure problem.

Ivan Gotovchits
@ivg

Concerning Rust bindings, they do exist, and we indeed didn't update them to the latest 2.0 interface. There is only one function that needs to be added/updated bap_init, @gitoleg will do this. Meantime, I will ask Matt to transfer the repo to us, so that it will be easier for people to find them (and so that they look more maitained).

Update: added the link

Enkelmann
@Enkelmann
Ah, thank you for the link! And good to know that they are / will be actively maintained. :-)
Enkelmann
@Enkelmann
Comparing error messages between Rust and Ocaml: Rust enforces that you have to annotate the type of all function parameters. Ocaml does not. But if you don't, Merlin's error messages often will not be in the line where the actual error occured. This really doesn't help if you are new to the language and it is probably one of the reasons why Rust simply forces you to annotate these types.
Ivan Gotovchits
@ivg

Merlin is incremental, it types as you types. So when you start writing a program it will immediately show you the error in the place where you type. Like if you write

let example () = 
   List.map (fun x -> x + 1) ["hello"]

it will immediately highlight "hello" saying This expression has type string but an expression was expected of type int. So merlin errors will be always at the place where the error occurs when you develop a program. If you take an already broken program and then ask Merlin what went wrong, then yes, your comment is applicable.

But, it is essential, not to postpone type checking, the same as with testing and integration. The sooner you type check the better.
Concerning annotations. You're not enforced to use them but highly encouraged. I usually pack all my functions into many small modules, each sealed with an interface. This limits both the scope of the typing and the scope that you have to reason about.
To summarize - small functions, packed into small modules, these are our building blocks. The smaller the block the easier it is to handle it.

FactionCube
@FactionCube
I too have been learning OCaml for the sake of using BAP. I think it's the first programming experience I've had where I've also jumped into the theoretical side to a greater than usual extent, to understand Lambda calculus in order to see how functional styles differ from Turing based languages. I do feel that I have to work harder to write code in OCaml, but it has been satisfying so far; it's just a learning curve to accustom myself to the OCaml idioms. Doing the FUN-MOOC course has been wonderful for keeping my nose to the grindstone in coming to grips with the language. Now - if only there were a course just like that for learning BAP...
Alastair White
@airxhi

Hi! I'm doing a project that involves running the disassembly of a program, I've been trying to get of_file to work but it errors with input file not supported by the specified loader after running

match of_file "/bin/ls" with

Any ideas what I could be doing wrong?
Thanks!

Alastair White
@airxhi

^ I encounter the same issue when using Image.create and then using of_image.

I've also tried using gcc compiled executables and object files with no luck.

Anton Kochkov
@XVilka
@ivg would be nice to add Rust bindings to the CI (and Python bindings too, if not yet) to check them on PRs/master pushes
Ivan Gotovchits
@ivg
@airxhi, looks like you didn't load plugins. Which version are you using?
And, I got a little bit I'll, so forgive me, if I will be responding with a delay :)
Ivan Gotovchits
@ivg

Also, it is usually a good idea to employ the bap utility and write your analysis as a plugin to it. For example, you can write a disassembly pass and get your function called on the disassembled file without having the burden of specifying all the parts manually. E.g., if you will write a simple program,

open Bap_main

let main proj = ()

let () = Extension.declare @@ fun _ctxt -> 
   Project.register_pass' main

the main function will be called with the project data structure that includes all the information associated with the disassembled target, e.g., Project.program proj will contain the disassembled program in IR, Project.disasm proj will be the disassembly. You can also obtain the image and other properties, if needed.

You can also consider our bap-tutorial for more in-depth introduction
https://github.com/BinaryAnalysisPlatform/bap-tutorial
Alastair White
@airxhi

I'm using bap 2.0.0, I'm wanting to call the bap disassembler (and maybe more) from inside a large pre-existing ocaml project. My main goals is to get the list of instruction names, parameters and labels to convert to a representation in this other project. i.e. insns produces type (Bap.Std.mem * Bap.Std.insn) Bap.Std.seq which would be ideal for my use case.

Is my plugin issue mainly that I need to init Project to initialize the correct global state? I'm not sure how well the plugin style dev will work for my use case (I can't use bapbuild etc and have to work with ocamlfind).

Thanks for all of the feedback :)

Ah, I just saw

register_pass' pass registers pass that doesn't modify the project effect and is run only for side effect. (See register_pass)

Think I might just need to do a bit more reading through the docs.

Ivan Gotovchits
@ivg
Before doing anything with BAP you need to initialize it. The Bap_main library is the entry point to BAP. http://binaryanalysisplatform.github.io/bap/api/master/bap-main/Bap_main/index.html
Read carefully the embedding section, especially the warning part. Other than the warning part it is just a simple call to Bap_main.init ()
As a side question, I wonder why can't you use bapbuild?
Benno Fünfstück
@bennofs
hi, I'm trying to use the bap frames tracers (bap-qemu and bap-pintraces), do you have some pointers about what environment I need to have to make them work?
I currently try on centos 7 (it has an old enough kernel for pintools 2) but pin still segfaults
also the qemu tracewrap produces a trace that readtrace cannot read
terminate called after throwing an instance of 'SerializedTrace::TraceException'
  what():  Unable to parse from string
Daniel Peters
@dtpeters
hey all I'm getting a weird error, where opam can't find bap "ERROR no solution for bap"
aaand I just realized its like 5:37 in the morning
Ivan Gotovchits
@ivg
@bennofs, We were using Ubuntu Trusty (and probably Xenial). And some particular version for Intel pintool. We're you following build instructions or had some deviations?
@dtpeters, ouch, it is better to start from a clean switch in that case. Basically, you have installed packages in your switch that are not compatible with BAP, or the compiler version is not.
Benno Fünfstück
@bennofs
the instructions from http://hacktracking.blogspot.com/2016/10/pin-and-triton-binary-instrumentation.html worked, you need to use ubuntu-14.04.1-server-amd64.iso, at least for getting pin to work
Ivan Gotovchits
@ivg
Good! Would you mind updating our readmes? I will welcome PR very welcome :) There is no need to be detailed, you can just add this link there as a small comment, but if you wish to provide more in depth explanations I will be very happy too) Especially since eventually I will be following your road)
Ivan Gotovchits
@ivg
saw the PR, thanks!, you've saved a lot of hours for those who will follow)
richinseattle
@richinseattle
@bennofs the pintracer works on ubuntu14 and debian8 installs
I do not know the current state of support for loading those traces into bap 2.0 and would be curious to know
I’d like to port our slicer asap
Ivan Gotovchits
@ivg

I do not know the current state of support for loading those traces into bap 2.0 and would be curious to know

This traces could be loaded and processed in OCaml using the trace plugin and the Bap_traces library. We're also planning to release after the holidays, code that enables support for old versions of Bil, e.g., those that were used in BAP 0.8.x, this should also help you in porting the old code :)

richinseattle
@richinseattle
Cool
Alastair White
@airxhi

Hi again,

@ivg bapbuild is used for building and installing plugins right? I'm trying to use a subset of bap's features as part of an interpreter to do some experimental things, so I'm not sure loading the extension as a plugin will work...

And even after paying attention to the warning in the Bap_main.init () I can't seem to call it without my program seg faulting.

I've managed to compiler a test file with ocamlfind ocamlopt -o out -linkpkg bap -package Findlib.dynload and not seg faulted. But when compiling the main project to lots of .cmx and then linking the resulting program segfaults.

I'm using these options

ocamlfind ocamlopt -bin-annot -g -I extraction -I lib -I common -I x86 -I backend -I cfrontend -I cparser -I driver -I exportclight -I debug -I <Menhirlib> -w +a-4-9-27 -strict-sequence -safe-string -package Findlib.dynload -linkpkg -package bap <lots of .cmx>

In case any of those might be interfering.

(I've tried using the Findlib.record_package function but I'm not sure what arguments to call it with lol

Ivan Gotovchits
@ivg
1) findlib.dynload is typed without capitalization (though if you're on mac it might not matter, or might do)
2) to link with bap you just need to use -package bap and do Bap_main.init you shall not specify any cmx from bap or anything like this
3) if your -I options refer to you own subfolders it could be very possible, that we have a name conflict in some compilation unit. Or that you have a conflict with the compiler-libs. In general, it is a good idea to prefix all your compilation units (ml files) with the name of your project, so not parser.ml but my_project_parser.ml (you can use dune, it will handle this mangling for you)
4) look into the log file at ~/.local/state/bap/log, it will tell what units and libraries it links.
5) you can drop me a link to your project, I might see if anything else is wrong
6) you can use a newer version of a compiler, that has a fixed loader, it will tell you the name of the offending module.
(I honestly don't remember in which version they fixed it, and it might be that we don't yet support it, at least in the master branch)
Ivan Gotovchits
@ivg
update, looks like it is 4.08
Alastair White
@airxhi

Hi

1) Woops thats a mistype in my gitter message, the build script uses findlib.dynload
2) yah, the cmx's are linking with all of the other cmx files at the build stage, I'm adding -linkpkg -package bap and package findlib.dynload to the ocamlfind options
3) Could be, will investigate
4) Will have a look
5) If 4 and 5 don't reveal anything, will do.
6) I'm using 4.06.1, will be a pain to update but might come to it

Thanks so much for the help!!

Alastair White
@airxhi
For stage 4) to use the findlib function should I call Findlib.record_package Findlib.Record_load "<path_to_plugin>" inside the file I wish to embed bap or am I way off track?
Ivan Gotovchits
@ivg
No, you shouldn't, this is done for you by the findlib.dynload
However, findlib.dynload may only detect compilation units that were introduced by packages, that are linked with the -package option. It won't track your local libraries specified on the command line via *.cmx[,a,s] files.
Ivan Gotovchits
@ivg

The rule is simple, there shouldn't be any compilation units with the same name. So most likely the conflict comes from some of your internal. It is unlikely that it conflicts with some bap module, as we prefix all our modules (unless you also prefixed your module with bap_<xxx>). I think the conflict is with some external library. Again, look at the log file it may limit our search space.

Also, make sure that you don't have modules named parser.* or lexer.* and other generic modules. Ideally, prefix all your modules with your namespace. It has nothing to do with bap in general, it is the limitation of OCaml linking scheme, just bap brings a lot of libraries to the namespace, so it increases the possibility of the conflict.

Also, just a long shot, I see you have some cparser and cfronted, and it could be that you have vendored FrontC, i.e., you have your own copy of frontc inside. This will definitely come into a confict with the frontc library from opam. So if it is true, rename the files, or use the upstream frontc if possible.

shamila wickramasuriya
@visitWicky_twitter
Is there a Dockerfile for ubuntu latest (18.04.3)?
gitoleg
@gitoleg
@visitWicky_twitter hey, there is no in bap repository, but you can use this one:
FROM ocaml/opam2:ubuntu-18.04

WORKDIR /home/opam

RUN sudo apt-get update \
 && sudo apt-get install libncurses5-dev --yes \
 && opam repo add bap git://github.com/BinaryAnalysisPlatform/opam-repository#testing --all \
 && opam switch 4.07 \
 && eval "$(opam env)" \
 && opam update \
 && opam depext --install bap --yes \
 && opam clean -acrs \
 && rm -rf /home/opam/.opam/4.0[2-6,8,9] \
 && rm -rf /home/opam/.opam/4.07/.opam-switch/sources/* \
 && rm -rf /home/opam/opam-repository

ENTRYPOINT ["opam", "config", "exec", "--"]
sjcappella
@sjcappella
Hi there. Awesome work on the 2.0.0 release! I finally got around to testing my project with the updated version, and I'm noticing some weirdness around expression normalization. I'm using the C bindings and before, on BAP 1.6.0, I would normalize let expressions. IN 1.6.0 this worked as expected and let was removed during the normalization. Now I'm seeing no change from the before and after normalization. Same goes for BIL normalization, regardless of BN1 or BN2. Was I suppose to do anything different?
Ivan Gotovchits
@ivg
Hi @sjcappella, thanks for the kind words! Indeed, a quick glimpse into the normalization procedure reveals that we disabled let-normalization in BNF2 (probably by accident, during our experimenting with floating-point operations). We definitely need to restore this normalization (and update our documentation on normalization). Not sure yet, whether we will restore it as a part of BNF2 or split it into two subforms. I will create an issue to track this.
sjcappella
@sjcappella
Thanks for checking on that and confirming! I personally like the idea of let normalization being it's own form, but that's just me :)