Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 06 19:42
    ivg opened #1562
  • Dec 03 01:56

    github-actions[bot] on master

    1077911 (compare)

  • Dec 03 01:07

    github-actions[bot] on v2.6.0-alpha

    (compare)

  • Nov 26 02:04

    github-actions[bot] on master

    1077911 (compare)

  • Nov 26 01:12

    github-actions[bot] on v2.6.0-alpha

    (compare)

  • Nov 22 05:05
    andrewj-brown synchronize #1546
  • Nov 19 02:21

    github-actions[bot] on master

    1077911 (compare)

  • Nov 19 01:20

    github-actions[bot] on v2.6.0-alpha

    (compare)

  • Nov 14 15:16
    kit-ty-kate opened #1561
  • Nov 12 02:30

    github-actions[bot] on master

    1077911 (compare)

  • Nov 12 01:04

    github-actions[bot] on v2.6.0-alpha

    Corrected primus_dictionary_mai… Fix not propagating term attrs … (compare)

  • Nov 10 21:03
    bmourad01 synchronize #1559
  • Nov 10 20:14
    bmourad01 opened #1559
  • Nov 10 19:58

    ivg on master

    Fix not propagating term attrs … (compare)

  • Nov 10 19:58
    ivg closed #1556
  • Nov 08 18:37
    ivg closed #1558
  • Nov 08 18:36

    ivg on master

    Corrected primus_dictionary_mai… (compare)

  • Nov 05 02:25

    github-actions[bot] on master

    77d09fe (compare)

  • Oct 31 17:22
    A-Benlolo opened #1558
  • Oct 29 02:37

    github-actions[bot] on master

    77d09fe (compare)

Benjamin Mourad
@bmourad01
@ivg I presume you're familiar with the revng project. Do you see problems with their approach to lifting to LLVM?
Kenneth Adam Miller
@KennethAdamMiller
Hey, I was thinking about that too. But there's one by microsoft and one by trail of bits and some others too.
Kenneth Adam Miller
@KennethAdamMiller
Does anyone know if any lifter supports kernel objects?
Ivan Gotovchits
@ivg
bap supports
Kenneth Adam Miller
@KennethAdamMiller
llvm can compile for kernel objects right?
Ivan Gotovchits
@ivg
it may but usually they use gcc
if we're speaking about linux
in macOS iirc they use clang
Kenneth Adam Miller
@KennethAdamMiller
Ok, thank you
Ivan Gotovchits
@ivg
@bmourad01, I don't know the low-level details of the translation and don't have any hands-on experience with their tools, so it is hard to judge. A few comments though. At least the way I see this project, please correct me if I am wrong, they focus on binary instrumentation to do dynamic analysis, mostly via fuzzing an instrumented program. They do not perform static analysis on the original binary and when they do any analysis they in fact analyze an emulator of the original binary, as they reify the CPU state into a C structure. It also looks like that they translate binary to LLVM IR via QEMU TCG on per instruction basis. Not on the whole-program level, nor even on a subroutine or block level. At least this is what I have picked from the papers and talks around. I would be interested to know more, as I still think that reusing LLVM (or GCC or some other compiler framework) is much more productive than writing your own instruction selector.
Kenneth Adam Miller
@KennethAdamMiller
I had a lot of trouble with the rule that llvm IR imposes. I think it is difficult to go directly from BIR to LLVM IR, at least not without further algorithm or processing to try to reach the demand of the target. But I think there exist ways around the problem that bypass the need to target going so far up. At the same time I think that this approach is good for certain applications, I think that it is probably bad for the CMU's team goal of lifting to higher languages. hmmm.
4 replies
cL0und
@cL0und

Hi I meet an error about "ocamlfind: Package `bap-main' not found" when I try to make bap-toolkit.Cloud anyone help me?

root@105c05d3d265:~/bap-toolkit# eval $(opam env)
[WARNING] Running as root is not recommended
root@105c05d3d265:~/bap-toolkit# make
sh build.sh clean
sh build.sh build
Entering directory av-rule-17' Leaving directoryav-rule-17'
Entering directory `av-rule-174'
bapbuild -pkgs bap,bap-main,bap-primus,bap-x86-cpu,bap-taint null_ptr_deref.plugin

  • ocamlfind ocamldep -package bap-taint -package bap-x86-cpu -package bap-primus -package bap-main -package bap -package ppx_jane -package core_kernel -pp 'ppx-jane -dump-ast -inline-test-drop' -predicates custom_ppx -predicates ppx_driver -modules null_ptr_deref.ml > null_ptr_deref.ml.depends
    ocamlfind: Package `bap-main' not found
    Command exited with code 2.
    Compilation unsuccessful after building 1 target (0 cached) in 00:00:00.
    Makefile:3: recipe for target 'build' failed
    make: * [build] Error 10
    root@105c05d3d265:~/bap-toolkit# bap --version
    1.4.0
My opam version is 2.1.0.
Ivan Gotovchits
@ivg
@cL0und, how did you install bap?
Ivan Gotovchits
@ivg

@ImanHosseini, I saw your message popping up, so I believe you answered to the thread (it is nearly impossible to find the thread in gitter), so answering here.

This option is having a weird syntax (which is mostly undocumented), the last time I used it I had to look into the compiler sources. Basically, it should be something like OCAMLPARAM="_,S=1", but let me double-check.

Ivan Gotovchits
@ivg

Yep, I checked it works,

$ OCAMLPARAM="_,S=1" ocamlbuild w.native
/home/ivg/.opam/4.09.0/bin/ocamldep.opt -modules w.ml > w.ml.depends
/home/ivg/.opam/4.09.0/bin/ocamlc.opt -c -o w.cmo w.ml
/home/ivg/.opam/4.09.0/bin/ocamlopt.opt -c -o w.cmx w.ml
/home/ivg/.opam/4.09.0/bin/ocamlopt.opt w.cmx -o w.native
$ ls _build/
_digests  _log  w.cmi  w.cmo  w.cmx  w.ml  w.ml.depends  w.native  w.o  w.s

Also, if you mess up with the OCAMLPARAM syntax then it will just spill a warning (which will land up somewhere inside of opam logs), e.g.,

$ OCAMLPARAM="S" ocamlbuild w.native
/home/ivg/.opam/4.09.0/bin/ocamldep.opt -modules w.ml > w.ml.depends
+ /home/ivg/.opam/4.09.0/bin/ocamldep.opt -modules w.ml > w.ml.depends
File "_none_", line 1:
Warning 46: illegal environment variable OCAMLPARAM : missing '=' in S
File "_none_", line 1:
For the context, we were discussing with @ImanHosseini how to force opam to generate *.s files fore the compiled code and in general how to globally pass a flag to the compiler without modifying any build scripts.
cL0und
@cL0und

@cL0und, how did you install bap?

I installed it by opam.

Ivan Gotovchits
@ivg

I think that the culprit is,

# eval $(opam env)
[WARNING] Running as root is not recommended

though it might be a red herring. Can you tell me the output of the following commands?

ocamlfind list | grep bap-main
which bapbuild
cL0und
@cL0und
Sorry I stay at home now, and can not connect my server.I might have to give it to you tomorrow.Because I execute it in docker so the default role is root.
Ivan Gotovchits
@ivg
yep, I get it, besides, have you heard that we already have a docker image that is built every day?
also, there is a dockerfile in the toolkit repo that uses it
cL0und
@cL0und
I don't know, however the container which I used contains a more complex environment which I hardly install manually, so I prone to use current docker. I will refer to this dockerfile. Thanks for your prompt reply:)
Philip Zucker
@philzook58

I'm going back through your talk and trying to understand. You show a type type 'a t = U : 'a * 'd k * 'd -> 'd t. This type is pretty confusing to me. As I understand it, it is equivalent to type _ t = U : 'a * 'd k * 'd -> 'd t meaning 'a is existential and hidden. In the example of interpreting your arithmetic expressions to strings, which I would think would look something like this in finally tagless with the type t in the module

module StringAExpr : S = struct
  type 'a t = string
  let int x = string_of_int x
  let (+) x y = sprintf "%s + %s" x y 
  let ite c t e = sprintf "if %s then %s else %s" c t e
end

How does this look using your type 'a t kept external to the module? Is the type variable 'a a string? How do you write (+) if so since this type stays existential?

cL0und
@cL0und

I think that the culprit is,

# eval $(opam env)
[WARNING] Running as root is not recommended

though it might be a red herring. Can you tell me the output of the following commands?

ocamlfind list | grep bap-main
which bapbuild

root@105c05d3d265:/# camlfind list | grep bap-main
bash: camlfind: command not found
root@105c05d3d265:/# which bapbuild
/usr/local/bin/bapbuild

I used a temp vpn and connected to server just now. ;)
Ivan Gotovchits
@ivg

@philzook58,
Okay, it is an error, the correct type should be

type _ t = U : 'a * 'd k * 'd -> ''a t

So 'd is the data type of the representation, which is kept existential, and 'a denotes the sort of the value, i.e., some static information about it. (In KB parlance it is either KB.Value.sort or KB.Effect.sort.

Concerning the second part of the question, the magic is in the 'a k type which is a type witness (which is hidden in the slot type in the KB implementation, that gives you functions like KB.Value.get and KB.Value.put).

So the key interface is something like,

val get : 'd k -> 's t -> 'd
val put : 'd k -> 's t  -> 'd -> 's t

So that the StringAExpr which denotes expressions as pretty strings is,

let dom : string k = ...

module StringAExpr : S = struct
   let int x = put dom (string_of_int (get dom x))
   ...

So the key idea is that we represent abstract types of each denotation with a single existential type. Basically, and extensible variant.

@cL0und, it is ocamlfind not camlfind :)

and the location of bapbuild is very strange, I think that you didn't install bap from opam, or you have two installations of bap. If you would install bap from opam then you will have bap installed in the opam switch, as opam never installs in the system folder.

My hypothesis is that you have installed bap from the debian package, which indeed installs to the /usr/local prefix. But the debian package doesn't install the dev files that are necessary to build the toolkit, so you have to do opam install bap

Ivan Gotovchits
@ivg
A yet another indicator that corraborates my hypothesis is that you bap --version returns 2.4.0, which is what the released debian package will do. If you would install bap from opam (or used bap from opam), the version will be either 2.3.0 (which is the latest publically released version), or 2.4.0-alpha+<sha>, e.g., 2.4.0-alpha+18eb123
cL0und
@cL0und

and the location of bapbuild is very strange, I think that you didn't install bap from opam, or you have two installations of bap. If you would install bap from opam then you will have bap installed in the opam switch, as opam never installs in the system folder.

My hypothesis is that you have installed bap from the debian package, which indeed installs to the /usr/local prefix. But the debian package doesn't install the dev files that are necessary to build the toolkit, so you have to do opam install bap

In fact, I install it by debian package at frist time,but I find it can not install plugin successfully,so i chose install it by opam.

root@105c05d3d265:/# ocamlfind list | grep bap-main
findlib: [WARNING] cannot read directory /usr/lib/ocaml/METAS: No such file or directory
root@105c05d3d265:/# opam install bap
[WARNING] Running as root is not recommended
[NOTE] Package bap is already installed (current version is 1.4.0).
Ivan Gotovchits
@ivg
ouuch, 1.4.0
it is not 2.4.0)
it is like 10 years ago)) I even forgot such numbers))) Not surprised that I didn't notice it
cL0und
@cL0und
yeah the debian package version is 2.4.0 ,but the opam version is just 1.4.0.
Ivan Gotovchits
@ivg
So it is funny coincidence of bugs))

You have two installations of bap, an acient bap installed from a prehistoric opam, and the newest bap installed from the deb packages.

Since you're using the new syntax for the old opam, namely eval $(opam env) it doesn't activate opam (in ye olde times it was eval $(opam config env) IIRC). So you still get the debian-installed bap.

cL0und
@cL0und
It seems like complex, but I think my opam is the newest version.
root@105c05d3d265:/# opam --version
2.1.0
Ivan Gotovchits
@ivg

Now how to resolve this issue.
1) make sure that you image doesn't contain the debian installed bap, it will reduce entropy and the size of the image.
2) install the modern version of opam, it should be opam 2.x
3) then install bap from opam and enjoy.

An alternative is to use a multistage build in your docker. You can keep your existing image and dockerfile and try to pull the bap image, e.g.,

FROM <your-base-image>
FROM binaryanalysisplatform/bap:latest as bap

RUN sudo apt-get install zip --yes

COPY --chown=opam:nogroup . /bap-toolkit
WORKDIR /bap-toolkit
RUN eval $(opam env) && make && make install


RUN apt-get update && apt-get install libgmp-dev binutils --yes
WORKDIR /home/opam
COPY --from=bap /home/opam/.opam/4.09/bin/bap /usr/bin/
COPY --from=bap /home/opam/.opam/4.09/lib/bap/*.plugin /home/opam/.opam/4.09/lib/bap/
COPY --from=bap /home/opam/.opam/4.09/share/bap /home/opam/.opam/4.09/share/bap

So you just pull the bap image and copy the built parts from it. It may not work, if you have a distribution wich is too different from the one that we use for bap, i.e., if you have a different version of libc. But worthwhile to try, at least.

Okay... so let's unwind back (but the multistage image is still a good idea). So when I asked you to do which bap and which ocamlfind I presumed that you did eval $(opam env) before that. Can you please repeat it with eval $(opam env)?
cL0und
@cL0und

Okay... so let's unwind back (but the multistage image is still a good idea). So when I asked you to do which bap and which ocamlfind I presumed that you did eval $(opam env) before that. Can you please repeat it with eval $(opam env)?

root@105c05d3d265:~/bap-toolkit# eval $(opam env)
[WARNING] Running as root is not recommended

Ivan Gotovchits
@ivg
Next, if you have bap 1.4 installed in opam 2.x then it should be because for some reason it wasn't able to install the latest bap. Probably it is because you have some other packages installed that are not compatible with the modern bap or because you chose a compiler version for which the newest version of bap is 1.4. Most likely the latter. Can you also show the output of opam switch?

~/bap-toolkit# eval $(opam env)
[WARNING] Running as root is not recommended

I meant to repeat those two commands after you have activated opam, e.g.,

eval $(opam env)
which bap
which ocamlfind
cL0und
@cL0und
root@105c05d3d265:~/bap-toolkit# eval $(opam env)
[WARNING] Running as root is not recommended
root@105c05d3d265:~/bap-toolkit# which bap
/root/.opam/system/bin/bap
root@105c05d3d265:~/bap-toolkit# which ocamlfind
/root/.opam/system/bin/ocamlfind
Ivan Gotovchits
@ivg
but in any case, bap-tookit doesn't work with bap 1.4, and indeed bap-main appeared only in bap 2.something.