ivg on master
updates testsuite (#1064) This… (compare)
ivg on master
fixes a bug in the primus reall… (compare)
ivg on master
adds a stub for realloc (#1035)… (compare)
is there an intuition on how BAP scales if I was able to throw more hardware at it?
We run several analysis on different binaries in parallel. The main bottleneck is memory, roughly per each binary of interest, reserve 10 to 20 Gigabytes. So, if you have a bunch of servers each having 128 Gb of RAM, you can run about 8 bap instances per server without trouble. This is a lower bound (i.e., I'm taking rather large binaries, such as firmwares with thousands of functions), in general, if your targets are smaller, i.e., a few dozens of kilobytes (like the average size of a program in
/usr/bin, you can easily run 20+ instances per server.
Anyway, for us, the analyst time is the bottleneck and we're more focused on optimizing this time (i.e., to minimize the effort of an analyst, i.e., the between an incident report and CVE report) :)
@qqtranqq, see #1059, now we can translate any unknown instruction into a call to an intrinsic function, and then stub this function in Primus Lisp. In the future, when BAP Lisp will be ready, it should be possible to reify those stubs into static IR, but right now it will work for Primus. I think I will stub tomorrow floating-point operations (via OCaml floating-point operations) and see how it works.
The fun part, is that you can now disable lifters with (--no-x86 for example) and disassemble the whole program as a set of intrinsic calls (not sure why one would do this, besides the sacred "because we can"). In addition, we can now try evaluate programs for which we don't have lifters.
How does Primus determine a valid stack pointer for x86_64 programs?
I noticed that if I run a simple program and set RSP to, say 0x0 or 0x7FFFFFFFFFFF, then when Primus encounters an instruction like RSP := RSP - 8, the program crashes/segfaults.
But if RSP is a value like 0x40000000, the program doesn't crash.
--primus-loader-stack-size, which default to
0x40000000and 8Mb correspondingly). Then it will map this region of memory, that's all.
--primus-loader-helpin general, for more information and details.
@qqtranqq, with #1059 we can now stub missing instructions with Primus Lisp. But, in general, an ability to stub instructions opens a wide range of opportunities for us, that will allow us to tackle with complex binaries that do crypto (SSE/AVX) and floating-points. Both of them suffer from code explosion (because they use extremely complex instructions commonly millicoded in CPU's firmware), that (a) very hard to lift (they correspond to hundreds lines of code in C, which will take even more in BIL) and (b) hard to analyze. However, from the security perspective, we do not really need to go into these instructions and unfold their implementation, we just need to compute them concretely and as fast as possible without bloating our analysis space. So, the plan is to stub those instructions and compute them using the host CPU, which will be much faster and will relieve Primus from doing the hard work (and increase coverage, as basically right now every such complex instruction is an impassable barrier for Primus - it runs out of gas before it reaches even the end of the instruction).
Right now, as a proof of concept, I implemented some floating-point operations (well, they are also useful for analysis too), so we can easily compute programs with floating points. I hope this will help you (and it already helps us).
Ok ()), e.g.,
open Core_kernel open Bap_main open Bap.Std let counter = object inherit [int * int] Term.visitor method! enter_term _ _ (jmps,total) = jmps,total+1 method! enter_jmp _ (jmps,total) = jmps+1,total end let main proj = let jmps,total = counter#run (Project.program proj) (0,0) in printf "ratio = %d/%d = %g\n" jmps total (float jmps /. float total) let () = Extension.declare @@ fun _ctxt -> Project.register_pass' main; Ok ()
I've merged #1059 and #1061 into the master. Now Primus is roughly three times faster and we can turn instructions into intrinsic calls. First of all, this is useful for instructions with unknown semantics (so that we can easily and fastly stub their semantics in Primus Lisp, e.g., for example, it takes only 6 lines of code in Primus Lisp to stub SSE floating-point addition). Stubbing unknown instructions in Primus Lisp not only enables them for the emulation mode but also, in the future, it should be possible to translate those stubs into the BIL/IR static representation. This would be possible when we will finish our work on BAP Lisp (planned in BAP 2.2, which will be in October/September, this year). So right now you can take, say, an s390x binary, and do
bap ./exe --no-ida --bil-enable-intrinsic=:unknown --run and gradually add semantics to the missing intrinsics.
This feature is also useful for various VEX instructions that are implemented as hardware millicode and unfold to tons of BIL code on which our analysis chokes (like many crypto instructions, CRC, and so on). One of the approaches is instead of modeling those instructions we can turn them into uninterpreted functions (and in Primus we can give those functions a concrete interpretation, the way we did with floating points).
(defun realloc/update-chunk (old-ptr new-len) (let ((old-len (malloc/get-chunk-size old-ptr))) (if (>= old-len len) (realloc/shrink-chunk ptr len) (let ((new-ptr (malloc new-len))) (when new-ptr (memcpy new-ptr old-ptr old-len) (free old-ptr)) new-ptr))))